Terraform Provider may return 'unexpected new value' during certain operations with Data Services Manager

Products

• VMware Data Services Manager

Issue/Introduction

Some operations on Data Services Manager using Terraform Provider may return 'unexpected new value'. For example, below backtrace is seen during attempt to change of DSM provisioned database admin password using Terraform:

module.minimal_setup.kubernetes_secret.pg_hba_config: Modifying... [id=default/namepg-testX-0-mod3-pg-hba]
module.minimal_setup.kubernetes_secret.pg_hba_config: Modifications complete after 0s [id=default/namepg-testX-0-mod3-pg-hba]
module.minimal_setup.kubernetes_manifest.this: Modifying...
module.minimal_setup.kubernetes_manifest.this: Still modifying... [10s elapsed]
module.minimal_setup.kubernetes_manifest.this: Still modifying... [20s elapsed]
module.minimal_setup.kubernetes_manifest.this: Still modifying... [30s elapsed]
module.minimal_setup.kubernetes_manifest.this: Still modifying... [40s elapsed]
module.minimal_setup.kubernetes_manifest.this: Still modifying... [50s elapsed]
╷
│ Error: Provider produced inconsistent result after apply
│
│ When applying changes to module.minimal_setup.kubernetes_manifest.this, provider "module.minimal_setup.provider[\"registry.terraform.io/hashicorp/kubernetes\"]" produced an unexpected new value: .object.metadata.labels: new element
│ "dsm.vmware.com/admin-password-name" has appeared.
│
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.

Environment

Data Services Manager

Cause

DSM maintains various labels and annotations for processing and performance reasons, typically set by mutation webhooks updated from any changed fields in the cluster spec. 

unexpected new value: .object.metadata.labels: new element

This is referring to the Terraform Kubernetes provider rather than the DSM one. Label seen in the example above (admin-password-name) is coming from a mutation webhook that Terraform does not expect during its apply operation, thus producing the error. 

Resolution

Configure Terraform Provider with ignore_annotations and ignore_labels attributes set for DSM e.g.:

ignore_annotations = ["dsm\\.vmware\\.com\\/*"]
ignore_labels      = ["dsm\\.vmware\\.com\\/*"]

Additional Information

https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs#ignore-kubernetes-annotations-and-labels