Stretched Workload Domain Workflow fails fails at subtask - Update NSX TransportNode Profile with SubCluster configuration

Products

• VMware SDDC Manager

Issue/Introduction

• Workflow fails at 86% on prepare transport node collection task
  
  
• On SDDC Manager UI we seeing the following error:
  
  Message: A problem has occurred on the server. Please retry or contact the service provider and provide the reference token.
Remediation Message:
Reference Token: XXXXX
Cause: Caught an exception in toggling vSAN Health Alarms for cluster : <domain-id>, silence : false
  
  
• In SDDC Manager logs we seeing the following errors:
  
  yyyy-mm-ddThh:mm:ss.zzzz DEBUG [vcf_dm,xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx,xxxx] [c.v.v.c.f.p.n.p.a.CreateTransportNodeCollectionAction,dm-exec-2]  Apply Transport Node profile <redacted> on cluster <redacted>:<domain-id> through {}
yyyy-mm-ddThh:mm:ss.zzzz DEBUG [vcf_dm,xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx,xxxx] [c.v.v.s.c.s.SecurityConfigurationServiceImpl,dm-exec-2]  Security config retrieved {"fipsMode":false}
yyyy-mm-ddThh:mm:ss.zzzz DEBUG [vcf_dm,xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx,xxxx] [c.v.v.s.c.s.SecurityConfigurationServiceImpl,dm-exec-2]  Security config retrieved {"fipsMode":false}
yyyy-mm-ddThh:mm:ss.zzzz DEBUG [vcf_dm,xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx,xxxx] [c.v.e.s.c.c.v.vsphere.VcManagerBase,dm-exec-2]  Connecting to https://<vc_fqdn>:443/sdk
yyyy-mm-ddThh:mm:ss.zzzz DEBUG [vcf_dm,xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx,xxxx] [c.v.v.s.t.DynamicTrustManager,dm-exec-2]  Checking validity of certificate chain <redacted>
yyyy-mm-ddThh:mm:ss.zzzz DEBUG [vcf_dm,xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx,xxxx] [c.v.v.s.t.DynamicTrustManager,dm-exec-2]  Certificate chain <redacted>
yyyy-mm-ddThh:mm:ss.zzzz WARN  [vcf_dm,xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx,xxxx] [c.v.v.v.c.h.i.HttpProtocolBindingBase,dm-exec-2]  Asynchronous execution requested but no Executor configured. The request will be executed as synchronous one.
yyyy-mm-ddThh:mm:ss.zzzz ERROR [vcf_dm,xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx,xxxx] [c.v.e.s.c.c.v.vsphere.VsphereClient,dm-exec-2]  Failed to connect to https://<vc_fqdn>:443/sdk
java.util.concurrent.ExecutionException: (vim.fault.InvalidLogin) {
   faultCause = null,
   faultMessage = null
}
yyyy-mm-ddThh:mm:ss.zzzz ERROR [vcf_dm,xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx,xxxx] [c.v.e.s.c.c.v.vsphere.VcManager
Base,dm-exec-2]  Cannot complete login due to incorrect credentials: <vc_fqdn>, <vc_service-account>@vsphere.local.
yyyy-mm-ddThh:mm:ss.zzzz ERROR [vcf_dm,xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx,xxxx] [c.v.v.c.f.p.n.h.NsxtVsphereService,dm-exec-2]  
Caught an exception in toggling vSAN Health Alarms for cluster : <domain-id>, silence : true
java.util.concurrent.ExecutionException: (vim.fault.InvalidLogin) {
   faultCause = null,
   faultMessage = null

Environment

SDDC Manager 5.x

Cause

Issue is caused by a vCenter service account credentials mismatch in the existing workload domain expansion workflow

Resolution

To resolve the issue retrieve the credentials of the vCenter VCF Service Account and update the workflow with the correct password.

To achieve the above task follow the resolution steps of articles KB399460 and KB314620