Expanding a stretch cluster in SDDC Manager fails at Remediate ESXi Host(s) to be Compliant with Cluster's Image

Products

• VMware SDDC Manager
• VMware Cloud Foundation

Issue/Introduction

• Expanding a stretch cluster in SDDC Manager fails at Remediate ESXi Host(s) to be Compliant with Cluster's Image
• Error in SDDC Manager UI
  
  

  Progress Messages	Applying personality to cluster domain-c# failed with error: ApplyStatus (com.vmware.esx.settings.clusters.software.apply_status) => { status = SKIPPED, progress = <null>, startTime = yyyy-mm-ddThh:mm:ss.325Z[GMT], endTime = yyyy-mm-ddThh:mm:ss.385Z[GMT], notifications = Notifications (com.vmware.esx.settings.notifications) => { info = <null>, warnings = <null>, errors = <null> } } Applying personality to cluster domain-c# failed with error: ApplyStatus (com.vmware.esx.settings.clusters.software.apply_status) => { status = SKIPPED, progress = <null>, startTime = yyyy-mm-ddThh:mm:ss.859Z[GMT], endTime = yyyy-mm-ddThh:mm:ss.667Z[GMT], notifications = Notifications (com.vmware.esx.settings.notifications) => { info = <null>, warnings = <null>, errors = <null> } } Applying personality to cluster domain-c# failed with error: ApplyStatus (com.vmware.esx.settings.clusters.software.apply_status) => { status = SKIPPED, progress = <null>, startTime = yyyy-mm-ddThh:mm:ss.427Z[GMT], endTime = yyyy-mm-ddThh:mm:ss.731Z[GMT], notifications = Notifications (com.vmware.esx.settings.notifications) => { info = <null>, warnings = <null>, errors = <null> } } Applying personality to cluster domain-c# failed with error: ApplyStatus (com.vmware.esx.settings.clusters.software.apply_status) => { status = SKIPPED, progress = <null>, startTime = yyyy-mm-ddThh:mm:ss.084Z[GMT], endTime = yyyy-mm-ddThh:mm:ss.155Z[GMT], notifications = Notifications (com.vmware.esx.settings.notifications) => { info = <null>, warnings = <null>, errors = <null> } }
  	Failed to validate since personality was already applied on cluster domain-c#.
  Error	Message: Applying personality to cluster domain-c# failed with error: ApplyStatus (com.vmware.esx.settings.clusters.software.apply_status) => { status = SKIPPED, progress = <null>, startTime = yyyy-mm-ddThh:mm:ss.427Z[GMT], endTime = yyyy-mm-ddThh:mm:ss.731Z[GMT], notifications = Notifications (com.vmware.esx.settings.notifications) => { info = <null>, warnings = <null>, errors = <null> } } Remediation Message: Reference Token: 6####6
  Error	Message: Failed to validate since personality was already applied on cluster domain-c#. Remediation Message: Reference Token: G####D Cause: Exception occurred during vAPI invocation: java.util.concurrent.ExecutionException: com.vmware.vapi.std.errors.Unauthenticated: Unauthenticated (com.vmware.vapi.std.errors.unauthenticated) => { messages = [], data = <null>, errorType = UNAUTHENTICATED, challenge = SIGN realm="A6:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:BA",sts="https://vcenter.example.com/sts/STSService/vsphere.local", Basic realm="vCenter" }

• Error in /var/log/vmware/vcf/domainmanager/domainmanager.log

  ERROR [vcf_dm,68#######################45,e448] [c.v.e.s.o.model.error.ErrorFactory,dm-exec-25]  [6425LR] VLCM_REMEDIATE_PERSONALITY_FAILED Applying personality to cluster domain-c# failed with error: ApplyStatus (com.vmware.esx.settings.clusters.software.apply_status) => {
      status = SKIPPED,
      progress = <null>,
      startTime = yyyy-mm-ddThh:mm:ss.084Z[GMT],
      endTime = yyyy-mm-ddThh:mm:ss.155Z[GMT],
      notifications = Notifications (com.vmware.esx.settings.notifications) => {
          info = <null>,
          warnings = <null>,
          errors = <null>
      }
  }
  com.vmware.evo.sddc.orchestrator.exceptions.OrchTaskException: Applying personality to cluster domain-c9 failed with error: ApplyStatus (com.vmware.esx.settings.clusters.software.apply_status) => {
      status = SKIPPED,
      progress = <null>,
      startTime = yyyy-mm-ddThh:mm:ss.084Z[GMT],
      endTime = yyyy-mm-ddThh:mm:ss.155Z[GMT],
      notifications = Notifications (com.vmware.esx.settings.notifications) => {
          info = <null>,
          warnings = <null>,
          errors = <null>
      }
  }
  

• ESX hosts are in compliance with the image in the vSphere Cluster

Environment

VMware Cloud Foundation 9.0

Cause

vCenter compliance checks are taking a significant amount of time to complete, the workflow is not returning to the domainmanager to continue the add host workflow. This causes an exception when remediating the cluster.

Resolution

Please contact Broadcom Support if you can not upgrade and would like to preform a workaround of this issue. Refer - Contact Broadcom support