SDDC Manager upgrade postcheck fails with vLCM cluster configuration error due to active vSphere HA failover loop

Products

• VMware SDDC Manager / VCF Installer

Issue/Introduction

During a VMware Cloud Foundation (VCF) lifecycle management (LCM) upgrade execution, the SDDC Manager update reports a failed state during its synchronous post-upgrade phase. Although all underlying software components successfully write to disk and register their updated build numbers in the SDDC Manager dashboard, the overarching upgrade task presents an error.

The lcm.log and lcmdebug.log expose the following specific error string:

Upgrade failed during post-checks. Retriable operation 'Checking if vLCM cluster config status is green' failed to complete after 3 retries.. Cluster is unhealthy. Reason: vSphere HA failover operation in progress in cluster <REDACTED_CLUSTER_NAME> in datacenter <REDACTED_DATACENTER_NAME>: 0 VMs being restarted, 2 VMs waiting for a retry, 0 VMs waiting for resources, 0 inaccessible vSAN VMs. Reference token *****

 

you will have the similar error on the UI

Environment

 

• VMware Cloud Foundation (VCF) 5.x 

• SDDC Manager lifecycle management (LCM)

• vSphere Lifecycle Manager (vLCM) enabled clusters

• vSphere High Availability (HA)

 

Cause

The SDDC Manager LCM service runs a strict synchronous validation loop during the postcheck phase. One mandatory task checks that the vLCM cluster status is completely healthy (green).

If a vSphere HA event occurs concurrently—or if virtual machines remain in a transient Fault Domain Manager (FDM) retry loop (waiting for a retry) following host remediation—vCenter Server flags the cluster status as unhealthy. This environment state causes the retriable SDDC Manager validation task to exhaust its maximum threshold of 3 retries and fail the postcheck workflow, even though the database platform and domain manager have no hanging internal tasks.

Resolution

Because the software bits are complete and correct build levels are verified, this postcheck error can be safely worked around to clear the state in SDDC Manager:

1. Verify Target Build State: Log in to the vSphere Client and SDDC Manager UI to verify that all transport hosts within the updated cluster reflect the target ESXi and vCenter build levels.

2. Clear Transient HA States: Navigate to the affected cluster within the vSphere Client, select the Monitor tab, and review vSphere HA status. Ensure any virtual machines stuck in a waiting or retry loop are stabilized.

3. Disable/Re-enable vSphere HA (Workaround):

   • Select the target cluster (<REDACTED_CLUSTER_NAME>) in the vSphere inventory.

   • Navigate to Configure > Services > vSphere Availability.

   • Click Edit, deselect Turn on vSphere HA, and click OK.

   • Wait for the unconfiguration task to finalize across all managed transport hosts.

   • Click Edit again, check Turn on vSphere HA, and click OK to cleanly re-initialize the cluster FDM agents.

4. Retry the Postcheck/Upgrade Task: Return to the SDDC Manager UI inventory view for the workload domain and select the option to resume or retry the upgrade sequence. The vLCM check will now return a healthy state, allowing the workflow to complete successfully.