VCF Cluster Creation fails at stage "Remediate ESXi Host(s) to be Compliant with Cluster's Image"

Products

• VMware SDDC Manager / VCF Installer

Issue/Introduction

When creating or expanding a cluster in VMware Cloud Foundation (VCF) SDDC Manager, the task fails during remediation. The SDDC Manager UI reports the cluster is NON_COMPLIANT

In the vCenter, we see that the remediation is complete

In /log/vmware/vcf/sddc-support/sos/sddc/var/log/vmware/vcf/domainmanagerdomainmanager.log, the following error is observed:

YYYY-MM-DD ERROR [vcf_dm,#######,#######] [c.v.v.clustermanager.ClusterSpecUtil,dm-exec-9] Cannot skip 3 ESXi host(s) ([<HOST1>, <HOST_2>, <HOST_3>]), as only 0 ESXi host(s) would remain which is not sufficient. Minimum requirement is 2
YYYY-MM-DD ERROR [vcf_dm,#######,#######] [c.v.e.s.o.model.error.ErrorFactory,dm-exec-9]  SKIP_FAILED_HOSTS_NOT_ENOUGH_HEALTHY_HOSTS_WOULD_REMAIN

Simultaneously, the var/log/vmware/vmware-updatemgr/vum-server/vmware-updavum-server.log (vSphere Lifecycle Manager) on the vCenter Server reports a policy conflict:

YYYY-MM-DD error vmware-vum-server[######] [Originator@6876 sub=RemediateClusterTask] [Task, 524] Task:com.vmware.vcIntegrity.lifecycle.RemediateClusterTask ID:. Task Failed. Error: com.vmware.vapi.std.errors.error
--> Messages: com.vmware.vcIntegrity.lifecycle.RemediateClusterTask.InvalidPolicyCombination <The remediation policies 'Enforce Live Patch, Parallel remediation' are not supported together and can not be enabled at the same time. Please keep only one of them enabled to proceed.>

Environment

• VMware Cloud Foundation 5.x
• VMware vSphere 8.0.x

Cause

The root cause is a configuration conflict in vSphere Lifecycle Manager (vLCM). The policies "Live Patch" and "Parallel Remediation" are enabled simultaneously. The remediation in parallel only applies for ESXi hosts that are already in maintenance mode. During parallel remediation, hosts do not enter or exit maintenance mode automatically. To remediate hosts in parallel, host must be set to enter and exit maintenance mode manually. If parallel remediation is enabled, vSphere Lifecycle Manager does not remediate the ESXi hosts that are not in maintenance mode.

Resolution

To resolve this issue, disable Parallel Remediation in the cluster settings:

1. Open the vSphere Client.
2. Navigate to the Cluster > Configure > Desired State > Configuration.
3. Edit the Remediation Settings.
4. Uncheck Parallel Remediation.
5. Save the settings.

Restart the failed task in SDDC Manager.