When patching ESXi hosts from version 8.0 Update 3f to 8.0 Update 3g via VMware SDDC Manager, one or more nodes fail to complete remediation while other hosts in the cluster update successfully.
Symptoms:
The ESXi host reports the updated version/build (e.g., 8.0 U3g) locally via SSH or Host Client, but the vCenter Server continues to mark the ESXi host as Non-Compliant.
The vCenter server log /var/log/vmware/vmware-updatemgr/vum-server/vmware-vum-server.log records the following error:com.vmware.vcIntegrity.lifecycle.RemediateClusterTask.HostNotCompliantAfterRemediation - "After host remediation completed, compliance check reported host as 'non-compliant'. The image on the host does not match the image set for the cluster."
Correspondingly, the ESXi host log /var/run/log/lifecycle.log records partition corruption and payload staging errors:yyyy-mm-ddThh:mm:ssZ In(14) lifecycle[PID]: HostImage:249 Installer <class 'vmware.esximage.Installer.LockerInstaller.LockerInstaller'> was not initiated - reason: Failed to create empty Database directory: [Errno 2] No such file or directory: '/locker/packages/var'.........yyyy-mm-ddThh:mm:ssZ Db(15) lifecycle[PID]: HostSeeding:847 AddOn details : HPE-Custom-AddOn, 803.0.0.12.1.0-11, HPE Customization for HPE Servers, Hewlett Packard Enterprise, 803.0.0.12.1.0-11yyyy-mm-ddThh:mm:ssZ Er(11) lifecycle[PID]: HostSeeding:697 BaseImg Comps are removed: {'VMware-VM-Tools'}yyyy-mm-ddThh:mm:ssZ Er(11) lifecycle[PID]: HostSeeding:879 Software info extract errors: The following Components have been removed on the host: VMware-VM-Toolsyyyy-mm-ddThh:mm:ssZ Er(11) lifecycle[PID]: imagemanagerctl:307 Get Software Info Failed: The following Components have been removed on the host: VMware-VM-Toolsyyyy-mm-ddThh:mm:ssZ In(14) lifecycle[PID]: imagemanagerctl:843 Calling with arguments: software --getsoftwareinfo
The Remove Hosts option in the VMware SDDC Manager UI is greyed out for affected nodes because the cluster task state is stuck in an ERROR status.
Reimaging the host out-of-band without matching vendor add-ons triggers an Incompatible status in vSphere Lifecycle Manager (vLCM) due to driver version drift.
VMware SDDC Manager
VMware vSphere ESXi 8.0.x
VMware vCenter Server 8.0.x
Corruption on the ESXi host bootbank partitions prevents directory creation under /locker/packages/var and corrupts the /productLocker symbolic link, preventing the VMware Tools payload staging required for vLCM compliance.
Follow this workflow to resolve filesystem corruption, align driver versions, and clear VMware SDDC Manager inventory locks:
Clear Stalled SDDC Manager Tasks: If the cluster is stuck in an inconsistent state preventing host removal, manually retire the stalled remediation tasks. For more information, see Removing host(s) from SDDC Manager fails with error.
Align Vendor Custom Add-on Driver Version: Ensure the Vendor Custom Add-on version on the ISO used during vLCM reimaging matches the exact version specified in the cluster desired state image in vLCM.
Trigger vLCM Compliance Check: Once host reinstallation finishes and the host comes back online, navigate to Cluster > Updates > Image > Check Compliance in the vSphere Client to confirm ESXi host compliance.
Synchronize SDDC Manager Inventory: Perform an inventory refresh/sync in VMware SDDC Manager to register the host status and restore normal UI operation.