Inventory Version Synchronization fails: Cannot connect to vCenter in VMware Cloud Foundation 9.x

Products

VMware SDDC Manager / VCF Installer

Issue/Introduction

The Synchronize Inventory Versions task for a workload domain fails consistently in the SDDC Manager user interface (UI). The SDDC Manager Tasks pane displays the following error details:

Description: Synchronize Inventory Versions
Progress Messages: Inventory version synchronization failed. Try to fix the cause for the issue and retry the operation.
Error Message: Inventory version synchronization failed. Try to fix the cause for the issue and retry the operation.
Remediation Message: Ensure the following: Ensure the vCenter #### is up and running.
Reference Token: ####
Cause: Inventory version synchronization failed with: Cannot connect to vCenter #### of domain #### to complete inventory sync.

There is a discrepancy between the number of ESXi hosts tracked by SDDC Manager for a specific Workload Domain versus the actual number of hosts present in the associated vCenter Server inventory.
(For example, SDDC Manager registers 11 hosts assigned to the Workload Domain, but the vCenter UI only shows 10 hosts physically present in the cluster).
In the SDDC Manager UI under Security > Password Management, the root and service accounts for the missing host display a Disconnected status.
Querying the SDDC Manager Database reveals that the missing host is still registered with an ACTIVE status, despite being absent from the vCenter inventory:
```
psql -h localhost -U postgres -d platform -c "SELECT id, hostname, status FROM host WHERE hostname='<FQDN-of-Missing-Host>';"
```

Reviewing the /var/log/vmware/vcf/operationsmanager/operationsmanager.log file shows an InventorySyncException occurring during the EsxiVersionSync phase.

DEBUG [c.v.v.i.s.s.impl.EsxiVersionSync] ESX host #### with ID #### has inventory version 8.0.3-####
DEBUG [c.v.e.s.c.c.v.vsphere.VcManagerBase] Finding host with fqdn ####
DEBUG [c.v.e.s.c.c.v.vsphere.VcManagerBase] Finding host with Ip address ####
DEBUG [c.v.e.s.c.c.v.vsphere.VsphereClient] Destroying 1 open views
ERROR [c.v.v.i.s.s.impl.EsxiVersionSync] Exception when constructing version diff for host(s) assigned to vcenter ####
com.vmware.vcf.inventory.sync.service.exception.InventorySyncException: Cannot connect to vCenter #### of domain #### to complete inventory sync.
        at com.vmware.vcf.inventory.sync.service.impl.EsxiVersionSync.constructVersionDiffForAssignedHosts

Environment

VMware Cloud Foundation (VCF) 9.x

Cause

This issue occurs due to an Out-of-Band (OOB) configuration change where an ESXi host is removed directly via the vCenter UI, bypassing standard VCF workflows. Since the host removal does not execute through SDDC Manager, the platform database retains a stale record of the host with an ACTIVE status. During the "Synchronize Inventory Versions" task, the EsxiVersionSync process iterates through its database and queries vCenter for the ESXi version of every host. When it queries the stale record, vCenter returns a null response, causing SDDC Manager to throw a fatal exception and terminate the workflow to prevent data corruption.

Resolution

Direct database modification is required to safely purge the stale host record and its associated relational mappings (cluster, domain, network pools, and credentials) from the SDDC Manager database.

Reference KB : Steps to remove ESXi Hosts from the SDDC Manager DB

Contact Broadcom Support and reference this KB article for executing the database cleanup safely.

Additional Information

Subscribe to a Broadcom knowledge article by article or product