Synchronize inventory versions not working after a vLCM ESXi upgrade failure in a VMware Cloud Foundation Environment

Products

• VMware SDDC Manager

Issue/Introduction

• vLCM cluster upgrade fails during post-checks with the error "Upgrade failed during post-checks."
• All hosts are upgraded to the target version in vCenter.
• SDDC-M reports that one or more hosts have not reached the target version.

The following behavior is observed in the operationsmanager.log

/var/log/vmware/vcf/operationsmanager/operationsmanager.log
======================================================================================
2025-03-10T12:10:13.148+0000 INFO  [vcf_om,67ced6a422517dc9d748460806519f4b,085a] [c.v.v.i.s.s.impl.EsxiVersionSync,pool-3-thread-13] Host #######.######.##### has inventory version {"version":"8.0.3-24022510"} and product version {"version":"8.0.3-24585383"}
2025-03-10T12:10:13.148+0000 DEBUG [vcf_om,67ced6a422517dc9d748460806519f4b,085a] [c.v.v.i.s.service.clients.LcmClient,pool-3-thread-13] Checking LCM if resource 5b9fcdc2-8616-4671-ad74-0d6cd6979cf5 with type ESX_HOST should be synced to version 8.0.3-24585383
2025-03-10T12:10:13.452+0000 DEBUG [vcf_om,67ced6a422517dc9d748460806519f4b,085a] [c.v.v.i.s.service.clients.LcmClient,pool-3-thread-13] Is sync required result from LCM: false

Environment

VMware Cloud Foundation 5.2.x

Cause

The Inventory Version Sync API skips components an upgrade is still in an available state. 

Resolution

Workaround Option#1 Disable version sync required check

1. SSH to SDDC Manager with vcf user and then elevate to root user with su.

2. Edit the application-prod.properties file

vi /etc/vmware/vcf/operationsmanager/application-prod.properties

3. At the end of the file, add the following line:

check.lcm.isVersionSyncRequired=false

4. Save and close the file.

:wq

5. Restart the Operation manager service on the SDDC Manager.

systemctl restart operationsmanager

6. Retry the Inventory Version Sync API.

Use KB article Synchronize inventory versions after out-of-band upgrade in a VMware Cloud Foundation Environment to perform inventory version sync.

7. Revert the Inventory Version Sync changes.

8. SSH to SDDC Manager with vcf user and then elevate to root user with su.

9. Remove "check.lcm.isVersionSyncRequired=false" from the application-prod.properties file.

vi /etc/vmware/vcf/operationsmanager/application-prod.properties

10. Save and close the file.

:wq

11. Restart the Operation manager service on the SDDC Manager.

systemctl restart operationsmanager

Workaround Option #2  Cancel the upgrade plan in the SDDC manager UI.

1. Cancel the upgrade plan in the SDDC manager UI. Verify that bundle is not available for the impacted resource. 



2. Retry the Inventory Version Sync API.

Use KB article Synchronize inventory versions after out-of-band upgrade in a VMware Cloud Foundation Environment to perform inventory version sync.

3. Recreate the upgrade plan in the SDDC manager UI (If applicable)