Synchronize inventory versions fails in a VMware Cloud Foundation scale environment

Products

• VMware SDDC Manager

Issue/Introduction

• Both inventory version synchronization and management domain upgrade precheck for out-of-band upgrade fail with the following cause:
  "Inventory version synchronization failed with: Unable to fetch standalone ESX credentials via /v1/system/credentials API".

Environment

• SDDC Manager 5.2.x

Cause

• In large scale environments with unassigned hosts present in the VCF inventory, the version synchronization workflow and precheck for out-of-band upgrade timeout while trying to fetch host credentials.

Resolution

Failures can occur in several places:

• Upgrade precheck for management domain - Precheck WARNING for out-of-band upgrade can be ignored.
  
  
• Daily scheduled job - Failures of the daily sync job should not cause any issues and can be ignored.
  
  
• Version synchronization workflow - Sync workflow needs to be triggered per domain and not globally for the VCF inventory.
  Steps  to trigger version synchronization per domain can be found in this article Synchronize inventory versions after out-of-band upgrade in a VMware Cloud Foundation Environment
  Note: For unassigned hosts, version synchronization is blocked until SDDC Manager is upgraded to a version containing the resolution.