vLCM Sync Updates fails with 'A depot is inaccessible or has invalid contents' in VCF 9.1

Products

• VMware SDDC Manager / VCF Installer

Issue/Introduction

• Log into the vSphere UI. Select hamburger icon > Lifecycle Manager > ACTIONS > Updates > Sync Updates.
• This error is observed:

Sync updates fails - vCenter error when trying to sync updates. A general system error occurred: "A depot is inaccessible or has invalid contents. Make sure an official depot source is used and verify connection to the depot."

• In the vCenter log - /var/log/vmware/vmware-updatemgr/vum-server/imageservice.log - this error is observed:

Two baseimages share the same releaseID: ESXi:9.1.0-0100.25433460, but have different attributes.

Environment

VCF 9.1.0.0

Cause

• The online correct base image and an imported base image are clashing.
• This is caused by creating a 9.1 ESXi image while the vCenter was running 9.0.

Resolution

(1) Perform a cleanup of the vLCM depot:

1. Take a snapshot, without memory, of the vCenter.
2. SSH to the vCenter, log in as root.
3. Launch the dcli:
   • dcli +i +show
4. List the offline depots (log in as [e-mail] when prompted):
   • esx settings depots offline list
5. Identify the ID for the problematic depot (e.g., OFFLINE_BUNDLE_xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx.zip - the one with the 9.1 ESXi image, imported when the vCenter was still running 9.0).
6. Delete the problematic depot:
   • esx settings depots offline delete-task --depot XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX --force true
7. Delete all orphan images in the vLCM image library, so that the only images that are left are the ones that are assigned to a cluster, via the vSphere UI > Lifecycle Manager > Image Library

(2) Perform a cleanup of UMDS content inside of SDDC Manager:

1. Take a snapshot, without memory, of the SDDC Manager.
2. SSH to the SDDC Manager as vcf, then su to root.
3. Remove the contents of the UMDS:
   • rm -rf /nfs/vmware/vcf/nfs-nount/umds/patch-store/hostupdate
4. Verify removal:
   • ls -l /nfs/vmware/vcf/nfs-mount/umds/patch-store/hostupdate/vmw/

To confirm that the solution recommendation has resolved the issue, log into the vSphere UI > Lifecycle Manager > ACTIONS > Updates > Sync Updates

This workflow is expected to succeed now, when it failed previously.