Unable to check updates via VAMI due to "List operation failed, please try again" error

Products

• VMware vCenter Server
• VMware vCenter Server 7.0
• VMware vCenter Server 8.0

Issue/Introduction

Symptoms:

• vCenter VAMI Update tab results in - List operation failed, please try again.
• VAMI would not present the vCenter list for updates.
• vCenter 8.0 patch update from VAMI may fail during Pre-Check with error "Internal error occurs during execution of update process."

             

• /var/log/vmware/applmgmt/PatchRunner.log

 vsphere_ui:ComponentDiscovery ERROR transport Command ['rpm', '-q', 'vsphere-ui'] exit-code=1, stdout=package vsphere-ui is not installed, stderr=
vsphere_ui:ComponentDiscovery ERROR vmware_b2b.patching.executor.hook_executor Patch hook 'vsphere_ui:ComponentDiscovery' failed.
Traceback (most recent call last):
  File "/storage/core/software-update/updates/8.0.3.00300/scripts/patches/py/vmware_b2b/patching/executor/hook_executor.py", line 74, in executeHook
    executionResult = systemExtension(args)
  File "/storage/core/software-update/updates/8.0.3.00300/scripts/patches/libs/sdk/extensions.py", line 106, in __call__
    result = self.extension(*args)
  File "/storage/core/software-update/updates/8.0.3.00300/scripts/patches/libs/sdk/extensions.py", line 123, in _func
    return func(*args)
  File "/storage/core/software-update/updates/8.0.3.00300/scripts/patches/payload/components-script/vsphere_ui/__init__.py", line 91, in discover
    source_client_version, source_client_cln = get_client_version_from_rpm()
  File "/storage/core/software-update/updates/8.0.3.00300/scripts/patches/payload/components-script/vsphere_ui/version_util.py", line 41, in get_client_version_from_rpm
    cmd_out = getCommandOutput(localOpsManager, command)
  File "/storage/core/software-update/updates/8.0.3.00300/scripts/patches/libs/sdk/transport/__init__.py", line 215, in getCommandOutput
    raise ExecutionException('Command failed with exit-code "%s"' % result.exitCode)
transport.ExecutionException: ('Command failed with exit-code "1"', 999)
2024-10-12T00:54:45.512Z ERROR vmware_b2b.patching.phases.discoverer Could not execute discovery hook in file: /storage/core/software-update/updates/8.0.3.00300/scripts/patches/payload/components-script/vsphere_ui

 

Environment

vCenter Server 7.x

vCenter Server 8.x

Cause

Possible corruption in RPM DB.

Resolution

Note: There are multiple factors to be considered while rebuilding RPM DB. vCenter 7.x runs on Photon 3.0 and 8.0 U2 onwards on Photon 4.0 and the steps to recreate the RPM DB will vary depending on the Photon OS Version.

Step - 1

SSH to vCenter Server

Check the version of underlying Photon OS version by doing "cat /etc/os-release"

cat /etc/os-release

Perform any of below steps depending on the underlying Photon Version:

• Photon 3.0:

  • Create backup of RPM DB:

mkdir -p /root/rpmDbBackup
cp -a /var/lib/rpm /root/rpmDbBackup

  • Remove the following files to avoid stale locks

rm -f /var/lib/rpm/__db*

  • Rebuild the RPM DB
    
    rpm --rebuilddb
rpm --verifydb && echo "DB verification success"

  • List the packages, this should show the full list of packages installed in the system

rpm -qa

• Photon 4.0:

  • Create backup of RPM DB:

mkdir -p /root/rpmDbBackup
cp -a /var/lib/rpm /root/rpmDbBackup
    
    
  • Rebuild the RPM DB

rpm --rebuilddb
rpm --verifydb && echo "DB verification success"
    
    
  • List the packages, this should show the full list of packages installed in the system
    
    rpm -qa