Symptoms:
vCenter Async Patch 7.0U3o patch fails with the following error in SDDC, "Software packages failed to install"
This issue impacts vCenters that have undergone an upgrade from VC 7.0 to VC 7.0U3 (any patch version earlier than U3i) and have been subsequently updated to 7.0U3o. We recommend following the pre-update remediation steps in this KB before applying the 7.0U3o patch.
You can check the upgrade sequence by running the following command from the appliance shell on the vCenter. To change shell from bin/bash run command appliancesh
software-packages list --history
Example of an affected upgrade sequence:
Command> software-packages list --history
[2023-11-15 20:17:39,387] :
'Name' 'Install Date'
VC701MP1-HP-2755530 2021-07-02T16:45:13.576Z
VC-7.0U2d 2021-10-13T06:22:19.013Z
VC-7.0U3d 2022-10-17T20:59:48.758Z
Similar errors are seen in the lcm.log on the SDDC Manager.
/var/log/vmware/vcf/lcm/lcm.log
2023-11-09T18:22:38.219+0000 ERROR [vcf_lcm,94e2e03a9d213e29,d7d0,upgradeId=########-####-####-####-########a93e,resourceType=
VCENTER,resourceId=########-####-####-####-########6f36,bundleElementId=########-####-####-####-########d3ac] [c.v.e.s.l.p.i.v
center.VCenterCommon,Async-4]
VCENTER FQDN software packages failed to install. Update info Info (com.vmware.appliance.update.info) => {
state = UP_TO_DATE,
task = <null>,
version = 7.0.3.00500,
latestQueryTime = 10/12/22 06:56:02 +0000
}
Similar errors are seen in the update_microservice.log on the vCenter.
/var/log/vmware/applmgmt/update_microservice.log
Updating / installing...
bash-4.4.18-4.ph3 ########################################
Cleaning up / removing...
bash-4.4.18-2.ph3 ########################################
][/bin/sh: error while loading shared libraries: libtinfo.so.6: cannot open shared object file: No such file or directory
warning: %postun(bash-4.4.18-2.ph3.x86_64) scriptlet failed, exit status 127
VMware is aware of this issue and working to resolve this in a future release.
1. Revert the failed vCenter to a previously working version
2. SSH into the vCenter server using root credentials.
3. Remove the stale Patching.db file from vCenter
mv /storage/db/patching.db /storage/db/patching.db.old
4. Retry the update to 7.0U3o from the SDDC UI.
1.SSH into the vCenter(s)
2.Check for the patching.db*
ls -lh /storage/db/patching.db
3.Remove the stale Patching.db file from the vCenter(s)
mv /storage/db/patching.db /storage/db/patching.db.old
4.Verify the file is no longer present:
ls -lh /storage/db/patching.db
5.Start the upgrade from the SDDC UI.