vCenter upgrade fails, SDDC Manager UI displays error "software packages failed to install due to an exception Could not convert filed 'task' of structure 'com.vmware.appliance.update.info'", vCenter VAMI UI displays patching error "Test RPM transaction f

Products

• VMware SDDC Manager

Issue/Introduction

• VCF triggers vCenter upgrade
• The upgrade fails, with this error being seen on the SDDC Manager UI - 

"software packages failed to install due to an exception Could not convert filed 'task' of structure 'com.vmware.appliance.update.info'"

• vCenter patching fails with the following VAMI (port 5480) UI error message:

"Test RPM transaction failed. Collect the logs for diagnostic"

• The `/var/log/vmware/applmgmt/update_microservice.log`  on the vCenter contains similar to the following:

YYYY-MM-DD hh:mm:ss,### - ##### -     update_functions::      runCommandAndCheckResult: ### -    DEBUG - runCommandAndCheckResult failed: '\tinstalling package VMware-vpxd-7.0.3.01500-11163753.x86_64 needs 4KB on the /storage/log filesystem\n\tinstalling package vmware-directory-client-7.0.3.25000-11151027.x86_64 needs 4KB on the /storage/log filesystem\n\tinstalling package vmware-eam-7.0.3.01500-11163753.x86_64 needs 8KB on the /storage/log filesystem\n\tinstalling package vmware-esx-netdumper-7.0.3.01500-0.0.11163753.i386 needs 4KB on the /storage/log filesystem\n\tinstalling package VMware-visl-integration-7.0.3.01500-11163753.x86_64 needs 4KB on the /storage/log filesystem\n\tinstalling package VMware-pod-7.0.3.01500-11163753.x86_64 needs 4KB on the /storage/log filesystem\n\tinstalling package VMware-cis-license-7.0.3.01400-10773104.x86_64 needs 12KB on the /storage/log filesystem\n\tinstalling package VMware-analytics-7.0.3.01400-10772069.x86_64 needs 4KB on the /storage/log filesystem\n'

Environment

VCF 4.5.x, vCenter 7.0.x

Cause

This happens when the /storage/log partition on the vCenter is full prior to starting the upgrade process, or has become full during the upgrade process (as might happen if it was almost full prior to starting the upgrade process).

This article is relevant - 

https://knowledge.broadcom.com/external/article/326227/log-disk-exhaustion-caused-by-wcp-stdstr.html

 

Resolution

1. Revert to the pre-upgrade snapshot.
2. Free up space in /storage/log - there may be an obvious file that can be removed, if not use article 313077 to identify what file(s) to remove
3. Re-run the upgrade by retrying the workflow on the SDDC Manager

Note: there may be circumstance where it is not feasible to revert to the pre-upgrade snapshot.
For example, there may have been environmental changes that have taken place.
In this instance it is preferable to try to get the failed upgrade to complete.

1. Don't revert to the pre-upgrade snapshot
2. Free up space in /storage/log
3. Re-run the upgrade

If the upgrade does not succeed (this error may be seen - ""Staging packages for vCenter" "upgrade has failed due to NotAllowedInCurrentState""), then proceed with patching the vCenter Server appliance by using the appliance shell - 
https://techdocs.broadcom.com/us/en/vmware-cis/vsphere/vsphere/7-0/vcenter-server-upgrade-7-0.html