VIBs "have unequal values of the 'payloads' attribute" while performing Update Manager compliance check

Products

• VMware vCenter Server
• VMware vSphere ESXi

Issue/Introduction

This article is to identify and resolve "have unequal values of the 'payloads' attribute" error.

Symptoms:

• While performing an Update Manager compliance check, the scan returns the error:

esxupdate error codes: -1. Check the Update Manager log files and esxupdate log files for more details.

• In the /var/run/log/esxupdate.log, you see entries similar to:

ERROR: ValueError: VIBs ELX_bootbank_elx-esx-libelxima.so_12.0.1108.0-03 and ELX_bootbank_elx-esx-libelxima.so_12.0.1108.0-03 have unequal values of the 'payloads' attribute: '[elx-esx-libelxi: 1602.936 KB]' != '[elx-esx-libelxi: 1493.833 KB]'

• Image Builder is unable to build an ISO out of the software depot because it sees two sizes of the VIBs and are unable to merge. You will see similar entries in vmon-redirect.log.stderr as mentioned below:

vmon-redirect.log.stderr:
INFO:DepotCollection:Error parsing metadata.zip from https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/esx/vmw/vmw-ESXi-6.7.0-metadata.zip: Cannot merge VIBs ELX_bootbank_elx-esx-libelxima.so_12.0.1108.0-03, ELX_bootbank_elx-esx-libelxima.so_12.0.1108.0-03 with unequal payloads attributes: ([elx-esx-libelxi: 1493.833 KB], [elx-esx-libelxi: 1602.936 KB])
Exception in thread Connect-task-11784841:
Traceback (most recent call last):
 File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
  self.run()
 File "/usr/lib/python2.7/threading.py", line 754, in run
  self.__target(*self.__args, **self.__kwargs)
 File "/usr/lib/vmware-imagebuilder/site-packages/imagebuilder/vmodl_impl/DepotManager.py", line 288, in _connectToDepot
  if errors:
UnboundLocalError: local variable 'errors' referenced before assignment

Environment

VMware vSphere Update Manager 6.5.x
VMware vSphere Update Manager 6.0.x
VMware vSphere Update Manager 6.7.x

Cause

This issue is caused by a VIB installed on the ESXi having a different payload value (size variants) than would be expected according to the information in its metadata.
 

VIB Name: ELX_bootbank_elx-esx-libelxima.so_12.0.1108.0-03

Resolution

To resolve this issue, remove the conflicting VIB and perform the patch update again. For more information on removing VIBs, see Remove VIBs from a Host.

Note: Before removing the conflicting VIB, confirm it is not in use on the host. For more information, see Determining Network/Storage firmware and driver version in ESXi

For example in case of elx-esx-libelxima.so VIB with unequal payload:

esxcli software vib remove -n elx-esx-libelxima.so

 

Workaround:

• Manually install http://vibsdepot.hpe.com/hpe/sep2018/esxi-670-devicedrivers/VMW-ESX-6.7.0-elxiscsi-12.0.1108.0-8119170.zip on the impacted ESX host by esxcli (not scalable if there are many hosts).
• Remove the HPE depot, reset VUM DB/patch store, and resume to only import the needed monthly release (http://vibsdepot.hpe.com/hpe/sep2018/index.xml), create a baseline and remediate the impacted host. This will mean redoing things from VUM side. Afterward you will need to reset VUM again and re-import the whole HPE depot to resume getting the latest contents.
  • Note: This workaround is to get the host updated with this one VIB and move on to new updates in the future. Once installed the unequal error will not appear since the VIB is no longer needed to be applied.
• Create a custom baseline without the conflicting VIB.

Additional Information

Impact/Risks:

• This issue will prevent the ESXi host from being able to be patched.
• Host baselines for critical and non-critical patches will show non-compliant all the time.