Compliance check fails on host while using single image with error : "An unknown error occurred while performing the operation"

Products

• VMware vSphere ESXi

Issue/Introduction

Inability to perform compliance checks on a host which is in a cluster that uses single image.

Error received: An unknown error occurred while performing the operation

 

The lifecycle.log file on ESXi has the following entries :

YYYY-MM-DDTHH:MM:00Z lifecycle: 6075408: Scanner:362 ERROR Failed to validate/extract the softwareSpec: 'Broadcom-ELX-lpfc'

YYYY-MM-DDTHH:MM:00Z lifecycle: 6075408: Scanner:363 ERROR Traceback (most recent call last): File "/lib64/python3.8/site-packages/vmware/esximage/ImageManager/Scanner.py", line 1993, in scan  self.currentSoftwareScanSpec = self.getImageProfileScanSpec() File "/lib64/python3.8/site-packages/vmware/esximage/ImageManager/Scanner.py", line 789, in getImageProfileScanSpec  compInfo = self.scanLocalComponents(baseImageComponents, addOnComponents, File "/lib64/python3.8/site-packages/vmware/esximage/ImageManager/Scanner.py", line 578, in scanLocalComponents  by = SOURCE_TO_ENTITY[self.hostCompSource[comp.com

  pNameStr]] KeyError: 'Broadcom-ELX-lpfc'

 

The vCenter shows the following entries in vmware-vum-server.log :

  YYYY-MM-DDTHH:MM:28.968Z error vmware-vum-server[31160] [Originator@6876 sub=ScanUtil] [ScanUtil 759] Host scan task failed for host host-270195

  YYYY-MM-DDTHH:MM:28.968Z error vmware-vum-server[31160] [Originator@6876 sub=com.vmware.vcIntegrity.lifecycle.DesiredScanClusterTask] [Task, 457] Task:com.vmware.vcIntegrity.lifecycle.DesiredScanClusterTask ID:52e9b3bc-8d29-2eb6-aadd-0894f77fe030. Failed to get host : esxi.example.com

  (host-#######) image result. Errors : Error:

  -->  com.vmware.vapi.std.errors.error

  --> Messages:

  -->  com.vmware.vcIntegrity.lifecycle.EsxImage.UnknownError<An unknown error occurred while performing the operation.>

  --> ;default compliance will be substituted.

 

The compliance check may still work while removing the addons from the image.

Environment

VMware vSphere ESXi 7.0

Cause

In the example above, Broadcom-ELX-lpfc is the missing/modified component causing the error.
 
This is caused when the VIB is manually removed and re-installed. Ideally while performing upgrades on hosts , if a different version of a VIB is required, we should create a custom image using imagebuilder to get the required VIB.
 
Its also possible to check the ESXi profile if the VIBs comprising of the component was removed previously :
 
For example  :
 
List VIBs in the component: (This may require using a working host.)

esxcli software component vib list -n Broadcom-ELX-lpfc
Name  Version                           Vendor  Acceptance Level  Install Date  Platforms
----  --------------------------------  ------  ----------------  ------------  ---------
lpfc  12.8.329.0-1OEM.700.1.0.15843807  EMU     VMwareCertified   YYYY-MM-DD        host

Check if the VIB was removed previously :

 esxcli software profile get | less

You would see an entry of the following type :

YYYY-MM-DDTHH:MM:00Z:11.906406+00:00: The following VIBs have been removed:
    lpfc       7.0.3-0.90.21686933

 

Resolution

This issue has been fixed in VMware ESXi 7.0 Update 3o, Build 2234881

Workaround:

Check if the component exists :
esxcli software component list | grep lpfc
 
Remove the VIB from the host that belongs to this component and reboot the host .
esxcli software vib remove -n lpfc
reboot
 
Download any offline bundle with the required version of the component
 
Check if the component exists in the bundle using the following command :
esxcli software sources component list -d "/vmfs/volumes/iSCSI-1/VMware-ESXi-7.0U3g-20328353-depot.zip"
 
Apply the component on the ESXi host :
esxcli software component apply -n "Broadcom-ELX-lpfc" -d "/vmfs/volumes/iSCSI-1/VMware-ESXi-7.0U3g-20328353-depot.zip"
 
Reboot the host

 

Additional Information

Impact/Risks:

Host scan task, compliance checks fail.