vSphere Lifecycle Manager (vLCM) Hardware Compatibility checks flags the VMD controller as "Non-Compliant"

Products

• VMware vCenter Server

Issue/Introduction

• Hardware Compatibility List (HCL) checks enable the user perform hardware compatibility checks against the set desired image to detect any compatibility issues that may arise during upgrade. HCL checks are done against the hardware present and desired state against the VMware Compatibility Guide matrix.
  
  
• On running the hardware compatibility checks on vSAN cluster and the host underneath the cluster have VMD controller and NVMe devices behind it, the VMD controller listed in PCI device listing is flagged as "Non-compliant". The VMD controller would be flagged as Non-compliant with the warning message:
  
  
  • "No Firmware present. Could not determine firmware compatibility for the device"
  • "The device is incompatible with the driver or firmware specified in the cluster image specification."
    
    
• This would happen when the driver version/name are compliant with the vSAN VMware Compatibility Guide (VCG), and the target firmware version returned by the (Hardware Support Manager) HSM and the firmware version present in the vSAN VCG is empty or N/A.
  
  
• Similar Entries will be observed while checking /var/log/vmware/vmware-updatemgr/vum-server/vmware-vum-server-xx.log:
  

  YYYY-MM-DDTHH:MM:SSZ info vmware-vum-server[#####] [Originator@6876 sub=HCL::HostHclValidate] [HostHclValidate 583] Device driver Name: [ nvme_pcie ], TARGET : Driver Name [ nvme_pcie ], Driver Version [ ##### ],Firmware Version [  ]
  YYYY-MM-DDTHH:MM:SSZ info vmware-vum-server[#####] [Originator@6876 sub=HCL::HostHclValidate] [HostHclValidate 731] Esxi Version Found in SupportedReleases.Proceed for Compatibility Check.
  YYYY-MM-DDTHH:MM:SSZ info vmware-vum-server[#####] [Originator@6876 sub=HCL::HostHclValidate] [HostHclValidate 734] Checking if DeviceConstraints has values
  YYYY-MM-DDTHH:MM:SSZ info vmware-vum-server[#####] [Originator@6876 sub=HCL::HostHclValidate] [HostHclValidate 754] Checking Device driver name in SpecDriverName
  YYYY-MM-DDTHH:MM:SSZ info vmware-vum-server[#####] [Originator@6876 sub=HCL::HostHclValidate] [HostHclValidate 768] Device driver Name [ nvme_pcie ], version [ ##### ] found in softSpecNameVersionPair. PciDevice info [ vid ####, did ####, svid ####, ssid #### ]
  YYYY-MM-DDTHH:MM:SSZ info vmware-vum-server[#####] [Originator@6876 sub=HCL::HostHclValidate] [HostHclValidate 792] TARGET : Spec Driver Version [ ##### ]
  YYYY-MM-DDTHH:MM:SSZ info vmware-vum-server[#####] [Originator@6876 sub=HCL::HostHclValidate] [HostHclValidate 817] Setting TargetFirmwareVersion = ''
  YYYY-MM-DDTHH:MM:SSZ info vmware-vum-server[#####] [Originator@6876 sub=HCL::HostHclValidate] [HostHclValidate 819] TARGET : Esxi Version [ ESXi 9.0 ], Driver Version [ ##### ],Firmware Version [  ]
  YYYY-MM-DDTHH:MM:SSZ info vmware-vum-server[#####] [Originator@6876 sub=HCL::HostHclValidate] [HostHclValidate 825] Device found on VCG; starting as compatible.
  YYYY-MM-DDTHH:MM:SSZ info vmware-vum-server[#####] [Originator@6876 sub=HCL::HostHclValidate] [HostHclValidate 863] Spec driver version match but Firmware NOT match in Device Constrants.Spec Firmware Version [ ##### ], Device Firmware Version [  ]

Environment

VMware vCenter Server 8.x

VMware vCenter Server 9.x

Cause

There should be no firmware version check required for VMD controllers, we only need to check the driver name and driver version to be installed on the VMD controller against the vSAN VCG. But we currently have a bug in the code, due to which there is an attempt to perform firmware version check.

Resolution

Broadcom is aware of this issue and will be fixed in the future releases.

Workaround:
If the user observes the above mentioned symptoms and verifies that the firmware reported in the PCI device listings on the HCL UI is empty, then the HCL warning can be ignored. If the remediation setting was set to block upgrades upon HCL issues, then this remediation setting *should be turned off* so that the upgrades would not be blocked by this HCL issue.

Notes:

• HCL warning message can only be ignored if related to VMD controller firmware version. If there is any warning/error messages around the driver name/version of the VMD controller should NOT be ignored.
• Remediation settings to ignore the HCL issues can be turned off if there is no other HCL issues apart from the one mentioned above. If there are other HCL issues reported, then these should be resolves before turning off the remediation settings to ignore the HCL issue.

Additional Information

As long as no firmware version showing on "Driver and Firmware in the Cluster's image", the warning will appear, and manually verification on the compatibility of driver/firmware is required.