ESXi Upgrade via SDDC Manager blocked due to multiple Precheck Failures related to hardware firmware and compatibility checks.

Products

• VMware SDDC Manager

Issue/Introduction

Prechecks fail with errors citing Hardware compatibility issues, Error fetching device firmware data and NVMe SSD Controller PM173Xa incompatibility with the cluster image.

• Image has Hardware compatibility issues
  
• Error fetching device firmware data, Device compatibility must be verified manually
  
• NVMe SSD Controller PM173Xa: Device is incompatible with the driver/firmware in cluster’s image.
  

Environment

VCF 5.x

Cause

The Hardware/NVMe compatibility errors indicate the Hardware Support Manager (HSM) plugin—the vendor OVF integrated into vCenter—is failing its hardware compatibility validation checks for vLCM.

Hardware compatibility validation failed when using the hardware vendor's Hardware Support Manager (HSM) plugin with VMware vSphere Lifecycle Manager (vLCM).

Note: The Hardware Support Manager (HSM) is a software solution provided by the hardware vendor, typically delivered as an OVF (Open Virtualization Format) image. Once deployed, the HSM includes a plug-in that registers itself as an extension within the vCenter Server. This integration allows for streamlined hardware management and support, enabling enhanced compatibility checks and firmware updates within the vSphere environment.

Resolution

These are Warning Message reported via Pre-Check in SDDC Manager. Due to HSM Plugin issue manual Validation at the vCenter is required.

Note: These warning messages can be safely ignored as these are not Upgrade Blocker.

1. Manually Validation steps
   • Login to vCenter and Select the Cluster where the Host is residing
   • Navigate to the Updates Tab in the Cluster
   • Run Precheck for the Image Attached
     • This should reflect the Components Version Installed on the ESXi Host and the Components Version that is available on the attached Image for upgrade
       
     • This helps validate if the Attached Image has Higher version of Firmware Components, so that we can safely ignore the Pre-Check warning in SDDC Manager and proceed with the ESXi upgrade.
       
       
2. NVMe Controller driver Validation
   • Validate the Existing version from ESXi SSH Session with below command
     • esxcli storage core adapter list
       
     • for a in $(esxcfg-scsidevs -a |awk ' {print $2) ') ;do vmkload_mod -s $a |grep -i version ; done
       
     • esxcli vsan debug controller list -- used-by-vsan=true
       
     • esxcli nvme device get -A vmhba0 | grep -A 4 PCIVID
       
     • The attached image has higher version of nvme
       
3. If in case the Firmware Component on the Image is lower than the current version, then manually upgrade the required version post the upgrade.
   - Supported drivers and firmware versions for I/O devices