"Manually added component <VIB Name> is an unsupported downgrade from version <version> on the host" while running check compliance on the vLCM Image cluster

Products

VMware vSphere ESX 8.x

Issue/Introduction

In the vLCM Cluster the check complains fails on one or more ESXi host with error "Manually added component <VIB Name> is an unsupported downgrade from version <version> on the host. Replace the Component in the image with one of the same or higher version."
ESXi host during the vLCM precheck fails with "Host is not compatible with the image"

The affected VIB installed on the ESXi host is of a higher version than the one present in the vlcm image version
Removing the affected VIB would give below error

[ProfileValidationError]
Profile (Updated) ESXi-8.0U3c-24414501-standard is missing component(s) Intel-Volume-Mgmt-Device which are necessary and removing them is prohibited. Please ensure that these components, which are part of the ESXi release version 8.0.3-0.55.24414501, are fully installed or overridden without partial VIB replacement or removal.

Additional symptoms reported:

"<VIB> driver for FC adapters is an unsupported downgrade from version <version> on the host."

Cause

The "Host is out of compliance with the image" status arises because vLCM detects that the iavmd VIB version present on the ESXi host is different from (specifically, higher than) the version defined within the cluster's image. vLCM's design prioritizes upgrades and consistency with the defined image; it does not inherently support downgrading individual components to match an older image specification. This requires explicit intervention to either align the host's VIB with the image (downgrade) or align the image with the host's VIB (upgrade the image definition).

Resolution

This article provides two methods to resolve the iavmd VIB version mismatch and bring ESXi 8.0 hosts back into vLCM compliance. You must choose the option that best suits your environment and desired VIB version:

Option 1: Downgrading the iavmd VIB on the ESXi Host (CLI): This option is used if you intend to align the host's iavmd VIB version with the older version specified in your vLCM image.
Option 2: Updating the vLCM Cluster Image with the Newer iavmd VIB: This option is used if you intend to standardize your cluster on the newer iavmd VIB version that is already present on the affected host.

Downloading the Intel VMD Driver (VIB Bundle)
The Intel VMD driver (VIB) is available as a .zip bundle from the Broadcom Support Portal.

Refer to the following Broadcom KB article for detailed guidance on navigating to the ESXi drivers section of their portal: Finding IO Drivers in the Broadcom Support Portal
Once in the "Drivers and Tools" section of the portal, search for "VMware ESXi 8.0 iavmd" to locate the desired VIB versions.
Important: Depending on the resolution option you choose below, you will need to download either the older (image-compliant) version or the newer (host-present) version of the iavmd VIB bundle.

Option 1: Updating the vLCM Cluster Image with the Newer iamvd VIB

This method updates the vLCM cluster image to include the newer iavmd VIB version, bringing the image in line with the host's current VIB.

Required Download: Download the higher version of the iavmd VIB bundle (the version currently present on your ESXi host) from the Broadcom Support Portal.

Import the Newer Driver into vLCM Depot:
- In the vSphere UI, navigate to Lifecycle Manager.
- Go to Updates > Import Updates.
- Select the downloaded higher version of the iavmd VIB bundle (.zip file) and upload it to the vLCM depot. The driver will appear as an ESXi component.
Edit the Cluster Image:
- Navigate to Inventory > [Your Cluster Name] > Updates > Image.
- Click the Edit button.
- In the "Components" section, click Add Components.
- Select the specific newer iavmd ESXi component(s) that were uploaded in Step 1.
- Validate and save the changes made to the image.
Check Compliance:
- Re-run the "Check Compliance" operation for the cluster hosts.
- The ESXi host(s) should now show "Host is in compliance," as the cluster image now matches the iavmd VIB version on the host. (Note: If other hosts in the cluster have the older VIB version, they will now show out of compliance, and you would then remediate them to upgrade their VIBs.)

Option 2: Downgrading the iavmd VIB on the ESXi Host (CLI)

This method manually downgrades the iavmd VIB on the affected ESXi host to match the version specified in the vLCM cluster image.

Required Download: Download the lower version of the iavmd VIB bundle (the version present in your vLCM image) from the Broadcom Support Portal.

Prepare the ESXi Host:
- Migrate all virtual machines off the affected ESXi host.
- Place the ESXi host into Maintenance Mode.
- Ensure SSH is enabled on the ESXi host.
Upload the VIB Bundle:
- Upload the downloaded lower version of the iavmd VIB bundle (.zip file) to a datastore accessible by the ESXi host.
Install the Lower VIB Version (Downgrade):
- Using an SSH/PuTTY session, connect to the ESXi host.
- Navigate to the datastore directory where the .zip file was uploaded (e.g., cd /vmfs/volumes/datastore_name/).
- Run the following command to install the iavmd driver, forcing the downgrade.
- Note: Replace /vmfs/volumes/datastore_name/Intel-Volume-Mgmt-Device_X.X.X.zip with the actual path and filename of the lower VIB bundle.
```
esxcli software vib install -d "/vmfs/volumes/datastore_name/Intel-Volume-Mgmt-Device_X.X.X.zip"
```
Reboot the ESXi Host:
- Reboot the ESXi host for the changes to take effect.
Verify Compliance:
- After the reboot, go to the cluster's Images section and re-run the "Check Compliance" for the ESXi host.
- The ESXi host should now show "Host is in compliance," as its iavmd VIB version matches the cluster image.

Additional Information

To confirm the iavmd VIB has been successfully installed or updated, run the following command on the ESXi host via SSH:

esxcli software vib list | grep iavmd

This should output details of the installed iavmd VIB, allowing you to confirm its version