“Downgrades of manually added Components… are not supported” during vLCM image remediation on ESXi hosts

Products

• VMware vSphere ESXi

Issue/Introduction

• Running a pre-check or image compliance check using image-based remediation to update ESXi hosts fails. This occurs when the ESXi host contains newer OEM, partner async, or manually installed driver components compared to the versions included in the desired vLCM image or vendor addon.

• The vSphere Lifecycle Manager (vLCM) UI reports the following errors: Host is not compatible with the image.

• Downgrades of manually added Components (<Component Name>) in the desired Addon are not supported. Add the Components of matching or higher version to the image.

• Examples of impacted components include:

  • Cisco Ethernet native driver (nenic)

  • Cisco Fibre Channel native driver (nfnic)

  • Intel Virtual RAID on CPU Driver for ESXi (iavmd)

  • Intel RDMA auxiliary driver (irdman)

  • Broadcom NetXtreme-E / RoCE drivers

  • QLogic NetXtreme II drivers

Environment

• VMware vSphere ESXi 8.x

• VMware vCenter Server 8.x

• vSphere Lifecycle Manager image-based cluster remediation

Cause

vSphere Lifecycle Manager (vLCM) image remediation does not proceed when the desired image downgrades manually added, OEM, or partner async components. This occurs when the ESXi host contains newer driver components or VIBs than the versions included in the desired vLCM image or vendor addon.

The desired image must contain matching or higher component versions, or the conflicting unused components must be removed or aligned before remediation.

Common scenarios include:

• The host was initially provisioned or refreshed using an OEM custom ESXi ISO (e.g., Dell, Cisco, HPE).

• The host was upgraded to a newer ESXi release while retaining newer OEM/async driver components from an earlier installation.

• A previous baseline, image, or manual esxcli software vib/component install operation installed newer component versions.

• The desired vLCM image or vendor addon contains older versions than what currently resides on the host.

Example:

• Host installed component: MRVL-E4-CNA-Driver-Bundle 5.0.402.0-1OEM

• Desired image component: MRVL-E4-CNA-Driver-Bundle 5.0.305.0-1OEM

In this condition, vLCM detects that remediation requires a component downgrade and blocks the operation.

Resolution

IMPORTANT: Do not remove or replace driver components until confirming they are not used by active hardware. Removing a driver that is bound to an active storage, network, or boot device causes host connectivity or boot issues.

Before making changes, verify the following prerequisites:

• Place the ESXi host into maintenance mode.

• Ensure ESXi host management access is available.

• Confirm all virtual machines are migrated or powered off.

• Ensure the host can be recovered if a driver removal causes an unexpected impact.

• Validate driver usage on each affected host individually.

Step 1: Identify the components blocking vLCM remediation

1. From the vLCM compliance or Full image comparison view, identify the components where the Host Version is newer than the Image Version.

2. On the ESXi host, collect the installed component list: esxcli software component list | grep -E "MRVL-E4|MRVL-E3|Broadcom-bnxt|Intel-Volume-Mgmt|Intel-irdman|nenic|nfnic|iavmd|irdman|bnxt"

3. Collect the installed VIB list: esxcli software vib list | grep -E "bnxtnet|bnxtroce|iavmd|irdman|qfle3|qfle3f|qfle3i|qcnic|qedentv|qedrntv|qedf|qedi"

Step 2: Verify whether the drivers are in use

1. List the kernel modules bound to PCI hardware: esxcli hardware pci list | awk '/Module Name:/ {print $3}' | sort -u

2. Check whether the suspected modules are loaded: esxcli system module list | grep -E "bnxtnet|bnxtroce|iavmd|irdman|qfle3|qfle3f|qfle3i|qcnic|qedentv|qedrntv|qedf|qedi"

3. Check physical NIC driver usage: esxcli network nic list

   • Note: Review the Driver column. If any NIC is using one of the flagged drivers, do not remove that component until the hardware dependency is reviewed.

4. Check storage adapter driver usage: esxcli storage core adapter list

   • Note: If storage adapters show drivers such as qedf, qedi, or another flagged storage driver, do not remove that component until the storage dependency is reviewed.

5. Check NVMe controller usage if Intel VMD / VROC is involved: esxcli storage nvme controller list

   • Note: If a controller or boot device is backed by iavmd, do not remove or downgrade iavmd without hardware vendor validation.

Step 3: Decide whether to add matching/newer components to vLCM or remove unused stale components

Option A: Add matching or newer components to the vLCM image

Use this option when the driver is required or is actively bound to hardware.

1. Download the matching or newer supported offline bundle (.zip) component for the ESXi release from the Broadcom Support Portal.

   • "Manually added component <VIB Name> is an unsupported downgrade from version <version> on the host" while running check compliance on the vLCM Image cluster

2. Import the offline bundle into the vLCM depot via Lifecycle Manager > Settings > Patch Setup.

3. Edit the cluster image.

4. Add the matching or newer component under Components.

5. Validate the image.

6. Run the vLCM compliance/pre-check again.

7. Remediate the host or cluster.

   • Note: This aligns the desired image with the currently installed host component version and avoids downgrade errors such as: Manually added component <VIB Name> is an unsupported downgrade from version <version>

Option B: Remove unused stale components from the ESXi host

Use this option only when the driver/component is confirmed not to be used by active hardware.

1. Place the host into maintenance mode.

2. Enable SSH on the ESXi host.

3. Validate that the flagged components are not bound to physical NICs, storage adapters, NVMe controllers, or boot devices.

4. Remove only the unused components.

   • Example commands:

     • esxcli software component remove -n MRVL-E4-CNA-Driver-Bundle

     • esxcli software component remove -n MRVL-E3-Ethernet-iSCSI-FCoE

     • esxcli software component remove -n Intel-irdman

     • esxcli software component remove -n Broadcom-bnxt-Net-RoCE

     • esxcli software component remove -n <Component-Name>

5. Reboot the host:

   • reboot

6. Rerun the vLCM Run Pre-Check after the host returns.

7. Continue with normal vLCM image remediation if the pre-check succeeds.

Step 4 (If needed): Handle components marked required by the vendor addon

Some components may not be removable because the vendor addon marks them as required. For example, removing Intel-Volume-Mgmt-Device may fail with: 

[ProfileValidationError] Profile <profile-name> is missing component(s) Intel-Volume-Mgmt-Device which are necessary and removing them is prohibited. Ensure that these components are fully installed or overridden without partial VIB replacement or removal.

• Note: In this condition, do not continue trying to remove the required component as a normal component removal.

Instead, align the component with a supported version using one of the following methods:

1. Add a matching or higher supported version of the component to the vLCM image (as described in Option A).

2. Install the vendor/addon-compatible VIB version if directed by vendor guidance, ensuring it is validated for the ESXi version and hardware. Example: esxcli software vib install -v "/vmfs/volumes/<datastore>/<path>/INT_bootbank_iavmd_<version>.vib"

3. Reboot the host after installing or aligning the required VIB: reboot

4. Rerun the vLCM Run Pre-Check.

Additional Information

Related KB articles:

• KB 392092: Manually added component <VIB Name> is an unsupported version in the desired vLCM image

• KB 394653: Error: Downgrades of host VIBs <name>(<version>) in the desired ESXi version are not supported

• KB 320562: vSphere Lifecycle Manager image compliance check fails with “Host not compatible with the image”

• KB 373802: Upload ESXi ZIP file to Image Depot on the Lifecycle Manager

Component to driver mapping examples:

Component	Common driver/VIB names	Description
Broadcom-bnxt-Net-RoCE	bnxtnet, bnxtroce	Broadcom NetXtreme-E network/RoCE drivers
Intel-Volume-Mgmt-Device	iavmd	Intel VMD / Intel VROC driver
Intel-irdman	irdman	Intel X722/E810 RDMA auxiliary driver
MRVL-E3-Ethernet-iSCSI-FCoE	qfle3, qfle3f, qfle3i, qcnic	QLogic/Marvell NetXtreme II E3 drivers
MRVL-E4-CNA-Driver-Bundle	qedentv, qedrntv, qedf, qedi	Marvell FastLinQ E4 network/RDMA/FCoE/iSCSI drivers