Roll back incorrect host updates

Products

VMware vSphere ESXi

Issue/Introduction

Administrators may encounter a scenario where incorrect Vendor-Specific (OEM) updates are applied to an ESXi host (e.g., Cisco Custom Image VIBs installed on Lenovo hardware). This can lead to driver instability, boot failures, or compliance warnings.
This article addresses two states:

Installed: The wrong updates have been applied, and the host has rebooted.
Staged: The wrong updates have been uploaded (staged) to the host but not yet applied/rebooted.

Environment

VMware vSphere 7.x / 8.x
VMware ESXi
vSphere Lifecycle Manager (vLCM)

Cause

This issue is typically caused by selecting the incorrect Image, Baseline, or Depot file during a manual upgrade or vLCM remediation task.

Resolution

Scenario 1: Reverting Installed Updates (Host A)

If the host has already been remediated and rebooted with the incorrect VIBs, the most reliable method to return to a stable state is utilizing the ESXi Recovery Mode.

Procedure:

Connect to the console of the ESXi host (via IPMI, ILO, IMM, or physical monitor).
Reboot the host.
Monitor the boot sequence. When the hypervisor progress bar appears, look for the prompt "Recovery Mode" in the bottom right corner.
If Recovery Mode is available:
- Press Shift+R.
- The console will display the currently installed build and the previous build (rollback target).
- Press Y to confirm the rollback.
- The host will boot into the previous hypervisor version.
If Recovery Mode is NOT available:
- If the "Recovery Mode" prompt does not appear, or if the system reports that no previous version is available, the altbootbank partition is likely empty or corrupted.
- Action Required: You must reinstall ESXi to restore functionality.
- Reinstall Steps:
  1. Mount the correct vendor-specific ISO (e.g., Lenovo Custom Image) to the host via virtual media.
  2. Boot from the ISO.
  3. During the installation workflow, select the boot drive containing the existing installation.
  4. CRITICAL: When prompted, select "Install ESXi, preserve VMFS datastore". This ensures virtual machine data is retained while the hypervisor is overwritten.
  5. Complete the installation and re-register virtual machines if necessary.

Scenario 2: Clearing Staged Updates (Host B)

If the updates are "Staged" but not installed, you must clear the bootbank buffer or scratch location to prevent the host from installing them on the next reboot.

Method A: vSphere Lifecycle Manager (Images) If managing via vLCM Images, you cannot explicitly "un-stage." You must overwrite the staged content.

In vCenter, navigate to the Cluster > Updates > Image.
Select the correct Image (e.g., the valid Lenovo image or standard ESXi image).
Click Stage All.
vLCM will overwrite the incorrect staged files with the correct files.

Method B: Manual CLI Cleanup (ESXCLI/Baselines) If the files were staged manually or via Update Manager baselines, they reside in the scratch partition.

Connect to the host via SSH as root.

Place the host in Maintenance Mode:

esxcli system maintenanceMode set --enable true

Locate the scratch partition path:
```
esxcli system scratch get
```
(Note the output path, e.g., /vmfs/volumes/<UUID>/scratch)
Navigate to the downloads directory within the scratch partition:
```
cd /vmfs/volumes/<UUID>/scratch/downloads
```
Verify the incorrect files are present:
```
ls -l
```
Delete the contents of the downloads directory:
```
rm -rf *
```
Warning: Ensure you are in the correct directory before executing rm -rf.

Exit Maintenance Mode:

esxcli system maintenanceMode set --enable false