ESXi Host May Unexpectedly Exit Maintenance Mode After Reboot During Patching, Upgrades, or VIB Operations

Products

• VMware vSphere ESXi

Issue/Introduction

• An ESXi host may not retain its maintenance mode state across a reboot if patching, upgrading, or VIB installation, update, or removal is executed prior to enabling maintenance mode.
  
  
• An ESXi host is placed into maintenance mode for planned maintenance activities.
  
  
• A Patch/VIB install, removal or update operation is performed before the host enters maintenance mode.
  
  
• The host is rebooted after the VIB removal.
  
  
• Upon restart, the host exits maintenance mode automatically.

Environment

VMware vSphere ESXi

Cause

• ESXi manages configuration states through a rotating partition system consisting of the bootbank and altbootbank. Whenever a VIB operation (install, update, or removal) occurs, the altbootbank is updated to reflect the current system state. Upon the next reboot, a "bank rotation" occurs: the altbootbank is promoted to become the active bootbank, while the previous bootbank is relegated to the altbootbank role.
  
  
• Any configuration changes made after a VIB operation (which do not involve VIB modifications) are recorded only in the currently active bootbank. This active partition is scheduled to become the standby altbootbank following a successful reboot.
  
  
• In the event of a non-graceful shutdown or hardware reset, ESXi fails to perform the final synchronization of the current state. This results in the loss of recent configuration data such as Maintenance Mode status as the system boots from a partition that was never updated with the most recent changes. Conversely, during a graceful reboot or shutdown, ESXi executes a full state synchronization to the altbootbank, ensuring all settings, including Maintenance Mode, are retained post-boot.

Resolution

To prevent this behavior, follow these operational best practices:

• Place ESXi Host in Maintenance Mode Before VIB Operations
  
  
  • Always ensure the host is in maintenance mode before performing any VIB removal, update, or installation.
    
    
• Check Maintenance Mode Status Before performing any such patching/update/upgrade activity
  
  
• Using vSphere Client UI:
  
  
  • Navigate to the Hosts and Clusters view in vSphere Client.
  • Select the ESXi host.
  • Check the Summary tab → Maintenance Mode field to confirm whether the host is in maintenance mode.
    
    
• Using ESXi CLI:
  
  
  • Connect to the ESXi host via SSH.
  • Run the following command:vim-cmd hostsvc/hostsummary | grep inMaintenanceMode

    Sample Output:
    
    true → Host is in maintenance mode.
    
    false → Host is in normal mode.

• It is recommended to use vLCM workflows as they maintain state consistency across partitions automatically in case of update/upgrade based activity.

Additional Information

This behavior is caused by a synchronization gap between the two boot partitions. The altbootbank captures the system's operational state specifically at the time of a VIB removal. When the host is subsequently placed into Maintenance Mode, that change is not mirrored to the altbootbank. Consequently, when the partitions are swapped during the next reboot, the altbootbank is promoted to the active bootbank, causing the host to revert to the state recorded prior to the Maintenance Mode transition. Adhering to the recommended sequence ensures the intended operational state is preserved and prevents an unexpected exit from Maintenance Mode.