ESXi host stuck "in progress" when entering maintenance mode

Products

• VMware vCenter Server
• VMware vSphere ESXi

Issue/Introduction

An ESXi host that has been instructed to enter maintenance mode may show "in progress" for an indefinite amount of time under Tasks in the vCenter server.

• No virtual machines are being migrated
• It may or may not be able to cancel the maintenance mode task
• Other tasks by vCenter are unable to be processed for the ESXi host.

The status bar does not move and the following message is displayed: 

 

Waiting for all VMs to be powered off or suspended or migrated. In a DRS cluster check the Faults page on the DRS tab for troubleshooting.

Cause

Before a host can enter maintenance mode all virtual machines need to be powered off or migrated to another host.

• For VMs to be moved automatically while entering maintenance mode, DRS must be enabled and in Fully Automated mode
• A quick test to determine if all virtual machines can be vMotioned manually can often expose the real reason for the Maintenance mode problem.
• Common problems are a connected CD-ROM or if the virtual machine is on local storage. These will prevent vMotion from being possible. It is prudent to verify all vMotion validation requirements before proceeding.

Resolution

If the ESXi never times out and is stuck at "In Progress" and the ESXi host appears to be in normal state when logged into its host client, perform the following steps:

• Look in the DRS settings for the cluster. If DRS is not enabled or not set to fully automated, cancel the maintenance mode action and retry maintenance mode after changing DRS settings.
  
  
  • Or migrate the VMs manually.
    
    
• If the maintenance mode action is not visible in vSphere Client, or a cancellation action does not appear to take effect, refresh the page.
  
  
• Review the VMs tab on the host to see if some but not all VMs could be migrated. If some VMs remained, try to migrate them manually and watch for pre-check messages indicating compatibility errors or conditions that prevent a migration.
  
  
• Verify the date and time is correct in the environment, including vCenter and all ESXi hosts. It is possible the host lost communication with the vCenter due to the date/time being out of sync.
  
  
  • Does the host say Disconnected or Not Responding?
  • For vCenter, review Date/Time config in the VAMI (port 5480) interface.
  • For ESXi, review the configuration tab for the host.
    
    
• Note: Before performing the following steps, please refer to Restarting the Management agents in ESXi for important information on restarting the Management agents.
  
  
  • Restart the VMware Virtual Center service (vpxd). For steps on restarting the Virtual Center Server service, see: Stop, Start or Restart Services on vCenter Server 6.x or Higher.