Canceling ESXi Maintenance Mode with Full Data Migration in VMware vSAN

Products

• VMware vSAN
• VMware vSphere ESXi

Issue/Introduction

Initiating maintenance mode with the Full data migration option in a VMware vSAN cluster starts immediate object resynchronization. this process can cause significant storage performance degradation. If maintenance mode task initiated accidentally, Canceling the task stops unnecessary data movement and restores normal cluster operations.

Environment

• VMware vSAN 7.x
• VMware vSAN 8.x
• VMware vSAN 9.x

Resolution

Cancel the maintenance mode task:

1. Login to effectuated host via SSH using root credentials.

2. Run the following command to cancel the maintenance mode task:
   
   esxcli vsan maintenancemode cancel

3. If steps above does not resolve the issue, try the following: 

   1. 1. Make sure all virtual machine are migrated from host affected

      2. Login to ESXI host (web client) and EXIT host from maintenance mode.

      3. If the host fails to exit maintenance mode, restart the management agents on the host:

         1. 1. /etc/init.d/vpxa restart

            2. /etc/init.d/hostd restart

            3. /etc/init.d/vsanmgmtd restart

      4. After restarting management services, Exit host from maintenance mode using from vCenter.

Validate whether the maintenance mode task was successfully canceled:

1. Login to effectuated host via SSH using root credentials.
2. Run the below command. 
   
   vim-cmd vimsvc/task_list 

'vim.Task:haTask-ha-host-vim.HostSystem.enterMaintenanceMode-#####',

• If the maintenance mode task lists on the above mentioned command, Run the below command to confirm the task is already stopped. 
  
  vim-cmd vimsvc/task_info <TASKNAME>,
  
  Example:
  
  vim-cmd vimsvc/task_info 'vim.Task:haTask-ha-host-vim.HostSystem.enterMaintenanceMode-#####'

vmodl.fault.ManagedObjectNotFound) {
   faultCause = (vmodl.MethodFault) null,
   faultMessage = <unset>,
   obj = 'vim.Task:vim.Task:haTask-ha-host-vim.HostSystem.enterMaintenanceMode-#####,'
   msg = "Received SOAP response fault from [<<io_obj p:0x#########, h:6, <TCP '###.#.#.# : 20989'>, <TCP '###.#.#.#: 8307'>>, /sdk>]: GetInfo
The object 'vim.Task:vim.Task:haTask-ha-host-vim.HostSystem.enterMaintenanceMode-#####,' has already been deleted or has not been completely created"
}

Additional Information

• Full data migration will move 100% of the data off the host. This is generally only required if you are permanently decommissioning/wiping the host.
• Ensure accessibility will move only what is necessary to keep the VMs running. This is recommended for patching, reboots, or hardware swaps.