Unable to enter maintenance mode for last ESXi host in cluster when vCLS does not stop
search cancel

Unable to enter maintenance mode for last ESXi host in cluster when vCLS does not stop

book

Article ID: 403400

calendar_today

Updated On:

Products

VMware vCenter Server

Issue/Introduction

Symptoms  

  • When you attempt to put the last ESXi host in a cluster into maintenance mode, vCLS normally stops automatically—allowing the host to enter maintenance mode. However, if the ESXi host you’re targeting is also running the vCenter Server VM and the vCenter Server service is shut down before the vCLS agents, the vCLS VMs will remain running and the host will fail to enter maintenance mode.

Environment

  • VMware vSphere 7.0.x / 8.0.x

  • VMware vSAN 7.x / 8.x

Cause

vSphere Cluster Services (vCLS) are managed by vCenter Server. When an ESXi host enters Maintenance Mode, it expects vCenter to shut down or migrate these agent VMs. If vCenter is offline, the local HA agent keeps the vCLS VMs powered on, which blocks the host's transition to Maintenance Mode.

Resolution

Method 1: vCenter is Accessible

  1. Shut down all virtual machines on the ESXi host except the vCenter Server VM and the vCLS VMs.

  2. Before entering maintenance mode on the last ESXi host in the cluster, put the cluster into Retreat Mode to delete the vCLS VMs.

  3. Using the Host Client, shut down the vCenter Server VM on that ESXi host.

  4. In the Host Client, place the ESXi host into Maintenance Mode.

Method 2: vCenter is Offline (Manual CLI)

If vCenter is offline and you cannot enable Retreat Mode via the GUI, perform these steps on the ESXi host terminal:

  1. Stop the HA Agent to prevent it from restarting VMs automatically:

    /etc/init.d/vmware-fdm stop

  2. List all VMs to identify the vCLS VM ID: Refer Powering off an unresponsive virtual machine on an ESXi host

    vim-cmd vmsvc/getallvms | grep -i vcls

  3. Power off the vCLS VM using the ID from the first column:

    vim-cmd vmsvc/power.off ####

  4. Cancel any stuck maintenance mode tasks and retry: Refer Proper use of esxcli/localcli vsan maintenancemode cancel

    esxcli/localcli vsan maintenancemode cancel

    esxcli system maintenanceMode set --enable true -m noAction

  5. Once maintenance tasks are complete, remember to restart the HA agent if the host remains in the cluster:

    /etc/init.d/vmware-fdm start

Note: When vCenter Server is restored, it will automatically redeploy new vCLS VMs once Retreat Mode is disabled or services are stabilized.

Additional Information

Powered by Wolken Software