The ESXi host failed to enter maintenance mode in the vSAN cluster when the user tried to shut it down via the manual method.

Products

• VMware vSAN

Issue/Introduction

Symptoms:

The user is trying to manually shut down and restart the vSAN cluster using the following method.

Manually Shut Down and Restart the vSAN Cluster

However, some ESXi hosts won't enter the maintenance mode when the user places all the hosts into maintenance mode with no action mode as per the above document.

When checking the "vobd" logs, the user notices the following excerpt.

2025-02-06T12:52:09.082Z No(29) clomd[2098774]: [Originator@6876] ClomDecomMultipleDecomCheck: Going to fail decom xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx with VMK_BUSY as there are more than one decom entities in cmmds. decomType Node, evacReason 0, IsOtherDecomDdh 0, IsOtherNodeDecomWaiting 1.

Cause

The reason why the ESXi host won't enter the maintenance mode is because there is another ESXi host in the process of entering the maintenance mode. The user can confirm this with the following log excerpt from the vobd logs.

2025-02-06T12:52:09.082Z No(29) clomd[2098774]: [Originator@6876] ClomDecomMultipleDecomCheck: Going to fail decom xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx with VMK_BUSY as there are more than one decom entities in cmmds. decomType Node, evacReason 0, IsOtherDecomDdh 0, IsOtherNodeDecomWaiting 1.

If the user notice the message "IsOtherNodeDecomWaiting 1", it indicates that another ESXi host is concurrently attempting to enter the maintenance mode.

This happens because the user is running the following command on all ESXi hosts at the same time to manually shut down the vSAN cluster.

esxcli system maintenanceMode set -e true -m noAction

And if the previous maintenance mode task for the ESXi host has not completed and the next ESXi host receives the request for the maintenance mode, then we expect the maintenance mode task to fail and log the above-mentioned excerpt in the vobd log file.

 

Resolution

There are two resolutions:

• The first resolution is to use the "Shutdown vSAN cluster wizard." The Shutdown cluster wizard can be used to gracefully shut down the vSAN cluster for maintenance or troubleshooting. Please check the following link for the details.

  Shut Down the vSAN Cluster Using the Shutdown Cluster Wizard

• If the user has to use the manual method, then the recommendation would be to check the output of the following command prior to putting down the next ESXi host in maintenance mode.

  esxcli system maintenanceMode get

  If user see the output as "Enabled: True" then it means that the ESXi host is in maintenance mode, and then the user can proceed with putting the next ESXi host in maintenance mode. If the output is "False," then the user can wait till the output of the command turns to "True" and then proceed with putting the next ESXi host in maintenance mode.