ESXi host fails to enter Maintenance Mode and stuck at 87 %
search cancel

ESXi host fails to enter Maintenance Mode and stuck at 87 %

book

Article ID: 432742

calendar_today

Updated On:

Products

VMware vSAN

Issue/Introduction

  • An ESXi host in a vSAN cluster remains stuck while entering Maintenance Mode using Ensure Accessibility or Full Data Migration. \
  • The maintenance mode task takes longer than expected and may eventually be cancelled manually.
  • Cluster Type : Standard VSAN cluster 

Environment

VMware vSAN 8.x

Cause

The issue occurs when a vSAN object or virtual machine is associated with a storage policy designed for a vSAN Stretched Cluster or 2-Node cluster.

The policy contains the rule as subFailuresToTolerate.This rule is only supported in stretched or 2-node vSAN configurations. When the environment is a standard vSAN cluster, the vSAN Cluster Level Object Manager (CLOM) cannot process the rule during object evacuation or reconfiguration.

As a result, CLOM returns a Bad parameter error and fails the reconfiguration task, preventing the host from completing the maintenance mode operation.

Cause Validation: 

During maintenance mode operations, vSAN attempts to evacuate or reconfigure objects residing on the host. However, the process fails due to storage policy validation errors.

Example clomd.log entry: 

Log path  : less /var/run/log/clomd.log

2026-02-27T03:06:03.945Z No(29) clomd[62980309]: [Originator@6876] CLOM_ProcessDecomUpdate: Node ########## state change. Old:DECOM_STATE_CANCEL New:DECOM_STATE_NONE Mode:0 evacReason:0 JobUuid:00000000-0000-0000-0000-000000000000
2026-02-27T03:06:03.945Z Er(27) clomd[62980309]: [Originator@6876] CLOMDecomCleanupInMemState: Cleaning in-memory state
2026-02-27T03:06:03.945Z No(29) clomd[62980309]: [Originator@6876] CLOMDecomCleanupObjLists: Cleaning up decommissioning lists.
2026-02-27T03:06:03.945Z No(29) clomd[62980309]: [Originator@6876] CLOMDecomCMMDSResponseCb: CMMDS update response received: Success
2026-02-27T03:06:03.945Z Cr(26) clomd[62980309]: [Originator@6876] CLOMDecomCMMDSResponseCb: Failed to get the decommissioning entity Not found
2026-02-27T03:06:04.944Z No(29) clomd[62980309]: [Originator@6876] CLOM_PostWorkItem: Posted a work item opID:1804317545 for ########## group: 00000000-0000-0000-0000-000000000000 Type: REPAIR delay 0 (Success)
2026-02-27T03:06:04.957Z No(29) clomd[62980309]: [Originator@6876 opID=1804317545] CLOMProcessWorkItem: Op REPAIR starts:1804317545
2026-02-27T03:06:04.960Z No(29) clomd[62980309]: [Originator@6876 opID=1804317545] CLOMReconfigure: Reconfiguring ########## workItem type REPAIR
2026-02-27T03:06:04.960Z Cr(26) clomd[62980309]: [Originator@6876 opID=1804317545] CLOMValidateAndUpdateTargetPolicy: subFailuresToTolerate is not supported in non-stretch!
2026-02-27T03:06:04.960Z Cr(26) clomd[62980309]: [Originator@6876 opID=1804317545] CLOMReconfigure: exit: obj ########## transiantCapGenerated - total: 0, site1: 0, site2: 0, workItem type REPAIR configDelay 0 newConfigGenerated 0 newCompWitnessOnly 0 status Bad parameter

2026-02-27T03:06:04.960Z No(29) clomd[62980309]: [Originator@6876 opID=1804317545] CLOM_PublishResyncBytes: About to fail the workItem(Bad parameter), reset queued resync bytes to 0
2026-02-27T03:06:04.961Z No(29) clomd[62980309]: [Originator@6876 opID=1804317545] CLOMProcessWorkItem: Op ends:1804317545
2026-02-27T03:06:05.945Z No(29) clomd[62980309]: [Originator@6876] CLOM_PostWorkItem: Posted a work item opID:1804317546 for ########## group: 00000000-0000-0000-0000-000000000000 Type: REPAIR delay 0 (Success)
2026-02-27T03:06:05.958Z No(29) clomd[62980309]: [Originator@6876 opID=1804317546] CLOMProcessWorkItem: Op REPAIR starts:1804317546
2026-02-27T03:06:05.959Z No(29) clomd[62980309]: [Originator@6876 opID=1804317546] CLOMReconfigure: Reconfiguring ########## workItem type REPAIR
2026-02-27T03:06:05.960Z Cr(26) clomd[62980309]: [Originator@6876 opID=1804317546] CLOMValidateAndUpdateTargetPolicy: subFailuresToTolerate is not supported in non-stretch!
2026-02-27T03:06:05.960Z Cr(26) clomd[62980309]: [Originator@6876 opID=1804317546] CLOMReconfigure: exit: obj ########## transiantCapGenerated - total: 0, site1: 0, site2: 0, workItem type REPAIR configDelay 0 newConfigGenerated 0 newCompWitnessOnly 0 status Bad parameter

The policy is set as "R1 Local - R1 Remote - Storage Policy"  and subFailuresToTolerate: 1 which is not supported in non-stretch.

  Command : localcli vsan debug object list 

  • structtype:
      ObjectInfo
    Health:
      healthy
    Object UUID:
      ##########
    Version:
      20
    Owner:
      Host
    Policy:
      stripeWidth: 1
          cacheReservation: 0
          proportionalCapacity: 0
          hostFailuresToTolerate: 1
          forceProvisioning: 0
          spbmProfileId: 7
          spbmProfileGenerationNumber: 0
          objectVersion: 20
          replicaPreference: Performance
          iopsLimit: 0
          checksumDisabled: 0
          subFailuresToTolerate: 1
          CSN: 25095
          SCSN: 25094
          spbmProfileName: R1 Local - R1 Remote - Storage Policy
          locality: None
    Used:
      25987907584
    Used 4K Blocks:
      22535139328
    Size:
      42949672960
    Type:
      vdisk
    Path:
      /vmfs/volumes/vsan:##########/######/#########.vmdk (Exists)
    Group UUID:
    #########
    Directory Name:
      N/A
    Configuration:
          RAID_1
             RAID_1
                Component: #####
                  
             RAID_1
                C#####bcp.cat.com
          Witness: #######
            Component State: ABSENT,  CSN: STALE (24611!=25095),  Address Space(B): 0 (0.00GB),  Disk UUID: 52c2d379-444b-c2b8-a955-001540d1c10b,  Disk Name: N/A
            Votes: 3,  Host UUID: None
          Witness: #########
            Component State: ACTIVE,  Address Space(B): 0 (0.00GB),  Disk UUID: ######,  Disk Name: naa.####:2
            Votes: 1,  Capacity Used(B): 12582912 (0.01GB),  Physical Capacity Used(B): 4194304 (0.00GB),  Total 4K Blocks Used(B): 4218880 (0.00GB),  Host Name: ###.
          Witness: ######
            Component State: ACTIVE,  Address Space(B): 0 (0.00GB),  Disk UUID: ######,  Disk Name: naa.#######:2
            Votes: 1,  Capacity Used(B): 12582912 (0.01GB),  Physical Capacity Used(B): 4194304 (0.00GB),  Total 4K Blocks Used(B): 4218880 (0.00GB),  Host Name: ####.
 

Resolution

1. If the ESXI host still in maintenance mode entering state exit it by running below command from cli 

    esxcli system maintenanceMode set --enable false

2. Identify the storage policy applied to the affected vSAN objects (for example: "R1 Local - R1 Remote - Storage Policy").

3. Verify the policy rules and confirm whether the policy includes the subFailuresToTolerate rule.

4. Modify the storage policy to remove the unsupported rule or create a new policy that is supported for a standard vSAN cluster.

5. Reassign the affected virtual machines or objects to the updated policy.

6. Wait for the objects to become compliant with the new policy.

7. Retry placing the ESXi host into Maintenance Mode.