The ESXI Host takes very long time entering maintenance mode

Products

• VMware vSAN

Issue/Introduction

This article explains the issue in vSAN cluster where ESXi host will take very long time to enter the Maintenance Mode  ( Default Mode - Ensure Accessibility ) 

• An ESXi host has been put in "maintenance mode" may show "in progress"  and takes very long time to finish the operation
• The vSAN Skyline Health may show all the objects are healthy and no other issue in the vSAN cluster. 

Environment

VMware ESXi (All Versions)

Cause

• Some of the VM are configured with VM Storage policy which has been set with "No data redundancy" 
  
  
• The time taken for a host to enter Maintenance Mode in a vSAN cluster, where objects are configured with an FTT=0 policy (residing on the host), can be significantly longer depending on the size of the objects. Larger objects may require more time to migrate to other hosts in the cluster before the Maintenance Mode operation can be completed.
  
  
• To identify the VM(s) configured with "No data redundancy" 
  
  
  • The VM Storage policy of the VM can be found  in Host and Cluster view  > Select VM  > Configure > Policies  > VM Storage Policy
    
    
  • The attributes the of the respective VM Storage Policy can be found in : vCenter UI client > vSphere Client > Under Monitoring menu > VM Storage Policies > Select the Policy > Select Rules from right bottom window 

 

 

• • • If VM(s) are configured with Storage Policy that has rule  "No data Redundancy"  ( also called as FTT=0 i.e. Failure to tolerate 0 ), means VMs are running on single copy of data 
      
      
    • The vSAN objects of the VM(s) configured with above policy will have singe component created on vSAN datastore shown as RAID 0 
       
    • VM(s) with FTT=0 :  Host and Cluster view  > select VM  > Monitor > Physical disk placement - see the image below
      
      
      
      
    • The name of VM Storage policy can be found from : Host and Cluster view  > Select VM  > Configure > policies  > VM Storage Policy

    • By default the Maintenance Mode is set with "Ensure Accessibility" which will ensure all the vSAN objects will be migrated to other hosts within the cluster only after which the Maintenance Mode operation will be marked as completed.

Resolution

• The VM Storage Policy with "No data redundancy" ( FTT=0) should be to changed to  "Failures to tolerate : 1 failure or 2 Failure"  ( FTT=1 )  ( depends on end users choice ) 
  
  
  • The change of policy should not  applied on all the VM as this will trigger cluster wide resynch and should be done only batches with close monitoring on resynch 
    
    
• The VM can be migrated off the vSAN storage or VM(s) can be deleted  ( if not needed )

Additional Information

Note : A detailed overview of the VM Storage Policies is provided in an article vSAN Policies ( refer section - Storage Policy - Availability )