vSAN Object Stuck in Reduced Availability State After Cluster Scale-Down
search cancel

vSAN Object Stuck in Reduced Availability State After Cluster Scale-Down

book

Article ID: 400420

calendar_today

Updated On:

Products

VMware vSAN VMware vSAN 6.x VMware vSAN 7.x VMware vSAN 8.x

Issue/Introduction

  • You see one object in a "Reduced availability with no rebuild" state: 

  • Similarly, you may also see a failed pre-check error: "The host cannot enter maintenance mode." "1 more standalone host is required", when trying to place the host you're removing into maintenance mode with Full Data Evacuation:

  • this issue often involves "non-local VMs", stale .vmdk.REDO_ files, or orphaned objects.

Environment

  • VMware vSAN (all versions)
  • vSAN ESA
  • Azure VMware Solution (AVS)

Cause

The object is governed by a storage policy (e.g., RAID 6) that requires more hosts than currently available in the cluster after a scale-down or host removal.
 For example:  if the object has a RAID 6 policy applied, but the vSAN cluster now has less than 6 hosts. 

  • To verify, you can login to one of the hosts in the cluster and confirm which policy the object has in the GUI
    Cluster -> Monitor Tab -> vSAN -> Virtual Object -> Select the check box of the object and click View Placement Details


  • From an ssh session to one of the ESXi hosts in the cluster and run the command following command:

 

 

esxcli vsan debug object list --all -u <UUID>

Object UUID: ########-####-####-#############

...
   Policy:
            ...
      spbmProfileName: vSAN Default Storage Policy
      
   Configuration:
      
      RAID_6
         Component: ######-####-####-#############
           Component State: ACTIVE,  Address Space(B): 68451041280 (63.75GB), Disk UUID: ######-####-####-#############, Disk Name naa.##############:2
             Votes: 2, Capacity Used(B): 176160768 (0.16GB), Physical Capacity Used(B): 171966464(0.16GB), Total 4K Blocks Used(B): 76054528 (0.07GB), Host Name:  myvsanhost20.testlab.local
             
         Component: ######-####-####-#############
           Component State: ACTIVE,  Address Space(B): 68451041280 (63.75GB), Disk UUID: ######-####-####-#############, Disk Name naa.##############:2
             Votes: 2, Capacity Used(B): 146800640 (0.14GB), Physical Capacity Used(B): 142606336(0.13GB), Total 4K Blocks Used(B): 78524416 (0.07GB), Host Name:  myvsanhost22.testlab.local
             
         Component: ######-####-####-#############
           Component State: ACTIVE,  Address Space(B): 68451041280 (63.75GB), Disk UUID: ######-####-####-#############, Disk Name naa.##############:2
             Votes: 2, Capacity Used(B): 159383552 (0.15GB), Physical Capacity Used(B): 155189248(0.14GB), Total 4K Blocks Used(B): 77770752 (0.07GB), Host Name:  myvsanhost19.testlab.local
             
         Component: ######-####-####-#############
           Component State: ACTIVE,  Address Space(B): 68451041280 (63.75GB), Disk UUID: ######-####-####-#############, Disk Name naa.##############:2
             Votes: 2, Capacity Used(B): 1635777856 (0.15GB), Physical Capacity Used(B): 159383552(0.15GB), Total 4K Blocks Used(B): 80252928 (0.07GB), Host Name:  myvsanhost17.testlab.local
             
         Component: ######-####-####-#############
           Component State: ACTIVE,  Address Space(B): 68451041280 (63.75GB), Disk UUID: ######-####-####-#############, Disk Name naa.##############:2
             Votes: 2, Capacity Used(B): 188743680 (0.18GB), Physical Capacity Used(B): 184549376(0.17GB), Total 4K Blocks Used(B): 80494592 (0.07GB), Host Name:  myvsanhost23.testlab.local
             
         Component: ######-####-####-#############
           Component State: ABSENT,   CSN: STALE (15795!=15855), Address Space(B): 68451041280 (63.75GB), Disk UUID: ######-####-####-#############, Disk Name naa.##############:2
             Votes: 1,  Host Name: None
             
   Type: vmnamespace
   Path: /vmfs/volumes/vsan:################-################/appvolumes (Exists)
   Group UUID:  ########-####-####-####-################
   Directory Names: appvolumes

 

Resolution

If the object is an ordinary VM (or vmdk, vmnamespace or swap object related to an ordinary VM) and if the policy looks correct (e.g. the assigned policy is the vSAN Default Policy [left at the default RAID1])

For non ordinary VM objects (e.g. non-local VM's, appvolumes directories, content libraries, .iso's, orphaned, unassociated, etc. )

Additional Information