Inaccessible objects in vSAN when all components show as Active
search cancel

Inaccessible objects in vSAN when all components show as Active

book

Article ID: 326833

calendar_today

Updated On:

Products

VMware vSAN

Issue/Introduction

Symptoms:
In vCenter you see inaccessible objects but no cluster partitions, failed disks, or hosts in maintenance mode/vSAN Decomm state
Running esxcli vsan debug object list --all|less you see similar output for the inaccessible objects: 
Object UUID: c0473b62-####-####-####-#########cb
   Version: 15
   Health: inaccessible - Object is initializing or creating.(APD)
   Owner: esxi1.example.com
   Size: 0.00 GB
   Used: 0.01 GB
   Policy:
   Configuration:

      RAID_1
         Component:c0473b62-####-####-####-#########cb
            Component State: ACTIVE,  Address Space(B): 1073741824 (1.00GB),  Disk UUID: 5226d47b-####-####-####-#########be,  Disk Name: mpx.vmhba2:C0:T1:L0:2
           Votes: 1,  Capacity Used(B): 12582912 (0.01GB),  Physical Capacity Used(B): 4194304 (0.00GB),  Host Name: esxi2.example.com
         Component: c1488363-####-####-####-#########d5
           Component State: ACTIVE,  Address Space(B): 1073741824 (1.00GB),  Disk UUID: 52a11798-####-####-####-#########c8,  Disk Name: mpx.vmhba2:C0:T1:L0:2
           Votes: 1,  Capacity Used(B): 12582912 (0.01GB),  Physical Capacity Used(B): 4194304 (0.00GB),  Host Name: esxi1.example.com
      Witness: 01d4e663-####-####-####-#########d5
        Component State: ACTIVE,  Address Space(B): 0 (0.00GB),  Disk UUID: 5205d73b-####-####-####-#########35,  Disk Name: mpx.vmhba1:C0:T1:L0:2
        Votes: 1,  Capacity Used(B): 12582912 (0.01GB),  Physical Capacity Used(B): 4194304 (0.00GB),  Host Name: esxi4.example.com

   Type: vdisk
   Path: /vmfs/volumes/vsan:52a9537793######-c08e3f91e6######/047d3762-####-####-####-#########cb/New Virtual Machine_1.vmdk (Missing)
   Group UUID:047d3762-####-####-####-#########cb    Directory Name: None


Environment

VMware vSAN 7.0.x
VMware vSAN 8.0.x

Cause

Advanced setting /VSAN/DOMPauseAllCCPs is set to 1 on one or more hosts in the cluster
Setting this option will block all CCP in owner setup, resulting in objects going to APD state. This setting is used for cluster reboot/shutdown

This issue may occur when moving vSAN hosts to another vSAN cluster. If we choose to 'shutdown' a vSAN cluster,  /VSAN/DOMPauseAllCCPs in the hosts will be set to 1. Then after moving these hosts to another existing vSAN cluster, re-creating disk groups, some components in DOM objects will be migrated to these new hosts for re-balance purpose. As /VSAN/DOMPauseAllCCPs is 1 in these hosts, the components are blocked and results in inaccessible DOM objects.

Resolution

When dealing with a large number of inaccessible objects do the following:
Run command "esxcfg-advcfg -g /VSAN/DOMPauseAllCCPs" on  all hosts to get the current value,

 

[root@esxi1:~] esxcfg-advcfg -g /VSAN/DOMPauseAllCCPs
Value of DOMPauseAllCCPs is 1

 

If the value is 1 means this is enabled and objects owned by this host will be in an inaccessible state until the setting is reverted to the default value of 0 by running esxcfg-advcfg -s 0 /VSAN/DOMPauseAllCCPs on the affected hosts.

When dealing with very few inaccessible objects you can follow the above steps just on the owning host for the inaccessible objects.

Additional Information

Impact/Risks:
Objects become inaccessible

Powered by Wolken Software