vSAN Health Service Reports Inaccessible Objects After Upgrade

Products

• VMware vSAN

Issue/Introduction

Symptoms 

• Inaccessible vSAN objects were reported under vSAN > Monitor > Health > vSAN Object Health.
• All the inaccessible objects are associated with only 1 host in vSAN cluster.
• vSAN Objects health status showing 

Health Status                                              Number Of Objects
---------------------------------------------------------  -----------------
remoteAccessible                                                           0
inaccessible                                                            1000
reduced-availability-with-no-rebuild                                       0
reduced-availability-with-no-rebuild-delay-timer                           0
reducedavailabilitywithpolicypending                                       0
reducedavailabilitywithpolicypendingfailed                                 0
reduced-availability-with-active-rebuild                                   0
reducedavailabilitywithpausedrebuild                                       0
data-move                                                                  0
nonavailability-related-reconfig                                           0
nonavailabilityrelatedincompliancewithpolicypending                        0
nonavailabilityrelatedincompliancewithpolicypendingfailed                  0
nonavailability-related-incompliance                                       0
nonavailabilityrelatedincompliancewithpausedrebuild                        0
healthy                                                                 1700

• Additionally, >200 inaccessible objects were identified as vswp (VM swap) files.

Validation Checks

• Host Decommission State:

[root@ESXi:~] cmmds-tool find -f json -t NODE_DECOM_STATE |grep -E "uuid|content"|sed 'N;s// /' |awk -F \" '{print $2 ":" $4": " $8 $9}' |sort;
uuid:623438f8-####-####-####-######c7d4c: decomState: 0,
uuid:6234392a-####-####-####-######c7d84: decomState: 0,
uuid:62343969-####-####-####-######c9ba4: decomState: 0,
uuid:62343974-####-####-####-######c9b1c: decomState: 0,
uuid:623439b9-####-####-####-######c9de4: decomState: 0,
uuid:623439ec-####-####-####-######c9df4: decomState: 0,
uuid:62343a15-####-####-####-######c7df5: decomState: 0,
uuid:62343a18-####-####-####-######c7d44: decomState: 0,
uuid:62343a28-####-####-####-######c8104: decomState: 0,
uuid:62343a3a-####-####-####-######c9dec: decomState: 0,
uuid:62343a50-####-####-####-######c4c64: decomState: 0,
uuid:62343a87-####-####-####-######c8e54: decomState: 0,

• • All hosts returned: decomState: 0
    → Indicates hosts are not marked for decommission.

• DOM CCP Pause State:

[root@ESXi:~] esxcfg-advcfg -g /VSAN/DOMPauseAllCCPs
Value of DOMPauseAllCCPs is 0

• • Output: Value of DOMPauseAllCCPs is 0
    → No ongoing pause that would impact object rebuild or accessibility.

Environment

VMware vSAN 8.x

 

Cause

The inaccessible objects were identified as stale vSAN objects, which are no longer part of any valid or active object configuration.

• Using the cmmds-tool, the CONFIG_STATUS values were examined:

   1010 state\": 44
      2 state\": 45
   1742 state\": 7

• Results indicated that a majority of objects had a status value of 44 and 45, confirming they were stale.

 

Resolution

Contact Broadcom Support