VM objects in "inaccessible" and "nonavailability-related-incompliance" state after multiple node failures.

Products

• VMware vSAN

Issue/Introduction

Symptoms:

• The issue is observed on a vSAN Stretched Cluster. After multiple node failures, VM objects may transition into the following states:
• • Inaccessible
  • Nonavailability-related incompliance
• The Skyline Health report will indicate a "vSAN object health" error.

Validation Step:

• To verify the health of the affected object, run the following command on the vSAN host command line:

esxcli vsan debug object health summary get

Health Status                                                                            Number Of Objects
---------------------------------------------------------                                 -----------------
remoteAccessible                                                                             0
inaccessible                                                                                 2
reduced-availability-with-no-rebuild                                                         0
reduced-availability-with-no-rebuild-delay-timer                                             0
reducedavailabilitywithpolicypending                                                         0
reducedavailabilitywithpolicypendingfailed                                                   0
reduced-availability-with-active-rebuild                                                     0
reducedavailabilitywithpausedrebuild                                                         0
data-move                                                                                    0
nonavailability-related-reconfig                                                             0
nonavailabilityrelatedincompliancewithpolicypending                                          0
nonavailabilityrelatedincompliancewithpolicypendingfailed                                    0
nonavailability-related-incompliance                                                        222
nonavailabilityrelatedincompliancewithpausedrebuild                                          0
healthy                                                                                     501

• To verify "nonavailability-related-incompliance" Objects:
  Run the following command on the vSAN host command line:"
  
  localcli vsan debug object list
Policy:
  stripeWidth: 1
      proportionalCapacity: 0
      hostFailuresToTolerate: 0
      affinity: ['xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx']
      affinityMandatory: 1
      spbmProfileId: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
      spbmProfileGenerationNumber: 1
      storageType: Allflash
      replicaPreference: Performance
      subFailuresToTolerate: 1
      CSN: xxxx
      SCSN: xxxx
      spbmProfileName: xxx-xxx-xxxxxx-xxx
      locality: NonPreferred

The locality of the affected components will be marked as "NonPreferred".

Environment

VMware vSAN 6.x
VMware vSAN 7.x
VMware vSAN 8.x

Cause

The issue occurs when there are multiple node failures in a vSAN Stretched Cluster.

Resolution

Inaccessible Objects:

• Please contact the Broadcom support team to resolve the issue with inaccessible objects. If the VMs have already been recovered from backup, they can assist with cleaning up the stale inaccessible objects.

"nonavailability-related-incompliance" Objects:

• To resolve the issue, restart the vsanmgmt service on all hosts in the vSAN cluster using the following command:
  
  "/etc/init.d/vsanmgmtd restart"