VMs objects goes inaccessible soon after VM created/cloned and powered-on on vSAN datastore

Products

• VMware vSAN

Issue/Introduction

Symptoms:

VMs are cloned/deployed successfully. But, when powered on, it powers on successfully and stuck at boot screen.

VMs migrated to existing vSAN datastore, but crash moments later.

Checking the underlying object health, the objects would show as inaccessible.

When check for the components for the inaccessible objects, all the components show ACTIVE. However, the object would show in APD state.

Environment

VMware vSAN 7.x

VMware vSAN 8.x

Cause

The issue is seen when the hosts have DOMPauseAllCCPs option set to true. This would block the I/O initialization on the components and would mark the object health as inaccessible.

This can be validated using the command below. Where Value '1' is true and '0' is false. 

[root@esxi1:~] esxcfg-advcfg -g /VSAN/DOMPauseAllCCPs
Value of DOMPauseAllCCPs is 1

One or more hosts in a vSAN cluster with value set to true (1) can cause the objects to report inaccessible with APD.

Resolution

To resolve the issue, we need to disable DOMPauseAllCCPs should be disabled. 

The below command can be used to disable DOMPauseAllCCPs on vSAN hosts.

esxcfg-advcfg -s 0 /VSAN/DOMPauseAllCCPs

Additional Information

NOTE: The parameter DOMPauseAllCCPs would have modified and set to true if vSAN cluster shutdown and restart activity is executed and the restart did not make the changes on hosts as expected.

Refer the below KB for more info:

Inaccessible objects in vSAN when all components show as Active

vSAN cluster unable to restart after a planned cluster shutdown

Engineering planning on adding a Skyline Health check to check for this flag in 9.2.