VM not accessible after physical disk failure in vSAN

Products

• VMware vSAN

Issue/Introduction

• After a physical drive failure on the ESXi host Virtual machines will be seen as 'Inaccessible' in the vCenter UI.
  
  
• vSAN objects shows as inaccessible in "vCenter UI > vSAN cluster > Monitor > Virtual object".
  
  
• vSAN on-disk format upgrade pre-check will fail reporting vSAN objects are inaccessible.

Environment

VMware vSAN 8.x
VMware vSAN 9.x

Cause

• vSAN object became inaccessible due a the disk failure when the object was in reduced availability state and rebuilding.
  
  
• Output of running command : # esxcli vsan debug object list -u <Inaccessible_Object_UUID>, is as follows
  
  (Note : This object UUID is received from vSAN Skyline Health)

RAID_1
    Component: ########-####-####-####-############
    Component State: ACTIVE, Address Space(B): 273804165120 (255.00GB), Disk UUID: ########-####-####-####-############, Disk Name: naa..#################:2
    Votes: 2, Capacity Used(B): 276396244992 (257.41GB), Physical Capacity Used(B): 273657364480 (254.86GB), Total 4K Blocks Used(B): 273635536896 (254.84GB), Host Name: <esxi fqdn>
    Component: ########-####-####-####-############
    Component State: ACTIVE, Address Space(B): 273804165120 (255.00GB), Disk UUID: ########-####-####-####-############, Disk Name: naa..#################:2
    Votes: 2, Capacity Used(B): 276396244992 (257.41GB), Physical Capacity Used(B): 273657364480 (254.86GB), Total 4K Blocks Used(B): 273635536896 (254.84GB), Host Name: <esxi fqdn>
RAID_1
    Component: ########-####-####-####-############
    Component State: ABSENT, Address Space(B): 273804165120 (255.00GB), Disk UUID: ########-####-####-####-############, Disk Name: N/A, Transient: 1
    Votes: 1, Host UUID: None
RAID_1
    Component: ########-####-####-####-############
    Component State: RECONFIGURING, Address Space(B): 273804165120 (255.00GB), Disk ########-####-####-####-############, Disk Name: naa. .#################:2
    Votes: 1, Capacity Used(B): 279315480576 (260.13GB), Physical Capacity Used(B): 2206203904 (2.05GB), Total 4K Blocks Used(B): 2193866752 (2.04GB), Host Name: <esxi fqdn>
    Component: ########-####-####-####-############
    Component State: RECONFIGURING,
    Votes: 1, Capacity Used(B): 279315480576 (260.13GB),
    Address Space(B): 273804165120 (255.00GB), Disk UUID: ########-####-####-####-############, Disk Name: naa.#################:2
    Physical Capacity Used(B): 2206203904 (2.05GB)

Due to the 'Absent' component, 'Reconfigure' of the other two components would not complete. This renders the object and hence the VM inaccessible.

Resolution

• Please contact Broadcom Support to investigate the inaccessible object and possible recovery options.