Cannot remove and replaced a failed vSAN disk from the vSAN host.

Products

• VMware vSAN
• VMware vSAN 7.x
• VMware vSAN 8.x

Issue/Introduction

vSAN capacity disk failed in one of Dell 730xd Host . When tried to remove the disk from the disk group even after putting the host into the Maintenance Mode the operation failed with the error that the disk can not be removed. 

Environment

VMware vSAN 7.x

VMware vSAN 8.x 

Cause

Removal of the failed vSAN disk failed since it was physically removed and replaced before removing it from the vSphere UI.

Resolution

1. Put the host with the failed vSAN disk into Maintenance Mode with ensure accessibility by using vSphere UI or CLI by using following command after login into the host using SSH as root user.
   
   esxcli system maintenanceMode set --enable true -m noAction

2. Record the absent capacity disk UUID by running this command

   esxcli vsan storage list

   Example output of an absent capacity device:

   Unknown
   Device: Unknown
   Display Name: Unknown
   Is SSD: true
   VSAN UUID: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx8fa3
   VSAN Disk Group UUID: 
   VSAN Disk Group Name:
   Used by this host: false
   In CMMDS: false
   On-disk format version: 5
   Deduplication: false
   Compression: false
   Checksum: 
   Checksum OK: false
   Is Capacity Tier: false
   Encryption: false
   Is Mounted: false
   DiskKeyLoaded: false
   
   If above does not show the failed disk, you can also verify the absent disk uuid by using the command 'vdq -iH'

   Sample Output:

   vdq -iH
   Mappings:
      DiskMapping[0]:
              SSD:  naa.xxxxxxxxxxxxxx
               MD:  naa.xxxxxxxxxxxxxx
               MD:  naa.xxxxxxxxxxxxxx
               MD:  naa.xxxxxxxxxxxxxx
               MD:  xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx8fa3

   In the above output, absent capacity disk will only have the uuid while all other healthy devices will have the device naa.id

3. Remove the absent disk using the below command

   esxcli vsan storage remove -u <UUID of the absent capacity disk to remove>

 

Additional Information

How to manually manage and configure a vSAN disk group using esxcli commands