Troubleshooting unhealthy disk alerts on disk management after ESXi Host reinstallation

Products

• VMware vSAN
• VMware vSAN 8.x

Issue/Introduction

• Unhealthy disks may be identified by navigating to vSAN Cluster > Configure > Disk Management, and selecting the specific ESXi host.

 

• Despite removing and recreating disk groups, disks may still be identified as unhealthy in the UI.
• The host’s vSAN status may be flagged as disabled as a result of running the following command.

# esxcli vsan cluster get

Errors:
VSAN Clustering is not enabled on this host

• Rejoining an ESXi host to a vSAN cluster using CLI command  "esxcli vsan cluster join -u sub_cluster_UUID" may be unsuccessful if the host is stuck in an 'Unknown' Maintenance Mode state.

esxcli vsan cluster get 

Vsanutil: Failed to get decommission state entry: Not found
Vsanutil: Failed to get decommission state entry: Not found
Cluster Information:
Enabled: true
Current Local Time: YYYY-MM-DDThh:mm:ss
Local Node UUID: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
Local Node Type! NORMAL
Local Node State: MASTER
Local Node Health State: HEALTHY
Sub-Cluster Master UUID: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
Sub-Cluster Backup UUID:
Sub-Cluster UUID: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
Sub-Cluster Membership Entry Revision: 0
##.###.##.###
Sub-Cluster Member UUIDs: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
Sub-Cluster Member HostNames: <ESXi host_name>
Sub-Cluster Membership UUID: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
Unicast Mode Enabled: true
Maintenance Mode State: UNKNOWN
Config Generation: None 0 0.0
Mode: REGULAR
vSAN ESA Enabled: false

Detailed instructions on re-adding a host can be found in the article How to add a host back to a vSAN cluster after an ESXi host rebuild/reimage.

 

Environment

VMware vSAN 9.x

VMware vSAN 8.x

Cause

Post-reinstallation, the ESXi host might not properly joining the vSAN cluster. This has caused the disk group configuration to fail, leaving the host excluded from the vSAN datastore.

Resolution

• Place the ESXi host in maintenance mode and perform a reboot.

 

Warning:

Before removing or recreating a disk group, it is critical to verify that no active data components remain on the disks. Removing a disk that still contains active components may cause the associated vSAN objects to become inaccessible. While a host in Maintenance Mode should ideally have no active components, this status must be confirmed to ensure all objects remain healthy before proceeding with the disk recreation.

To verify if there is any alert for object health:

• Navigate to vSAN cluster > Monitor > vSAN Skyline health 
• For a detailed view, go to Monitor > vSAN > Virtual Objects.

To check for inaccessible objects from the ESXi host CLI, execute the following command:

esxcli vsan debug object health summary get

If any inaccessible objects are found on the affected disks, please open a support request with Broadcom before attempting to remove or recreate the disk group.

 

• Recreate the disk group using vSphere UI by navigating vSAN Cluster > Configure > Disk Management > Select the specific ESXi host > View disks > Create disk group.

Follow the KB article for Managing and Configuring a vSAN disk group using esxcli commands.