"Host Storage Status" alarm in vCenter due to predictive drive failure on Lenovo ThinkSystem used for vSAN ESA

Products

• VMware vSphere ESXi

Issue/Introduction

Issue/Introduction A "Host Storage Status" alarm is triggered on an ESXi host in the vCenter Server UI. While the host remains operational, the hardware management console (Lenovo XClarity Controller) reports a predictive failure on a specific physical drive within a RAID array.

• Primary Product: VMware vSphere ESXi
• Component: Storage / Hardware Health
• Symptoms:
  • vCenter triggers a Host Storage Status alarm. 
    
    
  • Lenovo XClarity Controller (XCC) logs hardware events such as: Failure Predicted on Drive <ID> for array ThinkSystem <Model>. 
    
    
  • Verify the disk status from the ESXi command line to confirm it has been logically isolated:  The impacted drive is missing here. 
    
    # vdq -i 

"singleTier" : [
                     "t10.NVMe____KIOXIA_KCD######68_____________________A0##########8C",
                     "t10.NVMe____KIOXIA_KCD######68_____________________A0##########8C",
                     "t10.NVMe____KIOXIA_KCD######68_____________________A0##########8C",
                     "t10.NVMe____KIOXIA_KCD######68_____________________A0##########8C",
                     "t10.NVMe____KIOXIA_KCD######68_____________________A0##########8C",
                     "t10.NVMe____KIOXIA_KCD######68_____________________A0##########8C",
                     "t10.NVMe____KIOXIA_KCD######68_____________________A0##########8C"
                     ]    
  • Run esxcli storage core device physical get -d t10.NVMe____KIOXIA_KCD######68_____________________A0##########8C To validate the slots 
    
    Example:

# esxcli storage core device physical get -d t10.NVMe____KIOXIA_KCD######68_____________________A0##########8C
  Physical Location: Bay 0                                             

Environment

VMware vSphere ESXi 8.x
VMware vSphere ESX  9.x
VMware vSAN ESA All 

Cause

The issue is caused by a physical hard drive reporting a "Predictive Failure" state. Controllers monitor internal disk metrics; when these metrics exceed certain thresholds, the controller flags the drive as likely to fail soon. ESXi detects this hardware state change and triggers the storage health alarm.

 

Resolution

To resolve the alarm and maintain data redundancy, the predictive failing drive must be replaced.

• Validation and Prerequisites:
  
  
  • Log into the Lenovo XClarity Controller (XCC) to identify the exact slot number of the failing drive (e.g., Drive 5).
  • Ensure a valid backup of all Virtual Machines residing on the affected local datastore.
  • Take a snapshot or backup of the vCenter Server if it resides on the affected host.
• Maintenance Mode:
  
  
  • Right-click the ESXi host in vCenter and select Maintenance Mode > Enter Maintenance Mode.
  • Select the appropriate data evacuation policy (e.g., Ensure Accessibility for vSAN or migrate VMs manually for local storage).
• Reboot the host to validate if the Disk issue subsides
  
  
  • If post reboot the disk error still shows up then engage hardware vendor to get the disk replaced   
• Before you replace the disk follow the action plan: 

• • Remove impacted disk from storage pool
  • Navigate to the vSAN cluster.
  • On the Configure tab, click Disk Management under vSAN.
  • Select the storage pool device, and click Remove Disk.
    
  • In the Remove Disk dialog box, select Full data migration to transfer all the data available on the host to other hosts in the cluster.
    
    
  • Click Go To Pre-Check to find the impact on the cluster if the object is removed or placed in maintenance mode.
    
    
  • Click Remove to remove the storage pool device.
• Physical Replacement:

• • Coordinate with Lenovo Hardware Support to replace the identified drive.
  • Note: On most ThinkSystem models, drives are hot-swappable. However, entering Maintenance Mode is recommended to prevent I/O latency impacts during the RAID rebuild. 
• Add a new device to the host. The host automatically detects the device.
  
  
  • If the host is unable to detect the device, perform a device rescan.
    
    
  • Claim a disk using the vSAN cluster > Configure > vSAN > Disk Management. 
• Clear Alarms:
  
  
  • In the vSphere Client, navigate to the ESXi host Monitor > Hardware Health.
  • Click Reset Sensors to refresh the hardware state.
  • If the "Host Storage Status" alarm persists, select the host, go to the Alarms tab, and manually Reset to Green.
• Exit Maintenance Mode:
  
  
  • Right-click the host and select Maintenance Mode > Exit Maintenance Mode. 

Please reference the Official documentation: Replace a Storage Pool Device in vSAN ESA Cluster

Additional Information

If in case error occurred while removing the disk from UI then please use : Unable to Remove vSAN ESA Disk from ESXi Command Line with "esxcli vsan storage remove"