Unable to mount local datastore

Products

• VMware vSphere ESX 7.x
• VMware vSphere ESX 8.x

Issue/Introduction

• Local datastore is inaccessible on the host, leading to VMs residing on that datastore appearing as inaccessible
  
  
• The physical storage device appears to be attached, but not consumed in the vSphere Client (Choose Host -> Configure Tab -> Storage Devices)
  
  
• The datastore is not present within the /vmfs/volumes directory when viewed via the host's SSH session

Environment

VMware vSphere ESXi 7.x
VMware vSphere ESXi 8.x

Cause

• The datastore unmounted because the underlying storage device encountered an unrecoverable hardware error
  
  
• /var/run/log/vmkernel.log reports entries similar to:
  
  YYYY-MM-DDTHH:MM:SS Wa(180) vmkwarning: cpu13:2097663) WARNING: HBX: 1911: '<datastore_name>': HB at offset 3473408 - HB IO failed due to no connectivity:
  YYYY-MM-DDTHH:MM:SS In(182) vmkernel: cpu5:2097751) smartpqi01: pqisrc_show_sense_data_simple: 0162: [ERR INFO] BTL: 1:0:2 op=0x8a path=Raid K:C:Q: #:##:#
  YYYY-MM-DDTHH:MM:SS In(182) vmkernel: cpu5:2097751) smartpqi01: os_raid_response_error: 1258: err_info status=0x2 data_out=0x0. VMK host=0x1 device=0x2
  YYYY-MM-DDTHH:MM:SS In(182) vmkernel: cpu1:2097383)HPP: HppScsiAADetermineStatus:57: Device naa.########################## path vmhba0:C#:T#:L# hit an unrecoverable hardware error
  YYYY-MM-DDTHH:MM:SS In(182) vmkernel: cpu11:2097654)smartpqi01: pqisrc_display_device_info:0405: removed scsi B:T:L #:#:#  HPE      LOGICAL VOLUME   RAID 5      SSDSmartPathCap+ En- Exp+ qd=272 devtype=0
  YYYY-MM-DDTHH:MM:SS In(182) vmkernel: cpu11:2097654)smartpqi01: pqisrc_remove_device:1850: Removed vendor: HPE      model: LOGICAL VOLUME   B:T:L #:#:# is_physical_device:0x0
  YYYY-MM-DDTHH:MM:SS In(182) vmkernel: cpu1:2097375)smartpqi01: pqisrc_reject_if_device_unavailable:1499: The device is there, but will be marked as gone [ B:T:L #:#:# ]
  YYYY-MM-DDTHH:MM:SS Wa(180) vmkwarning: cpu13:2097804)WARNING: ScsiPath: 5988: Path vmhba0:C#:T#:L# is being removed
  YYYY-MM-DDTHH:MM:SS In(182) vmkernel: cpu11:2097654)smartpqi01: pqisrc_display_volume_status:1973: scsi BTL #:#:# Volume is in an unknown state.

(Entries will vary depending on the hardware vendor)

Resolution

Engage the hardware vendor to perform a complete diagnostic check on the RAID controller and associated physical components.