ESXi host in disconnected state after NetApp Storage Upgrade

Products

• VMware vSphere ESXi

Issue/Introduction

Symptoms:

• vCenter UI shows the ESXi host in disconnected state after the NetApp Storage Upgrade.
  
  
  
  
  
• Affected ESXi host will not be accessible via the Host Client as well.
• Host will be accessible via SSH and DCUI.

Environment

VMware vSphere ESXi 7.0.x

VMware vSphere ESXi 8.0.x

Cause

This is caused by communication issues between the storage and the host via the Fibre Channel protocol.

Following fnic_abort_cmd messages as seen in the log snippets below indicate command timeout to the storage device:

2025-03-04T06:04:28.802Z In(182) vmkernel: cpu83:2097812) nfnic: <1>: INFO: fnic taskMgmt: 2128: TaskMgmt abort sc->cdb: 0x28
2025-03-04T06:04:28.8022 In (182) vmkernel: cpu83:2097812) nfnic: <1>: INFO: fnic_abort_cmd: 3793: Abort cmd called for Tag: 0x204 issued time: 40002 ms CHD_STATE: FNIC_IOREQ_CHD_PENDING CDB Opcode: 0x28 sc:0x45b9b3657080 flags: 0x3 lun:16 target: 0x2b0101
2025-03-04T06:04:28.8212 In (182) vmkernel: cpu68:2098079) ScsiDeviceIO: 4617: Cmd(0x45b9b3657080) 0x28, cmdId.initiator=0x4309f8553180 CmdSN 0x2d7c1278 from world 0 to dev "naa.###################### failed H:0x5 D:0x0 P:0x0 Cancelled from driver layer. 

The read command 0x28 issued to the storage device is aborted with following scsi sense code (H:0x5 D: 0x0 P: 0x0) returned from the array:

This status is returned if the driver has to abort commands in-flight to the target. This can occur due to a command timeout or parity error in the frame.

Refer to this link for more information: VMware ESXi SCSI Sense Code Decoder

Resolution

Engage the storage vendor to investigate and address any communication issues between the storage and the host via the Fibre Channel protocol.

Additionally, verify the HBA compatibility with the ESXi version and ensure the HBA firmware and driver are up-to-date and supported.
To validate the driver and firmware version running on the Host Bus Adapters (HBA) refer to this article:  Determining Network/Storage firmware and driver version in ESXi