ESXi host disconnecting frequently from VCenter and VM's may go in hung state
search cancel

ESXi host disconnecting frequently from VCenter and VM's may go in hung state

book

Article ID: 416237

calendar_today

Updated On:

Products

VMware vSphere ESXi

Issue/Introduction

Symptoms: 

  • The ESXi host appears as Not Responding in vCenter Server.
  • The host remains accessible through DCUI.
  • Restarting the ESXi management agents restores connectivity to vCenter Server.
  • VM's may go in hung state due to constant I/O Abort

Environment

VMWare ESXi 8.x

Cause

The issue is caused by repeated aborts occurring on a Fibre Channel (FC) fabric switch port associated with a specific HBA adapter. Continuous command aborts from a single port can disrupt I/O operations and lead to host or datastore access instability

Verify whether multiple aborts are originating from a single HBA adapter.

Log path :/var/run/logvmkernel.log 

2025-10-04T03:22:46.460Z Wa(180) vmkwarning: cpu97:2098270)WARNING: lpfc : vmhba4 lpfc_abort_fcp_cmpl:7403: 3096 Abort  completion for abort cmd iotag x398 xri:0xb37req_tag x398, status x0, hwstatus x0
2025-10-04T03:22:46.460Z In(182) vmkernel: cpu0:2549315)NMP: nmp_ThrottleLogForDevice:3893: Cmd 0xc0 (0x45#########, 2097272) to dev "naa.########" on path "vmhba4:C0:T0:L13" Failed:
2025-10-04T03:22:46.460Z In(182) vmkernel: cpu97:2098270)lpfc: lpfc_handle_status:4260: vmhba4 3271: FCP cmd xc0 failed <0/15> sid x024940, did x01ed40, oxid xb5b iotag xe81 Abort Requested Host Abort Req
2025-10-04T03:22:46.460Z Wa(180) vmkwarning: cpu97:2100294)WARNING: lpfc : vmhba4 lpfc_abort_fcp_cmpl:7403: 3096 Abort  completion for abort cmd iotag x397 xri:0xb5breq_tag x397, status x0, hwstatus x0
2025-10-04T03:22:46.464Z In(182) vmkernel: cpu0:2549315)NMP: nmp_ThrottleLogForDevice:3893: Cmd 0xc0 (0x4#########, 2097272) to dev "naa.#########" on path "vmhba4:C0:T0:L15" Failed:
2025-10-04T03:22:46.464Z Wa(180) vmkwarning: cpu53:2548315)WARNING: lpfc : vmhba4 lpfc_validate_fcp_abort:7544: 3111 Outstanding FCP I/O Abort Request still pending on io_buf 0x45db23f52430, xri xb83   >>>>>>>>>>>>>>>>>>>>>>>>>and usually indicates that an abort command sent to terminate a pending I/O request has not completed yet 
2025-10-04T03:22:46.465Z In(182) vmkernel: cpu97:2100294)lpfc: lpfc_handle_status:4260: vmhba4 3271: FCP cmd xc0 failed <0/17> sid x024940, did x01ed40, oxid xb83 iotag xea9 Abort Requested Host Abort Req
2025-10-04T03:22:46.465Z Wa(180) vmkwarning: cpu97:2098270)WARNING: lpfc : vmhba4 lpfc_abort_fcp_cmpl:7403: 3096 Abort  completion for abort cmd iotag x396 xri:0xb83req_tag x396, status x0, hwstatus x0
2025-10-04T03:22:46.466Z In(182) vmkernel: cpu97:2100294)lpfc: lpfc_handle_status:4260: vmhba4 3271: FCP cmd xc0 failed <0/18> sid x024940, did x01ed40, oxid xb42 iotag xe68 Abort Requested Host Abort Req

State in doubt message appear in logs

2026-01-07T06:58:23.526Z cpu32:2098204)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "naa.####################" state in doubt; requested fast path state update...
2026-01-07T06:58:23.526Z cpu32:2098204)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "naa.####################" state in doubt; requested fast path state update...

FC aborts observed in logs 

2026-01-07T06:58:23.526Z cpu43:2098093)lpfc: lpfc_handle_status:5613: 0:(0):3271: FCP cmd x2a failed <3/3> sid x011300, did x012100, oxid x2ad iotag x5d3  Returning Host Busy
2026-01-07T06:58:23.526Z cpu43:2098093)WARNING: lpfc: lpfc_sli_cancel_iocbs:1329: 0:0x0x########## iotag 1565 idx 0 flag 516
2026-01-07T06:58:23.526Z cpu43:2098093)lpfc: lpfc_handle_status:5613: 0:(0):3271: FCP cmd x2a failed <3/3> sid x011300, did x012100, oxid x2f7 iotag x61d  Returning Host Busy
2026-01-07T06:58:23.526Z cpu43:2098093)WARNING: lpfc: lpfc_sli_cancel_iocbs:1329: 0:0x########## iotag 3067 idx 0 flag 516
2026-01-07T07:12:45.108Z cpu36:2097603)WARNING: lpfc: lpfc_sli_issue_abort:10923: 0:(0):3169 Abort failed: Abort INP: Data: xf06 x122c x3 x98
2026-01-07T07:12:45.216Z cpu36:2097603)WARNING: lpfc: lpfc_sli_issue_abort:10923: 0:(0):3169 Abort failed: Abort INP: Data: xeb6 x11dc x6 x98

NMP Throttle devices events observed .

2026-01-07T06:58:23.526Z cpu32:2098204)NMP: nmp_ThrottleLogForDevice:3867: Cmd 0x2a (0x45d9e1e05ac8, 2101735) to dev "naa.#################" on path "vmhba2:C0:T2:L2" Failed:
2026-01-07T06:58:23.526Z cpu32:2098204)NMP: nmp_ThrottleLogForDevice:3875: H:0x2 D:0x0 P:0x0 . Act:EVAL. cmdId.initiator=0x########## CmdSN 0x360
2026-01-07T06:58:23.526Z cpu32:2098204)NMP: nmp_ThrottleLogForDevice:3798: last error status from device naa.################### repeated 10 times

This is followed up with more aborts seen.

2026-01-07T06:58:23.533Z cpu63:2098277)WARNING: lpfc: lpfc_sli_issue_abort:10923: 0:(0):3169 Abort failed: Abort INP: Data: xef0 x1216 x3 x98
2026-01-07T06:58:23.533Z cpu63:2098277)WARNING: lpfc: lpfc_sli_issue_abort:10923: 0:(0):3169 Abort failed: Abort INP: Data: xf04 x122a x3 x98

Devloss warning's observed in logs after the aborts .

2026-01-07T06:58:23.527Z cpu43:2098093)WARNING: lpfc: lpfc_start_devloss:4559: 0:(0):3248 Start 10 sec devloss tmo WWPN ##:##::##::##::##::##::##::##: NPort x013000
2026-01-07T06:58:23.528Z cpu43:2098093)WARNING: lpfc: lpfc_start_devloss:4559: 0:(0):3248 Start 10 sec devloss tmo WWPN ##::##::##::##::##::##::##::##: NPort x013100

Resolution

  • Check the SAN switch ports for any connectivity or performance issues affecting the HBA path.
  • If a problematic or unstable port is identified, disable the affected switch port after validating the redundancy. 

  • After making the change, verify the ESXi host status and confirm that storage paths and I/O activity return to normal.