Host was not responding to any services and certain VMs on it were also not reachable on network

Products

• VMware vSphere ESXi

Issue/Introduction

An ESXi host becomes unresponsive to vCenter management commands and direct console access (hostd/vpxa lockup) during scheduled storage maintenance, such as rolling firmware upgrades on a Pure Storage array. Affected VMs may lose network connectivity, and restarting management agents (services.sh) fails to restore the host state.

 

 

• Host status changes to 'Not Responding' in vCenter Server.
• Multiple iSCSI session recovery timeouts observed in vobd.log: [vob.iscsi.session.recovery.timeout] iScsi session recovery timeout for vmhba####:C1:T0.
• Peer-initiated disconnects recorded in vmkwarning.log: Failed to receive data: Connection closed by peer.
• Massive I/O queuing and task management aborts in vmkernel.log.
• Datastore inaccessibility (APD) alerts for storage volumes.

 

Environment

 

• VMware ESXi 7.x / 8.x
• iSCSI Storage Arrays (specifically Pure Storage during Storage Processor reboots)

 

Cause

This issue occurs when an environmental mismatch prevents the ESXi host from successfully failing over active I/O to standby paths during a target reboot. Common causes include:

• Delayed ACK: Enabled by default, which can delay path failover during congestion or TCP resets.
• Login Timeout: A default value (5 seconds) that is insufficient for certain arrays to re-establish paths during a controller failover.
• Stale Sessions: 'Ghost' iSCSI sessions that do not tear down correctly, causing the initiator to wait indefinitely.

Resolution

Perform the following sequential steps to prevent host lockups during future storage maintenance:

1. Verify that Delayed ACK is globally disabled on all iSCSI software adapters.
2. Set the Login Timeout to 30 seconds for iSCSI initiators connected to Pure Storage arrays.
3. Audit physical NIC firmware and drivers to ensure they match the cluster baseline and Broadcom Compatibility Guide.
4. Engage the storage vendor to verify array logs (SCSI sense codes) to ensure proper TCP reset behavior was initiated during the failover.
5. See Download Broadcom products and software to download this release if firmware/driver updates are required.

Additional Information

If you require further assistance, see Contact Broadcom Support or Creating and Managing Broadcom Support Cases.