Virtual machines might become unresponsive when one or more paths between ESXi and an NVMe over FC storage array are lost

Products

• VMware vSphere ESXi 8.0

Issue/Introduction

Applications or virtual machines might become unresponsive when one or more paths between ESXi and an NVMe over FC storage array are lost.

ESXi typically uses multipathing to arrays, but in some cases, the loss of one or more paths can cause unresponsiveness of virtual machines. 

The issue occurs because some in-flight NVMe commands might reach failed paths and cannot complete or stop.

As a result, such commands remain blocked until the paths are restored, causing unresponsiveness of applications or VMs.

Environment

VMware ESXi 8.0

Cause

After the NVME storage target lost, the HBA driver or firmware doesn't abort/reject the outstanding commands, and vmknvme doesn't perform any controller reset.

The controller recovery world skips the controller reset as the target is offline.
Then the outstanding NVMe commands can't be completed.

It led the unresponsiveness of applications or VMs.

Resolution

This issue is resolved in the VMware ESXi 8.0 Update 3h (Build 25067014)

Additional Information

VMware ESXi 8.0 Update 3h Release Notes
https://techdocs.broadcom.com/us/en/vmware-cis/vsphere/vsphere/8-0/release-notes/esxi-update-and-patch-release-notes/vsphere-esxi-80u3h-release-notes.html