ESXi 8.0 Host Fails with PSOD PF Exception 14 in world nvmetcp_help

Products

• VMware vSphere ESXi

Issue/Introduction

• An ESXi 8.0 host fails with a purple diagnostic screen (PSOD) due to a page fault exception within the nvmetcp_help world. Link flapping or socket errors on the storage network interface card (NIC) typically precede the failure.

  The PSOD backtrace contains the following panic message and stack entries:

  Panic Message: @BlueScreen: #PF Exception 14 in world ###:nvmetcp_help IP ########## addr ###########
  Backtrace:
    MCSLockWork@vmkernel#nover+0x8
    vmk_SpinlockLock@vmkernel#nover+#xf
    [email protected]##.#.#.#+#x#d
    nt_DisconnectQueueAndReportHelperFunc@(nvmetcp)#<None>+0#ee
    HelperQueueFunc@vmkernel#nover+0x300
    CpuSched_StartWorld@vmkernel#nover+##bf
    Debug_IsInitialized@vmkernel#nover+##c

Environment

 VMware ESXi 8.0

Cause

The provided purple diagnostic screen (PSOD) displays a #PF Exception 14 occurring in world nvmetcp_help on VMware ESXi 8.0 Update 3. The stack trace confirms the kernel panic occurs within the nvmetcp module, specifically involving nvmetcp_helper, vmk_MutexLock, and Int_DisconnectQueueHelperFunc@(nvmetcp).

This kernel panic is caused by the disconnection of an NVMe over TCP/IP storage controller. An asynchronous queue disconnect call, triggered by connection failures to the storage device and subsequent socket errors, attempts to access a freed memory object. This invalid memory access results in the page fault exception.

Resolution

 

• Patch the affected host to ESXi 8.0 Update 3i Build: 25205845. Release notes or later.

• Alternatively, upgrade the environment to ESXi 9.0 or later, where the code defect is natively resolved.

• Investigate and stabilize the underlying physical network switch infrastructure and storage fabric to eliminate the link flapping and network disconnects that trigger the race condition.