PF Exception 14 in world xxxxxxx:nvmetcp_help IP XxXXXXXafXebXc addr XxXa

Products

• VMware vSphere ESXi

Issue/Introduction

• Sudden PSOD on VMware ESXi 8.x
• In the PSOD backtrace, you see stack entries similar to:

Version Details: VMware ESXi 8.x.x build-xxxxxxxx
Panic Details: Crash at YYYY-MM-DDTHH:MM:SS on CPU 11 running world XXXXXXX - nvmetcp_helper. VMK Uptime:HH:MM:SS
Panic Message: @BlueScreen: #PF Exception 14 in world xxxxxxx:nvmetcp_help IP XxXXXXXafXebXc addr XxXa
Backtrace:
  XxXXXXXXXXbefX:[XxXXXXXafXebXc]MCSLockWork@vmkernel#nover+0x8 stack: XxXXXXXbcXXfcX, XxXXXfXfXXXcXX, XxbXXaXXfXaXbac, XxXXXfXfXXXcXX, XxXXXfXfXXeXXX
  XxXXXXXXXXbfXX:[XxXXXXXaeXXcbX]vmk_SpinlockLock@vmkernel#nover+Xxf stack: XxXXXfXfXXXcXX, XxbXXaXXfXaXbac, XxXXXfXfXXXcXX, XxXXXfXfXXeXXX, XxXXXfXfXXXXdX
  XxXXXXXXXXbfXX:[XxXXXXXbcXXfcX][email protected]#X.X.X.X+XxXd stack: XxXXXfXfXXXcXX, XxXXXfXfXXeXXX, XxXXXfXfXXXXdX, XxXXXXXcXfaXbX, XxXXXXXXXfXXX
  XxXXXXXXXXbf4X:[XxXXXXXcXfaXbX]nt_DisconnectQueueAndReportHelperFunc@(nvmetcp)#<None>+0xee stack: XxXXXfXfXXXXdX, XxXXXXXafXbbXX, XxXXXfXfXXXXXX, XxX, XxXXXfXfXXXXXX
  XxXXXXXXXXbf6X:[XxXXXXXafXbbXf]HelperQueueFunc@vmkernel#nover+0x300 stack: XxXXXfXfXXXXXX, XxXXXXXXXXfXXX, XxXXXXXXXXXXXd, XxXXXXXcXfaXcX, XxXXXXXafXbbXX
  XxXXXXXXXXbfeX:[XxXXXXXbXdcXXe]CpuSched_StartWorld@vmkernel#nover+Xxbf stack: XxX, XxXXXXXafXXfbX, XxX, XxX, XxX
  XxXXXXXXXXcXXX:[XxXXXXXafXXfaf]Debug_IsInitialized@vmkernel#nover+Xxc stack: XxX, XxX, XxX, XxX, XxX

Environment

 VMware ESXi 8.x

Cause

Disconnect of the NVMe over TCPIP storage controller, asynchronous call of queue disconnect which seemed to have set due to socket error accessed the freed object.

The disconnection of the controller was triggered by the connection failures to the storage device.

Resolution

This issue is fixed in ESX 9.0.

There is currently no workaround available.

Customers on version 8.0 should subscribe to this article to be made aware when a fix for 8.0 has been released.