Recursive panic PSOD with #PF Exception 14 on VMware ESX

Products

• VMware vSphere ESXi

Issue/Introduction

• A VMware ESXi 8.0.3 host crashes with a Purple Screen of Death (PSOD). The crash screen or system logs indicate a Physical CPU (PCPU) failure with the following characteristics:
• ESXi host crashes with a PSOD, the crash screen contains a backtrace similar to this example:
  

  PCPU ##: no heartbeat (0/3 IPIs received)

  PanicvPanicInt
  Panic_WithBacktrace
  Heartbeat_DetectCPULockups
  TimerWheelHandler
  Timer_BHHandler
  BH_DrainAndDisableInterrupts
  CpuSchedDispatch
  CpuSchedWait
  CpuSchedVcpuHaltWork
  VMMVMKCall_Call
  VMKVMM_ArchEnterVMKernel

  Possible hardware problem: 1 PCPU(s) [##] did not respond to NMI
  Recursive panic (cpu ##, world #####, depth x): ip=###.###.###.### randomOff=#######:
  #PF Exception 14 in world #####:vnn#:####### IP ####### addr #######

   

• Additionally, the system fails to generate a vmkernel-zdump core file during the event.

Environment

VMware vSphere ESXi 8.0.3

Cause

The host encountered a recursive panic, typically triggered when a PCPU fails to update its heartbeat and stops responding to Non-Maskable Interrupts (NMIs). This behavior is generally caused by an underlying hardware-level fault in the processor or outdated microcode.

Resolution

1. Review host logs in /var/run/log/vmkernel.log or /var/run/log/LogEFI.log to determine if the crashes consistently occur on a specific PCPU core.
2. Update the server BIOS and all component firmware to the latest vendor-approved versions to ensure current microcode is applied.
3. Engage your hardware vendor (e.g., HPE, Dell, Lenovo) to perform a comprehensive hardware diagnostic, specifically targeting the identified PCPU.
4. If the issue persists despite firmware updates, follow the hardware vendor's recommendation for component replacement.

Additional Information

• To learn how to collect host logs for analysis, see Collecting diagnostic information for VMware ESXi.
• To speak with a customer representative or a Support Engineer see Contact Support. Scroll to the bottom of the page and click on your respective region.