ESXi Purple Diagnostic Screen (PSOD) with Heap Exhaustion when RSPAN Destination is configured

Products

• VMware vSphere ESXi

Issue/Introduction

• ESXi Purple Diagnostic Screen (PSOD) with Heap Exhaustion when RSPAN Destination is configured.
• PSOD Backtrace contains references to:
  PanicvPanicInt
Panic_WithBacktrace
NMI_Interrupt
IDTNMIWork
Int2_NMI
gate_entry
HeapCheckMemoryPressure
Heap_AlignWithTimeoutAndRA
Mirror_DispatchFromPort
VSwitchPortDispatch
Port_InputResume
Net_AcceptRxList
NetPollWorldCallback
• vmkernel.log entries:
  Mirror.cswitch: VSwitchRspanDstSessionDestinationFind: Failed to alloc memory for LightMirrorSession
• vmkwarning.log entries:
  WARNING: Heap: Heap cswitch already at its maximum size. Cannot expand.
WARNING: Heartbeat: 961: PCPU <ID> didn't have a heartbeat for xx seconds... *may* be locked up.

Environment

VMware vSphere ESXi

Cause

During RSPAN operations, memory is allocated to a data structure. This memory is not freed fast enough and eventually causes memory exhaustion. Further memory allocation failures run in a loop, resulting in PSOD on the host.

Resolution

To avoid hitting the PSOD again, the workaround is to remove RSPAN destination mirror sessions from VC UI.
Select dvs > configure > Port mirroring, remove "Remote Mirroring Destination" sessions.