ESX with NIC driver qfle3/qedentv fails with a PSOD

Products

• VMware vSphere ESXi 8.0
• VMware vSphere ESXi 7.0
• VMware vSphere ESXi 6.0

Issue/Introduction

•  VMware ESXi host fails with a PSOD
•  ESXi NIC uses driver qfle3/qedentv
•  MTU of 9000 is configured on the NIC
•  The crash screen contains a backtrace similar to this example:

   Heap_AlignWithTimeoutAndRA
   Pkt_AllocHandleWithSize
   Pkt_AllocWithFlags
   vmk_PktAllocForDMAEngine

 

Environment

VMware ESXi

Cause

When a pNIC driver attempts to allocate 9000 MTU size packets to replenish its receive queue, as system memory is fragmented, networking subsystem is not able to allocate large page memory to grow the memory pool that backs such packets. As a result, memory pool API spins for large pages for a long time, triggers CPU heartbeat check failure and crashes the host.

Resolution

Resolution: This is a known issue impacting VMware ESXi.

Workaround: Reserve memory pool for such pNIC drivers through command:

"esxcli system settings kernel set -s netPktJumboSlabMinSize -v <SIZE_OF_MEMORY_TO_RESERVE_IN_MB>"