PSOD with PANIC bora/vmkernel/main/dlmalloc.c:4944 - Usage crror in dlmalloc

Products

• VMware vSphere ESXi
• VMware vSphere ESXi 8.0

Issue/Introduction

ESXi Server might fail with a PSOD having backtrace similar to the following :

VMware ESXi 8.0.3 [Releascbuild-24784735 x86_64]
PANIC bora/vmkernel/main/dlmalloc.c:4944 - Usage crror in dlmalloc

Module(s) involved in panic: [vnfs3 Version VMFS6:24.82, VMFS6-L:24.82, VMFS5:14.81, VMFS5-L:14.60, Built on: Jun 8 2025]
cr0=0x8001003d cr2=0xffff8007ef8f3000 cr3=0x6b7ff000 cr4=0x14216c
FMS=06/6a/6 uCode=0xd0003f5
*PCPU2:2098339/FS3ResMgr
PCPU 0: SVSVVVVSVVVVSVSVVVSVVVVVVVVVVVVVVVVSVVVSSSVVVSVVVVVVVVVVSUSVVVSV
PCPU 64: VVVVVVVUVVVVVVVSVVSVVVSVVUUVSVVVVVVSVSSVVSVSVVVV
Code start: 0x420010600000 VMK uptime: 62:19:42:36.891
0x4539c019bd70 : [0x42001077bc80]PanicvPanic Int@vnkernel#nover+0x20c stack: 0x42001075b3f6
0x4539c019bc20:[0x42001077c5fc]Panic_NoSave@vmkernel#nover+0x4d stack: 0x4539c019bc80
0x4539c019be80:[0x4200107329b3]DLM_free@vmkernel#nover+0x15c stack: 0x4314ce5ce0a0
0x4539e019bea0: [0x42001073b4ad]Heap_Free@vnkernel#nover+0xb2 stack: 0x0
0x4539e019bef0: [0x420011aaf2bb]Res3ManagerVMFS6@esx#nover+0x1d8 stack: 0x1
0x4539e019bfe0: [0x420010cdc88e]CpuSched_StartHor1d@vmkernel#nover+0xbf stack: 0x0
0x4539e019c000 : [0x420010744faf ]Debug_Is Initialized@vmkernel#nover+0xc stack: 0x0
base fs=0x0 gs=0x420040800000 Kgs=0x0

 

/var/run/log/vmkernel.log

YYYY-MM-DDTHH.417Z cpu2:2098284)nfnic: <2>: INFO: fnic_fdls_validate_and_get_frame_type: 4121: Receiving Abort Request from s_id: 0x1438eb
YYYY-MM-DDTHH.417Z cpu2:2098284)nfnic: <2>: INFO: fnic_fdls_recv_frame: 4506: Received unknown FCoE frame of len: 140. Dropping frame.

Environment

VMware vSphere ESXi 8.0

 

Cause

 

PSOD cause by VMFS heap corruption & that can trigger by Firmware/Driver level issue with abort request initiated by nfnic driver.

 

 

Resolution

Validate Driver/Firmware Versions & Compatibility :

1.Log in to the ESXi host via SSH and refer the following KB to identify the currently installed nfnic driver version & Firmware version:

Determining Network/Storage firmware and driver version in ESXi

2.Please refer Compatibility Guide to validate if Firmware Info is available :

https://compatibilityguide.broadcom.com/

3. Please consult HW vendors to upgrade/downgrade if it's async driver.

4.Please consult HW vendors to run intense diagnosis and upgrade the drivers/firmware's to latest.