VMware ESXi 8.x host with NVMEoFC vVOLs fails with a purple diagnostic screen (PSOD) that includes "vmkdrivers/native/Proprietary/Storage/nfnic/fnic_nvme.c:757 -- VMK_ASSERT(io_req)"

Products

• VMware vSphere ESXi 8.0

Issue/Introduction

Symptoms: 

• An ESXi 8.x host stops responding and displays a purple diagnostic screen indicating "vmkdrivers/native/Proprietary/Storage/nfnic/fnic_nvme.c:757"
• The host is connected to SAN storage using NVMe over Fiber Channel based VVOLs
• The backtrace includes entries similar to the following:

2025-02-26T16:20:11.227Z cpu24:2097625)@BlueScreen: Failed at vmkdrivers/native/Proprietary/Storage/nfnic/fnic_nvme.c:757 -- VMK_ASSERT(io_req)
2025-02-26T16:20:11.227Z cpu24:2097625)Code start: 0x420007000000 VMK uptime: 8:00:52:42.575
2025-02-26T16:20:11.227Z cpu24:2097625)0x45390ec9b6a0:[0x42000717bbc0]PanicvPanicInt@vmkernel#nover+0x20c stack: 0x0
2025-02-26T16:20:11.228Z cpu24:2097625)0x45390ec9b750:[0x42000717c3bc]Panic_vPanic@vmkernel#nover+0x25 stack: 0x2
2025-02-26T16:20:11.228Z cpu24:2097625)0x45390ec9b770:[0x420007197cc0]vmk_PanicWithModuleID@vmkernel#nover+0x41 stack: 0x45390ec9b7d0
2025-02-26T16:20:11.228Z cpu24:2097625)0x45390ec9b7d0:[0x42000816f529]fnic_wq_copy_cmpl_handler@(nfnic)#<None>+0x69a stack: 0xffffffff46f547e0
2025-02-26T16:20:11.229Z cpu24:2097625)0x45390ec9b830:[0x4200081563c8]fnic_isr_msix_wq_copy@(nfnic)#<None>+0x2d stack: 0x18
2025-02-26T16:20:11.229Z cpu24:2097625)0x45390ec9b840:[0x42000715f77b]IntrCookieBH@vmkernel#nover+0x170 stack: 0x45390ec9b860
2025-02-26T16:20:11.229Z cpu24:2097625)0x45390ec9b8d0:[0x42000713f6cc]BH_DrainAndDisableInterrupts@vmkernel#nover+0x159 stack: 0x420046001570
2025-02-26T16:20:11.230Z cpu24:2097625)0x45390ec9b940:[0x42000716143f]IntrCookie_VmkernelInterrupt@vmkernel#nover+0xa4 stack: 0x76
2025-02-26T16:20:11.230Z cpu24:2097625)0x45390ec9b960:[0x4200076a290a]IDT_IntrHandler@vmkernel#nover+0x97 stack: 0x0
2025-02-26T16:20:11.230Z cpu24:2097625)0x45390ec9b980:[0x42000769b0c6]gate_entry@vmkernel#nover+0xa7 stack: 0x0
2025-02-26T16:20:11.230Z cpu24:2097625)0x45390ec9ba48:[0x420007090e24]Power_ArchPerformWait@vmkernel#nover+0xd4 stack: 0x420046001880
2025-02-26T16:20:11.231Z cpu24:2097625)0x45390ec9ba50:[0x420007090f75]Power_ArchSetCState@vmkernel#nover+0xba stack: 0x0
2025-02-26T16:20:11.231Z cpu24:2097625)0x45390ec9baa0:[0x4200076d4111]CpuSchedIdleLoopInt@vmkernel#nover+0x292 stack: 0x0
2025-02-26T16:20:11.231Z cpu24:2097625)0x45390ec9bb10:[0x4200076d863c]CpuSchedDispatch@vmkernel#nover+0x1e31 stack: 0x1
2025-02-26T16:20:11.232Z cpu24:2097625)0x45390ec9bd50:[0x4200076d904e]CpuSchedWait@vmkernel#nover+0x35b stack: 0x8000000000000006
2025-02-26T16:20:11.232Z cpu24:2097625)0x45390ec9bec0:[0x4200076d91b0]CpuSchedSleepUntilTC@vmkernel#nover+0xb5 stack: 0x748
2025-02-26T16:20:11.232Z cpu24:2097625)0x45390ec9bf60:[0x420007498068]SchedPowerEnergyPeriodicLoop@vmkernel#nover+0x3d stack: 0x9000000000
2025-02-26T16:20:11.232Z cpu24:2097625)0x45390ec9bfe0:[0x4200076d67b2]CpuSched_StartWorld@vmkernel#nover+0xbf stack: 0x0
2025-02-26T16:20:11.233Z cpu24:2097625)0x45390ec9c000:[0x420007144cef]Debug_IsInitialized@vmkernel#nover+0xc stack: 0x0

Environment

• VMware vSphere ESXi 8.x

Cause

• The cause of this issue under investigating by Cisco.

Resolution

• The affected host will need to be rebooted to recover.
• Please contact Cisco for further information.