ESXi host fails with purple diagnostic screen with "vmk_PktListPopFirstPkt" in the backtrace

Products

• VMware vSphere ESXi

Issue/Introduction

Symptoms:

• The PSOD backtrace may contain entries similar to:

###############################Backtrace for current CPU ##, worldID=#######, fp=##############
###############################vmk_PktListPopFirstPkt@vmkernel###############################
###############################[email protected]###############################
###############################Port_InputResume@vmkernel###############################
###############################Net_AcceptRxList@vmkernel###############################
###############################NetPollWorldCallback@vmkernel###############################
###############################CpuSched_StartWorld@vmkernel###############################
###############################Debug_IsInitialized@vmkernel###############################
#PF Exception 14 in world #######:vmnic#-pollW IP #x########### addr #x# PTEs:#x##########;#x##########;#x##########;#x#;

• In the /var/run/log/vmkernel.log, we see entries similar to:

YYYY-MM-DDTHH:MM:SSZ cpu66:2098643)ql_fcoe:vmhba67:SendFCoEVlanSolicitation:1404:Info: Sending FCoEVlanSolicitation request (0x1)
YYYY-MM-DDTHH:MM:SSZ cpu68:2098638)ql_fcoe:vmhba65:SendFCoEDiscoverySolicitation:1146:Info: Sending FIP discovery for vlan_id = 0x3ea (0x431264972000
YYYY-MM-DDTHH:MM:SSZ cpu47:2098642)ql_fcoe:vmhba67:SendFCoEVlanSolicitation:1404:Info: Sending FCoEVlanSolicitation request (0x2)
YYYY-MM-DDTHH:MM:SSZ cpu68:2098638)ql_fcoe:vmhba65:SendFCoEDiscoverySolicitation:1146:Info: Sending FIP discovery for vlan_id = 0x3ea (0x431264972000[7m
YYYY-MM-DDTHH:MM:SSZ cpu44:2097460)WARNING: qfle3: ecore_state_wait:317: timeout waiting for state 7m
YYYY-MM-DDTHH:MM:SSZ cpu44:2097460)WARNING: qfle3: ecore_queue_state_change:4992: [vmnic1] Ecore Wait Complete Timed out, manually polling fp[2]. from f01 to 10da
YYYY-MM-DDTHH:MM:SSZ cpu49:2098642)ql_fcoe:vmhba67:SendFCoEVlanSolicitation:1404:Info: Sending FCoEVlanSolicitation request (0x3)
YYYY-MM-DDTHH:MM:SSZ cpu7:2098219)World: 3069: PRDA 0x420041c00000 ss 0x0 ds 0xf50 es 0xf50 fs 0xf50 gs 0xf50
YYYY-MM-DDTHH:MM:SSZ cpu7:2098219)World: 3071: TR 0xf58 GDT 0x45398000e000 (0xf77) IDT 0x42001b94f000 (0xfff)
YYYY-MM-DDTHH:MM:SSZ cpu7:2098219)World: 3072: CR0 0x8001003d CR3 0x6ffff000 CR4 0x10216c7m
YYYY-MM-DDTHH:MM:SSZ cpu44:2097460)WARNING: qfle3: ecore_queue_state_change:4999: [vmnic1] fp[2], pending bits cleared

 

Note: The preceding log excerpts are only examples. Date, time and environmental variables may vary depending on the environment.

 

Environment

VMware vSphere ESXi 7.x
VMware vSphere ESXi 6.7

Cause

This issue is caused due to ecore timeout.

Resolution

This issue is resolved in following versions of Qlogic driver:

• Qfle3 driver version 1.1.25.0 in ESXi 6.7.
• Qfle3 driver version 1.4.25.0 in ESXi 7.0.

Steps to download the driver : Finding IO Drivers in the Broadcom Support Portal

Workaround: None

Additional Information

Impact/Risks:
Requires ESXi host reboot