PSOD Can Occur When Using QFLE3 Driver
Article ID: 317956
Updated On:
Products
VMware vSphere ESXi
Issue/Introduction
Symptoms:
cpu5:2097316)0x451a4521b9f0:[0x41802fe42262]qfle3_alloc_fp_buffers@(qfle3)#<None>+0x7f
cpu5:2097316)0x451a4521ba60:[0x41802fe2db9c]qfle3_rq_create@(qfle3)#<None>+0x3a9
cpu5:2097316)0x451a4521bae0:[0x41802fde4d37]qfle3_cmd_create_q@(qfle3)#<None>+0x15c
cpu5:2097316)0x451a4521bb30:[0x41802fe1c6c2]qfle3_sm_q_cmd@(qfle3)#<None>+0x147
cpu5:2097316)0x451a4521bbb0:[0x41802fe2d55e]qfle3_rq_alloc@(qfle3)#<None>+0x2d7
cpu5:2097316)0x451a4521bc40:[0x41802f6de61d]UplinkNetq_AllocHwQueueWithAttr@vmkernel#nover+0x92
cpu5:2097316)0x451a4521bc90:[0x41802f617ee5]NetqueueBalActivatePendingRxQueues@vmkernel#nover+0x156
cpu5:2097316)0x451a4521bd50:[0x41802f618b25]NetqueueBalRxQueueCommitChanges@vmkernel#nover+0x36
cpu5:2097316)0x451a4521bd90:[0x41802f61c127]UplinkNetqueueBal_BalanceCB@vmkernel#nover+0x19fc
cpu5:2097316)0x451a4521bf00:[0x41802f6d9069]UplinkAsyncProcessCallsHelperCB@vmkernel#nover+0x116
cpu5:2097316)0x451a4521bf30:[0x41802f4eb06a]HelperQueueFunc@vmkernel#nover+0x157
cpu5:2097316)0x451a4521bfe0:[0x41802f7107da]CpuSched_StartWorld@vmkernel#nover+0x77
2020-10-21T12:06:34.190Z cpu2:2097316)qfle3: qfle3_queue_alloc_with_attr:642: [vmnic1] Feature RSS requested.
2020-10-21T12:06:34.190Z cpu2:2097316)qfle3: qfle3_rq_alloc:327: [vmnic1] Rxq 2 is leading RSS with 4 RSS queues.
2020-10-21T12:06:34.190Z cpu2:2097316)WARNING: qfle3: qfle3_rq_create:376: [vmnic1] RQ seems to have already been created [0m
2020-10-21T12:06:39.190Z cpu10:2097316)qfle3: qfle3_queue_alloc_with_attr:642: [vmnic1] Feature RSS requested.
2020-10-21T12:06:39.190Z cpu10:2097316)qfle3: qfle3_rq_alloc:327: [vmnic1] Rxq 2 is leading RSS with 4 RSS queues.
2020-10-21T12:06:39.190Z cpu10:2097316)WARNING: qfle3: qfle3_rq_create:376: [vmnic1] RQ seems to have already been created [0m
2020-10-21T12:06:44.190Z cpu18:2097316)qfle3: qfle3_queue_alloc_with_attr:642: [vmnic1] Feature RSS requested.
2020-10-21T12:06:44.190Z cpu18:2097316)qfle3: qfle3_rq_alloc:327: [vmnic1] Rxq 2 is leading RSS with 4 RSS queues.
2020-10-21T12:06:44.190Z cpu18:2097316)WARNING: qfle3: qfle3_rq_create:376: [vmnic1] RQ seems to have already been created [0m
2020-10-21T12:06:49.191Z cpu8:2097316)qfle3: qfle3_queue_alloc_with_attr:642: [vmnic1] Feature RSS requested.
2020-10-21T12:06:49.191Z cpu8:2097316)qfle3: qfle3_rq_alloc:327: [vmnic1] Rxq 2 is leading RSS with 4 RSS queues.
2020-10-21T12:06:49.191Z cpu8:2097316)WARNING: qfle3: qfle3_rq_create:376: [vmnic1] RQ seems to have already been created [0m
2020-10-21T12:06:54.191Z cpu12:2097316)qfle3: qfle3_queue_alloc_with_attr:642: [vmnic1] Feature RSS requested.
- ESXi host running on 6.5 or 6.7 crashes with a PSOD with vmk_Memset@vmkernel#nover+0x9
- The PSOD backtrace entries will be similar to
cpu5:2097316)0x451a4521b9f0:[0x41802fe42262]qfle3_alloc_fp_buffers@(qfle3)#<None>+0x7f
cpu5:2097316)0x451a4521ba60:[0x41802fe2db9c]qfle3_rq_create@(qfle3)#<None>+0x3a9
cpu5:2097316)0x451a4521bae0:[0x41802fde4d37]qfle3_cmd_create_q@(qfle3)#<None>+0x15c
cpu5:2097316)0x451a4521bb30:[0x41802fe1c6c2]qfle3_sm_q_cmd@(qfle3)#<None>+0x147
cpu5:2097316)0x451a4521bbb0:[0x41802fe2d55e]qfle3_rq_alloc@(qfle3)#<None>+0x2d7
cpu5:2097316)0x451a4521bc40:[0x41802f6de61d]UplinkNetq_AllocHwQueueWithAttr@vmkernel#nover+0x92
cpu5:2097316)0x451a4521bc90:[0x41802f617ee5]NetqueueBalActivatePendingRxQueues@vmkernel#nover+0x156
cpu5:2097316)0x451a4521bd50:[0x41802f618b25]NetqueueBalRxQueueCommitChanges@vmkernel#nover+0x36
cpu5:2097316)0x451a4521bd90:[0x41802f61c127]UplinkNetqueueBal_BalanceCB@vmkernel#nover+0x19fc
cpu5:2097316)0x451a4521bf00:[0x41802f6d9069]UplinkAsyncProcessCallsHelperCB@vmkernel#nover+0x116
cpu5:2097316)0x451a4521bf30:[0x41802f4eb06a]HelperQueueFunc@vmkernel#nover+0x157
cpu5:2097316)0x451a4521bfe0:[0x41802f7107da]CpuSched_StartWorld@vmkernel#nover+0x77
- In the ESXi var/log/vmkernel.log you will see entries as below
2020-10-21T12:06:34.190Z cpu2:2097316)qfle3: qfle3_queue_alloc_with_attr:642: [vmnic1] Feature RSS requested.
2020-10-21T12:06:34.190Z cpu2:2097316)qfle3: qfle3_rq_alloc:327: [vmnic1] Rxq 2 is leading RSS with 4 RSS queues.
2020-10-21T12:06:34.190Z cpu2:2097316)WARNING: qfle3: qfle3_rq_create:376: [vmnic1] RQ seems to have already been created [0m
2020-10-21T12:06:39.190Z cpu10:2097316)qfle3: qfle3_queue_alloc_with_attr:642: [vmnic1] Feature RSS requested.
2020-10-21T12:06:39.190Z cpu10:2097316)qfle3: qfle3_rq_alloc:327: [vmnic1] Rxq 2 is leading RSS with 4 RSS queues.
2020-10-21T12:06:39.190Z cpu10:2097316)WARNING: qfle3: qfle3_rq_create:376: [vmnic1] RQ seems to have already been created [0m
2020-10-21T12:06:44.190Z cpu18:2097316)qfle3: qfle3_queue_alloc_with_attr:642: [vmnic1] Feature RSS requested.
2020-10-21T12:06:44.190Z cpu18:2097316)qfle3: qfle3_rq_alloc:327: [vmnic1] Rxq 2 is leading RSS with 4 RSS queues.
2020-10-21T12:06:44.190Z cpu18:2097316)WARNING: qfle3: qfle3_rq_create:376: [vmnic1] RQ seems to have already been created [0m
2020-10-21T12:06:49.191Z cpu8:2097316)qfle3: qfle3_queue_alloc_with_attr:642: [vmnic1] Feature RSS requested.
2020-10-21T12:06:49.191Z cpu8:2097316)qfle3: qfle3_rq_alloc:327: [vmnic1] Rxq 2 is leading RSS with 4 RSS queues.
2020-10-21T12:06:49.191Z cpu8:2097316)WARNING: qfle3: qfle3_rq_create:376: [vmnic1] RQ seems to have already been created [0m
2020-10-21T12:06:54.191Z cpu12:2097316)qfle3: qfle3_queue_alloc_with_attr:642: [vmnic1] Feature RSS requested.
Environment
VMware vSphere ESXi 6.7
Cause
This occurs because NULL pointer dereference on memory allocation failure for the driver (qfle3)
Resolution
This issue is resolved QLogic
Qlogic has released a new driver for ESXi 6.7 to address this issue:
Please upgrade to latest qfle3 driver bundle below:
ESXi 6.7: Version 1.1.13.0 (Nov 22, 2020) Internal FW: 7.13.18.1
https://customerconnect.vmware.com/downloads/details?downloadGroup=DT-ESXI67-MARVELL-E3-ETHERNET-ISCSI-FCOE-20139&productId=742
ESXi 7.0: Version 1.4.13.0 (Nov 22, 2020) Internal FW: 7.13.18.1
https://customerconnect.vmware.com/downloads/details?downloadGroup=DT-ESXI70-QLOGIC-MRVL-E3-ETHERNET-ISCSI-FCOE-301430&productId=936
Workaround:
A workaround to this disable netqueue , please note there might be a performance impact ,Additional option may be to consider Standard MTU instead of Jumbo MTU
Qlogic has released a new driver for ESXi 6.7 to address this issue:
Please upgrade to latest qfle3 driver bundle below:
ESXi 6.7: Version 1.1.13.0 (Nov 22, 2020) Internal FW: 7.13.18.1
https://customerconnect.vmware.com/downloads/details?downloadGroup=DT-ESXI67-MARVELL-E3-ETHERNET-ISCSI-FCOE-20139&productId=742
ESXi 7.0: Version 1.4.13.0 (Nov 22, 2020) Internal FW: 7.13.18.1
https://customerconnect.vmware.com/downloads/details?downloadGroup=DT-ESXI70-QLOGIC-MRVL-E3-ETHERNET-ISCSI-FCOE-301430&productId=936
Workaround:
A workaround to this disable netqueue , please note there might be a performance impact ,Additional option may be to consider Standard MTU instead of Jumbo MTU
Feedback
Yes
No
Powered by
