Network I/O performance regression after migrating from bnx2x driver to qfle3 driver
search cancel

Network I/O performance regression after migrating from bnx2x driver to qfle3 driver


Article ID: 318438


Updated On:


VMware vSphere ESXi


When you upgrade from VMware ESXi 6.5 to 6.7, you experience the below symptoms:
  • Qlogic E3 NIC driver migrates from bnx2x to qfle3 driver.
  • You experience performance regression, i.e higher CPU utilization, lower throughput in some cases.

 You see below PSOD on ESXi 6.7 host:

0x451a8f99bc20:[0x4180291533dd]FastSlab_AllocWithTimeout@vmkernel#nover+0x41 stack: 0x451a8f99bd68, 0x451a8f99bcb8, 0x451a8f99bd60, 0x430eba573560, 0x430eba6afda0
0x451a8f99bc70:[0x41802903c445]vmk_PageSlabAlloc@vmkernel#nover+0x22 stack: 0x451a00002400, 0x418000000000, 0x451a8f99bd00, 0x2400, 0x430eba573560
0x451a8f99bc80:[0x4180291cdfda]PktPageAlloc_AllocPages@vmkernel#nover+0x37 stack: 0x451a8f99bd00, 0x2400, 0x430eba573560, 0x430eba6affa0, 0x459aea403a30
0x451a8f99bd00:[0x4180292589cb]vmk_PktAllocPage@vmkernel#nover+0x10 stack: 0x430eba584c08, 0x4180299db709, 0xc04eac000, 0x2400, 0x430eba573560
0x451a8f99bd10:[0x4180299dbf51]qfle3_page_alloc_and_map@(qfle3)#<None>+0x22 stack: 0xc04eac000, 0x2400, 0x430eba573560, 0x26b, 0x430eba573560
0x451a8f99bd60:[0x4180299f05a5]qfle3_alloc_rx_sge_mbuf@(qfle3)#<None>+0x2e stack: 0x3, 0x1000, 0x0, 0x25bc, 0x430eba57e020
0x451a8f99bda0:[0x4180299f2164]qfle3_rxeof@(qfle3)#<None>+0x13d5 stack: 0x451a8f99be80, 0x418029226fbc, 0x1, 0x418029226ad3, 0x451a8f99bf00:[0x4180299f7d1e]qfle3_netpoll_rxcb@(qfle3)#<None>+0x21b stack: 0x8000000000001054, 0x4300d0db2680, 


VMware vSphere ESXi 6.7
VMware vSphere ESXi 6.5


This issue occurs due to the difference in data path mechanism: bnx2x supports queue-paring mode, while qfle3 does not. As the result, qfle3 consumes more MSI-X vectors in data path, which in turn causes higher system overhead. Because of the lack of queue-paring mode, networking Load balance is also unable to consolidate traffic in netqueue.


To resolve this issue please upgrade qfle3 driver to or later.


To workaround this issue on VMware ESXi 6.x:

1. Disable qfle3 driver and enable bnx2x driver by running the below commands:

esxcli system module set --enabled=true --module=bnx2x
esxcli system module set --enabled=false --module=qfle3

2. Reboot the ESXi host.

Note: VMware ESXi 6.7 image is shipped with both bnx2x and qfle3 drivers.