NICs using elxnet driver may experience small throughput regression for workloads

Products

• VMware vSphere ESXi

Issue/Introduction

Symptoms:

• You may experience reduction in throughput for elxnet driver when there is only one virtual machine which has heavy network traffic or when the network traffic is vmknic workload (such as vMotion, nfs) or encapsulated traffic using non-VXLAN protocols (such as Geneve).
• You see reduction of throughput of about 2% to 12% for some single-VM benchmarks.

Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.

Environment

VMware vSphere ESXi 6.7

Cause

In ESXi 6.7, netqueue RSS is disabled by default for elxnet, driver uses smaller number of queues to process network traffic. This is because netqueue RSS and dynamic netqueue are mutually exclusive for elxnet devices, and we chose to enable dynamic netqueue by default as it reduces CPU cost and offers similar throughput in typical multi-VM workloads.

Resolution

For elxnet driver, if lower throughput for vmknic workload (such as vmotion, nfs) or encapsulated traffic (such as Geneve) is not acceptable, you can choose to manually turn on netqueue RSS.

To turn on netqueue RSS for elxnet, set elxnet driver module parameter dyn_netq to 0.