High virtual network throughput performance tuning recommendation when using 100G network interface cards
Article ID: 312057
Updated On:
Products
Issue/Introduction
This article provides performance tuning recommendations for physical NICs in a vSphere environment.
Users might not see a high enough throughput for a VNIC to fill the physical NIC bandwidth.
Environment
- VMware NSX-T
- VMware NSX 4.0.0.1
- VMware vSphere ESXi 8.0.2
Cause
By default, ESX/NSX networking uses limited number of threads to service Tx and Rx of a vNIC. Throughput of a vNIC is limited by the processing capacity of those threads.
Resolution
If a vNIC requires high throughput, whether just high Rx side throughput or bi-directional, it is recommended to configure ethernetX.pnicFeatures=4, and ethernetX.ctxPerDev=3 in the virtual machine's VMX file, to get the parallelism in the networking layer. This gives a much more balanced Tx side and Rx side bandwidth, roughly equally in the bi-directional test. These tunings work best with pNIC RSS enabled.
Relevant documents:
Receive Side Scaling
Performance Best Practices for VMware vSphere 8.0
Additional Information
Enabling RSS, and other tuning on the Guests vNICs may cause higher CPU requirements for the edited VM, possibly affecting other VMs negatively by lowering the CPU resources available.
Feedback
