Intermittent packet loss and TX_DROPS is counted on Edge VM that uses ixgben pNIC

Products

• VMware NSX

Issue/Introduction

• You observe intermittent packet loss under heavy load.
• You see TX_DROPS counts up on Edge node.
  • You can check it by get physical-port fp-ethN stats command.
• The ESXi pnics that the Edge VM uses are ixgben.

Environment

VMware NSX

VMware ESXi 7.0 and later

Cause

By default, QPair is enabled and rx and tx traffic are handled by the same pollworlds.

If rx traffic rate is high, tx could be also suffered.
ESXi could not send packets from VMs fast enough and TX_DROPS could count up.

 

Resolution

It improves performance to disable QPair for ixgben under heavy load.
When it is disabled, tx and rx pollworlds are separated and tx is not suffered from high rx rate.

For more details, please refer ESXi 7.0 release note.
https://techdocs.broadcom.com/us/en/vmware-cis/vsphere/vsphere/7-0/release-notes/vsphere-esxi-vcenter-server-70-release-notes.html

Note:

The command in the release note is just an example.
# esxcli system module parameters set -p "QPair=0,0,0,0..." -m ixgben

Each 0 or 1 corresponds to each pNIC's configuration.
For example, if you have 8 ixgben pNICs and you want to disable QPair for all of them, use the command below.
# esxcli system module parameters set -p "QPair=0,0,0,0,0,0,0,0" -m ixgben