"High pNic RX/TX Pause rate detected" on vmnic with little to no throughput/packets

Products

• VMware vSAN

Issue/Introduction

Symptoms:

• The alert "High pNic RX/TX Pause rate detected on vmnicX" is seen on vSAN hosts.

• Where vmnicX could be the standby vmnic or even the active vmnic which little to no throughput or packets passing through it, as seen from the vSphere Client > Select host > Monitor > vSAN Performance Graph > Physical Adapters.

Environment

vSAN 7.x
vSAN 8.x

Cause

• Pause count or flow control packet may be observed on vmnics even when they are transmitting very low amount of data (less than 1-10 kb/s), this would be the case on standby vmnics or active vmnics when vSAN is not being used.
• This is expected behavior, as the driver for the vmnic would trigger ESXi to reduce the number of receive queues allocated to the vmnic during times of low/no traffic.
• Now, if there is a sudden burst in IO through this vmnic, it can quickly fill up the queue and lead to exhaustion and trigger pause frames.
  Example graph:
  

Resolution

• No action is needed here, the vmnic driver would inform ESXi host to increase the number of queues to handle the IOs.

• Once IOs start increasing, the amount of Flow control packets would reduce as the vmnic deploys enough queues to handle the IOs.

• The alert can be reset to green.

• Incase the alert reappears multiple times after the IOs are consistent, then open a case with Broadcom Technical Support to investigate further.