Alarm about high pNIC error rate being detected

Products

• VMware vCenter Server
• VMware vSphere ESXi

Issue/Introduction

In vSphere 7.0 U2 a new alarm named "High pNic error rate detected. Check the host's vSAN performance view for details" has been introduced, which monitors certain counters of the physical NICs used for vSAN traffic. Note that this also implies that the cause for triggering this alarm might have been present before upgrading to 7.0 U2 already.
This article explains which network metrics are monitored and what the thresholds are for the alarm to be triggered.

Symptoms:
After upgrading to vSphere 7.0 U2 or later the alarm named "High pNic error rate detected. Check the host's vSAN performance view for details" gets triggered for one or more ESXi hosts.

Environment

VMware vSphere ESXi 7.0.x
VMware vSphere ESXi 8.0.x

VMware vCenter Server 7.0.x
VMware vCenter Server 8.0.x

VMware vSAN 7.x
VMware vSAN 8.x

Cause

The following table shows the metrics for pNICs used for vSAN that are monitored and their alarm thresholds:
 

Metric	Warning Threshold	Critical Threshold
Rx CRC Errors	>0.1%	>1%
Tx Carrier Errors	>0.1%	>1%
Rx Errors	>0.1%	>1%
Tx Errors	>0.1%	>1%
Rx/Tx Pause	>1%	>10%
Rx Missed Errors	>0.1%	>1%
Rx Over Errors	>0.1%	>1%
Rx Fifo Errors	>0.1%	>1%

This information can be found with the following command, where vmnic# is the vmnic being checked. This is the information the health service uses to determine the error rate to alert on (or not).

esxcli network nic stats get -n vmnic#

Resolution

All metrics monitored by this alarm describe conditions between the vmnic port and the physical switch port. If these metrics exceed the thresholds listed above, the cause for this excess should be investigated in the physical network.

Note: Part of above alarms are caused by special network frame length mismatch packages, after confirmed by package source vendor, it could be safely ignored.