Alarm about high pNIC error rate being detected

Products

• VMware vCenter Server
• VMware vSphere ESXi
• VMware vSAN

Issue/Introduction

Symptoms:

• After upgrading to vSphere 7.0 U2 or later, the alarm named "High pNic error rate detected. Check the host's vSAN performance view for details" gets triggered for one or more ESXi hosts.

In vSphere 7.0 U2, a new alarm named "High pNic error rate detected. Check the host's vSAN performance view for details" has been introduced, which monitors certain counters of the physical NICs used for vSAN traffic. Note that this also implies that the cause for triggering this alarm might have been present before upgrading to 7.0 U2 already.

This article explains which network metrics are monitored and what the thresholds are for the alarm to be triggered.

Cause

The following table shows the metrics for pNICs used for vSAN that are monitored and their alarm thresholds:
 

Metric	Warning Threshold	Critical Threshold
Rx CRC Errors	>0.1%	>1%
Tx Carrier Errors	>0.1%	>1%
Rx Errors	>0.1%	>1%
Tx Errors	>0.1%	>1%
Rx/Tx Pause	>1%	>10%
Rx Missed Errors	>0.1%	>1%
Rx Over Errors	>0.1%	>1%
Rx Fifo Errors	>0.1%	>1%

This information can be found with the following command, where vmnic# is the vmnic being checked. This is the information the health service uses to determine the error rate to alert on (or not).

esxcli network nic stats get -n vmnic#

Resolution

All metrics monitored by this alarm describe conditions between the vmnic port and the physical switch port. If these metrics exceed the thresholds listed above, the cause for this excess should be investigated in the physical network.

Note:

• Part of above alarms are caused by special network frame length mismatch packets.  The physical switch and/or NIC driver vendor may be able to confirm whether it can be safely ignored.

• These issues are indicative of Layer 1 concerns, the issues are being reported by the health check - and the alarm does not indicate an issue with vSAN itself.

Additional Information

For Pause Frame alerts: Configuring Flow Control on VMware ESXi and VMware ESX

For troubleshooting NIC errors: Troubleshooting NIC errors and other network traffic faults in ESXi 

Ensure that ESXi uplinks are running on compatible firmware and driver version: Determining Network/Storage firmware and driver version in ESXi