High pNIC tx errors are detected on vmnicX on an ESXI host.

Products

• VMware vSphere ESXi

Issue/Introduction

• Monitoring tools may report high PNIC errors, and transmit errors may be observed.
  
  
• The esxcli network nic stats get -n vmnic# command may show error or drop counters that are greater than zero, on one or more physical network adapters.
  
  Below is an example output:  $ esxcli network nic stats get -n vmnic2
  

Environment

VMware vSphere ESXi

Cause

High pNIC errors on an ESXi host often indicate a driver/firmware issue, incorrect network configuration, unsupported SFP, or a physical cabling problem, and can be related to software bugs that miscount packets. 

Resolution

1. Monitoring error values:

The data these counters display are additive of issues that are external to the ESXi kernel and only what is being reported to the ESXi host from the NIC driver.

When these counters show higher values, the rate that the numbers increase can be a helpful tool. To monitor the counters and see if the numbers are increasing, use the command below:

$ watch esxcli network nic stats get -n vmnic#

It is important to note that in some instances the numbers associated with the counters can be rather large or it may not be known if the increased values are new or from a previous issue. It may be helpful to clear them before proceeding with the investigation. To clear the values the ESXi host will need to be rebooted:

1. 1. Record the current values of the counters.
   2. Place the ESXi host into Maintenance Mode.
   3. Perform a reboot.
   4. Once the ESXi host is back up the counters values will be reset to zero.  

In a healthy environment, "errors" should either be zero, or very small as a percentage of the overall total. 

2. Interpreting errors:

Transmit carrier errors:

• • These errors occur when a signal is disrupted during transmission, often caused by physical layer issues like a bad cable, faulty/unsupported SFP, interference, or misconfiguration. To troubleshoot, check the physical connection and cables for damage, ensure correct network settings for speed and duplex, and verify the network interface card (NIC) is functioning correctly.
    

3. Causes of carrier errors: 

• • Physical link disruption: A weak or damaged cable, loose connections, faulty/unsupported SFP, or electrical interference (from sources like Electromagnetic Interference) can easily disrupt the normal link pulse, triggering a carrier error.
    
    
  • Incorrect speed or duplex settings between devices, or problems during the auto negotiation process, can lead to carrier errors. These should stop once auto negotiation succeeds, but misconfiguration will cause them to continue.
    
    
  • A malfunctioning NIC or a failing component in the network can generate corrupted data that results in carrier errors.

Additional Information

Tips on troubleshooting physical network issues outside of ESXi:

• Check physical connections: Inspect all cables for damage and ensure they are securely plugged in at both ends.
• Verify network settings: Confirm that all devices on the network segment are configured with the same speed and duplex settings.
• Test the NIC: Try replacing the cable or the NIC, or test the NIC in another device to see if the error follows the hardware.
• Update drivers: Make sure the network interface card drivers are up to date.
  
  • Please refer to this KB to determine Network/Storage firmware and driver version.  Determining Network/Storage firmware and driver version in ESXi
  • Check to ensure firmware/drivers are listed in the VMware Hardware Compatibility Guide:  Broadcom Hardware Compatability Guide
• Check for interference: Ensure network cables are not running near sources of electromagnetic interference.

 

For other types of NIC errors in ESXi refer to the following KB article:   Troubleshooting NIC errors and other network traffic faults in ESXi