High pNIC tx errors are detected on vmnicX on an ESXI host.

Products

• VMware vSphere ESXi

Issue/Introduction

• Monitoring tools may report high PNIC errors and transmit errors may seen.  
  
  
• The esxcli network nic stats get -n vmnic# command may show error or drop counters that are greater than zero, on one or more physical network adapters.
  
  Below is an example output:  $ esxcli network nic stats get -n vmnic2

NIC statistics for vmnic3
   Packets received: 0
   Packets sent: 13975195
   Bytes received: 0
   Bytes sent: 13975195
   Receive packets dropped: 0
   Transmit packets dropped: 0
   Multicast packets received: 0
   Broadcast packets received: 0
   Multicast packets sent: 1761
   Broadcast packets sent: 188418
   Total receive errors: 0
   Receive length errors: 0
   Receive over errors: 0
   Receive CRC errors: 0
   Receive frame errors: 0
   Receive FIFO errors: 0
   Receive missed errors: 0
   Total transmit errors: 1665
   Transmit aborted errors: 1275
   Transmit carrier errors: 390
   Transmit FIFO errors: 0
   Transmit heartbeat errors: 0
   Transmit window errors: 0

 

Environment

• VMware Vsphere ESXi

Cause

High pNIC errors on an ESXi host often indicate a driver/firmware issue, incorrect network configuration, or a physical cabling problem, and can be related to software bugs that miscount packets. 

Resolution

Monitoring error values:

The data these counters display are additive of issues that are external to the ESXi kernel and only what is being reported to the ESXi host from the NIC driver.

When these counters show higher values, the rate that the numbers increase can be a helpful tool. To monitor the counters and see if the numbers are increasing, use the command below:

$ watch esxcli network nic stats get -n vmnic#

It is important to note that in some instances the numbers associated with the counters can be rather large or it may not be known if the increased values are new or from a previous issue. It may be helpful to clear them before proceeding with the investigation. To clear the values the ESXi host will need to be rebooted:

1. 1. Record the current values of the counters.
   2. Place the ESXi host into Maintenance Mode.
   3. Perform a reboot.
   4. Once the ESXi host is back up the counters values will be reset to zero.  

In a healthy environment, "errors" should either be zero, or very small as a percentage of the overall total. 

Interpreting errors:

Transmit carrier errors:

• • These errors occur when a signal is disrupted during transmission, often caused by physical layer issues like a bad cable, interference, or misconfiguration. To troubleshoot, check the physical connection and cables for damage, ensure correct network settings for speed and duplex, and verify the network interface card (NIC) is functioning correctly.

   Total transmit errors: 1665
   Transmit aborted errors: 1275
   Transmit carrier errors: 390
   Transmit FIFO errors: 0
   Transmit heartbeat errors: 0
   Transmit window errors: 0

Causes of carrier errors: 

• • Physical link disruption: A weak or damaged cable, loose connections, or electrical interference (from sources like EMR) can easily disrupt the normal link pulse, triggering a carrier error.
    
    
  • Incorrect speed or duplex settings between devices, or problems during the autonegotiation process, can lead to carrier errors. These should stop once autonegotiation succeeds, but misconfiguration will cause them to continue.
    
    
  • A malfunctioning NIC or a failing component in the network can generate corrupted data that results in carrier errors.

 

Tips on troubleshooting physical network issues outside of ESXi

• Check physical connections: Inspect all cables for damage and ensure they are securely plugged in at both ends.
• Verify network settings: Confirm that all devices on the network segment are configured with the same speed and duplex settings.
• Test the NIC: Try replacing the cable or the NIC, or test the NIC in another device to see if the error follows the hardware.
• Update drivers: Make sure the network interface card drivers are up to date.
  
  • Please refer to this KB to determine Network/Storage firmware and driver version.  Determining Network/Storage firmware and driver version in ESXi
  • Check to ensure firmware/drivers are listed in the VMware Hardware Compatibility Guide:  Broadcom Hardware Compatability Guide
• Check for interference: Ensure network cables are not running near sources of electromagnetic interference.

 

 

Additional Information

For other types of NIC errors in ESXi refer to the following KB article:   Troubleshooting NIC errors and other network traffic faults in ESXi