ESXi host displays "Uplink redundancy lost" alert in vCenter due to CRC errors on the NIC
search cancel

ESXi host displays "Uplink redundancy lost" alert in vCenter due to CRC errors on the NIC

book

Article ID: 410676

calendar_today

Updated On:

Products

VMware vSphere ESXi

Issue/Introduction

  • Receiving the following alerts in vCenter on the ESXi host:  "Lost uplink redundancy" and/or "Network uplink redundancy lost"
  • Confirmed that NIC driver/firmware is compatible as per VMware HCL: Determining Network/Storage firmware and driver version in ESXi
  • Checking NIC stats, it shows that there are CRC error increasing rapidly:
    • Login to ESXi host over SSH session and run the following command:  esxcli network nic stats get -n vmnic<X> 
      Example output:
      [root@ESXi:~] esxcli network nic stats get -n vmnic0
      NIC statistics for vmnic0
         Packets received: 599736627
         Packets sent: 489239060
         Bytes received: 189333612000
         Bytes sent: 1264126208861
         Receive packets dropped: 0
         Transmit packets dropped: 0
         Multicast packets received: 95624657
         Broadcast packets received: 13147021
         Multicast packets sent: 760144
         Broadcast packets sent: 4614094
       Total receive errors:2349184
       Receive length errors: 0
         Receive over errors: 0
       Receive CRC errors: 2349184 <<<<<<<-------- CRC errors increasing rapidly
         Receive frame errors: 0
         Receive FIFO errors: 0
         Receive missed errors: 0
       Total transmit errors: 0
       Transmit aborted errors: 0
       Transmit carrier errors: 0
         Transmit FIFO errors: 0
         Transmit heartbeat errors: 0
         Transmit window errors: 0

Cause

  • A Cyclic Redundancy Check (CRC) is an error-detecting code commonly used in digital networks and storage devices to detect accidental changes to digital data.
  • CRC error: FCS (Frame Check Sequence) field contains a 4-byte CRC value used for error checking.
  • When a source host assembles a packet, it performs a CRC calculation on all fields in the packet except the Preamble, SFD (Start Frame Delimiter), and FCS using a predetermined algorithm.
    The source host stores the value in the FCS field and transmits it as part of the packet.
    When the packet is received by the destination host, it performs a CRC test again by using the same algorithm.
    If the CRC value calculated at the destination host does not match the value in the FCS field, the destination host discards the packet, considering this as a CRC Error.

Resolution

  • Please reach to respective hardware vendors(Physical Switch/SFT/Cable/Network Adapter) to troubleshoot further.
  • Common troubleshooting checks that can be done:
    • Check if SFP is compatible with card and similar to the other vmnic's
    • Check and clean transceiver optic part
    • Check if SFP is correctly seated on the card and physical switch switchport
    • Verify cable path for damage or with tight bends/short radius that may increase cable attenuation for a fiber type.
    • Verify physical switchport stats for any transmit errors

Additional Information