vSAN Host/Cluster Summary page alert "High pNic error rate detected" on ESXi 9.x hosts within a mixed-version (ESXi 8.x / 9.x) cluster

Products

• VMware vSphere ESXi

Issue/Introduction

When a vSAN cluster is undergoing a rolling upgrade, where some hosts have been upgraded to ESXi 9.x while others remain on ESXi 8.x, the upgraded ESXi 9.x hosts trigger network alerts visible on the summary dashboards.

• Host or Cluster Summary Page Alerts:

  • Navigating to the Summary page of the affected ESXi 9.x host or the vSAN Cluster displays the following critical alert:

    High pNic error rate detected. Check the host's vSAN performance view for details

• vSAN Performance View Alerts:

  • Navigating to Monitor -> vSAN -> Performance on the affected ESXi 9.x host displays specific timestamped alerts such as:

    High pNic RX Generic error rate detected on vmnicX on <DATE>
    
    High pNic TX Generic error rate detected on vmnicX on <DATE> 

    (where vmnicX represents the vSAN uplink adapters, e.g., vmnic2, vmnic3...)

• Performance Metrics Breakdown:

  • Navigating to Monitor -> vSAN -> Performance -> PHYSICAL ADAPTERS shows that the "pNIC Packet Error Rate" is not 0.

  • Navigating to Monitor -> vSAN -> Performance -> HOST NETWORK shows that the "vSAN Host Packet Discard Rate" is not 0.

Environment

• VMware vSphere ESXi 8.x

• VMware vSphere ESXi 9.x

• Mixed-version vSAN clusters undergoing rolling upgrades

Cause

This alert is caused by the temporary co-existence of different major ESXi versions (8.x and 9.x) within the same vSAN cluster during a rolling upgrade.

The version disparity between heterogeneous nodes affects the cluster communication metrics, leading to the pNIC alerts on the Summary page. These alerts are transitional and will clear automatically once all cluster nodes are upgraded to a unified version.

Resolution

To resolve this issue.

1. Upgrade all remaining ESXi 8.x hosts in the cluster to the identical ESXi 9.x version.
2. Once all hosts are upgraded, acknowledge and clear the transient UI alarms from the vCenter Server.

Additional Information

• As per the KB VMware vSAN Upgrade Best Practices , a mixed-version vSAN cluster is strictly a temporary operational state during upgrades. It is highly recommended to complete the upgrade of all remaining cluster nodes as expeditiously as possible to prevent monitoring and health reporting discrepancies.
  
  
• To speak with a customer representative or a Support Engineer see Contact Support. Scroll to the bottom of the page and click on your respective region.
  
  
• Subscribe to this article to receive updates: How to subscribe to Knowledge Management articles.
  
  
• For information on downloading software updates, see Download Broadcom products and software.