25Gbps Intel E810-XXV vmnic down after switch reboot

Products

• VMware vSphere ESXi

Issue/Introduction

This article provides steps on how to troubleshoot when a vmnic is down primarily focusing on the physical server NIC settings as well as providing information to engage with the physical server OEM vendor efficiently.

Symptoms

• After a switch reboot, 25Gbps Intel E810-XXV vmnic goes down on some ESXi hosts randomly.
• "esxcli network nic list" output shows Link Status down on some vmnics with Speed 0 and Half Duplex
• Changing the vmnic link speed from 25Gbps to Auto-negotiation or from Auto-negotiation to 25Gbps on ESXi, or restart the ESXi host temporarily resolves the issue.

Environment

VMware vSphere ESXi 7.0

Resolution

Understanding vmnic Functionality and Troubleshooting Responsibilities

The proper functioning of a vmnic on an ESXi host depends on multiple components working together:

• NIC hardware
• NIC firmware
• NIC driver (VIB)
• ESXi operating system

To identify the responsible party for troubleshooting a network NIC issue, refer to Broadcom's Compatibility Guide for ownership details.

Ownership and Troubleshooting Scope

If the NIC driver is classified as Async, the OEM server vendor owns the NIC hardware, firmware, and driver. In this case, all troubleshooting should be handled by the OEM vendor, as the vmnic status propagates through the following path: hardware → firmware → driver → ESXi.

A common resolution for vmnic link-down issues involves replacing faulty NIC cards. While VMware can provide test-based workarounds, the primary responsibility lies with the OEM vendor.

Workaround

Note: The following workaround options (option b /option c) only reduce the likelihood of encountering the issue and are not guaranteed fixes.

According to Intel(R) E810 Adapter Firmware Release Notes for Version 20.0.18, stability issues can occur if Forward Error Correction (FEC) is enabled or if link speed negotiation is left on Auto. To improve compatibility with certain switches, disable FEC and set the link speed manually. 

Configuration Steps

1. Reboot the server and boot into Device Settings
2. Select the Intel E810-XXV NIC
3. In NIC Main Configuration Page, select NIC Configuration
4. Set the 'Link Speed Method' to Manual and force link speed to 25Gbps. (The default value is Auto.)
5. Return to the main NIC Configuration Page and scroll to Forward Error Correction
   • Set it to Disabled (default is Auto)
     Note: This field is only editable if Link Speed Method is set to Manual.
6. Ensure the switch-side FEC configuration matches the NIC settings, following the switch vendor's documentation.

Tested Configuration Options

Options B and C both offer improvements over Option A, with Option C yielding the most reliable performance.

Option	Permit Total Port Shutdown	Link Speed Method	Link Speed	FEC Mode	ESXi (Speed Duplex) Setting	Switch Setting	Notes
A (Default)	Disabled	Auto	Auto	Auto	Auto	Auto-negotiation & Auto FEC	Baseline configuration
B	Enabled	Auto	Auto	Auto	Auto	Auto-negotiation & Auto FEC	Slight improvement over default
C (Recommended)	Enabled	Manual	25Gbps	Disabled	25Gbps Full Duplex	25Gbps / Full Duplex / FEC Disabled	Best results in testing

Additional Information

Impact/Risks:

vmnic down may cause traffic loss on the ESXi host.