NIC Flapping and "Lost Uplink Redundancy" on ESXi nodes with Intel X710 (i40en)
search cancel

NIC Flapping and "Lost Uplink Redundancy" on ESXi nodes with Intel X710 (i40en)

book

Article ID: 436585

calendar_today

Updated On:

Products

VMware vSphere ESXi

Issue/Introduction

Symptoms:

  • One or more Intel X710 (i40en) VMNIC interfaces experience persistent link flapping (Up/Down).

  • vCenter reports "Network uplink redundancy lost" and "Network uplink redundancy restored" in rapid succession.

  • The following events are seen in vobd.log 

2026-03-22T01:54:49.844Z In(14) vobd[2098051]:  [netCorrelator] 420311338626us: [vob.net.dvport.uplink.transition.down] Uplink: vmnic# is down. Affected dvPort: ##/# ## ## ## ## ## ## ##-## ## ## ## ## ## ## ##. # uplinks up. Failed criteria: 128
2026-03-22T01:54:49.844Z In(14) vobd[2098051]:  [netCorrelator] 420311338702us: [vob.net.vmnic.linkstate.down] vmnic vmnic# linkstate down
  • The vmkernel.log contains netschedHClk (Network Scheduler) watchdog errors:

2026-03-22T01:54:49.843Z In(182) vmkernel: cpu56:2098354)i40en: i40en_UpdateUplinkLinkStatus:7043: ####:##:##.#: Link is down for device
2026-03-22T01:54:49.843Z In(182) vmkernel: cpu56:2098354)Uplink: 65: update uplink: vmnic#, state: 0
2026-03-22T01:54:49.844Z In(182) vmkernel: cpu56:2098354)netschedHClk: NetSchedHClkNotify:5069: vmnic#: link down notification
2026-03-22T01:54:49.844Z In(182) vmkernel: cpu56:2098354)Uplink: 14587: vmnic# link state 0 speed 0 half duplex
2026-03-22T01:54:50.501Z In(182) vmkernel: cpu28:3053988)netschedHClk: NetSchedHClkWatchdogSysWorld:7126: vmnic#: failed to get the coalescing settings: Not supported
2026-03-22T01:54:50.501Z In(182) vmkernel: cpu28:3053988)netschedHClk: NetSchedHClkWatchdogSysWorld:7146: vmnic#: failed to push the coalescing settings, increase inflight window to 172920 bytes: Not supported
2026-03-22T01:54:50.501Z In(182) vmkernel: cpu21:2097645)NetqueueBal: 5211: vmnic#: device Up notification, reset logical space needed
  • Physical cable unplug/re-plug and/or Switch port changing do not resolve the issue.

  • The issue appeared after upgrading from VxRail 8.0.361 to 8.0.370; it can affect VxRail environments post-upgrade.

Environment

VMware vSphere ESXi

Cause

  • The issue is typically caused by an interoperability conflict between the Intel i40en driver and the NIC firmware
  • Specifically, the ESXi Network Scheduler (netsched) watchdog fails to negotiate hardware coalescing settings with the firmware.
  • When this negotiation fails or when the scheduler perceives a "stuck" packet condition, it triggers a hardware reset of the NIC to attempt recovery, leading to the sub-second flapping observed in the logs.

Resolution

  • To resolve this issue , ensure the driver and firmware are fully compliant with the Broadcom's HCL.
  • Incase you are already running on the suggested or higher versions of that firmware , contact Hardware vendor 
  • Raise a case with Broadcom for further assistance and more information.