vmnic flaps and Tx Timeout observed on Broadcom bnxtnet driver.

Products

• VMware vSphere ESXi

Issue/Introduction

• ESXi hosts report repeated alerts for "Network Uplink Redundancy Lost" and "Network Connectivity Lost".
• Physical switch ports show no errors from the switchport side.
• var/log/vobd.log indicates vmnic link state going down due to Failed criteria 128:

2026-02-05T10:03:34.115Z In(14) vobd[######]:  [netCorrelator] 5009643848687us: [vob.net.dvport.uplink.transition.down] Uplink: vmnicX is down. Affected dvPort: ###/## ## ## ## ## ## ## ##-## ## ## ## ## ## ## ##. X uplinks up. Failed criteria: 128

• The /var/run/log/vmkernel.log file contains "TX timeout" errors and NetSched watchdog warnings preceding the link down event:
  
  2026-02-05T10:03:34.009Z Wa(180) vmkwarning: cpu13:11097924)WARNING: netschedHClk: NetSchedHClkWatchdogSysWorld:7099: vmnicX: packets completion seems stuck, issuing reset
2026-02-05T10:03:34.009Z Wa(180) vmkwarning: cpu14:2097709)WARNING: bnxtnet: bnxtnet_uplink_reset:8857: [vmnicX : #x############] TX timeout!
2026-02-05T10:03:34.114Z In(182) vmkernel: cpu14:2097709)netschedHClk: NetSchedHClkNotify:5067: vmnic0: link down notification
2026-02-05T10:03:34.114Z In(182) vmkernel: cpu14:2097709)netschedHClk: NetSchedHClkDoFlushQueue:6165: vmnicX: dropping X packets from queue netsched.pools.persist.default
  
  
• Additionally, PCPU lock ups may also be seen in /var/run/log/vmkernel.log:

2026-02-05T10:02:04.010Z Al(177) vmkalert: cpu13:11143158)ALERT: NMI: 738: NMI IPI: PC #x############, SP #x############ (Src 0x1, CPU13)
2026-02-05T10:02:04.010Z In(182) vmkernel: cpu13:11143158)#x############:[#x############]Res6AffMgrComputeSortIndices@esx#nover+0x### stack: 0x0
2026-02-05T10:02:04.010Z In(182) vmkernel: cpu13:11143158)#x###########:[#x############]Res6AffmgrBatchRegionRead@esx#nover+0x### stack: 0x####
2026-02-05T10:02:04.010Z In(182) vmkernel: cpu13:11143158)#x##########:[#x##########]HelperQueueFunc@vmkernel#nover+0x### stack: 0x43########
2026-02-05T10:02:04.010Z In(182) vmkernel: cpu13:11143158)#x###########:[#x############]CpuSched_StartWorld@vmkernel#nover+0xbf stack: 0x0
2026-02-05T10:02:04.010Z In(182) vmkernel: cpu13:11143158)#x############:[#x############]Debug_IsInitialized@vmkernel#nover+0xc stack: 0x0
2026-02-05T10:02:05.287Z In(182) vmkernel: cpu79:10040013)Unmap6: 10536: NGSOC-DR-LR-OV-DS40: Acquired UC with 0 OP ucOffset 421### ucIndex: 88
2026-02-05T10:02:05.287Z In(182) vmkernel: cpu79:10040013)Unmap6: 10568: NGSOC-DR-LR-OV-DS40: Unmap capability 0 OP i: 0 offset: 65536 endROffset: 58#### Acquired: TRUE
2026-02-05T10:03:32.010Z Wa(180) vmkwarning: cpu74:2098297)WARNING: Heartbeat: 961: PCPU X didn't have a heartbeat for X seconds, timeout is X, X IPIs sent; *may* be locked up.

Environment

VMware ESXi

Cause

This issue is caused by a driver defect in the bnxtnet driver version 230.0.136.0. The driver experiences transmit (TX) timeouts and hangs, triggering the NetSched watchdog to reset the interface. This reset causes the link to momentarily drop, resulting in the "Network Uplink Redundancy Lost" alert.

Resolution

To resolve this issue, upgrade the bnxtnet driver and corresponding firmware to the latest compatible version.

1. Verify the current driver version and to obtain information on the latest driver, please follow KB: Determining Network/Storage firmware and driver version in ESXi 
2. Update the driver to the latest version by following the KB: Download and install async drivers in VMware ESXi
3. Ensure the firmware version is matched to the driver version as per the compatibility matrix in step 1.