Uplink Flapping after enabling Enhanced Data Path in the Host Node profile in NSX-T
search cancel

Uplink Flapping after enabling Enhanced Data Path in the Host Node profile in NSX-T

book

Article ID: 434778

calendar_today

Updated On:

Products

VMware NSX

Issue/Introduction

In VMware NSX-T environments, enabling the Enhanced Data Path (EDP) on a Host Node Profile may result in intermittent uplink flapping. This behavior is characterized by physical NICs (vmnics) frequently transitioning between 'Up' and 'Down' states, potentially impacting network stability for overlay and VLAN traffic.

The following symptoms are observed in /var/run/log/vobd.log on the affected ESXi host:

2026-03-11T01:32:05.671Z In(14) vobd[2098533]: [netCorrelator] 139031250581us: [vob.net.dvport.uplink.transition.down] Uplink: vmnic4 is down. Affected dvPort: 2a18913d-ffa7-4ae8-b7fb-d1df2c5bdc05/50 2d 64 e7 b0 fc de c9-6c ac 9d 5a 3e 5e 84 81. 1 uplinks up. Failed criteria: 128
2026-03-11T01:32:05.672Z In(14) vobd[2098533]: [netCorrelator] 139031250587us: [vob.net.dvport.uplink.transition.down] Uplink: vmnic4 is down. Affected dvPort: c6b80cf1-b48e-4e6d-8e5e-5cf91791937b/50 2d 64 e7 b0 fc de c9-6c ac 9d 5a 3e 5e 84 81. 1 uplinks up. Failed criteria: 128

2026-03-11T01:32:06.291Z In(14) vobd[2098533]: [netCorrelator] 139031871005us: [vob.net.dvport.uplink.transition.up] Uplink: vmnic4 is up. Affected dvPort: 1c440943-42ad-4c2b-8fee-03620ed71bb9/50 2d 64 e7 b0 fc de c9-6c ac 9d 5a 3e 5e 84 81. 2 uplinks up
2026-03-11T01:32:06.291Z In(14) vobd[2098533]: [netCorrelator] 139031871006us: [vob.net.dvport.uplink.transition.up] Uplink: vmnic4 is up. Affected dvPort: f3af338c-41b0-4053-ab31-15a269dcdfda/50 2d 64 e7 b0 fc de c9-6c ac 9d 5a 3e 5e 84 81. 2 uplinks up

On ESXi host /var/run/log/vmkwaring.log below log entries are seen:

2026-03-11T01:32:05.396Z Wa(180) vmkwarning: cpu27:2121163)WARNING: netschedHClk: NetSchedHClkWatchdogSysWorld:7044: vmnic4 : scheduler(0x4325a52016e0)/device(0x4308f4447040) 1/0 lock up [stopped=0]:
2026-03-11T01:32:05.396Z Wa(180) vmkwarning: cpu27:2121163)WARNING: netschedHClk: NetSchedHClkWatchdogSysWorld:7052: detected at 139030974 while last xmit at 138914184 and 265518 bytes in flight [window 216216 bytes]
2026-03-11T01:32:05.396Z Wa(180) vmkwarning: cpu27:2121163)WARNING: netschedHClk: NetSchedHClkWatchdogSysWorld:7059: and last enqueued/dequeued at 139025467/138914183 [stress 0]
2026-03-11T01:32:05.396Z Wa(180) vmkwarning: cpu27:2121163)WARNING: netschedHClk: NetSchedHClkWatchdogSysWorld:7064: with 476 pkts inflight
2026-03-11T01:32:05.396Z Wa(180) vmkwarning: cpu27:2121163)WARNING: netschedHClk: NetSchedHClkWatchdogSysWorld:7099: vmnic4: packets completion seems stuck, issuing reset

On ESXi host /var/run/log/vmkernel.log below log entries are seen:

2026-03-11T01:32:05.396Z In(182) vmkernel: cpu14:2097965)ENS-LB: Ens_LBUnregisterUplinkPortBegin:15010: Uplink 7 is unregistered.
2026-03-11T01:32:05.601Z In(182) vmkernel: cpu14:2097965)<NMLX_INF> nmlx5_core: vmnic4: nmlx5_en_UplinkQuiesceIOLocked - (nmlx5_core_en_main.c:2811) called
2026-03-11T01:32:05.669Z In(182) vmkernel: cpu14:2097965)vswitch: VSwitchPortUpdateLinkChange:6674: [nsx@6876 comp="nsx-esx" subcomp="vswitch"]Uplink vmnic4(0x8400000c) link down event detected
2026-03-11T01:32:06.094Z In(182) vmkernel: cpu15:2097965)<NMLX_INF> nmlx5_core: vmnic4: nmlx5_en_UplinkStartIOLocked - (nmlx5_core_en_main.c:2643) called
2026-03-11T01:32:06.187Z In(182) vmkernel: cpu15:2097965)netschedHClk: NetSchedHClkNotify:5059: vmnic4: link up notification
2026-03-11T01:32:06.188Z In(182) vmkernel: cpu0:2097966)nsx_vdrb: VdrHandleUplinkEvent:2246: [nsx@6876 comp="nsx-esx" subcomp="vdrb-24954728"]SYS:DvsPortset-0: Uplink event 1 for port 0x8400000c, linkstate 1
2026-03-11T01:32:06.188Z In(182) vmkernel: cpu15:2097965)<NMLX_INF> nmlx5_core: vmnic4: nmlx5_en_UplinkLinkStateSetOS - (nmlx5_core_en_uplink.c:5231) Changing link status from DOWN Half Duplex 0 to UP Full Duplex 25000

Environment

VMware NSX-T Data Center

NSX-T 4.x

Cause

A netschedHClk watchdog timeout occurs when TX packet completions are delayed or stuck while Enhanced Data Path (EDP) is enabled. This results in the watchdog identifying a scheduler-to-device lockup, which triggers a mandatory NIC reset to recover the transmit queues, leading to an intermittent link flapping event.

Resolution

The following options are available to resolve or mitigate the netschedHClk TX stall and subsequent NIC resets.

Option 1: Perform ESXi Upgrade (Recommended) Upgrade the affected ESXi hosts to version 8.0.3 P08 or later. This version contains driver and scheduler optimizations that prevent the packet completion watchdog from incorrectly triggering a reset during Enhanced Data Path operations.

Option 2: Disable Network I/O Control (NetIOC) If an upgrade is not immediately feasible, disabling Network I/O Control (NetIOC) on the Distributed Switch (VDS) can mitigate the conflict.

[!CAUTION] Disabling NetIOC is a disruptive operation. This action triggers a reset of all uplinks for every host participating in the VDS. Perform this task only during an approved maintenance window to avoid unplanned production outages.

  1. Log in to the vSphere Client.

  2. Navigate to the Networking inventory and select the Distributed Switch.

  3. Select Configure > Settings > Properties.

  4. Click Edit.

  5. Select Network I/O Control and change the status to Disabled.

  6. Click OK.

Refer to the official documentation for detailed procedural steps: Enable or Disable Network I/O Control on a vSphere Distributed Switch

Powered by Wolken Software