ESXi VMkernel Interfaces vmk10/vmk11 Fallback to APIPA Causing NSX VTEP Failure

Products

• VMware vSphere ESXi 8.0
• VMware NSX

Issue/Introduction

On NSX-prepared VMware ESXi hosts, VMkernel interfaces (commonly vmk10 and vmk11) that are configured for NSX VTEP functionality may lose their DHCP-assigned IP addresses and revert to APIPA addresses (169.254.#.#). As a result, NSX overlay network connectivity is lost for the affected host.

• vmk10 or vmk11 shows an APIPA address (169.254.#.#) instead of the expected subnet IP.

You will see logs similar to below

• /var/run/log/vmkernel.log contains repeated messages such as:

WARNING: Tcpip: 1168: Failed to set netmask. (error = 0x11)
WARNING: Tcpip_Vmk: 884: vmk_set_ip_address failed with error = 0x11, status = 0xbad0005
Tcpip_Vmk: 123: rn_delete: return NULL after checking rn_dupedkey
WARNING: vdl2: VDL2CheckVmknicStatus:3592: No valid v4 TEP IP for a long time of vmknic[vmk10/vmk11]

• /var/run/log/dhclient.log shows DHCP responses being received and acknowledged but ultimately declined:

DHCPOFFER, DHCPACK received...
Unable to Set: Address not available: VSI node (...) . Declining offer.

• NSX Controller logs may show:

Routing Domain VTEP Membership Update: Num of removed entries: 1
The vtep ip fetching retry limit exceeded. Abort the task.

Environment

VMware ESXi 7.0 U3 and 8.0 with NSX-T 3.#/4.#

Cause

This issue is caused by a stale routing entry in the ESXi kernel networking stack that prevents successful binding of a new DHCP IP to the VMkernel interface.

When a DHCP lease expires or an IP change is triggered, the ESXi kernel attempts to unbind the existing IP. However, a failure to remove the corresponding route (indicated by the log entry rn_delete: return NULL after checking rn_dupedkey) causes the stack to incorrectly retain the old binding.

Even though the DHCP server successfully responds (DHCPOFFER, DHCPACK), the kernel fails to apply the new address due to the lingering route. This leads to:

Failed to set netmask → vmk_set_ip_address fails → vmk falls back to APIPA

The affected TEP (vmk10 or vmk11) cannot join the NSX overlay, and the Controller removes the host’s VTEP entry due to IP unavailability.

Resolution

Workaround:

Remove Stale Route via CLI 

• SSH into the affected ESXi host.

• List current IPv4 routes:

esxcli network ip route ipv4 list

• Identify any stale "Local Subnet" routes linked to the old IP/subnet of vmk10 or vmk11:

Network: 192.1.#.#  Netmask: 255.255.255.0  Interface: vmk11

• Remove the stale route:

esxcli network ip route ipv4 remove -n <network> -N <netmask> -i <vmk_interface>

Note: Ensure values are correct before removing routes to avoid unintended impact.

• Trigger DHCP renewal:

esxcli network ip interface ipv4 set -i vmk11 -t dhcp

• Verify the new IP:

esxcli network ip interface ipv4 get -i vmk11

Alternatively a reboot of the ESXi host can also fix this issue.