ESXi hosts lose overlay connectivity and TEP tunnels drop when specific TEP IP addresses are assigned.
Article ID: 434336
Updated On:
Products
Issue/Introduction
ESXi hosts may experience an intermittent loss of all overlay connectivity when specific Tunnel Endpoint (TEP) IP addresses are assigned by NSX. When a conflict occurs, the TEP interfaces (vmk10 and vmk11) detect duplicate IP addresses on the network, causing all tunnels on the impacted host to drop. This results in a complete loss of Virtual Machine (VM) connectivity, often requiring manual workload migration to alternate hosts within the cluster to restore service.
Symptoms include:
Intermittent "Tunnels Down" for transport nodes.
ESXi vmkernel interfaces (typically vmk10 and/or vmk11) logging alerts for duplicate IP addresses.
Entries similar to the below are observed in ESXI logs:/var/run/log/vobd.log:2025-03-06T12:05:47.382Z: [netCorrelator] 8492350294662us: [esx.problem.net.vmknic.ip.duplicate] Duplicate IP address detected for 10.###.###.12 on interface vmk10, current owner being 00:50:##:##:12:ac.2025-03-06T12:05:47.675Z: [netCorrelator] 8492323880740us: [vob.net.vmknic.ip.duplicate] A duplicate IP address was detected for 10.###.###.12 on interface vmk10. The current owner is 00:50:##:##:12:ac.2025-03-06T12:05:47.701Z: [netCorrelator] 8492350294662us: [esx.problem.net.vmknic.ip.duplicate] Duplicate IP address detected for 10.###.###.23 on interface vmk11, current owner being 00:50:##:##:23:aa.2025-03-06T12:05:47.723Z: [netCorrelator] 8492323880740us: [vob.net.vmknic.ip.duplicate] A duplicate IP address was detected for 10.###.###.23 on interface vmk11. The current owner is 00:50:##:##:23:aa.
Environment
VMware NSX
Cause
The root cause is an IP address conflict typically originating from one of two scenarios:
- Orphaned Hosts: A decommissioned ESXi host remains powered on and continues to advertise TEP IP addresses to the physical network.
- Management Plane Desynchronization: Physical network or VLAN misconfigurations disrupt communication between Transport Nodes and the NSX Manager. This prevents the Manager from receiving 'release' triggers, leading to stale entries in the IP Pool database where the Manager erroneously reassigns an 'in-use' IP to a new host."
Resolution
To resolve the connectivity loss, you must remove the source of the duplicate IP advertisement from the physical network:
Identify the Source: Use a physical network trace (via the physical switch) to locate the port associated with the conflicting MAC addresses from ESXi vmkernel logs.
Decommission the Orphaned Host: Once the physical host is located, perform one of the following actions:
Power down the decommissioned/orphaned ESXi host completely.
Remove the legacy NSX configuration from the orphaned host to officially release the TEP IPs back to the network.
Verify Resolution: After the conflict is cleared from the wire, the TEP interfaces on the active transport nodes will no longer trigger duplicate IP detection, allowing tunnels to establish correctly.
Additional Information
Feedback
