VMs lose network connectivity and TEP tunnels drop after upgrading NSX due to Transport Node Profile (TNP) synchronization.
Article ID: 412998
Updated On:
Products
Issue/Introduction
Virtual machines (VMs) experience a complete loss of network connectivity immediately following an upgrade of NSX.
All host Tunnel Endpoint (TEP) tunnels are observed to be in a DOWN state.
The ESXi hosts are part of vSphere clusters managed by Transport Node Profiles (TNPs).
Hosts within these clusters are distributed across different physical pods and were individually configured with specific, custom TEP IP Pools that override the parent TNP settings.
Post-upgrade, the individual host TEP IP configurations are missing, having been replaced by the IP pool configured in the parent TNP.
Environment
VMware NSX
Cause
This issue is caused by the re-synchronization of Transport Node Profiles (TNPs) triggered during the NSX Manager upgrade process.
When a vSphere cluster is managed by a TNP, the profile serves as the source of truth for host configuration. While it is technically possible to manually override an individual host's TEP IP Pool, the NSX upgrade process forces the TNPs to re-sync with their assigned clusters. During this synchronization, the TNP pushes its defined TEP IP Pool down to all hosts in the cluster, inadvertently overwriting any individual host-level overrides.
This overwrite misconfigures the host TEP interfaces, leading to the collapse of all host TEP tunnels and severing the overlay network connectivity for the virtual machines.
Resolution
Immediate Recovery: To quickly restore virtual machine network connectivity, manually configure NSX on each affected ESXi host and explicitly reapply the correct, individual TEP IP Pool. Once the correct IPs are applied, the TEP tunnels will re-establish.
Long-Term Prevention: To prevent future NSX upgrades or TNP synchronizations from overwriting individually configured TEP IP Pools, the NSX architectural design must be adjusted using one of the following supported methods:
Method 1: Implement Sub-Clusters and Sub-Cluster TNPs (Recommended) If you wish to continue using TNPs for automated lifecycle management while accommodating varied host configurations (such as different IP pools for different pods within the same cluster), you must implement Sub-Clusters and Sub-Cluster Transport Node Profiles.
Method 2: Detach the Transport Node Profile If utilizing Sub-Clusters is not feasible, and individual host TEP IP Pool configurations are preferred, you must completely detach the TNP from the cluster to prevent automated overwrites.
For detailed procedures on implementing the long-term preventive solutions, refer to the official VMware NSX Documentation:
Feedback
