Host Preparation Failure: VTEP Installation Incomplete and Host in Partial Success State.

Products

• VMware NSX

Issue/Introduction

• You may hit this issue when building a new workload domain through SDDC Manager UI. Getting error message.

         Error: One or more transport node(s) realization failed while creating the transport node collection with profile [Profile-ID] on compute collection [Collection-ID] through NSX Manager [nsxmanager.example.com]
         Remediation Message: Check the error(s) on NSX Manager [nsxmanager.example.com]
         Cause: failed to realize transport node.  Please refer logs

• Host configuration failed with below error seen in NSX UI
  
  Host configuration: VTEP [vmk11, <IP address>] failed to be applied: Failed to add VTEP: {'fault': 'PlatformConfigFault', 'msg': 'An error occurred during host configuration.', 'faultMessage': ['Operation failed, diagnostics report: Unable to Set: Network unreachable: VSI node (480:VSI_NODE_net_tcpip_instances_vNics_v4_gateway)']}

• Analysis of the host's vmkernel logs reveals that one of the VTEP interface is inadvertently configured to point to an incorrect gateway address

Environment

VMware NSX

Cause

The root cause is a gateway mismatch for VTEPs when a single TEP IP Pool contains multiple subnets and IPs from one subnet are exhausted, forcing subsequent VTEPs to pull IPs from another subnet within the same pool. The VXLAN netstack can only maintain a single default gateway for TEPs, which is derived from the first acquired subnet, leading to network unreachability for TEPs whose IPs belong to a different subnet. This behavior is considered a design limitation, not a defect.

Resolution

• Do not configure TEP Pool with multiple IP subnets
• Ensure there are sufficient IP address available in the subnets when applying the TNP to cluster
• If only one Host is affected, you can attempt to remove the Host from NSX and add it back to re-prepare the Host for NSX

 

For last resolution bullet, steps to remove and re-add the Host to NSX for re-preparation

• Place ESXi Host into vSphere Maintenance Mode in the vSphere UI
• Drag Host out of cluster into the Datacenter level object in the vSphere UI
• Monitor the uninstall process in NSX UI > System > Fabric > Hosts > Other Nodes
• Once Host shows as "Not Configured" in the NSX UI, drag Host back into cluster in the vSphere UI
• Monitor install process in the NSX UI under the cluster in question

If there are issues during the removal and re-add process, it could be due to failure to remove NSX VIBs from the Host.  

Uninstall NSX from a vSphere Cluster