NSX-T Edge tunnels are down to ESXi hosts when sharing the same VLAN for TEP traffic
search cancel

NSX-T Edge tunnels are down to ESXi hosts when sharing the same VLAN for TEP traffic

book

Article ID: 312645

calendar_today

Updated On:

Products

VMware NSX

Issue/Introduction

The following tunnel connectivity issues are observed and Hosts and Edge Nodes are configured to use the same VLAN for TEP traffic:

  • Tunnels between Hosts and Edge Nodes are down.
  • Tunnels between Hosts are up.
  • Tunnels between Edge Nodes are up.

Environment

  • VMware NSX 4.x
  • VMware NSX-T Datacenter 3.x
  • VCF 9.x

Cause

Tunnels between Edge Nodes and Hosts will be down under the following conditions:

  • Edge deployed on NSX-prepared Host with "NSX on DVPG" disabled
    • When an Edge Node is deployed on an NSX-prepared host where "NSX on DVPG" is disabled
    • AND Edge TEPs (Tunnel Endpoints) are attached to standard vSphere Port Groups
  • Edge deployed on non-NSX-prepared Host that is later prepared for NSX without "NSX on DVPG" enabled.
    • When an Edge Node is deployed on a non-NSX-prepared host
    • AND the host is subsequently prepared for NSX: the necessary configuration (`nestedTNConfig` property) sent by the Management Plane (MP) to the Host may not be transmitted, causing the Host to only listen to traffic for its TEP interface and drop all traffic for the TEP interface of the Edge Node.

Resolution

Workaround:

Multiple workarounds are available for this issue, listed in order of preference based on NSX version.

 

For VCF 9.x

Pre-Condition: Edge TEP is connected to standard distributed Port Groups

 

Workaround 1: Enable NSX on DVPG (Recommended)
If NSX on DVPG is disabled, follow the steps in the technical documentation to activate it:

Reference Guide: [Activate NSX on DVPG] (Activate NSX on Distributed Virtual Port Groups (DVPGs))

 

Workaround 2: Configure a VLAN-Backed Segment

Create a trunked VLAN backed segment and connect that segment to Edge TEP interface.

Steps: 

1. Create a VLAN Transport Zone (if you don't already have one).

2. Create a new NSX VLAN Segment within that VLAN transport zone.

3. Configure the Segment for Trunking: In the segment's settings, specify a VLAN ID range (e.g., 0-4094) instead of a single VLAN ID.

4. Create an Edge Uplink Profile and IP Pool: Configure these to use the shared TEP VLAN ID and subnet.

5. Configure the Edge Node: When deploying or reconfiguring your Edge Node, map the TEP interface to the newly created Trunk Segment.

Note: If using a VLAN segment on an NSX prepared VDS, adding only a single VLAN to the segment will result in the segment processing traffic for that VLAN only and stripping the tag for ingress traffic and adding the tag for egress traffic. This will result in the edge TEPs being unable to communicate as they require a trunked segment. This can be achieved by applying a range to the VLAN value (inclusive of your required VLAN) .

 

Workaround 3: Use Different VLAN for Edge TEP Traffic
Place the TEP traffic of the Edge Node on a different VLAN than the Host's TEP VLAN.

Steps:

1. Create a new uplink profile with a new VLAN
2. If an IP pool is used, create a new IP pool that allocates IP addresses in the newly created VLAN
3. Apply this profile to the Edge Node

Note: This workaround requires a separate subnet/VLAN.

 

Workaround 4: Create New NVDS/VDS for ESXi Host
Create a new NVDS/VDS for the ESXi Host and assign a different uplink (vmnic) to the NVDS/VDS. Apply the same uplink profile (including VLAN) and IP pool to the new uplink.

Note: This workaround requires additional vmnic(s) for the new NVDS/VDS.

 

For NSX 4.2.x

Pre-Condition: Edge TEP is connected to standard distributed Port Groups

 

Workaround 1: Enable NSX on DVPG (Recommended)

NSX on DVPG is disabled by default in NSX 4.2.x. Follow the steps in the technical documentation to activate it:

Reference Guide: [Activate NSX on DVPG] (Activate NSX on Distributed Virtual Port Groups)

 

Workaround 2: Configure a VLAN-Backed Segment

Create a trunked VLAN backed  segment and connect that segment to Edge TEP interface.

Steps: 

1. Create a VLAN Transport Zone (if you don't already have one).

2. Create a new NSX VLAN Segment within that VLAN transport zone.

3. Configure the Segment for Trunking: In the segment's settings, specify a VLAN ID range (e.g., 0-4094) instead of a single VLAN ID.

4. Create an Edge Uplink Profile and IP Pool: Configure these to use the shared TEP VLAN ID and subnet.

5. Configure the Edge Node: When deploying or reconfiguring your Edge Node, map the TEP interface to the newly created trunk segment.

Note: If using a VLAN segment on an NSX prepared VDS, adding only a single VLAN to the segment will result in the segment processing traffic for that VLAN only and stripping the tag for ingress traffic and adding the tag for egress traffic. This will result in the edge TEPs being unable to communicate as they require a trunked segment. This can be achieved by applying a range to the VLAN value (inclusive of your required VLAN) .

 

Workaround 3: Use Different VLAN for Edge TEP Traffic
Place the TEP traffic of the Edge Node on a different VLAN than the Host's TEP VLAN.

Steps:

1. Create a new uplink profile with a new VLAN
2. If an IP pool is used, create a new IP pool that allocates IP addresses in the newly created VLAN
3. Apply this profile to the Edge Node

Note: This workaround requires a separate subnet/VLAN.

 

Workaround 4: Create New NVDS/VDS for ESXi Host
Create a new NVDS/VDS for the ESXi Host and assign a different uplink (vmnic) to the NVDS/VDS. Apply the same uplink profile (including VLAN) and IP pool to the new uplink.

Note: This workaround requires additional vmnic(s) for the new NVDS/VDS.

 

For NSX 4.x and 3.x

 

Workaround 1: Configure a VLAN-Backed Segment

Create a trunked VLAN backed  segment and connect that segment to Edge TEP interface.

Steps: 

1. Create a VLAN Transport Zone (if you don't already have one).

2. Create a new NSX VLAN Segment within that VLAN transport zone.

3. Configure the Segment for Trunking: In the segment's settings, specify a VLAN ID range (e.g., 0-4094) instead of a single VLAN ID.

4. Create an Edge Uplink Profile and IP Pool: Configure these to use the shared TEP VLAN ID and subnet.

5. Configure the Edge Node: When deploying or reconfiguring your Edge Node, map the TEP interface to the newly created trunk segment.

Note: If using a VLAN segment on an NSX prepared VDS, adding only a single VLAN to the segment will result in the segment processing traffic for that VLAN only and stripping the tag for ingress traffic and adding the tag for egress traffic. This will result in the edge TEPs being unable to communicate as they require a trunked segment. This can be achieved by applying a range to the VLAN value (inclusive of your required VLAN) .

 

Workaround 2: Use Different VLAN for Edge TEP Traffic
Place the TEP traffic of the Edge Node on a different VLAN than the Host's TEP VLAN.

Steps:

1. Create a new uplink profile with a new VLAN
2. If an IP pool is used, create a new IP pool that allocates IP addresses in the newly created VLAN
3. Apply this profile to the Edge Node

Note: This workaround requires a separate subnet/VLAN.

 

Workaround 3: Create New NVDS/VDS for ESXi Host
Create a new NVDS/VDS for the ESXi Host and assign a different uplink (vmnic) to the NVDS/VDS. Apply the same uplink profile (including VLAN) and IP pool to the new uplink.

Note: This workaround requires additional vmnic(s) for the new NVDS/VDS.

Additional Information

For detailed information about this feature NSX on DVPGs, please refer to the following technical doc links:

Activate NSX on Distributed Virtual Port Groups (DVPGs)

Activate NSX on Distributed Virtual Port Groups

 

Powered by Wolken Software