NSX-T Edge tunnels are down to ESXi hosts when sharing the same VLAN for TEP traffic
search cancel

NSX-T Edge tunnels are down to ESXi hosts when sharing the same VLAN for TEP traffic

book

Article ID: 312645

calendar_today

Updated On:

Products

VMware NSX

Issue/Introduction

The following tunnel connectivity issues are observed and Hosts and Edge Nodes are configured to use the same VLAN for TEP traffic:

  • Tunnels between Hosts and Edge Nodes are down.
  • Tunnels between Hosts are up.
  • Tunnels between Edge Nodes are up.

Environment

  • VMware NSX 4.x
  • VMware NSX-T Datacenter 3.x
  • VCF 9.x

Cause

Tunnels between the edge node and prepped ESXi host will be down if the edge node's TEP interface uses a vSphere-created distributed virtual port group (DVPG) and shares the same TEP VLAN as the host.

Resolution

Workaround:

Multiple workarounds are available for this issue, listed in order of preference based on NSX version.

For VCF 9.x

Pre-Condition: Edge TEP is connected to standard distributed Port Groups

Workaround 1: Enable NSX on DVPG (Recommended)

If NSX on DVPG is disabled, follow the steps in the technical documentation to activate it:

Reference Guide: (Activate NSX on Distributed Virtual Port Groups (DVPGs))

Note on exclusion lists: If the affected VDS is used exclusively by NSX (for example, all VMs are on a different VDS using VLAN-backed distributed port groups), an exclusion list is not required to activate NSX on DVPG. Before proceeding, verify this by reviewing the transport node profile to confirm which VDS is in scope and whether any non-NSX workloads share that VDS.

Workaround 2: Configure a VLAN-Backed Segment

Create a trunked VLAN backed segment and connect that segment to the Edge TEP interface.

Steps:

  1. Create a VLAN Transport Zone (if you don't already have one).
  2. Create a new NSX VLAN Segment within that VLAN transport zone.
  3. Configure the Segment for Trunking: In the segment's settings, specify a VLAN ID range (e.g., 0-4094) instead of a single VLAN ID.
  4. Create an Edge Uplink Profile and IP Pool: Configure these to use the shared TEP VLAN ID and subnet.
  5. Configure the Edge Node: When deploying or reconfiguring your Edge Node, map the TEP interface to the newly created Trunk Segment.

Note: If using a VLAN segment on an NSX prepared VDS, adding only a single VLAN to the segment will result in the segment processing traffic for that VLAN only and stripping the tag for ingress traffic and adding the tag for egress traffic. This will result in the edge TEPs being unable to communicate as they require a trunked segment. This can be achieved by applying a range to the VLAN value (inclusive of your required VLAN).

Note on VCF-controlled environments: In VCF-controlled environments, this workaround cannot be applied as a simple in-place configuration change. The TEP VLAN is held in the edge configuration as deployed by VCF, so moving the Edge TEP from a DVPG to a trunked VLAN-backed segment requires redeploying the edges through VCF. If a redeploy is not feasible, use Workaround 1 instead.

Workaround 3: Use Different VLAN for Edge TEP Traffic

Place the TEP traffic of the Edge Node on a different VLAN than the Host's TEP VLAN.

Steps:

  1. Create a new uplink profile with a new VLAN
  2. If an IP pool is used, create a new IP pool that allocates IP addresses in the newly created VLAN
  3. Apply this profile to the Edge Node

Note: This workaround requires a separate subnet/VLAN.

Workaround 4: Create New NVDS/VDS for ESXi Host

Create a new NVDS/VDS for the ESXi Host and assign a different uplink (vmnic) to the NVDS/VDS. Apply the same uplink profile (including VLAN) and IP pool to the new uplink.

Note: This workaround requires additional vmnic(s) for the new NVDS/VDS.

For NSX 4.2.x

Pre-Condition: Edge TEP is connected to standard distributed Port Groups

Workaround 1: Configure a VLAN-Backed Segment

Create a trunked VLAN backed segment and connect that segment to the Edge TEP interface.

Steps:

  1. Create a VLAN Transport Zone (if you don't already have one).
  2. Create a new NSX VLAN Segment within that VLAN transport zone.
  3. Configure the Segment for Trunking: In the segment's settings, specify a VLAN ID range (e.g., 0-4094) instead of a single VLAN ID.
  4. Create an Edge Uplink Profile and IP Pool: Configure these to use the shared TEP VLAN ID and subnet.
  5. Configure the Edge Node: When deploying or reconfiguring your Edge Node, map the TEP interface to the newly created trunk segment.

Note: If using a VLAN segment on an NSX prepared VDS, adding only a single VLAN to the segment will result in the segment processing traffic for that VLAN only and stripping the tag for ingress traffic and adding the tag for egress traffic. This will result in the edge TEPs being unable to communicate as they require a trunked segment. This can be achieved by applying a range to the VLAN value (inclusive of your required VLAN).

Workaround 2: Use Different VLAN for Edge TEP Traffic

Place the TEP traffic of the Edge Node on a different VLAN than the Host's TEP VLAN.

Steps:

  1. Create a new uplink profile with a new VLAN
  2. If an IP pool is used, create a new IP pool that allocates IP addresses in the newly created VLAN
  3. Apply this profile to the Edge Node

Note: This workaround requires a separate subnet/VLAN.

Workaround 3: Create New NVDS/VDS for ESXi Host

Create a new NVDS/VDS for the ESXi Host and assign a different uplink (vmnic) to the NVDS/VDS. Apply the same uplink profile (including VLAN) and IP pool to the new uplink.

Note: This workaround requires additional vmnic(s) for the new NVDS/VDS.

For NSX 4.x and 3.x

Workaround 1: Configure a VLAN-Backed Segment

Create a trunked VLAN backed segment and connect that segment to the Edge TEP interface.

Steps:

  1. Create a VLAN Transport Zone (if you don't already have one).
  2. Create a new NSX VLAN Segment within that VLAN transport zone.
  3. Configure the Segment for Trunking: In the segment's settings, specify a VLAN ID range (e.g., 0-4094) instead of a single VLAN ID.
  4. Create an Edge Uplink Profile and IP Pool: Configure these to use the shared TEP VLAN ID and subnet.
  5. Configure the Edge Node: When deploying or reconfiguring your Edge Node, map the TEP interface to the newly created trunk segment.

Note: If using a VLAN segment on an NSX prepared VDS, adding only a single VLAN to the segment will result in the segment processing traffic for that VLAN only and stripping the tag for ingress traffic and adding the tag for egress traffic. This will result in the edge TEPs being unable to communicate as they require a trunked segment. This can be achieved by applying a range to the VLAN value (inclusive of your required VLAN).

Workaround 2: Use Different VLAN for Edge TEP Traffic

Place the TEP traffic of the Edge Node on a different VLAN than the Host's TEP VLAN.

Steps:

  1. Create a new uplink profile with a new VLAN
  2. If an IP pool is used, create a new IP pool that allocates IP addresses in the newly created VLAN
  3. Apply this profile to the Edge Node

Note: This workaround requires a separate subnet/VLAN.

Workaround 3: Create New NVDS/VDS for ESXi Host

Create a new NVDS/VDS for the ESXi Host and assign a different uplink (vmnic) to the NVDS/VDS. Apply the same uplink profile (including VLAN) and IP pool to the new uplink.

Note: This workaround requires additional vmnic(s) for the new NVDS/VDS.

Additional Information

For detailed information about this feature NSX on DVPGs, please refer to the following technical doc links:

Activate NSX on Distributed Virtual Port Groups (DVPGs)

Activate NSX on Distributed Virtual Port Groups

 

Powered by Wolken Software