Connectivity issue on the workload VMs residing on the host where the Edge VM resides

Products

• VMware NSX

Issue/Introduction

Packet drops are observed when normal workload VMs are placed on an NSX prepped ESXi host where the Edge node is residing.

This issue is seen in the following configurations:

• When the Edge Uplink Profile does not have a Transport VLAN set, or the transport VLAN (TEP VLAN) is set to 0.

• The fastpath(fp-ethx) interface corresponding to the Edge VTEP is connected to a non-trunk Distributed Virtual Port Group (DVPG).

• Besides the Edge VTEP interface, the Edge management interface or VLAN HostSwitch(Edge NVDS) Interface is also connected to a non-trunk DVPG.

Environment

VMware Cloud Foundation 9.1.0.0

Cause

• When the Edge Uplink Profile does not have a Transport VLAN set (or is set to 0), the NestedTNConfig property is expected to be set only on the DVPG port corresponding to the VTEP interface.
• The packet drops occur because the NestedTNConfig property (com.vmware.port.extraConfig.vdl2.nestedTNConfig = version=1;vlan=0,label=<labelID>) is incorrectly set on non-VTEP Edge DVPG interfaces as well, such as the management interface and the fastpath(fp-ethx) interfaces corresponding to the VLAN Host Switch.

Resolution

This issue is planned to be resolved in upcoming NSX versions.

Please follow the workaround below:

1. Update the TEP VLAN in the Edge Overlay HostSwitch to a non-zero value:

• • Open Edge Node Uplink Mapping -> VIEW DETAILS for the impacted Edge node.
    
  • As shown below in highlighted part, enter the VTEP VLAN id which you want to use for Edge VTEP.
  • VTEP VLAN specified should be a non-zero value.

       2.  Attach the Edge VTEP vNIC to a "VLAN Trunk DVPG" that contains the VTEP VLAN in the Trunk range:

• • DVPG Configuration before Workaround will look something like below

• • If DVPG connected to edge VTEP interfaces is a non-trunk as mentioned above. Then we need to make this DVPG as "VLAN Trunk-DVPG" with below steps.  
  • Make DVPG as "VLAN Trunk-DVPG" , login to vCenter → Go to Networks → Select DVPG → Then configure VLAN trunk range for this DVPG. In VLAN trunk range just specify edge VTEP VLAN ID as mentioned below -

• • Connect this "VLAN Trunk-DVPG" to Edge VTEP Interface → 
  • After above configuration we need to attach this VLAN Trunk DVPG to edge VTEP interface i.e. "Uplink interfaces" from Overlay HostSwitch

      3.  After above 2 steps are performed, we need to disconnect and re-connect "Vlan Host Switch interfaces" and "Management interfaces"

• • In 1st two steps issue for VTEP fast path(fp-ethx) interface will be resolved
  • In this 3rd step, we are resolving issue for "Vlan Host Switch interfaces"(fp-ethx) and "Management interface".
    • To fix issue for "Vlan Host Switch interfaces"(fp-ethx) and "Management interface",
    • From VC UI, Disconnect and Re-connect DVPGs connected to "Vlan Host Switch interfaces"(fp-ethx) interfaces.
    • From VC UI, Disconnect and Re-connect DVPGs connected to "Management interface".

      After all 3 steps are performed, the issue should be resolved.

     Note: A maintenance window should be taken before applying this workaround.