L2VPN connectivity failure between NSX VLAN segment (Server-side) and Overlay Segment (Client-side).

Products

• VMware NSX

Issue/Introduction

• L2VPN has been configured between 2 NSX managed sites.
• The server side is configured with a VLAN segment with the gateway on the physical network while the client side is configured with overlay NSX segment.
• Overlay-to-Overlay connectivity across the L2VPN tunnel functions as expected.
• However, VLAN-to-Overlay connectivity is failing.
• Validated the tunnel configuration status and the tunnel was reporting its status as UP. 
  • get l2vpn sessions config
  • get l2vpn sessions <session_UUID_from_above> logical-switch
  • 
  • get tunnel-port <tunnel-port_UUID_from_above> stats
  • 
• Based on above validation on the active Edge node, this indicates a healthy tunnel state with no incrementing "No-match" or "L2-loop" drop counters.
• Hence, as per scenario B of the article: Troubleshooting NSX L2 VPN, performed packet-captures at the switch-port, tunnel-port to validate the traffic is passing through the VPN or not.
• Packet captures executed at the switch-port layer of the VPN confirmed ARP requests and replies from the client site were received by the source Edge node.
  • 
• Packet captures executed at the Edge Node fastpath interface confirmed ARP replies are processed and transmitted outside of the Edge VM.
  • 
• However, these outgoing ARP replies fail to leave the ESXi host hosting the active Edge node. 
• Performing packet-captures and trace on this ESXi host highlighted, that the traffic is dropped at the vSphere Distributed Switch (vDS) port group level.
  • 
• The Drop reason for these ARP replies was reported "MAC Forgery Drop"

Environment

VMware NSX

Cause

• The ESXi host was actively dropping outgoing ARP replies originating from the Edge node due to a "MAC Forgery Drop" condition.
• The underlying vDS port group backing the Edge node uplink has the "Forged Transmits" security policy explicitly set to "Reject". 
• In this specific L2VPN configuration, the NSX Edge node must transmit packets using the source MAC addresses belonging to the remote client VMs. 
• Because the source MAC address of the outgoing payload does not match the effective MAC address registered to the Edge VM's adapter, the ESXi vSwitch security module specifically "L2Sec_FilterSrcMACForgeries" identifies the transmission as MAC spoofing and drops the packet.

Resolution

Modify the vDS port group security policy backing the Edge uplinks to set "Forged Transmits" to "Accept". This explicitly allows the port group to pass outgoing frames containing MAC addresses that do not match the assigned vNIC MAC, resolving the MAC Forgery Drop.

Workaround

• If due to security compliance, enabling "Forged Transmits" on standard vDS port groups is not possible, please follow the below workaround:
• Disconnect the Edge VM fastpath interfaces from the standard vDS port groups.
• Connect the Edge VM fastpath interfaces to VLAN-backed NSX segment.
  
  • Please ensure the target NSX VLAN segment is configured with MAC Learning enabled under "MAC Discovery" segment security profile of the segment.
  • Also, ensure the required Trunk VLANs are allowed on the segment.

Attaching the Edge uplinks to VLAN segment configured with MAC Learning allows the traffic from dynamically learned MAC addresses from L2VPN to pass through.

Additional Information

For more information on the creation of the required VLAN segment with the required MAC Discovery profile, please refer: Create an NSX MAC Discovery Segment Profile

Please refer article: Troubleshooting NSX L2 VPN for more information on troubleshooting L2VPN issues.