Bridging overlay segment to overlay segment between two different NSX Instances is failing

Products

• VMware NSX

Issue/Introduction

• Two separate NSX environments connected to the same vCenter Server with Multi-NSX enabled

• Attempting to bridge from overlay segment to overlay segment between the two different NSX environments

• Migrating workloads from one NSX environment to another via the above mentioned bridge

• The destination NSX environment's Edge Bridge VM was manually deployed on the source NSX instance's host transport node and the bridge interface is plugged into the source NSX instance's overlay segment

• Packet captures show ICMP packets make it to the NSX Edge Bridge VM, but the ARP packets were not getting encapsulated as they leave the NSX Edge VM

• Using pktcap-uw --switchport <switchportID_of_interface_connected_to_overlay> --trace on the ESXi host where the NSX Edge Node VM lives, the drop reason shows 'Mac Forgery Drop'. For Example:

Environment

VMware NSX

Cause

The bridge interface connected to the source side overlay segment needs to have promiscuous mode, mac change and forged transmits enabled for the overlay to overlay bridging to work.

Resolution

Resolution steps are as follows:

• nsxdp-cli vswitch l2sec set --dvport <dvportID> --dvs-alias <dvs-name> --promisc --mac-change --forge-src

• To check the current settings per port ID:

  • nsxdp-cli vswitch l2sec get -dvs <dvs-name>

• To obtain the information needed for the above commands:

  • nsxdp-cli vswitch instance list

NOTE: This is a non persistent change, if the VM reboots or is vMotioned the settings will revert back to their defaults.

Additional Information

Configuring L2 port security settings on an NSX backed portgroup