NSX-V L2 Bridging stops working when MAC learning enabled workloads are present in the bridge node and multiple uplinks are used in non-LACP based teaming

Products

• VMware NSX Networking

Issue/Introduction

Symptoms:
Communication between VLAN and VXLAN does not work if a VM residing on bridge node has mac learning enabled and there are multiple uplinks connected to the host switch in non-LACP based teaming.

Environment

VMware NSX Data Center for vSphere 6.4.x

Cause

Issue would happen if there are VM's with mac learning enabled on the bridge node (both mac learning enabled VM and DLR VM resides on same host)

When MAC learning is enabled on any dvport of the host switch, MACs are learned from the uplinks that are connected to the host switch too. If multiple uplinks are used without LACP, broadcast packets sent out of one uplink could reach the other uplink as the physical switch is unaware of the teaming inside the ESX host. Such packets are filtered out correctly if sender of the pkt used:

1) its assigned / fixed vNIC MAC as the source MAC (or)
2) a forged MAC with MAC learning enabled on the dvport(or)
3) a forged MAC with forged transmit + promiscuous setting on the dvport and /Net/ReversePathFwdCheckPromisc advanced option enabled.


Since VDR port L2 bridging solution uses forged transmit + sink port configuration, the loopback packets are not getting filtered and MAC learning ends up learning the overlay MAC behind VLAN incorrectly. This causes response from VLAN workloads to not reach the bridge port and hence not the overlay side too.

Resolution

This is a known behaviour and currently there is no resolution.

Workaround:
1) Move away the workloads with mac-learning enabled from the bridge node.
2) Either use LACP when multiple uplinks are used, or use a single uplink.