BFD packets from external application doesn't reach DPDK enabled vnic interfaces of VM once after vmnic comes up after external switch redundancy test

Products

• VMware NSX

Issue/Introduction

After external switch redundancy tests connected to ESXi hosts, when the switch port comes UP,  the BFD packets are seen reaching the switchport of the VMs but are unable are not seen reaching the ethernet interface of VM by the application. The vnics of VMs are DPDK enabled 

The uplink profile on NSX is configured with two vmnics with Failover order or Load balance source teaming policy.

Environment

3.2.0.1

Cause

• When the vmnicX is brought down from physical switch end, all the MAC entries on the physical switch port connecting to the vmnicX are cleared. Traffic fails over to another vmnicY.
• Now, when vmnicX is brought back up, all the traffic to workloads move back to vmnicX. This occurs since failback is enabled in the teaming policy.
• Vswitch attempts to send RARP packets for all the learned MACs out of vmnicX. However, the RARP packets are sent untagged (VLAN 0) which are dropped since the VNIC port is on trunk VLAN.
• Traffic then recovers once the MAC entries are aged out on the physical switch or the MAC moves to the correct switchport on the physical switch.

Resolution

This scenario will not occur if fallback disabled in uplink profile. The option to enable/disable failback has been added from 4.2.0.

Additional Information

Additionally the issue will not be seen if a single uplink or LACP is configured to ensure traffic doesn't break with static MACs on trunk VLAN.