Huge Packet Loss on Active/Standby Tier-0 with NAT Enabled

Products

• VMware NSX

Issue/Introduction

• Packet loss towards the North for the apps with SNAT enabled.
• No issues seen for the traffic without SNAT at Tier-0.
• ECMP is enabled between Tier-0 & TOR.
• BFD was enabled on top of BGP towards TOR & is marked as down in the "get bgp neighbour summary" output.
• The issue is seen starting with GC releases (3.0.x) only.

Environment

VMware NSX Datacenter.

Cause

As BFD is down, BFD marks the next hop as unusable, due to which traffic for the same flow is sent across two different uplinks.
Due to this, SNAT fails.

Resolution

Workaround:

A) Resolve the BFD issue and ensure BFD is UP
or
B) Disable BFD on the BGP peers and then disable/re-enable ECMP
or
C) Disable ECMP

Resolution:

Fixed in 3.1.2 and later.

Additional Information

Impact:

North-South traffic is impacted.