Traffic blackholed on standby Tier-0 Gateway node when using ECMP on physical Leaf switches (NSX)
Article ID: 447793
Updated On:
Products
Issue/Introduction
Users may experience intermittent connectivity issues and packet loss for traffic ingressing tier-0
When testing connectivity, traceroutes show intermittent failures depending on which uplink path the physical Leaf switch selects. Traffic that successfully routes through the Active Tier-0 node reaches its destination, but traffic arriving at the Standby node of an Active/Standby Tier-0 Gateway is dropped (blackholed), typically stopping at the Transit Gateway.
Environment
Cause
In an Active/Standby Tier-0 configuration, only the Active Edge node is programmed to process data plane traffic. The Standby node maintains BGP peering sessions and routing tables to ensure rapid failover, but its data plane (HA ports) is intentionally kept down.
To discourage physical routers from sending traffic to the Standby node, NSX automatically uses AS-Path prepending to make the Standby route artificially longer. However, if the physical Leaf switches are configured to ignore AS-Path length (e.g., using a command like bestpath as-path ignore) and use ECMP (maximum-paths) to load-balance traffic across all uplinks, they will mistakenly hash roughly 50% of the traffic to the Standby node's uplinks. Because the Standby node does not forward transit traffic by design, these packets are blackholed.
Resolution
To resolve this issue, you must either align the physical network configuration with the NSX Active/Standby design or change the NSX HA architecture to support ECMP.
Option 1: Modify Physical Network Configuration (Required for Stateful Services) If your Tier-0 Gateway utilizes stateful services (such as NAT, Gateway Firewalls, or VPNs), it must remain in Active/Standby mode.
Access the upstream physical Leaf switches.
Remove any BGP configurations that ignore AS-Path length (e.g., remove the bestpath as-path ignore command).
Disable ECMP for the paths toward the Tier-0 Gateway, allowing the switches to respect the AS-Path Prepending (or MED) sent by NSX. This ensures the physical network correctly prefers the Active Edge node as the single forwarding path.
Option 2: Convert to Active/Active Mode (For Stateless Routing Only)
If multi-pathing and simultaneous forwarding across all Edge nodes are required to maximize bandwidth, you can convert the Tier-0 Gateway to Active/Active mode.
To configure this:
Log in to the NSX Manager UI.
Navigate to Networking > Tier-0 Gateways.
Edit the specific Tier-0 Gateway.
Change the HA Mode from Active-Standby to Active-Active.
Note: Active/Active mode is designed exclusively for stateless routing. Using stateful services (such as NAT or Gateway Firewall) on a Tier-0 VRF in Active/Active mode is not supported, as it breaks traffic symmetry.
If you choose to proceed with this option, please carefully review and follow the guidelines outlined in this Broadcom Knowledge Base article 429539 (NSX Tier-0 Gateway HA Mode Change Fails with Logical Interface Error)
Additional Information
To speak with a customer representative or a Support Engineer, see . Scroll to the bottom of the page and click on your respective region.
Feedback
