[VMC] "Static route update failed" error on Tier-0 for overlapping Direct Connect and Transit Connect (vTGW) routes

Products

• VMware Cloud on AWS

Issue/Introduction

When configuring network redundancy or advertising new routes from On-Prem to VMware Cloud on AWS SDDC, learned routes for specific subnets (often larger CIDR blocks like /24 or /16) fail to be programmed into the NSX Edge Tier-0 routing table.

The following symptoms are observed during this issue:

• N-S Network connectivity is disrupted for Workload VMs associated with the affected subnets.
• Traceroute from the affected workload VM shows packets getting dropped at the Direct Connect or Transit Gateway (TGW) interface.
• Other, smaller (e.g. /25) or non-overlapping CIDR blocks may propagate successfully while the larger/affected CIDR remains in a `Failed` state.
• In the SDDC's NSX Manager UI -> Cloud Services -> Direct Connect or Transit Connect -> Learned Routes tab,  the specific Network CIDR appears in a "Failed" status with the error message: `Static route update failed`.
  
  

 

Environment

VMware Cloud on AWS

Cause

This issue is caused by a routing conflict triggered when identical prefixes are advertised simultaneously from both Transit Gateway (TGW) and Direct Connect (DX).
The routing engine encounters an inconsistency during route programming, causing the NSX Edge T0 router to reject the update for that specific prefix.

Resolution

To resolve this conflict, you must ensure that routing advertisements are unique and non-overlapping across the TGW and DX paths.

1.  Identify Duplicate Routes: Review your BGP route advertisements on both the on-premises Router and the AWS TGW.
2.  Remove Overlaps: If a prefix is being advertised from both TGW and DX simultaneously, remove the advertisement from one of the two paths so that the route is learned via a single preferred source only.
3.  Once duplicates are removed, the NSX Manager should automatically retry the route programming. 
4.  Monitor the SDDC NSX Manager UI to confirm the route state changes to `SUCCEEDED`: Navigate to the NSX Manager UI -> Cloud Services -> Direct Connect or Transit Connect  -> Learned Routes tab and verify that the Network is now present in the Learned Routes.

Note: If a redundant route is required via both TGW and DX, as a work-around you could advertise a more specific route via your preferred path (e.g. /25 via DX) and a less specific route via the backup path (e.g. /24 via TGW).  

If you require urgent assistance with mitigation or additional clarifications on this issue, please raise a Technical case with Broadcom Support for the Product VMware Cloud on AWS (Get Support) with the following details:

• SDDC ID
• Detailed description of the issue, including the exact requirements and failed CIDR(s).
• Business Impact

Additional Information

VMC Engineering is working on a permanent fix for the issue in a future release. Currently there is no ETA for the fix availability.