[VMC] Routes are not being learned or advertised over Direct Connect or VPN

Products

• VMware Cloud on AWS

Issue/Introduction

• Routes are no longer being learned or advertised over Direct Connect (DX) or VPN. 
• When new routes are added to DX, they are showing up red (failed) in the DX interface.
• Existing routes in the DX interface still show as blue (successful).
• The VPN status is intermittently showing as DOWN after new routes are added. 
• The VPN tunnel status is UP, but the BGP status is DOWN. 

• VPN error similar to below may be seen when additional routes are added:
  BGP: %MAXPFXEXCEED: No. of IPv4 Unicast prefix received from <VPN_Endpoint_IP> exceed, limit 100

Environment

VMware Cloud on AWS

Cause

• There is a 100 route hard limit when using AWS Site-to-Site VPN connections. 
• This includes DX virtual interfaces and VPN connections configured in VMC on AWS. 
• This limit applies to both dynamic and static routes. 

Resolution

• This is an AWS limitation and the hard limit cannot be adjusted. 
• The recommendation is to either filter the routes being advertised or to implement route summarization. 
  • If using a Route-Based VPN, updates to route-filtering or route summarization will be configured at the external endpoint.
  • If using a Policy-Based VPN, route summarization can be configured at the external endpoint or the VMC SDDC endpoint. Route-filtering will need to be configured at the external endpoint. 

Additional Information

• VMware Configuration Maximums: VMware Cloud on AWS
• AWS Site-to-Site VPN quotas