Statically configured T1 routes are not realized on the dataplane forwarding tables

Products

• VMware NSX

Issue/Introduction

• Static routes are configured on the T1, but are not seen in the forwarding table, when using the nsxcli command 'get forwarding' within the T1 router context.
• Redistribution of the static routes is enabled, but they are not seen in the routing table of the T0 router.
• In the NSX Manager proton logs in /var/log/proton/nsxapi.log you will see the configured route was deleted:

2025-01-22T11:11:41.369Z  INFO workerTaskExecutor-1 UserStaticRoutesCCPPublisher 30334 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Deleted existing ccp route RouteNextHopConfigUFOProxy{networkId=<NETWORK UUID>, networkConfig=<NETWORK PREFIX\SUBNET TO BE ROUTED - X.X.X.X/X>, routerId=LogicalRouterConfig/<ROUTER UUID>, routeType=STATIC, description='null', logicalRouterPortId=null, peerLifId=null, nextHopIp=<NEXT HOP IP ADDRESS>, administrativeDistance=1, blackhole=false, blackholeAction=ROUTE_BLACKHOLE_ACTION_INVALID, vrfName='null', disabled=false, originEntityId=StaticRoute/<UUID>, lrResourceId=LogicalRouter/<UUID>, identifier=RouteNextHopConfig/<UUID>}

Environment

• VMware NSX

Cause

• There are two components that need to have values specified when creating a static route. The first part is the 'Network' to be routed, which requires a subnet to be specified and the second part is a 'Next Hop' IP address. The flow that leads up to the issue is the following:

1. A static route is configured with a valid 'Network' and 'Next Hop' IP address.
2. Another static route is erroneously configured were the 'Network' component is configured with the 'Next Hop' IP address used in step 1, and another subnet is configured as the 'Next Hop' IP for this new route.
3. Now the route from step 1 is pointing to the route created in step 2 via its 'Next Hop' IP address. The 'Next Hop' IP address from step 1 cannot be resolved, as that Next Hop network now points to the 'Next Hop' IP address configured in the second route, which is not reachable.
4. Due to this the NSX Manager is not able to resolve the direction of the route, so it deletes all routes which point to the 'Next Hop' IP address from step 1. Please note this is expected behaviour.

• In the In the NSX Manager proton logs in the /var/log/proton/nsxapi.log you will see logs like the following, which indicate that the CCP is unable to determine the direction of the route, and are reported as 'UNKNOWN':

2025-01-22T11:11:41.368Z  INFO workerTaskExecutor-1 RouteGraphManager 30334 ROUTING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Affected networks from route graph are : [{ Network : <NETWORK PREFIX\SUBNET TO BE ROUTED - X.X.X.X/X>, Direction : UNKNOWN, nhToRealExitNextHopMap : { { nhIp: <NEXT HOP IP ADDRESS>,static route id : StaticRoute/<UUID>, adminDistance: 1, isDeleted: false } : [], } }, { Network : <NETWORK PREFIX\SUBNET TO BE ROUTED - X.X.X.X/X>, Direction : UNKNOWN, nhToRealExitNextHopMap : { { nhIp: <NEXT HOP IP ADDRESS>,static route id : StaticRoute/<UUID>, adminDistance: 1, isDeleted: false } : [], } }], for lr LogicalRouter/<ROUTER UUID>

• Attempts to correct the issue by deleting the route created in step 2 or by correcting its values does not lead to the routes being realized in the T1 forwarding tables.

Resolution

• This is a known issue in NSX, and it will be fixed in a future release.
• The workaround is to first delete the configured static routes, and then run the following NSX 4.2.1 API Guide. Once the API has been run successfully reconfigure the static routes as intended, and they will be realized in the forwarding table:

POST /policy/api/v1/infra/tier-1s/<T1 ID>?action=reprocess

Some details on this API are the following:

• The Manager internally creates a graph of all routes, that then uses a algorithm to see what route is reachable and from where.
• The Manager maintains this graph in a memory cache.
• That graph has gone into a bad state due to the incorrectly added route.
• When the reprocess API is called, the route graph is cleared and recreated back in a state prior to the bad state.