EVPN Type-5 routes are not installed in the RIB and FIB after rebooting the TOR

Products

• VMware NSX

Issue/Introduction

• The TOR was rebooted following which, EVPN type-5 routes are not installed in the Edge routing table.
• The below logs are present on Edge node that show the VXLAN TEP tunnel going down as expected during TOR reboot:
  
  [timestamp]EDGE NSX 1 FABRIC [nsx@6876 comp="nsx-edge" subcomp="nsxa" s2comp="tunnel" level="INFO"] Tunnel 10.1.1.1:10.1.1.2(vxlan) state updated from up to down


• The route for the remote VXLAN TEP was re-learned on Edge post the TOR reboot.
  The same is present in the "get route" command on Edge  and is reachable:
  
  [timestamp] edge NSX 11181 FABRIC [nsx@6876 comp="nsx-edge" subcomp="datapathd" s2comp="dpc-pb" tname="dp-ipc19" level="INFO"] Update lrouter ########-####-####-####-######## FIB entry for 10.1.1.2/32


• However, the routes present in the "get bgp evpn" command output are not marked as best path (*>) and thus, not installed in the routing table.
• In the "/var/log/bgp_support_bundle.log" file of Edge support bundle, we can see the next hop (VXLAN TEP) is marked as invalid:
  
  

  >>[timestamp]show bgp vrf all nexthop
10.1.1.2 invalid, #paths 9
  Last update: 

Environment

VMware NSX 4.2.x

VMware NSX 9.x

Cause

This is a known issue in route selection logic of BGP on NSX Edge.

Resolution

The issue is fixed in 9.1.1 version of NSX, and upcoming 4.2.x release of NSX.

Workaround:

• Reset BGP session
• Put edge into Maintenance mode.