Route learned from eBGP neighbor is always preferred over same route learnt from inter-SR iBGP, even with high local preference on inter-SR iBGP route

Products

• VMware NSX

Issue/Introduction

• Two edge nodes, edge1 and edge2. edge1 has eBGP peering with ISP1 and edge2 has eBGP peering with ISP2. They also have inter-SR enabled so there is iBGP peering as well.
• Both ISP1 and ISP2 are advertising same prefix. Edge2 has two routes in BGP table, one from ISP2 and other from Edge1 through inter-SR iBGP. The route learned from inter-SR iBGP has high local preference however it is installing route from eBGP neighbor in routing table.

Environment

VMware NSX-T Data Center 3.x
VMware NSX 4.x

Cause

When inter-SR iBGP is enabled, weight with value 32765 gets added to eBGP routes. So eBGP route is always preferred.

edge1 is selecting route from eBGP neighbor 10.9#.#.# based on best local preference value.

root@edge01:~# vtysh -p

edge1# show ip bgp ipv4 10.#.#.#/16
BGP routing table entry for 10.#.#.#/16
Paths: (2 available, best #1, table default)
  Advertised to non peer-group peers:
  172.1#.#.#
  6#### 6####
    10.9#.#.# from 10.9#.#.# (10.9#.#.#)
      Origin IGP, localpref 90, valid, external, best (Local Pref)
      Last update: Tue Aug  6 18:00:42 2024

# edge2 is selecting route from eBGP neighbor 10.2#.#.# based on AS path value.

edge2#  show ip bgp ipv4 10.#.#.#/16
BGP routing table entry for 10.#.#.#/16
Paths: (2 available, best #1, table default)
  Advertised to non peer-group peers:
  172.#.#.#
  2#### 6####
    10.2#.#.# from 10.2#.#.# (2##.##.##.##)
      Origin IGP, localpref 60, valid, external, best (AS Path)
      Last update: Tue Aug  6 18:00:47 2024

Since Inter-SR is enabled, these edges also have iBGP running between them.

edge2# show ip bgp vrf all summary 

Instance default:

IPv4 Unicast Summary:
BGP router identifier 172.1#.#.#, local AS number ##### vrf-id 0
BGP table version 644
RIB entries 306, using 57 KiB of memory
Peers 2, using 43 KiB of memory

Neighbor        V         AS   MsgRcvd   MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcdPfxSnt
10.2#.#.#      4       2764      3768      3682        0    0    0 2d13h18m           13           8
172.#.#.#     4     134555      7774      7533        0    0    0 2d13h18m          146           0

Total number of neighbors 2

Instance inter_sr_vrf:

IPv4 Unicast Summary:
BGP router identifier 172.1#.#.#, local AS number ##### vrf-id 2
BGP table version 794
RIB entries 313, using 59 KiB of memory
Peers 1, using 22 KiB of memory

Neighbor        V         AS   MsgRcvd   MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcdPfxSnt
169.254.0.130   4 #####   2095161   2095219        0    0    0 6d14h10m          157         158 <--

Total number of neighbors 1

edge2 has following routes for 10.#.#.#/16. The ones from eBGP neighbor have weight 32765 set but the one coming from iBGP neighbor have weight set to 0. Since weight takes precedence, eBGP route is installed in routing table.

edge2# show ip bgp vrf all

<output snipped>

Instance inter_sr_vrf:
BGP table version is 794, local router ID is 172.1#.#.#, vrf id 2
Default local pref 100, local AS #####
Status codes:  s suppressed, d damped, h history, * valid, > best, = multipath,
               i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes:  i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path

*> 10.#.#.#/16      10.2#.#.#@0                   60  32765 #### #### i <--
* i                 169.254.0.130                  90      0 #### #### i 

<output snipped>

edge2# show ip route vrf all
Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
       F - PBR, f - OpenFabric,
       > - selected route, * - FIB route

B>* 0.0.0.0/0 [20/0] via 172.#.#.#, uplink-275, 2d13h25m
B>* 10.#.#.#/16 [20/0] via 10.2#.#.#, uplink-285, 2d13h24m <--

Resolution

This is expected behavior. A route learned by an Edge node from a northbound router will always be preferred to the same route learned over inter-SR iBGP. It is not possible to change this preference.

Additional Information

Configure BGP