BGP Flapping on new environment

Products

• VMware NSX

Issue/Introduction

NSX BGP peering never establishes

• Observe the BGP SYN ACK handshake doesn't receive the ACK however ARP requests receive ARP replies within Edge packet capture on the T0 interface.  

  • #######-en01> start capture interface ########-####-####-####-######## direction dual expression host 10.0.2.1
19:57:34.134333 ##:##:##:##:##:## > ##:##:##:##:##:##, ethertype 802. 1Q (0x8100), length 78: vlan 0, p 0, ethertype IPv4 (0x0800), 10.0.0.11.4477 > 10.0.0.1.179: Flags [S], seq 624952279, win 62720, options [mss 8960, sackOK, TS val 2919977501 ecr 0, nop, wscale 8], length 0
<bame64>xKt.NDn3/AFRWnBungQAAAAgARcAAPFINQAAKBmmGCi7QCwou0AGuywCzJUAD1wAAAACgAvUAqToAAATETwAFAggKrgtHQAAAAARAwMI</bane64>

    19:57:35.011413 ##:##:##:##:##:## > ##:##:##:##:##:##, ethertype 802.1Q (0x0100), length 78: vlan 0, p 0, ethertype IPv4 (0x0800), 10.0.0.11.45003 > 10.0.2.1.179: Flags [s], seq 1037790102, win 62720, options [mss 8960, sackOK, TS val 2919980293 ecr 0, nop, wscale 0], length 0
<base64>xKtNDn3/AFBWnBungQAAAAgARcAAPHXrQAAKBkWoCi7QCwou0AGy6wCzPduK1gAAAACgAvUA+ecAAAIEIwAEAggKrgtdBQAAAAABAwMI</base64>

    19:57:36.023043 ##:##:##:##:##:## > ##:##:##:##:##:##, ethertype 802.1Q (0x8100), length 78: vlan 0, p 0, ethertype IPv4 (0x0800), 10.0.0.11.45803 > 10.0.0.1.179: Flags [S], seq 1037798102, win 62720, options [mss 8960, sackOK, TS val 2919981305 ecr 0, nop, wacale 8], length 0
<base64>xKLNDn3/AFBWnBungQAAAAqARcAAPHXsQAAKBkWnC17QCwou0AGy6wCzPduK1qAAAACqAvUA9[MAAAIEIwAEAggKrgtg+QAAAAABAwMI</base64>

    19:57:38.039751 ##:##:##:##:##:## > ##:##:##:##:##:##, ethertype 802.1Q (0x8100), length 78: vlan 0, p 0, ethertype IPv4 (0x0800), 10.0.0.11.45803 > 10.0.0.1.179: Flags [S], seq 1037798102, win 62720, options [mss 8960, sackOK, TS val 2919983321 ecr 0, nop, wscale 8], length 0
<bane64>xKtNDn3/AFRWnBungQAAAAgARcAAPHXtQAAKBkWmCi7QCwou0AGy6wCzPduK1gAAAACgAvUA7hMAAATETwAEAggKrgto2QAAAAABAwMI</base64>

    19:57:42.200317 ##:##:##:##:##:## > ##:##:##:##:##:##, ethertype 802.1Q (0x0100), length 78: vlan 0, p 0, ethertype IPv4 (0x0800), 10.0.2.11.45003 > 10.0.2.1.179: Flags [s], seq 1037790102, win 62720, options [mss 8960, sackOK, TS val 2919987481 ecr 0, nop, wscale 0], longth 0
<base64>xKtNDn3/AFBWnBungQAAAAgARcAAPHXuQAAKBkW1Ci7QCwou0AGy6wCzPduK1gAAAACgAvUA3dMAAAIEIwAEAgg-rgt5GQAAAAABAWMI</base64>

    19:57:43.221640 ##:##:##:##:##:## > ##:##:##:##:##:##, ethertype ARP (0x0806), length 64: vlan 0, p 0, ethertype ARP (0x0806), Request who-has 10.0.0.1 tell 10.0.0.11, length 46
<base64>xKtNDn3/AFBWnBungQAAAAgGAAEIAAYEAAEAUFacG6cKLtALAAAAAAAAC1 7QAQAAAAAAAAAAAAAAAAAAAAAAAA ==< /base64>

    19:57:43.222693 ##:##:##:##:##:## > ##:##:##:##:##:##, ethertype ARP (0x0806), length 60: vlan 0, p 0, ethertype ARP (0x0806), Reply 10.0.0.1 is-at ##:##:##:##:##:##, length 46
base64>AFBWnBunxKtNDn3/CAYAAQgABgQAA=SrTQ59/wou0AEAUFacG6cKLtALAAAAAAAAAAAAAAAAAAAbyk7S</baae64>

Environment

NSX All version

Cause

The physical BGP peer Has never been established 

• Neighbor    AS       State Up/DownTime    BFD   InMsgs   OutMsgs  InPfx    OutPfx

10.0.0.1    65508    Conne never          NC    0        0        0        0
10.0.0.3    65512    Eatab 04w1d20h       NC    123456   321      1234     98
10.0.1.2    65508    Conne never          NC    0        0        0        0
10.0.1.4    65512    Eatab 04w1d20h       NC    123456   321      1234     98
• This is indicative of the physical device peer (10.0.0.1 and  10.0.1.2) not being configured correctly.

 

Resolution

Validate BGP configuration on the physical infrastructure

Additional Information

Review for BGP  troubleshooting:

• BGP session diagnostics for troubleshooting BGP session flaps on NSX-T edge node