Unable to establish BGP session using HSRP virtual IP

Products

• VMware NSX

Issue/Introduction

BGP neighbor adjacency is stuck in active state and has never been established before.

Cause

Using HSRP virtual IP to establish BGP sessions causes the ESXi host to look for the virtual MAC address and not the interface MAC address for the neighbor. 

Within a packet capture of the BGP TCP handshake, the Edge sends out the TCP SYN packet to the virtual MAC address 00:00:0c:07:##:##, but the physical switch responds with a TCP reset packet with physical/interface MAC address. In the below logs, you will find that the highlighted mac address for the BGP neighbor are different which hints at virtual IP address being used for BGP neighbor.

 

Packet capture logs on the edge node over SR gateway uplink interface:

 00:50:56:##:##:## > 00:00:0c:07:ac:##, ethertype IPv4 (0x0800), length ##: <src_bgp_IP>.### > <bgp_neighbor_IP>.179: Flags [S], seq <seq_num>, win ##, options [mss #,sack#], length 0
 ##:##:##:##:##:## > 00:50:56:##:##:##, ethertype IPv4 (0x0800), length ##: <bgp_neighbor_IP>.179 > <src_bgp_IP>.###: Flags [R.], seq 0, ack <seq_num +1> win 0, length 0

 

Above logs illustrates that the NSX edge VM sends TCP SYN packet to the BGP neighbor i.e virtual IP for HSRP(00:00:0c:07:ac:## is the virtual MAC used for HSRP v1 protocol) on upstream physical switch and there is no TCP ACK received from HSRP virtual MAC used on the upstream switch. 

BGP neighborship cannot be formed with virtual IP address as TCP connections need to be initiated from a switchport interface with routable IP address, not a virtual one. The TCP packets for BGP peering is allowed to source only from a switchport interface IP, not from a virtual IP address. As a result, you never get an acknowledgement for the TCP SYN to complete the TCP handshake and establish a TCP connection.

The second log message is a TCP RST packet from the upstream/peer(HSRP active router's interface MAC address instead of the HSRP virtual MAC address). This confirms upstream switch terminating the BGP session since it's not able to establish one using virtual HSRP IP address.

 

Resolution

Change the "Set BGP neighbors" in the UI to the active router's interface IP address instead of the HSRP virtual IP. 

Additional Information

Here is a Cisco BGP Peering with HSRP Virtual Address community link discussing BGP session not able to establish with HSRP virtual IP.