BGP session diagnostics for troubleshooting BGP session flaps on NSX-T edge node

Products

• VMware NSX
• VMware NSX-T Data Center

Issue/Introduction

This article explains the procedure to enable BGP session diagnostics for a BGP neighbor on a NSX-T Edge node. The BGP session diagnostic information is useful to troubleshoot issues like BGP session flaps on edge node and cases where BGP peering does not go into Established state (or "does not come up").

BGP session diagnostics information for a BGP neighbor includes below details which will be helpful for troubleshooting BGP session flaps:

1. Current state of the peer connection.
2. BGP session last reset reason.
3. Timestamps of the last 5 keep-alives received on the peer connection.
4. Timestamps of the last 5 keep-alives sent on the peer connection.
5. Details of the last 8 events that was observed on the peer connection.
6. Details of the last 4 session flaps seen on the peer connection.

NOTE:

• In VMware NSX 4.1.2 release, by default the BGP session diagnostics collection is disabled on all the configured BGP neighbors.
• Edge CLI commands are available to enable/disable the session diagnostics per BGP neighbor.
• User need to enable the BGP session diagnostics for troubleshooting on the required BGP neighbor where the BGP session flaps are being observed.
• Once the BGP session diagnostics is enabled for a BGP neighbor, it shall remain enabled unless explicitly disabled by CLI or edge node/FRR restarts. On an event of edge node reboot or service FRR restart, the BGP session diagnostics need to be re-enabled on the BGP neighbor, as it is not persistent across edge/FRR restarts.
• The command to fetch the BGP session diagnostics is also included in the edge support bundle for offline debugging.

Environment

VMware NSX-T Data Center
VMware NSX

Resolution

The BGP session diagnostics, needs to be enabled/disabled from the respective Tier0 Service router in the edge node CLI.
Use the below commands in the edge node CLI as admin user:

Edge> get logical-routers
Edge> vrf <vrf_id of SERVICE_ROUTER_TIER0>

To enable BGP session diagnostics for a BGP neighbor on the edge node:

Edge(tier0_sr)> set debug
Edge(tier0_sr)> set routing debug session-diagnostics neighbor <neighbor-ip>
Edge(tier0_sr)> exit

To disable BGP session diagnostics for a BGP neighbor on the edge node:

Edge(tier0_sr)> set debug
Edge(tier0_sr)> del routing debug session-diagnostics neighbor <neighbor-ip>
Edge(tier0_sr)> exit

To get the BGP session diagnostics information for a specific BGP neighbor on the edge node:

Edge> get logical-routers
Edge> vrf <vrf_id of SERVICE_ROUTER_TIER0>
Edge(tier0_sr)> get bgp neighbor <neighbor-ip> session-diagnostics

To get the BGP session diagnostics information for all the neighbors on the Tier0 Service router:

Edge(tier0_sr)> get bgp neighbor session-diagnostics.

 

To get the BGP logs to examine the establishment of BGP peering relationships:

Edge> get log-file routing