BGP session diagnostics for troubleshooting BGP session flaps on NSX-T edge node

Products

• VMware NSX
• VMware NSX-T Data Center

Issue/Introduction

This article explains the procedure to enable BGP session diagnostics for a BGP neighbor on a NSX-T Edge node. The BGP session diagnostic information is useful to troubleshoot issues like BGP session flaps on edge node.

BGP session diagnostics information for a BGP neighbor includes below details which will be helpful for troubleshooting BGP session flaps:

1. Current state of the peer connection.
2. BGP session last reset reason.
3. Timestamps of the last 5 keep-alives received on the peer connection.
4. Timestamps of the last 5 keep-alives sent on the peer connection.
5. Details of the last 8 events that was observed on the peer connection.
6. Details of the last 4 session flaps seen on the peer connection.

NOTE:

• In VMware NSX 4.1.2 release, by default the BGP session diagnostics collection is disabled on all the configured BGP neighbors.
• Edge CLI commands are available to enable/disable the session diagnostics per BGP neighbor.
• User need to enable the BGP session diagnostics for troubleshooting on the required BGP neighbor where the BGP session flaps are being observed.
• Once the BGP session diagnostics is enabled for a BGP neighbor, it shall remain enabled unless explicitly disabled by CLI or edge node/FRR restarts. On an event of edge node reboot or service FRR restart, the BGP session diagnostics need to be re-enabled on the BGP neighbor, as it is not persistent across edge/FRR restarts.
• The command to fetch the BGP session diagnostics is also included in the edge support bundle for offline debugging.

Environment

VMware NSX 4.x
VMware NSX-T Data Center 3.x

Resolution

The BGP session diagnostics, needs to be enabled/disabled from the respective Tier0 Service router in the edge node CLI.
Use the below commands in the edge node CLI as admin user:

Edge> get logical-routers
Edge> vrf <vrf_id of SERVICE_ROUTER_TIER0>

To enable BGP session diagnostics for a BGP neighbor on the edge node:

Edge(tier0_sr)> set debug
Edge(tier0_sr)> set routing debug session-diagnostics neighbor <neighbor-ip>
Edge(tier0_sr)> exit

To disable BGP session diagnostics for a BGP neighbor on the edge node:

Edge(tier0_sr)> set debug
Edge(tier0_sr)> del routing debug session-diagnostics neighbor <neighbor-ip>
Edge(tier0_sr)> exit

To get the BGP session diagnostics information for a specific BGP neighbor on the edge node:

Edge> get logical-routers
Edge> vrf <vrf_id of SERVICE_ROUTER_TIER0>
Edge(tier0_sr)> get bgp neighbor <neighbor-ip> session-diagnostics

To get the BGP session diagnostics information for all the neighbors on the Tier0 Service router:

Edge(tier0_sr)> get bgp neighbor session-diagnostics