BGP goes down due to bgpd crash

Products

• VMware NSX Networking
• VMware NSX-T Data Center

Issue/Introduction

1.  BGPD crash and generate a core file.
2. The crash was triggered when "show bgp vrf all neighbors json" was run by agg-service.

     Relevant logs to look to understand the what are the events triggered during the crash.:

•  /var/log/syslog,

    2024-06-25T13:34  nsxedge NSX 29556 - [nsx@6876 comp="nsx-edge" subcomp="node-mgmt" username="root" level="WARNING"] Core file generated: /var/log/core/core.bgpd.1719322490.28580.160.6.gz

• /var/log/frr/frr.log

    [Thread debugging using libthread_db enabled]
    Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
    Core was generated by `/usr/lib/frr/bgpd -d -A 127.0.0.1'.
    Program terminated with signal SIGABRT, Aborted.

 

Environment

NSX-T 3.1.3.0 Highline

Cause

BGP goes down due to bgpd crash, which might impact the traffic disruption.

• From the support bundle and core bt, it is observed that crash is happening due to the memory corruption. As, we can see the crash is happening at different places.
• The cli `show ip bgp vrf all neighbors json` is getting executed by agg-service in every interval of 1 minute.
• While iterating over `show ip bgp vrf all neighbors json` cli in the code, it is found that the json variable is being double freed. This is the reason for the memory corruption.

Resolution

This issue is resolved in VMware NSX-T 3.2.3, available at Broadcom Downloads.
If you are having difficulty finding and downloading software, please review the Download Broadcom products and software KB.