NSX Edge BGP flaps and connection resets due to continuous "cannot clear L2 track on null cookie" memory panic

Products

• VMware NSX
• VMware Telco Cloud Platform

Issue/Introduction

Network instability, BGP flapping, and routing drop-offs to Border Leafs are observed on the affected NSX Edge node. Application workloads experience repeated connection disconnects even after BGP sessions appear to establish.

A review of the Edge /var/log/syslog shows a continuous stream of the following error on the datapath firewall thread:

[ERROR] cannot clear L2 track on null cookie

This error loop prevents the datapath from successfully forwarding traffic. The datapath process (datapathd) does not crash or experience a segmentation fault, meaning no core dump is generated by the host despite the traffic disruption.

Environment

VMware NSX-T Data Center 3.2.2

Cause

A memory leak within the datapath firewall thread causes a memory panic related to Layer 2 tracking cookies. Once the thread enters this corrupted state, it is unable to clear the null cookies and fails to process further state updates or forward traffic, leading to the BGP connection resets.

Resolution

Resolved in NSX 4.1.x and higher.

Additional Information

Workaround:

To clear the corrupted memory state and restore normal traffic forwarding on affected versions, the datapath service must be reinitialized:

1. Place the affected NSX Edge node into Maintenance Mode to gracefully halt the datapath service and drain traffic.

2. Exit Maintenance Mode to force a clean restart of the datapath service, allowing the Layer 2 tracking tables to allocate cleanly.

3. Verify that the BGP sessions stabilize and the continuous null cookie errors have ceased in the syslog.