High CPU Utilization Causes BGP Hang Leading to Edge Node System Failure and Routing Outage

Products

• VMware NSX

Issue/Introduction

An NSX Edge Node experiences a severe performance degradation or enters a hung state, resulting in a loss of network routing capabilities across certain segments.

During the outage, the following symptoms and logs are observed:

• The datapathd process consumes excessive CPU resources (often exceeding 200%), which severely degrades fast-path packet processing.

• The bgpd process exhibits high CPU usage (often exceeding 190%), impacting the stability of network routing.

• Management agents are unable to communicate with the datapath, generating the following warning in the syslog:
  
  failed to connect to /var/run/vmware/edge/dpd.ctl

Environment

VMware NSX-T Data Center

Cause

This issue stems from high Datapath CPU consumption, which creates a processing bottleneck. When the Datapath is overloaded, the system cannot process flow-cache entries or manage communication tasks in a timely manner.

This condition is typically driven by two primary factors:

High Flow Creation Rate: A continuous, rapid influx of new network connections forces the initial packet of each individual flow into the "slow path" so the system can set up the cache. This process is significantly more resource-intensive than standard fast-path forwarding.

Underlying Host Resource Contention: Physical CPU resource constraints, ready time, or performance spikes on the underlying ESXi host where the NSX Edge VM is hosted directly restrict the appliance's processing capacity.

Resolution

See KB NSX Edge Performance Issues and High CPU on Datapath and BGP for next steps.