NSX Enhanced Datapath (EDP) Flow Table Usage High Alarm

Products

• VMware NSX

Issue/Introduction

An alarm is triggered in the NSX Manager UI:

• An alarm is triggered in the NSX Manager UI: There is at least one Enhanced Datapath flow table on the Host [UUID] in which more than 90% of entries are filled.
• Event ID: esx_health.enhanced_dp_flow_table_usage_high
• Observed performance degradation or increased latency for traffic on specific transport nodes.

Environment

VMware NSX

Cause

This alarm triggers when the number of active network flows exceeds 90% of the configured capacity of the Enhanced Datapath flow table. This occurs in two primary scenarios:

1. Stable High Volume: The workload maintains a high number of concurrent active connections that exceed the default table size (typically 131,072).
2. Short-Lived Connections: A high rate of dynamic, short-lived connections fills the table faster than entries can be aged out.

Note: This article is for high-usage (90-95% occupancy). For very-high-usage (above 95% occupancy), please refer to KB Article 95769 .

Resolution

For Enhanced Datapath (EDP) host switch mode, consider increasing the flow table size by invoking the following command, if performance degradation is observed.

 

Get the flow table size. 

nsxdp-cli ens flow-table size get

Ex-

Increasing the flow table size.

nsxdp-cli ens flow-table size set -s $NUM_ENTRIES

• $NUM_ENTRIES must be a power of 2
  
  • 2¹⁷ = 131072
  • 2¹⁸ = 262144
  • 2¹⁹ = 524288
• The host must be rebooted.

Ex -

Increasing the number of flow entries doesn't always improve performance, if short lived connections keep coming in. The flow table might be always full regardless of the flow table size. A large flow-table size wouldn't help in this case. EDP has a logic to detect this and automatically enable and disable flow tables to handle such a case.
Increasing the number of flow entries may increase the memory footprint.

Note: The above settings will be persistent across reboots in 4.2 and onward - Increasing the flow table size on ESXi node does not persist after a reboot

Additional Information

API Guide: https://developer.broadcom.com/xapis/nsx-t-data-center-rest-api/4.1.0/ 
Admin Guide: https://techdocs.broadcom.com/us/en/vmware-cis/nsx/vmware-nsx/4-1/administration-guide.html