Enhanced Datapath (EDP) Flow Table Usage High Alarm in VMware NSX

Products

• VMware NSX

Issue/Introduction

An alarm is triggered in the NSX Manager UI:

• An alarm is triggered in the NSX Manager UI: There is at least one Enhanced Datapath flow table on the Host [UUID] in which more than 90% of entries are filled.
• Event ID: esx_health.enhanced_dp_flow_table_usage_high
• Observed performance degradation or increased latency for traffic on specific transport nodes.

Environment

VMware NSX

VMware Cloud Foundation

Cause

This alarm triggers when the number of active network flows exceeds 90% of the configured capacity of the Enhanced Datapath flow table. This occurs in two primary scenarios:

1. Stable High Volume: The workload maintains a high number of concurrent active connections that exceed the default table size (typically 131,072).
2. Short-Lived Connections: A high rate of dynamic, short-lived connections fills the table faster than entries can be aged out.

Note: This article is for high-usage (90-95% occupancy). For very-high-usage (above 95% occupancy), please refer to KB Article 95769 .

Resolution

For Enhanced Datapath (EDP) host switch mode, consider increasing the flow table size by invoking the following command, if performance degradation is observed.

 

Get the flow table size. 

nsxdp-cli ens flow-table size get

Ex-

Increasing the flow table size.

nsxdp-cli ens flow-table size set -s $NUM_ENTRIES

• $NUM_ENTRIES must be a power of 2
  
  • 2¹⁷ = 131072
  • 2¹⁸ = 262144
  • 2¹⁹ = 524288
• The host must be rebooted.

Ex -

Increasing the number of flow entries doesn't always improve performance, if short lived connections keep coming in. The flow table might be always full regardless of the flow table size. A large flow-table size wouldn't help in this case. EDP has a logic to detect this and automatically enable and disable flow tables to handle such a case.
Increasing the number of flow entries may increase the memory footprint.

Note: The above settings will be persistent across reboots in 4.2 and onward - Increasing the flow table size on ESXi node does not persist after a reboot

Additional Information

API Guide: https://developer.broadcom.com/xapis/nsx-t-data-center-rest-api/4.1.0/

Admin Guide: https://techdocs.broadcom.com/us/en/vmware-cis/nsx/vmware-nsx/4-1/administration-guide.html