Alarm for very high Enhanced Datapath flow table usage in NSX.

Products

• VMware NSX

Issue/Introduction

Title: Alarm for very high Enhanced Datapath flow table usage.
Event ID: esx_health.enhanced_dp_flow_table_usage_very_high
Alarm Description

• Purpose: There is at least one flow-table on the Host in which more than 95% of entries are filled.
• Impact: Slower traffic may be observed due to flow miss as there is high chance new flow entry creation could fail.

Note: This article is for both very-high-usage (above 95% occupancy) and high-usage (90-95% occupancy).

Environment

VMware NSX-T Data Center

Cause

If the number of active flows exceeds the flow-table size, or if there are many short-lived connections coming in, the flow tables can fill up. Increasing the flow table size is beneficial when the number of active flows is stable over time. However, this might not improve performance in cases where short-lived connections continuously come in. In such scenarios, the flow table might always be full, regardless of its size.

Resolution

For Enhanced Datapath (EDP) host switch mode, for versions 4.2.1 and above, if performance degradation is observed consider increasing the flow table size by invoking the following command:

# nsxdp-cli ens flow-table size set -s $NUM_ENTRIES

Host must be rebooted.

Note:

$NUM_ENTRIES must be a power of 2 to a max value of 524288
The value set is persistent across reboots from NSX version 4.2.1
Increasing the number of flow entries doesn't always improve performance, if short lived connections keep coming in. The flow table might be always full regardless of the flow table size. A large flow-table size wouldn't help in this case. EDP has a logic to detect this and automatically enable and disable flow tables to handle such a case.
Increasing the number of flow entries may increase the memory footprint.

Additional Information

Flow tables being full does not indicate any impact, just that performance improvements from EDP are reduced. For this reason, this alarm will be a lower warning level in future releases of NSX. 

API Guide
Admin Guide
Increasing the flow table size on ESXi node does not persist after a reboot