Alarm for high or very high Enhanced Datapath flow table usage in NSX.

Products

• VMware NSX

Issue/Introduction

• An alarm is showing in the UI that states either "Alarm for high EDP flow table usage" or "Alarm for very high EDP flow table usage".

 

Environment

• VMware NSX 4.2.x

Cause

If the number of active flows exceeds the flow-table size, or if there are many short-lived connections coming in, the flow tables can fill up.

The alarm did not exist prior to version 4.2.0, so the table usage may still have been high and would not have been reported prior to 4.2.0.

Resolution

For Enhanced Datapath (EDP) host switch mode, for versions 4.2.1 and above, if performance degradation is observed consider increasing the flow table size by invoking the following command:

# nsxdp-cli ens flow-table size set -s $NUM_ENTRIES

Host must be rebooted.

Note:

$NUM_ENTRIES must be a power of 2 to a max value of 524288
The value set is persistent across reboots from NSX version 4.2.1
Increasing the number of flow entries doesn't always improve performance, if short lived connections keep coming in. The flow table might be always full regardless of the flow table size. A large flow-table size wouldn't help in this case. EDP has a logic to detect this and automatically enable and disable flow tables to handle such a case.
Increasing the number of flow entries may increase the memory footprint.

Additional Information

Flow tables being full does not indicate any impact, just that performance improvements from EDP are reduced. For this reason, this alarm will be a lower warning level in future releases of NSX. 

API Guide
Admin Guide
Increasing the flow table size on ESXi node does not persist after a reboot

Increasing the flow table size is beneficial when the number of active flows is stable over time. However, this might not improve performance in cases where short-lived connections continuously come in. In such scenarios, the flow table might always be full, regardless of its size.