When IDPS is enabled, traffic experiences latency and packet drops

Products

• VMware NSX

Issue/Introduction

Symptoms:

• NSX-T Data Center is installed
• IDS/IPS enabled even in detection-only mode
• IDS/IPS is enabled on an src:Any dst:Any prot:Any rule
• Traffic experiences latency and packet drops
• On ESXi host, /var/run/log/vobd.log indicate the idps service is crashing

46385:2022-02-10T02:49:59.947Z: [UserWorldCorrelator] 5054803445447us: [vob.uw.core.dumped] /usr/lib/vmware/nsx-idps/bin/nsx-idps(31502338) /var/core/nsx-idps-zdump.000
46386:2022-02-10T03:00:38.398Z: [UserWorldCorrelator] 5055441840903us: [vob.uw.core.dumped] /usr/lib/vmware/nsx-idps/bin/nsx-idps(31513632) /var/core/nsx-idps-zdump.001

Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.

Environment

VMware NSX-T Data Center

Cause

When a src:Any dst:Any prot:Any rule is created for IDPS, it results in all traffic going through the IDPS service for inspection.
In a busy environment, it is possible this may exhaust the memory pool available to the IDPS service running on the ESXi host.
As a result the service will crash and restart. When this happens traffic is dropped.

Resolution

This issue is resolved in NSX-T Data Center 3.2.2, see Download Broadcom products and software

This remains a known issue on NSX 4.0.0.1 and 4.0.1.1.

Workaround:
Remove the Any Any rule and replace it with specific rules targeted to traffic flows to be observed.

Additional Information

Note: If SMB traffic is being inspected by IDPS rules see Traffic disruption when NSX-T IDPS inspects SMB traffic flows