NSX-T IDPS generates Critical alarm: NSX-IDPS engine is down

Products

• VMware vDefend Firewall

Issue/Introduction

• Running IDPS in Detect Only or Detect and Detect and Prevent.
• In the NSX manager, Critical alarms are generated:

• Core dumps are observed on the transport node where alarm was generated for:

/var/core/nsx-idps-zdump.000

• And seen being generated in log /var/run/log/vobd.log

1360:2023-04-06T14:11:04.293Z: [UserWorldCorrelator] 6397661389us: [vob.uw.core.dumped] /usr/lib/vmware/nsx-idps/bin/nsx-idps(2100913) /var/core/nsx-idps-zdump.000

1361:2023-04-06T14:11:04.293Z: [UserWorldCorrelator] 6397695566us: [esx.problem.application.core.dumped] An application (/usr/lib/vmware/nsx-idps/bin/nsx-idps) running on ESXi host has crashed (1 time(s) so far). A core file may have been created at /var/core/nsx-idps-zdump.000.

• Also in the ESXi host /var/run/log/vmkernel.log:

30520:2023-04-06T14:10:56.061Z cpu26:2102190)UserDump: 2635: W#03: Dumping cartel 2100913 (from world 2102190) to file /var/core/nsx-idps-zdump.000 ...

• On the impacted host, if you enter the NSX cli and run the command 'get ids engine stats' , you see the traffic types: dns, ftp, http, smb, smtp, snmp.

  6  app_layer:                                                                     
  7  ---------                                                                      
  8            flow:                                                                
  9                dcerpc_tcp: 2285                                                 
 10                   dns_udp: 720                                                  
 11                failed_tcp: 346                                                  
 12                failed_udp: 369                                                  
 13                       ftp: 19                                                   
 14                      http: 144                                                  
 15                      krb5: 1516                                                 
 16                       smb: 1914                                                 
 17                      smtp: 1                                                    
 18                      snmp: 8995                                                 
 19                       tls: 4039                                                 
 20              tx:                                                                
 21                   dns_udp: 1458                                                 
 22                       ftp: 186                                                  
 23                      http: 162                                                  
 24                       smb: 12039                                                
 25                      smtp: 4                                                    
 26                      snmp: 17999  

Environment

VMware NSX-T Data Center/VMware NSX 4.2.1 or below

Cause

IDPS was attempting to track the complete sessions for these protocols, this can lead to an out of memory condition and crash of the service.

Resolution

Resolution:
Upgrade to NSX 4.2.2 and then enable high-performance Turbo mode (SCRX) for Distributed IDS/IPS.

For Turbo mode (SCRX) information and installation instructions, see - https://techdocs.broadcom.com/us/en/vmware-security-load-balancing/vdefend/vdefend-atp/4-2/nsx-ids-ips-and-nsx-malware-prevention/ddpi-engine/ddpi-engine-prerequisites.html

For Turbo mode (SCRX) compatibility pre-check script see - https://knowledge.broadcom.com/external/article?articleNumber=396277

Workaround:
Reduce the amount of traffic the IDPS service deals with, in relation to these protocols, by using applied to and limiting source and destination in the Security - IDS/IPS & Malware Prevention - Distributed Rules, should help alleviate the load on the service.

If you are unable to apply this workaround, contact Broadcom Support and refer to this KB article.