Out Of Memory error causing edge VPN tunnel down

Products

• VMware NSX

Issue/Introduction

Problem Definition: VPN sessions on edge are going to Down status with reason "Out of memory".

Symptoms:

• Edge is "out of memory" even though memory is available.

• From edge syslog:

  2024-01-12T12:14:01.402Z nsx.edge.valdiated NSX 14255 VPN [nsx@6876 comp="nsx-edge" subcomp="iked" s2comp="ike-stack" level="WARN"] The maximum number of active Phase-1 SAs reached

  2024-01-12T12:14:01.402Z nsx.edge.valdiated NSX 14255 VPN [nsx@6876 comp="nsx-edge" subcomp="iked" s2comp="ike-stack" level="ERROR" errorCode="EDG1000028"]   Message: Out of memory (65537)

Environment

VMware NSX-T Data Center 3.x

Cause

Known issue with Phase 1 IKE SA memory is getting allocated but not released.

Resolution

Issue is fixed in following versions:

VMware NSX 4.2.1, 3.2.5 (Telco) & VCF 9.0

 

Workaround: 
- If generic firewall any any accept rule is configured, change the same to any any deny. And add specific rules for traffic which needs to be allowed. This will help with dropping malicious packets to edge.
- If changing firewall rules is not possible, only way is to perform failover for all the SRs having IPsec VPN service on this particular edge.

Additional Information

Impact to customer: Datapath traffic stops working for local and remote workloads over VPN tunnels.