On NSX-T 2.4.0/2.4.1 Edge Load Balancer service status is UNKNOWN

Products

• VMware NSX Networking

Issue/Introduction

Symptoms:

• NSX-T 2.4.0 or 2.4.1
•  Edge has high memory usage
•  The load balancer process has generated core dumps in /var/log/core e.g.

  core.nginx.1559278043.gz

•  Edge syslog messages indicate a failure to allocate memory

2019-07-01T02:57:37.108029+00:00 hostname NSX 1681 LB [nsx@6876 comp="nsx-edge" subcomp="nsx-edge-lb.lb_log" level="FATAL"] [dece7c45-df1a-4ab8-835c-7a678d16b0d5] [emerg] 1681#0: malloc(44800000) failed (12: Cannot allocate memory)

•  There is a high rate of configuration change on the LB.  This can be seen by monitoring the counter lbs_generation_id which increments with each configuration change

 #grep lbs_generation_id /config/vmware/edge/lb/etc/<LB_UUID>/nginx.conf

Environment

VMware NSX-T Data Center
VMware NSX-T Data Center 2.x

Cause

The Load Balancer server experiences this problem due to a memory leak triggered by a high rate of configuration changes.

Resolution

This issue is resolved in VMware NSX-T Data Center 2.4.2, available at VMware Downloads.

Workaround:
Once memory has been exhausted on the Edge VM, it has to be rebooted to workaround the issue.

If a 3rd party ingress controller is configured in the environment, it may be contributing to the high rate of configuration changes on the Edge LB. In this case please open a Support Request with VMware GSS to discuss workaround options.