NSX-T Load Balancer stop working.

Products

• VMware NSX Networking

Issue/Introduction

Symptoms:

• Traffic going through NSX-T Load Balancer stop working.
• In the NSX-T Edge logs you can see:

/var/log/syslog
<5>1 2020-06-04T07:48:29.299510+00:00 <Edge-TN-Name> kernel - - - [17530602.434999] grsec: From 10.10.10.27: Segmentation fault occurred at 0000000000000008 in /opt/vmware/nsx-edge/bin/nginx[nginx:30892] uid/euid:134/134 gid/egid:140/140, parent /opt/vmware/nsx-edge/bin/nginx[nginx:11821] uid/euid:134/134 gid/egid:140/140
<180>1 2020-06-04T07:48:29.358Z <Edge-TN-Name> NSX 3886 - [nsx@6876 comp="nsx-edge" subcomp="node-mgmt" username="root" level="WARNING"] Core file generated: /var/log/core/core.nginx.1599034158.30892.134.11.gz
<29>1 2020-06-04T07:48:29.842439+00:00 <Edge-TN-Name> NSX 11821 LB [nsx@6876 comp="nsx-edge" subcomp="nsx-edge-lb.lb" level="INFO"] [d38a6943-ffb4-45e1-8cbc-b4438bd78945] signal 17 (SIGCHLD) received from 30892
<25>1 2020-09-02T07:50:30.842782+00:00 <Edge-TN-Name> NSX 11821 LB [nsx@6876 comp="nsx-edge" subcomp="nsx-edge-lb.lb" level="FATAL"] [d38a6943-ffb4-45e1-8cbc-b4438bd78945] worker process 30892 exited on signal 11 (core dumped)

• One or more Load Balancer Core files are present in the Edge directory: /var/log/core

Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment

Environment

VMware NSX-T Data Center 2.5.x
VMware NSX-T Data Center

Resolution

This issue is resolved in NSX-T 2.5.2 and 3.0.2

Workaround:
As temporary workaround, put the Edge TN in Maintenance mode to failover the traffic to the non impact Edge in the cluster and reboot the impacted Edge TN.

Additional Information

Impact/Risks:
The NSX-T VIPs hosted by NSX-T Load Balancer will not respond to traffic anymore. Resulting in a service impact.