New connections via the NSX Load Balancer cannot be established because the 'attached to memzone' event for the specific generation ID has not been processed.

Products

• VMware NSX

Issue/Introduction

Reviewing /var/log/syslog on the Edge node, we observed that the attached to memzone event is missing for a specific generation ID following the initialized memzone event.

Normally, the attached to memzone event should occur immediately after the initialized memzone event.

<DATE_TIME> <HOSTNAME> NSX #### LOAD-BALANCER [nsx@#### comp="nsx-edge" subcomp="lb" s2comp="lb" level="INFO"] [########-####-####-####-############] initialized memzone /lb_cp_########-####-####-####-############:1 [base@0x############] [size:#########] <DATE_TIME> <HOSTNAME> NSX # LOAD-BALANCER [nsx@#### comp="nsx-edge" subcomp="lb-dispatcher" s2comp="lb-disp" level="INFO"] attached to memzone /lb_cp_########-####-####-####-############:1 [base@0x############] [size:#]   <DATE_TIME> <HOSTNAME> NSX #### LOAD-BALANCER [nsx@#### comp="nsx-edge" subcomp="lb" s2comp="lb" level="INFO"] [########-####-####-####-############] initialized memzone /lb_cp_########-####-####-####-############:2 [base@0x############] [size:#########] Note: There is no 'attached to memzone' event logged for generation ID 2.

 

Environment

VMware NSX 4.2.3

Cause

This issue is triggered when the lbconf process is killed due to an Out-Of-Memory (OOM) condition. When the lbconf process restarts, it resets and begins using the generation ID from 0 again. However, the lb-dispatcher process continues to refer to the old generation ID from its cache.

The OOM kill event for the lbconf process can be verified in /var/log/kern.log:

<DATE_TIME> <HOSTNAME> kernel - - - [########.######] Out of memory: Killed process ####### (nginx) total-vm:#######kB, anon-rss:#######kB, file-rss:#kB, shmem-rss:##kB, UID:### pgtables:####kB oom_score_adj:#

Resolution

As a workaround, please place the Edge node into maintenance mode to free up the memory.

Broadcom Engineering is aware of this issue, and a fix will be provided in a future release.