Load balancer status is failing with LB engine query
edge01.your.domain> get load-balancer b7c148e4-###-##########2327d status
Tue Oct 29 2024 UTC 08:02:47.484
24304: Internal Error: Query LB Engine Failed.
Lb-service is not running.
edge01.your.domain> get load-balancer b7c148e4-###-##########2327d health-check-table
Tue Oct 29 2024 UTC 08:02:58.924
% An unexpected error occurred: LB service is not running currently
Load balancer service is unavailable and generates frequent nginx core dumps.
edge01.your.domain> get load-balancer b7c148e4-###-##########2327d diagnosis
Checking
Action : checking system
Result : passed
Action : checking crash
Result : core file found
core.nginx.1730188987.278549.134.11.gz is generated.
core.nginx.1730188987.278531.134.11.gz is generated.
Action : checking daemon status
Result : passed
Action : checking configuration
Result : not_found_in_db: LbCurrentMsg
Segmentation fault errors are found in Edge's /var/log/kern.log:
kernel.log:
2024-10-29T19:22:00.557Z edge01.your.domain.com kernel - - - [48535.297668] nginx[1667454]: segfault at 0 ip 000074c26e0e4bc2 sp 00007880f4ec9910 error 4 in libprotobuf-c.so.1.0.0[74c26e0e4000+6000]
2024-10-29T19:22:00.558Z edge01.your.domain.com kernel - - - [48535.962629] nginx[1667475]: segfault at 0 ip 000074c26e0e4bc2 sp 00007880f4ec9910 error 4 in libprotobuf-c.so.1.0.0[74c26e0e4000+6000]
nsxcli.log shows the rule-log is enabled
cli.descriptors.cli_command_service INFO CMD: set load-balancer b7c148e4-###-##########2327d rule-log virtual-server 988c4c91-#########-cde43fadd196 enabled
cli.audit INFO CMD: set load-balancer b7c148e4-###-##########2327d rule-log virtual-server 988c4c91-#########-cde43fadd196 enable, Operation status: CMD_EXECUTED
VMware NSX
VMware NSX-T Data Center
If a rule-log is enabled for the LB virtual server and it is not disabled, before the virtual server or the pool in this LB service is detach/deleted.
And once the docker service for lb restarts( or edge reboots) we may see the nginx crashes happening frequently due to the LB rule log debug command is in place.
Note: Edge CLI command "set load-balancer <uuid> rule-log ..."
is only used for internal debug. This rule-log has nothing to do with debug level logging with load-balancer and virtual server logging.
Please make sure that rule-log debug is disabled before the virtual server or the pool in this LB service is detach/deleted.
If you encounter the above issue with please contact to Broadcom support.