Avi Service Engine crash with Segmentation fault at ngx_http_upstream_clear_ssl_state
search cancel

Avi Service Engine crash with Segmentation fault at ngx_http_upstream_clear_ssl_state

book

Article ID: 412474

calendar_today

Updated On:

Products

VMware Avi Load Balancer

Issue/Introduction

ServiceEngine may crash if a memory allocation failure happens during GSLB site persistence operation.

 

The crash stack trace will include the function: ngx_http_upstream_clear_ssl_state. (It should be present in initial #0 method calls)

Sample StackTrace:

 

 

To investigate further, you can review the latest stack traces from the Controller or SE by accessing the following path:

CLI:

Login to Controller via ssh and run this command.Please note you have to replace the name of se_dp file here.

root@<Controller ip>:#  cat /opt/avi/archive/stack_traces/<se_dp.timestamp>.stack_trace
 
UI:
Navigate to Administration > Support > Crash Reports > Expand the latest crash file.

Environment

Affects Version(s):

22.1.1 - 22.1.7

30.1.1 - 30.1.2

30.2.1 - 30.2.3

31.1.1

Cause

This is a new issue that was identified when GSLB Site persistence is enabled.

 

SE trace logs indicating the issue

C03 12:38:37.563017 [ngx_http_persist_deserialize_http_cookie:1558] Persistence Cookie expired.
C03 12:38:37.665633 [ngx_http_persist_deserialize_http_cookie:1558] Persistence Cookie expired.
C03 12:38:38.752720 [ngx_palloc:182] *98923628 error [0] - allocation of 8246779701625834555 bytes failed in pool 0x000055617b487130: allocation exceeds pool limit of 67108864. (pool size: 16272)
C03 12:38:38.752720 [ngx_http_upstream_connect:1608] *98923628 upstream create request failed
C03 12:38:38.752720 [ngx_http_free_request:4406] *98923628 Memory limit exceeded during the request processing.

Resolution

Please upgrade the system to the fix version.

AV-240094: Service Engine may fail if the memory allocation exceeds the limit during GSLB site persistence operation
Fix Version: 30.2.4
Workaround: No workaround available