SE Crash in ngx_http_remove_cookie due to Memory Allocation Failure

Products

• VMware Avi Load Balancer

Issue/Introduction

• The SE crash occurs when processing client requests that use HTTP cookie persistence during high client connection traffic towards the Virtual Service.

Environment

Affected Avi version: <= 30.2.6, <=31.1.2, <=31.2.2, 32.1.1

Cause

The crash is triggered by a sequence of events related to memory allocation failure within the ngx_http_remove_cookie function.

• A stack trace similar to below will be noticed 

  #2  ngx_http_remove_cookie (r=r@entry=0x556cffeda240, name=name@entry=0x7ffc86464f70)
  #3  0x0000556cd07d631e in ngx_get_http_ck_persist_data 
  #4  0x0000556cd0bb11a0 in se_lb_gen_get_server 
  #5  0x0000556cd07d77e3 in ngx_http_upstream_loadbalancer_pool_get_svr 
  :
  

• Analysis from Stacktrace confirms buffer allocation failed in wait request handler, which confirms OOM condition on the SE.
• Refer to the output of "show serviceengine <name of service engine> mallocstats" output from Controller shell - Notice the Fail Count for type SE_MTYPE_L7_REQUEST_MEMPOOL

Resolution

• There's no workaround for this issue currently, apart from rebooting the affected SE.
• Fix for this issue is targeted to be released in future version release of Avi LB software. 
  • Subscribe to this article to be updated on the fix status; see Subscribe to a Broadcom knowledge article by article or product