Service Engine Crash due to SE Agent out of memory
book
Article ID: 413358
calendar_today
Updated On:
Products
VMware Avi Load Balancer
Issue/Introduction
There could be varied symptoms of the issue
Avi Service Engine Failed event noticed for Service engine.
SE crash file may get generated. The stacktrace points to crash at ipstk_dns_tcp_read_client_so function.
There can be events related to connection drops due to SE being low on resources such as CONN_DROP_NO_CONN_MEM, CONN_THROTTLED_MEMFAIL_FLOW_TBL
Cause
Detection of an DoS attack may cause the Service Engine to experience memory leakage for the SE_agent process.
We can identify the same by looking at the footprint of SE_agent memory utilization along with "num_dos_msgs_sent" counter in the output of the command : "show serviceengine <se-name> metrics debug". An increment in this counter points towards the system being prone to hitting this issue.
Below is a sample screenshot for reference from Service Engine tech-support logs to identify the issue:
Another way to identify the issue is by looking at DOS related logs in SE under /var/lib/avi/log/se_trace* . High value for such logs can point towards SE hitting this issue.
zgrep "DOS" se_trace.* | wc -l
500281
Resolution
The issue is fixed in the latest 22.1.x, 30.2.x releases
Additional Information
Below Techdoc also documents the bug under AV-185506: