IDS Service not running issue in 1371

Products

• VMware vDefend Network Detection and Response

Issue/Introduction

Sensor gets into the issue of IDS service not running post the upgrade to 1371

Environment

On-prem environment, On the sensor run the below commands to see if timon utilisation would show abnormally high like above 80%

root@sensor# docker stats --no-stream | grep -i timon
47365acf4a37 timon_timon_1 120.63% 33.75GiB / 39.34GiB 85.79% 0B / 0B 1.51GB / 15.9MB 18


root@sensor# ps aux --sort -rss | head -n 5
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
suricata 30276 157 49.8 33605704 32535364 ? Ssl 07:24 32:07 /usr/bin/timon -c /etc/lastline/timon.yaml
monitor+ 21879 0.7 1.1 1690960 778912 ? S 06:17 0:38 /usr/bin/uwsgi-core --ini /usr/share/analyst_sdk_api/analyst_sdk_api_uwsgi.conf --uid root --gid root --processes 10 --threads 1 --max-requests 1000 --harakiri 90 --socket :3031 --http :9091
monitor+ 5241 5.3 0.6 996904 439740 ? S 07:37 0:25 /usr/bin/uwsgi-core --ini /usr/share/analyst_sdk_api/analyst_sdk_api_uwsgi.conf --uid root --gid root --processes 10 --threads 1 --max-requests 1000 --harakiri 90 --socket :3031 --http :9091
monitor+ 18681 0.1 0.5 786292 384964 ? S Apr26 7:12 /usr/bin/python /usr/bin/analyst_sdk_analysis_completion.py -c /etc/lastline/config.ini

In UI, the status of sensor you would see the 'IDS Service not running'.

Cause

As observed from the output of the commands, it's due to memory leak related to timon and lack of memory. 

The disable status of watchdog that ensures that the IDS service is not automatically restarted in case of crashes.

Resolution

echo "ZWNobyAiUGF0Y2hpbmcgdGltb24iCnNlZCAtaSAtZSAicy90aW1vbjo6aW1hZ2VfdmVyc2lvbjogJzQ2LTUwMDYyNTA4LmZvY2FsJy90aW1vbjo6aW1hZ2VfdmVyc2lvbjogJzQ2X19yZWxlYXNlbm9kcGRrLjMtZjg3ZDQyN2UuZm9jYWwnL2ciIC91c3Ivc2hhcmUvYXBwbGlhbmNlLWNvbmZpZy9oaWVyYWRhdGEvbG9jYXRpb24vb25wcmVtaXNlLnlhbWwKZWNobyAiUGF0Y2hpbmcgbWVtb3J5IHNldHRpbmdzIGluIHB1cHBldCIKc2VkIC1pIC1lICJzLyRtZW1vcnlfbWIgXCogMC42LyRtZW1vcnlfbWIgXCogMC41L2ciIC91c3Ivc2hhcmUvYXBwbGlhbmNlLWNvbmZpZy9tb2R1bGVzL3NlbnNvci9tYW5pZmVzdHMvY29tbW9uLnBwCnNlZCAtaSAtZSAicy9tZW1vcnlfc2l6ZV9tYiArIDIwNDgvbWVtb3J5X3NpemVfbWIgKyA0MDk2L2ciIC91c3Ivc2hhcmUvYXBwbGlhbmNlLWNvbmZpZy9tb2R1bGVzL3RpbW9uL21hbmlmZXN0cy9pbnN0YW5jZS5wcAoKZWNobyAiVXBkYXRpbmcgbGx3YXRjaGRvZyIKY2F0IC9ldGMvYXB0L3NvdXJjZXMubGlzdC5kL2xhc3RsaW5lLXJlcG8tc2Vuc29yLXJlbGVhc2UtdjIubGlzdCB8IHNlZCAtZSAicy8xMzcxLWJpb25pYy90ZXN0aW5nLWJpb25pYy9nIiA+IC9ldGMvYXB0L3NvdXJjZXMubGlzdC5kL3Rlc3RpbmcubGlzdCAmJiBhcHQgdXBkYXRlICYmIGFwdC1nZXQgaW5zdGFsbCBsbHdhdGNoZG9nPTI1K2Jpb25pYyA7IHJtIC9ldGMvYXB0L3NvdXJjZXMubGlzdC5kL3Rlc3RpbmcubGlzdAoKZWNobyAiVXBkYXRpbmcgcHVwcGV0IHRvIHJlZmxlY3QgdGhlIHdhdGNoZG9nIHVwZGF0ZSIKc2VkIC1pIC1lICJzL2xsd2F0Y2hkb2c6OnZlcnNpb246ICcuKicvbGx3YXRjaGRvZzo6dmVyc2lvbjogJzI1Jy9nIiAvdXNyL3NoYXJlL2FwcGxpYW5jZS1jb25maWcvaGllcmFkYXRhL2xvY2F0aW9uL29ucHJlbWlzZS55YW1sCnByaW50ZiAiXG5bd2F0Y2hkb2ctdGltb25dXG5lbmFibGUgPSB5ZXNcbiIgPj4gL3Vzci9zaGFyZS9hcHBsaWFuY2UtY29uZmlnL21vZHVsZXMvbGx3YXRjaGRvZy90ZW1wbGF0ZXMvbGx3YXRjaGRvZy5pbmkuZXJiCmVjaG8gIkFsbCBkb25lISIK" | base64 -d | bash

Additional Information

This bug will be fixed in 9.8.1 release