System Overload messages in /var/log/messages is preventing the syslog action from being enabled on Security Analytics Appliance

Products

• Security Analytics

Issue/Introduction

If the data enrichment queue is too full, it is possible that certain actions may not trigger or you may not be able to enable the actions.  We want to try reducing the extractions and providers to see if the actions can then be enabled.  

 

Cause

The fact that the data enrichment queue is full, may be preventing you from enabling syslog (or other) actions.  You want to disable all of the providers and then turn up the one that is most common or most important.  Then try enabling the action. 

Resolution

To provide the most reliable test, disable all providers, check /var/log/messages and see if the "System overload Dropping" is stopped for at least fifteen minutes.

The data enrichment module processes the extractions and sends them to providers for verdicts.  In the messages file, there are many instances of "System overload. Dropping".  This is telling us that there are more requests for data enrichment than can be drained from the queue.  

In looking at the configuration for the artifacts that are being extracted, the raw text is:

derp_filter=pdf:rar:rpm:zip:conf:ftp:torrent:avi:asf:flash:mov:mpg:ra:wav:wmv:doc:docx:ppt:pptx:wpd:xls:xlsx:dll:exe

This indicates that torrent, move, mpg, wav, wmv are being sent to the extractor for data enrichment.

Other providers may also be turned on.  This includes Yara, js-unpack, Static Analysis, and ClamAV.  If a large number of filetypes for all providers are queued up, the queue file become full, or overloaded.  

Please try only enabling a couple of filetypes for one provider, and then add the syslog action.