- Use filters in the technote to import selective activity_type, severity, object_type can reduce the number of nonessential logs to be queried
- python <tool> _agent.py [--proxy <host_and_port> ] [-u <username> -p <password> ] [--severity <severity ...> ] [--app <app ...> ] [--object_type <object_type ...> ] [--activity_type <activity_type ...> ] [--elastica_app <elastica_app ...> ] [-c] [-r] [-v] [-d] [--rate] [-o/--output] [--start_date <start_date> ] [-s/--stream <stream> ] [-t/--target <socket> ] [--socket_type <udp_or_tcp> ] [-f/--filename <filename> ] [--max_bytes <maximum_bytes> ] [--backup_count <backup_count> ]
- Set up multiple agents with each agent pulls different Securlet logs using the --app option
- Check the time it takes to complete a job. Reduce the interval between jobs. For example, if a job takes 10 mins to complete, then adjust the schedule so the agent will run the job every 15 mins.
- Check the log writing rate. The default rate is 40 logs per second, this can be increased to a maximum of 5000 logs per second.
- For example, the logs below indicate it takes about 10 mins to write 25900.
YYYY-MM-DD 10:58:44,278-Log_Exporter_Client-INFO-Writing log to Syslog.
YYYY-MM-DD 11:09:40,112-Log_Exporter_Client-INFO-Wrote 25900 logs to Syslog.
-
- This can be improved and adjusted so it will take less time to write the logs and therefore finish the job faster