Setup the an Event-Forwarder on a Separate Server to Receive Audit Logs

Products

• Carbon Black EDR (formerly Cb Response)

Issue/Introduction

How to setup an event forwarder on a separate box to receive audit logs over rsyslog to be picked up by the event forwarder

Environment

• EDR: All Versions
• CB-Event-Forwarder: All Versions

Resolution

Event-Forwarder Server

1. Open /etc/rsyslog.conf
2. Look for these two sections. Both are disabled by default, depending on if you plan to be listening on TCP or UDP or both, uncomment the lines. The example has UDP options uncommented. Note: adjust the port if you are not going to use 514.
   • Note: Rsyslog receivers do not handle TCP well in that it will send multiple lines at once and the receiver will bundle them into a single line. UDP will send a single message at a time allowing the receiver to write to the file correctly. UDP is recommended here due to this, but the admin should make sure connections are always up to avoid the data being dropped.  
   • Note: adjust the port if you are not going to use 514.

     Provides UDP syslog reception
     $ModLoad imudp
     $UDPServerRun 514
     
     # Provides TCP syslog reception
     #$ModLoad imtcp
     #$InputTCPServerRun 514

3. Under those two lines, add the following lines. Contains searches require single quote surrounding, note the space as well in the search term. Directories and file will be created up on the first event received. 

   if $msg contains '/api/' then /var/log/cb/audit/useractivity.log
   &~
   if $msg contains ' ban' then /var/log/cb/audit/banning.log
   &~
   if $msg contains ' command' then /var/log/cb/audit/live-response.log
   & ~
   if $msg contains ' isolation' then /var/log/cb/audit/isolation.log
   &~

   Example:

   if $host == "192.168.1.20" then /var/log/cb/audit/useractivity.log
   &~

4. Restart rsyslog

   Centos 6 = service rsyslog restart
   Centos 7/8 = systemctl restart rsyslog

5. Open the firewall to allow port 514 communication from EDR server
6. Create a log rotate file /etc/cb/cb-logrotate.conf with the below information

   # Set the same defaults configured in /etc/logrotate.conf
   # rotate log files weekly
   weekly
   
   # keep 4 weeks worth of backlogs
   rotate 4
   
   # create new (empty) log files after rotating old ones
   create
   
   # use date as a suffix of the rotated file
   dateext
   
   /var/log/cb/audit/*.log
   {
       compress
       daily
       dateext
       dateformat -%Y%m%d.%s
       delaycompress
       maxsize 500M
       maxage 7
       missingok
       postrotate
           /bin/kill -s HUP `cat /var/run/syslogd.pid 2> /dev/null` 2> /dev/null || true
       endscript
       rotate 7
       sharedscripts
   }

7. Create a cronjob to check on rotation. Files will be rotated without any service restarts needed.

   # Run logrotate every 5 minutes to perform a rotation if necessary.
     */5  *  *  *  * root /usr/sbin/logrotate /etc/cb/cb-logrotate.conf

EDR Server:

1. Open /etc/rsyslog.d/cb-coreservices.conf
2. Find the audit log lines you wish to send. These are the 4 that are written to /var/log/cb/audit. @@ = TCP, @ = UDP

   if $programname == 'cb-audit-isolation' then /var/log/cb/isolation.log;CbAuditLogFormat
   & @<EF IP HERE>:514;CbAuditLogFormat
   & ~
   if $programname == 'cb-audit-banning' then /var/log/cb/audit/banning.log;CbAuditLogFormat
   & @<EF IP HERE>:514;CbAuditLogFormat
   & ~
   if $programname == 'cb-audit-live-response' then /var/log/cb/audit/live-response.log;CbAuditLogFormat
   & @<EF IP HERE>:514;CbAuditLogFormat
   & ~
   if $programname == 'cb-audit-useractivity' then /var/log/cb/audit/useractivity.log;CbAuditLogFormat
   & @<EF IP HERE>:514;CbAuditLogFormat
   &~

3. Open /etc/rsyslog.conf. Under the #### Modules #### section add the following config

   $MaxMessageSize 4096

4. Restart rsyslog

   Centos 6 = service rsyslog restart
   Centos 7/8 = systemctl restart rsyslog

5. May need communication to be open in the firewall if outbound is blocked for 514
6. Open /etc/cb/cb.conf and uncomment/adjust these lines to 4096

   MaxSyslogSenderMessageSize=
   MaxCbLoggingMessageSize=

7. Restart cb-enterprise services

Additional Information

• If you are not seeing messages come over, please check that they are not in /var/log/messages on the receiving server
• To confirm messages are being sent, use tcpdump on both sender and receiver to confirm

  yum install tcpdump (if not already installed)
  
  tcpdump -X -i and port 514

• Rsyslog requires administration troubleshooting as this is a built in OS application that is being utilized by EDR to write log files similar to the OS