How to troubleshoot Yara components

Products

• Carbon Black EDR (formerly Cb Response)
• Carbon Black EDR

Issue/Introduction

How to troubleshoot the Carbon Black EDR Yara components

Environment

• Carbon Black EDR Server: 7.7.x and higher
• Yara Connector: 2.2.0
• Yara Manager: 2.2.0

Resolution

Stepping through the Yara process:

1. Validate the services are running:

   systemctl status cb-yara-connector 
   systemctl status cb-yara-manager
   #Restart services and check if the PID has changed after the restart
   systemctl restart cb-yara-connector
   systemctl restart cb-yara-manager

2. Troubleshoot broken services:

   Yara Manager is in the left menu column of EDR Console after cb.conf changes.
   Look for errors:
   less /var/log/cb/integrations/cb-yara-connector/yaraconnector.log
   less /var/log/cb/integrations/cb-yara-manager/cb-yara-manager.log

3. Was the binary uploaded?

   psql -d cb -p 5002 -c "select * from storefiles where md5hash='\x<hash>';"

4. Did Yara analyze the file?

   select md5,last_scan_date from binarydetonationresult where md5="<hash>"

5. Was there a match?

   grep <hash> /var/cb/data/cb-yara-connector/feed.json

6. Where is the Yara alert?

   curl 'http://localhost:8080/solr/cbfeeds/select?q=id:"binary_<md5 hash lower case here>"'

 

Additional Troubleshooting

1. Yara Connector logs 

   /var/log/cb/integrations/cb-yara-connector/yaraconnector.log 
   journalctl -fexu cb-yara-connector

2. Yara Manager logs 

   /var/log/cb/integrations/cb-yara-manager/cb-yara-manager.log

3. Monitor binary.db.  Does it grow in size as new binaries arrive to EDR?  First command presents the total binaries scanned (should increase). The second command identifies any missing binaries, perhaps aged out or uploaded to Alliance (should be 0).

   sqlite3 /var/cb/data/cb-yara-connector/feed_db/binary.db "select count(*) from binarydetonationresult;" 
   sqlite3 /var/cb/data/cb-yara-connector/feed_db/binary.db "select count(*) from binarydetonationresult where ‘binary_not_available’ and ‘score’>0;”

4. Run a Process Search in EDR console and expand the time range to "All available".

   alliance_score_yara:*

5. Check for new binaries in Postgres.  Modify the timestamp as needed.

   psql -p 5002 cb -c “select md5hash,node_id from storefiles where present_locally=true and timestamp>=’2023-05-19 00:00:00’ order by timestamp desc;”

6. Feed tile does not appear on Threat Intelligence page.
   1. Redis could be overwhelmed.  Remove the client output buffer limit in /etc/cb/redis.conf.template.  Change to:

      client-output-buffer-limit pubsub 0 0 0

   2. OR using EDR Console > Threat Intelligence > Add New Feed

      URL: file://var/cb/data/cb-yara-connector/feed.json

7. By design, due to performance reasons, the connector does not retroactively go back to rescan binaries for matches when a rule is uploaded. The db would need to be cleared and trigger a new scan from the beginning.
8. Reset and Restart. 

   systemctl stop cb-yara-manager
   systemctl stop cb-yara-connector
   cp /etc/cb/integrations/cb-yara-connector/yara_rules/.YARA_RULES* /etc/cb/integrations/cb-yara-connector/yara_rules/.YARA_RULES-bkup
   rm /etc/cb/integrations/cb-yara-connector/yara_rules/.YARA_RULES*
   rm /var/cb/data/cb-yara-connector/feed.json
   rm /var/cb/data/cb-yara-connector/feed_db/binary*
   systemctl start cb-yara-connector
   systemctl start cb-yara-manager

    

9. Yara-connector does not automatically start after reboot.
10. Check symlink from /etc/systemd/system/multi-user.target.wants/cb-yara-connector.service to etc/system/system/cb-yara-connector.sevice
11. Debug mode.   
    1. Add to /etc/cb/integrations/cb-yara-connector/yaraconnector.conf then restart cb-yara-connector. Binary scan does not start so feed.json does not populate.

       log_level=DEBUG

12. Verify the Threat Report is populated from the Yara Feed tile or run the following command on the EDR Primary server.

    curl 'http://localhost:8080/solr/cbfeeds/select?q=id:"binary_<md5 hash lower case here>"'

13. The binary scan does not start and feed.json remains empty.

    * Confirm the mode is set to 'standalone' (even in an EDR cluster on primary server)
    * Confirm the token in the yara configuration has adequate permissions to access the binaries and the account is active.

14. The binary.db and feed.json are populated but alerts do not appear in the console.  The connection with Solr may be broken.  

    * Check the Solr logs and yara-connector logs for errors.
    * Consider resetting the yara-connector noted previously.