EDR: Minion Fails to Ingest Data Because Datastore Thread Stuck in Loop

Products

• Carbon Black EDR (formerly Cb Response)

Issue/Introduction

• Minion(s) ingest ZERO events and create SOLR cores with no data
• PSQL on the Master server has the following error in /var/log/cb/pgsql/postgresql_<date>.txt

  2019-06-20 00:00:10 UTC [cbfs-http(111967) @ <MINION_IP_ADDRESS>(58542)] ERROR: invalid byte sequence for encoding "UTF8": 0x00
  2019-06-20 00:00:10 UTC [cbfs-http(111967) @ <MINION_IP_ADDRESS>(58542)] STATEMENT: INSERT INTO vt_write_events (timestamp, parent_md5hash, file_md5hash, parent_name, file_name, sensor_group_id) VALUES ( $1, $2, $3, $4, $5, $6 ):

Environment

• EDR Server: 6.x

Cause

Any SQLException while writing to vt_write_event table in PSQL will cause the thread to be in loop and the datastore service would remain busy and not digest data.

Resolution

1. Upgrade to a supported version of EDR as this issue has been resolved
2. As a workaround, restart the cb-datastore service on the Master node, which will drop the "bad event data" and moves on to consume next event. 

service cb-datastore stop
service cb-datastore start