EDR: Minion Fails to Ingest Data Because Datastore Thread Stuck in Loop
book
Article ID: 289013
calendar_today
Updated On:
Products
Carbon Black EDR (formerly Cb Response)
Issue/Introduction
Minion(s) ingest ZERO events and create SOLR cores with no data
PSQL on the Master server has the following error in /var/log/cb/pgsql/postgresql_<date>.txt
2019-06-20 00:00:10 UTC [cbfs-http(111967) @ <MINION_IP_ADDRESS>(58542)] ERROR: invalid byte sequence for encoding "UTF8": 0x00
2019-06-20 00:00:10 UTC [cbfs-http(111967) @ <MINION_IP_ADDRESS>(58542)] STATEMENT: INSERT INTO vt_write_events (timestamp, parent_md5hash, file_md5hash, parent_name, file_name, sensor_group_id) VALUES ( $1, $2, $3, $4, $5, $6 ):
Environment
EDR Server: 6.x
Cause
Any SQLException while writing to vt_write_event table in PSQL will cause the thread to be in loop and the datastore service would remain busy and not digest data.
Resolution
Upgrade to a supported version of EDR as this issue has been resolved
As a workaround, restart the cb-datastore service on the Master node, which will drop the "bad event data" and moves on to consume next event.
service cb-datastore stop
service cb-datastore start