EDR: Binary Search page does not return any binaries

Products

• Carbon Black EDR (formerly Cb Response)

Issue/Introduction

• Binary Search page does not return any binary hits.
• These errors can be observed in /var/log/cb/solr/debug.log:

1@4f7ad88d rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@606baf0f[Running, pool size = 256, active threads = 256, queued tasks = 0, completed tasks = 7793122]
java.util.concurrent.RejectedExecutionException: Task

 

Environment

CB Response Server: 7.x+
CB Response Sensor: All Versions

Cause

Under certain loads, SOLR can exhaust the number of required threads. 

Resolution

Increase the number of threads in the pool from 256 to 512 as so:

1. Increase "maximumPoolSize" in /etc/cb/solr6/core_conf/solr.xml from 256 to 512 on ALL nodes.
2. Restart the cluster.

/usr/share/cb/cbcluster stop
/usr/share/cb/cbcluster start