Watchlist run times slowing with frequent IO read spikes on server

Products

• Carbon Black EDR (formerly Cb Response)

Issue/Introduction

• Watchlist run times have increased
• The server shows high spikes in disk IO, specifically for read requests
• Event or binary backlog may spike around the same time as disk IO reads jump

Environment

• EDR Server: All Supported Versions
• EDR API is utilized in recurring scripts

Cause

​​​​​There is an inefficient API query running too often and overloading Solr

Resolution

• Any queries running on a regular schedule should be adjusted to include last_server_update based on the last time the query was sent. Without that setting the query will always request information from the earliest data point
• Adjust the length of time the query runs. If the query was running every minute, push the time to every 10 or 20 minutes

Additional Information

• Watchlists added to the server will run every 10 minutes by default