/api/v1/sensor cmds failing when results are over 100,000 entries and timeout

Products

• Carbon Black EDR (formerly Cb Response)

Issue/Introduction

API errors like:

Bad Gateway</h1>
<h3>Whoa! Sorry about that!</h3>
<p>Not sure what happened, but it's not meant to work like that!</p>
<p>If you're seeing this consistently, can you tell us about it so we can fix it?
Send us an email at <a href="mailto:[email protected]" target="_new">[email protected]</a>.</p>

OR

Traceback (most recent call last):
File "<path>\Python*\lib\site-packages\requests\adapters.py", line 439, in send
resp = conn.urlopen(
File ""<path>\Python*\lib\site-packages\urllib3\connectionpool.py", line 747, in urlopen
return self.urlopen(
File ""<path>\Python*\lib\site-packages\urllib3\connectionpool.py", line 747, in urlopen
return self.urlopen(
File ""<path>\Python*\lib\site-packages\urllib3\connectionpool.py", line 747, in urlopen
return self.urlopen(
[Previous line repeated 2 more times]
File ""<path>\Python*\lib\site-packages\urllib3\connectionpool.py", line 719, in urlopen
retries = retries.increment(
File ""<path>\Python*\lib\site-packages\urllib3\util\retry.py", line 436, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='<SERVERNAME>', port=443): Max retries exceeded with url: /api/v1/sensor (Caused by ReadTimeoutError("HTTPSConnectionPool(host='<SERVERNAME>', port=443): Read timed out. (read timeout=120)"))

Environment

• EDR Server: All Supported Versions
• API: api/v1/sensor 
• Sensors: Over 100,000 sensor_ids

Cause

The api/v1/sensor query will timeout after a certain period of time based on a default configuration of the gunicorn webserver (60 seconds), and if the results exceed 100,000 entries its possible for the query to exceed the timeout.

Resolution

1. Change the API call to limit the number of rows returned and do it in multiple runs.
   1. For example:
      1. 'curl "https://localhost:443/api/v1/sensor?rows=100000"' to return the first 100k results.
      2. Then "curl "https://localhost:443/api/v1/sensor?rows=100000&start=100000" to get the next 100k starting at 100k+