EDR: Recover Sensors after Strict Mode was Enabled

Products

• Carbon Black EDR (formerly Cb Response)

Issue/Introduction

• Sensors are going offline.
• The /var/log/cb/nginx/access.log (for Hosted EDR /var/log/cb/nginx/sensor-access.log) contains HTTP 400 errors (SSL incorrect) for sensor checkins.
• Forcing a check-in 'sc control carbonblack 200' does not trigger a connection to the server.
• Linux or Mac sensordiags show sensorsettings.ini with CbServerSSLCertStrictCheck=true.
• Windows sensordiags show registry setting 'CbServerSSLCertStrictCheck' set to 1.
• Proof that Strict Mode was enabled is found on the EDR server in the /var/log/cb/audit/useractivity file and contain:

entry path=‘/api/v1/settings/global/advanced’  and
content=‘{server_ssl_cert_strict”: {“changeable”: true, “value”: true}

• Current status of the global strict mode setting is in the postgres database:

psql -p 5002 -d cb -c “select server_ssl_cert_strict from global_query_settings;”

 

Environment

EDR Server: 7.x

Cause

• On the EDR Console, under Admin > Settings > Server Certificates, the mode 'Strict certificate validation' was selected and Saved.   If EDR is using legacy certificates (or the Certificate Authority is not configured properly) when Strict Mode is enabled, the sensors checking-in enable this mode locally and then fail to properly authenticate.

• 'Strict certificate validation' has been enabled mistakenly while troubleshooting Server-to-Sensor communications, only to break additional sensor communications.

Resolution

• On each sensor, the CbServerSSLCertStrictCheck variable needs to be set to zero (or False) and then restart the CB service.

• On each Windows sensor:

  @echo off
  reg add HKLM\Software\CarbonBlack\config -v CbServerSSLCertStrictCheck -t REG_DWORD -d 0 -f
  sc stop carbonblack  
  sc query carbonblack
  sc query carbonblackk
  sc stop carbonblackk
  sc start carbonblack

* May require a reboot.

• On each Linux sensor:

Add the variable to /var/opt/carbonblack/response/sensorsetting.ini file.

CbServerSSLCertStrictCheck=false

Restart sensor services.

kill -s 1 $(pidof cbdaemon)

• On each Mac sensor:​​​​​​

Run:

'defaults write com.carbonblack.sensor-service CbServerSSLCertStrictCheck 0'

Alternatively, if sensorsetting.ini is available:

Stop sensor services:
'launchctl unload /Library/LaunchDaemons/com.carbonclack.daemon.plist'
Apply the change:
Add/Modify  'CbServerSSLCertStrictCheck = False' in /var/lib/cb/sensorsettings.ini
Restart sensor services:
'launchctl load /Library/LaunchDaemons/com.carbonclack.daemon.plist'

 

Additional Information

• Setting Strict Certificate Validation on legacy certificates will cause sensors to go offline.

• Sensors do not attempt to communicate with the EDR server after receiving approximately three HTTP 400 errors.