"Missing Key usage extension" Error on EDR Certificates cb-server.crt and cb-client-ca.crt

Products

Carbon Black EDR

Issue/Introduction

Both the EDR server certificate (cb-server.crt) and the client CA certificate (cb-client-ca.crt) display the error message:

Missing Key usage extension

However, the system continues to function correctly and sensors remain operational.

Environment

Carbon Black EDR Server: 7.7.0 and higher

Resolution

Recommendation: Backup Before Proceeding

Before any certificate operations, it is highly recommended to perform a full backup of your EDR server and certificates.

Server backup guide:
Broadcom Carbon Black EDR Server Configuration Manager Backup
Backup certificates command:

sudo /usr/share/cb/cbssl backup --out /root/certs.bak

Generating a New Client Certificate Set

Once you have verified sensors are able to connect and show online, the next step is to generate a new set of client certs. (Primary only)

Stop the EDR services

Backup the current expired client and server certificates.

mkdir /etc/cb/certs/expired_certs && cp /etc/cb/certs/cb-client* /etc/cb/certs/expired_certs/ && cp /etc/cb/certs/cb-server.* /etc/cb/certs/expired_certs/

Create new client certificates.
1. If EDR is installed on RHEL/CentOS 7 please follow this step, else continue to step 2. This temporary modifies the release file to will work around a el8 check in the utility code.
```
cp /etc/redhat-release /etc/redhat-release.bkp && sed -i 's/7/8/' /etc/redhat-release
```
2. Set an environmental variable to remove the accidental regeneration safety check.
```
export FORCE_REGENERATE=client-ca
```
3. Regenerate the cb-client-ca certificate set.
```
/usr/share/cb/cbssl certs --regenerate client-ca
```
4. Revert the OS version If EDR is installed on RHEL/CentOS 7
```
mv /etc/redhat-release.bkp /etc/redhat-release
```
5. Unset the environmental variable
  - ```
  unset FORCE_REGENERATE
```
6. In a cluster environment, synchronize the regenerated certificates from the Primary to the Minions.
  - ```
  /usr/share/cb/cbcluster sync-certs --cert client-ca
```
Start the EDR services.

Generating a Temporary Self-signed Certificate

The next step is to generate a temporary server certificate once all sensors are online with the checks still enforced. (Sensors will drop offline in the console after 5 minutes if the procedure failed).

Copy the script from the additional information section and create a temp_server_cert.sh file to the EDR server backend.
- It's recommended to create a new directory so the cert files are easy to find. cd into this directory as it will drop the files into the current working directory.
Update the script permissions
```
chmod 700 temp_server_cert.sh
```
View the script, you can change any of the parameters under the EOF section that will be entered into the script or leave it default. The DNS.1 and DNS.2 entries can be left default or modified, however they should not be a DNS resolvable name.
Run the script with a "filename" indicator, hostname is suggested.
```
./temp_server_cert.sh $(hostname)
```
This will drop three files, the .conf to make the cert and the .key and .pem.
Log into the EDR console as a global admin.
Go to your username > Settings > Server Certificates
Click to + Add certificate
1. Add a name, this is alphanumeric restricted, no spaces or special characters. "temporary" can be used to help distinguish for deletion later.
2. Upload the .pem under the "Upload certificate" section
3. Upload the .key under the "Upload private key" section
4. Click Add to create the temp cert.
Go to the Sensors page
Create new sensor group(s) and assign the new temporary certificate.
Move all sensors to these groups.
Wait for all sensors to show the new temporary certificate in the "Server Certificate" column before proceeding.

Generate a New "Legacy" Self-signed Certificate

Stop the EDR services.
Set a new environmental variable.
```
export FORCE_REGENERATE=legacy
```
If EDR is installed on RHEL/CentOS 7 please follow this step, else continue to step 2. This temporary modifies the release file to will work around a el8 check in the utility code.
```
cp /etc/redhat-release /etc/redhat-release.bkp && sed -i 's/7/8/' /etc/redhat-release
```
Create the new legacy self signed certificate.
```
/usr/share/cb/cbssl certs --regenerate legacy
```
Revert the OS version file change and unset the protection environmental variable.
```
mv /etc/redhat-release.bkp /etc/redhat-release && unset FORCE_REGENERATE
```
If on an EDR Cluster run this command to sync the certs on the Primary Server to the Minion Servers
```
/usr/share/cb/cbcluster sync-certs
```
Start the EDR services.
Log back into the console.
1. Go into the username > Settings > Server Certificates
2. Verify the "legacy" certificate has a new thumbprint and non-expired date.
3. Go to the sensors page and edit the test group by setting "legacy" as the "Assign Server Certificate"
4. Wait 10 minutes to verify the sensor stays online.
If the sensor stays online, modify each sensor group with this new legacy certificate.
When all sensors are now using the new legacy server certificate, the temporary certificate can be deleted from the console.

Revoke Sensor Group Certificates

Revoke all sensor group certificates to generate new group certificates signed by the new cb-client-ca.
See detailed instructions here: How to Revoke Sensor Group Certificates
Allow some time for sensors to collect the updated sensor group certificates, which depends on how many endpoints are online.

Additional Information

Due to the nature of certificates and their purpose, the steps are complicated to swap out the certificates. This option provides the ability to swap certificates with minimal re-installation required. There may still be sensors that need installation after following these instructions if the endpoint was not online during the time the steps were performed.
The temporary cert can be used normally over a period of time to allow sensors to connect and get the cert so that they don't need to be reinstalled.
Any sensor installer package created prior to this change will need to be regenerated. Failure to do this will result in new sensors not connecting. Verify with your teams that use an application management software such as JAMF, SCCM, GPO etc. that they are using newly generated packages for the installation automation.
The other option is to re-install all sensors with a newly generated package. If this is the option you prefer, follow the steps outlined in these two sections only prior to downloading sensor installation packages.
- Generating a New Client Certificate Set
- Generate a New "Legacy" Self-signed Certificate

temp_server_cert.sh

#!/bin/sh

: '

### THIS SCRIPT GENERATES A TEMPORARY SELF SIGNED CERT WITH TWO SAN ENTRIES ###

When running the script, please add a domain name. 
This is just used to name the files, however it may be easier to use the hostname.

./temp_server_cert.sh <domain name> 

The two DNS entries are the SAN entries required for a custom certificate. 
DNS.1
DNS.2
These can be anything you want as long as they are not DNS resolvable. Or you can use the default entered already.

Feel free to update certificate entry information

'


if [ "$#" -ne 1 ]
then
  echo "Usage: Must supply a domain"
  exit 1
fi
 
DOMAIN=$1
 
cat << EOF >> ${DOMAIN}_config.conf
[req]
distinguished_name = req_distinguished_name
x509_extensions = v3_req
prompt = no
[req_distinguished_name]
C = US
ST = MA
L = Waltham
O = Broadcom Carbon Black
OU = Support
CN = XXXX
[email protected]
[v3_req]
keyUsage = keyEncipherment, dataEncipherment
extendedKeyUsage = serverAuth
subjectAltName = @alt_names
[alt_names]
DNS.1 = edr.primary
DNS.2 = edr.minion
EOF
 
echo "Creating the self signed cert and key"
openssl req -x509 -nodes -days 1825 -newkey rsa:2048 -keyout ${DOMAIN}_private_sensor.key -out ${DOMAIN}_public_sensor.pem -config ${DOMAIN}_config.conf -extensions 'v3_req'