EDR: Service Fails to Start/Upgrade

Products

• Carbon Black EDR (formerly Cb Response)

Issue/Introduction

• Attempting to start services produces a message to run /usr/share/cb/cbupgrade, despite not upgrading recently.
• When executing /usr/share/cb/cbupgrade, the following error occurs:

----------------------------
UPGRADE ABORTED
--------------------------------------------------------------------------------
Unable to upgrade from 3.0.1.130605.35 (<3.2.0) to $<CB_R_server_version>.
Please contact support for further instruction.

Environment

• EDR (Formerly CB Response) Server: 5.x and Higher
• Logically mounted partition specifically for the /var directory

Cause

• The logical disk partition that houses the /var/cb/data/SCHEMA_VERSION file has become dismounted from the server and the directory is unavailable.

Resolution

1. Stop cb-enterprise on the affected EDR server or cluster.

# for standalone 
service cb-enterprise stop

# for cluster
/usr/share/cb/cbcluster stop

2. Remount the unmounted disk partitions to the EDR Server nodes
3. Start the EDR server or cluster

# for standalone
service cb-enterprise start

# for cluster
/usr/share/cb/cbcluster start

4. Validate all services start correctly

Additional Information

• If the server or cluster start failed previously, it might be necessary to run through the RabbitMQ MNESIA fix solution on the User Exchange to get the nodes of the cluster back in sync with one another.
• Validate the permissions on the /var directory folders are unchanged and still correct.
• Confirm the /var/cb/data/ partition is not uncommented in the /etc/fstab file
• Last resort is to restore the nodes from a backup from before the configuration was affected.