Cb Response: Unable to start pgsql after upgrading to 6.2.4 due to disk space

Products

• Carbon Black EDR (formerly Cb Response)

Issue/Introduction

• Unable to start cb-pgsql services after running out of disk space during the cbupgrade process, specifically during the migration phase
• /var/cb/data/pgsql9.3 file exists, but /var/cb/data/pgsql does not
• /usr/share/cb/cbupgrade says it's already been upgraded

Environment

• Carbon Black Response Server: 6.2.4 and above
• Postgres9.3
• Postgres10

Cause

Migration from pgsql9.3 to pgsql10 did not complete

Resolution

1. Clean up disk space. You will need enough to expand what is currently in the pgsql9.3 folder. Use the following to check the current size.

   du -sh /var/cb/data/pgsql.93
2. Rename the pgsql9.3 back to pgsql

   mv /var/cb/data/pgsql.93 /var/cb/data/pgsql
3. copy the current SCHEMA_VERSION file and rename it

   cp /var/cb/data/SCHEMA_VERSION /var/cb/data/SCHEMA_VERSION.bak
4. Edit the SCHEMA_VERSION to the 6.2.3 version. 

   6.2.3.180809.1703
5. Run the cbupgrade script again

   /usr/share/cb/cbupgrade
6. Let the services start and confirm you are able to get into the console
7. Remove the pgsql.93 folder

   rm -rf /var/cb/data/pgsql.93

Additional Information

• During the migration, the pgsql database will get truncated/vacuumed to a smaller size
• Changing the contents of the SCHEMA_VERSION will force the cbupgrade script to run again