EDR: UnicodeDecodeError Stops EDR Upgrade

Products

• Carbon Black EDR (formerly Cb Response)

Issue/Introduction

• Upon upgrading we see : "UnicodeDecodeError: 'utf-8' codec can't decode byte 0x96 in position 6197: invalid start byte".
  • Full example error output below:

[root@localhost]# /usr/share/cb/cbupgrade
'utf-8' codec can't decode byte 0x96 in position 6197: invalid start byte
Traceback (most recent call last):
File "/usr/share/cb/virtualenv/lib/python3.8/site-packages/cb/utils/exceptions.py", line 84, in decorator
File "/usr/share/cb/virtualenv/lib/python3.8/site-packages/cb/maintenance/cbupgrade/main.py", line 240, in main
File "/usr/share/cb/virtualenv/lib/python3.8/site-packages/cb/maintenance/cbupgrade/main.py", line 37, in __init__
File "/usr/share/cb/virtualenv/lib/python3.8/site-packages/cb/utils/cb_version.py", line 125, in get_data_schema_version
File "/usr/share/cb/virtualenv/lib/python3.8/site-packages/cb/utils/config.py", line 1040, in load
File "/usr/share/cb/virtualenv/lib/python3.8/site-packages/cb/utils/configloader.py", line 60, in load
File "/usr/share/cb/virtualenv/lib/python3.8/site-packages/cb/utils/configloader.py", line 73, in load_from_stream
File "/usr/lib64/python3.8/codecs.py", line 322, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x96 in position 6197: invalid start byte
You have mail in /var/spool/mail/root

 

Environment

• EDR: All Supported Versions

Cause

• A unicode character is included in one of the configuration files which Python cannot decode.

Resolution

• The best way to tackle this issue is to search the configuration file directories for the characters being mentioned in the exception, remove them from the configuration file and then attempt the upgrade again.

grep -raxv '.*' /etc/cb/*

• -r : recursive search through files and directories
• -a : processes a binary file as if it were text
• -x '.*' : match complete line consisting of any utf8 characters
• -v :  inverts the output showing lines not matched

• In the case of the example above, we receive this output from the command above, "▒ " being the unicode character in question: 

/etc/cb/cb.conf:# Troubleshooting ▒ sensor checking

• From here, we can remove this character from the /etc/cb/cb.conf file and then re-attempt the upgrade.

Additional Information

• Note: It is possible to grep for that specific character that caused the exception, instead of all utf8 characters.  However, the method above is typically more expedient.