vRealize Operations Manager Upgrade Fails Due to Invalid Character in Log File
search cancel

vRealize Operations Manager Upgrade Fails Due to Invalid Character in Log File

book

Article ID: 367482

calendar_today

Updated On:

Products

VMware Aria Suite

Issue/Introduction

This article describes an issue encountered during an upgrade to vRealize Operations Manager (vROps) 8.10.2 that results in a failure with the error message "FailedThe PAK action 'apply_system_update' script 'source ./pak_python_wrapper.sh updateCoordinator.py -p'".

2024-03-30T17:36:25 DEBUG - b2b -updateCoordinator - runScript - Script command: "/usr/bin/python /storage/db/pakRepoLocal/vRealizeOperationsManagerEnterprise-810020570120/extracted/upgrade_scripts/VA/pre/vsutilities-preb2b.py --status_file /storage/db/pakRepoLocal/vRealizeOperationsManagerEnterprise-810020570120/vRealizeOperationsManagerEnterprise-810020570120_apply_system_update.json"
2024-03-30T17:36:27 DEBUG - b2b -updateCoordinator - runScript - stdout: "Entering "vsutilities-preb2b.py"
Begin touchup_vcopssuitevm
End touchup_vcopssuitevm
Begin scrub_logfiles
"
2024-03-30T17:36:27 DEBUG - b2b -updateCoordinator - runScript - stderr: "Traceback (most recent call last):
  File "/storage/db/pakRepoLocal/vRealizeOperationsManagerEnterprise-810020570120/extracted/upgrade_scripts/VA/pre/vsutilities-preb2b.py", line 160, in <module>
    main()
  File "/storage/db/pakRepoLocal/vRealizeOperationsManagerEnterprise-810020570120/extracted/upgrade_scripts/VA/pre/vsutilities-preb2b.py", line 149, in main
    scrub_logfiles()
  File "/storage/db/pakRepoLocal/vRealizeOperationsManagerEnterprise-810020570120/extracted/upgrade_scripts/VA/pre/vsutilities-preb2b.py", line 116, in scrub_logfiles
    if grep_file(filepath):
  File "/storage/db/pakRepoLocal/vRealizeOperationsManagerEnterprise-810020570120/extracted/upgrade_scripts/VA/pre/vsutilities-preb2b.py", line 88, in grep_file
    for line in f:
  File "/usr/lib/python3.7/codecs.py", line 322, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x98 in position 4104: invalid start byte
"
2024-03-30T17:36:27 DEBUG - b2b -updateCoordinator - runScript - exit code: 1
2024-03-30T17:36:27 ERROR - b2b -updateCoordinator - runScript - Script command: "/usr/bin/python /storage/db/pakRepoLocal/vRealizeOperationsManagerEnterprise-810020570120/extracted/upgrade_scripts/VA/pre/vsutilities-preb2b.py --status_file /storage/db/pakRepoLocal/vRealizeOperationsManagerEnterprise-810020570120/vRealizeOperationsManagerEnterprise-810020570120_apply_system_update.json" failed with exit code: 1

Environment

vRealize Operations Manager (vROps) 8.10.2

Cause

The upgrade fails due to the presence of an invalid character in one of the vROps log files. The character causing the issue is encoded as byte 0x98, which cannot be decoded using the UTF-8 character encoding.

Resolution

There is no permanent fix to prevent this issue from occurring again, as the cause of the invalid character being written to the log file is unknown. However, a workaround exists:

Identify the affected log file:

Run the following command on the node where the upgrade failed:

grep -c $'\x80' /storage/log/vcops/log/pakManager/*.log | grep -v ":0$"

This command will search for the character encoded as byte 0x98 (represented as $'\x80') within all log files located in the /storage/log/vcops/log/pakManager directory.

The output will display the filename and line number where the invalid character is found.

There are two options to proceed:

Note: Make sure you take a backup of the files before proceeding with either of the below options:

Option 1: Delete the affected file(s):
If the identified file is not critical for vROps operation, you can simply delete it.

Option 2: Edit the affected file(s):
If the identified file contains important information, you can edit the file and remove the invalid character. Be cautious when editing system files.

After addressing the affected file(s), retry the vROps upgrade.

Additional Information

The provided logs indicate the error occurred during the execution of the script "vsutilities-preb2b.py" or can occur for any other scripts too.