Unable to upgrade Aria Operations due to an error "PAK Manager: The PAK action 'deploy_new_upgrade_content' script 'source ./pak_python_wrapper.sh b2bServiceConfigurator.py' failed"

Products

• VMware Aria Suite

Issue/Introduction

• Aria Operations upgrade fails due to an error occurred at staging "PAK Manager: The PAK action 'deploy_new_upgrade_content' script 'source ./pak_python_wrapper.sh b2bServiceConfigurator.py' failed"
• Cluster shows as going offline with nodes in offline state and button to take node offline or online is greyed out. 
• Log File /storage/vcops/log/casa/casa.log reports the error:

Command '/usr/bin/sudo -n ... vcopsPakManager.py --action deploy_new_upgrade_content ...' 
threw exception: CommandLineExitException: key=general.failure; args=1

ERROR ... upgrade.pak.PakService:4865 -
com.vmware.vcops.casa.exception.CasaLocalizableException: key=pak_manager.script_failed; 
args=deploy_new_upgrade_content,source ./pak_python_wrapper.sh b2bServiceConfigurator.py

Could not read filename ... casa_pak_history_vRealizeOperationsManagerEnterprise-816123365473.json (No such file or directory)

• Log file "deploy_new_upgrade_content_err.log" shows below error: 

WARNING:root:Open URL: 'https://localhost/casa/cluster/security/ping' failed with a URLERROR returned reason: [SSL:CERTIFICATE VERIFY FAILED] certificate verify failed: certificate has expired ( ssl.c: 1076), exception: <urlopen error [SSL: CERTIFICATE VERIFY FAILED] certificate verify failed: certificate has expired ( SSL.c: 1076)
Traceback (most recent call last):
File "b2bServiceConfigurator.py", line 292, in <module>
  main()
File "b2bServiceConfigurator.py", lune 288, in main
  b2bService.configure()
File "b2bServiceConfigurator.py", line 256, in configure
  is_admin_node = self.ping_node_to_find_admin()
File "b2bServiceConfigurator.py", line 194, in ping_node_to_find_admin
  response = CasaManager().get_casa_call(casa_ping_url, logger)
File "/storage/db/pakRepoLoca1/vRealizeOperationsManagerEnterprise—d14122798982/extracted/va1idate.py", line 2199, in get_casa_call
  return CasaManager.hand1e_http_response(url, response_code, response, logger, log_response=log_response)
File "/storage/db/pakRepoLoca1/vRealizeOperationsManagerEnterprise—814122798982/extracted/va1idate.py", line 2215, in handle_http_response 
  raise CaSARestException(response code, error message)
validate.CaSARestException: Failed to call CaSA ur1 https://localhost/casa/cluster/security/ping. CASA returned status code 9000, response:

Environment

Aria Operations 8.x

Cause

Custom web certificate is expired

Resolution

There are two options mentioned below, please ensure to take snapshots prior to implementing any steps below

Option 1: 

• Revert to snapshots
• Reload the default certificate
• Retry the upgrade

 

Option 2: 

• Revert to snapshots
• Configure a Certificate
• Replace the custom certificate with a newly generate certificate
• Retry the upgrade