ERROR Failed to upgrade release 'client-secrets' in namespace 'prelude' while running deploy.sh on Aria Automation

Products

• VCF Operations/Automation (formerly VMware Aria Suite)

Issue/Introduction

• Unable to access vRA UI and getting 404 page not found
• Running /opt/scripts/deploy.sh fails at the stage "Populating initial identity-service data"
• This stage fails with a 500 error from Identity Manager and shows the error from the description:
  • ERROR Failed to upgrade release 'client-secrets' in namespace 'prelude'
• Checking Identity Service logs while the client-secrets is trying to start (/services-logs/prelude/identity-service-app/file-logs/identity-service-app.log) :
  • ERROR identity-service [host='identity-service-app-###########' thread='reactor-http-epoll-1' user='' org='' trace='###########'] c.v.i.c.RestResponseEntityExceptionHandler.logError:225 - Handling bad gateway exception: REST error received: {"Errors":[{"code":"500","description":"Internal Server Error"}]}, status code: 500 INTERNAL_SERVER_ERROR
        com.vmware.identity.exception.BadGatewayException: REST error received: {"Errors":[{"code":"500","description":"Internal Server Error"}]}, status code: 500 INTERNAL_SERVER_ERROR
• Checking the vracli vidm info on Automation, the SHA-256 thumbprint stored doesn't match to the current vIDM certificate
• Attempting to update this data as per KB 322719, we again receive a 500 error from vIDM (vracli vidm set https://vidm.fqdn.com admin vidmConfigAdmin)

Environment

• VMware Aria Automation 8.x
• VMware Identity Manager 3.3.7

Cause

Identity Manager is sending a 500 error when Aria Automation is looking for updated identity info after an IDM certificate change.

Resolution

1. Take a snapshot of the environment's machines
2. Reboot of all vIDM nodes: power off and power on via LCM
3. Optionally trigger Automation to update its IDM data as per KB 322719 - this will be done as part of deploy.sh 
4. Run /opt/scripts/deploy.sh