Aria Automation UI fails to load with "no Healthy upstream" error
search cancel

Aria Automation UI fails to load with "no Healthy upstream" error

book

Article ID: 429415

calendar_today

Updated On:

Products

VCF Operations/Automation (formerly VMware Aria Suite)

Issue/Introduction

  • When attempting to log into the VMware Aria Automation UI, the login screen fails to load. Instead, an alert is displayed on the screen with the following error: no Healthy upstream

Environment

  • Aria Automation 8.18.x
  • VMware Identity Manager 3.3.7

Cause

  • This issue can occur when the identity-service-app pod in  Aria Automation appliance fails to resolve or establish connectivity with the VMware Identity Manager (vIDM) appliance.
  • When performing a deploy.sh to restart the services of the Automation appliance the following error is seen: ERROR Release 'identity-service' in namespace 'prelude' failed to come up
  • If the upstream vIDM cluster is unhealthy or experiencing issues, the identity-service-app pods cannot initialize.
  • Consequently, Aria Automation fails to initialize with the vIDM appliance, resulting in the UI failing to load.

Resolution

To resolve this issue, you must restore the health and connectivity of the upstream VMware Identity Manager cluster.

  1. Verify the status of your pods to confirm if the identity-service-app is failing to initialize. You can monitor the pod initialization by running the following command:

watch kubectl get pods -n prelude
  1. If the identity-service-app pods are failing to initialize, investigate your VMware Identity Manager (vIDM) cluster for health issues or offline nodes.

  2. Work with your identity team or support to resolve the underlying vIDM cluster errors.

  3. Bring at least one node in the vIDM cluster back online. Once a singular vIDM node is healthy and reachable, the identity-service-app pods will successfully initialize.

    • Shut down services by running:

      If necessary, after restoring vIDM health, restart the vRA services with an /opt/scripts/deploy.sh command. If this still fails, perform a graceful reboot of the VMware Aria Automation system to restart the services:

       

  4. deploy.sh --shutdown
    
    • Power cycle the appliance VM in vCenter.

    • Restart the services by running:

     
    deploy.sh
    

Once the pods fully initialize, the VMware Aria Automation UI will load successfully.