Aria Automation upgrade to 8.14 fails with error "LCMVRAVACONFIG90029" Deploying project services

Products

• VMware Aria Suite

Issue/Introduction

Symptoms:

• Aria Automation upgrade to 8.14 fails with error "LCMVRAVACONFIG90029" Deploying project services
• The upgrade failed while deploying the services we can see the project service app in Error and crash loop

[INFO][2023-11-15 06:19:23][XXXXXXXXXXXX.vmware.cba] Synced upgrade data with node: xxxxxxxxxx.vmware.cba

[INFO][2023-11-15 06:19:24][xxxxxxxxxx.vmware.cba] Deployment of infrastructure and application services started.

[WARNING][2023-11-15 07:22:27][xxxxxxxxxx.vmware.cba][Exit Code: 123] Error while deploying infrastructure and application services.

[INFO][2023-11-15 07:22:27][xxxxxxxxxx.vmware.cba] Shutting down application services...

Environment

VMware vRealize Automation 8.x

Cause

• The problem causing the failure are duplicated database entries in project-db.

Resolution

• This is a known issue and likely to be made available as code fix in Aria Automation 8.16.1.
• There is also a workaround.

Workaround:

• The workaround is to delete the duplicate entries from the project-db in Aria Automation DB.
• Steps:

1. Take a snapshot of Aria automation nodes.
2. SSH to one of the the nodes as root
3. Eter the vRA DB: vracli dev psql
4. Change to project db: \c project-db
5. Execute the below query to clear the duplicate entries:

DELETE FROM project_principals WHERE id IN (

  -- entries that are 1:1 copies (both have user_id/group_id or both do not have ids)

  SELECT fst.id FROM project_principals fst

  JOIN project_principals snd ON fst.id > snd.id

    AND fst.principal_id = snd.principal_id

    AND fst.project_id = snd.project_id

    AND fst.role = snd.role

  WHERE fst.user_id = snd.user_id OR fst.group_id = snd.group_id

    OR (fst.user_id IS NULL AND fst.group_id IS NULL AND snd.user_id IS NULL AND snd.group_id IS NULL)



  UNION



  -- entries that differ in having user_id/group_id - the one that does not have id is chosen to be deleted

  SELECT fst.id FROM project_principals fst

  JOIN project_principals snd ON fst.id != snd.id

    AND fst.principal_id = snd.principal_id

    AND fst.project_id = snd.project_id

    AND fst.role = snd.role

  WHERE fst.user_id IS NULL AND fst.group_id IS NULL AND (snd.user_id IS NOT NULL OR snd.group_id IS NOT NULL)

); 

 

6. Take a new snapshot of the nodes
7. Start the upgrade from LCM again.

Additional Information

Impact/Risks:

• Upgrades to 8.14 onwards will fail.