After vRO has been migrated from vRO 7.x to vRO 8.6 or earlier versions the vRO pod status is Init:CrashLoopBackOff and it not able to start due to sql script failure

Products

• VMware Aria Suite

Issue/Introduction

Procedure for customers to resolve the issue by themselves.

Symptoms:

vRO pod remains in Init:CrashLoopBackOff status and the following error can be found in file log /services-logs/prelude/vco-app/console-logs/install-rpms.log

[ERROR: column "createdat" does not exist Where: PL/pgSQL function merge_duplicates(text,text,text) line 105 at EXECUTE

 

Environment

VMware vRealize Automation 8.6.x
VMware vRealize Operations 8.6.x

Cause

The issue is that in vRO versions 7.x,  customer is able to create folders with the same name in the same parent folder.
In 8.x, it is not allowed for folders with the same name to exist in the same parent folder, so in one of the initial containers we run a script that fixes this.
In one of the sql queries, the script uses a column for ordering which does not exist and in this case the script will fail and won't merge the duplicates therefore the server won't be able to start.

Resolution

Resolution is expected in the upcoming release vRO/vRA 8.6.1 and later

Workaround:

To work around this issue, follow these steps:

1. Create a snapshot of the environment.

NOTE: In case of clustered environment the next steps from 2 to 9 need to be executed only on one of the nodes in the cluster.

2. Connect to the vRO/vRA node using ssh.

3. Stop the vco-app pods by running the following command:
   kubectl -n prelude scale deployment vco-app --replicas=0

4. Wait for the pods to be terminated. The status can be checked with the following command:
   kubectl -n prelude get pods

5. Connect to the database using the following command:
   vracli dev psql vco-db

6. Once connected, paste the contents of the attached script and execute. You should receive output similar to the
following:

 NOTICE: MERGING DUPLICATES IN vmo_resourceelementcategory AND vmo_resourceelement
 NOTICE: table "tree" does not exist, skipping
 NOTICE: Remaining groups: 1
 NOTICE: Target id: 8a7480b87173057501717305bb090000, name: Configuration, depth: 3, tenantId: __SYSTEM
 NOTICE: MERGING DUPLICATES IN vmo_resourceelementcategory AND vmo_resourceelement
 NOTICE: Remaining groups: 0
 NOTICE: Done!
 NOTICE: MERGING DUPLICATES IN vmo_workflowcategory AND vmo_workflow
 NOTICE: Remaining groups: 0
 NOTICE: Done!
 NOTICE: MERGING DUPLICATES IN vmo_configelementcategory AND vmo_configelement
 NOTICE: Remaining groups: 0
 NOTICE: Done!
 NOTICE: MERGING DUPLICATES IN vmo_policytemplatecategory AND vmo_policytemplate
 NOTICE: Remaining groups: 0
 NOTICE: Done!

7. Start the vco-app pods using the following command:

   kubectl -n prelude scale deployment vco-app --replicas=1

NOTE: In case of clustered environment replicas should be 3

8. Check the status of the pods using the following command:
   kubectl -n prelude get pods

9. Wait for pods status to be Running.