"LCMVRAVACONFIG590003" error code when configuring vRA HA

Products

• VMware Aria Suite

Issue/Introduction

Symptoms:

• HA environment contains 3 nodes.
• HA deployment fails on initialization step (stage nine) with the following error:

Issue - Error Code: LCMVRAVACONFIG590003
Cluster initialization failed on vRA.
vRA va initialize failed on : xxxx-xxxx-xxxx, 
Please login to the vRA and check /var/log/deploy.log file for more information on failure. If its a vRA HA and to reset the load balancer, set the 'isLBRetry' to 'true' and enter a valid load balancer hostname.

• In the install-rpms.log, vco-controlcenter-app.previous.log or in vco-controlcenter-app.log, you see similar to:

2020-06-19 18:13:33,993 [ERROR] Failed to create the initial configuration data. Reason: query did not return a unique result: 2; nested exception is javax.persistence.NonUniqueResultException: query did not return a unique result: 2
Failed to update database configuration with local changes.
java.lang.RuntimeException: Failed to create the initial configuration data.
 at com.vmware.o11n.settings.proxies.DbConfigurationInitializator.run(DbConfigurationInitializator.java:53) ~[o11n-configuration-services-8.1.0.jar:?]
 at com.vmware.o11n.cli.configuration.commands.sync.SyncLocalCommand.updateDbConfiguration(SyncLocalCommand.java:28) ~[o11n-commandline-configuration-8.1.0.jar:?]
 at com.vmware.o11n.cli.configuration.commands.ConfigurationCommand.updateDbConfiguration(ConfigurationCommand.java:133) [o11n-commandline-configuration-8.1.0.jar:?]
 at com.vmware.o11n.cli.configuration.commands.ConfigurationCommand.executeCmd(ConfigurationCommand.java:122) [o11n-commandline-configuration-8.1.0.jar:?]
 at com.vmware.o11n.cli.configuration.ConfigurationCli.executeCommand(ConfigurationCli.java:123) [o11n-commandline-configuration-8.1.0.jar:?]
 at com.vmware.o11n.cli.configuration.ConfigurationCli.main(ConfigurationCli.java:99) [o11n-commandline-configuration-8.1.0.jar:?]

Environment

VMware vRealize Automation 8.1.x

Cause

This issue occurs due to the vRA installation through LCM failing due to vRO pods restarting causing a race condition in the vRO configuration manager.

Resolution

1. SSH to the primary node.
2. Scale down vco-app deployment to zero pods: 

kubectl -n prelude scale deployment vco-app --replicas=0

3. Connect to the database with:

vracli dev psql

4. Once logged into the database you have to switch the connection to the vro database with:

\c vco-db

5. Execute this SQL command:

DELETE FROM vmo_vroconfiguration config1 USING vmo_vroconfiguration config2 WHERE config1.id < config2.id AND config1.actionid IS NOT NULL AND config1.actionid = config2.actionid;

6. Log out of psql

      \q

7. Scale vco-app deployment to one replica and wait until the pod becomes in running status and all its containers are ready before proceeding with the next step.

kubectl -n prelude scale deployment vco-app --replicas=1

8. Run /opt/scripts/deploy.sh