vRealize Orchestrator services fail to start. 'Invalid client secret' are seen in server.log

Products

• VCF Operations/Automation (formerly VMware Aria Suite)

Issue/Introduction

Symptoms:

• The vco-server service fails to start. Messages similar to the following are found in server.log:

2018-12-28 13:01:00.039+0000 [pool-1-thread-1] ERROR {} [Bootstrap$InitVcoAppContextCommand] vRealize Orchestrator applicaiton context initialization failed! Retrying in 30s: Unable to initialize group definition. Group resource name [classpath*:/META-INF/spring/singletonBeanFactoryLocator-context.xml], factory key [vcoApplicationContext]; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'vcoApplicationContext' defined in URL [jar:file:/usr/lib/vco/app-server/deploy/vco/WEB-INF/lib/o11n-services-bootstrap-7.5.0.jar!/META-INF/spring/singletonBeanFactoryLocator-context.xml]: Bean instantiation via constructor failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [com.vmware.o11n.platform.PlatformContext]: Constructor threw exception; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'com.vmware.o11n.configuration.snapshot.sections.LicenseSection#0' defined in URL [jar:file:/usr/lib/vco/app-server/deploy/vco/WEB-INF/lib/o11n-snapshot-7.5.0.jar!/META-INF/spring/platform/snapshot-context.xml]: Unsatisfied dependency expressed through constructor parameter 1; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'com.vmware.o11n.configuration.licensing.LegacyLicenseXmlMigrator#0': Unsatisfied dependency expressed through field 'vcLicenseDataFactory'; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'vc': Unsatisfied dependency expressed through field 'licenseTester'; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'licenseTester': Unsatisfied dependency expressed through field 'ssoTokenServiceAdaptor'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'tokenServiceAdaptor' defined in class path resource [com/vmware/o11n/security/config/VcoSecurityTokenConfig.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [com.vmware.o11n.security.sso.SsoTokenServiceAdaptor]: Factory method 'tokenServiceAdaptor' threw exception; nested exception is org.springframework.security.authentication.BadCredentialsException: {
  "error": "invalid_client",
  "error_description": "Invalid client secret."

Environment

VMware vRealize Orchestrator 7.x

Cause

In certain circumstances, the client.id and client.secret values can be changed in the /etc/vco/app-server/sso.properties file. In order to resolve this, a new client.id and client.secret will need to be regenerated.

Resolution

If running these steps on an embedded vRealize Orchestrator instance which is clustered, only run this on the primary node. Changes will be synchronized to other nodes.

1. Take snapshots of both appliances
2. Stop the vco-server service
   • service vco-server stop
3. Delete the registration ID file
   • rm /etc/vco/app-server/vco-registration-id
4. Reset registered vco service:
   • vcac-vami vco-service-reconfigure

Standalone Orchestrator Instance:

1. SSH into the vRealize Orchestrator server.
2. Run the command: /var/lib/vco/tools/configuration-cli/bin/vro-configure.sh reset-authentication