Connector page is blank when connectors are missing an oAuth2ClientId in the database Connector table.

Products

• VMware Aria Suite

Issue/Introduction

Symptoms:

• A node is re-joined to the cluster after it had previously been removed following  

  How to remove a vRealize Automation appliance from a cluster

• Within Administration, Directories Management or Connectors the re-joined node does not have a connector listed.
• Within /var/log/vmware/horizon/horizon.log the below errors are observed when authentication requests hit the re-joined node connector.
   
  • Note: The 4 digit connector ID will differ in each environment

2019-01-26 13:58:19,525 INFO (Timer-16) [[email protected];;] com.vmware.horizon.client.rest.IApi - Failed to getOAuth2AccessToken: https://XXXXX/SAAS/t/VSPHERE.LOCAL/API/1.0/oauth2/token : 401

2019-01-26 13:58:19,525 WARN (Timer-16) [[email protected];;] com.vmware.horizon.connector.admin.service.FetchOrgConfigForSAMLAttributeService - Failed to fetch org config for sending password. Continuing to use existing setting.
com.vmware.horizon.connector.exception.HorizonException: Could not get the Suite Token from Service.
 at com.vmware.horizon.connector.admin.service.impl.TokenGenerationServiceImpl.getSuiteToken(TokenGenerationServiceImpl.java:46)
 at com.vmware.horizon.connector.admin.service.FetchOrgConfigForSAMLAttributeService.getSendPasswordOrgConfig(FetchOrgConfigForSAMLAttributeService.java:99)
 at com.vmware.horizon.connector.admin.service.FetchOrgConfigForSAMLAttributeService.syncIfAppropriate(FetchOrgConfigForSAMLAttributeService.java:86)
 at com.vmware.horizon.connector.admin.service.FetchOrgConfigForSAMLAttributeService.access$100(FetchOrgConfigForSAMLAttributeService.java:35)
 at com.vmware.horizon.connector.admin.service.FetchOrgConfigForSAMLAttributeService$1.run(FetchOrgConfigForSAMLAttributeService.java:61)
 at java.util.TimerThread.mainLoop(Timer.java:555)
 at java.util.TimerThread.run(Timer.java:505)
Caused by: com.vmware.horizon.client.rest.Exception.ApiException: {
  "error": "invalid_client",
  "error_description": "Client is not found."
}
 at com.vmware.horizon.client.rest.Exception.ApiException.buildApiException(ApiException.java:69)
 at com.vmware.horizon.client.rest.Api.getOAuth2AccessTokenByClientCredentials(Api.java:235)
 at com.vmware.horizon.client.rest.Api.getOAuth2AccessToken(Api.java:113)
 at com.vmware.horizon.client.rest.Api.signIn(Api.java:92)
 at com.vmware.horizon.client.rest.Api.signIn(Api.java:87)
 at com.vmware.horizon.connector.admin.RestHorizonServiceImpl.signIn(RestHorizonServiceImpl.java:136)
 at com.vmware.horizon.connector.admin.service.impl.TokenGenerationServiceImpl.getSuiteToken(TokenGenerationServiceImpl.java:40)
 ... 6 more
Caused by: com.vmware.horizon.client.rest.Exception.ProtocolResponseException: /token
 at com.vmware.horizon.client.rest.Api.getOAuth2AccessTokenByClientCredentials(Api.java:228)

Environment

VMware vRealize Automation 7.x

Cause

There is no related OAuth2Client for the Connector worker with an ID of "####" in OAuth2Client vPostgres table.

Resolution

Prerequisite: 

•  Follow KB2149866 to clean up the impacted node.  How to remove a vRealize Automation appliance from a cluster

 

Procedures:

1. Identify the Connector ID from the Connector table which is missing an "oAuth2ClientID".

SELECT id, "tenantId", host, "createdDate", "oAuth2ClientId", version FROM saas."Connector";

2. Delete the Connector by ID that is missing an "oAuth2ClientID" from the Connector table.

DELETE from "Connector" where id=6674;

3. Remove the impacted Connector from /usr/local/horizon/conf/states/VSPHERE.LOCAL/ directory.

cd /usr/local/horizon/conf/states/VSPHERE.LOCAL
mv 3209 6674 /tmp
mv 3209 6725 /tmp

4. Reboot all Replica Nodes
5. On the impacted replica Node, remove the below files

cd /usr/local/horizon/conf/states
rm -r /usr/local/horizon/conf/states
cd /usr/local/horizon/conf/flags
rm sysconfig.firstboot
rm sysconfig.mac_address
rm sysconfig.hostname

6. Restart the horizon service

service horizon-workspace restart

7. From the impacted replica node, perform a Cluster Join operation within the vRA VAMI within the Cluster tab.
8. After the Cluster Join operation completes successfully, confirm that the impacted node has a unique connector ID and a value in the "oAuth2ClientId" table.

SELECT "id", "tenantId", "host", "createdDate", "oAuth2ClientId", "version" FROM saas."Connector";