Endpoint communication is broken after upgrade to vRealize Automation 7.3 and above

Products

• VMware Aria Suite

Issue/Introduction

This article describes how to resolve issues surrounding new endpoint certificate requirements in vRealize Automation 7.3 and above.

Symptoms:

• Following a vRealize Automation upgrade from a version less than or equal to 7.2  to 7.3 or above, endpoint communication no longer functions properly.
  • Provisioning fails in general.
  • The Agent Status on the vSphere Compute resources shows as down.
  • Data collection fails.
  • Cannot establish connection with the vCenter/NSX endpoints
     
• In the Infrastructure-as-a-Service (IaaS) logs, you see certificate/trust related errors similar to:
  
  This exception was caught:
  The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel.
  Inner Exception: The remote certificate is invalid according to the validation procedure.

Environment

VMware vRealize Automation 7.3.x
VMware vRealize Automation 7.4.x
VMware vRealize Automation 7.5.x
VMware vRealize Automation 7.6.x

Cause

Beginning with vRealize Automation 7.3, vSphere and NSX endpoints have certificate validation enabled. You can no longer use an untrusted certificate with these endpoints.

Although you can use the "Test Connection" button to validate the certificate thumbprint on these endpoints, if the certificate is generated so that the root certificate in the certificate chain is not self signed, the certificate validation process for these two endpoints can fail and cause a functional failure in data collection, provisioning, or post-provisioning actions.

Resolution

This is a known issue affecting VMware vRealize Automation 7.3 and above.

 

This issue is discussed in the downloaded zip folder. vRealize-automation-7.6

For vSphere Endpoints

In most scenarios testing the endpoint connections and accepting the certificate prompt will resolve. However if the certificate chain has untrusted root then below steps are necessary

Download the root certificate in the endpoint certificate chain.

• For vCenter endpoint 6.0 or later, see How to download and install vCenter Server root certificates to avoid Web Browser certificate warnings.
• For vCenter endpoint 5.5 or earlier, download the ROOT certificate from the endpoint certificate's certification path.

Complete these steps.

1. First download the endpoint certificate by accessing the endpoint directly in the browser.
2. Go to Certification Path to get the root certificate.
3. Download the root certificate in the chain.
4. Install the certificate in the Trusted root store of the Agent and DEM machines.
5. Restart DEMs and Agents.

For the NSX Endpoint

1. Download the endpoint certificate( the NSX certificate, as well as the Root certificate) by accessing the endpoint directly in the browser.
2. Go to Certification Path to get the root certificates.
3. Download the certificates in the chain.
4. Install the NSX certificate and the root certificate in the Trusted root store of the DEM machines.
5. Restart DEMs.

Note:

• Certificate name validation also occurs so its important that, the endpoint name matches the CN in the certificate. Otherwise you see the validation error:
  Certificate is not trusted (RemoteCertificateNameMismatch)
  
  
• To place a certificate in the Trusted root store, the browser should be run with Administrator privileges. Refer to instructions in https://blogs.technet.microsoft.com/sbs/2008/05/08/installing-a-self-signed-certificate-as-a-trusted-root-ca-in-windows-vista/