Cannot establish a remote console connection with certificate issues when trying to use Connect to Remote Console after upgrading to VMware Aria Automation 8.18

Products

• VMware Aria Suite

Issue/Introduction

• Remote Console Day 2 Action does not work for cloud accounts with missing 'certificate' property.
  • The certificate property (normally found in customProperties and endpointProperties) may be missing if the cloud account is using 'acceptSelfSignedCertificate=true', cloud account is added across multiple tenants or was deleted and re-added.

• The /var/log/services-logs/prelude/provisioning-service-app/file-logs/provisioning-service-app.log file has a NullPointerException error similar to:

  2024-09-03T05:19:27.504Z ERROR provisioning [host='provisioning-service-app-<UID>' thread='reactor-http-epoll-5' user='' org='' trace='' parent='' span=''] c.v.w.p.server.WebsocketHandlerFromUI.handleSSLContext:160 - [vRA host: <AriaAutomationFQDN> communicating with vCenter Host: wss://<vCenterServerFQDN>:443/ticket/<ID>]: Trusted connection cannot be established with host wss://<vCenterServerFQDN>:443/ticket/<ID>. Error:
java.lang.NullPointerException: Cannot invoke "com.vmware.webmks.proxy.ticket.api.ssl.WebMksCertificateThumbprint.toString()" because the return value of "com.vmware.webmks.proxy.ticket.api.WebMksTicketPayload.getEndpointThumbprint()" is null

Environment

VMware Aria Automation 8.18.x

There are two methods to connect to VMRC. Older MKS and a newer WebMKS.

vCenter 6.x and 7.x support both MKS and WebMKS
vCenter 8.x supports only WebMKS

Cause

Beginning with VMware Aria Automation 8.18, a new remote console proxy was introduced to add support for webMKS based remote web console sessions which is mandatory in vSphere 8.0.

For Cloud Accounts in Aria Automation with acceptSelfSignedCertificate set to true (found within the Cloud Account customProperties), the remote web console day 2 action will fail.

This is due to the new proxy validating the certificate stored in the Cloud Account state (found in endpointProperties and customProperties), for any Cloud Account created with acceptSelfSignedCertificate=true, this certificate property will not be added automatically.

By default, vSphere Cloud Accounts in Aria Automation are not created with this property set, only when created (or updated) via the API.

Impact

Patching the Cloud Account with the correct certificate as described in Workaround 2 below will add the certificate to the cloud account state within both the endpointProperties and customProperties.

Resolution

Workaround

There are two possible workarounds for 8.18.x:

1. Disable the new remote proxy by toggling the feature remote.console.proxy.webmks.enabled to false. This will fall back to the remote console behavior that exists in previous versions of Aria Automation.
   

   https://AriaAutomationFQDN/automation/#/service/automation-ui/provisioning-ui;ash=%2FconfigurationProperties`

   
   
2. Patch the Cloud Account with the valid vCenter certificate as outlined in vSphere Cloud Account certificate is changed causing errors such as Unavailable for Deployment and Failed to validate

Additional Edge Cases:

1. Aria Automation Cloud Account tried to revalidate and shows error message 'Unable to find valid certification path to requested target"

Additional Information

If the issue persists after performing steps in vSphere Cloud Account certificate is changed causing errors such as Unavailable for Deployment and Failed to validate

Check again the provisioning service logs /var/log/services-logs/prelude/provisioning-service-app/file-logs/provisioning-service-app.log

If you now see an entry similar to below its likely that the certificate has been patched but in an incorrect format including line break characters '/n'.

2024-09-05T15:16:40.312Z ERROR provisioning [host='provisioning-service-app-<UID>' thread='reactor-http-epoll-6' user='<User>' org='<OrgId>' trace='<TraceId>' parent='<ParentId>' span='<SpanId>'] c.v.a.r.c.ComputeRemoteConsoleController.lambda$authenticateAndGetWebMksTicket$3:257 - Failed processing certificate from endpoint for remote console operation. Failure:
    java.security.cert.CertificateException: Unable to initialize, java.io.EOFException: not enough content

Follow the steps below to capture the correct certificate format and submit again the api calls from vSphere Cloud Account certificate is changed causing errors such as Unavailable for Deployment and Failed to validate:

Step 1. Export/Download the Certificate

Step 2. Right Click the certificate>>Edit with Notepad++

Step 3. This will show the right certificate without unwanted "/n"

Step 4. You can also get the same by command: openssl s_client -showcerts -connect IP/FQDN:443