vSphere Cloud Account certificate is changed causing errors such as Unavailable for Deployment and Failed to validate

Products

• VMware Aria Suite

Issue/Introduction

Symptoms:

• You have recently changed the vSphere certificate that is used as a Cloud Account in VMware Aria Automation SaaS / 8.x (formerly VMware vRealize Automation 8.x)
• You notice errors similar to "Unavailable for Deployment or Failed to validate" when attempting to update the Cloud Account and see error message
  
• You notice one of the following errors or similar when validating and saving the Cloud Account:
  "Unable to update endpoint of type vsphere with hostname: vcenter-server.example.com ClientResponse has erroneous status code: 400 Bad Request"
      Note: In some cases, the endpoint will validate and save, but the Update will show as "Unavailable for deployment" again after a few minutes.
• You notice errors similar to "Cannot find available cloud zones in the current project. Maybe there is no connectivity to the endpoint(s)" when trying to provision.
• You see error like the following in provisioning-service-app.log:
  "ERROR provisioning [host='provisioning-service-###' thread='###' user='###' org='###' trace='###' parent='###' span=###'] c.v.p.c.m.a.v.VSphereEndpointAdapterService.lambda$validate$8:280 - handleCertificate serviceErrorResponse: {"message":"Error storing certificate. Retry validation","statusCode":400,"errorCode":0,"serverErrorId":"###","documentKind":"com:vmware:xenon:common:ServiceErrorResponse"} for host: ###"
  
  
• You have recently upgraded to Aria Automation 8.18.0 and now are unable to connect to the remote console of vCenter VMs through Automation.
• You notice errors when click "Update" on "Available for Deployment":
  • 
  • Error shows like so:

    • Endpoint validation failed during health check: FAILED with Failed to validate.. Original Task Error: 'Failed to validate.'

    • 

Environment

• VMware Aria Automation 8.x
• VMware vRealize Automation 8.x

Cause

When a vSphere certificate is changed, the Aria Automation user revalidates the Cloud account and accepts the new certificate but closes the Cloud account without saving. The certificate is then accepted in the trust store but not saved to the endpoint.

Resolution

This issue is resolved in the future release of VMware Cloud Foundation 9.0.

• See the Workaround section below for additional details.

Workaround:

Prerequisites

• Please take simultaneous non-memory snapshots of each virtual appliance(s) in the cluster.
  • Use the quiesce option for versions 8.9.x and above.

Requirements:

• Aria Automation Orchestrator plugin for Aria Automation

Steps:

1. Download the attached Automation Orchestrator package net.broadcom.vra.update.ca.certificate
2. Import this package into Orchestrator. Import a Package in the Automation Orchestrator Client
3. Create an Aria Automation plugin host with tenant administrator credentials. (This step can be skipped if you already have an Aria Automation plugin host for the impacted Automation Cluster.) You can verify this by looking under vRO>Inventory>and looking for a VMware Aria Automation endpoint. If it already exists, it will show up there and you'll use the Default hostname Add a VMware Aria Automation host
4. Start the workflow Update Cloud Account Certificate
5. Select the Aria Automation host from Step#3 for field Tenant Admin Host.
6. Once the host is selected, all configured vSphere cloud accounts will be populated in the vSphere Cloud Accounts to Update data grid.
7. Remove any cloud accounts which do not require a certificate update or leave the field Administrator password empty.
8. Enter data for the Administrator password field.
9. Trigger the workflow run and wait until the execution completes.
10. Once the workflow execution is completed you will see a summary of information for the update certificate operation in the workflow execution log.
11. Close the Cloud Account if it is open in the UI, then Validate the health of the Cloud Account. This may take up to 10 minutes to complete the data collection.

 

If this did not resolve the issue then please contact Broadcom Support and reference Knowledge Article ID 318756 in the problem description. For more information, see How to Submit a Support Request.