vSphere Cloud Account certificate change causes errors such as Unavailable for Deployment and Failed to validate in Aria Automation
search cancel

vSphere Cloud Account certificate change causes errors such as Unavailable for Deployment and Failed to validate in Aria Automation

book

Article ID: 318756

calendar_today

Updated On:

Products

VCF Operations/Automation (formerly VMware Aria Suite)

Issue/Introduction

  • You have recently changed the vSphere certificate that is used as a Cloud Account in VMware Aria Automation 8.x 
  • You notice errors similar to "Unavailable for Deployment or Failed to validate" when attempting to update the Cloud Account and see error message
  • You notice one of the following errors or similar when validating and saving the Cloud Account:
    "Unable to update endpoint of type vsphere with hostname: vcenter-server.example.com ClientResponse has erroneous status code: 400 Bad Request"
        Note: In some cases, the endpoint will validate and save, but the Update will show as "Unavailable for deployment" again after a few minutes.
  • You notice errors similar to "Cannot find available cloud zones in the current project. Maybe there is no connectivity to the endpoint(s)" when trying to provision.
  • You see error like the following in provisioning-service-app.log: 
    "ERROR provisioning [host='provisioning-service-###' thread='###' user='###' org='###' trace='###' parent='###' span=###'] c.v.p.c.m.a.v.VSphereEndpointAdapterService.lambda$validate$8:280 - handleCertificate serviceErrorResponse: {"message":"Error storing certificate. Retry validation","statusCode":400,"errorCode":0,"serverErrorId":"###","documentKind":"com:vmware:xenon:common:ServiceErrorResponse"} for host: ###"
  • You have recently upgraded to Aria Automation 8.18.0 and now are unable to connect to the remote console of vCenter VMs through Automation.
  • You notice errors when click "Update" on "Available for Deployment":
  • Error message similar to the one show below are displayed on the Cloud Account view: 
    Endpoint validation failed during health check: FAILED with Failed to validate.. Original Task Error: 'Failed to validate.'

Environment

VMware Aria Automation 8.x

Cause

When a vSphere certificate is changed, the Aria Automation user revalidates the Cloud account and accepts the new certificate but closes the Cloud account without saving. The certificate is then accepted in the trust store but not saved to the endpoint.

Resolution

This issue is resolved in the future release of VMware Cloud Foundation 9.0.

  • See the Workaround section below for additional details.

Workaround:

Prerequisites

  • Please take simultaneous non-memory snapshots of each virtual appliance(s) in the cluster.
    • Use the quiesce option for versions 8.9.x and above.

Requirements:

  • Aria Automation Orchestrator plugin for Aria Automation

Note:

  • The procedure outlined below is only applicable for a vSphere Cloud Account and not applicable in case of certificate changes for other cloud account types such as NSX / VCF cloud account. 
  • In case of NSX or VCF cloud account having the reported issue, please contact Broadcom Support and reference Knowledge Article ID 318756 in the problem description. For more information, see How to Submit a Support Request.

Steps:

  1. Download the attached Automation Orchestrator package net.broadcom.vra.update.ca.certificate
  2. Import this package into Orchestrator. Import a Package in the Automation Orchestrator Client
  3. Create an Aria Automation plugin host with tenant administrator credentials. (This step can be skipped if you already have an Aria Automation plugin host for the impacted Automation Cluster.) You can verify this by looking under vRO>Inventory>and looking for a VMware Aria Automation endpoint. If it already exists, it will show up there and you'll use the Default hostname Add a VMware Aria Automation host
  4. Start the workflow Update Cloud Account Certificate
  5. Select the Aria Automation host from Step#3 for field Tenant Admin Host.
  6. Once the host is selected, all configured vSphere cloud accounts will be populated in the vSphere Cloud Accounts to Update data grid.
  7. Remove any cloud accounts which do not require a certificate update or leave the field Administrator password empty.
  8. Enter data for the Administrator password field.
  9. Trigger the workflow run and wait until the execution completes.
  10. Once the workflow execution is completed you will see a summary of information for the update certificate operation in the workflow execution log.
  11. Close the Cloud Account if it is open in the UI, then Validate the health of the Cloud Account. This may take up to 10 minutes to complete the data collection.

If this did not resolve the issue then please contact Broadcom Support and reference Knowledge Article ID 318756 in the problem description. For more information, see How to Submit a Support Request.

Additional Information

Due to repeated attempts to validate the cloud account at an invalid state, the cloud account may be marked 'SUSPENDED' in the Aria Automation. 
Refer: Error while collecting data from vCenter cloud account on Aria Automation 8.x

If the Update Cloud Account Certificate workflow fails with the below error message then the service account used in Step 3 of the above procedure may not have the Assembler Administrator role. 

ERROR HTTP 403 - Forbidden : {"message":"forbidden","statusCode":403,"errorCode":0,"serverErrorId":"<UUID>","documentKind":"com:vmware:xenon:common:ServiceErrorResponse"}

To resolve the issue. assign the Assembler Administrator role from the Identity and Access management and re-run the workflow.

Attachments

net.broadcom.vra.update.ca.certificate.package get_app
Powered by Wolken Software