vRealize Automation 7.x certificate process overview, common issues, and troubleshooting guidance
search cancel

vRealize Automation 7.x certificate process overview, common issues, and troubleshooting guidance

book

Article ID: 325853

calendar_today

Updated On:

Products

VMware Aria Suite

Issue/Introduction

This article contains step by step instructions to update and/or replace all certificates in vRealize Automation 7.x with additional troubleshooting guidance and other known certificate product issues that may need to be corrected before continuing with product certificate updates.

See Updating vRealize Automation Certificates for additional information.

Note: End of General Support for VMware vRealize Automation 7.x

Environment

VMware vRealize Automation 7.x

Resolution

Prerequisites

Accounts and Credentials

  • root account for each vRA appliance
  • [email protected]
  • Domain / Local administrator access to each Windows domain joined IaaS component node

Files or Utilities

  • openssl
  • External or Internal CA to generate certificates
  • Access to any external connector appliances if using Smart Card authentication

Common expired certificate troubleshooting commands

  • To be used when manually validating the expiry of
    • /storage/db/root.crt
    • /storage/db/pgdata/server.crt
    • /etc/apache2/server.pem
    • /opt/vmware/etc/lighttpd/server.pem
openssl x509 -in /storage/db/pgdata/server.crt -text -noout

Known Issues and Additional Notes

  • If you are presented with HSTS warnings within your client browser when accessing any vRO or vRA web interface, it is recommended to secure the virtual appliances with publicly trusted certificates. Several public CAs offer free automatically renewable certificates.
  • Error:  connect: Connection refused seen when logging into the vRA Appliance management interface vRA > Certificates.
    • To be used when manually updating a vRealize Automation 7.x IaaS Manager Service VMPS website certificate when not co-located with an IaaS Web component 
      netsh http show sslcert
netsh http update sslcert ipport=0.0.0.0:443 certhash=thumbprint appid='{xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx}'

Note: Update the certhash and appid values to match your environment.

  • *.cer and *.crt are pem formatted files with a different extension, one that is recognized by Windows Explorer as a certificate.
Note: *.crt and *.pem files extensions can be swapped to open the certificates in Windows

Smart Cards with external vIDM Connectors

  • If using Smart Card authentication, the following symptoms may be seen after replacing vRA certificates
    • vRA application logs for the Event broker component, labeled as cafe:event-broker, located at /var/log/vmware/vcac/catalina.out contains messages similar to
com.vmware.vcac.eventlog.auditing.saveEvent:90 - Request to vCO failed.  Error: 400
  • vRO logins fail with messages similar to
The provided credentials are not valid.
 
  • The workflow in vRO Update a vRA host / Update the Host fails with a message similar to 
    Can not update the vRA host.

Updating & replacing all vRA 7.x certificates

  1. Validate the internal vPostgres certificate is not expired:  vRA 7.x APIs return error 500 when connecting to Postgres DB
    Note: The above issue will prevent VAMI operations from completing as expected.  Ensure this is not impacting your environment before moving forward as the proceeding commands will fail otherwise.
    Note: Do not copy or store manually backed up configuration files within /storage/db or /storage/db/pgdata.
  2. If ManagementAgent is about to expire when logged into the vRA VAMI vRA > Cluster tab: Replace a Management Agent Certificate
  3. Validate solution users in IaaS are not expired:  At least one security token in the message could not be validated reported in Proxy Agent logfiles
  4. Replace Certificates in the vRealize Automation Appliance
  5. Replace the Infrastructure as a Service Certificate
  6. Replace the IaaS Manager Service Certificate
    1. See the Virtual Machine Template section under Updating vRealize Automation Certificates for additional information on Guest Agent and Software Agent certificate update issues if templates are not updated.
Note:  This information is called out here:

The trust command syntaxes shown herein are representative rather than definitive. While they are appropriate for most typical deployments, there may be situations in which you need to experiment with variations on the commands.

  • If you specify --certificate you must provide the path to a valid certificate file in PEM format.
  • If you specify --uri, you must provide the uri from which the command can fetch a trusted certificate.
  • If you specify the --registry-certificate option, you indicate that the requested certificate should be treated as the certificate for the component registry and the trusted certificate is added to the truststore under a specific alias used by the component registry certificate.
  1. cp /etc/apache2/server.pem /opt/vmware/etc/lighttpd/server.pem && service vami-lighttp restart && service haproxy restart
    Note:  The above will implement Step #8 and the documentation steps as one command.
  2. Test accessing each appliance web interfaces over
    • VAMI over port 5480
    • vRA tenant portals over 443
    • vRO Control Center over port 8283
    • HTML5 vRO Client over port 8283
  3. All web services should now share the same certificate thumbprint or be fully trusted with updated certificates.
  4. Move onto the next section if using external vIDM connectors for Smart Card authentication.

Smart Card Configurations

  1. Copy the vIDM connector certificate in pem format to the /tmp directory on the vRA appliance(s).
  2. Trust the external connector appliance certificates by running the below command 
    /usr/local/horizon/scripts/installExternalSslRootCA.hzn --ca /path/to/certchain.pem --alias connector-root

Additional Information

If embedded vRO issues persist on a vRA appliance see: Resetting the Embedded vRealize Orchestrator 7.x configuration on a vRealize Automation 7.x appliance

For additional guidance in generating a signing request and Signing vRA certificates using an internal Microsoft CA signing authority.

Video guidance on generating a valid certificate signing request from custom configuration files with subject alternative name entries:  vRealize 7.x – Replacing vRA Certificates

HSTS is HTTP Strict Transport Security: a way for sites to elect to always use HTTPS. See https://www.chromium.org/hsts.

Powered by Wolken Software