No resources present in the VMware Cloud Director Provider Portal after certificate update in multisite environment

Products

• VMware Cloud Director

Issue/Introduction

• After certificate expiration the provider and tenant portals will still load. When navigating through the UI there will be no resources, certificates, vCenters, Organizations, or Cloud Cells returned. Pages that would normally be populated will be blank.

• Multisite is configure in the environment.

• Organizations and cells are not visible in the Cloud Director UI.

• The UI remains in a "loading" state or displays blank pages for resources.

• Browser developer tools show failed fanout API calls with SSL validation errors.

• vcloud-container-debug.log contains similar errors: 

  javax.net.ssl.SSLHandshakeException: PKIX path building failed.

• Direct API calls will still return resources.

Environment

VMware Cloud Director 10.5.x

VMware Cloud Director 10.6.x

Cause

The issue is caused by a certificate trust mismatch. When a certificate is updated on a local site, the change is not automatically trusted by the other sites in the multisite association, or the local site's new certificate must be manually re-trusted within its own "Trusted Certificates" store for fanout API calls to succeed.

Resolution

To restore visibility of organizations and cells, import and trust the certificate within the Cloud Director Provider UI:

1. Log in to the Provider UI.
2. Navigate to Administration.
3. Under Settings, select Trusted Certificates.
4. Click Test Remote Connection.
5. Enter the Fully Qualified Domain Name (FQDN) of the local Cloud Director site.
6. Click Connect.
7. At the Trust On First Use (TOFU) prompt, review the certificate and click Trust.
8. Repeat these steps for any remote sites in the multisite association if they also report missing resources.

It may be necessary to delete and recreate the multi-site configuration:

1. In the provider portal access Administration.
2. From Administration go to Settings -> Multisite.
3. Save any details for the remote site present.
4. From the "Actions" drop down menu delete the remote site.
5. Refresh the page and check for objects to appear in the UI again.
6. Replace the cell certificates following Replace VMware Cloud Director Public/https Certificate, and re-add the remote site.

Additional Information

This behavior can also occur if the site is no longer deployed or available. Removing the site as detailed above will restore functionality.