HCX Site Pairing is 'disconnected' post certificate renewal on Target HCX Cloud

Products

VMware HCX VMware Cloud on AWS

Issue/Introduction

HCX Site Pairing is down and it shows the error message below.
Host name '<HCX-Cloud_IP>' does not match the certificate subject provided by the peer (CN=hcx.sddc-###-###.vmwarevmc.com, O="VMware, Inc", L=Palo Alto, ST=California, C=US)
While the screenshot above is taken from a VMC-HCX deployment, this issue is not specific to VMC and may occur in any HCX environment.
The HCX Site Pairing is configured using an IP address in the "Remote HCX URL" field.

Environment

VMware HCX
HCX deployed on VMware Cloud on AWS

Cause

The HCX Manager certificate at the Cloud (Target) site was updated/replaced. The HCX Site Pairing is currently configured using an IP address in the "Remote HCX URL" field. However, the new certificate on the target HCX Manager uses a Fully Qualified Domain Name (FQDN) as its Common Name (CN), resulting in a mismatch.

Resolution

This issue can be resolved by following the steps outlined below. This procedure must be performed on your Source (On-Premises) HCX environment.

Click HCX 443 UI > Infrastructure > Site Pairs/Site Pairing
Click the "EDIT CONNECTION" OR "EDIT SITE PAIR" option in the existing Site Pairing.
Fill in the correct 'Username' and 'Password' and click the “EDIT” button.
Click the "IMPORT CERTIFICATE" button on the "Certificate Warning" popup.
Validate that the earlier warning/error is gone.

NOTE: In case the above does not work or you do not get a pop up to "IMPORT CERTIFICATE".
Kindly import the target HCX certificate manually into the source HCX environment. Refer : Importing Trusted Certificates from a Remote Site

If above does not work, you can export certificate manually and import it over HCX Manager 9443 UI:

1. Open Destination/Cloud HCX UI (not vCenter plugin) in the browser.
2. Inspect the certificate in the browser when HCX UI is open.
3. Go to details and export certificate, save it:
4. Go to HCX Connector UI on port 9443 and go to the Administration tab -> Trusted CA Certificate -> Import:
5. Select File, click browse and select the certificate file that was exported in step 3 and click on APPLY.
6. Once the certificate is imported, you can go back to HCX UI and edit Site Pairing and re-enter the credentials.

This solution should be regarded as a last option; however, it is generally unsuitable for the majority of clients who have existing.
If these steps do not resolve the issue, you may need to delete the Site Pairs (including the Service Mesh), and recreate the Site Pairing using the FQDN.

NOTE: Ensure you note the Service Mesh configuration details from Infrastructure > Interconnect so that you can recreate the mesh later.
Navigate to Infrastructure > Interconnect and select "Delete" for any Service Mesh that depends on the Site Pairing we need to replace
1. Removing the Existing Pairing (if necessary):
  - Log in to the HCX Manager (either on-premises or in the cloud).
  - Navigate to Infrastructure > Site Pairs.
  - Locate the site pair you want to recreate.
  - If you need to remove it, select the site pairing and choose the option to remove or disconnect it.
2. Creating a New Site Pairing:
  - Log in to the HCX Manager
  - Navigate to Infrastructure > Site Pairs.
  - Click Add a Site Pairing.
  - Enter the Remote HCX URL FQDN and credentials (username and password).

Additional Information

Importing Trusted Certificates from a Remote Site

If the Site Pairing is down, configuration workflows will fail and no migrations can be scheduled from HCX Connector or source Cloud Manager. Existing Network Extension services will remain active indefinitely but no configuration changes can be made on those, except for "unstretch", which can be forced from the target HCX Cloud Manager's side.
When creating Site Pairing, use the Target HCX FQDN in "Remote HCX URL" instead of an IP Address.