Unable to register replication with vCenter when vCenter/PSC certs gets changed or after Convergence.

Products

• VMware Live Recovery
• VMware vSphere ESXi

Issue/Introduction

• Unable to re-register the replication appliance to vCenter when vCenter/PSC certs gets changed or after Convergence.


HMS Server Error

java.lang.RuntimeException: HMS Server failed to start successfully:

        at com.vmware.hms.App.main(App.java:117)

Caused by: com.vmware.vim.vmomi.client.exception.SslException: javax.net.ssl.SSLException: Certificate thumbprint mismatch, expected: FE:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:CF but encountered:9D:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:64

        at com.vmware.vim.vmomi.client.common.impl.ResponseImpl.setError(ResponseImpl.java:255)

        at com.vmware.vim.vmomi.client.http.impl.HttpExchange.run(HttpExchange.java:56)

dr.log
2020-07-26 22:16:49,923 [https-jsse-nio-0:0:0:0:0:0:0:0-8443-exec-2] ERROR com.vmware.srm.client.infrastructure.websso.WebSsoWorkflow axxxxx5-5xx5-4xx0-axx5-5xxxxxxxx3  - WebSso operation failed: com.vmware.vim.vmomi.client.exception.SslException: Failed to connect to SSO Server at  https://<vcenter-fqdn>/sso-adminserver/sdk/txxxxx.local. Reason: com.vmware.vim.vmomi.core.exception.CertificateValidationException: Server certificate chain is not trusted and thumbprint verification is not configured

 

Environment

VMware vSphere Replication 8.x
VMware vSphere Replication 6.x

Cause

The ovfEnv.xml and the hms config file  /opt/vmware/hms/conf/hms-configuration.xml  and also the replication HMS database will not get updated with the correct PSC/vCenter SSL thumbprints.

 

Resolution

Impact/Risks:
Please take a snapshot of replication appliance before these changes

Firstly run the lsdoctor tool.

Power down all VR SRM and VC nodes and take a snapshot.

Follow the steps in this article.  Using the 'lsdoctor' Tool

1. On the VR applinace get the SHA256 thumbprints of vCenter and PSCs 
   
   root@dxxxxxxx1 [ ~ ]# openssl s_client -connect <vcenter-fqdn>:443 2>/dev/null | openssl x509 -noout -fingerprint -sha256

SHA256 Fingerprint=8D:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:92
   
   
2. Manually update the correct/current certs thumbprint on ovfEnv.xml and the hms config file  /opt/vmware/hms/conf/hms-configuration.xml and also the replication HMS database.
    
   1. To check the details on VR database, log on to HMS database and run the below command to check the details.
      
      vrmsdb=# select * from vmomiserverentity;
vrmsdb=# select * from localvcentity;
   2. Update the correct SSL SH256 thumbprints
      
      Example:-
vrmsdb=# update vmomiserverentity set thumbprint='7A:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:AA' where dbid=X;

vrmsdb=# update localvcentity set thumbprint='31:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:DA' where dbid=X;

3. After updating the above, remove the service registrations  of replication from PSC and then reboot the appliance and then re-registers the VR back to vCenter/PSC to resolve the issue.

Additional Information