VMware Site Recovery not connecting | A general system error occurred: N7Vmacore3Ssl18SSLVerifyExceptionE SSL Exception: Verification parameters | unable to get local issuer certificate | Certificate thumbprint mismatch
Article ID: 391926
Updated On:
Products
Issue/Introduction
Symptoms:
-
When the SSL certificate on the vCenter is updated, thumbprint changes.
-
Unable to reconfigure the SRM due to SSL certificate change, takes long time and times out to login page.
-
Manually started srm-server service from VAMI, gives an error as :
A general system error occurred:N7Vmacore3Ssl18SSLVerifyExceptionESSL Exception: Verification parameters:PeerThumbprint:01:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:89:AB:CD:EFExpectedThumbprint: FE:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:76:54:32:10ExpectedPeerName: vmwarexx.local.com. The remote host certificate has these problems:unable to get local issuer certificate.Operation ID: #######-####-####-####-##########
Validation:
In SRM, /opt/vmware/support/logs/srm/vmware-dr.log, log indicates certificate mismatch problem:
2024-04-08T13:44:57.599-07:00 info vmware-dr[92978] [SRM@6876 sub=IO.Http] Set user agent error; state: 1, (null), N7Vmacore3Ssl18SSLVerifyExceptionE(SSL Exception: Verification parameters:--> PeerThumbprint: 01:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:89:AB:CD:EF--> ExpectedThumbprint: FE:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:76:54:32:10--> ExpectedPeerName: vmwareVC01.local.com--> The remote host certificate has these problems:-->--> * unable to get local issuer certificate)--> [context]zKq7AVECAAQAAMX4YgELdm13YXJlLWRyAADM6xtsaWJ2bWFjb3JlLnNvAACkbjMAmV0zAGkEMwAgIDMA9iEzAN5INQDiYTUAsItKAbCOAGxpYnB0aHJlYWQuc28uMAAC7/oPbGliYy5zby42AA==[/context]2024-04-08T13:44:57.601-07:00 error vmware-dr[92978] [SRM@6876 sub=IO.Http] User agent failed to send request; (null), N7Vmacore3Ssl18SSLVerifyExceptionE(SSL Exception: Verification parameters:--> PeerThumbprint:01:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:89:AB:CD:EF--> ExpectedThumbprint:FE:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:##:76:54:32:10--> ExpectedPeerName: vmwarexx.local.com
--> The remote host certificate has these problems:
--> * unable to get local issuer certificate)xx
Environment
VMware Site Recovery Manager 8.x
VMware Site Recovery Manager 9.x
vSphere Replication 8.x
vSphere Replication 9.x
Cause
This occurs when the vCenter(s) certificate(s) are changed. The reconfiguration handshake fails, and the new thumbprint cannot be registered due to a mismatch. This issue is often caused by incorrect thumbprints in the SRM database tables.
Resolution
1. Power cycle SRM and VRMS (Power off then power on, not reboot)
2. Reconfigure SRM and VRMS via the VAMI interface on port 5480 (One at a time)
3. Open site recovery and make sure it is connected. If not, perform a 'Reconnect'.
4. If you still see the issue, run VMware lsdoctor tool lsdoctor on the vCenter Server to identify the SSLThumbprint mismatch error.
WARNING
Before using lsdoctor to make any changes, ensure you have taken proper snapshots of your SSO domain. This means that you must shut down all VCs or PSCs that are in the SSO domain at the same time, then snapshot them, and power them on again. If you need to revert to one of these snapshots, shut all the nodes down, and revert all nodes to the snapshot. Failure to perform these steps will lead to replication problems across the PSC databases.
First run “python lsdoctor.py -l” to check for common issues in the lookup service. Does not make any changes to the environment. This will show issues found on any node in the SSO domain. See output for findings and path to JSON report.
Output will be similiar to below -
root@vcenter [ ~/lsdoctor-240201 ]# python lsdoctor.py -l
ATTENTION: You are running a reporting function. This doesn't make any changes to your environment.
You can find the report and logs here: /var/log/vmware/lsdoctor
2025-01-30T09:13:13 INFO main: You are reporting on problems found across the SSO domain in the lookup service. This doesn't make changes.
2025-01-30T09:13:13 INFO live_checkCerts: Checking services for trust mismatches...
2025-01-30T09:13:13 INFO generateReport: Listing lookup service problems found in SSO domain
2025-01-30T09:13:13 INFO generateReport: No issues detected in the lookup service entries for vCenter.vmware.com (VC 7.0 or CGW).
2025-01-30T09:13:13 INFO generateReport: No issues detected in the lookup service entries for vspherereplication.vmware.com (vSphere Replication).
2025-01-30T09:13:13 INFO generateReport: No issues detected in the lookup service entries for srmserver.vmware.com (SRM).
2025-01-30T09:13:13 ERROR generateReport: default-first-site\srmserver.vmware.com (SRM) found SSL Trust Mismatch: Please run python ls_doctor.py --trustfix option on this node.
2025-01-30T09:13:13 INFO generateReport: Report generated: /var/log/vmware/lsdoctor/vCenter.vmware.com-2025-01-30-091313.json
If the above output reports SSL trust mismatch errors, run the command python lsdoctor.py -t. This option resolves SSL trust mismatch issues in the lookup service.
The lookup service registrations may have an SSL trust value that doesn’t match the MACHINE_SSL_CERT on port 443 of the node. This can be caused by a failure during certificate replacement, among other failures.
root@vCenter [ ~/lsdoctor-240201 ]# python lsdoctor.py -t
WARNING: This script makes permanent changes. Before running, please take *OFFLINE* snapshots
of all VC's and PSC's at the SAME TIME. Failure to do so can result in PSC or VC inconsistencies.
Logs can be found here: /var/log/vmware/lsdoctor
2025-01-30T09:16:04 INFO main: You are checking for and fixing SSL trust mismatches in the local SSO site. NOTE: Please run this script one PSC or VC per SSO site.
Have you taken offline (PSCs and VCs powered down at the same time) snapshots of all nodes in the SSO domain or supported backups?[y/n]Y
Provide password for [email protected]:
2025-01-30T09:16:30 INFO __init__: Retrieved services from SSO site: Default-First-Site
2025-01-30T09:16:30 INFO findAndFix: Checking services for trust mismatches...
2025-01-30T09:16:31 INFO findAndFix: Attempting to reregister 738b033e-XXXX-XXXX-XXXX-f18b745a996a for srmserver.vmware.com
2025-01-30T09:16:32 INFO findAndFix: Attempting to reregister h5-dr-738b033e-XXXX-XXXX-XXXX-f18b745a996a for srmserver.vmware.com
2025-01-30T09:16:32 INFO findAndFix: We found 2 mismatch(s) and fixed them :)
2025-01-30T09:16:32 INFO main: Please restart services on all PSC's and VC's when you're done.
If the issue persists, please Contact Broadcom Support
Feedback
