Exporting and importing vCenter certificates into SRM Appliance OR Unable to pair SRM sites - SRM server XX cannot validate SSL certificate from server at YY. The remote host certificate has these problems: Unknown SSL certificate error
Article ID: 312683
Updated On:
Products
Issue/Introduction
1. Unable to pair sites / You get an error “unknown SSL certificate error” in the SRM GUI when pairing SRM sites.
2. Unable to connect to Site Recovery Manager Server at https://srmpr.vr.local:443/drserver/vcdr/vmomi/sdk. Reason: java.net.SocketTimeoutException: 30,000 milliseconds timeout on connection http-outgoing-496 [ACTIVE]
3. Site pairing is not working after replacing Machine SSL certs.
4. PSC convergence can also lead to this error
VMware-dr.log :
2023-01-30T18:40:35.555Z warning vmware-dr[02656] [SRM@6876 sub=IO.Connection] Failed to SSL handshake; SSL(<io_obj p:0x00007efbd8010d98, h:38, <TCP '10.200.201.150 : 48042'>, <TCP '10.200.202.244 : 443'>>), e: 336134278(certificate verify failed), duration: 15msec
2023-01-30T18:40:35.555Z warning vmware-dr[02656] [SRM@6876 sub=HttpConnectionPool-000001] Failed to get pooled connection; <cs p:00000000024fd7a0, TCP:vcenter.prod.org:443>, SSL(<io_obj p
:0x00007efbd8010d98, h:38, <TCP '10.200.201.150 : 48042'>, <TCP '10.200.202.244 : 443'>>), duration: 25msec, N7Vmacore3Ssl18SSLVerifyExceptionE(SSL Exception: Verification parameters:
--> PeerThumbprint: D0:6B:B9:A0:8B:F3:F1:24:FA:D3:BF:BD:D4:79:8F:90:65:95:02:80:D5:92:FE:32:F8:19:6F:65:0D:27:19:76
--> ExpectedThumbprint: 6D:BE:63:05:20:9B:D1:76:2E:1F:C4:90:46:57:AC:B2:48:1D:C6:EF:EC:86:E0:D3:9D:8B:FA:C3:90:E9:00:62
--> ExpectedPeerName: vcenter.prod.org
--> The remote host certificate has these problems:
-->
--> * unable to get local issuer certificate)
--> [context]zKq7AVECAAQAAFE5JgEKdm13YXJlLWRyAABN3BxsaWJ2bWFjb3JlLnNvAADBDzAAKNIvADfWLwAv5jEAHWkxAHKFMQBeuUIBh38AbGlicHRocmVhZC5zby4wAALvNQ9saWJjLnNvLjYA[/context]
2023-01-30T18:40:35.557Z info vmware-dr[02656] [SRM@6876 sub=IO.Http] Set user agent error; state: 1, (null), N7Vmacore3Ssl18SSLVerifyExceptionE(SSL Exception: Verification parameters:
--> PeerThumbprint: D0:6B:B9:A0:8B:F3:F1:24:FA:D3:BF:BD:D4:79:8F:90:65:95:02:80:D5:92:FE:32:F8:19:6F:65:0D:27:19:76
--> ExpectedThumbprint: 6D:BE:63:05:20:9B:D1:76:2E:1F:C4:90:46:57:AC:B2:48:1D:C6:EF:EC:86:E0:D3:9D:8B:FA:C3:90:E9:00:62
--> ExpectedPeerName: vcenter.prod.org
--> The remote host certificate has these problems:
-->
--> * unable to get local issuer certificate)
--> [context]zKq7AVECAAQAAFE5JgEKdm13YXJlLWRyAABN3BxsaWJ2bWFjb3JlLnNvAADBDzAAKNIvADfWLwAv5jEAHWkxAHKFMQBeuUIBh38AbGlicHRocmVhZC5zby4wAALvNQ9saWJjLnNvLjYA[/context]
2023-01-30T18:40:35.558Z error vmware-dr[02656] [SRM@6876 sub=IO.Http] User agent failed to send request; (null), N7Vmacore3Ssl18SSLVerifyExceptionE(SSL Exception: Verification parameters:
--> PeerThumbprint: D0:6B:B9:A0:8B:F3:F1:24:FA:D3:BF:BD:D4:79:8F:90:65:95:02:80:D5:92:FE:32:F8:19:6F:65:0D:27:19:76
--> ExpectedThumbprint: 6D:BE:63:05:20:9B:D1:76:2E:1F:C4:90:46:57:AC:B2:48:1D:C6:EF:EC:86:E0:D3:9D:8B:FA:C3:90:E9:00:62
--> ExpectedPeerName: vcenter.prod.org
--> The remote host certificate has these problems:
-->
--> * unable to get local issuer certificate)
--> [context]zKq7AVECAAQAAFE5JgEKdm13YXJlLWRyAABN3BxsaWJ2bWFjb3JlLnNvAADBDzAAKNIvADfWLwAv5jEAHWkxAHKFMQBeuUIBh38AbGlicHRocmVhZC5zby4wAALvNQ9saWJjLnNvLjYA[/context]
2023-01-30T18:40:35.563Z warning vmware-dr[02666] [SRM@6876 sub=Default connID=lkp-admin-2854] StubExcTranslator : Error while calling stub for 'lookup.ServiceInstance:ServiceInstance'
--> N7Vmacore3Ssl18SSLVerifyExceptionE SSL Exception: Verification parameters:
--> PeerThumbprint: D0:6B:B9:A0:8B:F3:F1:24:FA:D3:BF:BD:D4:79:8F:90:65:95:02:80:D5:92:FE:32:F8:19:6F:65:0D:27:19:76
--> ExpectedThumbprint: 6D:BE:63:05:20:9B:D1:76:2E:1F:C4:90:46:57:AC:B2:48:1D:C6:EF:EC:86:E0:D3:9D:8B:FA:C3:90:E9:00:62
--> ExpectedPeerName: vcenter.prod.org
--> The remote host certificate has these problems:
-->
--> * unable to get local issuer certificate
--> [context]zKq7AVECAAQAAFE5JgEKdm13YXJlLWRyAABN3BxsaWJ2bWFjb3JlLnNvAADBDzAAKNIvADfWLwAv5jEAHWkxAHKFMQBeuUIBh38AbGlicHRocmVhZC5zby4wAALvNQ9saWJjLnNvLjYA[/context]
--> [backtrace begin] product: VMware vCenter Site Recovery Manager, version: 8.5.0, build: build-19282257, tag: vmware-dr, cpu: x86_64, os: linux, buildType: release
--> backtrace[03] libvmacore.so[0x001CDC4D]
--> backtrace[04] libvmacore.so[0x00300FC1]
--> backtrace[05] libvmacore.so[0x002FD228]
--> backtrace[06] libvmacore.so[0x002FD637]
--> backtrace[07] libvmacore.so[0x0031E62F]
--> backtrace[08] libvmacore.so[0x0031691D]
--> backtrace[09] libvmacore.so[0x00318572]
--> backtrace[10] libvmacore.so[0x0042B95E]
--> backtrace[11] libpthread.so.0[0x00007F87]
--> backtrace[12] libc.so.6[0x000F35EF]
--> [backtrace end]
2023-01-30T18:40:35.564Z warning vmware-dr[02666] [SRM@6876 sub=Default connID=lkp-admin-2854] Unrecognized SSL certificate error flags: 0x0000000008000000
2023-01-30T18:40:35.564Z warning vmware-dr[02666] [SRM@6876 sub=RemoteSite.RemoteLkpServer connID=lkp-admin-2854] Failed to connect:
--> (dr.fault.CertificateNotTrustedByDr) {
--> faultCause = (dr.fault.CertificateUnknownError) {
--> faultCause = (vmodl.MethodFault) null,
--> faultMessage = <unset>,
--> name = "vSphereSRM2",
--> uuid = "78170aab-3fed-4894-a60d-84eab2c5d1e7",
--> address = "vcenter.prod.org",
--> port = "443",
--> reason = (vmodl.MethodFault) null
--> msg = ""
--> },
--> faultMessage = <unset>,
--> name = "vSphereSRM2",
--> uuid = "78170aab-3fed-4894-a60d-84eab2c5d1e7",
--> address = "vcenter.prod.org",
--> port = "443",
--> reason = (vmodl.MethodFault) null
--> msg = ""
--> }
--> [context]zKq7AVECAAQAAFE5JgEOdm13YXJlLWRyAABN3BxsaWJ2bWFjb3JlLnNvAAEOc2ZsaWJkci10eXBlcy5zbwABcqdmAjOZA2xpYmRyLXZtb21pLnNvAAPUbQZsaWJjb25uZWN0aW9uLWJhc2Uuc28ABAfSHWxpYmNvbm5lY3Rpb24tcHNjLnNvAAMsVRADaX0QBRonW2xpYmRyLXJlY292ZXJ5LnNvAAAdaTEAcoUxAF65QgaHfwBsaWJwdGhyZWFkLnNvLjAAB+81D2xpYmMuc28uNgA=[/context]
--> [backtrace begin] product: VMware vCenter Site Recovery Manager, version: 8.5.0, build: build-19282257, tag: vmware-dr, cpu: x86_64, os: linux, buildType: release
--> backtrace[03] libvmacore.so[0x001CDC4D]
--> backtrace[04] libdr-types.so[0x0066730E]
--> backtrace[05] libdr-types.so[0x0066A772]
--> backtrace[06] libdr-vmomi.so[0x00039933]
--> backtrace[07] libconnection-base.so[0x00066DD4]
--> backtrace[08] libconnection-psc.so[0x001DD207]
--> backtrace[09] libconnection-base.so[0x0010552C]
--> backtrace[10] libconnection-base.so[0x00107D69]
--> backtrace[11] libdr-recovery.so[0x005B271A]
--> backtrace[12] libvmacore.so[0x0031691D]
--> backtrace[13] libvmacore.so[0x00318572]
--> backtrace[14] libvmacore.so[0x0042B95E]
--> backtrace[15] libpthread.so.0[0x00007F87]
--> backtrace[16] libc.so.6[0x000F35EF]
--> [backtrace end]
Cause
SRM server is not having the new vCenter root certificates in its certificate store that either changed post replacing vCenter certificates or post PSC convergence.
Resolution
Please follow the instructions below to save certificates from vCenter Windows/Appliance & import into SRM appliance.
1. Right click on Download trusted root CA certificates & Save Link as..
3. You may be required to use Windows or Linux certificates depending on the host OS you are importing it to. For SRM appliance, we will be using linux certificates.
For Enhanced Link Mode vCenter
Using WinSCP copy the "lin" certificate folder to /home/admin/ directory in the SRM appliance.
a.Login to SRM appliance as root and list the contents of /home/admin/
root@srmpr [ /home/admin ]# ls
lin
b.Change directory to lin
root@srmpr [ /home/admin ]# cd lin
c.Copy all the files in folder ''lin" to /etc/ssl/certs/ directory by running the command cp *.* /etc/ssl/certs/ and follow the steps from 4.
For Standalone vCenter (Not in linked mode)
Using WinSCP copy the contents of both vCenters "lin" folder into the folder created as common under /home/admin directory in the SRM appliance.
a. Login to SRM appliance as root and list the contents of /home/admin/
root@srmpr [ /home/admin ]# ls
common
b.Change directory to common
root@srmpr [ /home/admin ]# cd common
c.Copy all the files in folder ''common" to /etc/ssl/certs/ directory by running the command cp *.* /etc/ssl/certs/ and follow the steps from 4.
4. To modify the certificates' permissions, run the following command - chmod a+r /etc/ssl/certs/*
5. Run - c_rehash
NOTE: c_rehash command does not exist in SRM 8.8 and higher versions. Please run '/usr/bin/rehash_ca_certificates.sh'
6. Reboot the appliance
7. Reconfigure the appliance
8. Reconnect site pair.
Workaround:
c_rehash scans directories and calculates a hash value of each .pem, .crt, .cer, or .crl file in the specified directory list and creates symbolic links for each file, where the name of the link is the hash value.
Feedback
