Exporting and importing vCenter certificates into SRM Appliance OR Unable to pair SRM sites - SRM server XX cannot validate SSL certificate from server at YY. The remote host certificate has these problems: Unknown SSL certificate error
search cancel

Exporting and importing vCenter certificates into SRM Appliance OR Unable to pair SRM sites - SRM server XX cannot validate SSL certificate from server at YY. The remote host certificate has these problems: Unknown SSL certificate error

book

Article ID: 312683

calendar_today

Updated On:

Products

VMware Live Recovery VMware vCenter Server

Issue/Introduction

Symptoms:

1. Unable to pair sites /  You get  an error “unknown SSL certificate error”  in the SRM GUI when pairing SRM sites.

2. Unable to connect to Site Recovery Manager Server at https://srmpr.vr.local:443/drserver/vcdr/vmomi/sdk. Reason: java.net.SocketTimeoutException: 30,000 milliseconds timeout on connection http-outgoing-496 [ACTIVE]

3. Site pairing is not working after replacing Machine SSL certs.

4. PSC convergence can also lead to this error 

VMware-dr.log : 

2023-01-30T18:40:35.555Z warning vmware-dr[02656] [SRM@6876 sub=IO.Connection] Failed to SSL handshake; SSL(<io_obj p:0x00007efbd8010d98, h:38, <TCP '10.200.201.150 : 48042'>, <TCP '10.200.202.244 : 443'>>), e: 336134278(certificate verify failed), duration: 15msec
2023-01-30T18:40:35.555Z warning vmware-dr[02656] [SRM@6876 sub=HttpConnectionPool-000001] Failed to get pooled connection; <cs p:00000000024fd7a0, TCP:vcenter.prod.org:443>, SSL(<io_obj p
:0x00007efbd8010d98, h:38, <TCP '10.200.201.150 : 48042'>, <TCP '10.200.202.244 : 443'>>), duration: 25msec, N7Vmacore3Ssl18SSLVerifyExceptionE(SSL Exception: Verification parameters:
--> PeerThumbprint: D0:6B:B9:A0:8B:F3:F1:24:FA:D3:BF:BD:D4:79:8F:90:65:95:02:80:D5:92:FE:32:F8:19:6F:65:0D:27:19:76
--> ExpectedThumbprint: 6D:BE:63:05:20:9B:D1:76:2E:1F:C4:90:46:57:AC:B2:48:1D:C6:EF:EC:86:E0:D3:9D:8B:FA:C3:90:E9:00:62
--> ExpectedPeerName: vcenter.prod.org
--> The remote host certificate has these problems:
-->
--> * unable to get local issuer certificate)
--> [context]zKq7AVECAAQAAFE5JgEKdm13YXJlLWRyAABN3BxsaWJ2bWFjb3JlLnNvAADBDzAAKNIvADfWLwAv5jEAHWkxAHKFMQBeuUIBh38AbGlicHRocmVhZC5zby4wAALvNQ9saWJjLnNvLjYA[/context]
2023-01-30T18:40:35.557Z info vmware-dr[02656] [SRM@6876 sub=IO.Http] Set user agent error; state: 1, (null), N7Vmacore3Ssl18SSLVerifyExceptionE(SSL Exception: Verification parameters:
--> PeerThumbprint: D0:6B:B9:A0:8B:F3:F1:24:FA:D3:BF:BD:D4:79:8F:90:65:95:02:80:D5:92:FE:32:F8:19:6F:65:0D:27:19:76
--> ExpectedThumbprint: 6D:BE:63:05:20:9B:D1:76:2E:1F:C4:90:46:57:AC:B2:48:1D:C6:EF:EC:86:E0:D3:9D:8B:FA:C3:90:E9:00:62
--> ExpectedPeerName: vcenter.prod.org
--> The remote host certificate has these problems:
-->
--> * unable to get local issuer certificate)
--> [context]zKq7AVECAAQAAFE5JgEKdm13YXJlLWRyAABN3BxsaWJ2bWFjb3JlLnNvAADBDzAAKNIvADfWLwAv5jEAHWkxAHKFMQBeuUIBh38AbGlicHRocmVhZC5zby4wAALvNQ9saWJjLnNvLjYA[/context]
2023-01-30T18:40:35.558Z error vmware-dr[02656] [SRM@6876 sub=IO.Http] User agent failed to send request; (null), N7Vmacore3Ssl18SSLVerifyExceptionE(SSL Exception: Verification parameters:
--> PeerThumbprint: D0:6B:B9:A0:8B:F3:F1:24:FA:D3:BF:BD:D4:79:8F:90:65:95:02:80:D5:92:FE:32:F8:19:6F:65:0D:27:19:76
--> ExpectedThumbprint: 6D:BE:63:05:20:9B:D1:76:2E:1F:C4:90:46:57:AC:B2:48:1D:C6:EF:EC:86:E0:D3:9D:8B:FA:C3:90:E9:00:62
--> ExpectedPeerName: vcenter.prod.org
--> The remote host certificate has these problems:
-->
--> * unable to get local issuer certificate)
--> [context]zKq7AVECAAQAAFE5JgEKdm13YXJlLWRyAABN3BxsaWJ2bWFjb3JlLnNvAADBDzAAKNIvADfWLwAv5jEAHWkxAHKFMQBeuUIBh38AbGlicHRocmVhZC5zby4wAALvNQ9saWJjLnNvLjYA[/context]
2023-01-30T18:40:35.563Z warning vmware-dr[02666] [SRM@6876 sub=Default connID=lkp-admin-2854] StubExcTranslator : Error while calling stub for 'lookup.ServiceInstance:ServiceInstance'
--> N7Vmacore3Ssl18SSLVerifyExceptionE SSL Exception: Verification parameters:
--> PeerThumbprint: D0:6B:B9:A0:8B:F3:F1:24:FA:D3:BF:BD:D4:79:8F:90:65:95:02:80:D5:92:FE:32:F8:19:6F:65:0D:27:19:76
--> ExpectedThumbprint: 6D:BE:63:05:20:9B:D1:76:2E:1F:C4:90:46:57:AC:B2:48:1D:C6:EF:EC:86:E0:D3:9D:8B:FA:C3:90:E9:00:62
--> ExpectedPeerName: vcenter.prod.org
--> The remote host certificate has these problems:
-->
--> * unable to get local issuer certificate
--> [context]zKq7AVECAAQAAFE5JgEKdm13YXJlLWRyAABN3BxsaWJ2bWFjb3JlLnNvAADBDzAAKNIvADfWLwAv5jEAHWkxAHKFMQBeuUIBh38AbGlicHRocmVhZC5zby4wAALvNQ9saWJjLnNvLjYA[/context]
--> [backtrace begin] product: VMware vCenter Site Recovery Manager, version: 8.5.0, build: build-19282257, tag: vmware-dr, cpu: x86_64, os: linux, buildType: release
--> backtrace[03] libvmacore.so[0x001CDC4D]
--> backtrace[04] libvmacore.so[0x00300FC1]
--> backtrace[05] libvmacore.so[0x002FD228]
--> backtrace[06] libvmacore.so[0x002FD637]
--> backtrace[07] libvmacore.so[0x0031E62F]
--> backtrace[08] libvmacore.so[0x0031691D]
--> backtrace[09] libvmacore.so[0x00318572]
--> backtrace[10] libvmacore.so[0x0042B95E]
--> backtrace[11] libpthread.so.0[0x00007F87]
--> backtrace[12] libc.so.6[0x000F35EF]
--> [backtrace end]

2023-01-30T18:40:35.564Z warning vmware-dr[02666] [SRM@6876 sub=Default connID=lkp-admin-2854] Unrecognized SSL certificate error flags: 0x0000000008000000
2023-01-30T18:40:35.564Z warning vmware-dr[02666] [SRM@6876 sub=RemoteSite.RemoteLkpServer connID=lkp-admin-2854] Failed to connect:
--> (dr.fault.CertificateNotTrustedByDr) {
-->    faultCause = (dr.fault.CertificateUnknownError) {

-->       faultCause = (vmodl.MethodFault) null,
-->       faultMessage = <unset>,
-->       name = "vSphereSRM2",
-->       uuid = "78170aab-3fed-4894-a60d-84eab2c5d1e7",
-->       address = "vcenter.prod.org",
-->       port = "443",
-->       reason = (vmodl.MethodFault) null
-->       msg = ""
-->    },
-->    faultMessage = <unset>,
-->    name = "vSphereSRM2",
-->    uuid = "78170aab-3fed-4894-a60d-84eab2c5d1e7",
-->    address = "vcenter.prod.org",
-->    port = "443",
-->    reason = (vmodl.MethodFault) null
-->    msg = ""
--> }
--> [context]zKq7AVECAAQAAFE5JgEOdm13YXJlLWRyAABN3BxsaWJ2bWFjb3JlLnNvAAEOc2ZsaWJkci10eXBlcy5zbwABcqdmAjOZA2xpYmRyLXZtb21pLnNvAAPUbQZsaWJjb25uZWN0aW9uLWJhc2Uuc28ABAfSHWxpYmNvbm5lY3Rpb24tcHNjLnNvAAMsVRADaX0QBRonW2xpYmRyLXJlY292ZXJ5LnNvAAAdaTEAcoUxAF65QgaHfwBsaWJwdGhyZWFkLnNvLjAAB+81D2xpYmMuc28uNgA=[/context]
--> [backtrace begin] product: VMware vCenter Site Recovery Manager, version: 8.5.0, build: build-19282257, tag: vmware-dr, cpu: x86_64, os: linux, buildType: release
--> backtrace[03] libvmacore.so[0x001CDC4D]
--> backtrace[04] libdr-types.so[0x0066730E]
--> backtrace[05] libdr-types.so[0x0066A772]
--> backtrace[06] libdr-vmomi.so[0x00039933]
--> backtrace[07] libconnection-base.so[0x00066DD4]
--> backtrace[08] libconnection-psc.so[0x001DD207]
--> backtrace[09] libconnection-base.so[0x0010552C]
--> backtrace[10] libconnection-base.so[0x00107D69]
--> backtrace[11] libdr-recovery.so[0x005B271A]
--> backtrace[12] libvmacore.so[0x0031691D]
--> backtrace[13] libvmacore.so[0x00318572]
--> backtrace[14] libvmacore.so[0x0042B95E]
--> backtrace[15] libpthread.so.0[0x00007F87]
--> backtrace[16] libc.so.6[0x000F35EF]
--> [backtrace end]


Cause


SRM server is not having the new vCenter root certificates in its certificate store that either changed post replacing vCenter certificates or post PSC convergence. 

Resolution


Please follow the instructions below to save certificates from vCenter Windows/Appliance & import into SRM appliance. 

1. Right click on Download trusted root CA certificates & save it. 

 


 
2. Extract the certs zip folder 
 

image.png


3. You may be required to use Windows or Linux certificates depending on the host OS you are importing it to. For SRM appliance, we will be using linux certificates. 

4. Using WinSCP copy the "lin" certificate folder to /home/admin/ directory in the SRM appliance. If you have 2 individual vCenters (note in linked mode), copy the contents of both the vCenters "lin" folder into a common folder before copying it to the appliance. 

5. Login to SRM appliance as root and list the contents of /home/admin/

root@srmpr [ /home/admin ]# ls
lin

6. Copy all the files in folder ''lin" to /etc/ssl/certs/ directory by running the command - 

root@srmpr [ /home/admin/lin ]# mv * /etc/ssl/certs/

7. To modify the certificates' permissions, run the following command - chmod a+r /etc/ssh/certs/*

8. Run - c_rehash

NOTE: c_rehash command does not exist in SRM 8.8 and higher versions. Please run '/usr/bin/rehash_ca_certificates.sh'

9. Reboot the appliance 

10. Reconfigure the appliance 

11. Reconnect site pair. 


Workaround:

c_rehash scans directories and calculates a hash value of each .pem, .crt, .cer, or .crl file in the specified directory list and creates symbolic links for each file, where the name of the link is the hash value.

Additional Information


[Internal] "Failed to obtain MoRef/ServiceInstanceContent, context not available within timeout period" when starting SRM service (54729)
How to download and install vCenter Server root certificates to avoid Web Browser certificate warnings (2108294)
Pairing sites in SRM fails with the error: the host certificate chain is not complete (2117578)