Replication failing with error : No connection to VR server and no access to datastore path.

Products

VMware Live Recovery

Issue/Introduction

Symptoms:

After changing the VR certificate to CA signed VMs are in RPO violation with error: "No host can be used to access the datastore path.vmdk"

Environment

vSphere Replication 9.x

Cause

The cause of the connection failure is a Certificate Thumbprint Mismatch within the vSphere Replication database.

When the VR certificate was replaced with a CA-signed version, the appliance's unique "digital fingerprint" changed. However, the internal database tables (hbrserverentity and hbrserverinfoentity) were not updated with this new thumbprint.

Analysis of the hms.log confirms that the connection failed because the HMS service attempted to push a stale SSL thumbprint from its database to the ESXi host; this mismatch triggered a communication timeout and caused the host to reject the secure handshake.

2026-03-02 04:21:13.500 DEBUG com.vmware.hms.monitor.hostEnableHostOnHbrHelper [hms-config-host-at-hbr-thread-105] (..monitor.host.EnableHostOnHbrHelper) [operationID=8d909fe8-e07b-4ce5-834d-1e59d10ce1e6-HMSINT-20767681] | putThumbprintsIntoHost host:host-###### <- [EB:FE:AC:04:92:##:##:##:##:##:##:##:##:7D:0E:B4:5E:59:D5:B7:36:21:F5:D8:75:95:E8:9F:1B:9E:33:04]
2026-03-02 04:26:13.717 ERROR com.vmware.hms.monitor.hostEnableHostAtHbrTaskRunner [hms-config-host-at-hbr-thread-105] (..monitor.host.EnableHostOnHbrHelper) [operationID=8d909fe8-e07b-4ce5-834d-1e59d10ce1e6-HMSINT-20767681] | Failed to enable hostesxi_hostname(host-######) for addresses [esxi_ip], using NICs [management.key-vim.host.VirtualNic-vmk0].
com.vmware.vim.vmomi.client.exception.ConnectionException: https://vr_hostname:8123/ invocation failed with "java.net.SocketTimeoutException: Read timed out"
2026-03-02 04:26:13.718 ERROR com.vmware.hms.monitor.hostEnableHostAtHbrTaskRunner [hms-config-host-at-hbr-thread-105] (..jvsl.util.Slf4jUtil) [operationID=8d909fe8-e07b-4ce5-834d-1e59d10ce1e6-HMSINT-20767681] | Failed to enable host esxi_hostname(host-######) on any NIC in VR server vr_hostname(524b468b-4964-3374-a58e-17aabf9cdcf8).
com.vmware.hms.monitor.host.HostCannotBeEnabledException: Failed to enable host esxi_hostname(host-######) on any NIC in VR server vr_hostname(524b468b-4964-3374-a58e-17aabf9cdcf8).
2026-03-02 04:26:13.718 ERROR com.vmware.hms.monitor.hostEnableHostAtHbrTaskRunner [hms-config-host-at-hbr-thread-105] (..host.task.EnableHostAtHbrTaskRunner) [operationID=8d909fe8-e07b-4ce5-834d-1e59d10ce1e6-HMSINT-20767681] | Error while enabling host host-###### in VR Server 127.0.0.1
com.vmware.hms.monitor.host.HostCannotBeEnabledException: Failed to enable host esxi_hostname(host-######) on any NIC in VR server vr_hostname(524b468b-4964-3374-a58e-17aabf9cdcf8).

Analysis of the hbrsrv.log confirms that the connection failed because the vSphere Replication server was unable to authenticate with the ESXi host; the logs show a vim.fault.NoClientCertificate followed by a vim.fault.InvalidLogin, which indicates the host rejected the handshake because the presented certificate did not match the stale thumbprint metadata stored in the database.

Mar 02 05:27:45 vr_hostname hbrsrv[3352]: 2026-03-02T05:27:45.651Z info hbrsrv[03414] [Originator@6876 sub=vmomi.soapStub[27] opID=c0a7e065-5d2f-4b73-b2c8-0e2482dd7728] SOAP request returned HTTP failure; <SSL(<io_obj p:0x00007f808400ec30, h:59, <TCP 'vr_ipadd : 38154'>, <TCP 'esxi_ip : 443'>>), /sdk>, method: loginBySSLThumbprint; code: 500(Internal Server Error); fault: (vim.fault.NoClientCertificate) {
Mar 02 05:27:45 vr_hostname hbrsrv[3352]: --> faultCause = (vmodl.MethodFault) null,
Mar 02 05:27:45 vr_hostname hbrsrv[3352]: --> faultMessage = <unset>
Mar 02 05:27:45 vr_hostname hbrsrv[3352]: --> msg = "Received SOAP response fault from [<SSL(<io_obj p:0x00007f808400ec30, h:59, <TCP 'vr_ipadd : 38154'>, <TCP 'esxi_ip : 443'>>), /sdk>]: loginBySSLThumbprint
Mar 02 05:27:45 vr_hostname hbrsrv[3352]: --> Client connected without supplying a certificate."
Mar 02 05:27:45 vr_hostname hbrsrv[3352]: --> }
Mar 02 05:27:45 vr_hostname hbrsrv[3352]: 2026-03-02T05:27:45.651Z info hbrsrv[03414] [Originator@6876 sub=AgentConnection opID=c0a7e065-5d2f-4b73-b2c8-0e2482dd7728] Agent host-5917770/hostd: failed to log in. Connection type: /sdk
Mar 02 05:27:45 vr_hostname hbrsrv[3352]: 2026-03-02T05:27:45.651Z error hbrsrv[03414] [Originator@6876 sub=AgentConnection opID=c0a7e065-5d2f-4b73-b2c8-0e2482dd7728] Connection failed to agent host-######/hostd (esxi_ip): Can't login to the host
Mar 02 05:27:45 vr_hostname hbrsrv[3352]: 2026-03-02T05:27:45.651Z error hbrsrv[03414] [Originator@6876 sub=Main opID=c0a7e065-5d2f-4b73-b2c8-0e2482dd7728] HbrError stack:
Mar 02 05:27:45 vr_hostname hbrsrv[3352]: 2026-03-02T05:27:45.651Z error hbrsrv[03414] [Originator@6876 sub=Main opID=c0a7e065-5d2f-4b73-b2c8-0e2482dd7728] [0] Exception Vmacore::Exception: Can't login to the host
Mar 02 05:27:45 vr_hostname hbrsrv[3352]: 2026-03-02T05:27:45.651Z error hbrsrv[03414] [Originator@6876 sub=Main opID=c0a7e065-5d2f-4b73-b2c8-0e2482dd7728] [1] Connection failed for agent host-######/hostd:

Mar 02 05:28:45 vr_hostname hbrsrv[3352]: 2026-03-02T05:28:45.651Z info hbrsrv[04391] [Originator@6876 sub=Host opID=c0a7e065-5d2f-4b73-b2c8-0e2482dd7728] Heartbeat handler detected dead connection for agent: host-######/hostd
Mar 02 05:28:45 vr_hostname hbrsrv[3352]: 2026-03-02T05:28:45.652Z info hbrsrv[04391] [Originator@6876 sub=Main opID=c0a7e065-5d2f-4b73-b2c8-0e2482dd7728] HbrError stack:
Mar 02 05:28:45 vr_hostname hbrsrv[3352]: 2026-03-02T05:28:45.652Z info hbrsrv[04391] [Originator@6876 sub=Main opID=c0a7e065-5d2f-4b73-b2c8-0e2482dd7728] [0] Exception Vmacore::InvalidStateException: No connection (host-5917770/hostd)
Mar 02 05:28:45 vr_hostname hbrsrv[3352]: 2026-03-02T05:28:45.652Z info hbrsrv[04391] [Originator@6876 sub=Main opID=c0a7e065-5d2f-4b73-b2c8-0e2482dd7728] [1] Heartbeat failed (host-######/hostd)
Mar 02 05:28:45 vr_hostname hbrsrv[3352]: 2026-03-02T05:28:45.652Z info hbrsrv[04391] [Originator@6876 sub=Main opID=c0a7e065-5d2f-4b73-b2c8-0e2482dd7728] [2] Ignored error.
Mar 02 05:28:45 vr_hostname hbrsrv[3352]: 2026-03-02T05:28:45.652Z info hbrsrv[04391] [Originator@6876 sub=Main opID=c0a7e065-5d2f-4b73-b2c8-0e2482dd7728] HbrError stack:
Mar 02 05:28:45 vr_hostname hbrsrv[3352]: 2026-03-02T05:28:45.652Z info hbrsrv[04391] [Originator@6876 sub=Main opID=c0a7e065-5d2f-4b73-b2c8-0e2482dd7728] [0] Database object not found
Mar 02 05:28:45 vr_hostname hbrsrv[3352]: 2026-03-02T05:28:45.652Z info hbrsrv[04391] [Originator@6876 sub=Main opID=c0a7e065-5d2f-4b73-b2c8-0e2482dd7728] [1] Looking up global key in the database guestinfo.hbr.hbrsrv-certificate-revoked/guestinfo-cache
Mar 02 05:28:45 vr_hostname hbrsrv[3352]: 2026-03-02T05:28:45.652Z info hbrsrv[04391] [Originator@6876 sub=Main opID=c0a7e065-5d2f-4b73-b2c8-0e2482dd7728] [2] Couldn't read cached value for key guestinfo.hbr.hbrsrv-certificate-revoked
Mar 02 05:28:45 vr_hostname hbrsrv[3352]: 2026-03-02T05:28:45.652Z info hbrsrv[04391] [Originator@6876 sub=Main opID=c0a7e065-5d2f-4b73-b2c8-0e2482dd7728] [3] Ignored error.
Mar 02 05:28:45 vr_hostname hbrsrv[3352]: 2026-03-02T05:28:45.652Z info hbrsrv[04391] [Originator@6876 sub=AgentConnection opID=c0a7e065-5d2f-4b73-b2c8-0e2482dd7728] Agent host-5917770/hostd: restarting with address esxi_ip

Mar 02 05:28:48 vr_hostname hbrsrv[3352]: 2026-03-02T05:28:48.697Z info hbrsrv[04391] [Originator@6876 sub=vmomi.soapStub[175] opID=c0a7e065-5d2f-4b73-b2c8-0e2482dd7728] SOAP request returned HTTP failure; <SSL(SSL(<io_obj p:0x00007f8088022190, h:211, <TCP 'vr_ipadd : 33762'>, <TCP 'esxi_ip : 443'>>)), /sdk>, method: loginBySSLThumbprint; code: 500(Internal Server Error); fault: (vim.fault.InvalidLogin) {
Mar 02 05:28:48 vr_hostname hbrsrv[3352]: --> faultCause = (vmodl.MethodFault) null,
Mar 02 05:28:48 vr_hostname hbrsrv[3352]: --> faultMessage = <unset>
Mar 02 05:28:48 vr_hostname hbrsrv[3352]: --> msg = "Received SOAP response fault from [<SSL(SSL(<io_obj p:0x00007f8088022190, h:211, <TCP 'vr_ipadd : 33762'>, <TCP 'esxi_ip : 443'>>)), /sdk>]: loginBySSLThumbprint
Mar 02 05:28:48 vr_hostname hbrsrv[3352]: --> Cannot complete login due to an incorrect user name or password."
Mar 02 05:28:48 vr_hostname hbrsrv[3352]: --> }

Validate the thumbprint of the VR appliance by running the openssl command

root [ /etc/vmware ]# echo | openssl s_client -connect vr_ipadd:443 | openssl x509 -noout -fingerprint -sha256
depth=0 C = US, ST = California, L = Oakland, O = Blue Shield of California, OU = IT, CN = vr_hostname
verify error:num=20:unable to get local issuer certificate
verify return:1
depth=0 C = US, ST = California, L = Oakland, O = Blue Shield of California, OU = IT, CN = vr_hostname
verify error:num=21:unable to verify the first certificate
verify return:1
depth=0 C = US, ST = California, L = Oakland, O = Blue Shield of California, OU = IT, CN = vr_hostname
verify return:1
DONE
sha256 Fingerprint=A1:07:C1:C9:##:##:##:##:##:##:##:##:BE:C9:D0:A8:B5:C5:A1:4A:D4:27:80:35:C7:1E:DE:5E:04:DE:AA:49

Validated the hbrserverentity table and confirmed the thumbprint is incorrect and the hbrserverinfoentity table had the thumbprint missing.

vrmsdb=# select vsrv_address,replicationtrafficaddress,vsrv_thumbprint from hbrserverentity where vsrv_address='127.0.0.1';

-[ RECORD 1 ]-------------+------------------------------------------------------------------------------------------------
vsrv_address | 127.0.0.1
replicationtrafficaddress | vr_ipaddress
vsrv_thumbprint | EB:FE:AC:04:92:##:##:##:##:##:##:##:##:7D:0E:B4:5E:59:D5:B7:36:21:F5:D8:75:95:E8:9F:1B:9E:33:04

Resolution

If the symptoms and issue matches, please contact Broadcom Support to investigate the issue