Error: "java.net.ConnectException: Connection refused" when HMS service crashes in vSphere Replication
search cancel

Error: "java.net.ConnectException: Connection refused" when HMS service crashes in vSphere Replication

book

Article ID: 449517

calendar_today

Updated On:

Products

VMware Live Recovery

Issue/Introduction

Symptoms:

  • The HMS service fails to start or crashes intermittently.
  • Replications will be in Not Active state with below error message on the VM:

"A replication error occurred at the vSphere Replication Server for replication 'VM_NAME'. Details: 'No connection to VR Server for virtual machine on host [HOST_NAME] in cluster [CLUSTER] in datacenter:Unknown'"

  • In the SRM interface, ESXi hosts are listed as disconnected under Replication Servers

  • The Site Recovery UI displays:

"Unable to retrieve pairs from extension server at https://vr_hostname:8043.Unable to connect to HBR Management Server at https://vr_hostname:8043. Reason: https://vr_hostname:8043 invocation failed with "java.net.ConnectException: Connection refused"

  • The /heapdump partition may fill up, preventing replication operations.
  • Logs hms-stderr.log show java.lang.OutOfMemoryError: Java heap space

Jul 02, 2026 12:55:36 PM org.apache.catalina.loader.WebappClassLoaderBase clearReferencesThreads
WARNING: The web application [ROOT] appears to have started a thread named [hms-main-thread-8] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:

Jul 02, 2026 12:55:36 PM org.apache.coyote.http11.Http11NioProtocol destroy
INFO: Destroying ProtocolHandler ["https-jsse-nio-0.0.0.0-8043"]
Exception in thread "hms-main-thread-1" org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'com.vmware.jvsl.sessions.net.impl.PersistentConnection': Injection of resource dependencies failed; nested exception is org.springframework.beans.factory.BeanCreationNotAllowedException: Error creating bean with name 'pingScheduledThreadPool': Singleton bean creation not allowed while singletons of this factory are in destruction (Do not request a bean from a BeanFactory in a destroy method implementation!)

Jul 02, 2026 12:57:16 PM org.apache.coyote.http11.Http11NioProtocol start
INFO: Starting ProtocolHandler ["https-jsse-nio-0.0.0.0-8043"]
Exception in thread "hms-vlsi-client-thread-1052" java.lang.OutOfMemoryError: Java heap space

  • Validating the hms log confirms that the service attempts to communicate with the ESXi hosts and fails:

2026-08-13 04:39:08.690 INFO  com.vmware.jvsl.util.SingleThumbprintVerifier [hms-ping-scheduled-thread-9] (..jvsl.util.SingleThumbprintVerifier) [operationID=67c0b6a5-e80e-4103-852a-84daf61da462-HMSINT-10, operationID=f45a2a8d-a89c-4346-9ef4-31c0118df3b9-HMS-PING] | Thumbprint C7:FC:BF:0E:F0:44:87:CD:F9:58:##:##:##:##:##:##:##:##:##:##:##:12:9F:CE:D3:32:E7:A5:08 for host 10.##.###.## is trusted.
2026-08-13 04:39:08.693 ERROR com.vmware.hms.net.hbr.ping.svr.6a9b2f4b-ff9f-e811-0000-00000000005d [hms-ping-scheduled-thread-9] (..net.impl.VmomiPingConnectionHandler) [operationID=67c0b6a5-e80e-4103-852a-84daf61da462-HMSINT-10, operationID=f45a2a8d-a89c-4346-9ef4-31c0118df3b9-HMS-PING] | Ping for server 10.##.###.##:443/hbr for session: N/A failed:
com.vmware.vim.binding.vmodl.fault.SystemError: null

2026-08-13 04:39:20.571 ERROR com.vmware.hms.net.impl.hbr.HbrBrokerServiceImpl [hms-main-scheduled-thread-1] (..impl.hbr.HbrBrokerServiceImpl) [operationID=67c0b6a5-e80e-4103-852a-84daf61da462-HMSINT-10] | Failed to scheduleBrokerTokenVerificationSync for hbrsrvuw 10.##.###.##
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'com.vmware.jvsl.sessions.net.impl.PersistentConnection': Injection of resource dependencies failed; nested exception is org.springframework.beans.factory.BeanCreationNotAllowedException: Error creating bean with name 'pingScheduledThreadPool': Singleton bean creation not allowed while singletons of this factory are in destruction (Do not request a bean from a BeanFactory in a destroy method implementation!)

  • Validating the port connection using the curl command confirms that the hms service is crashing and rejecting connections:

root@vr_hostname[ / ]# curl -v telnet://127.0.0.1:8043
*   Trying 127.0.0.1:8043...
* connect to 127.0.0.1 port 8043 from 127.0.0.1 port 50098 failed: Connection refused
* Failed to connect to 127.0.0.1 port 8043 after 0 ms: Could not connect to server
* closing connection #0
curl: (7) Failed to connect to 127.0.0.1 port 8043 after 0 ms: Could not connect to server

root@vr_hostname[ / ]# curl -v telnet://127.0.0.1:8043
*   Trying 127.0.0.1:8043...
* Established connection to 127.0.0.1 (127.0.0.1 port 8043) from 127.0.0.1 port 35180
       

 

Environment

  • VMware vSphere Replication 9.x
  • VMware Live Recovery 9.x

Cause

  • This issue occurs when the proper procedure for decommissioning an ESXi host is not followed.
  • Removing or disconnecting an ESXi host from vCenter while replications remain active causes stale data to persist within the vSphere Replication database. The replication system is unable to clear this residual data, which leads the HMS service to consume excessive Java heap space and ultimately crash.

Resolution

If the symptoms and log entries described in this article match your environment, and the cause has been confirmed, contact Broadcom Support for assistance. The resolution requires validation of the environment and application of the appropriate corrective action.

Powered by Wolken Software