Errors during Re-balancing of replications between Hbrsrvuws

Products

• VMware Live Recovery

Issue/Introduction

Symptoms:

• Seeing multiple failed tasks in vCenter for one or more ESXI hosts:
  
  Task name : "Re-balancing replications between Hbrsrvuws".
Status : "There have been errors during vSphere Replication Servers loadbalancer execution, please refer The vSphere Replication Management Server log for detailed information

• Loadbalancer execution errors reported in hms logs:
  
  (in /opt/vmware/hms/logs/hms.log)
  

  YYYY-MM-DD HH:MM:SS.145 ERROR com.vmware.hms.hbrsrvuw.loadbalancer.HbrsrvuwLoadBalanceWorkflow [hms-main-thread-7] (..hbrsrvuw.loadbalancer.HbrsrvuwLoadBalanceWorkflow) [operationID=#1, task=HTID-########-####-####-####-############] | Error during Hbrsrvuw loadbalancer execution

  YYYY-MM-DD HH:MM:SS.147 DEBUG com.vmware.hms.phcollector.ops.VrOperationsSender [hms-main-thread-6] (..phcollector.ops.VrOperationsSender) [operationID=#1] | handleHmsTaskFinishedEvent[id: HTID-########-####-####-####-############; taskName: Re-balancing replications between hbrsrvuw's; taskManagedEntityId: null; taskTypeId: HbrsrvuwBalanceReplicationsTask; requestedExecutorId: hms-main; taskTag: null; autoAbort: true; queueTime: ##############; startTime: ###############; completeTime: null; error: null; success: false; result: null]

  YYYY-MM-DD HH:MM:SS.103 INFO  com.vmware.hms [hms-main-scheduled-thread-5] (..hms.util.HmsAsyncLock) [operationID=########-####-####-####-############-HMSINT-667] | (Owner: 63:AbstractGroup.java:321:lock,AbstractGroup.java:417:refreshCurrentIssues_aroundBody10(GID-########-####-####-####-#############), elapsed: 0 milliseconds) obtains the lock
YYYY-MM-DD HH:MM:SS.145 ERROR com.vmware.hms.hbrsrvuw.loadbalancer.HbrsrvuwLoadBalanceWorkflow [hms-main-thread-7] (..hbrsrvuw.loadbalancer.HbrsrvuwLoadBalanceWorkflow) [operationID=#1, task=HTID-########-####-####-####-############] | Error during Hbrsrvuw loadbalancer execution
com.vmware.vim.binding.hms.fault.ReplicationError: Replication error encountered at VR Server for replication '<VM-name>'. Details: 'Error for (groupId: "GID-########-####-####-####-#############"): Database file doesn't exist.; While performing distdb route for path '/vmfs/volumes/vsan:################-################/########-####-####-####-############/groupdb.db'; Code set to: No such group; While performing LoadGroupFromDatastore'.
        at jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(Unknown Source) ~[?:?]
        at java.lang.reflect.Constructor.newInstanceWithCaller(Unknown Source) ~[?:?]
        at java.lang.reflect.ReflectAccess.newInstance(Unknown Source) ~[?:?]
        at jdk.internal.reflect.ReflectionFactory.newInstance(Unknown Source) ~[?:?]
        at java.lang.Class.newInstance(Unknown Source) ~[?:?]

  
  YYYY-MM-DD HH:MM:SS.147 DEBUG com.vmware.hms.hbrsrvuw.HbrsrvuwMonitor [hms-main-thread-7] (..hms.hbrsrvuw.HbrsrvuwMonitor) [operationID=#1, task=HTID-########-####-####-####-############] | rebalance FAILED (HTID-########-####-####-####-############)
com.vmware.vim.binding.hms.fault.HbrLoadbalancerFault: null

  YYYY-MM-DD HH:MM:SS.149 DEBUG com.vmware.hms.util.taskupdates.TaskInfoUpdater [hms-main-thread-7] (..util.taskupdates.TaskInfoUpdater) [operationID=#1, task=HTID-########-####-####-####-############] | State update enqueued for HMS task 'HTID-########-####-####-####-############', State = error
2026-05-22 07:30:49.149 DEBUG com.vmware.hms.util.taskupdates.TaskInfoUpdater [hms-main-thread-7] (..util.taskupdates.TaskInfoUpdater) [operationID=#1, task=HTID-########-####-####-####-############] | Start processing updates for HMS task 'HTID-########-####-####-####-############'
2026-05-22 07:30:49.159 DEBUG com.vmware.hms.util.taskupdates.TaskInfoUpdater [hms-main-thread-7] (..util.taskupdates.TaskInfoUpdater) [operationID=#1, task=HTID-########-####-####-####-############] | State update succeeded for task 'HTID-########-####-####-####-############'. State = error

Environment

• vSphere Replication 8.8.x through 9.x (Enhanced Replication)
  
  
• vSphere Live Recovery 9.0.4

Cause

• These errors are reported in vSphere Client when vSphere Replication tries to load balanced errored replications between different ESXi hosts.

Resolution

• Identify and resolve issues with any replications that are in an error state.
  
  
• If the issue persists, log into each vSphere Replication appliance URL (also known as VMware Live Recovery after version 9.0.4).
  For example: https://vspherereplication_site1/dr AND https://vspherereplication_site2/dr
  
  
• Likely need to force-remove the replication on both sides.
  
  
• If the issue still persists after these steps, contact Broadcom support for further assistance.

Additional Information

Change frequency of Load balance task