vSphere Replication HMS service fails when pairing
Article ID: 395801
Updated On:
Products
Issue/Introduction
Sites shows "Not connected" under the pairing page.
Re-configuration works fine.
HMS service crashes when attempted to pair the sites.
hms.log
Caused by: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'datastoreInfoMap': Unsatisfied dependency expressed through field 'hbrsrvuwMonitor'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'hbrsrvuwMonitor' defined in com.vmware.hms.cfg.spring.ScaleOutEnabledConfiguration: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [com.vmware.hms.hbrsrvuw.HbrsrvuwMonitor]: Factory method 'hbrsrvuwMonitor' threw exception; nested exception is java.lang.IllegalStateException: Duplicate key host-49384 (attempted merging values 30373237-####-4d32-####-303530363456 and 30373237-####-4d32-####-35323039444e)
Environment
vSphere Replication 8.x
vSphere Replication 9.x
Cause
The error is caused by duplicate host entries in the vSphere Replication Server database. These duplicate entries conflict during the replication configuration process, leading to a runtime error.
If during/after upgrades, any ESXI hosts are removed/disconnected/added to vCenter, while legacy replications exit, it causes the VR log record duplicate entries in the hbrserverentity table.
Resolution
1. Snapshot the vSphere Replication appliances
2. Stop the HMS service:systemctl stop hms
3.cat /opt/vmware/hms/conf/embedded_db.cfg--> to get VR Database password
4./opt/vmware/vpostgres/current/bin/psql -U vrmsdb--> to login to VR DB
5. Query the hbrsrvuw which does not have replications from hbrtagentity & hbrserverentity tables:
select hbrserver_movalue from hbrtagentity where hbrserver_movalue IN (select movalue from hbrserverentity where vsrv_port = 443 AND NOT EXISTS (select hbrserver_movalue from secondarygroupentity where hbrserverentity.movalue = secondarygroupentity.hbrserver_movalue));select hbrservername,movalue from hbrserverentity where vsrv_port = 443 AND NOT EXISTS (select hbrserver_movalue from secondarygroupentity where hbrserverentity.movalue = secondarygroupentity.hbrserver_movalue);
6. Delete the hbrsrvuw which does not have replications from hbrtagentity & hbrserverentity tables:
delete from hbrtagentity where hbrserver_movalue IN (select movalue from hbrserverentity where vsrv_port = 443 AND NOT EXISTS (select hbrserver_movalue from secondarygroupentity where hbrserverentity.movalue = secondarygroupentity.hbrserver_movalue));delete from hbrserverentity where vsrv_port = 443 AND NOT EXISTS (select hbrserver_movalue from secondarygroupentity where hbrserverentity.movalue = secondarygroupentity.hbrserver_movalue);
7. Start HMS service:systemctl start hms
Feedback
