Enhanced Replication Mappings Testing Connection spins indefinitely

Products

• VMware Live Recovery

Issue/Introduction

1. Connection testing spins indefinitely 

2. Continuous testing causes the hosts to enter a not responding state 

3. Enhanced Replication TEST results fail

Fault occurred while performing health check. Details: 'Connect: Connection timed out'.
Fault occurred while performing health check. Details: 'Connect: Connection timed out'.

Fault occurred while performing health check. Details: 'A generic error occurred in the vSphere Replication Management Server. Exception details: 'Failed to build configuration spec for hbrsrvuw '####6877-####-####-1001-00000000####'.'.'.
Fault occurred while performing health check. Details: 'A generic error occurred in the vSphere Replication Management Server. Exception details: 'Failed to build configuration spec for hbrsrvuw '####6877-####-####-1001-00000000####'.'.'.

4. Innumerable placeholder datastores are found in the placeholder mappings tab which are mapped to unrequired clusters.



Environment

VMware vSphere Replication 9.x
VMware Live Site Recovery 9.x
VMware ESXi 8.x

Cause

1. The <scale-out-mode> setting is set to false (This setting is required for Enhanced Replication to function).

2. Placeholder datastores mappings has innumerable datastores in it mapped to several unknown clusters 

3. ESXi hosts are locking the datastores used by Enhanced replication 

Resolution

1. Check out the <scale-out-mode> setting in hms-configuration.xml and set it to true

   <scale-out-mode>false</scale-out-mode>
   <!--
   Timeout to wait before tagging hbrsrv as decommissioned due to maintenance mode.
   At the moment set to 0 since replications are auto-released on hbrsrvuw when ESX host enters MM.
   -->

root@VRMS [ /opt/vmware/hms/conf ]# less hms-configuration.xml | grep -i scale-out-mode
   <scale-out-mode>true</scale-out-mode>

2. Following best design practices, each cluster must have its own Placeholder datastore (Size = 5GB). For further information, consult the documentation on Placeholder Virtual Machines. This is not applicable if you are using only a vSAN datastore at both the source and target sites. 

3. Review ESXi host logs to identify any locked datastores used by Enhanced replication or potential storage issues.

Additional Information

Investigate Lock Contention on ESXi hosts (384496)