Re-protect task fails with Error: "Operation timed out: 14580 seconds" in the DR UI

Products

• VMware Live Recovery

Issue/Introduction

Symptoms

• When performing a re-protect task through SRM after failover an error is encountered:

Operation timed out: 14580 seconds

• The above error will appear in the recovery plan report generated for the specific recovery plan.

Validation

• In the /opt/vmware/support/logs/srm/vmware-dr.log, you will see this parameter vrReplication.synchronizationTimeout value is set to default 7200 seconds:

2025-01-06T11:39:02.359+05:30 verbose vmware-dr[01237] [SRM@6876 sub=HbrRecoveryEngine opID=b66d8a0a-9e07-43f0-ae63-4dd421d15a3c-reprotect:b53a:e8ea:2f9b:6d14-onlineSync:2f27] [52f24] Sync: Using sync time out of 7200 seconds.

• The synchronization process runs for 4 hours but eventually times out:

2025-01-06T15:39:02.428+05:30 verbose vmware-dr[01238] [SRM@6876 sub=RemoteTask.dr.hbrProvider.HmsProxy.remoteSync48 opID=b66d8a0a-9e07-43f0-ae63-4dd421d15a3c-reprotect:b53a:e8ea:2f9b:6d14-onlineSync:2f27 tid=dr.hbrProvider.HmsProxy.remoteSync48.HmsRemoteSync] The remote task 'vim.Task:19f9e716-4a4d-4f30-afd4-9fb50254a3d2:dr.hbrProvider.HmsProxy.remoteSync48' timeout passed. _updated=true

 

 

Environment

VMware Site Recovery Manager 8.x
VMware Live Site Recovery 9.x

Cause

The default timeout value was insufficient for the synchronization process to complete, leading to a timeout error after 14580 seconds

This may be caused due to:

• Slow network
• Amount to delta to synchronize

Resolution

• vrReplication.synchronizationTimeout is a configurable parameter in vSphere Replication, that defines the maximum allowed time (in seconds) for a replication synchronization operation to complete
• vrReplication.synchronizationTimeout value is set to default 7200 seconds.
• The synchronization timeout is calculated as: 2 x vrReplication.synchronizationTimeout = 14400 seconds (4 hours).
• Additionally, the remoteManager.taskProgressDefaultTimeout is set to 180 seconds by default.
• Combining these two values, the total timeout duration comes to 14580 seconds.
  
  
• To resolve the issue, adjust the vrReplication.synchronizationTimeout to 10800 seconds (3 hours) instead of the default 7200 seconds (2 hours).
• When vrReplication.synchronizationTimeout is changed from 7200 to 10800, total timeout is then adjusted to 2 x vrReplication.synchronizationTimeout = 21600 
• This adjustment will allow for a longer synchronization period, reducing the likelihood of premature timeouts during replication operations.

To increase the vrReplication.synchronizationTimeout value on both the sites go to the Advanced Settings section.

1. In the vSphere Client, click Site Recovery > Open Site Recovery.
2. On the Site Recovery home tab, select a site pair, and click View Details.
3. In the left pane, click Configure > Advanced Settings > vSphere Replication.
4. Select a site and click Edit to modify the vSphere Replication settings.

Additional Information

For more details, please refer the below document: 

https://techdocs.broadcom.com/us/en/vmware-cis/live-recovery/site-recovery-manager/8-7/site-recovery-manager-administration-8-7/advanced-srm-configuration/reconfigure-srm-settings/change-vsphere-replication-settings.html