Failed to create snapshots of replica devices

Products

• VMware Live Recovery

Issue/Introduction

Running a TEST recovery using SRM fails with the error - Failed to create snapshots of replica devices

Environment

VMware Live Site Recovery 9.x
Site Recovery Manager - 8.x

Cause

Factors such as lack of space, low cache, and misconfigurations on the storage array can contribute to the occurrence this error.

From SRM log, /opt/vmare/support/logs/Support/vmware-dr.log

-->       errors = (vmodl.MethodFault) [
-->          (dr.storageProvider.fault.StorageTestFailoverStartFailed) {
-->             faultCause = (dr.storage.fault.CommandTimedout) {
-->                faultCause = (vmodl.MethodFault) null,
-->                faultMessage = <unset>,
-->                commandName = "testFailoverStart",
-->                timeout = 900
-->                msg = "Timed out (900 seconds) while waiting for SRA to complete 'testFailoverStart' command."
-->             },
-->             faultMessage = <unset>
-->             msg = "Failed to create snapshots of replica devices. Timed out (900 seconds) while waiting for SRA to complete 'testFailoverStart' command."
-->          },

The SRA fails to complete and update the recovery task to SRM within set timeout value

Resolution

SRA manages the target devices and their associated operations when working in tandem with SRM, time-consuming tasks such as creation, mapping, unmapping and deletion of target volumes often lead to SRM timeouts, impacting a recovery operation. 

To address such issues, you can choose to adjust the SRM advanced option storage.commandTimeout to accommodate longer durations. By default, this value is set to 300 seconds. Based on practical experience, it's been observed that setting this value within the range of 500 to 900 seconds and in some cases 1800 seconds will show positive results. This setting must be changed on SITE A & SITE B. 

However, it's worth noting that any adjustments to this setting should be approached with caution and ideally be discussed with the VMware support team, as it has implications on TEST recovery & recovery operations in general as it tends to prolong the duration of time required to complete a recovery plan. 

If you've tried adjusting this setting and still can't resolve the issue, it's best to reach out to your storage vendor and provide them with the SRM log bundles for further investigation.