All Snapshot from Protected site failed | VCDR couldn't communicate correctly with vCenter

Products

• VMware Live Recovery

Issue/Introduction

To fix the issue and resume the backups. And the snapshots to work again and resume the scheduled jobs.

Symptoms:

All snapshots from the protected site have failed, and VCDR is unable to communicate with vCenter. Scheduled snapshots for all protection groups have failed, resulting in a critical status for the protection group, accompanied by the error: "The last snapshot of protection group xxx failed because it did not include any VMs."

In the event log, the following error appears for one of the protection groups:

Target: pg_Server

Description: Error taking snapshot for protection group 'pg_Server'. Internal error: RPC service unresponsive. Task ID: drc-xxxxxxw-xxxxx-xxxx-xxxx-0xxxx0xxxx.

 

 

When attempting to perform a manual snapshot, the same error persists. Additionally, creating a new protection group results in the browser for the protected vCenter remaining "loading," indicating that VCDR is unable to communicate with vCenter.

 

 

TROUBLESHOOTING:

The error "RPC service unresponsive" may indicate a communication issue with the DRC Connector VM. The connector VMs should be rebooted, and then allow 30 minutes before checking if you can create/edit the protection group and take a manual snapshot.

 

Also, verify the network connectivity of all DR connectors by following the instructions in the document:

DRC Network Test — drc network test --scope local

Cyber Recovery connector Connectivity Check

Cause

A change in the on-premises environment has resulted in a firewall block that is preventing communication with the VCDR service.

Resolution

After removing the firewall block, the snapshots function correctly, and scheduled jobs can resume.

 

Additional Information

Impact/Risks:

There is a production impact; the customer's sites are unprotected, and backup/snapshot jobs cannot resume.