Unable to create a test bubble image for group ID on vSphere replication server.

Products

• VMware Site Recovery Manager 8.x

Issue/Introduction

Symptoms:

•  Site Recovery Manager recovery fails with the following error messages in the Recovery Plan execution report:

"Cannot create a failover image for group 'GID-4fc0c3f5-####-####-####-ca2add940db6' on vSphere Replication Server 'hostname' (address '###.##.##.###'). No storage is found for datastore path '[datastore_name] fc77a766-b4c7-797c-68b7-b496915a8e3c/hbrcfg.GID-4fc0c3f5-####-####-####-ca2add940db6.293168.nvram.1893'. Cannot create a failover image for group 'GID-4fc0c3f5-####-####-####-ca2add940db6' on vSphere Replication Server (address '###.##.##.###'). No storage is found for datastore path '[datastore_name] 1c82a766-609c-52c0-df1c-043201517df0/hbrcfg.GID-4fc0c3f5-####-####-####-ca2add940db6.293167.nvram.1891"

Cause

• The file hbrcfg.GID-4fc0c3f5-####-####-####-ca2add940db6.293168.nvram.1893 is critical for restoring the virtual machine's memory state during SRM-driven test recovery or failover operations. Its absence prevents vSphere Replication from completing recovery workflows that involve memory state retention.
• Manual deletion of a test bubble VM can remove the vSAN objects it references. Since vSphere Replication is disk-based and doesn't use a traditional placeholder VM structure, deletion of those vSAN objects breaks the replication chain. Refer VMware vSphere Replication 9.0 Release Notes for more details.
• A review of the /opt/vmware/support/logs/hbrsrv.log file on the vSphere Replication appliance revealed multiple entries indicating file was not found.

2025-03-03T15:22:30.478+05:30 error hbrsrv[24983] [Originator@6876 sub=Main groupID=GID-4fc0c3f5-####-####-####-ca2add940db6 opID=fa690057-c93f-41dc-a4ce-e89b8f491cfc-failover:188a:6764:de9b:886c-HMS-1443352] HbrError for (datastoreUUID: "vsan:52ee3b4ac0652491-####fca4e38221fa"), (hostIP: ""), (pathname: "1c82a766-609c-52c0-df1c-043201517df0/hbrcfg.GID-4fc0c3f5-####-####-####-ca2add940db6.293168.nvram.1891") stack:
2025-03-03T15:22:30.479+05:30 error hbrsrv[24983] [Originator@6876 sub=Main groupID=GID-720ef374-d423-33db-bde7-d19107482c92 opID=fa690057-c93f-41dc-a4ce-e89b8f491cfc-failover:188a:6764:de9b:886c-HMS-1443352]    [0] Fault: (vim.fault.FileNotFound) {
-->    faultCause = (vmodl.MethodFault) null,
-->    faultMessage = <unset>,
-->    file = "/vmfs/volumes/vsan:52ee3b4ac0652491-####fca4e38221fa/1c82a766-609c-52c0-df1c-043201517df0/hbrcfg.GID-4fc0c3f5-####-####-####-ca2add940db6.293168.nvram.1893"
-->    msg = "File /vmfs/volumes/vsan:52ee3b4ac0652491-####fca4e38221fa/1c82a766-609c-52c0-df1c-043201517df0/hbrcfg.GID-4fc0c3f5-####-####-####-ca2add940db6.293168.nvram.1893 was not found"

Resolution

To resolve the issue of the missing .nvram and other critical configuration files on the target datastore, follow these steps:

1. Remove the VM from replication, ensuring that the replication seeds are retained on the target datastore.

2. Reconfigure replication for the same VM using the existing seed disks on the target site.

3. Monitor and validate replication progress to confirm that the initial sync completes successfully.

4. After synchronization, verify the presence of the .nvram file and other configuration files (.vmx, .vmdk, etc.) on the target datastore.

5. Perform a test recovery operation to ensure it completes without errors.

By removing and re-adding the VM to replication using the same seeds, vSphere Replication is able to reconstruct missing metadata and files, including the .nvram. This ensures that all required files are in place on the recovery site, enabling successful test recovery or failover operations.