VMware SRM test failover results in missing VMware Tools/system files or Windows booting into Recovery Environment (WinRE)

Products

• VMware vSphere ESXi

Issue/Introduction

During or immediately following a VMware Site Recovery Manager (SRM) test failover, virtual machines (VMs) that function normally in the protected site encounter one or both of the following post-failover symptoms in the recovery site:

1. Missing Files: Critical files within C:\Program Files\VMware\VMware Tools or other system directories are missing or stripped out immediately following the initial boot in the recovery environment.

2. Boot Failure to WinRE: Recovered Windows VMs fail to complete a normal boot cycle and repeatedly launch into the Windows Recovery Environment (WinRE) or drop into an automatic repair loop.

Because unbootable VMs in WinRE cannot be fully logged into, additional system files or drivers in guest OS directories (such as C:\Windows\System32\drivers) may also be missing or incomplete due to similar automated file actions.

Environment

 

• VMware Site Recovery Manager (SRM) 8.x / 9.x

• vSphere Replication (VR) / Array-Based Replication (ABR)

• Microsoft Windows Server (All supported versions)

• Third-Party Endpoint Detection and Response (EDR) / Antivirus (AV) software (e.g., SentinelOne, CrowdStrike Falcon, VMware Carbon Black, Microsoft Defender for Endpoint)

 

Cause

This issue can be caused by third-party EDR/AV agents within the Guest OS flagging and quarantining critical binaries, drivers, or executables during the post-failover boot process.

When SRM executes a test failover, VMs boot in an isolated test network ("test bubble") and often encounter minor hardware environment shifts (e.g., new MAC addresses, host CPU topology differences, or hypervisor host identity changes). Additionally, VMware Tools automatically executes background tasks and initialization scripts to adapt to the recovery site.

Behavioral engines within EDR software can interpret this sudden combination of offline network posture, hardware profile shifts, and automated script execution as anomalous or malicious activity (such as a ransomware payload or unauthorized system modification). Consequently, the EDR software silently quarantines or deletes the executing files—such as VMware Tools executables or critical Windows boot drivers—leaving the guest OS in an incomplete state or triggering a WinRE boot failure.

Resolution

Step 1: Confirm EDR Quarantine Activity

1. Log into the EDR management console (e.g., SentinelOne, CrowdStrike, Carbon Black, Defender for Endpoint).

2. Filter the incident or alert logs for the impacted recovery VM hostnames or IP addresses during the exact timeframe of the test failover.

3. Verify if files under C:\Program Files\VMware\VMware Tools or C:\Windows\System32\ were flagged and quarantined.

4. If files are quarantined, release/restore the impacted files via the EDR console (if the OS is accessible) or prepare to re-install VMware Tools upon successful OS boot.

Step 2: Implement EDR Exclusions

To prevent recurring quarantine events during test failovers, configure proper exclusions in the EDR solution:

• Publisher / Signer Identity Exclusion (Highly Recommended): Configure an exclusion based on the Code Signing Certificate / Signer Identity for VMware, Inc. This allows the EDR agent to trust binaries signed by VMware regardless of path or file hash changes during future updates, maintaining security while ensuring functional interoperability.

• Path / Interoperability Exclusions (Alternative): If Publisher exclusions are not supported by the EDR vendor, configure targeted path exclusions for C:\Program Files\VMware\VMware Tools\*. Ensure the exclusion mode is set to Interoperability or Bypass monitoring hooks for those specific executables.

Security Best Practice: Apply these exclusions strictly to the endpoint policy group containing disaster recovery/replicated workloads rather than applying them globally across the entire enterprise estate.

Step 3: Validate Recovery

1. Cleanly shut down the impacted VMs in the recovery site and remove the SRM test environment.

2. Perform a new synchronization and run an SRM Test Failover.

3. Verify that the VMs boot into Windows normally and that all VMware Tools services start without missing files.

Additional Information

For VMs that fail to boot past WinRE, access the WinRE Command Prompt and inspect the diagnostic log located at: C:\Windows\System32\Logfiles\Srt\SrtTrail.txt This log explicitly identifies the missing driver, corrupted boot binary, or dirty-shutdown flag that triggered the recovery loop.