VMware Tools installer can fail during Windows updates or when security software locks files

Products

• VMware vSphere ESXi

Issue/Introduction

When a Windows virtual machine (VM) configured with the VMware Tools upgradeAtPowerCycle policy undergoes automated Windows updates, conflicts can arise between the Guest OS boot routines and the VMware Tools auto-upgrade process. This overlap often leads to installation failures, resulting in missing drivers (e.g., VMXNET3) or the uninstallation of existing VMware Tools components.

Symptoms

• VMware Tools status in vCenter displays "Not running, not installed" or "unmanaged TOOLS_STATUS_NO_TOOLS".
• Guest network connectivity is lost (missing/uninstalled VMXNET3 adapter).
• Installer logs (e.g., vmupgrader_msi.log) show errors: 1723, 1603, 19, or 21009.
• VMware Tools installer service is deleted or purged during boot-time rollback.

Environment

VMware vSphere ESXi 8.x

VMware Tools 11.x, 12.x, 13.x

Guest OS: Windows Server (all versions)

Cause

This issue occurs due to a race condition between the Windows startup/shutdown routines and the VMware Tools auto-upgrade engine with the VMware Tools upgrade policy upgradeAtPowerCycle. This contention is often exacerbated by real-time security software (e.g., Antivirus/EDR) locking core installer binaries or registry keys during the update process.

Resolution

VMware Tools 11.x and later installers handle this error by alerting the user while allowing the process to continue. If a failure occurs:

1. Ensure all Windows updates and associated reboots are complete.
2. Run the VMware Tools installer manually within the Guest OS.
3. Select the Repair option to restore any missing or corrupted drivers.
4. Reboot the Guest OS to finalize the repair.

Workaround:

1. Disable Auto-Upgrade Policy:

   • Set tools.upgrade.policy to Manual on production virtual machines.
   • In the vSphere Client, go to VM > Edit Settings > VM Options > VMware Tools > Tools Upgrades.
   • Uncheck "Check and upgrade VMware Tools before each power on."

2. Configure Security Exclusions: Exclude the following paths and processes from your real-time security software to prevent file-locking collisions:

   • Folders: C:\Program Files\VMware\VMware Tools\ , C:\ProgramData\VMware\ , C:\Windows\Installer\ , and C:\Windows\Temp\vmware-SYSTEM\
   • Processes: setup64.exe, vmtoolsd.exe, VGAuthService.exe

3. Perform Manual Reinstallation:

   • Clear leftover directories from failed installations.
   • Run the installer manually inside the Guest OS: setup64.exe /s /v "/qn REBOOT=R"

Additional Information

Decouple update cycles: VMware Tools updates should be managed out-of-band via vSphere Lifecycle Manager (vLCM) or manual deployment, rather than relying on automatic power-on upgrades for production workloads.