VMware Tools installer can fail when overlapping with Windows updates

Products

• VMware vSphere ESXi

Issue/Introduction

Symptoms:

After performing a Windows update on a Guest OS, the following symptoms may be observed:

• Guest networking is lost, and the VMXNET3 adapter may appear as missing or uninstalled.
• Other VMware drivers (such as VMCI) or VMware Tools components are removed from the guest.
• Windows System Event logs may show error 0x800f0246 during the update process.
• vSphere UI displays the VMware Tools status as "Not running, not installed."

Environment

• ESXi: 6.0, 6.5, 6.7, 7.0, and 8.0.x
• VMware tools: Versions prior to 11.3.5 are highly susceptible. Versions 11.3.5 through 12.x include improved error handling but can still experience contention during overlapping reboots.

Cause

This issue occurs due to a race condition when a Windows VM is configured with the VMware Tools upgrade policy upgradeAtPowerCycle (Check and upgrade VMware Tools before each power on).

When the Guest OS reboots to complete a Windows System Update, the automatic VMware Tools upgrade initiates simultaneously. This overlap causes a conflict while modifying kernel drivers, leading to a Windows system error (0x800f0246). Consequently, the VMware Tools installer rolls back, which can result in the removal of existing drivers and components.

Resolution

VMware Tools 11.3.5 and later installers handle this error by alerting the user while allowing the process to continue. If a failure occurs:

1. Ensure all Windows updates and associated reboots are complete.
2. Run the VMware Tools installer manually within the Guest OS.
3. Select the Repair option to restore any missing or corrupted drivers.
4. Reboot the Guest OS to finalize the repair.

Workaround:

To prevent this contention, choose the appropriate method based on your environment:

Manual Policy (Recommended for Automated Patching)
For Windows VMs using automatic updates, temporarily switch the VMware Tools upgrade policy to Manual. This ensures VMware Tools upgrades only occur when the OS is stable.

ProductLocker Management

• Admin-Managed ProductLocker: Only deploy new VMware Tools to the ProductLocker datastore after all VMs have completed their Windows update cycles.
• Default ProductLocker: Perform ESXi host updates (which include new Tools versions) only after the Windows update window for the hosted VMs has closed.

Disabling Power Cycle Check
If updates have already been staged on the host, uncheck "Check and upgrade VMware Tools before each power on" in the VM settings (Edit Settings > VM Options > VMware Tools) before the next reboot.