Error: VMware Tools uninstalls automatically during Windows patching

Products

• VMware vSphere ESXi 8.0

Issue/Introduction

• During a Windows patching cycle, if the Automatic VMware Tools upgrade option is enabled, a race condition may occur. If the VM reboots before the VMware Tools upgrade completes, the task remains in a suspended state.

• Re-installation of VMware Tools will not complete while the task is suspended

• VMware Tools drivers, including the vmxnet3 network driver, are unloaded.

• The VM becomes unreachable over the network.

 

• In vmware.log we can see similar snippets:

  /vmfs/volumes/datastore-UUID/vmname/vmware.log

  YYYY-MM-DDTHH:MM:SSZ In(05) vcpu-0 - Guest: vnet: AUDIT: DriverUnload : vNetWFPFilter 12.1.0.0 build-19947491  unloaded

  YYYY-MM-DDTHH:MM:SSZ In(05) vcpu-4 - TOOLS Received tools.set.version rpc call, version = TOOLS_VERSION_NONE (uninstalled), type is unset

  YYYY-MM-DDTHH:MM:SSZ In(05) vcpu-4 - TOOLS Setting toolsVersionStatus = TOOLS_STATUS_NO_TOOLS

  YYYY-MM-DDTHH:MM:SSZ In(05) vcpu-4 - TOOLS setting legacy tools version to '0' type -1, manifest status is 7

  YYYY-MM-DDTHH:MM:SSZ In(05) vcpu-4 - Tools_SetVersionAndType: Updating the Tools Version in the disk

  YYYY-MM-DDTHH:MM:SSZ In(05) vcpu-6 - VMXNET3 user: Ethernet0 RSS fields requested by vmx: 3

  YYYY-MM-DDTHH:MM:SSZ In(05) vcpu-5 - Guest: Driver=vmxnet3, Version=1.9.14.0 - unloaded

  YYYY-MM-DDTHH:MM:SSZ In(05) vcpu-5 - Guest: Driver=vsock, Version=9.8.19.0 - unloaded

  YYYY-MM-DDTHH:MM:SSZ In(05) vcpu-7 - Guest: Driver=vmci, Version=9.8.18.0 - unloaded

  YYYY-MM-DDTHH:MM:SSZ In(05) vcpu-0 - Tools: Tools heartbeat timeout.

  YYYY-MM-DDTHH:MM:SSZ In(05) vcpu-0 - Tools: Running status rpc handler: 1 => 0.

  YYYY-MM-DDTHH:MM:SSZ In(05) vcpu-0 - Tools: Changing running status: 1 => 0.

  YYYY-MM-DDTHH:MM:SSZ In(05) vcpu-0 - Tools: [RunningStatus] Last heartbeat value 83527 (last received 20s ago)

 

Environment

VMware vSphere ESXi 8.x

VCF (VMware Cloud Foundation) 

 

Cause

This issue occurs due to a race condition between WSUS patching and VM reboots, where the system restarts before the VMware Tools installation completes. As a result, the VMware Tools upgrade task remains in a suspended state without being properly monitored.

Resolution

There is no resolution. Reach out to the guest OS vendor

Workaround:

1. 1. Turn off "Check and upgrade VMware tools before each power on", and then perform Windows patching.
   2. Manually upgrade the VMware Tools.  

Additional Information

If Windows SCCM is being used, and any updates to the SCCM were made that preceded the removal of VMware Tools on VMs it manages, reach out to Microsoft to investigate those inadvertent removal tasks.

 

Tasks in the Windows event log might look similar to the following.

Product: VMware Tools -- Removal completed successfully.

Windows Installer removed the product. Product Name: VMware Tools. Product Version: 12.4.0.23259341. Product Language: 1033. Manufacturer: VMware, Inc.. Removal success or error status: 0.

Windows Installer requires a system restart. Product Name: VMware Tools. Product Version: 12.4.0.23259341. Product Language: 1033. Manufacturer: VMware, Inc.. Type of System Restart: 1. Reason for Restart: 2.

The Windows Installer initiated a system restart to complete or continue the configuration of 'VMware Tools'.

Contact Broadcom Support

Broadcom Product Downloads