VMware Tools install/upgrade freezes (hangs) at vmci "driver installation"

Products

VMware vSphere ESXi

Issue/Introduction

Symptoms:

On VMware Tools installer (fresh installation or upgrade), it hangs at "vmci driver installation"

Then it fails after some time with error "Setup failed to install the VMCI Virtual Machine Communication Interface driver automatically. This driver will have to installed manually"

Log snippet in C:\Windows\INF\setupapi.dev.log:

>> [Device Install (UpdateDriverForPlugAndPlayDevices) - PCI\VEN_15AD&DEV_0740&SUBSYS_074015AD]
>> Section start 2022/11/28 15:09:38.085
cmd: C:\Windows\System32\MsiExec.exe -Embedding D6EA8E5B0FEF289D96534FDF641438EC E Global\MSI0000
ndv: INF path: C:\Program Files\Common Files\VMware\Drivers\vmci\device\Win8\vmci.inf
ndv: Install flags: 0x00000001
ndv: {Update Device Driver - PCI\VEN_15AD&DEV_0740&SUBSYS_074015AD&REV_10\3&61AAA01&0&3F}
ndv: Search options: 0x00000080
ndv: Searching single INF 'C:\Program Files\Common Files\VMware\Drivers\vmci\device\Win8\vmci.inf'
dvi: {Build Driver List} 15:09:38.101
…
dvi: Class GUID of device changed to: {4d36e97d-e325-11ce-bfc1-08002be10318}.
dvi: {Plug and Play Service: Device Install for PCI\VEN_15AD&DEV_0740&SUBSYS_074015AD&REV_10\3&61AAA01&0&3F}
ump: {Plug and Play Service: Device Install exit(000005b4)}
!!! ndv: Device install failed for device.
!!! ndv: Error 1460: This operation returned because the timeout period expired.
ndv: Installing NULL driver.
dvi: {Plug and Play Service: Device Install for PCI\VEN_15AD&DEV_0740&SUBSYS_074015AD&REV_10\3&61AAA01&0&3F}
! dvi: Installing NULL driver!
…
dvi: Default installer: Enter 15:26:12.962
! dvi: Installing NULL driver!
…
ndv: {Update Device Driver - exit(000005b4)}
!!! ndv: Failed to install device instance 'PCI\VEN_15AD&DEV_0740&SUBSYS_074015AD&REV_10\3&61AAA01&0&3F'. Error = 0x000005b4
<<< Section end 2022/11/28 15:26:45.012
<<< [Exit status: FAILURE(0x000005b4)]

Note: The driver failed to install could be vmxnet3, pvscsi, vmusbmouse, vmmouse or vm3dmp in other upgrade cases.

Cause

One of the identified causes for the issue is:
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Device Installer\DebugInstall is set to DWORD value 2.

Resolution

To fix this issue, please make sure that the below value is set to 0 or deleted if it exists:

Note: This procedure modifies the Windows registry. Before making any registry modifications, ensure that you have a current and valid backup of the registry and the virtual machine. For more information on backing up and restoring the registry, see the Microsoft Knowledge Base article 136393.

Select Start > Run, type regedit, and click OK. The Registry Editor window opens.
Navigate to HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Device Installer\
If the registry DWORD value DebugInstall is present, make sure it is set to value 0.

Please refer to Microsoft Knowledge Base article for more information.

For other cases, please suspend the virtual machine at issue time, collect vmss/vmem files and file a Support Request with VMware Technical Support.

For more information about collecting vmss/vmem, see Converting a snapshot file to memory dump using the vmss2core tool (2003941)

Disclaimer: VMware is not responsible for the reliability of any data, opinions, advice, or statements made on third-party websites. Inclusion of such links does not imply that VMware endorses, recommends, or accepts any responsibility for the content of such sites.

Additional Information

On certain scenarios, if you cannot install new drivers on the Guest OS.
Validate the parameter - DeviceInstallDisabled

"HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\DeviceInstall\Parameters\DeviceInstallDisabled"

If the value is set 1, change to 0.
This will help in drivers installation and also the Vmware tools installation.