VMware Tools upgrade fails with Setup failed to install the paravirtual SCSI driver automatically on Windows Server

Products

• VMware vCenter Server
• VMware vSphere ESXi

Issue/Introduction

• When attempting to upgrade VMware Tools on a Windows guest operating system, the installation fails and rolls back with the following error message: Setup failed to install the paravirtual SCSI driver automatically. This driver will have to be installed manually.
  
  
• In the guest OS C:/Windows/INF/setupapi.dev.log, the following error signature is observed:
  
       dvi:                               Detecting devices pending query-removal: HH:MM:SS.155
     dvi:                                 STORAGE\VOLUME\{########-####-####-####-##########}#0000000000100000
     dvi:                                 STORAGE\VOLUME\{######-####-####-####-######}#000000001F400000
     dvi:                                 STORAGE\VOLUME\{######-####-####-####-######}#0000000025800000
     dvi:                                 STORAGE\VOLUME\{######-####-####-####-######}#0000000026800000
     dvi:                                   STORAGE\VOLUMESNAPSHOT\HARDDISKVOLUMESNAPSHOT1
     dvi:                                   STORAGE\VOLUMESNAPSHOT\HARDDISKVOLUMESNAPSHOT2
     dvi:                                   STORAGE\VOLUMESNAPSHOT\HARDDISKVOLUMESNAPSHOT3
     dvi:                                   STORAGE\VOLUMESNAPSHOT\HARDDISKVOLUMESNAPSHOT4
     ...Text omitted...
     dvi:                                   STORAGE\VOLUMESNAPSHOT\HARDDISKVOLUMESNAPSHOTXXXX
     dvi:                                   STORAGE\VOLUMESNAPSHOT\HARDDISKVOLUMESNAPSHOTXXXX
     dvi:                                   STORAGE\VOLUMESNAPSHOT\HARDDISKVOLUMESNAPSHOTXXXX
     dvi:                                   STORAGE\VOLUMESNAPSHOT\HARDDISKVOLUMESNAPSHOTXXXX
     dvi:                                   STORAGE\VOLUMESNAPSHOT\HARDDISKVOLUMESNAPSHOTXXXX
     dvi:                                   STORAGE\VOLUMESNAPSHOT\HARDDISKVOLUMESNAPSHOTXXXX
     dvi:                                   STORAGE\VOLUMESNAPSHOT\HARDDISKVOLUMESNAPSHOTXXXX
     dvi:                                   SWD\WPDBUSENUM\{######-####-####-####-######}#0000000008100000
     dvi:                                 PCI\#####################\AB12345678900
     dvi:                                   SCSI\DISK&VEN_VMWARE&PROD_VIRTUAL_DISK\000000
     dvi:                                   SCSI\DISK&VEN_VMWARE&PROD_VIRTUAL_DISK\000100
!!!  dvi:                               Detected driver hang in related device while installing device 'PCI\#####################\AB12345678900'. HH:MM:SS.576
!!!  dvi:                               Error 481: The operation timed out waiting for this device to complete a PnP query-remove request due to a potential hang in the device stack of a related device. The system may need to be rebooted to complete the operation.
 HH:MM:SS.576
     ump:                          {Plug and Play Service: Device Install exit(000001e1)}
!!!  ndv:                          Device install failed for device.
!!!  ndv:                          Error 481: The operation timed out waiting for this device to complete a PnP query-remove request due to a potential hang in the device stack of a related device. The system may need to be rebooted to complete the operation.
     ndv:                     {Update Device Driver - exit(000001e1)}
!!!  ndv:                     Failed to install device instance 'PCI\#####################\AB12345678900'. Error = 0x000001e1
<<<  Section end YYYY/MM/DD HH:MM:SS.028
<<<  [Exit status: FAILURE(0x000001e1)]

Environment

VMware vSphere ESX 9.x
VMware vSphere ESXi 8.x
Windows Servers

Cause

• The failure is caused by a Windows Plug-and-Play (PnP) "query-remove" timeout (Error 481 / 0x000001e1). During the upgrade, the VMware Tools installer requests the OS to temporarily quiesce and detach the active PVSCSI device to update the driver.
• This process fails to complete within the Windows-allocated time window, as the device stack is locked by an excessive number of Volume Shadow Copy Service (VSS) snapshots (shadow copies). 

Resolution

To resolve this issue, Contact Microsoft Support for analysis of the PnP manager timeout (Error 481). 

Workaround:

• To complete the VMware Tools upgrade without triggering the OS driver-update timeout: Perform a Custom Setup of VMware Tools and deselect the PVSCSI Driver.
• This allows the rest of the Tools suite to upgrade while leaving the existing PVSCSI driver in place.
  Refer kb for workaround steps: VMware Tools upgraded fail while installing PVSCSI driver

Additional Information

vssadmin delete shadows

To speak with a Broadcom Support Engineer, see Contact Support