Add datastore from vCenter fails with "query execution timed out because of a back-end property-provider"

Products

• VMware vCenter Server
• VMware vSphere ESXi

Issue/Introduction

Symptoms:

• When adding a new datastore from Web Client, the following error occurs and Web Client fails to show available LUNs:

The query execution timed out because of a back-end property-provider 'com.vmware.vsphere.client.storage.impl.DatastorePropertyProvider' which took more than 120 seconds

• There are a large amount of unresolved unmounted datastores or snapshot LUNs.

Environment

VMware vCenter Server 6.7.x
VMware vSphere ESXi 6.5
VMware vSphere ESXi 6.0
VMware vCenter Server 6.0.x
VMware vSphere ESXi 6.7
VMware vCenter Server 6.5.x

Cause

This is a known issue. When querying list of unresolved VMFS volumes and snapshot LUNs from vCenter, ESXi host performs additional filesystem liveness check for all unresolved volumes. This liveness check takes at least 16 seconds per volume. So there are a large number of unresolved volumes, the query does not complete within the timeout of dataservice at Web Client side and fails to show the volumes.

Resolution

Option 1: Disabling the filesystem liveness check at ESXi host side:

1. Log in to any of client such as Host Client or Web Client where you can manage the target ESXi host.
2. Change VMFS.UnresolvedVolumeLiveCheck to FALSE in the ESXi host's Advanced System Settings.
3. Save changes.

or

1. Log in to ESXi host as root via console or SSH.
2. Edit  /etc/vmware/hostd/config.xml and add the following parameter under plugins/hostsvc/storage node.
   <checkLiveFSUnresolvedVolume>FALSE</checkLiveFSUnresolvedVolume>​
3. Save changes.
4. Restart hostd.
   # /etc/init.d/hostd restart

Option 2: Increase the timeout value of dataservice in Web Client:

1. Log in to vCenter Server.
2. Edit webclient.properties. This file is found in the following location:
   - /etc/vmware/vsphere-client/webclient.properties  in vCSA
   - %ALLUSERSPROFILE%\VMware\vCenterServer\cfg\vsphere-client in Windows vCenter Server
3. Change the timeout value of dataservice.timeoutSeconds.
   dataservice.timeoutSeconds = [TIMEOUT VALUE IN SECONDS]
4. Save changes.
5. Restart Web Client service
   service-control --stop vsphere-client
   service-control --start vsphere-client