vCenter Inventory disappears from the UI occasionally due to excessive RPC calls
search cancel

vCenter Inventory disappears from the UI occasionally due to excessive RPC calls

book

Article ID: 318212

calendar_today

Updated On:

Products

VMware vCenter Server

Issue/Introduction

Symptoms:
  • All of the objects in a vCenter Server's inventory will not display in the UI. This can be intermittent or consistent.
  • Information about a specific VC, such as Node health in System Configuration might be missing from the UI.
  • A very high concentration of getManifestArray calls being made in a short period of time can be seen in /var/log/vmware/vsphere-ui/logs/dataservice.log
A PropertyProviderBean method's execution took too long: 2444 milliseconds. (Provider instance: com.vmware.vsphere.client.host.impl.HostPropertyProvider@1eb13a12, method: public com.vmware.vsphere.client.host.HostManifestInfo com.vmware.vsphere.client.host.impl.HostPropertyProvider.getManifestArray(com.vmware.vim.binding.vmodl.ManagedObjectReference) throws java.io.IOException,java.lang.Exception, property: ref: ManagedObjectReference: type = HostSystem, value = host-1234, serverGuid = 123456-2819-4cea-a95b-c3, method args: [ManagedObjectReference: type = HostSystem, value = host-1234, serverGuid = 23456-2819-4cea-a95b-c3])
  • Timeout errors containing PropertyName: manifestArray will likely also be seen in dataservice.log
[2023-02-21T02:14:44.280-03:00] [ERROR] http-nio-5090-exec-456       70115902 103194 200221 DataServiceImpl [queryId: q-100695:h5ui-getPropertiesForObjects:1335823833] Error occurred while executing query:
The query sent to the Data Service is:

QuerySpec
 QueryName: q-100695:h5ui-getPropertiesForObjects:1335823
 ResourceSpec
  Constraint: CompositeConstraint
   TargetType: HostSystem
   Conjoiner: OR
   Constraint: ObjectIdentityConstraint
    TargetType: null
    Target: ManagedObjectReference: type = HostSystem, value = host-1111, serverGuid = 2852BBB5-0BC0-4142-B8A9-ABCDEFGHI123
   Constraint: ObjectIdentityConstraint
    TargetType: null
    Target: ManagedObjectReference: type = HostSystem, value = host-2222, serverGuid = 2852BBB5-0BC0-4142-B8A9-ABCDEFGHI123
   Constraint: ObjectIdentityConstraint
    TargetType: null
    Target: ManagedObjectReference: type = HostSystem, value = host-3333, serverGuid = 2852BBB5-0BC0-4142-B8A9-ABCDEFGHI123
   Constraint: ObjectIdentityConstraint
    TargetType: null
    Target: ManagedObjectReference: type = HostSystem, value = host-4444, serverGuid = 2852BBB5-0BC0-4142-B8A9-ABCDEFGHI123
   ...(105 more)...
  PropertySpec[1]
   ProviderType: HostSystem
   Relationship: null
   Properties[9]
    PropertyName: name
    PropertyName: primaryIconId
    PropertyName: summary.overallStatus
    PropertyName: hostClusterName
    PropertyName: summary.config.product.version
    PropertyName: stateLabel.@formatted
    PropertyName: dcOrUserCreatedHostFolder
    PropertyName: connectionState
    PropertyName: manifestArray
 ResultSpec:
  Offset: null
  MaxResultCount: null
  OrderingCriteria
   OrderPropertySpec[1]
    ProviderType: Object
    Relationship: null
    SortType: ASCENDING
    OrderingProperties[1]
     PropertyName: id
 Options:
  VISE_TO_RISE_TRANSLATOR_STRICT_MODE=true
 com.vmware.vise.data.query.internal.DataServiceException: The query execution timed out because of a back-end property provider 'Property provider com.vmware.vsphere.client.host.impl.HostPropertyProvider' which took more than 120 seconds.


Environment

VMware vCenter Server 7.0.x

Cause

This issue is caused by improper handling of the manifestArray property in queries, resulting in excessive RPC calls and, eventually timeouts which cause the inventory to disappear.

Resolution

This issue is resolved in vCenter Server 7.0 U3L (build 21477706)