Integrations UI becomes unresponsive or times out due to high number of vCenter adapter instances

Products

• VCF Operations
• VMware Cloud Foundation

Issue/Introduction

• Accessing Administration > Integrations causes the browser to freeze or return a net::ERR_CONNECTION_TIMED_OUT error.

• An unusually high and incorrect vCenter adapter instances  (e.g., 10,000+) are being displayed in the inventory (Infrastructure Operations > Configurations > Inventory Management > Adapter Instances > vCenter) 

• analytics-###.log displays:

WARN analytics 314408 [ops@4413 threadId="278" threadName="Suspended Alert Recovery Thread"] [com.vmware.statsplatform.persistence.impl.AlertSqlQueryBuilder.createQueryCondition] - resourceIdSet is too large (#####) , return only first 100000 resource ids

• UI message "Cluster failed to come online" 

• Cluster remains in Going Online state indefinitely or fails with the message Cluster failed to come online

• Nodes stuck in waiting for Analytics state

Environment

• VCF Ops: 9.0.x, 9.0.2
• VCF: 9.0

Cause

Cause identification is currently under active investigation.

Resolution

To resolve this issue, contact Broadcom Support and note this Article ID (440197) in the problem description. For more information, see Creating and managing Broadcom support cases.

Additional Information

Prior to contacting Broadcom Support, please enable DEBUG level logging for below parameters:

• com.vmware.vcops.controller.data.utils.AdapterInstanceAccessor
• com.vmware.vcops.controller.data.utils.ResourceAccessorBase
• com.vmware.vcops.controller.data.utils.GeneralResourceAccessor

After substantial logging happens, please collect the FULL support bundle from VCF Operations component.