Errors in Spectrum VNM.OUT regarding performance issues with Real Time Searching

Products

• Spectrum
• Network Observability

Issue/Introduction

The following messages may be seen in the VNM.OUT file: 

ERROR TRACE at SearchManager.cc(1812): Possible performance problem seen, Remedial action taken - Real Time Search Queue has reached max size of 200000. Real Time Searches are being updated now and queued work released. This may be the result of one or more Real Time searches watching very frequently changing attributes.

Environment

Release: Any
Component: SPCCSS

Cause

Real-Time Global Collections will cause these messages and the subsequent performance problems.

Resolution

Generally speaking, Spectrum Support does not recommend using real-time updates on larger collections, and should be avoided unless business needs require it.

We suggest using a 1 hr search.  However, there may be instances in which a 30 minute search would be needed.

By default the UI will only allow 1 hour minimum on the update interval for a collection.

This can be set lower using the attribute dynamicUpdateInterval (0x12a68).  The value here is minutes.

So it could be set to 30 minutes for example.

To set this attribute:

1. Navigate to OneClick Console.

2. Select "Global Collections" on the Navigation Pane

3. Click "List" tab under Contents Pane

4. Now select the Global Collection you wish to change

5. Click on Attributes tab under Component Detail Panel

6. Filter the attributes for dynamicUpdateInterval (0x12a68)

7. Double Click the dynamicUpdateInterval (0x12a68) attribute.

8. Click the "Edit" button on the right side (paper and pencil icon)

9.  Uncheck "No Change" and put in the desired value. 

10. Click Ok.

The attribute is now properly set.

Caveat to this: If the Global Collection is edited (search options, or anything else) this value will get over written by the UI and it will need to be reset again.  This is by design.

Additional Information

Searching Best Practices and Tips and Tricks are provided in the documentation:

https://techdocs.broadcom.com/us/en/ca-enterprise-software/it-operations-management/spectrum/23-3/administrating/oneclick-administration/managing-searches.html