Archive Manager hanging while browsing events in OneClick

Products

• CA Spectrum

Issue/Introduction

We had a problem with our DDM, where it stops to deliver the service despite the process being up.  A restart resolves the problem, but how can we avoid this?

We had these messages in the logs :

Jul 31 10:45:28 WARNING at CsWorkSched.cc(173): Low thread resources detected. Work scheduler requesting 2min/40max threads, but not available.
Jul 31 10:45:35 WARNING at CsWorkSched.cc(173): Low thread resources detected. Work scheduler requesting 2min/40max threads, but not available.

This may happen with containers that have an extremely large number of models, especially parent containers that hold multiple child containers, that have a large number of events.

From the slow query log (see below for instructions on how to enable the Slow Query Logging for MySQL), we see the multiple slow queries:

The largest ones are standard event queries for a group of devices, as would happen when you browse in OneClick to a new container in the event tab.
E.g. SELECT E.model_h, M.model_name, M.mtype_h, MT.mtype_name, E.utime, E.counter, E.clk_seq, E.version, E.node_id, U.user_name, E.type, E.severity, E.vardata_string FROM event as E, model as M, model_type as MT, user_def as U WHERE E.model_h = M.model_h AND M.mtype_h = MT.mtype_h AND E.user_key = U.user_key AND M.model_h IN

These are followed by 1000’s of models so a huge number of events are being pulled from the DDMDB and using a lot of resources.

This causes performance problems for the Archive Manager database.

Environment

Release: Any
Component: SPCAEM

Resolution

It is not recommendable to open the events tab on such a large container of models.

To avoid this, avoid opening the events tab on parent containers that hold multiple child containers of models.

Drill down into a child container before opening the events tab.

Additional Information

Please see knowledge article "How can I enable the Slow Query Logging for MySQL?" for more information.