CA Spectrum mysql uses 100% CPU for several hours

book

Article ID: 76585

calendar_today

Updated On:

Products

CA Spectrum

Issue/Introduction

On one of our landscapes, mysql stayed at 100% CPU (one core) for several hours. Event tab in OC stayed at "Obtaining Events ...".

After mysql cpu returned to normal (1-2%), OC client needed to be restarted before any events would appear in event tab. 

Cause

After investigating we found there was a large amount of traps (over 300 per second) being sent to the SpectroSERVER for 6 hours.  These traps were set to be logged in the DDMdb.

The mysql was busy processing the millions of traps.

Environment

Release:
Component: SPCAEM

Resolution

The problem is due to the influx/overload of traps to the mysql db. When the trap rate is 300/second there is just too much data for mysql to be able to handle. This is due to the mysql myisam structure. In CA Spectrum 10.3 we will be moving to the mysql innodb structure which allows for greater flexibility.  There are query efficiencies we are able to make with the updated structure.   Prior to 10.2.3 you will need to either stop the trap storm or disable the logging of the traps that were flooding the SS.