How does CA Spectrum determine a trap storm has been detected?

book

Article ID: 14660

calendar_today

Updated On:

Products

CA Spectrum

Issue/Introduction

We are seeing "A TRAP STORM HAS BEEN DETECTED" alarms frequently generated on some devices.  How does Spectrum determine a trap storm has been detected?

Cause

The CA Spectrum documentation explains how the Spectrum Trap Storm detection works under the "How Trap Storm Detection Works" section:  

Broadcom TechDocs : CA Spectrum 10.4.x - Trap Management Subview


What it does not explain is the underlying code used to make that determination. 

Environment

Spectrum 10.4.x

Resolution

According to the references noted above, you can enable the trap storm detection at your SpectroSERVER or at the level of a modelled device. When devices that are modelled in CA Spectrum send more than 20 traps per second, you must adjust traps_per_sec_storm_threshold so that trap storm detection does not limit the ability to receive traps.


You can enable the trap storm detection at any level by configuring the following two attributes. These attributes are available under the Attributes Tab in the Component detail pane for the selected VNM model or for a selected device model:

traps_per_sec_storm_threshold

Defines the rate at which traps are received per second from a managed or unmanaged device. When this rate is sustained for the amount of time that is specified by the TrapStormLength, the SpectroSERVER stops the processing of traps from that unmanaged or managed device.
Default: 20 traps per second

TrapStormLength


Defines the time in seconds for which the traps_per_sec_storm_threshold value is sustained. SpectroSERVER considers it a trap storm and disables the processing of traps from that unmanaged or managed device.
Default: 5 seconds


When traps received from any device reach the configured thresholds, the SpectroSERVER identifies this rate as a trap storm. The SpectroSERVER stops handling traps from that device and traps from other devices are not blocked. SpectroSERVER trap storm detection logic is based on each IP address of an unmanaged or a managed device (trap source) that sends traps to SpectroSERVER. As a result, you can configure each device to send traps to the SpectroSERVER at the appropriate rate."


One important thing to remember from the above text is the word "rate". The underlying formula Spectrum uses to determine if there is a trap storm is as follows:

in_storm = ( sum/TrapStormLength >= trap_storm_size ) ? TRUE : FALSE;

The "sum" is the number of traps received over a time period. Using the above formula above and the default values for traps_per_sec_storm_threshold and TrapStormLength, if the device received 100 (sum) traps in 3 seconds, the calculation would be as follows:

100/5 >=20

In the above scenario, even though the sample of traps was received over a 3 second period, according to the formula used, the average number of traps is equal to or exceeds 20 traps per second over a 5 second period so Spectrum will detect a trap storm, assert an alarm and stop processing traps for that device until the rate falls below the configured parameters.