Alarm Generation on Spectrum Server Restart

Products

• CA Spectrum
• DX NetOps

Issue/Introduction

Fault Tolerant Primary Spectrum Server crashed and the database was restored with a healthy SSDB save and restarted.  When the primary was running we noticed there were new alarms for alarms that had already generated.  This caused new tickets to be created in our ticketing system.

What could be the reason alarms are regenerated?

Environment

Release : 10.2.3+

 

Resolution

When the primary SpectroSERVER starts up the secondary SpectroSERVER will send the alarms it knows about back to the primary SS.  When that happens:

• Contact alarms (for example, Device Has Stopped Responding to Polls and Management Agent Lost alarms) will reassert on SS startup and will have new timestamps.  This is because the SS will attempt to contact the device on startup and if it is down, will assert this as a new alarm even if it was down when the secondary SS took over.
• All other alarms which came from the secondary SS will be marked as Stale=Yes and the timestamp of the original alarm will be maintained.

• Any alarms which are listed as Stale=No would indicate these are valid new alarms which are triggered as part of the SS Initialization process.

Additional Information

https://techdocs.broadcom.com/us/en/ca-enterprise-software/it-operations-management/spectrum/22-2/administrating/distributed-spectroserver-administration/spectroserver-alarm-synchronization.html

https://techdocs.broadcom.com/us/en/ca-enterprise-software/it-operations-management/spectrum/22-2/managing-client-applications/alarmnotifier/operating-alarmnotifier/persistent-and-stale-alarms.html