search cancel

Persistent alarms re-generated at SpectroSEVER restart.

book

Article ID: 211869

calendar_today

Updated On:

Products

CA Spectrum DX NetOps

Issue/Introduction

On a non Fault Tolerance SpectroSERVER environment, all existing alarms (even if they are persistent) are re-generated at SpectroSERVER restart.

To verify the issue, add the following debug options in $SPECROOT/SS/.vnmrc file:

ftasv_debug=TRUE
debug=TRUE
persistent_alarms_active=TRUE

Restart the SpectroSERVER process and check for the following in the $SPECROOT/SS/VNM.OUT file:

Mar 11 10:47:39 : Please wait. SpectroSERVER
     is loading landscape 0x700000 at precedence 10...

Mar 11 10:47:39 ERROR TRACE at CsPstAlServ.cc(614): IH REF Attr has no value... cannot restore alarm
Mar 11 10:47:39 ERROR TRACE at CsPstAlServ.cc(614): IH REF Attr has no value... cannot restore alarm
Mar 11 10:47:39 ERROR TRACE at CsPstAlServ.cc(614): IH REF Attr has no value... cannot restore alarm
Mar 11 10:47:39 ERROR TRACE at CsPstAlServ.cc(614): IH REF Attr has no value... cannot restore alarm

Environment

Release : 20.2.7 (10.4.3) 20.2.10 (10.4.3.1)

Component : Spectrum Core / SpectroSERVER

Cause

Code flaw

Resolution

After upgrading to 22.2.2 and before starting the SpectroSERVER, add the following to the .vnmrc file:

restore_persistent_alarms=TRUE 

Additional Information

Take note this only happens in environments that do not have Fault Tolerant backup SpectroSERVERs as there are no backups for alarm synchronization to happen.  You can prevent this entirely by installing Fault Tolerant SS.

 

This was originally thought to have been fixed in 21.2.6 however there was an additional problem with the solution found later in that it will regenerate all alarms that ever existed in the alarm table.  The final solution is included in 22.2.2.

Here is the original 21.2.6 info:

https://techdocs.broadcom.com/us/en/ca-enterprise-software/it-operations-management/spectrum/21-2/release-information/issues-resolved.html

Symptom: Alarms INFERENCE_HANDLER_REF attribute corruption causes SpectroSERVER to re-create the alarms.
Resolution: Code changes are done to use the .vnmrc flag to avoid recreating alarms in SpectroSERVER. (DE518596, 32892848, 21.2.6)

The following KB article covers how to reload the .vnmrc without requiring an SS restart:
https://knowledge.broadcom.com/external/article?articleId=190121