Spectrum server crashes in AddSecureDomainValue after upgrade

Products

• CA Spectrum
• DX NetOps

Issue/Introduction

Spectrum server crashes after 10.4.1/20.x/21.x/22.x upgrade with AddSecureDomainValue in the stack:

Linux stack:

#3  0x00007f146fdf275c in SnmpV3ProfileGlobalConfigIH::AddSecureDomainValue(CsModelHandle const&, CsBuffer*, int, int) () from /opt/SPECTRUM/lib/../SS/libmdlsvint.so.1
#4  0x00007f146fdf4101 in SnmpV3ProfileGlobalConfigIH::trig_model_create(CsModelHandle const&) (  from /opt/SPECTRUM/lib/../SS/libmdlsvint.so.1

Windows stack:

0000006f`b455f440 00007ffe`3788fb5f : 00000000`00000000 0000006f`b455f560 0000021f`83c60810 00000000`00000000 : libhpse!CsHPSESnmpv3Info::CsHPSESnmpv3Info+0x197

0000006f`b455f500 00007ffe`37891a57 : 0000021f`8464d410 0000021f`83d13ea0 0000745a`00000004 0000006f`00000079 : libmdlsvint!SnmpV3ProfileGlobalConfigIH::AddSecureDomainValue+0xcf

0000006f`b455f5a0 00007ffe`381ca3ae : 0000021f`846557dc 0000006f`b455f780 0000021f`00000000 00000000`00000000 : libmdlsvint!SnmpV3ProfileGlobalConfigIH::trig_model_create+0xe3

Environment

Release : All Supported Releases

Component : SPCCSS - Spectrum Core / SpectroSERVER

Cause

SNMPv3 Profile Sync Issue

Resolution

To resolve this, you need to reload the SpectroSERVER models only portion of the database from before the upgrade.

The Spectrum install saves the SSdb before it migrates it.  The savefile is in the SS directory, and will have a ,1 after it.  For example:

db_20191008,1.SSdb

Then after the upgrade, the db is saved again:

db_20191008,1.after.SSdb

To resolve this, you can initialize the SSdb to legacy, then load the models only with the following instructions:

As spectrum install owner, in a bash shell navigate to the SS directory.

a) Delete the $SPECROOT/SS/*.db and *.ix files:

• $ rm -f *.db
• $ rm -f *.ix

Ensure they were removed.

b) Delete the $SPECROOT/SS/.VNMDB.LOCK file:

• $ rm -f .VNMDB.LOCK

c) Load the SSdb file:

../SS-Tools/SSdbload -il legacy.SSdb
../SS-Tools/SSdbload -m db_20191008,1.SSdb

OR

$ ../SS-Tools/SSdbload -il db_yyyymmdd,1.after.ssdb

d) Start the SpectroSERVER

Additional Information

Clear the Linux cache:

# echo 1 > /proc/sys/vm/drop_caches (logged as root)

 

If the crash keeps happening after following these steps for Step C. please use an older database.

 

../SS-Tools/SSdbload -m db_20191008,1.SSdb

 

use a SSdb before the upgrade from your saved SSdb databases