Primary SS keeps crashing

Products

• DX NetOps
• CA Spectrum

Issue/Introduction

Primary SS has crashed twice...

First time I pulled up SCP and did a restore and it seemed to restart fine....

It finished activating the models, but then crashed again...

Environment

All supported DX NetOps Performance Management releases

Cause

How to collect gdb stack trace from a SpectroSERVER core dump on Linux

Thread 1 (Thread 0x7fa14832fcc0 (LWP 32549)):
#0  0x00007fa13cc71468 in CsFindSpec::get_id() const () from /opt/SPECTRUM/lib/libVPapi.so.1
No symbol table info available.
#1  0x00007fa147bf98f1 in CsLandscape::search_calculated_keys(CsFindSpec const*, CsULHashTable*, CsFindSpec::LogicalOp_e, CsULHashTable*, int) () from /opt/SPECTRUM/lib/../SS/libsskrnl.so.1
No symbol table info available.
#2  0x00007fa147bfc370 in CsLandscape::terminal_find_model_handles(CsFindSpec const*, CsULHashTable*, CsFindSpec::LogicalOp_e, CsError::CsError_e*, int) () from /opt/SPECTRUM/lib/../SS/libsskrnl.so.1
No symbol table info available.
#3  0x00007fa147c012d1 in CsLandscape::find_model_handles(CsFindSpec const*, CsError::CsError_e*, CsSecurityIf const*, int) ()
   from /opt/SPECTRUM/lib/../SS/libsskrnl.so.1
No symbol table info available.
#4  0x00007fa147c8d1b9 in SearchManager::model_matches(CsModelHandle const&, SearchManagerNode*, int&) ()
   from /opt/SPECTRUM/lib/../SS/libsskrnl.so.1
No symbol table info available.
#5  0x00007fa147c8d5b7 in SearchManager::process_all_searches(CsModelHandle const&) () from /opt/SPECTRUM/lib/../SS/libsskrnl.so.1
No symbol table info available.

Resolution

We restored to a 2 day old backup that did not experience the crash.

Then we added this value to the .vnmrc (on both primary and secondary SS). After adding it restart the SS.

stack_size_modifier=4

What this will do is allow for an additional 512mb of memory to be used by the SpectroSERVER to help with searches.