SpectroSERVER on Linux crashes with SIGSEGV in /var/log/messages

Products

• CA Spectrum

Issue/Introduction

Our SpectroSERVER on Linux crashes occasionally, sometimes with a core file and sometimes without.  When we check the GDB trace from the core file we are not able to determine the cause.

We do not see anything written to the VNM.OUT at the time of the crash.

When we check the /var/logs/messages we see the following at the time of the crash

 

Nov 14 09:08:33 <hostname> abrt-hook-ccpp: Process 28249 (SpectroSERVER) of user 16697 killed by SIGSEGV - dumping core
Nov 14 09:08:33 <hostname> abrt-hook-ccpp: Process 28249 (SpectroSERVER) of user 16697 killed by SIGSEGV - dumping core

Environment

Release : 10.x, 20.x, 21.x

Component :SPCSS  CORE

Red Hat Linux

Cause

SpectroSERVER process is being killed by kernel due to insufficient memory resource issues

Resolution

The memory increase may be the result of a higher incoming trap load or a network backbone issue that causes Spectrum´s Fault Isolation to require more resources, including memory.

The memory increase may not be directly attributed to the SpectroSERVER process, but could also be the result of higher trap loads causing MYSQL and Archive Manager process to increase and grow higher then before.

Increasing the physical memory on the server is the only long term solution as reducing trap loads may only be a temporary solution until the devices start sending more traps again.