SpectroSERVER Linux crashes without core or with core but GDB trace shows no data

Products

• Spectrum

Issue/Introduction

One of your SpectroSERVERs is crashing regularly and when we run the GDB trace we see a stack without good information in it.

(gdb) where
#0  0x00007f78b944fe26 in ?? ()
#1  0x00007f78bdcbe78e in ?? ()
#2  0x00007f76e9a88548 in ?? ()
#3  0x00007f76e9a883c0 in ?? ()
#4  0x00007f770edd4490 in ?? ()
#5  0x00007f78bdcc715f in ?? ()
#6  0x00007f781cb2f200 in ?? ()
#7  0x00007f76e9a88548 in ?? ()
#8  0x00007f76e9a8b598 in ?? ()
#9  0x0000000000000000 in ?? ()

 

Or no core is dumped at all.

Cause

Not enough memory to write the stack at the time of crash when core file was created.

Resolution

To reduce the chance of this happening again, add the following line to both the primary and secondary $SPECROOT/SS/.vnmrc file

stack_size_modifier=4

The SpectroSERVERs will need to be restarted for the setting to take effect.  Please notes the Secondary SpectroSERVER will restart itself after a successful online backup.

Additional Information

What this setting does is allocate an extra 512mb of ram for the SpectroSERVER process which will help to write a good stack in case of a future crash.  Also it provides extra memory to buffer the SpectroSERVER in case of an inefficient search and will help avoid crashes.

Please note the SpectroSERVER process will grow by 512 megs of memory at the very most, which should not be a concern in today´s environments.