How to collect GDB stack trace from a SpectroSERVER core dump on Linux

Products

• Spectrum
• Network Observability

Issue/Introduction

When a Spectrum process crashes on Linux, we may ask you to process the core file on the system where is was generated.

This is because the version and files referenced only exist on your system.

The tool you would use is GDB.

Environment

DX NetOps Spectrum: All Supported Versions

Resolution

Please install the Linux GNU GDB Debugger on the system, if not already installed, by running the following command as the root user:

yum install gdb
 
These steps will provide the  "where",  "info threads" and "thread apply all bt full" information from the GDB output.

1 - Login, su or sudo to the root user

2. cd to the $SPECROOT/SS directory and enter the following commands where <CORE> is the filename of the core file:

echo `date` >> gdb.output ; echo "where output" >> gdb.output;

gdb -ex "where" -ex "quit" SpectroSERVER <CORE> >> gdb.output;

echo "info threads output" >> gdb.output;

gdb -ex "info threads" -ex "quit" SpectroSERVER <CORE> >> gdb.output;

echo "thread apply all bt full output" >> gdb.output;

gdb -ex "thread apply all bt full" -ex "quit" SpectroSERVER <CORE> >> gdb.output;

3 - Upload the gdb.output file to the case for analysis.

 

Example of a core file generated by the SpectroSERVER process:

core-SpectroSERVER-sig11-userXXXX-groupXXXX-pid2079677-time1705969295

 

NOTE: any process can generate a core file on Linux, in this example we are using the SpectroSERVER,

but the same steps apply to any core, just run it from the same directory as the core and use the process name that crashed.

Additional Information

To run the above steps manually:

a) As root, cd to the $SPECROOT/SS directory

b) If the file in the SS directory has not been renamed, rename the core
file to something other than core so what if GDB crashes it will not
overwrite the core file (ie. mv core core.1)

c) Execute the following and a few lines of data will print to the screen:

d) gdb SpectroSERVER <core_file>
Example: gdb SpectroSERVER core.1000

Type the following to redirect the output to a text file (gdb.txt)
(gdb) set logging on

You can tell it which file to use:
(gdb) set logging file my_gdb_object.log

And you can examine the current logging configuration:
(gdb) show logging

e) Type "where" (without the quotes) and hit enter

f) Several lines of data (the contents of the stack) will print out

g) Type "info threads" (without the quotes) and hit enter

h) Several lines of data (the contents of the stack) will print out

i) Type "thread apply all bt full" (without the quotes) and hit enter

j) Type $Q to quit out of gdb

k) Upload the $SPECROOT/SS/gdb.txt file.