SpectroSERVER crashing due to multiple instances running

Products

• CA Spectrum
• DX NetOps

Issue/Introduction

The SpectroSERVER process appears to have multiple instances running:

So it is not running stably and crashes randomly.

Environment

DX NetOps Spectrum 22.2.x

Cause

The following is seen repeatedly in the VNM.OUT SpectroSERVER log:

Jun 12 07:58:32 : Validating and loading SPECTRUM Intelligence Modules.
Jun 12 07:58:34 : All SPECTRUM Intelligence Modules successfully loaded.

Database already locked by: spectrum,
                by process: /appl/Spectrum/SS/SpectroSERVER,
             by process ID: 75521,
           on network node: my_Spectro,
          which started at: 06/12/2024_07:31:20.

Could not open the database. VNM exiting.
Landscape not initialized. VNM exiting.

Attempting to bind connection port 0xbeef

VNM.OUT.bak.202406120758370000644004511200451100000000412714632234415014534 0ustar  spectrumspectrumJun 12 07:58:35 : /appl/Spectrum/SS/SpectroSERVER starting
Jun 12 07:58:35 : MTS enabled, thread heaps = 5

After crashing, restarting it again results in the above due to a prior instance still running on restart. So the system is unable to operate stably due to the fact there are always two instances of the SpectroSERVER running.

Resolution

Shutdown the SpectroSERVER process properly:

$SPECROOT/bin/stopSS.pl

Make sure there are no other instance of it before restarting. On linux, run:

ps -ef | grep -i spectro

This will show if there are any instance of the SpectroSERVER still running. On Windows, Check in the Task Manager for "SpectroSERVER" process.

Ensure that the SpectroSERVER is only restarted under the Spectrum Admin user account, and not under any others, including by automated scripts running under Linux cron that may spawn a separate user shell when starting.