OLB stops secondary SS and fails to restart it.

Products

• CA Spectrum

Issue/Introduction

When the OLB is performed the rcpd stops the secondary SS, but It then fails to restart SS

The RPCD.OUT shows

Jun 18 12:39:45 : rcpd started

 command:  RECV
    file:  .ft_save_file.SSdb
    rcpd:  0xcafe

file name: .ft_save_file.SSdb.gz
file size: 291581741
 compress: 1
peer host: xxxxxx-primary

Jun 18 12:40:00 : Starting file transfer using 1048576 byte TCP receive socket buffer.
Jun 18 12:49:08 : Successfully received .ft_save_file.SSdb.gz
Jun 18 12:49:08 : Uncompressing the database file...success.
Jun 18 12:49:22 : Stopping the SpectroSERVER...success.
Jun 18 12:49:26 : Running SSdbload on the uncompressed database file...success.
Jun 18 12:50:35 : Restarting the SpectroSERVER...success.
Jun 18 12:50:35 : Exiting.

The RPCD.OUT does not show any issue, in fact It shows "Restarting the SpectroSERVER...success.", but if you run "ps -ef | grep SpectroSERVER" the SS is not running.

Enabling the debug on processd  in the processd_log you can see a segmentation violation...

Jun 18 12:50:37 DATE OF THIS REQUEST

ticket function id         = 257
ticket cb function id      = 256
ticket exchange id         = 8
ticket more data           = 0
ticket version number      = 6
ticket process_pid         = 25245
ticket process_ppid        = 0
ticket process_gpid        = 25245
ticket process_state       = 1
ticket how_many_procs      = 3
ticket auto_restart        = 0
ticket admin privileges    = 1
ticket server process      = 1
ticket windows service     = 0
ticket dont save ticket    = 0
ticket state based         = 1
ticket user_id             = 500
ticket user_name           = nmsdist
ticket requester user_name = 
ticket group user_id       = 100
ticket group user_name     = users
ticket part_name           = SS
ticket application_name    = SpectroSERVER Daemon
ticket work_pathname       = /spectrum/SS
ticket log_name            = /spectrum/SS/VNM.OUT
ticket log_backup_max      = 10
ticket output file pos     = 0
ticket output_file         = 
ticket signal upon exit    = 11
signal error string = a Segmentation Violation
ticket num of errnos       = 1
errnos                     = 0
ticket retry timeout       = 0
ticket retry maximum       = 0
ticket retry current       = 0
ticket timestamp started   = 1624005925
ticket start_priority      = 20

...

Environment

Release : 20.2.3 (10.4.2.1)

Component : Spectrum Core / SpectroSERVER

Resolution

 

From Spectrum 10.4.2.1 the DX NetOps Spectrum processes are registered with systemctl instead of /etc/init.d 

For processd it is recommended to use the “systemctl” command to control “processd” daemon.

Starting the processd on the secondary SS as follows:

 systemctl daemon-reload  

 systemctl stop processd

 systemctl start processd

and then SpectroSERVER with the install owner user, allowed OLB to work properly and the SpectroSERVER to be successfully restarted.