Spectrum Online Backup of the SpectroSERVER database is not syncing to the secondary SpectroSERVER

Products

• CA Spectrum
• DX NetOps

Issue/Introduction

Spectrum Online Backup of the SpectroSERVER database is not syncing to the secondary SpectroSERVER.

The $SPECROOT/SS/RCPD.OUT file contains the following information:

Jan 28 12:33:33 : Waiting for remote processd to startup rcpd...
Jan 28 12:34:03 : Waiting for remote processd to startup rcpd...
Jan 28 12:34:33 : Waiting for remote processd to startup rcpd...
Jan 28 12:35:03 : Waiting for remote processd to startup rcpd...

processd on the secondary SpectroSERVER crashes. The following is the stack trace of the core file:

#0  0x080d15ab in CsDynList::add_element(void*) ()
#1  0x080d1804 in CsStrList::add_element(char*) ()
#2  0x080d10e0 in CsPTTMgr::start_process(CsPTTAttr&, int) ()
#3  0x080d126b in CsPTTMgr::start_ticket(CsPTTAttr&, int) ()
#4  0x080cc1d1 in CsServerRequestManager::process_ticket(CsRuntimeTicket&) ()
#5  0x080cd5bb in CsServerRequestManager::create_and_start_ticket(CsPTTAttr::function_ids_e, CsPTTAttr&) ()
#6  0x080cf5c3 in CsServerRequestManager::service_ticket(CsPTTAttr&) ()
#7  0x080c0371 in CsPTTServer::inner_processd_event_parser() ()
#8  0x080c0c76 in CsPTTServer::processd_event_loop() ()
#9  0x080bf8b4 in Processd::run() ()
#10 0x08120bad in UnixDaemonBase::start(bool, int, char**) ()
#11 0x080be8a8 in UnixDaemonBase::main(int, char**) ()
#12 0x080beb93 in main ()

Environment

Release : 20.2.x running on Linux

 

Cause

The environment variable list getting corrupted during online backup sync process.

Resolution

This issue is address in the following patches:

The 10.04.02.01.D111 patch for Spectrum 10.4.2.1

The 10.04.01.D98 patch for Spectrum 10.4.1

This issue will also be address in Spectrum 10.4.3.1 when released. No specific release date set at the time this knowledge article was published.

Additional Information

After installing the D98 or D111 patches and before restarting the Spectrum application, you will need to set following environmental variable in the /etc/environment file:

export PROCESSD_LCOPY=TRUE

Please note:

"export PROCESSD_LCOPY=TRUE" environment variable needs to be available to processd daemon in customer secondary SS environment after OS patch or system reboot/restart. 
check the OLB sync issue after adding "export PROCESSD_LCOPY=TRUE" env variable to $SPECROOT/lib/SDPM/processd_init.sh file on secondary SS machine.
Afterwards reboot the secondary m/c server to check whether PROCESSD_LCOPY env variable properly set before “processd” daemon UP. 
 

Example:(After adding env variable sample "processd_init.sh" file looks as below)

#!/bin/sh
# chkconfig: 35 97 99
# description: SPECTRUM Process Daemon

### BEGIN INIT INFO
# Provides:          processd
# Required-Start:    $local_fs $network
# Required-Stop:     $local_fs $network
# Default-Start:     3 5
# Default-Stop:      0 1 2 6
# Short-Description: SPECTRUM Process Daemon
# Description:  The SPECTRUM Process daemon is
#      responsible for starting/stopping and
#      monitoring SPECTRUM components.
### END INIT INFO
export PROCESSD_LCOPY=TRUE