SOI Connectors losing connections to Spectrum Servers

Products

• CA Spectrum
• CA Service Operations Insight (SOI)

Issue/Introduction

After upgrading SOI to 4.2 Spectrum connector stays in Initializing state and no alarm are imported from spectrum. 
Connector has been upgraded to Connector 259 debug patch.zip and no change.

 

The Spectrum Connector is skipping the connector re-initialization when a heartbeat is missing.

Spectrum IM connector uses CORBA API to regularly communicate and retrieve data from CA Spectrum servers.

You must make sure that there is minimal network latency between the connector and Spectrum server. Poor network performance or high network latency can cause missing heartbeats and re-initialization of connector.


Errors seen in the Spectrum Connector log

Notifying ErrorListeners of exception com.ca.ucf.api.MDRUnavailableException: MDR host xxxxxxxx is unavailable 
com.ca.ucf.api.MDRUnavailableException: MDR host xxxxxxxx is unavailable 

Spectrum Heartbeat missed during Connector startup, ignoring 

 

SAM-integrationService_wrapper.log

NFO   | jvm 1    | 2021/03/30 06:12:38 | 2021/03/30 06:12:38 : xxxxx.xxxx.cxxx : [email protected] : OFFLINE - Connector is restarting due to critical error: com.ca.ucf.api.MDRUnavailableException: MDR host hostname is unavailable - heartbeat missed. : Broadcasting HeartBeat Message

INFO   | jvm 1    | 2021/03/30 06:12:38 | ********** xxxxx.cxxx : [email protected]Stopped.  Connector is restarting due to critical error: com.ca.ucf.api.MDRUnavailableException: MDR host hostname is unavailable - heartbeat missed.

Environment

SOI 4.2

CA Spectrum version 10.4.3

Connector 259 debug patch.zip 

 

Resolution

A new system property (spectrum.connector.heartbeatmiss.ignore) is added to skip the reinitializing the connector if the heartbeat is missing. By default the value of this property is set to 'false'. To enable this property, follow these steps:

1. Go to $SOIROOT\jsw\conf\SAM-IntegrationServices.conf.
2. Change the “spectrum.connector.heartbeatmiss.ignore” property value to true.) **
   (for example, wrapper.java.additional.10=-Dspectrum.connector.heartbeatmiss.ignore=true) 
   Update: You must start with additional.1 and count up without gaps (This is a convention for shoe-horning lists into Java properties syntax). 
3. Restart the IFW Service. (CA SOI Integration Services, previously known as CA SAM Integration Services)

 

**If the key is not in the file, add it:

wrapper.java.additional.25=-Dspectrum.connector.heartbeatmiss.ignore=true

NOTE:

This file may have multiple wrapper.java.additional.xx lines, some are ACTIVE & INACTIVE
 
When we need to add an additional line, it should be increment of last ACTIVE line
 
Ex: This file has wrapper.java.additional.99 but the last active one is 17
So, we need to add wrapper.java.additional.18