EventForwarder :97 | Fatal error occur

Products

• CA Privileged Access Manager - Server Control (PAMSC)

Issue/Introduction

Hi,

We have PIM 12.9SP2 Distribution servers on Windows 1012R2. We are not able to send data to Splunk and we are seeing below error in EventForwarder log.

2021-12-20 07:56:15.697 | ERROR | main       | com.ca.ppm.eventForwarder.core.EventForwarder     :97    | Fatal error occur, restarting the configuration
2021-12-20 07:56:15.697 | INFO  | main       | com.ca.ppm.eventForwarder.core.EventForwarder     :100   | Closing configuration
2021-12-20 07:56:22.705 | INFO  | Listener-1 | com.ca.ppm.eventForwarder.services.DefaultMessageListenerContainerOverride:39    | org.springframework.jms.listener.AbstractJmsListeningContainer$SharedConnectionNotInitializedException: This listener container's shared Connection has not been initialized yet
2021-12-20 07:56:22.705 | INFO  | Listener-1 | com.ca.ppm.eventForwarder.services.DefaultMessageListenerContainerOverride:42    | org.springframework.jms.listener.AbstractJmsListeningContainer$SharedConnectionNotInitializedException: This listener container's shared Connection has not been initialized yet
2021-12-20 07:56:25.709 | INFO  | Listener-1 | com.ca.ppm.eventForwarder.services.DefaultMessageListenerContainerOverride:39    | org.springframework.jms.listener.AbstractJmsListeningContainer$SharedConnectionNotInitializedException: This listener container's shared Connection has not been initialized yet
2021-12-20 07:56:25.709 | INFO  | main       | com.ca.ppm.eventForwarder.core.EventForwarder     :113   | Starting configuration
2021-12-20 07:56:25.709 | INFO  | Listener-1 | com.ca.ppm.eventForwarder.services.DefaultMessageListenerContainerOverride:42    | org.springframework.jms.listener.AbstractJmsListeningContainer$SharedConnectionNotInitializedException: This listener container's shared Connection has not been initialized yet
2021-12-20 07:56:25.709 | INFO  | Listener-1 | com.ca.ppm.eventForwarder.services.DefaultMessageListenerContainerOverride:65    | javax.jms.JMSSecurityException: Failed to connect via SSL to [ssl://127.0.0.1:7243]: Socket closed
2021-12-20 07:56:25.709 | INFO  | Listener-1 | com.ca.ppm.eventForwarder.services.DefaultMessageListenerContainerOverride:70    | javax.jms.JMSSecurityException: Failed to connect via SSL to [ssl://127.0.0.1:7243]: Socket closed
2021-12-20 07:56:45.726 | INFO  | main       | com.ca.ppm.springAppCore.jms.PropertyPlaceholderConfigurerOverride:101   |  messageQueueUserPassXmlFIlePath received as ::  ../../../properties-service.xml
2021-12-20 07:56:45.726 | INFO  | main       | com.ca.ppm.springAppCore.jms.PropertyPlaceholderConfigurerOverride:108   |  xmlPathParser Initialization is successful 
2021-12-20 07:56:45.726 | INFO  | main       | com.ca.ppm.springAppCore.jms.PropertyPlaceholderConfigurerOverride:48    | Message Queue server URL: ssl://127.0.0.1:7243
2021-12-20 07:56:45.726 | INFO  | main       | com.ca.ppm.springAppCore.jms.PropertyPlaceholderConfigurerOverride:60    | Host name resolved to: XXXXXXXXXv
2021-12-20 07:56:45.726 | INFO  | main       | com.ca.ppm.eventForwarder.services.EventForwarderDataImpl:133   | Load 96 properties from:../conf/CM_CEF_MAPPING.properties
2021-12-20 07:56:45.726 | INFO  | main       | com.ca.ppm.eventForwarder.services.EventForwarderDataImpl:133   | Load 3 properties from:../conf/config.properties
2021-12-20 07:56:45.726 | INFO  | main       | com.ca.ppm.eventForwarder.services.EventForwarderDataImpl:133   | Load 1 properties from:../conf/standalone.properties
2021-12-20 07:56:45.726 | INFO  | main       | com.ca.ppm.eventForwarder.services.EventForwarderDataImpl:88    | Initializing event forwarder
2021-12-20 07:56:45.726 | INFO  | main       | com.ca.ppm.eventForwarder.services.ForwarderManagerImpl:244   | configuration message received from the server:
TENANT:
TENANT=1
ENABLE=TRUE
FORMAT=AUDIT_FORMAT_CEF
HOST=yyyyyyyyyy:10584
2021-12-20 07:56:45.726 | INFO  | main       | com.ca.ppm.eventForwarder.services.Log4JForwarder :62    | Loading syslog configuration from: ../conf/events_log4j2.xml
2021-12-20 07:56:47.929 | INFO  | main       | com.ca.ppm.eventForwarder.services.Log4JForwarder :190   | Add primary syslog appender: yyyyyyyyyy

Also

2021-12-20 07:56:25.709 | INFO  | Listener-1 | com.ca.ppm.eventForwarder.services.DefaultMessageListenerContainerOverride:42    | org.springframework.jms.listener.AbstractJmsListeningContainer$SharedConnectionNotInitializedException: This listener container's shared Connection has not been initialized yet
2021-12-20 07:56:25.709 | INFO  | Listener-1 | com.ca.ppm.eventForwarder.services.DefaultMessageListenerContainerOverride:65    | javax.jms.JMSSecurityException: Failed to connect via SSL to [ssl://127.0.0.1:7243]: Socket closed
2021-12-20 07:56:25.709 | INFO  | Listener-1 | com.ca.ppm.eventForwarder.services.DefaultMessageListenerContainerOverride:70    | javax.jms.JMSSecurityException: Failed to connect via SSL to [ssl://127.0.0.1:7243]: Socket closed
2021-12-20 07:56:45.726 | INFO  | main       | com.ca.ppm.springAppCore.jms.PropertyPlaceholderConfigurerOverride:101   |  messageQueueUserPassXmlFIlePath received as ::  ../../../properties-service.xml
2021-12-20 07:56:45.726 | INFO  | main       | com.ca.ppm.springAppCore.jms.PropertyPlaceholderConfigurerOverride:108   |  xmlPathParser Initialization is successful 
2021-12-20 07:56:45.726 | INFO  | main       | com.ca.ppm.springAppCore.jms.PropertyPlaceholderConfigurerOverride:48    | Message Queue server URL: ssl://127.0.0.1:7243
2021-12-20 07:56:45.726 | INFO  | main       | com.ca.ppm.springAppCore.jms.PropertyPlaceholderConfigurerOverride:60    | Host name resolved to: XXXXXXXXX
2021-12-20 07:56:45.726 | INFO  | main       | com.ca.ppm.eventForwarder.services.EventForwarderDataImpl:133   | Load 96 properties from:../conf/CM_CEF_MAPPING.properties

Environment

Release : 14.0

Component :

Cause

A temporary network issue caused the tibco queues to fill up and stop responding to eventforwarder requests.

Resolution

After ensuring the Splunk service could receive the data restarting both the Tibco and EventForwarder services restored functionality