Spectrum OneClick message error:
MM DD AAAA HH:MM:SS AM (PoolThread-30: GlobalPool => RubiconEventPoller) (org.apache.commons.httpclient.HttpMethodDirector) - (INFO) - I/O exception (org.apache.commons.httpclient.NoHttpResponseException) caught when processing request: The server <IP> failed to respond
MM DD AAAA HH:MM:SS AM (PoolThread-30: GlobalPool => RubiconEventPoller) (org.apache.axis2.transport.http.impl.httpclient3.HTTPSenderImpl) - (INFO) - Unable to sendViaPost to url[http://<IP>:8281/EventManager/EventManagerWS.asmx]
java.net.ConnectException: Connection refused: connect
MM DD AAAA HH:MM:SS PM (PoolThread-346: GlobalPool => RubiconEventPoller) (org.apache.axis2.transport.http.impl.httpclient3.HTTPSenderImpl) - (INFO) - Unable to sendViaPost to url[http://<IP>:8281/EventManager/EventManagerWS.asmx]
java.net.SocketTimeoutException: Read timed out
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
options.setProperty(HTTPConstants.SO_TIMEOUT,
(int) this.eventManager.getWebServiceTimeoutInMS() );
PM Version 21.x
Spectrum Version 21.x
The Spectrum OC is sending a big block of events for PM Event Manager.
Check in wrapper-*.log file from /opt/CA/PerformanceCenter/EM/logs folder on NetOps Portal Server:
This message show how many events was found:
INFO | jvm 1 | AAAA/MM/DD HH:MM:SS | INFO | qtp517420854-21 | AAAA/MM/DD HH:MM:SS,131 | com.ca.im.portal.em.util.NPCEventManagerWSImpl
INFO | jvm 1 | AAAA/MM/DD HH:MM:SS | | getEventListOrUpdatesWithAllProperties, ItemID=116, utcStartTime=1636469258, utcEndTime=1636555658: 53338 events
But then get an error:
INFO | jvm 1 | AAAA/MM/DD HH:MM:SS | INFO | qtp517420854-21 | AAAA/MM/DD HH:MM:SS,134 | org.apache.cxf.binding.soap.interceptor.Soap12FaultOutInterceptor
INFO | jvm 1 | AAAA/MM/DD HH:MM:SS | | class org.apache.cxf.binding.soap.interceptor.Soap12FaultOutInterceptor$Soap12FaultOutInterceptorInternalapplication/soap+xml
And "response code 500" meaning server error.
INFO | jvm 1 | AAAA/MM/DD HH:MM:SS | INFO | qtp517420854-21 | AAAA/MM/DD HH:MM:SS,135 | org.apache.cxf.services.EventManagerWS.EventManagerWSSoap.EventManagerWSSoap
INFO | jvm 1 | AAAA/MM/DD HH:MM:SS | | Outbound Message
INFO | jvm 1 | AAAA/MM/DD HH:MM:SS | ---------------------------
INFO | jvm 1 | AAAA/MM/DD HH:MM:SS | ID: 4680
INFO | jvm 1 | AAAA/MM/DD HH:MM:SS | Response-Code: 500
INFO | jvm 1 | AAAA/MM/DD HH:MM:SS | Encoding: UTF-8
INFO | jvm 1 | AAAA/MM/DD HH:MM:SS | Content-Type: application/soap+xml
INFO | jvm 1 | AAAA/MM/DD HH:MM:SS | Headers: {}
INFO | jvm 1 | AAAA/MM/DD HH:MM:SS | Payload: <soap:Envelope xmlns:soap="http://www.w3.org/2003/05/soap-envelope"><soap:Body>
The problem is it found 53k events and the Spectrum OC doesn't send a limit so the EM try and build a SOAP body of 53k events and it fail
According to the EM log message the EM is trying to get a big block of events and it hasn't completed the call since the start timestamp: