Error connecting to ServiceNow service, shutting down connector

Products

• CA Service Operations Insight (SOI)

Issue/Introduction

I see the following error message in the log file.  However, the connector status is showing as Green and running normally. Would like to know why these error's appearing

2019-11-21 22:11:09.688 [FATAL] (MonitorThread-ComputerSystem) com.ca.usm.ucf.servicenow.ServiceNowConnector  - Error connecting to ServiceNow service, shutting down connector

2019-11-21 22:11:09.688 [ERROR] (MonitorThread-ComputerSystem) com.ca.usm.ucf.servicenow.ServiceNowConnector  - Error detected during detectEvents(): Error invoking web service 'getRecords' operation for table 'cmdb_ci_computer'.  Fields: [__encoded_query,sys_updated_on>=2019-11-22 04:09:23^sys_updated_on<2019-11-22 04:10:18^sys_class_name=cmdb_ci_computer]. Connector will restart.

[DomainManagerException: errorType=10, errorCode=FATAL, connectorName=N/A, message=Error invoking web service 'getRecords' operation for table 'cmdb_ci_computer'.  Fields: [__encoded_query,sys_updated_on>=2019-11-22 04:09:23^sys_updated_on<2019-11-22 04:10:18^sys_class_name=cmdb_ci_computer]. Connector will restart., faultData={}, cause=com.ca.usm.ucf.servicenow.mdr.ServiceNowConnectorException: Error invoking web service 'getRecords' operation for table 'cmdb_ci_computer'.  Fields: [__encoded_query,sys_updated_on>=2019-11-22 04:09:23^sys_updated_on<2019-11-22 04:10:18^sys_class_name=cmdb_ci_computer]. Connector will restart.]

at com.ca.usm.ucf.servicenow.mdr.MonitorThread.detectEvents(MonitorThread.java:142)[156:com.ca.usm.ucf.ServiceNowConnector:4.6.0]

at com.ca.usm.ucf.servicenow.mdr.MonitorThread.run(MonitorThread.java:79)[156:com.ca.usm.ucf.ServiceNowConnector:4.6.0]

Caused by: com.ca.usm.ucf.servicenow.mdr.ServiceNowConnectorException: Error invoking web service 'getRecords' operation for table 'cmdb_ci_computer'.  Fields: [__encoded_query,sys_updated_on>=2019-11-22 04:09:23^sys_updated_on<2019-11-22 04:10:18^sys_class_name=cmdb_ci_computer]. Connector will restart.

at com.ca.usm.ucf.servicenow.mdr.ExceptionFactory.newServiceNowConnectorEx(ExceptionFactory.java:237)[156:com.ca.usm.ucf.ServiceNowConnector:4.6.0]

at com.ca.usm.ucf.servicenow.mdr.ExceptionFactory.domainManagerException(ExceptionFactory.java:64)[156:com.ca.usm.ucf.ServiceNowConnector:4.6.0]

at com.ca.usm.ucf.servicenow.service.DispatchWebService.doInvoke(DispatchWebService.java:109)[156:com.ca.usm.ucf.ServiceNowConnector:4.6.0]

at com.ca.usm.ucf.servicenow.service.DispatchWebService.invoke(DispatchWebService.java:94)[156:com.ca.usm.ucf.ServiceNowConnector:4.6.0]

at com.ca.usm.ucf.servicenow.mdr.MonitorThread.getRecords(MonitorThread.java:208)[156:com.ca.usm.ucf.ServiceNowConnector:4.6.0]

at com.ca.usm.ucf.servicenow.mdr.MonitorThread.publishRecordsBetween(MonitorThread.java:186)[156:com.ca.usm.ucf.ServiceNowConnector:4.6.0]

at com.ca.usm.ucf.servicenow.mdr.MonitorThread.detectEvents(MonitorThread.java:123)[156:com.ca.usm.ucf.ServiceNowConnector:4.6.0]

... 1 more

Caused by: com.sun.xml.internal.ws.client.ClientTransportException: HTTP transport error: java.net.UnknownHostException: baeprod.service-now.com

at com.sun.xml.internal.ws.transport.http.client.HttpClientTransport.getOutput(HttpClientTransport.java:117)[:1.8.0_212]

at com.sun.xml.internal.ws.transport.http.client.HttpTransportPipe.process(HttpTransportPipe.java:208)[:1.8.0_212]

at com.sun.xml.internal.ws.transport.http.client.HttpTransportPipe.processRequest(HttpTransportPipe.java:130)[:1.8.0_212]

at com.sun.xml.internal.ws.transport.DeferredTransportPipe.processRequest(DeferredTransportPipe.java:124)[:1.8.0_212]

at com.sun.xml.internal.ws.api.pipe.Fiber.__doRun(Fiber.java:1121)[:1.8.0_212]

at com.sun.xml.internal.ws.api.pipe.Fiber._doRun(Fiber.java:1035)[:1.8.0_212]

at com.sun.xml.internal.ws.api.pipe.Fiber.doRun(Fiber.java:1004)[:1.8.0_212]

at com.sun.xml.internal.ws.api.pipe.Fiber.runSync(Fiber.java:862)[:1.8.0_212]

at com.sun.xml.internal.ws.client.Stub.process(Stub.java:448)[:1.8.0_212]

at com.sun.xml.internal.ws.client.dispatch.DispatchImpl.doInvoke(DispatchImpl.java:250)[:1.8.0_212]

at com.sun.xml.internal.ws.client.dispatch.DispatchImpl.invoke(DispatchImpl.java:289)[:1.8.0_212]

at com.ca.usm.ucf.servicenow.service.DispatchWebService.fetchServiceResult(DispatchWebService.java:190)[156:com.ca.usm.ucf.ServiceNowConnector:4.6.0]

at com.ca.usm.ucf.servicenow.service.DispatchWebService.invokeTry(DispatchWebService.java:154)[156:com.ca.usm.ucf.ServiceNowConnector:4.6.0]

at com.ca.usm.ucf.servicenow.service.DispatchWebService.invokeWithRetry(DispatchWebService.java:125)[156:com.ca.usm.ucf.ServiceNowConnector:4.6.0]

at com.ca.usm.ucf.servicenow.service.DispatchWebService.doInvoke(DispatchWebService.java:101)[156:com.ca.usm.ucf.ServiceNowConnector:4.6.0]

... 5 more

Environment

Release : 4.2

Component : Service Operations Insight (SOI) Manager

Cause

The connector "MonitorThread" having problem after a disconnect from ServiceNow.

Resolution

Most of the time the ServiceNow connector "MonitorThread" will recover from the disconnect from the ServiceNow but if you still see the error's then do the following

1. Stop CA Catalyst Container Service.

2. Clear logs/files from below folders, including the logs in folder's within these folder's

    \CA\Catalyst\CatalystConnector\ifw\log

    \CA\Catalyst\CatalystConnector\jsw\logs

    \CA\Catalyst\CatalystConnector\container\data\log

    \CA\Catalyst\CatalystConnector\container\registry-cache

    \CA\Catalyst\CatalystConnector\container\CatalystDatastore

    \CA\Catalyst\CatalystConnector\container\nls-store

3. Start the service

Note:

The SN connector attempts to recover from a lost network connection retrycount times (default = 3), every retryinterval seconds (default = 5000 ms). You can increase these numbers if you see frequent disconnection. 

The configuration file is located in CA\Catalyst\CatalystConnector\registry\topology\physical\<HostName>_CatalystConnector\modules\configuration. Catalyst Container service needs to be recycled once the change is made.