GetEventTypes error seen in the NetOps Portal EMService.log

Products

CA Performance Management - Usage and Administration DX NetOps

Issue/Introduction

The following error is seen in the EMService.log file. Default location for the file is /opt/CA/PerformanceCenter/EM/logs.

ERROR | pool-1-thread-289 | 2021-08-04 15:37:19,568 | com.ca.im.portal.em.ds.EventProducerPoller
| Exception during GetEventTypes
javax.xml.ws.WebServiceException: Could not send Message.
at org.apache.cxf.jaxws.JaxWsClientProxy.invoke(JaxWsClientProxy.java:145)
at com.sun.proxy.$Proxy103.getEventTypes(Unknown Source)
at com.ca.im.portal.em.ds.EventProducerPoller.pullMetaData(EventProducerPoller.java:518)
at com.ca.im.portal.em.ds.EventProducerPoller.poll(EventProducerPoller.java:303)
at com.ca.im.portal.em.ds.EventProducerPoller.poll(EventProducerPoller.java:147)
at com.ca.im.portal.em.util.PollCycleTask.runPollingSteps(PollCycleTask.java:114)
at com.ca.im.portal.em.util.PollCycleTask.run(PollCycleTask.java:67)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.ConnectException: ConnectException invoking http://123.123.123.123:8181/pc/center/eventProducerWS: Connection refused (Connection refused)
at sun.reflect.GeneratedConstructorAccessor288.newInstance(Unknown Source)

When SSL is enabled, the error can be seen using the correct schema (HTTPS) and port (8182 default) but the wrong hostname or IP address. This scenario is often observed after a migration to new systems.

Environment

All supported DX NetOps Performance Management releases

Cause

After a host migration, or after HTTPS configuration is enabled, the information in the MySql em database is not updated with newly configured schema, port or hostname.

Resolution

To resolve this complete the following steps.

Open a terminal to the PC Portal web server host.
As the install owner run the following MySql command.
- Default install path shown. Enter password when prompted.
- /opt/CA/MySql/bin/mysql -uroot -p -e 'update em.data_sources set LastEvent=0 where sourceid=0;'
After running the command wait 15-20 minutes (30 minutes max) for the updated values to be passed around the system.

After everything is completed we should no longer see the same error in the EMService.log file.

The command forces the Event Manager service to gather fresh information about the target endpoint, the PC Console service. Once the em database is updated with the correct information it will allow successful communication attempts to succeed, resolving the error.

Additional Information

How do we know if the information changes in the database? The following query can be used to watch the value while waiting for Data Source synchronizations to complete and pass around the new value.

Default install path shown. Enter password when prompted.

/opt/CA/MySql/bin/mysql -uroot -p -e 'select SourceID,SourceGUID,ConsoleName,Host,SourceName,IEventProducer from em.data_sources\G;'