No new Threshold Violation Events seen in DX NetOps Portal

Products

CA Performance Management - Usage and Administration DX NetOps

Issue/Introduction

Events not seen in CAPC

After upgrade Events are not seen in the Performance Management.

We see the following error is seen in the Data Aggregator /opt/IMDataAggregator/apache-karaf-2.4.3/data/log/karaf.log* files (your installation path may vary if not default).

It also references the wrong hostname or IP address for the Performance Management server.

Another place to look is the DA rest endpoint for event registration:

http://YOURDA:8581/rest/emregistration

The eventConsumerURL should point to the correct Performance Management host but may not if this error is seen.

To troubleshoot do the following:

Run the following from the Data Aggregator:
- For 20.2.x: curl -v http://<expected IP>:8281/EventManager/EventManagerWS.asmx?wsdl
- For 21.2.x: curl -kv -u admin http://<expected IP>:8281/EventManager/EventManagerWS.asmx?wsdl
  - This one will prompt for the admin password

Those curl commands should succeed with a 200 OK message and show the correct host information. If it doesn't then the IP is wrong in the event manager general table possibly because there are multiple NICs on the box

Environment

All supported DX NetOps Performance Management releases

Cause

Wrong IP address registered with the DA

Resolution

Complete the following to resolve this.

Go to Administration->Data Sources->Data Sources.
1. Select the Event Manager Data Source.
2. Choose the Edit button.
3. In the Edit UI for the Event Manager Data Source:
  1. Edit the Host Name ensuring the correct one is set.
This is going to update the MySql em database general table after it syncs. You can verify that by doing the following on the CAPC host.
1. Enter the mysql prompt from /opt/CA/MySql/bin (default dir):
  1. mysql -unetqos -p
  2. Enter the password for mysql
  3. Run "use em;" to set the correct DB for use.
  4. Run "select * from general;"
  5. Confirm the hostnames or IP addresses are for the correct Portal host.

Once that updates run the following SQL to identify the sourceid of the Data Aggregator data source:

select sourceid, sourcetype,consolename from netqosportal.data_sources2 where SourceType = 262144;
Example output:

+----------+------------+------------------------------------------------+

| sourceid | sourcetype | consolename                                    |

+----------+------------+------------------------------------------------+

|        3 |     262144 | Data [email protected]          |

+----------+------------+------------------------------------------------+

In that example the sourceid is 3, your installation may have a different sourceid.

Take the sourceid value for the Data Aggregator Data Source form the previous step and run the following SQL:
1. update em.data_sources set LastEvent=0,ConsumerID=0 where sourceid = <DA_SourceID>;
2. Example: Using the example above (again your id may vary) you would run:
  1. update em.data_sources set LastEvent=0,ConsumerID=0 where sourceid = 3;
3. This will trigger a re-registration process for the Event Manager and the Data Aggregator. It may take upwards of 15-20 minutes for the values to transfer around via normal synchronization cycles in Performance Management.
The errors will be resolved when we see the eventConsumerURL value at the http://DA:8581/rest/emregistration page reflecting the correct Performance Center server host name or IP address.
After this a Full Event Manager sync is recommended.
1. Go to Administration->Data Sources->Data Sources
2. Select the Event Manager Data Source
3. Select the Resync button.
4. Select the checkbox for "Perform a Full Resynchronization".
5. Select the Resync button.