DX OI 1.3.2 integration with UIM - Troubleshooting guidelines

Products

• DX Operational Intelligence

Issue/Introduction

The following is a list of techniques and suggestions to employ when troubleshooting OI to UIM integration issues

A) Checklist
B) What diagnostic files should I gather for Broadcom Support?

Environment

DX Operational Intelligence 1.3.x

Resolution

A) CHECKLIST

OI Connector

1) Ensure the configuration is correct

Go to the UIM Admin console > OI_connector configuration page

Use the Verify options to confirm that provided information is correct, for example:

 

2. Review the oi_connector log

Default location : C:\Program Files (x86)\Nimsoft\probes\gateway\oi_connector\oi_connector.log

a) Ensure there are not errors or exceptions, here is an example of a message indicating a connectivity issue:

Jan 27 17:19:46:877 [QOS_PROCESSOR_THREAD, oi_connector] Exception while executing doPost request to Jarvis url: http://jarvis.gvecsl6636ipm/ingestion :: org.apache.http.NoHttpResponseException: jarvis.gvecsl6636ipm:80 failed to respond

b) Search for key string to help you confirm that date is being pushed to Jarvis or DOI:

“Total no of” QoS posted on jarvis via post”
“Total no of UIM Alarms posted on jarvis via post”
“Total no of UIM Groups posted on jarvis via post”

Here are some examples:

Dec 09 09:23:12:176 [UIM_OPEN_ALARM_PROCESSOR_THREAD, oi_connector] Total no of UIM Alarms posted on jarvis via post : 2
Dec 09 09:23:12:177 [DEVICE_INVENTORY_PROCESSOR_THREAD, oi_connector] Total no of UIM Device Inventory posted on jarvis via post : 2
Jan 28 08:21:31:017 [QOS_PROCESSOR_THREAD, oi_connector] Total no of QoS posted on jarvis via post : 6

c) If required enable TRACE logging from rom UIM console >  OIConnector probe configuration page

Restart the connector:

 

ElasticSearch

Verify that data (metrics, alerts, inventory) is being ingested 

a) List all UIM product indices

http://es.<servername>/_cat/indices/*uim*?v

Check if the doc.count and size columns values increases.

b) If you need more detail, you can list the content of each index

http://es.<severname>/<index-name>/_search?pretty&sort=timestamp:desc&size=500

For example: http://es.<severname>/ao_itoa_alarms_all_1_1/_search?pretty&sort=timestamp:desc&size=500

 

Jarvis/ Kafka

If you don't see any error in the oi_connector log and data is not getting ingested into Elastic, then it could be an issue in Jarvis or Kafka, see:

DX OI - Jarvis / Kafka Troubleshooting, Common Issues and Best Practices
https://knowledge.broadcom.com/external/article/189119

 

B) WHAT DATA SHOULD I GATHER FOR BROADCOM SUPPORT?

a) Provide screenshots illustrating the discrepancy between UIM and DX OI, If the problem is related to difference is metrics and values

b) From OI_Connector:

-Enable TRACE logging and gather 
C:\Program Files (x86)\Nimsoft\probes\gateway\oi_connector/logs/*
C:\Program Files (x86)\Nimsoft\probes\gateway\oi_connector/ oi_connector.cfg 

c) From Elastic

-Result of below queries:
http(s)://es.<servername>/_cat/indices/*uim*?v
http(s)://es.<severname>/ao_itoa_alarms_all_1_1/_search?pretty&sort=timestamp:desc&size=500

d) Kafka logs, see

DX OI - Jarvis / Kafka Troubleshooting, Common Issues and Best Practices
https://knowledge.broadcom.com/external/article/189119

 

Additional Information

 

DX AIOPs (OI and APM) - Troubleshooting, Common Issues and Best Practices
https://knowledge.broadcom.com/external/article/190815/dx-oi-troubleshooting-common-issues-and.html