DX OI - Metrics are not visible in Performance analytics and Alarms are not reported

Products

• DX Operational Intelligence

Issue/Introduction

Symptoms:

- We can see topology information but no metrics from Performance Analytics

- No metrics are getting ingested into http(s):/es.<endpoint>/ao_itoa_metrics_metadata_2_1/_search?pretty

- Alarms ingestion is also affected, we don't see new alarms.

- OIConnectors are working a expected and no issues is reported in their logs

 

 

Environment

DX Operational Intelligence 20.2.x

Cause

NFS or network issue affecting Jarvis:

- zookeeper was unable to connect to kafka brokers
- the below exception was reported randomly when checking for consumer activity:

"Error: Executing consumer group command failed due to org.apache.kafka.common.errors.CoordinatorNotAvailableException: The coordinator is not available"

For more information on checklist refers to:

DX Platform - Jarvis (kafka, zookeeper, elasticSearch) Troubleshooting
https://knowledge.broadcom.com/external/article/189119

 

 

Resolution

Restart Jarvis core services as documented in below KB:

DX Platform - How to restart Jarvis services?
https://knowledge.broadcom.com/external/article/223094

Additional Information

DX AIOPs Troubleshooting, Common Issues and Best Practices