Kafka components Down, Unable to process data

book

Article ID: 129001

calendar_today

Updated On:

Products

CA App Experience Analytics

Issue/Introduction

Kafka components are down and unable to process data, effectively bringing cluster to a down status.

Environment

Release:
Component: AXAENG

Resolution

The issue seems like related to heap settings on the axa components (specially Transformer and Aggregator).
Low heap setting likely causing issues with data processing.
We made the adjustments in heap size and restarted axa components.
It looks like AXA components are starting to processing data. 
Initially, there is still some Kafka lag , but after axa components continue run for few hours, Kafka lag has gone down and cluster performance is back to normal.