DX NetOps Performance Management (PM) Data Aggregator (DA) Unable to Export metrics to Kafka

Products

• Network Observability
• CA Performance Management

Issue/Introduction

Unable to Export metrics to Kafka from the DA to a Kafka Cluster, and the DA log is showing messages being dropped

INFO  | Updater-thread-1 | 2025-12-16T08:36:00,057 | ProducerStatisticsMonitor | .kafka.ProducerStatisticsMonitor  154 | egator.loader.integrator |       | ProducerStatisticsMonitor[serviceActive=true, overallStats=ProducerStats[sentMessageCount=0, sentMetricCount=0, droppedMessageCount=4976437, droppedMetricCount=30328904], lastMinuteStats=ProducerStats[sentMessageCount=0, sentMetricCount=0, droppedMessageCount=3440, droppedMetricCount=21183], lastFiveMinutesStats=ProducerStats[sentMessageCount=0, sentMetricCount=0, droppedMessageCount=17295, droppedMetricCount=105449]]

 

Our Enterprise Kafka Cluster version is 4.1.1, However it is backward compatible with 3.8 clients.

Following is config for kafkaexport.producer.cfg

#This is the default configuration file for KafkaResponseProducer
#Fri Nov 04 16:31:18 AEDT 2022
#Custom Kafka Producer configuration to send data to Operational Data Repository

producer.bootstrap.servers=first.kafka.com:9096,second.kafka.com:9096,third.kafka.com:9096

#feature.enabled=off
feature.enabled=on
topic=metric-export

producer.heartbeat.topic=metric-heartbeat

# SASL/SCRAM authentication
producer.security.protocol=SASL_SSL
producer.sasl.mechanism=SCRAM-SHA-512

# Update password with correct one from EPV.

producer.sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required username="prod-netops-da" password="";

# TLS encryption (truststore only - no keystore needed)
# AP - Should not be required, but keeping just in case. AWS Keys should be in default JRE Trust store

producer.ssl.truststore.location=/opt/CA/IMDataAggregator/apache-karaf/etc/kafka.client.truststore.jks
producer.ssl.truststore.password=<PASSWORD>

# Optional: Endpoint identification (may need to disable for MSK)
# AP - Leave this out, shouldn't be required

producer.ssl.endpoint.identification.algorithm=

# Line to trigger config reload

NOTE: Environment is in Fault Tolerance, above config is applied onto both Data Aggregators.

Environment

DX NetOps CAPM all currently supported releases

Cause

The issue is that the heartbeat starts with a bootstrap server of changeme:9092 (the default) some time ago as shown by the following message in the KafkaLog.log:

WARN  | rtbeat--thread-1 | 2025-08-09T20:12:56,117 | ClientUtils | apache.kafka.clients.ClientUtils   89 | ix.bundles.kafka-clients |       | Couldn't resolve server changeme:9092 from bootstrap.servers as DNS resolution failed for changeme

If the DA was never restarted after the bootstrap server was corrected, we might have this problem.

The heartbeat will start correctly if the configuration is fixed AND the feature.enabledis set to off, the file saved, then feature.enabled is returned to on and the file is saved again. 

Resolution

The configuration is as defined in:

KB : Required parameters DA and Kafka connections when using SSL/TLS

 

So, to fix this issue, in the file:

<DataAggregator_Home>/apache-karaf/etc/kafkaexport.producer.cfg 

Set

feature.enabled=off

Save the file, then edit it again and set:

feature.enabled=on

and save the file again. That should restart the heartbeat.