Kafka configuration queries for sizing
search cancel

Kafka configuration queries for sizing

book

Article ID: 266895

calendar_today

Updated On: 06-13-2025

Products

Network Observability CA Performance Management

Issue/Introduction

Need clarification on the following.

Is it possible to have the messages compressed in Kafka? Due to the large volume of data involved, it's crucial to focus on efficiency and optimization.

Could you provide a Kafka message sample for each metric family (port, device, and IP SLA)? With this information, we can begin working on the deployment structure, save time and improve the accuracy of our estimates.

Will messages from each metric family be in separate Kafka topics?

Environment

All supported Network Observability DX NetOps Performance Management releases working with kafka

Resolution

The metric records are not compressed by default. It is possible to export them in a compressed form by selecting the desired compression type from the values available.

See the 3.1 Broker Configs section and the compression.type details for options available when adding a producer.compression.type=<value> entry to the kafkaexport.producer.cfg file.

Sample messages for all Metric Families are not available. It is possible to have multiple Metric Families for a given device. The messages for various Metric Families will have some basic properties. Due to the possible variety of metrics and data being collected no full sample set is available.

Only polled metrics with non-null values will be exported. Metrics can be filtered from export per metric family via the ExportProfile item.

The Metric export sends all metrics to a single topic.

The messages are keyed by Data Aggregator Item_ID to keep the data in order per item represented by the data received.