DX APM - Kafka /JMX Setup

Products

• DX SaaS

Issue/Introduction

How to set JMX in Kafka.

Unable to gather Kafka metrics, and keep getting the below error when the Kafka Extension is configured with APMIA 20.11

2/01/21 09:52:30 AM EST [DEBUG] [IntroscopeAgent.KafkaExtension] Broker Data: {"jmx_port":-1,"timestamp":"1611461585073","endpoints":["PLAINTEXT://TESTweb0001e.linux.us.BRCM.com:9092"],"host":"TESTweb0001e.linux.us.BRCM.com","version":3,"port":9092}
2/01/21 09:52:30 AM EST [INFO] [IntroscopeAgent.KafkaExtension] List of available brokers []
2/01/21 09:52:30 AM EST [DEBUG] [IntroscopeAgent.KafkaExtension] No Kafka brokers online to monitor
2/01/21 09:52:30 AM EST [INFO] [IntroscopeAgent.KafkaExtension] No Kafka brokers configured to monitor

Environment

Release : DX SAAS, 

Component : APMIA , AXA

Cause

Since there is no JMX server or Kafka broker list configured, we can only get the brokers from the configured Zookeeper server list:

2/01/21 09:52:30 AM EST [DEBUG] [IntroscopeAgent.KafkaExtension] zookeeper server list: TESTpapp00374.linux.us.BRCM.com:2181,TESTweb0001e.linux.us.BRCM.com:2181,TEST002fd.linux.us.BRCM.com:2181
 

But none of brokers from Zookeeper has JMX enabled ("jmx_port":-1), 

2/01/21 09:52:30 AM EST [DEBUG] [IntroscopeAgent.KafkaExtension] Broker ID: 382075
2/01/21 09:52:30 AM EST [DEBUG] [IntroscopeAgent.KafkaExtension] Broker Data: {"jmx_port":-1,"timestamp":"1612026725818","endpoints":["PLAINTEXT://TESTweb0001e.linux.us.BRCM.com:9092"],"host":"TESTweb0001e.linux.us.BRCM.com","version":3,"port":9092}

 

So we end up with no broker to monitor:

2/01/21 09:52:30 AM EST [INFO] [IntroscopeAgent.KafkaExtension] List of available brokers []
2/01/21 09:52:30 AM EST [DEBUG] [IntroscopeAgent.KafkaExtension] No Kafka brokers online to monitor
 

Please enable jmx for the brokers to monitor, or configure jmx server list to monitor.

Resolution

edit ./bin/jarvis/startKafka.sh, 

add a line before the last line:

export JMX_PORT=<any free port#>

 

Need to do this for every broker.