IG-Kafka Producer job fails and Agent stops

Products

• CA Automic Workload Automation - Automation Engine

Issue/Introduction

After running a Kafka Producer job, the Agent stops.

The following error is displayed in the Agent log:

20260323/100954.143 - U02000005 Job 'KAFKA_PRODUCER_JOB' with RunID '2085974' is to be started.
20260323/100954.143 - U02000003 Job 'KAFKA_PRODUCER_JOB' started with RunID '2085974'.
20260323/100954.192 - U00029000 Report '2092345' for file 'C:\Automic\Kafka\temp\PLOG_OAAEORTU.TXT' has been started.
20260323/100954.210 - U00029001 Report '2092345' ended normally.
20260323/100954.210 - U00045014 Exception 'java.lang.NoClassDefFoundError: "org/apache/kafka/clients/producer/Producer"' at 'com.automic.agent.kafka.jobs.ProducerJob.runJob():46'.
20260323/100954.210 - U00045015 The previous error was caused by 'java.lang.ClassNotFoundException: "org.apache.kafka.clients.producer.Producer"' at 'jdk.internal.loader.BuiltinClassLoader.loadClass():641'.
20260323/100954.210 - U00003620 Routine 'com.uc4.ex.ThreadPool$RunnableDecorator' forces trace because of error.
20260323/100954.225 - U00003450 The TRACE file was opened with the switches '0000000000000000'.
20260323/100954.319 - U00003449 Output to the TRACE file is finished.
20260323/100954.319 - U02000041 Shutdown Agent 'KAFKA'.
20260323/100954.335 - U02000010 Connection to Server '<URL Automation Engine>/<unresolved>:8443' terminated.
20260323/100954.339 - U02000001 Agent 'KAFKA' version '24.0.0+build.1760334181407' ended normally.

The trace shows:

20260323/100954.272 - 19               ----------------------- Stack Trace -----------------------
20260323/100954.272 - 19               java.lang.NoClassDefFoundError: org/apache/kafka/clients/producer/Producer
20260323/100954.272 - 19                   at com.automic.agent.kafka.jobs.ProducerJob.runJob(ProducerJob.java:46)
20260323/100954.272 - 19                   at com.automic.ra.agent.RaJobProcessorImpl.execute(RaJobProcessorImpl.java:93)
20260323/100954.272 - 19                   at com.uc4.ex.Job.run(Job.java:474)
20260323/100954.272 - 19                   at com.uc4.ex.ThreadPool$RunnableDecorator.run(ThreadPool.java:105)
20260323/100954.272 - 19                   at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
20260323/100954.272 - 19                   at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
20260323/100954.288 - 19                   at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
20260323/100954.288 - 19                   at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
20260323/100954.288 - 19                   at java.base/java.lang.Thread.run(Thread.java:840)
20260323/100954.288 - 19               Caused by: java.lang.ClassNotFoundException: org.apache.kafka.clients.producer.Producer
20260323/100954.288 - 19                   at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641)
20260323/100954.288 - 19                   at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
20260323/100954.288 - 19                   at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:525)
20260323/100954.288 - 19                   ... 9 more

NOTE: if the Agent doesn't start in the first place, the following jar-files need to be copied from the Kafka instance to the <agent>\bin\lib folder:

kafka-clients-x.y.z.jar
lz4-java-x.y.z.jar
zstd-jni-x.y-z.jar

(where x, y, z represent version numbers)

Cause

The Kafka Agent expects libraries without version numbers

DE184511

Resolution

Solution:

This will be fixed in a future version of the IG Kafka

Workaround:

Remove the version numbers from the imported jar-files. For instance:

Rename kafka-clients-3.9.2.jar, lz4-java-1.8.0.jar and zstd-jni-1.5.6-4.jar to kafka-clients.jar, lz4-java.jar and zstd-jni.jar