Apache Kafka connection errors while trying to create assets.

Products

• Service Virtualization

Issue/Introduction

When trying to create Kafka connection, we are seeing the connection timing out, as shown below:

 

connecting to Bootstrap server- 

Verifying Asset
Opening consumer connection
Listing Topics
Error: Error listing topics: Timeout expired while fetching topic metadata

============================================================================
| Exception: 
============================================================================
| Message:     Error listing topics: Timeout expired while fetching topic metadata
----------------------------------------------------------------------------
| Trapped Exception: Timeout expired while fetching topic metadata
| Trapped Message:   org.apache.kafka.common.errors.TimeoutException: Timeout expired while fetching topic metadata
----------------------------------------------------------------------------
STACK TRACE
org.apache.kafka.common.errors.TimeoutException: Timeout expired while fetching topic metadata
============================================================================

 at com.itko.lisa.messaging.providers.kafka.asset.KafkaUtils.listTopics(KafkaUtils.java:485)
 at com.itko.lisa.messaging.providers.kafka.asset.KafkaUtils.verifyConnection(KafkaUtils.java:505)
 at com.itko.lisa.messaging.providers.kafka.asset.KafkaConfigAsset.verifyInstance(KafkaConfigAsset.java:171)
 at com.itko.lisa.messaging.providers.kafka.asset.KafkaConfigAsset.verifyInstance(KafkaConfigAsset.java:45)
 at com.itko.lisa.asset.AssetUtils.verifyRuntimeAsset(AssetUtils.java:1300)
 at com.itko.lisa.asset.ui.AssetEditor$5.run(AssetEditor.java:306)
 at com.itko.lisa.asset.ui.ProgressLog$Worker.doInBackground(ProgressLog.java:142)
 at com.itko.lisa.asset.ui.ProgressLog$Worker.doInBackground(ProgressLog.java:100)
 at javax.swing.SwingWorker$1.call(SwingWorker.java:295)
 at java.util.concurrent.FutureTask.run(FutureTask.java:266)
 at javax.swing.SwingWorker.run(SwingWorker.java:334)
 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
 at java.lang.Thread.run(Thread.java:745)

Environment

Release : 10.7 and up

Component : CA Service Virtualization

Resolution

Suggested to upgrade since Kafka better supported in 4.6

Below is all that was needed to Support SSL Kafka

Prerequisites for creating Kafka connection:

• Create connections for Keystore, Truststore, and SSL.
• Copy the following third-party jar files to the <LISA_HOME>/lib/shared directory:
  • kafka-clients-2.5.0.jar
  • lz4-java-1.4.jar
  • snappy-java-1.1.7.1.jar

 

Information that we got from the Kafka Team concerning connection:

All the rest Default.

If you need to add a keystore or a Trust , this is the time to add it.

If not, skip these next steps please.

Click the add Key Store button (arrow) and Select the Key Store File

To add a Trust Store do the following - Press the add Trust Store button:

Browse to your cacerts file and add its password.  Please note, for the trustore you need to import  the issuing CA, intermediate and root certs into a trustore file.  Use this against the Kafka Connection. 

 

 SSL Context with SSL configuration: 

Note : the Protocol is set to the one needed by the Server - TLSv1.2

 

Now Click the Checker to make sure it connects

And you should be able to see the topics that were found