Route to Kafka max message size RecordTooLargeException

Products

• CA API Gateway

Issue/Introduction

Trying to send huge message size to Kafka but I get:

Problem routing to Kafka Cluster. org.apache.kafka.common.errors.RecordTooLargeException: The message is 3830961 bytes when serialized which is larger than the maximum request size you have configured with the max.request.size configuration.

What is setting I can modify ?

Environment

Release : 10.x

Resolution

The Kafka client does not take the gateway default message settings, the advanced tab in the Route  to Kafka  assertion can be used to pass Kafka configuration settings  to the client . 

The Kafka default is 1 Mb  to raise this you can set  Kafka client config  "max.request.size"  = 20485880

some consideration to review from the Kafka site

https://www.conduktor.io/kafka/how-to-send-large-messages-in-apache-kafka

 "