How to set Jarvis kafka Logs Rotation

Products

• CA Application Performance Management Agent (APM / Wily / Introscope)
• INTROSCOPE

Issue/Introduction

An example of the process to log retention changes, this steps to advise the customer keeps the log size at 1 hour and 100 MB.

As disk space limited, customer requests to change the log retention in Kafka server.properties

Environment

APM 10.7

Resolution

An example to how process the log retention changes, this steps to advise the customer keeps the log size at 1 hour and 100 MB, and the users can change the values to the hours or bigger size according to their disk size.

1. Stop all java processes
2. Clear all data from the Data directory for Kafka (and zookeeper, if there is)
3. Clear data from /tmp directory (zookeeper keeps some reference data there)
4. Change the following setting in server.properties
a. Set log.flush.interval.ms=1000 - (that is 1 sec) The maximum amount of time a message can sit in a log before we force a flush
b. Set log.retention.check.interval.ms=60000 - (that is 1 sec) The interval at which log segments are checked to see if they can be deleted according to the retention policies
c. log.segment.bytes=100000000 ( that is 100 MB) The maximum size of a log segment file. When this size is reached a new log segment will be created.
d. log.retention.hours=1 ( that is 1 hour) The minimum age of a log file to be eligible for deletion (this is optional)
5. Start Kafka (and Zookeeper, if there is)
6. Start pumping data