ALERT: Some images may not load properly within the Knowledge Base Article. If you see a broken image, please right-click and select 'Open image in a new tab'. We apologize for this inconvenience.

Kafka stops during multi-node install


Article ID: 194868


Updated On:


CA App Experience Analytics



When the installation is run it is started on SERVER3 (ES, K, J) then after about a minute SERVER4 & SERVER5 (ES, K) together. 

Whichever is started first of 4 & 5 completes with Kafka server down and the other will stay on STATUS:...... until a stop and start of the one that failed does resolve this.



Release : 17.3




Here are some suggestions to investigate this sort of problem:


Check the Kafka Logs (under /opt/ca/jarvis/logs/) called server.log and zookeeper logs called zookeeper.log

That should tell what the issue is and why KAFKA starts and stops



Has been seen on a one node install regularly. Kafka needs to be restarted manually to make sure it runs. Once that is done, it usually continues to run.

Had stopped running because of missing disk space in the beginning. 


Possibly Kafka topic didn’t get created. Maybe it is a kafka/zookeeper unique id issue  


Zookeeper Error: Consumer not on the list of brokers. Or an error related to broker ids


Jarvis installer touches the file, and if incorrect IP address is detected by Jarvis, the broker ids in this file might get overwritten

Need to manually make sure the broker id's are in sync with each other in three zookeeper/kafka files:



1.   /jarvis/kafka_2.11- -file id number 

2. → has to be the same as /jarvis/kafka_2.11- -file parameter:


3. /jarvis/kafka_2.11- -file list of brokers must match the value from the above two, and the host name:



.... etc


After the file matches unique broker id's in myid -file and file for each node, restart zookeeper/kafka

The installer constructs the IDs in those files based on IP addresses - make sure the hostname and /etc/hosts -files are properly configured on each node….and that the IP addresses used are the correct ones, and that if FQDN is used as a hostname, it is in first place in /etc/hosts against the IP address"