In Smarts 10.x Kafka and Zookeeper are considered 3rd party products by VMware and not bundled with the install of Smarts Server Manager. However they are required if you enable VeloCloud SD-WAN discovery and monitoring.
It has been noted that installing Kafka and Zookeeper do not install as services by default, so one would have to restart zookeeper and kafka manually after a server restart.
1. Kafka is generally download and extracted from tgz file to a directory on Linux machine.
Example: /opt/kafka_2.11-2.0.0
2. Add Unix group/user for Kafka and set directory ownership to this use with following commands: groupadd kafka
useradd -g kafka kafka
passwd kafka
3. Change directory ownership. mkdir will fail if directory already exists.
mkdir /var/zookeeper
chown -R kafka:kafka /var/zookeeper
mkdir /var/log/zookeeper
chown -R kafka:kafka /var/log/zookeeper
chown -R kafka:kafka /opt/kafka_2.11-2.0.0
4. Create file kafka.service under /etc/systemd/system, add the following lines & save the file:
[Unit]
Requires=zookeeper.service
After=zookeeper.service
[Service]
Environment="KAFKA_OPTS=-Djava.security.auth.login.config=/opt/kafka_2.11-2.0.0/config/kafka_server_jaas.conf"
Type=simple
User=kafka
ExecStart=/bin/sh -c '/opt/kafka_2.11-2.0.0/bin/kafka-server-start.sh /opt/kafka_2.11-2.0.0/config/server.properties > /opt/kafka_2.11-2.0.0/kafka.log 2>&1'
ExecStop=/opt/kafka_2.11-2.0.0/bin/kafka-server-stop.sh
Restart=on-abnormal
[Install]
WantedBy=multi-user.target
5. Create file zookeeper.service under /etc/systemd/system, add the following lines & save the file:
[Unit]
Requires=network.target remote-fs.target
After=network.target remote-fs.target
[Service]
Environment="KAFKA_OPTS=-Djava.security.auth.login.config=/opt/kafka_2.11-2.0.0/config/zookeeper_jaas.conf"
Type=simple
User=kafka
ExecStart=/opt/kafka_2.11-2.0.0/bin/zookeeper-server-start.sh /opt/kafka_2.11-2.0.0/config/zookeeper.properties
ExecStop=/opt/kafka_2.11-2.0.0/bin/zookeeper-server-stop.sh
Restart=on-abnormal
[Install]
WantedBy=multi-user.target
6. Start and enable the services with following commands so they are automatically restarted when the system is restarted .
systemctl start zookeeper.service
systemctl enable zookeeper.service
systemctl start kafka.service
systemctl enable kafka.service
7. Check the status of Kafka & Zookeeper services with below commands:
systemctl status kafka
kafka.service
Loaded: loaded (/etc/systemd/system/kafka.service; enabled; vendor preset: disabled)
Active: active (running) since Thu 2019-12-12 16:19:26 GMT; 1 weeks 0 days ago
Process: 57455 ExecStop=/opt/kafka/bin/kafka-server-stop.sh (code=exited, status=1/FAILURE)
Main PID: 52923 (sh)
CGroup: /system.slice/kafka.service
+-52923 /bin/sh -c /opt/kafka/bin/kafka-server-start.sh /opt/kafka/config/server.properties ...
+-52926 java -Xmx1G -Xms1G -server -XX:+UseG1GC -XX:MaxGCPauseMillis=20 -XX:InitiatingHeapOccu...
systemctl status zookeeper
zookeeper.service
Loaded: loaded (/etc/systemd/system/zookeeper.service; enabled; vendor preset: disabled)
Active: active (running) since Thu 2019-12-12 16:19:26 GMT; 1 weeks 0 days ago
Process: 45496 ExecStop=/opt/kafka/bin/zookeeper-server-stop.sh (code=exited, status=1/FAILURE)
Main PID: 52922 (java)
CGroup: /system.slice/zookeeper.service
+-52922 java -Xmx512M -Xms512M -server -XX:+UseG1GC -XX:MaxGCPauseMillis=20 -XX:InitiatingHeap...
Below is the command to restart Kafka service with systemctl, it does not print a status message.
sudo systemctl restart kafka
In case of problems, errors from systemctl are logged in /var/log/messages for reference.