vsx-license-hub-deploy Service Failure Due to Kafka FQDN IP Mismatch in License Hub 2.0

Products

• VMware vDefend Firewall
• VMware vDefend Firewall with Advanced Threat Prevention
• VMware vDefend Network Detection and Response

Issue/Introduction

The License Hub 2.0 SSH banner shows that systemctl status vsx-license-hub-deploy is failed.

---------------------------- Information --------------------------------------------------------------
Service            | Status | Monitor using
-------------------------------------------------------------------------------------------------------
node-setup         | active | systemctl status node-setup
envoy              | active | systemctl status envoy
secop              | active | systemctl status secop
depot              | active | systemctl status depot
secophelper        | active | systemctl status secop-api-helper
authelia           | active | systemctl status authelia
authserver         | active | systemctl status authserver
license-hub-deploy | failed | systemctl status vsx-license-hub-deploy
--------------------------------------------------------------------------------------------------------

Environment

License Hub 2.0.0 or later

Cause

During the first boot up of License Hub 2.0, License Hub performs multiple checks. One of the checks is to validate if the kafka FQDN is in the range of the IP pool or not. If not, License Hub does not deploy the necessary pods.

To troubleshoot the problem, enter 'sudo journalctl -u vsx-license-hub-deploy' in a ssh session to License Hub.

sysadmin@lh200-01:~$ sudo journalctl -u vsx-license-hub-deploy
[sudo] password for sysadmin: 
Jul 20 18:30:14 lh200-01 systemd[1]: Starting vsx-license-hub-deploy.service - License Hub Deploy Service...
Jul 20 18:30:14 lh200-01 deploy-license-hub.sh[5751]: Starting License Hub deployment script...
Jul 20 18:30:14 lh200-01 deploy-license-hub.sh[5751]: Waiting for local secop service to be ready on port 8080...
Jul 20 18:30:19 lh200-01 deploy-license-hub.sh[5751]: secop service is ready.
Jul 20 18:30:19 lh200-01 deploy-license-hub.sh[5906]: ERROR: Kafka FQDN 'kafka-lh200-01.lab.local' resolves to {'10.1.1.4'} but none match IP pool {'10.1.1.2', '10.1.1.3'}
Jul 20 18:30:19 lh200-01 deploy-license-hub.sh[5751]: Kafka FQDN validation failed. Aborting deployment.
Jul 20 18:30:19 lh200-01 systemd[1]: vsx-license-hub-deploy.service: Main process exited, code=exited, status=1/FAILURE
Jul 20 18:30:19 lh200-01 systemd[1]: vsx-license-hub-deploy.service: Failed with result 'exit-code'.
Jul 20 18:30:19 lh200-01 systemd[1]: Failed to start vsx-license-hub-deploy.service - License Hub Deploy Service.

The example above shows that kafka-lh200-01.lab.local, which is the kafka FQDN, is resolved as 10.1.1.4 by DNS servers. 10.1.1.4 is outside the configured IP pool 10.1.1.2-10.1.1.3.

Resolution

If a correct IP address can be resolved by changing the DNS server configuration (e.g. change an IP address for kafka FQDN in DNS server), make changes to the DNS servers and then reboot the License Hub VM.

If the problem is caused by an incorrect configuration in License Hub (e.g. incorrect domain name in kafka FQDN, incorrect IP subnet configured in IP pool), you have to delete the License Hub VM and re-deploy the License Hub VM with the correct configuration.