Symptoms:
This is seen in simple deployments with 1 Platform node only.
After upgrading to VMware Aria Operations for Networks version 6.11, you may see Launcher Service Not Running error on GUI.
1. Below is alert/error is seen on VMware Aria Operations for Networks GUI.
2. If Alert notifications are configured then below alert notification is generated as well.
3. On Platform node navigate to launcher latest.log file. Log location: /home/ubuntu/logs/launcher
Search with keyword vrni-platform-release (This is old name) to see exceptions similar to such as below.
You see entries similar to below:
2023-09-14T13:43:48.627Z ERROR zookeeper.client.StaticHostProvider ReadOnlyZKClient-vrni-platform-release:2181@0x5c0d876c-SendThread(vrni-platform-release:2181) resolve:154 Unable to resolve address: vrni-platform-release:2181
java.net.UnknownHostException: vrni-platform-release
at java.net.InetAddress_CachedAddresses.get(InetAddress.java:797) _[_:_]
at java.net.InetAddress.getAllByName0(InetAddress.java:1519) _[_:_]
at java.net.InetAddress.getAllByName(InetAddress.java:1378) _[_:_]
at java.net.InetAddress.getAllByName(InetAddress.java:1306) _[_:_]
at org.apache.hadoop.hbase.shaded.org.apache.zookeeper.client.StaticHostProvider_1.getAllByName(StaticHostProvider.java:92) _[launcher-0.001-SNAPSHOT.jar:_]
at org.apache.hadoop.hbase.shaded.org.apache.zookeeper.client.StaticHostProvider.resolve(StaticHostProvider.java:147) [launcher-0.001-SNAPSHOT.jar:_]
at org.apache.hadoop.hbase.shaded.org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:375) [launcher-0.001-SNAPSHOT.jar:_]
at org.apache.hadoop.hbase.shaded.org.apache.zookeeper.ClientCnxn_SendThread.run(ClientCnxn.java:1137) [launcher-0.001-SNAPSHOT.jar:_]
2023-09-14T13:43:48.628Z WARN apache.zookeeper.ClientCnxn_SendThread ReadOnlyZKClient-vrni-platform-release:2181@0x5c0d876c-SendThread(vrni-platform-release:2181) run:1246 Session 0x0 for server vrni-platform-release:2181, unexpected error, closing socket connection and attempting reconnect
java.lang.IllegalArgumentException: Unable to canonicalize address vrni-platform-release:2181 because it's not resolvable
at org.apache.hadoop.hbase.shaded.org.apache.zookeeper.SaslServerPrincipal.getServerPrincipal(SaslServerPrincipal.java:71) _[launcher-0.001-SNAPSHOT.jar:_]
at org.apache.hadoop.hbase.shaded.org.apache.zookeeper.SaslServerPrincipal.getServerPrincipal(SaslServerPrincipal.java:39) _[launcher-0.001-SNAPSHOT.jar:_]
at org.apache.hadoop.hbase.shaded.org.apache.zookeeper.ClientCnxn_SendThread.startConnect(ClientCnxn.java:1087) _[launcher-0.001-SNAPSHOT.jar:_]
at org.apache.hadoop.hbase.shaded.org.apache.zookeeper.ClientCnxn_SendThread.run(ClientCnxn.java:1139) [launcher-0.001-SNAPSHOT.jar:_]
2023-09-14T13:43:48.728Z WARN hbase.zookeeper.ReadOnlyZKClient_ZKTask_1 ReadOnlyZKClient-vrni-platform-release:2181@0x5c0d876c exec:192 0x5c0d876c to vrni-platform-release:2181 failed for list of /hbase, code = CONNECTIONLOSS, retries = 11
2023-09-14T13:43:49.728Z ERROR zookeeper.client.StaticHostProvider ReadOnlyZKClient-vrni-platform-release:2181@0x5c0d876c-SendThread(vrni-platform-release:2181) resolve:154 Unable to resolve address: vrni-platform-release:2181
java.net.UnknownHostException: vrni-platform-release
4. Navigate to live log location /home/ubuntu/build-target/launcher
Here you will see 3 files such as below:
hbase_metric_store.configuration
tsdb_client.configuration
elasticsearch.configuration
Content of above file shows older name "vrni-platform-release"
ubuntu@aria-networks-platform:~/build-target/launcher$ cat tsdb_client.configuration
tsdbHosts: "vrni-platform-release"
tsdbPort: 55111
tsdbShards: 60
ubuntu@aria-networks-platform:~
ubuntu@aria-networks-platform:~/build-target/launcher$ cat elasticsearch.configuration
host_port {
host: "localhost"
port: 9300
}
cluster_name: "elasticsearch"
dedup_max_hits_fetch_mk: 6000000
dedup_max_hits_no_mk: 10000000
indexer_threads: 4
zookeeper_connection_string: "vrni-platform-release:2181"
ubuntu@aria-networks-platform:~
Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.
To workaround this issue, follow steps mentioned below:
1. SSH to Aria Operation for Network (vRNI) platform appliance as support.
2. Execute below 3 commands:
ub
cd /home/ubuntu/build-target/server-config-mgmt
/bin/bash /home/ubuntu/build-target/server-config-mgmt/run-server-config-mgmt.sh /home/ubuntu/build-target/launcher/server-config.configuration.single /home/ubuntu/build-target/launcher
sudo service launcher-service restart
3. Validate launcher-service has restarted by executing below command/script.
./check-service-health.sh -p -d
4. On Aria Operation for Network (vRNI) GUI, all errors should now be cleared for Launcher Service Not Running and no activity errors.
Also alerts email notification should now not trigger for Launcher Service Not Running and no activity.