VMware vRealize Log Insight 2.5 startup fails with the error: Unable to gossip with any seeds
search cancel

VMware vRealize Log Insight 2.5 startup fails with the error: Unable to gossip with any seeds

book

Article ID: 328682

calendar_today

Updated On:

Products

VMware

Issue/Introduction

This article describes a specific problem affecting VMware vRealize Log Insight 2.5. If you experience the error Failed to start Cassandra Server without Unable to gossip with any seeds, see Starting VMware vRealize Log Insight 2.5 fails with the error: Failed to start Cassandra Server (2107243).


Symptoms:
  • Starting VMware vRealize Log Insight 2.5 fails.

  • In the /storage/var/loginsight/loginsight_daemon_stdout.log file or /storage/var/loginsight/runtime.log file, you see an entry similar to:

    Daemon startup failed: Failed to start Cassandra Server: StartupException(description:Unable to connect to Cassandra node at localhost:nnnn: com.vmware.loginsight.cassandra.CassandraException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: localhost/127.0.0.1 (com.datastax.driver.core.TransportException: [localhost/127.0.0.1] Cannot connect))).

  • In the /storage/var/loginsight/cassandra.log file, you see an entry similar to:

    ERROR [main] yyyy-mm-dd hh:mm:ss,nnn CassandraDaemon.java (line ccc) Exception encountered during startup
    java.lang.RuntimeException: Unable to gossip with any seeds

Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.


Cause

This issue occurs when the Apache Cassandra service within the Log Insight 2.5 virtual appliance cannot establish communication between cluster nodes. This may be noticed immediately after upgrading a cluster from Log Insight 2.0 to 2.5.

This issue is triggered by an incorrect DNS resolution for the Log Insight Master node's own IP address.

Resolution

This is a known issue affecting VMware vRealize Log Insight 2.5.

This issue is resolved in VMware vRealize Log Insight 3.0, available from VMware Downloads.

To work around this issue, modify the /etc/hosts configuration file on the Master node and set the Fully Qualified Domain Name (FQDN) of the host and the hostname associated with the IP address of the node.

To correct the /etc/hosts configuration file on the Master node in the Log Insight cluster:

  1. Open a console or SSH session to the Log Insight virtual appliance and log in as root.

  2. Validate that the Log insight service is not running by running this command:

    service loginsight status

    You should see daemonNotRunning. If the service is running, these steps are not necessary.

  3. Determine the Log Insight node's IP address by running this command:

    /opt/vmware/share/vami/vami_ip_addr eth0

  4. Open the /etc/hosts configuration file in a text editor. For more information, see the Editing using vi section of Editing files on an ESX host using vi or nano (1020302).

  5. Inspect the contents of the /etc/hosts configuration file. A problematic configuration has the Log Insight node's fully-qualified domain name (FQDN) or hostname listed on the same line as the loopback address 127.0.0.1. For example:

    127.0.0.1 loginsight-master.example.com loginsight-master localhost

  6. Modify the /etc/hosts configuration file to specify the node's fully-qualified domain name and hostname associated with the IP address of the node noted in Step 3. For example:

    10.11.12.13 loginsight-master.example.com loginsight-master
    127.0.0.1 localhost


  7. If the /etc/hosts configuration file contains a line for the ::1 IPv6 loopback address, remove it. For example, remove this line entirely:

    ::1 loginsight-master.example.com loginsight-master localhost ip6-localhost ip6-loopback

  8. Save and close the /etc/hosts configuration file.

  9. Start the Log Insight service using the command:

    service loginsight start

Note: If Worker nodes in the Log Insight cluster restarted and failed to connect to the Master node during this procedure, restart the Log Insight service on each Worker node.

  1. Open a console or SSH session to each Log Insight cluster Worker node and log in as root.

  2. Restart the Log Insight service by running this command:

    service loginsight restart


If the issue persists even after performing this procedure, open a Support Request with VMware Support and quote this Knowledge Base article ID (2117471) in the problem description. For more information, see How to file a Support Request in Customer Connect (2006985).


Additional Information

To be alerted when this document is updated, click the Subscribe to Article link in the Actions box.
Editing files on an ESX host using vi or nano
How to file a Support Request in Customer Connect
Starting VMware vRealize Log Insight 2.5 fails with the error: Failed to start Cassandra Server