Cluster page of worker nodes throws error "The latest cluster data is not available, most likely primary node is not accessible"
search cancel

Cluster page of worker nodes throws error "The latest cluster data is not available, most likely primary node is not accessible"

book

Article ID: 387853

calendar_today

Updated On:

Products

VMware Aria Suite

Issue/Introduction

The cluster page of worker nodes fails to load and shows the message "The latest cluster data is not available, most likely primary node is not accessible"  while the nodes in the cluster page loads fine in the primary node

The errors in the runtime.log will match the below snippet

Log location:/var/log/vmware/loginsight/runtime.log

[1234-12-12 11:11:11.] ["main"/##.##.##.## INFO] [com.vmware.loginsight.daemon.DaemonClientStubManager] [Initializing daemon commands client pools.]
[1234-12-12 11:11:11.] ["main"/##.##.##.## INFO] [com.vmware.loginsight.daemon.LogInsightDaemon] [Exception during setup environment] [Exception during setup environment]
[1234-12-12 11:11:11.] ["main"/##.##.##.## FATAL] [com.vmware.loginsight.daemon.LogInsightDaemon] [Error starting services]
com.vmware.loginsight.config.MasterFQDNUnresolvableException: <hostname> cannot be resolved and there is not master IP buffer
[1234-12-12 11:11:11.] ["BucketManager-thread-1"/##.##.##.## ERROR] [com.vmware.loginsight.commons.rpc.clientconnpool.ClientPoolsHolder] [Need IP to create Thrift client. Got '<hostname>']

Environment

Aria operations for logs 8.x

Cause

The inter node communication client pools are constructed incorrectly when aria operations for logs is not able to correctly resolve provided master FQDN to an IP address

Resolution

     Note: Take cold snapshots of all the Aria operations for logs nodes in the cluster

  1. Backup the file by running cp /storage/core/loginsight/config/loginsight-config.xml#<highest_number>  /storage/core/loginsight/config/loginsight-config.xml#<highest_number>.bak
  2. Edit the below file and make changes 

    vi /storage/core/loginsight/config/loginsight-config.xml#<highest_number>

  3. Under section called distributed, replace FQDN of primary node in host attribute with the primary node IP address

  4. Restart loginsight service

    service loginsight restart
  5. Repeat steps 1 to 4 in all the nodes. 

 

Additional Information

This change is internal to Aria operations for logs hence it wont affect DNS entries. Aria operations for logs will still be accessible via FQDN. 

Powered by Wolken Software