"FAILED: Unable to get user data. Possible Cassandra is down" - Aria Operations for Logs

Products

• VMware Aria Suite

Issue/Introduction

• Unable to login Web Interface of Aria Operations for Logs. Attempts to login fail with "Error authenticating user"
• When trying to check the status of admin password, get the following error: "FAILED: Unable to get user data. Possible Cassandra is down."
• Running the li-reset-admin-passwd.sh script results in error output: "FAILED: Unable to get user data. Possible Cassandra is down."

Environment

VMware Aria Operations for Logs 8.x

Cause

Cassandra is down on one or more nodes

Resolution

1. Take a snapshot of a cluster without a memory before making any changes.
   How to take a snapshot
   
   
2. Log in to each Aria Operations for logs node over SSH and run the following command on each node
   
nodetool-no-pass status 
   
   Each node should have a status of UN, a status of DN indicates that the Cassandra service is not running as expected on that node (UN = Up Node ; DN = Down Node) 
3. Stop loginsight Daemon service on all nodes:
   
systemctl stop loginsight
   
   
4. Start Cassandra on each nodes

/usr/lib/loginsight/application/sbin/li-cassandra.sh --startnow --force                 
   
   
5. Check the status of Cassandra on all nodes again using the command from step 1. again
6. If you see that some nodes are not UN status, stop/start cassandra on those nodes 
   
   /usr/lib/loginsight/application/sbin/li-cassandra.sh --stopnow --force
/usr/lib/loginsight/application/sbin/li-cassandra.sh --startnow --force
   
   
7. If all nodes are up (UN status) run flush/repair on all nodes:
   
   nodetool-no-pass flush
nodetool-no-pass repair
   
   
8. Once repair is over, stop Cassandra and start loginsight Daemon on each node. Make sure a node is up and running before proceeding to the next one:
   
/usr/lib/loginsight/application/sbin/li-cassandra.sh --stopnow --force
systemctl start loginsight
   
   
9. After starting all of the nodes make sure that Cassandra is up and running on each node
   
   nodetool-no-pass status

Note:

Remove all snapshots as soon as they are no longer needed.

Snapshots existing for longer than 72 hours will cause performance issues. Check Best practices for using VMware snapshots in the vSphere environment for more details.