Aria Operations for Logs "error authenticating user" Due to Corrupt Cassandra Hints

Products

• VCF Operations/Automation (formerly VMware Aria Suite)

Issue/Introduction

• Aria Operations for Logs UI login fails for both local admin and domain accounts with the exact error message: "error authenticating user".
• Output from the 'nodetool-no=pass status' command indicates an inconsistent Cassandra cluster state across nodes, displaying a mix of Down/Normal (DN) and Up/Normal (UN) statuses depending on the queried node.
• Cassandra logs show errors similar to the following;

  • ERROR [HintsDispatcher:1] ####-##-## 07:00:24,378 HintsDispatchExecutor.java:243 - Failed to dispatch hints file ########-####-####-####-############-#############-#.hints: file is corrupted ({})"

or

• • ERROR <DATE> [HintsDispatcher:1]  JVMStabilityInspector.java:68 - Exception in thread Thread[HintsDispatcher:1,1,HintsDispatcher]org.apache.cassandra.io.FSReadError: java.io.IOException: Corrupt hint file found

Environment

• VMware Aria Operations for Logs 8.18.x

Cause

The authentication failure is caused by a Cassandra database cluster desynchronization triggered by corrupt hints files, which disrupts inter-node communication and database quorum.

Resolution

1. Verify configuration drift by executing 'ls /storage/core/loginsight/config' across all nodes in the cluster to identify any missing configuration versions.

2. Transfer the latest configuration file to the out-of-sync node(s) using the following command:

   • scp /storage/core/loginsight/config/loginsight-config.xml#<VERSION> root@<Out_of_sync_Node_IP>:/storage/core/loginsight/config/

3. Clear the corrupt hints files from all nodes in the cluster to restore database consistency by following the KB below;

   • Error "Failed to dispatch hints file" "file is corrupted" in cassandra,log

4. Validate successful UI login using the local admin account.