Improve Query performance by removing corrupted buckets in Aria Operations for Logs

Products

• VMware Aria Suite

Issue/Introduction

Symptoms:
Query performance of Aria Operations for Logs (formerly vRealize Log Insight) may degrade while under average or low load.

Environment

Aria Operations for Logs 8.x

Cause

Corrupted data buckets in the inventory of Aria Operations for Logs can cause slow querying.

Resolution

To Improve performance, remove the corrupted buckets from the Aria Operations for Logs inventory.

To identifying and remove corrupted buckets from the inventory, use one of these options:

Option 1

1. Analyze /storage/var/loginsight/runtime.log and compile a list of bucket UUIDs to remove.
2. Stop the Log Insight service by running the following command:

service loginsight stop

3. Run the following command to remove a bucket:

/usr/lib/loginsight/application/sbin/bucket-index UUID delete

Note: Replace UUID with the UUID of a bucket found in step 1.

4. Repeat step 3 for all UUIDs in your list.
5. Start the Log Insight service by running the following command:

service loginsight start

Option 2

1. Change to the /usr/lib/loginsight/application/sbin/ directory

cd /usr/lib/loginsight/application/sbin/

2. Run the bucket validator to determine the UUIDs of the corrupted buckets

./validate-bucket -Vv

3. Stop the Log Insight service by running the following command: 

service loginsight stop​​

4. Run the following command to remove a bucket:

./bucket-index UUID delete

Note: Replace UUID with the UUID of a bucket found in step 1.

5. Repeat step 4 for all UUIDs in your list.
6. Start the Log Insight service by running the following command:

service loginsight start

 

Additional Information

When running the command ./validate-bucket -Vv, this error could show:

ERROR com.vmware.loginsight.analytics.IndexDirectoryValidator - Exception while validating taxonomy, not marking as invalid: org.apache.lucene.store.NoSuchDirectoryException: directory /storage/core/loginsight/cidata/store/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/index' does not exist

This error will show for buckets that were present at the start of the validation, and were retired/removed before being validated. This will happen if you haven't stopped Log Insight service before running the bucket validator. Make sure to stop the Log Insight service before running the bucket validator..