Alarm Being Generated - REPORT MANAGER CONTAINS STALE BUCKETS

Products

• Network Observability
• Spectrum

Issue/Introduction

There are too many unprocessed files in the reporting folder of SRM, resulting in the following alarm being generated:

"REPORT MANAGER CONTAINS STALE BUCKETS"

Environment

Spectrum All Versions

Resolution

First it would be best to check what type of buckets are being created by logging into the reporting database:

From the Oneclick Server running Report manager:

1. Cd to SQL Directory:
cd $SPECROOT/mysql/bin

2. Enter MySQL reporting database:
./mysql -uroot --pMySqlR00t reporting

3. At the mySQL prompt, run the below queries. Null will show count of all bucket files. The other queries will show which type of bucket file are there.

mysql> select count(1) from bucketactivitylog where destroy_time is null;
mysql> select count(1) from bucketactivitylog where destroy_time is null and bucket_table_name like "security%";
mysql> select count(1) from bucketactivitylog where destroy_time is null and bucket_table_name like "alarm%";
mysql> select count(1) from bucketactivitylog where destroy_time is null and bucket_table_name like "ncm%";
mysql> select count(1) from bucketactivitylog where destroy_time is null and bucket_table_name like "modelcreatedestroybucket%";
mysql> select count(1) from bucketactivitylog where destroy_time is null and bucket_table_name like "modelavailbucket%";
mysql> select count(1) from bucketactivitylog where destroy_time is null and bucket_table_name like "spmeventbucket%";
 
4. Once verified which type of files are taking the brunt of the bucket, turn on debugging from Oneclick > Administration > Debugging Link

5. Stop Tomcat.
cd $SPECROOT/tomcat/
./stopTomcat.sh

6. Tail catalina.out log
cd $SPECROOT/tomcat/logs/
tail -f catalina.out

7. Start Tomcat.
cd $SPECROOT/tomcat/
./startTomcat.sh

**Typically restarting tomcat will fix the issue, if not you will see errors in catalina.out**