Not able to access the login page after multiple unplanned reboots on Security Analytics

Products

• Security Analytics

Issue/Introduction

There are many database tables being written to during normal operation.  If a system is powered off multiple times in quick succession, one of the tables may become corrupted.  This might prevent access to the login page or create errors in /var/log/messages indicating that data cannot be inserted into tables.

Cause

Unscheduled power cycles causing corruption in redis.

Resolution

To refresh the redis caching service, login as root and run each of the following lines individually:

systemctl stop monit

systemctl stop redis-ui.service

mv /var/lib/redis-ui/dump-ui.rdb /home/admin/

systemctl start redis-ui.service

systemctl start monit

 

To clean up corrupted tables, you can run this command, but should be run cautiously.

 

Login as root and run:

 

/sbin/runuser -l postgres -c "psql -X -t -d dsweb -c \"SELECT 'SET zero_damaged_pages = on; VACUUM (FULL, ANALYZE, VERBOSE) ' || table_name || '; REINDEX TABLE ' || table_name || ';' AS q FROM information_schema.tables WHERE table_schema = 'public' AND table_catalog = 'dsweb' ORDER BY table_name;\"" > /tmp/vacuum_tables.sql; /sbin/runuser -l postgres -c "psql -X -t -d dsweb < /tmp/vacuum_tables.sql"; rm -f /tmp/vacuum_tables.sql

 

Call support if you are not comfortable with the postgres cleanup command.