WARNING: database must be vacuumed

book

Article ID: 167814

calendar_today

Updated On:

Products

XOS

Issue/Introduction

This article describes a work-around and a solution for the error message 'database must be vacuumed '.Symptoms:

When trying to execute any command from the XOS CLI, the CPM may show a WARNING message similar to the following:

WARNING: database "running-config_alt" must be vacuumed within 7354631 transactions

A similar entry is recorded in /var/log/messages:

WARNING: database "running-config_alt" must be vacuumed within 10994670 transactions HINT: To avoid a database shutdown, execute a full-database VACUUM in "running-config_alt".

The same warning may appear for databases postgres or alarmNtfDbase. The warnings can be also seen in the show tech-support file.

In some cases, the /cbconfig partition may run out of space due to this issue and XOS CLI becomes unavailable.

Cause

Problem:

If your system displays these messages, maintenance of the Postgres database is required in order to vacuum old transactions. 

A transaction ID is a global parameter in Postgres and shared by all databases. The wrap limit that triggers the need for the vacuum operation is calculated with the oldest transaction ID that might exist in any database. As soon as the limit is reached, the /cbconfig disk partition gets filled up quickly due to logging of the warning at every transaction, the database eventually stops and the XOS CLI cannot be accessed. 

Resolution

To prevent the messages from appearing in the future, you should add the following cron job on each CPM of the chassis to ensure that the database is regularly vacuumed. Edit the /var/spool/cron/postgres file on the CPM and add the highlighted line with "vacuumdb -z -a" if it is missing:

0 2 * * 1-6 /usr/bin/vacuumdb -z -d running-config 
0 2 * * 0 /usr/bin/vacuumdb -z -a 
0 */2 * * * /usr/bin/vacuumdb -z -t webSessionTable -d running-config 
0 */2 * * * /usr/bin/vacuumdb -z -t alarmTable -d running-config 



This is the recommended content of the postgres crontab file and has been included with XOS V9.7.2. The change is referenced in the Release Notes:

ID 102207 The vacuum command now runs on all postgres databases on a daily schedule.

Workaround

To eliminate these messages, you need to manually vacuum the postgres database. Use the following steps:

1.  Create a copy of the current running-configuration:

CBS# copy running-config -flat /tmp/backup_xos_config
 
Save and keep a copy of the configuration external to the chassis.

2.  Run the following commands to vacuum the database and to clear the error:

     /usr/bin/vacuumdb -z -d running-config_alt -U postgres
     /usr/bin/vacuumdb -z -d postgres -U postgres