Root partition out of space due to vrlcm database index bloat

Products

• VCF Operations/Automation (formerly VMware Aria Suite)

Issue/Introduction

Users report that the appliance is out of space or receiving "No space left on device" errors on the root (/) partition. Investigation reveals the vrlcm PostgreSQL database is the primary consumer of disk space, specifically due to index fragmentation in the vm_rs_request table.

Environment

 

• Product: VMware Aria Suite Lifecycle

• Version: 8.18.0

• Component: vrlcm Database (PostgreSQL)

 

Cause

The postgres service account expiration prevents routine maintenance (Autovacuum), leading to massive index bloat in the vm_rs_request table.

Resolution

Follow these steps to restore the service account and reclaim disk space:

Step 1: Restore the postgres Service Account

Log in to the appliance via SSH as root and run the following commands to unlock the account and set it to never expire:

chage -m 0 -M 99999 -I -1 -E -1 postgres
passwd -u postgres

Step 2: Reclaim Database Space

Connect to the PostgreSQL instance and perform deep maintenance to collapse the index bloat:

1. Access the database: vrlcm-db-status (or your standard local PSQL login).

2. Run the following SQL commands:

REINDEX TABLE vm_rs_request;
VACUUM FULL vm_rs_request;

Note: VACUUM FULL requires exclusive locks on the table and may take time depending on the size of the bloat.

Step 3 (Optional): Enable Persistence for Maintenance

Note: This should be enabled out-of-the-box.

Ensure the database correctly tracks statistics for the Autovacuum daemon:

ALTER DATABASE vrlcm SET track_counts = 'on';

Additional Information

Additional Information

Subscribe to this knowledge article to get updates on this issue. If you are unable to clear enough space to run the VACUUM command, please attempt to increase the partition space: Extend root and alt_root volumes.