The /storage/dblog partition on vCenter filing up to 100% causing disk exhaustion

Products

• VMware vCenter Server
• VMware vCenter Server 8.0

Issue/Introduction

• The /storage/dblog partition on the vCenter Server Appliance (VCSA) is reaching 100% utilization
  
  
• /var/log/vmware/vpostgres/postgresql.log

YYYY-MM-DD HH:MM:SS 66bf245f.9af 0    2479 59573WARNING:  Available disk space of data folder is under pressure:allowed percentage is 0.2 available percentage is 0.0056725
YYYY-MM-DD HH:MM:SS 66bf245f.9af 0    2479 59574LOG:  Checking bloat of WAL caused by replication slots
YYYY-MM-DD HH:MM:SS 66bf245f.9af 0    2479 59575LOG:  Memory check: flag = rss, mem_used_bytes = 378804 kB, mem_avail = 1301504 kB
YYYY-MM-DD HH:MM:SS 66bf245f.9af 0    2479 59576LOG:  Writing instance status...
YYYY-MM-DD HH:MM:SS 66bf245f.9af 0    2479 59577LOG:  Wrote instance status successfully.
YYYY-MM-DD HH:MM:SS 66bf245f.9af 0    2479 59578LOG:  Updated instance status successfully.
YYYY-MM-DD HH:MM:SS 66bf245f.9ad 0    2477 4392WARNING:  archive_mode enabled, yet archive_command is not set

• When the /storage/dblog partition reaches 100% utilization, the vCenter Server service (vpxd) stops functioning.

Cause

The system is retaining WAL files because archive_mode is enabled but archive_command is not set. PostgreSQL requires a valid archive_command to confirm successful archiving before WAL files can be deleted.

Resolution

To resolve the issue, perform the below steps

• Log in to the VCSA via ssh
  
  
• Stop all services and start vmware-vpostgres service

service-control --stop --all && service-control --start vmware-vpostgres

• Execute the below command to update the configuration file

sed -i 's/archive_mode=.*/archive_mode=off/' /storage/db/vpostgres/postgresql.conf

• Start all services on the VCSA using the below command

service-control --start --all