SDDC Manager data partition getting filled up at 100%

Products

• VMware SDDC Manager

Issue/Introduction

 

• SDDC Manager UI is not accessible
• On restarting SDDC Manager, SDDC Manager services don't come up because of postgres related problems
• postgres service startup fails
• The device mounted on filesystem mount point - /dev/mapper/data_vg-data - is full at 100%

Environment

SDDC Manager 5.x
SDDC Manager 4.x

Resolution

Reclaim lost space by running the following psql command after logging in to each database and also on the default postgres user database:

1. Connect to the Postgres server
   
   psql -h localhost -U postgres;
   
   
2. Get the list of large tables by running the following query.
   
   SELECT nspname || '.' || relname AS "relation",
  pg_size_pretty(pg_total_relation_size(C.oid)) AS "total_size"
 FROM pg_class C
 LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace)
 WHERE nspname NOT IN ('pg_catalog', 'information_schema')
  AND C.relkind <> 'i'
  AND nspname !~ '^pg_toast'
 ORDER BY pg_total_relation_size(C.oid) DESC
 LIMIT 20;
   
   
3. Space can be reclaimed by running vacuum as follows:(reference: https://www.postgresql.org/docs/current/sql-vacuum.html)
   
   vacuum full verbose;

   
   
4. Then connect to each of the databases and repeat steps 2 and 3.
   
   databases: platform, domainmanager, lcm, operationsmanager, sddc_manager_ui
   
   example: \c platform

Additional Information

For SDDC manager 5.2 and above to access the database:

# /usr/pgsql/13/bin/psql -h localhost -U postgres
# \c platform;