SDDC Manager UI becomes inaccessible due to root (/dev/sda4) partition filling up

Products

• VMware Cloud Foundation

Issue/Introduction

Symptoms:

• SDDC Manager UI may be inaccessible with 503 Service Unavailable errors.
• Root partition on SDDC is full or almost full. 
• auth.log and/or message log are not rotating and filling up the root partition
   

 

Environment

VMware Cloud Foundation

Cause

The required log rotation file is missing from SDDC.

Resolution

If the root partition is full or messages log is over 1GB you should manually clear the messages log and auth.log prior to running this workaround.
echo > /var/log/messages
echo > /var/log/audit/auth.log
echo > /var/log/auth.log

1. SSH into the SDDC Manager and elevate to root
2. Navigate to the logrotate.d directory
cd /etc/logrotate.d
3. Copy the syslog file from the KB to the directory.
4. Update the permissions on the file.
chown root:root syslog
chmod 644 syslog
5. Initiate the log rotation 
logrotate -f /etc/logrotate.conf