vROps /storage/log partition full when Local6 logs are not rotated

Products

• VMware Aria Suite

Issue/Introduction

The purpose of this KB is to provide a list of steps to verify whether logs fail to rotate and how to resolve the issue.

Local6 logs are not rotated in vROPS due to a directory permission and /storage/log partition might be full due to bloated logs.

Environment

VMware Aria Operations 8.12.x

Cause

• Log file /var/log/local6 fails to rotate due to incorrect owner and group of parent directories.
• Directories /storage/log/var and /storage/log/var/log has wrong owner and group as per example below:
  

  drwxrwxr-x 13 admin admin 4.0K May 24 14:53 log

• Directories /storage/log/var and /storage/log/var/log must be owned by 'root' user and group

Resolution

This issue has been resolved in Aria Operations 8.12 HF1.

 

Workaround:

chown root:root /storage/log/var
chown root:root /storage/log/var/log
/usr/sbin/logrotate --verbose /etc/logrotate.conf

 

As an emergency procedure when root filesystem is full, you can truncate the file to 0 bytes to free up space used by the log file:

truncate -s 0 /var/log/local6

Additional Information

The owner and group of /storage/log/var and /storage/log/var/log should be 'root' and 'root', as per sample output below:

root@vrops01 [ /storage/log ]# ls -al
total 32
drwxrwxr-x 5 root  root   4096 Jul 16 16:44 .
drwxrwxr-x 8 root  root   4096 Aug 10 09:25 ..
drwxrwxr-x 2 admin admin 16384 Jul 16 16:38 lost+found
drwxrwxr-x 3 root  root   4096 Jul 16 16:43 var
drwxrwxr-x 4 admin admin  4096 Jul 16 16:45 vcops

root@vrops01 [ /storage/log/var ]# ls -al
total 12
drwxrwxr-x  3 root root 4096 Jul 16 16:43 .
drwxrwxr-x  5 root root 4096 Jul 16 16:44 ..
drwxrwxr-x 13 root root 4096 Sep 25 03:30 log