/storage/log is full on Aria Operations

Products

• VMware Aria Suite

Issue/Introduction

Symptoms:

• The /storage/log partition is full on one or more Aria Operations node and these log files are consuming this space:
  • /var/log/auth.log
  • /var/log/messages
  • /var/log/warn

Environment

Aria Operations 8.x

Cause

In Aria Operations 8.x this is caused by the move to Photon OS and an incorrect log rotation mechanism.

Resolution

To resolve this issue remove the /var/log/warn, /var/log/auth.log, and /var/log/messages files.

1. Log into the Aria Operations node as root via SSH or Console, pressing ALT+F1 in a Console to log in.
2. Run the following command to stop the syslog service:

service syslog stop

3. Remove the /var/log/warn, /var/log/auth.log, and /var/log/messages files by running this command:

​rm -f /var/log/warn* /var/log/auth.log* /var/log/messages*

4. Run the following command to start the syslog service:

service syslog start

5. Repeat steps 1-4 on any other affected nodes.

Note: If this article does not resolve your issue:

• see Troubleshooting Storage Issues in Aria Operations (318408)
  
  
• If running 8.12.x there is a known issue with the local6 file not being rotated

Additional Information

Impact/Risks:
Logs only fill the /storage/log partition. Removing log data will not reduce space in the other partitions.

It is recommended to take a snapshot following How to take a Snapshot of VMware Aria Operations (342576) before proceeding with the steps below.