To resolve this issue, upgrade vRealize Log Insight to 8.6 and higher contain the resolution for this issue.
Workaround:
To workaround the issue, boot into Single User mode to clear the filled log files, and configure log rotation.
- In the vSphere Client, open the console of the desired vRealize Log Insight node.
- With the console open, restart or power on the virtual machine.
- When the GRUB loader menu appears, immediately use the up and down arrow keys to navigate to the end of the line that starts with Photon OS or linux for new 8.x deployments.
Notes:
- Press the up and down arrow keys even if the option appears to already be selected. Otherwise, the machine continues to boot, and you have to start over.
- Type the letter e to go to the GNU GRUB edit menu.
- The cursor appears at the end of a line of boot options near the bottom of the display.
- If you cannot reach the boot menu before it disappears, enable Force BIOS setup in the Virtual Machine's Settings > VM Options > Boot Options and reboot.
- At the end of the line, add a space, then type rw init=/bin/bash which adds another option to the line.
- Press F10.
Note: The virtual appliance starts in single-user mode.
- Run the following commands to delete the audit.log and auth.log-xxxxxxx files.
rm /var/log/audit/audit.log
rm /var/log/auth.log*
- Exit Single User mode and boot the virtual machine normally
reboot -f
- Log into the node as root via SSH or Console, pressing ALT+F1 in a Console to log in.
- Open /etc/audit/auditd.conf in a text editor and set the max_log_file_action value to ROTATE, then save and close the file.
Note: Skip this step on vRealize Log Insight 8.4 and later.
- Run the following command to create the auth-logrotate file:
Note: Skip this step on vRealize Log Insight 8.1 and later.
touch /etc/logrotate.d/auth-logrotate
- Open /etc/logrotate.d/auth-logrotate in a text editor and add the following content, then save and close the file:
Note: Skip this step on vRealize Log Insight 8.6 and later.
/var/log/auth.log {
daily
missingok
rotate 5
compress
delaycompress
notifempty
create 640 root root
}
- Open the logrotate file in a text editor.
Notes:
- For vRealize Log Insight 8.4.1 and earlier the path is /etc/cron.daily/logrotate.
- For vRealize Log Insight 8.6 and later the path is /etc/cron.hourly/logrotate.
- Before the last line, add the following content then save and close the file:
if [[ -f /var/log/auth.log && ! -s /var/log/auth.log ]]; then
systemctl restart rsyslog
fi
Example: After editing, the file should look similar to the following.
#!/bin/sh
/usr/sbin/logrotate /etc/logrotate.conf
EXITVALUE=$?
if [ $EXITVALUE != 0 ]; then
/usr/bin/logger -t logrotate "ALERT exited abnormally with [$EXITVALUE]"
fi
if [[ -f /var/log/auth.log && ! -s /var/log/auth.log ]]; then
systemctl restart rsyslog
fi
exit $EXITVALUE