"Edge node disk usage is high" alarm is generated on NSX manager due to /var/log/frr/frr.log

Products

• VMware NSX

Issue/Introduction

• Running NSX version 4.2.1.x, up to, but not including 4.2.1.3.
• Alarms are generated on the NSX manager for the edge indicating the disk usage is high or very high.
• Log in to the Edge node indicated in the alarm as root user run df -h shows the partition is very large:
  
• Running du -sh shows a very large /var/log/frr/frr.log file:

         

• Due to this disk space usage, Edge node log extraction will be failing on the NSX UI.
• You may notice intermittent packet drops for traffic traversing the edge node.

Environment

VMware NSX

Cause

In upgrades to VMware NSX 4.2.1.x (less than 4.2.1.3), the frr service is missing from log rotate.

This issue does not occur with fresh greenfield deployment of NSX 4.2.1.x (less than 4.2.1.3).

Resolution

This issue is resolved in VMware NSX 4.2.1.3, available at Broadcom downloads.

If you are having difficulty finding and downloading software, please review the Download Broadcom products and software KB.

Workaround:

Copy and paste the content of frr.txt file (attached to this KB) into the /etc/logrotate.d directory as a new file called frr, using vi or vim editor. The next time log rotate runs, it will read this file and rotate the /var/log/frr/frr.log file(s), log rotate runs every hour.

The file permission should look like:

-rw-r--r-- 1 root root  973 Oct 25  2018 frr  

Additional Information

If utilization of /var/log/frr is 100%, the log rotation script will not work, you need need to truncate the file /var/log/frr/frr.log

If the file is needed for historical reasons, use Winscp/SCP and copy the file of the appliance and truncate the file by running '> frr.log'