NSX-T NAPP Platform Services disk usage is above the high threshold value of 75%
Article ID: 319119
Updated On:
Products
Issue/Introduction
Symptoms:
- NSX Intelligence version is 3.2.0, 3.2.1, or 3.2.1.1.
- Alerts indicate the disk usage of Platform Services service is above the high threshold value of 75%.
- Connecting to the fluentd-0 pod and looking in /opt/bitnami/fluentd/logs/buffers show a number of logs called:
-rw-r--r-- 1 domain users 24M Sep 26 21:42 continuousmonitoringjob-72a5bd837d3c1926-exec-1.log
-rw-r--r-- 1 domain users 29M Sep 26 22:42 continuousmonitoringjob-6983e8837d730804-exec-1.log
-rw-r--r-- 1 domain users 28M Sep 26 23:42 continuousmonitoringjob-7952ea837da9f62e-exec-1.log
-rw-r--r-- 1 domain users 28M Sep 27 00:42 continuousmonitoringjob-960941837de0e544-exec-1.log
The following command can be used from the NSX-T manager as root to view the files:
Environment
VMware NSX-T Data Center 3.x
VMware NSX-T Data Center
Resolution
The resolution to prevent this issue from occurring is in NSX intelligence 4.0.1 and later releases.
To clean up existing log files, see the Workaround section.
Workaround:
Create a cron job which will cleanup old log files.
To start cron job weekly to clean up the log files older than 1 week, run following commands from kubernates master node.
The below example sets up a cron job to execute at 20:00 hrs every Monday:
#0 20 * * 1 kubectl -n nsxi-platform exec -it fluentd-0 -- /bin/bash -c "find /opt/bitnami/fluentd/logs/buffers -type f -iname "continuousmonitoringjob*" -mtime +7 -delete"
If the above does not resolve the issue, it is also possible there are stale log folder entries inside the fluend-0 pod. Please log in to the pod by running the command below
Run this command to view the disk usage
and cd to directory below to verify if there is any stale worker node folder for the worker nodes which no longer exist.
Feedback
