Log Disk Exhaustion caused by wcp stdstream.log
Article ID: 326227
Updated On:
Products
Issue/Introduction
To prevent disk space exhaustion in the /storage/log partition of vCenter Server
Symptoms:
-
vSphere Client is displaying a warning "Log Disk Exhaustion on <vcenter-name>" on the summary page for the vCenter Server
-
Upon investigation the folder /var/log/vmware/wcp is found to be either one of or the largest folder on the /storage/log partition
-
Looking in the folder, there is one or more very large stdstream.log files, similar to the example below:
# ls -althrS /var/log/vmware/wcp | tail -5 1.4M -rw------- 1 wcp lwisRegReader 1.4M <DATE> <TIME> stdstream.log-1.stderr 34M -rw------- 1 wcp lwisRegReader 34M <DATE> <TIME>stdstream.log-4.stderr 111M -rw------- 1 wcp lwisRegReader 111M <DATE> <TIME> stdstream.log.stderr 1.1G -rw------- 1 wcp lwisRegReader 1.1G <DATE> <TIME> stdstream.log-3.stderr 1.6G -rw------- 1 wcp lwisRegReader 1.6G <DATE> <TIME> stdstream.log-0.stderr
Environment
Cause
The stdstream.log.stderr logs are currently not rotated by size, but only when the wcp service restarts. When wcp runs for longer amounts of this, their size can therefore grow quite large, thus allocating large portions of the space available in /storage/log, and potentially resulting in the space getting exhausted.
Resolution
Currently there is no permanent solution available.
VMware engineering is aware of this issue and working towards a fix in a future release
Workaround:
To mitigate this problem
1. Please take a fresh backup or powered off snapshot of the vCenter Server Appliance
2. Remove all stdstream.log* files from the folder
# rm /storage/log/vmware/wcp/stdstream.log*.stderr
3. Restart the wcp service:
# service-control --stop wcp && service-control --start wcpAdditional Information
Feedback
