/storage/log high in vCenter due to service-control.log occupying more space

Products

• VMware vCenter Server

Issue/Introduction

The /storage/log directory was utilizing a large amount of space in vCenter. Upon investigating the files with high space usage, we discovered that the service-control.log file was consuming several GB.

Environment

vCenter Server 7.x/8.x

Cause

Multiple loggings in the service-control about getting the status of services because of "watch" command triggered.

service-control.log getting status of all services every 4 seconds:
--------

XXXX-XX-XXT12:08:52.92Z INFO service-control ********** Start ['--status', '--all'] **********
XXXX-XX-XXT12:08:56.249Z INFO service-control ********** Start ['--status', '--all'] **********
XXXX-XX-XXT12:09:00.292Z INFO service-control ********** Start ['--status', '--all'] **********
XXXX-XX-XXT12:09:04.432Z INFO service-control ********** Start ['--status', '--all'] **********
XXXX-XX-XXT12:09:08.611Z INFO service-control ********** Start ['--status', '--all'] **********
XXXX-XX-XXT12:09:12.654Z INFO service-control ********** Start ['--status', '--all'] **********
XXXX-XX-XXT12:09:16.778Z INFO service-control ********** Start ['--status', '--all'] **********
XXXX-XX-XXT12:09:20.765Z INFO service-control ********** Start ['--status', '--all'] **********
XXXX-XX-XXT12:09:25.31Z INFO service-control ********** Start ['--status', '--all'] **********
XXXX-XX-XXT12:09:29.103Z INFO service-control ********** Start ['--status', '--all'] **********
XXXX-XX-XXT12:09:33.321Z INFO service-control ********** Start ['--status', '--all'] **********

Resolution

We can check the service-control processes using the below command:

ps -aef | grep 'service-control'

Example:

Based on the attached screenshot, we noticed that a "watch" command has been triggered on service-control since August, as indicated by the first entry.

We terminated the process with the following command: 

kill -9 <Process_ID>

Example: kill -9 9546

After terminating the process, please review the logging in the service-control.log file again.

Once the abnormal logging stops, you can clear the existing logs and continue monitoring the issue.

Additional Information

This issue can happen if customer uses any custom script to monitor the vCenter. So please check with the customer and also the processes for service-control.log using the above command.