Why would the allocated proxy space for storing logs became exhausted?

Products

ProxySG Software - SGOS

Issue/Introduction

The same issue was resolved by setting the logging in the global settings from "Stop logging" to "Overwrite oldest logs" when the logging file is full.

Was this due to a sudden increase in something which took up space?

Could this have been avoided through regular maintenance?

Was this caused by a change?

Environment

Release : 6.7.5.12

Resolution

To respond, we speak to two scenarios:

Scenario 1:

Please note that if there is a communication error between ProxySG and the remote server, the access log gets accumulated on disk until it reaches 20GB, the designed limit on the appliance. The default behavior of SG is to stop logging if the sum of all log sizes reaches 20GB.

Ref. doc.: https://knowledge.broadcom.com/external/article/165323/proxysg-accesslog-failed-to-upload-to-a.html

If an Access Log file fails to upload as configured, Proxy will continue to store this file on local disk until it reaches the maximum disk size allocated for Access-Log. The file will be rotated locally based on the upload interval settings. The default disk space allocation for Access-Log is 20GB which can be viewed/changed at Configuration [Tab] > Access Logging > General > Global Settings.

Once the upload server resumes or start accepting log files, Proxy will start uploading the stored files to it. Once upload of these stored files completes, the local copy will be removed to free up space.

If the server is still not functional to receive logs when the Maximum log size is reached, Proxy will perform one of the below action based on the configuration:

Stop Logging
Delete oldest log entries

Note: Eventlog entries are created when an Access-Log upload is failing to configured server. Monitoring the same and making corrective action in time will avoid such an issue

Ref. doc.: https://knowledge.broadcom.com/external/article/167237/accesslog-storage-if-it-fails-to-upload.html

Scenario 2:

Important: If setting up access logging and configure the main log to be uploaded, be sure to configure other protocols that are intercepted, such as SSL, P2P, and IM. If they are not configured, the main log is uploaded in a regular and timely manner but the other protocols and logs are not uploaded at all. Over time these other log files grow to the point where the main access log has less room to grow and an early upload occurs because the global early upload limit is hit.

Workaround

In the Management Console, select Configuration > Access Logging > General > Global Settings. Increase the value for "Start an early upload if the log reaches xxxxx megabytes" and save changes. This helps alleviate the problem in the short term, but the problem could recur unless further action is taken. Review the Resolution in the shared article to permanently alleviate the problem.

For the above scenario, the following resolutions are possible:

Resolution 1: Configure upload settings for each log and upload all logs

Resolution 2: Log only protocols with a configured upload client and delete all logs

For the resolutions details, please refer to the Tech. Article with the URL below.

https://knowledge.broadcom.com/external/article/165321/access-logs-start-unwanted-early-uploads.html

So, whether it's scenario 1, where the access logs accumulate locally, on disk, within the ProxySG, because the remote server had issues and could not receive access logs, or it's scenario 2, where it's a matter with logging for the main log and not doing the same for the other protocols, as described in the shared article, we have shared details on why any of these scenarios would happen and we have also shared the workaround and resolution options.