TAS Cloud Controller Ephemeral Disk reaching 100%

Products

• VMware Tanzu Application Service

Issue/Introduction

When using TAS, it may be observed that the Cloud Controller ephemeral disk reaches 100% (or close to it) of it's ephemeral disk usage.

Checking the directory /var/vcap/data/cloud_controller_ng/tmp/ on the VM shows many files with the name local_bits_packerXXX

Another effect of this issue is that the Cloud Controller memory usage might also reach 100%. To verify this is happening, you can check the file /var/vcap/monit/monit.log. Within this file, you will see messages like the following:

[UTC Nov 23 11:10:20] error    : 'cloud_controller_worker_local_1' total mem amount of 88.7% matches resource limit [total mem amount>85.0%]

The VM will run with it's memory maxed out, until it forces a restart.

Environment

TAS 4.x and TAS 6.x

Cause

This issue occurs when the cloud controller attempts to unzip an applications packages file, but it unable to complete the operation. This has been observed when the zip file is malformed.

The zip command attempts to unzip the file, but then hangs as it is unable to. When the memory then forces the process to start again, it attempts the same again. This leaves the local_bits_packer directories behind, building up the disk usage again in a cycle.

More details can be found here:

https://github.com/cloudfoundry/cloud_controller_ng/pull/3987

Resolution

This issue is fixed in CAPI version 1.194.0. This is included in the following versions of TAS:

TAS 4.0.30+

TAS 6.0.10+

Please upgrade to one of these releases to resolve the issue.