Applications failing to stage after upgrading to VMware Tanzu Application Service (TAS) for VMs 2.7.12 - 2.7.15

Products

• VMware Tanzu Application Service for VMs

Issue/Introduction

Apps are failing to stage with the following errors in VMware Tanzu Application Service (TAS) for VMs:

stderr: Error staging application: StagingTimeExpired

OR

  "description": "Stager error: bbs stager client staging failed: the requested resource already exists",
  "error_code": "CF-StagerError"

Note: When you SSH to the diego_database and diego_brain VM, the htop command shows the process, loggr-forwarder-agent, is causing high CPU usage.

Environment

Product Version: 2.7

Resolution

This is a known issue with TAS for VMs  2.7.12 - 2.7.15 releases caused by a recent change of the GoLang version to 1.14.1.

There are two workarounds which will mitigate this issue:

Workaround 1

Schedule a cron job for restarting the loggr-forwarder-agent for diego_database and diego_brain - this BOSH command can be used:

bosh -d <CF_DEPLOYMENT> ssh diego_database -c "PATH=$PATH:/var/vcap/bosh/bin sudo monit restart  loggr-forwarder-agent"

Workaround 2

Increase the vm_type for the diego_database and diego_brain (Diego BBS and Diego brain in the TAS tile) to assign more CPU instances.

To fix this known Golang issue, please upgrade to TAS for VMs 2.7.16+, where the GoLang version has been bumped to v1.14.2:

• https://github.com/cloudfoundry/loggregator-agent-release/releases/tag/v3.21.10
• https://docs.pivotal.io/platform/2-7/release-notes/runtime-rn.html#2.7.16

This issue with GoLang is fixed in TAS 2.7.16: https://docs.pivotal.io/platform/2-7/release-notes/runtime-rn.html#2.7.16