Usage Service Errand for Tanzu Platform for Cloud Foundry fails with error 'app-usage-server-venerable' does not exist.

Products

• VMware Tanzu Platform
• VMware Tanzu Application Service

Issue/Introduction

When running Apply Changes on Tanzu Platform for Cloud Foundry tile Usage Service Errand fails continuously. This causes the Apply Changes to fail preventing successful deployment of TPCF tile. Following errors are observed in the errand logs

App 'app-usage-worker-venerable' does not exist.
App 'app-usage-scheduler-venerable' does not exist.
App 'app-usage-server-venerable' does not exist.
App 'app-usage-worker-venerable' does not exist.
App 'app-usage-scheduler-venerable' does not exist.

Cause

This issue can occur if there are insufficient resources allocated available to run the app-usage-* applications. To confirm resource issue logs in the clock_global instances can be reviewed.

clock_global.xyz:/var/vcapsys/log/push-usage-service/push-usage-service.stdout.log:

InsufficientResources - Insufficient resources
InsufficientResources - Insufficient resources
InsufficientResources - Insufficient resources
InsufficientResources - Insufficient resources

To find the type of resource that is insufficient for the application, review the cloud_controller_ng logs on cloud_controller instances. In this scenario on  further examination it was discovered that the application staging is failing due to unavailability of disk resources.

cloud_controller:/var/vcap/sys/log/cloud_controller_ng/cloud_controller_ng.log:

{"timestamp":"2025-06-16T04:50:07.421516185Z","message":"diego.staging.buildpack.finished","log_level":"info","source":"cc.stager","data":{"request_guid":"cd57dfba-cb3e-4e6e-8b8f-73ec421b8225","response":{"error":{"id":"InsufficientResources","message":"insufficient resources: disk"}}},
"thread_id":73740,"fiber_id":73760,"process_id":6,
"file":"/var/vcap/data/packages/cloud_controller_ng/49389ba4ba7173868a1ed7ae4975259eb1a02bab/cloud_controller_ng/lib/cloud_controller/diego/staging_completion_handler.rb","lineno":27,"method":"staging_complete"}

 

Resolution

To resolve this issue the number of diego_cells can be increased to add more disk capacity or existing diego_cells can be resized to use a larger ephemeral disk by changing it's VM type.