How to view failed delayed jobs in App Usage Service in Pivotal Application Service (PAS)

Products

• VMware Tanzu Application Service for VMs

Issue/Introduction

The usage service relies on delayed jobs to fetch usage events from the Cloud Controller (CC) and process those events into rollup tables. If there are anomalies in the data returned by the usage service API, a common debugging technique is to see if any of the delayed jobs have failed and provide the failure stack traces to the usage service team.

Environment

Resolution

Note: These instructions will cause brief downtime while the usage service app is restarted.

How to retrieve stack traces from the Delayed Job GUI in usage service

1. Pick an arbitrary secure username and password and replace them in the commands below:

cf set-env app-usage-server DELAYED_JOB_UI_PASSWORD <password>
cf set-env app-usage-server DELAYED_JOB_UI_USERNAME <username>
cf set-env app-usage-server ENABLE_DELAYED_JOB_UI true
cf restart app-usage-server

2. Retrieve the usage service url by running the following: cf app app-usage-server

3. Navigate too: <usage_service_url>/delayed_job/overview.

4. (Optional) Once you have obtained the stack traces, you can optionally remove access to the delayed job GUI with the following commands: 

cf unset-env app-usage-server DELAYED_JOB_UI_PASSWORD
cf unset-env app-usage-server DELAYED_JOB_UI_USERNAME
cf unset-env app-usage-server ENABLE_DELAYED_JOB_UI
cf restart app-usage-server