How to get details on the last 1000 crashed apps from CAPI.

Products

• VMware Tanzu Application Service for VMs

Issue/Introduction

This KB article demonstrates a CAPI call that obtains app details relating to the last 1000 crashed app instances. The details it retrieves are:

1. TIME
2. ORG_GUID
3. SPACE_GUID
4. APP_GUID
5. APP_NAME
6. APP_INDEX
7. APP_INSTANCE
8. CELL_ID
9. CRASH_REASON

Environment

Product Version: 2.11

Resolution

Note: The below command will leverage jq , and was constructed to deliver the output in a more reader friendly format.

1. Will need to be logged in as ADMIN. 
2. Leverage CAPI endpoint using the below cf curl command:

cf curl '/v3/audit_events?types=audit.app.process.crash&order_by=-created_at&per_page=1000'
| jq -r '.resources[] | 
"TIME " + .created_at + 
"\nORG_GUID " + .organization.guid +
"\nSPACE_GUID " + .space.guid +
"\nAPP_GUID " + .target.guid +
"\nAPP_NAME " + .target.name +
"\nAPP_INDEX " + (.data.index|tostring) +
"\nAPP_INSTANCE " + .data.instance +
"\nCELL_ID " + .data.cell_id +
"\nCRASH_REASON " + .data.exit_description +
"\n\n=============\n"' > last-1k-crashes.txt

• The above will generate a file called last-1k-crashes.txt. This file will contain the last 1000 app crash details per crash event. The output will look similar to the following:

=============
 
TIME 2023-11-03T20:02:25Z
ORG_GUID a10fc090-3bc0-44bf-8cb3-03e0649708ef
SPACE_GUID e9404f35-e4a6-438d-881e-2a6fafc91dc1
APP_GUID ce566b62-ba6c-497a-8ed2-8879aafdf229
APP_NAME cpu-burn
APP_INDEX 0
APP_INSTANCE d9633229-730a-4c31-61a9-a060
CELL_ID a38a18f5-efba-4182-9696-63cc7952858a
CRASH_REASON APP/PROC/WEB: Exited with status 137
 
=============
 
TIME 2023-11-03T19:53:46Z
ORG_GUID 4853bdab-13ec-4695-9d14-e317b27eed0e
SPACE_GUID d29b4045-4698-4765-9142-d9fd922a3f87
APP_GUID 25c94c07-c02f-48d0-ba70-b88e28b755b5
APP_NAME rmq
APP_INDEX 0
APP_INSTANCE 3e46274f-03e8-44ce-4606-2f22
CELL_ID a38a18f5-efba-4182-9696-63cc7952858a
CRASH_REASON Instance never healthy after 1m0.45s: Timed out after 1m0s (30 attempts) waiting for readiness check to succeed: failed to make TCP connection to 10.255.8.53:8080: dial tcp 10.255.8.53:8080: connect: connection refused
 
=============

Noteable:

• This specific CAPI endpoint allows requesting up to 5000 per page, so if desired you can change [per_page=1000] to [per_page=5000] and update the output file name.
• This endpoint also allows you to specify each page. For example, if you need the last 10k crashes you would append page= to the query. (as shown below)

  cf curl '/v3/audit_events?types=audit.app.process.crash&order_by=-created_at&per_page=5000&page=1'
  cf curl '/v3/audit_events?types=audit.app.process.crash&order_by=-created_at&per_page=5000&page=2'