Using dontpanic to Collect Diagnostic Data from Diego Cell

Products

• VMware Tanzu Application Service for VMs

Issue/Introduction

In some scenarios platform users may need to diagnose issues specific to the running Diego Cells in a foundation.

Along with relevant log collection (Diego Cell logs, Support Bundle, any other relevant Diego Component logs) the DontPanic diagnostic can be gathered and shared for more information. 

Resolution

Go to a relevant Diego Cell. If you are curious about which Diego cell is a relevant cell to your app issue please use the following command within the space of the app. (Note the following grep requires Perl extension)

cf apps | grep --color started | awk '{print $1}' | xargs -I{} cf app {} --guid | xargs -I{} cf curl /v2/apps/{}/stats | grep --color -oP '"name":\s*"\K([^"]+)|"host":\s*"\K([^"]+)|"port":\s*\K([^,]+)|}$' | xargs | sed 's# }[ }]*#\n#g' | awk /./ 

The output will inform of which Cell and the relevant port the container app container is hosted. Using the identified IP of the cell, SSH into it via Bosh and run the following.

sudo /var/vcap/packages/dontpanic/bin/dontpanic 

The script can run for a few minutes collecting data. While running, the script will instruct to copy the relevant details being output and finally list the stored location of the bundle in /var/vcap/data/tmp. 

Scp the script off the VM or move the bundle into the /var/vcap/data/sys/log directory to be collected from a Diego log collection:

mv /var/vcap/data/tmp/os-report-bundleID.tar.gz /var/vcap/sys/log/ 

Collecting Diego Cell log:
https://community.pivotal.io/s/article/How-to-collect-bosh-logs-for-your-PCF-deployments?language=en_US
https://docs.pivotal.io/application-service/2-11/operating/diagnostics.html#component_logs 

SCP the file off of your VM and upload to the case for review. 

--------

Note: The Don't Panic script will collect the following logs:

## Date
## Uptime
## Garden Version
## Hostname
## Memory Usage
## Kernel Details
## Monit Summary
## Number of Open Files
## Max Number of Open Files
## Disk Usage
## GrootFS Unprivileged Usage
## GrootFS Privileged Usage
## List of Open Files
## Map of Inodes to Paths
>> Map of Inodes to Paths failed: timed out after 1m0s
## Process Information
## Process Tree
## Kernel Messages
## Network Interfaces
## IP Tables
## NAT IP Tables
## Mount Table
## Garden Depot Contents
## XFS Fragmentation
## XFS Info
## Slabinfo
## Meminfo
## IOSTAT -xdm (slow)
## VMSTAT -s
## VMSTAT -d (slow)
## VMSTAT -a (slow)
## Mass Process Data
## Kernel Log
## Monit Log
## Syslog
## Garden Config
## Garden Logs
## Sysstat
## Garden Containers
## Containerd Init Containers
## Containerd Pea Containers
## Containerd Tasks