How to debug Java application running on PWS?

Products

Services Suite

Issue/Introduction

The Java Virtual Machine (JVM) ships with several tools for debugging a running Java application. These are tools like jvisualvm and jstack, which connect to the app and dump various bits of information. However, in order to run these tools, you'll need to get direct access to your application's container or access via JMX. This article walks through setting up access to debug your application running on PWS.

Environment

Resolution

Since PWS migrated it's users to Diego, it is now possible to access an application instance directly via SSH or indirectly via an SSH tunnel. For Java applications, this can be used to connect and execute command line tools or to connect remotely via tools like jvisualvm. The two sections below describe how to do both of these tasks.

Command Line Debugging

To access the container to debug from the command line, you'll need to perform the following steps.

Make sure you have a recent version of the CF CLI installed. You need version 6.13 or greater. The most recent version can be downloaded from here.
Login with cf and target your Org and Space.
Run cf ssh <app-name>. If you have multiple instances of your application, you can use the -i argument to specify the specific app index number.

Once connected, run these two commands.

export JAVA_HOME=$HOME/app/.java-buildpack/open_jdk_jre/
export PATH=$PATH:$JAVA_HOME/bin

These two commands will setup the environment.

You can now run any of the typical utilities for Java troubleshooting, like jstack, jmap, jcmd and keytool.

Remote Debugging

To access your application remotely, you'll need to perform the following steps.

Make sure you have a recent version of the cf cli installed. You need version 6.13 or greater. The most recent version can be downloaded from here.
Login with cf and target your org and space.
Run cf set-env <app-name> JBP_CONFIG_JMX '{enabled: true}' and run `cf restage <app-name` to restage the app or add the line JBP_CONFIG_JMX: '{enabled: true}' to the env block of your manifest.yml file and run `cf push <app-name>` to update your app. Either option will instruct the Java build pack to enable JMX for your application. Please note this requires Java build pack version 3.4 or greater.
Run cf ssh -N -T -L 5000:localhost:5000 <app-name>. The command will appear to hang, but it's working. As long as it's running, it will keep open an ssh tunnel for your JMX traffic.
Start jvisualvm or jconsole on your PC. Enter localhost:5000 as the address to connect to your application. No authentication or SSL is required since we're going through the SSH tunnel.

For more details on connecting via JMX, please see the Java build pack documentation.

Additional Information

YourKit

If you have access to YourKit, which is a commercial Java Profiler, you can also use that to debug your Java applications. The process is very similar to what's described for remote JMX debugging above but uses a proprietary protocol to communicate over the SSH tunnel. Instructions for configuring can be found in the Java build pack documentation.

Heap Dumps

Please be aware that applications with a large heap, over 2GB, will not be able to generate heap dumps. This is because the JVM will write the heap dump information to the container's file system and the maximum size of the container's file system is 2GB. If your application's heap is greater than 1GB, don't forget to set the -k argument to cf push to increase the disk size from 1GB to 2GB, as 1GB is the default size for the container's file system.