Manual heap dump for application hanging or slow performance

Products

• Clarity PPM On Premise
• Clarity PPM SaaS

Issue/Introduction

Your application is hanging (or has high CPU, high RAM, or just slowness) but there is no Out of Memory (OOM) or outage, how can you determine what is going on? 

Environment

Release : All Supported Clarity Releases

Resolution

To further investigate the reasons for slow performance, you can generate a manual JVM heap dump on the server. This has to happen at the precise time you experience the slowness.

The heap dump is a snapshot of what is going on the server. This will provide a better idea of what is running and what is taking resources.

Note: This manual heap dump is different from a Heap dump for an OOM (aka automatic heap dump) as the application is not in outage so the OOM heap dump may not be generated.

Steps:

1. Connect to the Clarity server having the slowness issue. 
2. Run the Java command below (you may have to run it from Java directory):
jmap -dump:format=b,file=heapdump <process id>
3. This will create a dump from the moment of the hanging thread like a current snapshot.

• • Open and analyze as a heap dump.
  • Please make sure that you have enough disk space.

IMPORTANT: 

You always have to provide the PID for the java.exe, not the Clarity Tanuki Service Wrapper (e.g. serviceappcmd.exe). Otherwise, the command will fail. 

Windows Server:

Use the Task Manager to add the Command Line and identify the process PID you need. Example below:

If the service you are trying to work with is an app or NSA, you can also usually check for which process the port is listening on in order to determine through the netstat command.  For example, if you have your app URL listening on port 1561 so that users visiting Clarity enter http://servername:1561 to connect, you can find the PID with this (Windows):

   D:\Clarity>netstat -ano | find /i ":1561"

      TCP    0.0.0.0:1561         0.0.0.0:0              LISTENING       7164

   D:\Clarity>

Linux:

Use this command to get the PID of the "app" or "bg" server

ps -ef | grep java | grep bg | grep jdk | awk {'print$2'}

ps -ef | grep java | grep app | grep jdk | awk {'print$2'}

Additional Information

Note: To perform this action is a low impact, as the system is already slow. The only thing that could happen if the JVM is non responsive then the heap dump will not be taken until it gets to be responsive again so you may not have a correct snapshot of the situation.