How to generate a heapdump in VMware Aria Operations

Products

• VMware Aria Suite

Issue/Introduction

While working with VMware by Broadcom Support, you may be requested to provide a heapdump from your VMware Aria Operations deployment for further troubleshooting.  This article provide the steps to generate a heapdump in VMware Aria Operations

Environment

VMware Aria Operations (formerly known as vRealize Operations) 8.x

Resolution

The heapdump generation is on a per node basis.  If multiple heapdumps per process or node are requested, repeat the below steps as needed

1. Log into the desired VMware Aria Operations node as root via SSH or Console
2. Ensure the service for which the heapdump should be generated is in an active (running) state by running the following command:

service vmware-vcops status service_name

Note: Replace service_name with the service name of the service you want to generate the heapdump for.  Use the table outlined in step 5 for a list of service names

Example: service vmware-vcops status collector

3. Run the following command to see the available diskspace of the /storage/db drive:

df -H /storage/db

Note: The size of heapdump depends on multiple factors, such as node size, or the service the heapdump is generated on.  For example, an Analytics heapdump generated on an extra large sized node could be 110GB or more.  Ensure there is enough available space left on the /storage/db drive.  See Troubleshooting Storage Issues in vRealize Operations (83239) if you need to free up space

4. Run the following command to change to the /usr/lib/vmware-vcops/common/lib directory:

cd /usr/lib/vmware-vcops/common/lib

5. Run the following command to generate the heapdump:

VMware Aria Operations 8.6.4 - 8.12.x

java -Djavax.net.ssl.trustStore=NONE -cp "alive_platform.jar:log4j-core-2.17.2.jar:vrops-logging-1.0-SNAPSHOT.jar:log4j-api-2.17.2.jar:vrops-jmx-metrics-1.0.jar:vrops-trustmanager-3.0-SNAPSHOT.jar:vcops-security-1.0-SNAPSHOT.jar" com.vmware.vcops.common.JMXCommandExecutor service-port-number dumpHeap /storage/db/vcops/heapdump

VMware Aria Operations 8.14.x and later

java -Djavax.net.ssl.trustStore=NONE -cp "alive_platform.jar:log4j-core-2.17.2.jar:vrops-logging-1.0-SNAPSHOT.jar:log4j-api-2.17.2.jar:vrops-jmx-metrics-1.0.jar:vrops-trustmanager-3.0-SNAPSHOT.jar:vcops-security-1.0-SNAPSHOT.jar:bc-fips-1.0.2.3.jar:bctls-fips-1.0.12.3.jar" com.vmware.vcops.common.JMXCommandExecutor service-port-number dumpHeap /storage/db/vcops/heapdump  

Note: Replace service-port-number with the port number of the VMware Aria Operations service you need to generate a heapdump for.

Service	Service Name	Port Number
Analytics	analytics	9004
Collector	collector	9005
UI (tcServer)	ui	9006
API (REST)	api	9007

Example: java -Djavax.net.ssl.trustStore=NONE -cp "alive_platform.jar:log4j-core-2.17.2.jar:vrops-logging-1.0-SNAPSHOT.jar:log4j-api-2.17.2.jar:vrops-jmx-metrics-1.0.jar:vrops-trustmanager-3.0-SNAPSHOT.jar:vcops-security-1.0-SNAPSHOT.jar:bc-fips-1.0.2.3.jar:bctls-fips-1.0.12.3.jar" com.vmware.vcops.common.JMXCommandExecutor 9004 dumpHeap /storage/db/vcops/heapdump

6. Once the generation is complete, the resulting file will be located in the /storage/db/vcops/heapdump directory.  An SCP Utility, such as WinSCP, can be used to copy the heapdump so it can be uploaded to VMware Support.