Steps to enable the JMX port to capture thread dump/ heap dump for vRealize Automation

Products

• VMware Aria Suite

Issue/Introduction

Symptoms:

• vRA takes long time for Workflows to execute.
• vRA Service might crash because of Memory leak.
• In the /var/log/vmware/vco/app-server/Server.log | Catalina.out file you see entries similar to:

SEVERE [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoader.clearReferencesThreads The web application [/vco] appears to have started a thread named [Timer-0] but has failed to stop it. This is very likely to create a memory leak.
SEVERE [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoader.clearReferencesThreads The web application [/vco] appears to have started a thread named [ldapCacheManager] but has failed to stop it. This is very likely to create a memory leak.
SEVERE [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoader.clearReferencesThreads The web application [/vco] appears to have started a thread named [sessionRegistryCacheManager] but has failed to stop it. This is very likely to create a memory leak.
SEVERE [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoader.clearReferencesThreads The web application [/vco] appears to have started a thread named [hornetq-buffer-timeout] but has failed to stop it. This is very likely to create a memory leak.

Environment

VMware vRealize Automation 7.x

Cause

This issue occurs due to Memory Leak on Java.

Resolution

To resolve this issue, you must enable JMX to troubleshoot session of vRA application.

To enable JMX port on vRA 7.x appliance:

1. Open the /etc/vcac/setenv-user file and add the following entries:

#Enable jmx console
VCAC_OPTS="$VCAC_OPTS -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=2839 -Dcom.sun.management.jmxremote.local.only=false -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Djava.rmi.server.hostname=IP_of_vRA_server"

 

2. Open the /etc/vcac/catalina.properties file and add entries at the bottom:

#configure jmx port listener:
java.rmi.server.hostname=IP_of_vRA_server

 

3. Restart vcac-server:

service vcac-server restart

Note: Set the IP of the vRA server instead of IP_of_vRA_server.