search cancel

AWI tomcat goes down with java heap space error

book

Article ID: 243065

calendar_today

Updated On:

Products

CA Automic Workload Automation - Automation Engine

Issue/Introduction

Automic web interface performance degrades at times. The following exception is captured in the tomcat's std error file (%CATALINA_BASE%\logs\apachetomcat8-stderr.<date>.log)
28-May-2022 10:38:50.639 SEVERE [ContainerBackgroundProcessor[StandardEngine[Catalina]]] org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run Unexpected death of background thread [ContainerBackgroundProcessor[StandardEngine[Catalina]]]
java.lang.OutOfMemoryError: Java heap space
28-May-2022 14:06:18.698 INFO [Thread-51] org.apache.coyote.AbstractProtocol.pause Pausing ProtocolHandler ["http-nio-8080"]uler-1554" java.lang.OutOfMemoryError: Java heap space
java.lang.OutOfMemoryError: Java heap space
Exception in thread "UC4 System=AUTOMIC, Client=0100, User=AUTOMIC, SessionID=0000000004882026" java.lang.OutOfMemoryError: Java heap space
Exception in thread "AsyncFileHandlerWriter-664740647" java.lang.OutOfMemoryError: Java heap space
Exception in thread "ContainerBackgroundProcessor[StandardEngine[Catalina]]" java.lang.OutOfMemoryError: Java heap space

Restarting the tomcat seems to work around the problem. However, it returns in a few hours of continuous operation.

Cause

The problem is due to the tomcat does not have enough heap size to process the user request.

Environment

Release : 12.2 and higher

Component :

Resolution

The heap size for the tomcat needs to be increased to resolve the problem. By default, Tomcat allocates a low amount of memory to an application (256MB by default). This is not sufficient for AWI to process the concurrent user requests.

Windows:

    1. Go to the bin folder in your Tomcat installation (...Apache Software Foundation\Tomcat<version>\bin).
    2. Right-click the Commons Daemon Service Manager file (tomcat9w.exe) file and, Run as administrator.
    3. On the Java tab, set the heap size by setting the Maximum memory pool to the maximum possible on your system, for example, 8192MB (-Xmx8192m).

Linux:

In the CATALINA_OPTS environment variable, set the heap space to the maximum possible on your system, for example, 8192MB.

export CATALINA_OPTS=-Xmx8192m

On Linux environments, the variable can be set in the tomcat server startup script ($CATALINA_BASE/bin/start.sh).

Restart the tomcat service and verify the heap message in the Catalina log file ($CATALINA_BASE/log/catalina.log)  reflects the updated value.


02-Jun-2022 22:42:46.292 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Java Home:             C:\Program Files\Java\jdk1.8.0_202\jre
02-Jun-2022 22:42:46.292 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log JVM Version:           1.8.0_202-b08
02-Jun-2022 22:42:46.292 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log JVM Vendor:            Oracle Corporation
02-Jun-2022 22:42:46.292 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log CATALINA_BASE:         C:\Automic\External.Resources\tomcat9
02-Jun-2022 22:42:46.292 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log CATALINA_HOME:         C:\Automic\External.Resources\tomcat9
02-Jun-2022 22:42:46.296 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dcatalina.home=C:\Automic\External.Resources\tomcat9
02-Jun-2022 22:42:46.296 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dcatalina.base=C:\Automic\External.Resources\tomcat9
02-Jun-2022 22:42:46.296 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dignore.endorsed.dirs=C:\Automic\External.Resources\tomcat9\endorsed
02-Jun-2022 22:42:46.296 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.io.tmpdir=C:\Automic\External.Resources\tomcat9\temp
02-Jun-2022 22:42:46.298 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
02-Jun-2022 22:42:46.298 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.util.logging.config.file=C:\Automic\External.Resources\tomcat9\conf\logging.properties
02-Jun-2022 22:42:46.298 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: exit
02-Jun-2022 22:42:46.298 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: abort
02-Jun-2022 22:42:46.298 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Xms128m
02-Jun-2022 22:42:46.298 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Xmx8192m
02-Jun-2022 22:42:46.303 INFO [main] org.apache.catalina.core.AprLifecycleListener.lifecycleEvent Loaded Apache Tomcat Native library [1.2.32] using APR version [1.7.0].
02-Jun-2022 22:42:46.303 INFO [main] org.apache.catalina.core.AprLifecycleListener.lifecycleEvent APR capabilities: IPv6 [true], sendfile [true], accept filters [false], random [true], UDS [true].
02-Jun-2022 22:42:46.303 INFO [main] org.apache.catalina.core.AprLifecycleListener.lifecycleEvent APR/OpenSSL configuration: useAprConnector [false], useOpenSSL [true]
02-Jun-2022 22:42:46.309 INFO [main] org.apache.catalina.core.AprLifecycleListener.initializeSSL OpenSSL successfully initialized [OpenSSL 1.1.1n  15 Mar 2022]
02-Jun-2022 22:42:46.824 INFO [main] org.apache.coyote.AbstractProtocol.init Initializing ProtocolHandler ["http-nio-8080"]

Additional Information

There is no recommendation on how much memory would it require to process the user request. It totally depends on the user-specific operations and concurrent requests. Generally, it is a good idea to start with 8GB of the max heap and further increase it if required.

Attachments