Spectrum Webapp client OutOfMemory heap errors in tomcat log.

Products

• Spectrum
• Network Observability

Issue/Introduction

In the webTomcat catalina.out we see the following errors:

[oneclickwebapp_anonym_2938b5c942b2_1646378038150] Exception in thread "AWT-EventQueue-0" java.lang.OutOfMemoryError: Java heap space

2022-03-04 09:01:55,055 INFO  [Webswing Process Handler] (SwingProcessImpl.java:384) [oneclickwebapp_anonym_2938b5c942b2_1646378038150] WARNING: Application is not responding for 10 seconds. Thread dump generated . 

Environment

DX NetOps Release - Spectrum 21.2 / 22.2.x/23.2.x/24.3.x

Cause

The default WebApp client memory is set at 1GB.  Larger OneClick environments will have a higher demand on webApp client memory.  

In a OneClick environment with over 10,000 active alarms, we see a bases client memory usage of 70% in the GUI.  It can easily spike to greater than 80% which causes the application to dump Java heaps and register errors to the tomcat log, without any performance issue seen by the client.

Resolution

For OneClicks with up to 1,500 active alarms, the default should be sufficient.  For larger environments, we recommend increasing the maximum webapp client memory to a larger value, such as 2gb and up to 4gb as a maximum value depending on the number of active alarms present.  We would first recommend upgrading from 1gb to 2gb and see if any issues are noted before upgrading to 4gb. Allocating too much xmx ram can cause further issues in Linux as the full amount of memory can be reserved which can cause out of memory issues on other applications that need system ram.

To increase the memory, change the setting in webswing.config under:

$SPECROOT/webtomcat/webswing/

Find the line:

    "vmArgs" : "-Xmx1024m -Djavaws.cfg.jauthenticator=true -DbrowserIp=${clientIp} -Dsun.awt.noerasebackground=true -Duser.timezone=${clientTimeZone} -Djavax.net.ssl.trustStore=${SPECROOT}/custom/keystore/cacerts -Djavax.net.ssl.trustStorePassword=changeit",

Change the Xmx setting from 1024m to 4096m:

    "vmArgs" : "-Xmx4096m -Djavaws.cfg.jauthenticator=true -DbrowserIp=${clientIp} -Dsun.awt.noerasebackground=true -Duser.timezone=${clientTimeZone} -Djavax.net.ssl.trustStore=${SPECROOT}/custom/keystore/cacerts -Djavax.net.ssl.trustStorePassword=changeit",

Then restart webTomcat:

cd $SPECROOT/webtomcat/bin

./startWebTomcat.sh

Additional Information

TechDocs : DX NetOps 24.3 Spectrum : Memory Requirements for OneClick WebApp Web Server