Increase Derby Memory For WCC
Article ID: 219547
Updated On:
Products
Issue/Introduction
We are getting autosys console unable to accessible issues very frequently on everyday.
Issues:
1. Console down
2. Unable to access and monitor the batch jobs.
Derby database is loosing connectivity with tomcat due to out of memory
INFO | jvm 2 | 10:05:14 | Exception in thread "DRDAConnThread_15" java.lang.OutOfMemoryError: Java heap space
INFO | jvm 2 | 10:05:14 | WrapperManager Error: java.lang.OutOfMemoryError: Java heap space
INFO | jvm 4 | 10:06:52 | Exception in thread "DRDAConnThread_13" java.lang.OutOfMemoryError: GC overhead limit exceeded
INFO | jvm 4 | 10:06:59 | Exception in thread "DRDAConnThread_66" java.lang.OutOfMemoryError: GC overhead limit exceeded
Environment
Release : 11.3
Component : WORKLOAD CONTROL CENTER
Cause
From Logs:
Tomcat lost connectivity to database
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
INFO | jvm 1 | | 167094 | at java.lang.Thread.run(Thread.java:745)
INFO | jvm 1 | | 167094 | Caused by: org.apache.derby.client.am.DisconnectException: java.net.ConnectException : Error connecting to server localhost on port 1527 with message Connection refused: connect.
Tomcat Went out memory:
INFO | jvm 2 || Exception in thread "DRDAConnThread_15" java.lang.OutOfMemoryError: Java heap space
INFO | jvm 2 | | WrapperManager Error: java.lang.OutOfMemoryError: Java heap space
INFO | jvm 4 | | Exception in thread "DRDAConnThread_13" java.lang.OutOfMemoryError: GC overhead limit exceeded
INFO | jvm 4 || Exception in thread "DRDAConnThread_66" java.lang.OutOfMemoryError: GC overhead limit exceeded
-
-
Line 1437: INFO | jvm 7 | | java.lang.OutOfMemoryError: GC overhead limit exceeded
Line 1444: INFO | jvm 7 | | java.lang.OutOfMemoryError: GC overhead limit exceeded
Line 1461: INFO | jvm 1 | | WrapperManager Error: java.lang.OutOfMemoryError: GC overhead limit exceeded
Line 1473: INFO | jvm 2 | | Exception in thread "NetworkServerThread_2" java.lang.OutOfMemoryError: GC overhead limit exceeded
Resolution
Refer the consolidated steps to increase derby memory only
==> //Stop Services//
Then stop all of them with:
unisrvcntr stop CA-wcc
unisrvcntr stop CA-wcc-services
unisrvcntr stop CA-wcc-db
==> //Take Back Up//
"1. Export all views, filters, alert policies using the following command:
Windows: wcc_monitor.bat -u username -p password -x monitor_filename
Linux: ./wcc_monitor.sh -u username -p password -x monitor_filename
2. Export all configurations (including AE definitions) using the following command:
Windows: wcc_config.bat -u user_name -p password -x config_filename
Linux: ./wcc_config.sh -u user_name -p password -x config_filename"
3. Please take backup of DB_FiILES (zip/tar it) as it is Windows/Linux.
==> //Increase memory//
Increase Derby memory in below file
Unix Path: $CA_WCC_LOCATION/derby/conf/wrapper.conf
Windows Path: %CA_WCC_LOCATION%/derby/conf/wrapper.conf
# Java Application
wrapper.java.command=../../jre/bin/java
#wrapper.java.initmemory=64
wrapper.java.initmemory=1024
#wrapper.java.maxmemory=512
wrapper.java.maxmemory=3072
(Later on you can keep these values to improve Derby performances)
Start Derby with:
unisrvcntr start CA-wcc-db
Executed CA-wcc-db start.........................................OK
Start Other Services.
Note: If for whatever reason there is a problem during this compress operation, rename '$CA_WCC_LOCATION/derby/DB_FILES ' and restore the tar file taken at the beginning of this procedure.
Additional Information
Feedback
