Slow AWI performance on High Availability configuration

Products

• CA Automic Workload Automation - Automation Engine

Issue/Introduction

Intermittent slow or unresponsive Automic Web Interface (AWI) performance in all non-client 0 clients, installed on 3 load-balancing Tomcat servers.
Log in to client 0 is quick, but slow or unresponsive when logging into client X.

Performance is not slow when accessing from the Java UI or when configured without High Availability (HA).

Error:
java.lang.RuntimeException: This took too long.

Environment

AE version 12.0.4+build.1483 (active-active)
AWI version 12.0.1 HF01-362 
Amazon Web Services (AWS)
Three Tomcat 8 servers
Oracle Java 1.8.0_121-b13

Cause

It is discovered that sizing of the tomcat servers is too small - had a lot of ‘out of memory’ exceptions.

Resolution

All three Tomcat servers are rehydrated to larger EC2 images.

If issue persists, enable tracing:
1. Modify your uc4config.xml for AE Client Trace:
-- Located at <Apache Tomcat Installation>/webapps/<ECC Name>/config/uc4config.xml

Change the line reading:

<trace count="10" xml="0"></trace>
to
<trace count="10" xml="3"></trace>

2. Modify your logback.xml for Tomcat Trace:
-- Located at <Apache Tomcat Installation>/webapps/<ECC Name>/config/logback.xml

Change the line reading:

<root level="INFO">
to
<root level="TRACE">

3. Backup your current Tomcat log directory (<Apache Tomcat Installation>/logs) in case the error is not easily reproducible.

4. Remove contents of log directory.

5. Restart the Tomcat Server.

6. Enable trace on WP (TCP/IP=2, DB=4) via the UI System Overview.

7. Replicate the issue.

8. Stop the Tomcat Server.

9. Change the values back to original settings in steps 1, 2, and 6.

10. Zip the contents of the Tomcat log directory, WP logs, WP trace files, and send to CA Automic Support.

11. Restart the Tomcat Server.

 

Additional Information

Workaround :
1. Install a supported version of Oracle Java (not Openjdk Java), and have both root and service accounts use the same supported version.
2. Ensure the AWI is on the same version as the AWA.
3. Create workflows to enable users to restart JWPs, if needed.