Logins not possible; slowness; JWP: U00003623 Detected possible intrusion from 'null'.

Products

• CA Automic Workload Automation - Automation Engine
• CA Automic One Automation

Issue/Introduction

At some point there is a lot of slowness and new logins to the system are not possible.  The JWPs have a lot of errors like the following:

U00003623 Detected possible intrusion from 'null'. The Automation Engine received a message without valid login. A trace of the entire message will be written to the Server log.

There are a small amount of JWPs (one or two per AE server).

Environment

AE: 21.0, 24.X
JWPs: 1 or 2 per server

Cause

JWPs are busy with process monitoring queries and cannot work on IDX (search index) or AUT (authorization) role functions

Resolution

We see this sometimes when there are only one or two JWPs running per server in a fairly large, active system.  The JWPs become locked in Process Monitoring queries and then can't follow through with their authorization (AUT) and Index (IDX) roles and the system locks up with no one being to log in and there being performance problems.

The way to prevent this from happening and an absolute best practice is to start 4 JWPs per server in a system with a lot of activities.  Two of these JWPs per server should have explicit roles set on startup: one as -role=AUT and one as -role=IDX.  

This can be implemented on a system where JWPs are started by service manager by opening the service manager dialog and for each server running JWPs and following the below steps:

1. Duplicate the current JWP entry four times - rename these as you'd like
2. Open one of the duplicated JWP entries and update the start command to include the parameter -role=IDX before the -svc%port% like the following:
   
   java -Xrs -Xmx2G -jar ucsrvjp.jar -IC:\automic\AutomationEngine\bin\ucsrv.ini -role=IDX -svc%port%
   
   
3. Open another of the duplicated JWP entries and update the start command to include the parameter -role=AUT before the -svc%port% like the following:
   
   java -Xrs -Xmx2G -jar ucsrvjp.jar -IC:\automic\AutomationEngine\bin\ucsrv.ini -role=AUT -svc%port%
   
   
4. The other 2 JWPs do not have to have a specified role (although this can be done if needed)

More information on setting roles can be found here: JWP Roles