AE not processing with all WP's at 100% and long running DB calls on CHK_HG

Products

• CA Automic Workload Automation - Automation Engine

Issue/Introduction

Error Message :
N/A

The Automation Engine system performance slows down when processing several Hostgroup (HOSTG) objects. 

Investigation
Long running DB critical calls on 'Waiting for the host of an agent group' cause the WPs to be at 100% utilization.

A review of the WP logs contain the following: 

U00003524 UCUDB: ===> Time critical DB call! OPC: 'SLCU' time: '344:311.628.999' 
U00029131 UCUDB - List of bind parameters: 
U00029132 UCUDB: 'EH_Status = 1688' 
U00003525 UCUDB: ===> 'SELECT * FROM EH WHERE EH_Status = ? ORDER BY EH_QueuePriority, EH_UC4Priority, EH_ComplRate DESC, EH_FiFoNr' 

Cause

Cause type:
Defect
Root Cause: The checkwait (CHCKWAIT) for an agent is sending a CHK_HG for all agent groups. If 100 agents are started at the same time, all will serialize on the SELECT for UPDATE on EH with 1688.

Environment

Release: AUTWAB99000-12.0-Automic Workload Automation-Base Edition
Component:

Resolution

Update to a fix version listed below or a newer version, if available, where the performance has been improved with processing agents in HOSTG objects.

Fix Status: Fixed

Fix Version(s):
Component(s): AE Server and Initialdata

Automation Engine 12.2.0 - Available
Automation Engine 12.1.1 - Available
Automation Engine 12.0.4 - Available

Additional Information

Workaround :
1. Have the DBA run the SELECT statement against the database directly: 

'SELECT * FROM EH WHERE EH_Status = 1688 ORDER BY EH_QueuePriority, EH_UC4Priority, EH_ComplRate DESC, EH_FiFoNr' 

2. Have the DBA kill the job.