JCP queues flooded with messages

book

Article ID: 210827

calendar_today

Updated On:

Products

CA Automic Workload Automation - Automation Engine

Issue/Introduction

During high load it can happen that the JCP queue gets flooded with messages (10.000+, up to 1 million)

In some occasions this may lead the Primary Work Process stopping to process even after the JCP is restarted.

Investigation

When the issue happens run the following query (Oracle only):

select count(*), utl_raw.cast_to_varchar2(dbms_lob.substr(MQCP_Msg, 200, 3)) as msg, mqcp_priority from mqXcpYYY
group by utl_raw.cast_to_varchar2(dbms_lob.substr(MQCP_Msg, 200, 3)), mqcp_priority order by 1 desc

where mqXcpYYY has to be adapted to the system:

  • X is the current MQSET (1|2). The MQSET can be retrieved with "select ZDU_ACTIVE from UC_ZDU"
  • YYY is the number of the JCP
If your system is suffering from the issue the query will return the following:
COUNT(*) MSG MQCP_PRIORITY
10.000+ <uc-env session=""> <kick system="UC4" client="client#" idnr="^0?">EH^ ? </kick></uc-env> 200
10.000+ <uc-env session=""> <kick system="UC4" client="other_client#" idnr="^0?">EH^ ? </kick> </uc-env> 200
... ... 200
30+ <uc-env session=""> <asyncmsg> <message date="yyyy-mm-ddThh:mm:ss" nr="xxxxxxxx" insert=" message"^ ?Ü/> </asyncmsg> </uc-env> 200
30+ <uc-env session=""> <asyncmsg> <message date="yyyy-mm-ddThh:mm:ss" nr="xxxxxxxx" insert=" message"^ ?Ü/> </asyncmsg> </uc-env> 200
...   200

 

Cause

This is a bug

Environment

Release : 12.2 / 12.3

Component : AUTOMATION ENGINE

Resolution

Workaround:

Restarting the JCP will empty its MQxCPyyy table.

If the MQxCPyy table contains many millions of entries it may be advantages to stop the JCP and truncate the corresponding MQxCPyyy table before restarting.

 

Solution:

Update to a fix version listed below or a newer version if available.


Fix version:

Component(s): Automation Engine

Automation.Engine 12.2.9 - Available

Automation.Engine 12.3.5 HF4 - Available

Automation.Engine 12.3.6 - Available