Slowness in Backlog Job status changes

Products

• CA Automic Applications Manager (AM)

Issue/Introduction

After upgrading to Applications Manager(AM) 9.6 or 9.6.1, there are intermittent or consistent slowness Job status changes in the Backlog. 

This includes, but not limited to, changes from Queued to Running, Finished to dropping into History, and time between the finishing of one component to the starting of another component in a Process Flow.

A review of the RmiServer logs during the time of issue show random slowness in the return times for the procedure aw_submit_chains. Example in bold below:

13:59:32.218 Timer-5: .DBAccess: getProcedureData():  sID-12649272 aw5.submit_chains elapsed ms 20
14:04:08.149 Timer-5: .DBAccess: getProcedureData():  sID-12649301 aw5.submit_chains elapsed ms 248149
14:06:05.879 Timer-5: .DBAccess: getProcedureData():  sID-12649301 aw5.submit_chains elapsed ms 117729
14:09:03.752 Timer-5: .DBAccess: getProcedureData():  sID-12649301 aw5.submit_chains elapsed ms 177873
14:09:50.658 Timer-5: .DBAccess: getProcedureData():  sID-12649300 aw5.submit_chains elapsed ms 46906
14:09:50.705 Timer-5: .DBAccess: getProcedureData():  sID-12649300 aw5.submit_chains elapsed ms 46 

Environment

Applications Manager version 9.6.0 to 9.6.2

Cause

For the first time, Applications Manager 9.6 and above moves from using Oracle dbms_pipes (in memory) to Oracle Advanced Queues (physical tables). While Oracle AQ is more robust and persistent than dbms_pipe, improvements to AM code is required to make the best use of the change from dbms_pipe to AQ. 

Resolution

Solution:

A fix will be included in Applications Manager version 9.6.3 and 10.0

Workaround for 9.6 to 9.6.2:

Fixes can be applied by running two sql scripts that are attached to this article, the am_aq_fix_20260110.sql and aw_operation_body_fix_20260110.sql.

1. Download attached am_aq_fix.zip
2. Extract am_aq_fix_20260110.sql and aw_operation_body_fix_20260110.sql from zip and move to the master's $AW_HOME/sql directory
3. Stop Applications Manager processes
4. From the $AW_HOME/sql directory, run a sqlplus command and connect to the AM database as the AM Oracle User
5. Run the below commands from the sqlplus command line and exit:
   
   SQL> @am_aq_fix_20260110.sql
   SQL> @aw_operation_body_fix_20260110.sql
   SQL> exec am_aq.drop_and_create_all_am_queue_tables
   
   
6. Restart Applications Manager processes

If changes need to be reverted:

1. From the same sql directory, follow the same steps replacing the commands with the below commands:
   
   SQL> @am_aq.sql
   SQL> @aw_operation_body.sql
   SQL> exec am_aq.drop_and_create_all_am_queue_tables