Jobs remain in QUEUED status and "ORA-00001: unique constraint SO_JOB_QUEUE_IDX violated" error displayed

book

Article ID: 84882

calendar_today

Updated On:

Products

CA Automic Applications Manager (AM)

Issue/Introduction

Error Message :
ErrorMsg: AwE-5001 Database Query Error
ava.sql.SQLException: ORA-00001: unique constraint (AWPD.SO_JOB_QUEUE_IDX) violated

Agents initially not connecting and a “java.net.ConnectException: Connection refused” error is displayed.  The Applications Manager (AM) processes were stopped and restarted including both the Master and Remote Agent processes.  After all the Remote Agent connections were restored all newly launched jobs remain in a "QUEUED" status.

Investigation

The following error messages can be seen in the $AW_HOME/log/RmiServer log.
 
08:37:06.617 Timer-3: .AxOptions: NoErrorMsgProperties=false
ErrorMsg: AwE-5001 Database Query Error (12/21/16 8:37 AM)
Details: aw5.process_state
0 name: IN:VARCHAR2:java.lang.String:PRODCORP
1 progname: IN:VARCHAR2:java.lang.String:RMIMASTER
2 sessionId: IN:NUMERIC:java.math.BigDecimal:9750571
3 command: IN:OUT:VARCHAR2:java.lang.String:HEARTBEAT
4 cpuUsage: IN:OUT:NUMERIC:java.lang.Integer:0
jdbc:oracle:thin:@dc1rh606.clubcorp.com:1551:POPS:AWPD
java.sql.SQLException: ORA-00001: unique constraint (AWPD.SO_JOB_QUEUE_IDX) violated

We are still researching the cause of the unique constraint violation. If you should face this issue please contact Automic Support for assistance.
 
 

Cause

Cause type:
Other
Root Cause: The cause of this unique constraint on jobid is still under investigation.

Environment

Release: AAMOS499000-8.0-Automic Applications Manager-OS400 Agent
Component:

Resolution

The following information may be beneficial to further troubleshoot the issue. Please ensure the following table backups are kept from the time when the issue occurs.
 
1. Backup of the so_job_queue table
2. Backup of the aw_job_queue_activity table
3. Backup of the so_job_history


Fix Status: No Fix

Fix Version(s):
N/A

Additional Information

Workaround :
Please contact Automic Support for assistance.