search cancel

DEV Scheduler failed to connect to Oracle server

book

Article ID: 257226

calendar_today

Updated On:

Products

CA Workload Automation AE

Issue/Introduction

Please review the tar file and help me determine why the scheduler keeps rolling over to shadow.

From the log provide we can see the Scheduler starts having issue communicating with the database around 12/30/2022 00:49:56

Line   952: [12/30/2022 00:49:56]      CAUAJM_E_18400 An error has occurred while interfacing with ORACLE.
    Line   953: [12/30/2022 00:49:56]      CAUAJM_E_18401 Function <doExecute> invoked from <execute> failed <902>
    Line   966: [12/30/2022 00:50:30]      CAUAJM_E_18416 Event Server: <AUTOSYSC>  Failed Query: <UPDATE ujo_alamode SET int_val=28800 WHERE type='gmt_offset'>
    Line   967: [12/30/2022 00:50:30]      CAUAJM_E_18412 The database client has been interrupted while query execution is in progress.
    Line   968: [12/30/2022 00:50:30]      CAUAJM_E_18400 An error has occurred while interfacing with ORACLE.
    Line   969: [12/30/2022 00:50:30]      CAUAJM_E_18401 Function <doExecute> invoked from <execute> failed <902>
    

We can see that we were able to retry the queries and they would succeed
    Line   997: [12/30/2022 00:51:22]      CAUAJM_I_10165 The previously failed query below succeeded after 2 attempt(s):
    Line   999: [12/30/2022 00:51:22]      CAUAJM_I_10165 The previously failed query below succeeded after 2 attempt(s):
    Line  1001: [12/30/2022 00:51:22]      CAUAJM_I_10165 The previously failed query below succeeded after 2 attempt(s):
    Line  1003: [12/30/2022 00:51:22]      CAUAJM_I_10165 The previously failed query below succeeded after 2 attempt(s):

This went of for several hours and would appear to be a performance issue on your back-end SQL server or a network issue during this time.

The slowdown seems to have resolved after about 12/30/2022 02:52:28]


We then can see a fail over happen at [12/30/2022 22:25:55 when the database can not longer be reached.
    Line 26808: [12/30/2022 22:25:55]      CAUAJM_E_18407 Failed to connect to Oracle server: AUTOSYSC
    Line 26810: [12/30/2022 22:25:55]      CAUAJM_E_18401 Function <Olog> invoked from <openProc> failed <162>
    Line 26811: [12/30/2022 22:25:55]      CAUAJM_E_18402 ORA-12545: Connect failed because target host or object does not exist
    Line 26817: [12/30/2022 22:28:54]      CAUAJM_W_40328 System appears to be in failover state. Checking for shadow activity.
    Line 26819: [12/30/2022 22:28:55]      CAUAJM_I_00202 State         : Failed-over
    Line 26824: [12/30/2022 22:28:55]      CAUAJM_E_00132 Database indicates failover status. Shutting down.
    Line 26828: [12/30/2022 22:28:55]      CAUAJM_E_40187 ERROR: Failed to retrieve job from RESWAIT_QUE

 

Environment

Autosys  12.0

Resolution

We then can see a fail over happen at [12/30/2022 22:25:55 when the database can not longer be reached.
    Line 26808: [12/30/2022 22:25:55]      CAUAJM_E_18407 Failed to connect to Oracle server: AUTOSYSC
    Line 26810: [12/30/2022 22:25:55]      CAUAJM_E_18401 Function <Olog> invoked from <openProc> failed <162>
    Line 26811: [12/30/2022 22:25:55]      CAUAJM_E_18402 ORA-12545: Connect failed because target host or object does not exist
    Line 26817: [12/30/2022 22:28:54]      CAUAJM_W_40328 System appears to be in failover state. Checking for shadow activity.
    Line 26819: [12/30/2022 22:28:55]      CAUAJM_I_00202 State         : Failed-over
    Line 26824: [12/30/2022 22:28:55]      CAUAJM_E_00132 Database indicates failover status. Shutting down.
    Line 26828: [12/30/2022 22:28:55]      CAUAJM_E_40187 ERROR: Failed to retrieve job from RESWAIT_QUE

You will need to check with your DBA on the Oracle error ORA-12545 error and have them provide a reason the Oracle client returned this.