Forecast stays in preparing and fails with ForecastHeader not found

Products

• CA Automic Workload Automation - Automation Engine

Issue/Introduction

Autoforecast is prone to hang in AE version 21.0

These errors show up in the JWP log file:

20231114/125314.050 - 40     U00045014 Exception 'com.automic.database.api.DBException: "UPDATE FH SET FH_Count = FH_Count - 1 WHERE FH_Idnr = ?"' at 'com.automic.database.impl.DBConnectionImpl.executeInternal():573'.
20231114/125314.052 - 40     U00045015 The previous error was caused by 'org.postgresql.util.PSQLException: "ERROR: could not serialize access due to concurrent update"' at 'org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse():2497'.
20231114/125314.055 - 40     U00045362 ForecastHeader with ID '1015011' not found.
20231114/125314.055 - 40     U00003434 Server routine  'ForecastMessageHandler/JFCST' required '0' minutes and '12' seconds for processing.

In the trace of the same JWP:

20231114/125313.972 - 40     INSERT INTO FE (FE_AH_IDNR,FE_CHILDCNT,FE_COUNT,FE_EndTime,FE_ERT,FE_FCSTSTATUS,FE_FH_IDNR,FE_FileNameDst,FE_FileNameSrc,FE_HostAttrTypeDst,FE_HostDst,FE_HostSrc,FE_IDNR,FE_LoginDst,FE_LoginSrc,FE_MAXPARALLEL,FE_MAXPARALLELHG,FE_MRT,FE_MRTERT,FE_MRTMETHOD,FE_Name,FE_OCCURCOUNT,FE_Otype,FE_PARENTPRC,FE_RealEndTime,FE_RealStartTime,FE_REALSTATUS,FE_REUSEHOSTGROUP,FE_SIGNAL,FE_SRT,FE_SRTERT,FE_StartTime,FE_STATUS,FE_TASKCOUNT,FE_UC4PRIORITY) VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)
20231114/125313.975 - 40     UCUDB32 INBK RET 0000 HSTMT: 0000000000000000 VALUE: 0000000000000001 ALL:  0.00000 DB:  0.00000 ODBC:  0.00000 UDB:  0.00000
20231114/125314.003 - 40     UCUDB32 EXEC RET 0000 HSTMT: 0000000000000000 VALUE: 0000000000000285 ALL:  0.02700 DB:  0.00000 ODBC:  0.00000 UDB:  0.00000
20231114/125314.006 - 40               STRT com.automic.forecast.ForecastMessageHandler$$Lambda$241/0x0000000840629040
20231114/125314.009 - 40     U00009909 TRACE:(BINDPAR:  1) size 000004
20231114/125314.010 - 40               00000000  000F7CE3                             >..|.<
20231114/125314.010 - 40                                            ><RUNID><
20231114/125314.012 - 40     UPDATE FH SET FH_Count = FH_Count - 1 WHERE FH_Idnr = ?
20231114/125314.043 - 40     UCUDB32 EXEC RET 0001 HSTMT: 0000000000000000 VALUE: 0000000000000000 ALL:  0.03000 DB:  0.00000 ODBC:  0.00000 UDB:  0.00000
20231114/125314.045 - 40     UCUDB32 CLST RET 0000 HSTMT: 0000000000000000 VALUE: 0000000000000000
20231114/125314.045 - 40               EXIT com.automic.forecast.ForecastMessageHandler$$Lambda$241/0x0000000840629040 TIME: 36ms
20231114/125314.047 - 40     UCUDB32 RBCK RET 0000 HSTMT: 0000000000000000 VALUE: 0000000000000000 ALL:  0.00000 DB:  0.00000 ODBC:  0.00000 UDB:  0.00000
20231114/125314.051 - 40     U00045014 Exception 'com.automic.database.api.DBException: "UPDATE FH SET FH_Count = FH_Count - 1 WHERE FH_Idnr = ?"' at 'com.automic.database.impl.DBConnectionImpl.executeInternal():573'.
20231114/125314.053 - 40     U00045015 The previous error was caused by 'org.postgresql.util.PSQLException: "ERROR: could not serialize access due to concurrent update"' at 'org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse():2497'.
20231114/125314.055 - 40     U00045362 ForecastHeader with ID '1015011' not found.
20231114/125314.056 - 40     U00003434 Server routine  'ForecastMessageHandler/JFCST' required '0' minutes and '12' seconds for processing.
20231114/125314.057 - 40               EXIT ForecastMessageHandler TIME: 12221ms
20231114/125314.062 - 40     UCUDB32 CMIT RET 0000 HSTMT: 0000000000000000 VALUE: 0000000000000000 ALL:  0.00400 DB:  0.00000 ODBC:  0.00000 UDB:  0.00000
20231114/125314.062 - 40               Record found -> read again.

Environment

Automation Engine 21.0.5 with Postgres 15

Cause

This is a defect of the Automation Engine.

Resolution

Solution:

Update to a fix version listed below or a newer version if available.

Fix version:

Component(s): Automation Engine

Automation.Engine 21.0.12 - Available

Automation.Engine 24.3.0 - Planned release November/December 2024

Additional Information

The reference for the bug is DE128494