AutoSys Job Definitions stop updating in AAI -com.termalabs.common.datastore.DataAccessException: Error while saving instance: JobProperty[457="s(<jobName>)"]

Products

• Automation Analytics & Intelligence

Issue/Introduction

Autosys job definitions have stopped updating in AAI.

 

When reviewing the AAI jaws.log the error below can be seen when the ConverterService and/or DOWNLOAD_SCHEDULER process finishes:

WARN  [SchedulerHelper] Error while writing updates for scheduler "<AutoSys Scheduler Name>"

com.termalabs.server.domain.api.SchedulerUpdate$UpdateFailedException: SchedulerUpdate failed in the PRE_UPDATE phase

at deployment.aai.war/com.termalabs.server.cache.scheduler.SchedulerUpdateImpl.commit(SchedulerUpdateImpl.java:344) ~[jaws-server-base-24.3.0-2.jar:?]

at deployment.aai.war/com.termalabs.server.cache.scheduler.SchedulerUpdateImpl.commit(SchedulerUpdateImpl.java:307) ~[jaws-server-base-24.3.0-2.jar:?]

at deployment.aai.war/com.termalabs.server.app.scheduler.SchedulerHelper.convertScheduler(SchedulerHelper.java:290) ~[jaws-server-base-24.3.0-2.jar:?]

at deployment.aai.war/com.termalabs.server.app.scheduler.SchedulerHelper.downloadScheduler(SchedulerHelper.java:242) ~[jaws-server-base-24.3.0-2.jar:?]

at deployment.aai.war/com.termalabs.server.app.scheduler.SchedulerHelper.downloadNow(SchedulerHelper.java:190) ~[jaws-server-base-24.3.0-2.jar:?]

at deployment.aai.war/com.termalabs.server.restapi.service.SchedulersService.lambda$getSchedulerUpdate$5(SchedulersService.java:954) ~[jaws-server-rest-24.3.0-2.jar:?]

at deployment.aai.war/io.smallrye.context.impl.wrappers.SlowContextualRunnable.run(SlowContextualRunnable.java:19) [smallrye-context-propagation-2.0.0.jar:?]

at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) [?:?]

at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]

at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]

at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]

at java.base/java.lang.Thread.run(Thread.java:840) [?:?]

Caused by: com.termalabs.common.datastore.DataAccessException: Error while saving instance: JobProperty[457="s(<jobName>)"]

at deployment.aai.war/com.termalabs.server.hibernate.jaws.JawsSessionImpl.update(JawsSessionImpl.java:257) ~[jaws-server-base-24.3.0-2.jar:?]

at deployment.aai.war/com.termalabs.server.hibernate.jaws.HibernateJawsJobData.updateJobProperty(HibernateJawsJobData.java:489) ~[jaws-server-base-24.3.0-2.jar:?]

at deployment.aai.war/com.termalabs.server.cache.scheduler.ContextDatabaseHelper.doJobPropertyChanges(ContextDatabaseHelper.java:654) ~[jaws-server-base-24.3.0-2.jar:?]

at deployment.aai.war/com.termalabs.server.cache.scheduler.ContextDatabaseHelper.commitChanges(ContextDatabaseHelper.java:310) ~[jaws-server-base-24.3.0-2.jar:?]

at deployment.aai.war/com.termalabs.server.cache.scheduler.SchedulerUpdateImpl.saveObjects(SchedulerUpdateImpl.java:590) ~[jaws-server-base-24.3.0-2.jar:?]

at deployment.aai.war/com.termalabs.server.cache.scheduler.SchedulerUpdateImpl.commit(SchedulerUpdateImpl.java:334) ~[jaws-server-base-24.3.0-2.jar:?]

... 11 more

Caused by: org.hibernate.NonUniqueObjectException: A different object with the same identifier value was already associated with the session: [com.termalabs.common.hibernate.jaws.JobProperty#com.termalabs.common.hibernate.jaws.JobPropertyPK@f584bf20]

at deployment.aai.war/org.hibernate.engine.internal.StatefulPersistenceContext.checkUniqueness(StatefulPersistenceContext.java:843) ~[hibernate-core-6.6.13.Final.jar:6.6.13.Final]

Resolution

The error makes note of AAI trying to update a Job Property below:

com.termalabs.common.datastore.DataAccessException: Error while saving instance: JobProperty[457="s(<jobName>)"]

 

 

The example above is trying to update an AutoSys success condition, however the logging doesn't show which job is failing to be updated.

 

1. In order to find the problematic job(s), you must enable two DEBUG loggers in the AAI Configuration Tool

• ContextChecker
• ContextDatabaseHelper

 

2. Once the DEBUG loggers are set, go to the affected scheduler, and click the "Update" button

 

3. The logging for the download process when clicking the Update button for the scheduler will start and end with the DOWNLOAD_SCHEDULER message

 

4. Search for the string with the job condition in the error above, for example "s(<jobName>)". in the jaws.log after the DEBUG logging starts and you should be able to see the job or jobs that are trying to have this job condition added. 

 

5. If you have access to Autosys find those jobs in Autosys and if possible temporarily delete the jobs and then force a new Update of the scheduler in AAI.

 

6. If the error is still occurring please gather all logs and contact Broadcom Support