DST: Jobs executing twice at turnover or at incorrect time

Products

• CA Automic Workload Automation - Automation Engine
• AUTOMIC WORKLOAD AUTOMATION

Issue/Introduction

Some of the jobs contained within Schedules (JSCH) are executed twice or did not restart at all after period turnaround during Daylight Savings Time (DST). Possible symptoms include:

1. Schedule object refreshed over DST at 11 PM instead of 12 PM. 

2. After refreshing, the Schedule executed jobs twice for the previous day, even though they successfully completed on time.

3. Some Job objects were not executed. 

4. Some Schedule objects did not refresh at all.

Note: All information within this article relates specifically to the switchover to DST from Standard time (occurring in Spring).

Cause

Important Note: This issue affects individual timezone objects - if all of your timezones have passed DST, you will have no further effect at this time.

A bug in the routine that is calculating the next period turnaround of Schedule Objects causes multiple or no executions of jobs after the day light saving switch. Any JSCH object in any system, independent of the system's local time zone can be affected. A Schedule Object is affected in case it has a time zone object defined that refers to a time zone in which a daylight saving switch is defined. Schedules with no time zone defined will fall back to the client's time zone. In summary:

If a schedule object has no timezone defined AND is in a client with no timezone defined, it will not be affected.
If a schedule object has a timezone defined, it will be affected.
If a schedule object has no timezone defined, but there is a timezone on the client, that schedule will be affected

 

Environment

The issue affects the following service packs per release: 

Automation.Engine 11.2.9 and 11.2.10
Automation.Engine 12.0.6 and 12.0.7
Automation.Engine 12.1.3 and 12.1.4
Automation.Engine 12.2.1 and 12.2.2

Please note that systems running on earlier service packs are not affected, so for instance v12.0.6 is affected but AE 12.0.0 thru v12.0.5 are not affected.

Resolution

Workaround:

If affected, restarting the schedules will resolve the issue for impacted schedules. Additionally, a script has been created to identify affected JSCH objects. The script will also provide update statements (one per object) to fix the turnaround time, which will correct the situation.  This script was updated 29 March 2019 to cover additional situations.

In order to be able to import the objects, you need to have SQLVAR_INTERNAL set to YES in the UC_SYSTEM_SETTINGS.

The script needs to run after the DST and before the next period turnarround of the the JSCH.

An e-mail address can be defined in the top of the script to have a notification sent out in case faulty JSCH objects are found.

In case the Script identifies affected JSCH Objects, please execute the provided 'update' statement to fix it.

The script will not perform the update statement automatically for safety reasons.

You can easily grab the entire report and strip the timestamp at the beginning of the lines using a text editor to execute all statements with the tool of your choice.

Please don't forget to perform a 'commit' at the end, in cases the database used requires it.

A definitive fix is being worked on currently.

An example use of the script:

1.) DST is on Sunday 31st of March 02:00 am, period turnaround is at 00:00 

2.) You need to wait until 3:00 am, so the DST should already happened

3.) Between the DST and the next period turnaround of your JSCH, you need to execute the script and execute the provided update statements.

Fix:

We have identified this issue internally, and are working on a fix which will become available in the next releases for version 12.x (and the latest version 11.x), tentatively scheduled for release on the below dates: 

Automation.Engine 12.2.2 HF2 - Available

Automation.Engine 12.1.4 HF2 - Available

Automation.Engine 12.0.7 HF1 - Available

Automation.Engine 11.2.10 HF1 - Available

Automation.Engine 12.0.8 - Available

Automation.Engine 12.1.5 - Available

Automation.Engine 12.2.3 - Available

Automation.Engine 12.3.0 - Available 

Additional Information

Daylight saving switches occur on several dates from early March to mid April depending on the country. The country the system is physically located doesn't affect the vulnerability. It's the setup of the TZ Object in Automic Automation.Engine only, which determines if the bug will occur or not.

An overview of countries with DST switch and when they are due can be found here:

http://www.webexhibits.org/daylightsaving/g.html
and here
https://www.timeanddate.com/time/dst/2019.html