iDash Job Collection questions for AUTOSYS Jobs

Products

• iDash Workload Automation

Issue/Introduction

Would like to know on what basis iDash collects job data, it seems somewhat random. Have 1087 SLA and data is collected for those. However have many other jobs in Autosys for jobs that do not have SLA's and for some of those jobs there is SLA data. 

For example, we have an app code for jobs au_cba_hado% and there is job data for some jobs in the iDash database, but not all of them. I am wondering why this is? 

What basis does iDash Autosys collect job data into the iDash DB. Is is it if SLA's, OR reports, or some other. 

On reading the manual / techdoc content and the sql script, it seems that a table index xak_idash_ujo_proc_event is created on the Autosys Database, and not on the iDash database. Please confirm if this is correct.

From the techdoc: 

b) Run aedb_index_for_import.sql.
This will create an index in the AutoSys database if the auto import feature in iDash is enabled and an existing index isn't already in the database.

Have checked the Autosys database for the index xak_idash_ujo_proc_event and it does not exist, so seems auto import is not enabled. 

The manual / techdoc entry is a bit ambiguous. How can i determine if the auto import feature is enabled? I don't see anything in the iDash GUI for it? 

Please confirm if my understanding is correct.

1. If I enable auto import, job run data will be automatically imported to the iDash database for all jobs, regardless of whether an SLA is defined or not. 
2. with auto import enabled in iDash, when I run an iDash job run report, or job status report, the data will be sourced only from the iDash database.
3. Job run and status data will be stored on the iDash database for all jobs, based on the iDash archive retention periods
4. Job run and status data will be available on iDash database (iDash data retention is 365 days) for longer than the 7 days which apply to the Autosys (database archive events)
5. job data is automatically imported at some time frequency. Is this daily ? Is it configurable and how ? 
6. how do I enable auto import in iDash?  

 

Environment

Release : 12.1.02

Resolution

The index referenced in the iDash Techdoc is to be created on the Autosys database.  

This index is on two columns of the Autosys ujo_proc_event table: que_status_stamp and evt_num. If there is an existing index for these two columns, even if it has a different name, you do not need to create this index. If there is no matching index, you should create this one for increased performance on the import. Under normal operations, this will not make a lot of difference. If you are ever in a position where you are recovering from an iDash outage and importing multiple days of data from Autosys, this index will help the import go quicker. 

To determine if the auto import feature is enabled, take a look at the Autosys instance definition in the iDash Admin Tool. You will find a tab for Archive Data Settings. There are three data types that iDash will import: Job Runs, Events, and AutoTrack. If any of these have a checkmark in the Archive box, then auto import is enabled for that datatype. If it is enabled, there is also a setting to tell iDash how much data to keep. 

To be clear: the index on the Autosys database is not a requirement to have auto import enabled. It can, however, improve import performance under certain circumstances.

In answer to your questions:

1. Correct. Run data for all jobs is imported, whether or not those jobs belong to an SLA flow. iDash already captures current status data for all jobs, even without auto import enabled. 
2. Correct. The expectation is that with auto import enabled, the iDash database will be current with the Autosys database. There is a minimal delay in refreshing the iDash database, as it runs on a refresh timer. This is only around 25 seconds, though.
3. Correct. Job run data is kept in the database for the retention period. Job status data is transitory and kept in memory only, not stored in the iDash database for any length of time.
4. Correct, for job run data only. Job status data is kept in memory only and refreshed from the Autosys instance database every 25 seconds.
5. It is imported when the data is refreshed from the Autosys instance. In the Admin Tool, on the SLA settings tab and the Forecast sub-tab, you will find the DB Refresh Interval setting. This defaults to 25 seconds. You can change this to whatever value you wish, but I would caution against making it too large as the downstream effect is that you will see less frequent status updates for your Autosys SLAs. Setting it too small will not result in much visible difference, as the iDash display is refreshed according to the next setting on the Forecast sub-tab: Forecast Interval. I would suggest moving these two values together if you want to move them at all, with the DB Refresh slightly more frequent than the Forecast Interval.
6. Check the Archive box in the Archive Data Settings tab for the data type you wish to import. These settings are applied instance by instance, so if you have more than one Autosys instance, they can be configured differently for auto import. The DB Refresh Interval is a global setting, though. 

 

Regarding file settings on the iDashDB Archive Tab settings.

The file names in the iDash DB / Archive tab are for uploading Job, Event, or Autotrack data from the Autosys archive files. In past releases, these files could be uploaded using the Admin Tool, and this tab is where you would do that. Currently, this feature is disabled from the Admin Tool, and uploads of Autosys archive files are done using the CLI. There is no impact in not having any files listed on these tabs.

The SLA Settings tab contains time limits for archiving and deleting data related to SLA runs and CA 7 alerts. If you click the blue question mark button, the online help panel that opens will give you a breakdown of what specific things each of these values controls.