Monitoring Lost Control status for AutoSys jobs

Products

• Workload Automation Agent
• Autosys Workload Automation

Issue/Introduction

This article explains why AutoSys jobs may report a "Lost Control" status and how to proactively detect this failure even when standard STDOUT or STDERR logs are not generated.

Symptoms

• An AutoSys job fails with a status of Lost Control.
• No standard output (STDOUT) or standard error (STDERR) logs are created for the job run.
• The job may remain in a STARTING or RUNNING state for an extended period before transitioning to FAILURE with the "Lost Control" description.

Environment

• Product: AutoSys Workload Automation AE
• Component: System Agent (All Versions)
• Platform: Windows, Linux, UNIX

Cause

A "Lost Control" status occurs when the AutoSys agent can no longer track the Process ID (PID) of a running job. This typically happens due to:

1. An unexpected restart of the AutoSys Agent service while a job is in progress.
2. A hard reboot of the server where the agent is hosted.
3. The agent being unable to reconnect to the process after a service interruption.

Because the tracking link (PID) is broken, the agent cannot capture the final output of the process, resulting in the absence of job logs.

Resolution

To ensure visibility and proactive alerting for these events, follow these steps:

1. Enable Job Failure Alarms

Ensure that mission-critical job definitions include the alarm_if_fail attribute set to 1. Since AutoSys treats a "Lost Control" event as a job FAILURE, this will trigger a JOBFAILURE alarm.

jil

/* Example JIL Snippet */insert_job: my_sample_jobjob_type: CMDcommand: /bin/script.shmachine: my_agent_hostalarm_if_fail: 1

 

2. Monitor System Event Logs

Configure your monitoring system to scan the event_demon log (e.g., event_demon.$AUTOSERV) for the following specific strings:

• STATUS: FAILURE
• Lost control (Found in the extended event description)

3. Review Agent Logs

On the agent machine, you can verify these events by searching the runner_os_component.log for entries matching: Status(Lost control)

4. Implementation Recommendation

If you require automated detection, implement a PowerShell or Shell script to parse the event_demon log daily for these keywords. This ensures that failures occurring without standard log generation are captured and reported to your operations team.