Autosys Jobs Fail with Exit Code 4030: Invalid Command Not Listed in oscomponent.validcommand

Products

Autosys Workload Automation Workload Automation Agent ESP dSeries Workload Automation - System Agent (dSeries) ESP Workload Automation ESP dSeries Workload Automation - Scheduler (dSeries) ESP dSeries Workload Automation - Business Agents (dSeries) ESP dSeries Workload Automation

Issue/Introduction

After upgrading to Autosys System Agent r12.x, jobs sporadically fail with exit code 4030 and the following error message: <Invalid command: not listed in oscomponent.validcommand>

Symptoms

An autorep -J command shows the job failing shortly after starting:

  Status/[Event]  Time                 Ntry ES  ProcessTime           Machine  
--------------  --------------------- --  --  --------------------- -------------------------  
STARTING        12/12/2022 03:06:00    1  PD  12/12/2022 03:06:00   agent-machine  
RUNNING         12/12/2022 03:06:00    1  PD  12/12/2022 03:06:01   agent-machine  
FAILURE         12/12/2022 03:09:00    1  PD  12/12/2022 03:09:01       <Invalid command: not listed in oscomponent.validcommand>

The agent's runner_os_component.log typically contains warnings regarding disk space thresholds being breached prior to the failure:

Critical: Disk resource threshold of 10485760 breached... Agent is entering persist to memory mode.
Transmitter: Sending AFM: ... State SUBERROR Failed SetEnd Status(Invalid command: not listed in oscomponent.validcommand) Cmpc(4030)

Environment

Component: Workload Automation Agent

Cause

The server hosting the Autosys Agent has run out of disk space, falling below the critical threshold defined in the agentparm.txt file.

Default or configured thresholds in agentparm.txt may look like this:

agent.resourcemon.threshold.disk.warning.notice=30M
agent.resourcemon.threshold.disk.warning.severe=20M
agent.resourcemon.threshold.disk.critical=10M

When disk space falls below the critical threshold, the agent enters "persist to memory" mode and stops using the agent database directory. The job fails with exit code 4030 because the agent is unable to create or execute the temporary wrapper scripts required to trigger the job on the file system.

Resolution

To resolve this issue, you must free up disk space on the server where the agent is installed.

1. Resolve Disk Space Infrastructure Issue

Contact your System Administrator to identify and clear space on the partition hosting the agent installation. As soon as disk space rises above the thresholds, the agent will log "Disk resource no longer breaching any thresholds" and automatically switch persistence back to the disk.

2. Adjust Monitoring Thresholds (Optional)

If you require more lead time to address disk issues before jobs fail, increase the warning thresholds in the agentparm.txt file:

Note: Increasing these values does not fix the space issue but provides earlier alerts.

3. Disable Disk Space Monitoring (Optional)

To prevent the agent from monitoring the available disk space, you can set agent.resourcemon.enable parameter to false:

Edit agentparm.txt.
Add or modify: agent.resourcemon.enable=false.
If this is disabled, the agent must be manually restarted for the change to take effect.

4. Configure Automatic Shutdown (Optional)

To prevent the agent from attempting to run jobs in an unstable memory-only state, you can configure it to shut down automatically when the critical limit is hit:

Edit agentparm.txt.
Add or modify: agent.resourcemon.threshold.disk.critical.shutdown=true.
If this is enabled, the agent must be manually restarted once disk space is recovered.

Additional Information

For detailed configuration steps, refer to the Broadcom TechDocs: Configure the Agent to Monitor Available Disk Space