Autosys jobs stuck in a Starting state

Products

• CA Workload Automation AE - Business Agents (AutoSys)
• CA Workload Automation AE - Scheduler (AutoSys)
• Workload Automation Agent

Issue/Introduction

There are 5 Autosys Trigger jobs stuck in a Starting state, what could be the cause?

Environment

Component: Workload Automation Agent

Resolution

Users may sometimes notice that jobs stay in a READY or STARTING state.  The jobs run on the agents successfully and all required ports are open.  The logs may also show that jobs complete on the agent side, but no update is sent back to the Scheduler manager (transmitter log).  

In some Linux filesystems, there is a limitation of number of sub-directories within a  directory.  The ext3 has a limitation of 32000 subdirectories within a directory.  If the spool directory is not cleared in a timely manner, the subdirectory limitation can prevent agent from creating any spool output.  This will also result in a failure to update the manager of job(s) status.  The spool directory is located here: /opt/CA/WorkloadAutomationAE/SystemAgent/WA_AGENT.

Clear the spool directory and then run the job(s) again.  The agent can also be configured to automatically clear the spools.  Add the following parameters to the agentparm.txt and restart the agent.

# Change settings as per your requirement.  
runnerplugin.spool.clean.enable=true  
# Clear all spool older than 10 days
runnerplugin.spool.expire=10D
# Clear once every 24 hours 
runnerplugin.spool.sleep=24H