The jobs stay in READY or PROCESSING state. This issue may affect only one or few UNIX and Linux agents.
UNIX Operating System such as Solaris and AIX on UFS and other non journaling filesystem
Linux on ext3 and ext2 filesystem.
Before running each job, the agent creates a directory to capture output in the spool. If the agent cannot make a directory for an application in spool directory, then the job will stay in READY state.
If a user is unable to create a directory (not touch a file) manually, and the OS gives following error:
mkdir: Failed to make directory "directory_name"; Too many links.
Then, the issue with the OS filesystem.
The issue is with the limit of subdirectories within a directory. The UFS and older Linux filesystem only allow 32,726 (2^15) subdirectories. To create new directories, older directories will have to be removed or cleared out.
Note: User may run the following command in the spool directory to get the directory count. For example;
ls -d */ | wc
User may also archive the spool directory by renaming it to something like spool.old and restart the agent. The agent will create a new spool directory upon restart. However, it is highly recommended to periodically clean the spool. It can also be done by the agent by enabling the following parameters in the agentparm.txt (agent restart is required)
If above parameters are enabled already, then reduce the 'runnerplugin.spool.expire' value to keep only few days of spool files.
For more information on the above parameters, please consult Workload Automation Agent documentation here.