Workload Automation AE command "autosyslog" returns error 28 (No space left on device)

Products

• CA Workload Automation AE - Business Agents (AutoSys)
• CA Workload Automation AE - System Agent (AutoSys)
• CA Workload Automation AE - Scheduler (AutoSys)
• Workload Automation Agent
• Autosys Workload Automation

Issue/Introduction

Error returned while running the command 'autosyslog'

$autosyslog -j test -r 0 -n 0 -t S

----------------------------------------------------------------

Output of messages for workload object 9226580.352215874_1/WAAE_WF0.1/MAIN

Start date Thu Jan 23 11:50:00 2020

----------------------------------------------------------------

STDOUT = "${log}/$AUTO_JOB_NAME.log`/bin/date +\\%y\\%m

%d`"

STDERR = "${log}/$AUTO_JOB_NAME.err`/bin/date +\\%y\\%m

%d`"

cybspawn: Thu Jan 23 11:50:02 2020:Error opening stderr "/disk2/data/archive/log/testjob.err200123": error 28 (No space left on device)

cybspawn: Thu Jan 23 11:50:02 2020:Can't create spool files, error 28 (No space left on device)

Environment

Product: CA Workload Automation System Agent
OS: Supported Linux platforms

Cause

"No space left on the device" could also mean that the partition is out of inodes.
It does not matter how large the files are but each file, directory, and symbolic link requires an inode. 

 

Resolution

• As a root user try the native Linux utilities (df -i) to check the inode utilization. If the utilization is 100% on the partition consider deleting or moving the unused or old files.
• Ensure the partition has enough disk space along with the inode.
• Some of the files could largely contribute for the issue are the job stdout/stderr logs directory (/disk2/data/archive/log)
• Consider deleting or moving files from CA Workload Automation Agent's spool directory (/opt/CA/WorkloadAutomationAE/SystemAgent/WA_AGENT/spool/)
• Restart CA Workload Automation AE Agent service.

Additional Information

The CA Workload Automation Agents have the configuration properties for the spool and log files maintenance, see here.