Command (CMD) Job fails with exit code 127

Products

• Autosys Workload Automation

Issue/Introduction

We have a situation where a job using a service account '<USERNAME>' is not able to execute 'sendevent' commands and the job fails with exit code 127.

Below is the job JIL:

* ----------------- fake job ----------------- */
insert_job: <JOBNAME>  job_type: CMD
box_name: <BOXNAME>
command: sendevent -E FORCE_STARTJOB -J <OTHER_JOB_NAME>
machine: <MACHINE_NAME>
owner: <USERNAME>
permission: gx
date_conditions: 0
std_out_file: "> /opt/app/psoft/autosys/fin.out"
std_err_file: "> /opt/app/psoft/autosys/fin.err"
alarm_if_fail: 1
alarm_if_terminated: 1

Job run details:

Job Name                                   Last Start           Last End             ST/Ex Run/Ntry Pri/Xit
________________________________________________________________ ____________________ ____________________ _____ ________ _______
<JOBNAME>                                 10/05/2022 07:45:00  10/05/2022 07:45:00  FA    0/3      127

  Status/[Event]  Time                 Ntry ES  ProcessTime           Machine
  --------------  --------------------- --  --  --------------------- ----------------------------------------
  [FORCE_STARTJOB]  09/29/2022 00:12:32    0  PD  09/29/2022 00:12:35
  STARTING        09/29/2022 00:12:35    2  PD  09/29/2022 00:12:35   Agent_dev
  RUNNING         09/29/2022 00:12:36    2  PD  09/29/2022 00:12:38   Agent_dev
    <Executing at WA_AGENT_Agent_dev>
  FAILURE         09/29/2022 00:12:37    2  PD  09/29/2022 00:12:38
  [*** ALARM ***]
    JOBFAILURE    09/29/2022 00:12:39    2  PD  09/29/2022 00:12:39   Agent_dev
  [FORCE_STARTJOB]  10/05/2022 07:44:57    0  PD  10/05/2022 07:44:58
  STARTING        10/05/2022 07:44:59    3  PD  10/05/2022 07:44:58   Agent_dev
  RUNNING         10/05/2022 07:45:00    3  PD  10/05/2022 07:45:00   Agent_dev
    <Executing at WA_AGENT_Agent_dev>
  FAILURE         10/05/2022 07:45:00    3  PD  10/05/2022 07:45:00
  [*** ALARM ***]
    JOBFAILURE    10/05/2022 07:45:01    3  PD  10/05/2022 07:45:01   Agent_dev

Below is the machine definition:

/* ----------------- virtual-name_fsdev ----------------- */
insert_machine: virtual-name_fsdev
type: v
machine: Agent_dev
factor: 0.00

 

/* ----------------- Agent_dev ----------------- */

insert_machine: Agent_dev
type: a
factor: 1.00
port: 7525
node_name: AGENT-NAME
agent_name: WA_AGENT_Agent_dev
encryption_type: DEFAULT
character_code: ASCII

Machine status:

D:\Test>autorep -M virtual-name_fsdev

Machine Name                                                                     Max Load   Current Load Factor  O/S         Status
________________________________________________________________________________ __________ ____________ _______ ___________ ______
virtual-name_fsdev.Agent_dev                                                ---        ---          0.00    Sys Agent   Online

The user '<USERNAME>' is able to login to the machine 'virtual-name_fsdev' and able to execute the same command that is defined in the above job (sendevent -E FORCE_STARTJOB -J <OTHER_JOB_NAME>) without any issues.

Only while executing thru Autosys the job is failing with exit code 127. 

 

Environment

Autosys 11.x 12.X

Cause

The error suggests the file specified in the job's command is not found in the existing path OR the environment profile was not properly sourced in the job.

 

Resolution

Either provide the full path to the file or include a profile in the job definition which includes the needed environment (Example:      profile: "/opt/app/AutoSys/CA/WorkloadAutomationAE/autouser.TST/autosys.bash"  ) 

OR consider adjusting the path statement in something like the "/etc/profile" and exiting and restarting the agent so it may consume the new settings.

 

Additional Information

Another example of the error might be like: 

/opt/CA/autosys/WorkloadAutomationAE/SystemAgent/WA_AGENT/spool/ACE_SCH/MAIN/WAAE_WF0.1/181808.172185_17.C5980374C7B48221B0B09A286CB8919CEAE3863C.sh: line 2: foobar_test: command not found