Autosys File watcher jobs are unable to resolve the variables defined in the job profile file at runtime. Below is the example job w/ the problem -
insert_job: test_fw_var_watch_file job_type: fw
machine: au_cba_nasrem_racoapp_test_buh
owner: fw_owner
max_run_alarm: 60
alarm_if_fail: n
alarm_if_terminated: n
days_of_week: su,mo,tu,we,th,fr,sa
start_times: "03:30"
profile: /File_Transfer/profile/atsys_profile
term_run_time: 300
send_notification: n
watch_file: ${FILE_HISTORY}/FILE_HISTORY_${TODAY_YYYYMMDD}.DAT
watch_file_min_size: 0
watch_interval: 60
The profile file " /File_Transfer/profile/atsys_profile" from the job -
# cat /File_Transfer/profile/atsys_profile
#!/usr/bin/bash
# Set yesterday's date in YYYYMMDD format
YDAY_YYYYMMDD=$(date -d"yesterday" +%Y%m%d); export YDAY_YYYYMMDD
# Set today's date in YYYYMMDD format
TODAY_YYYYMMDD=$(date '+%Y%m%d'); export TODAY_YYYYMMDD
# Setup the landing directory for schema, manifest and data files
# for various apps
# Daily extracts locations
FILE_CUS=/folder1/directory1/CUSDAT
FILE_ASBCUS=/folder1/directory1/ASBCusScrnDAT
FILE_ASBPEP=/folder1/directory1/ASBPEPDAT
FILE_CASEFOLDER=/folder1/directory1/ISAACDAT
FILE_HISTORY=/folder1/directory1/HISTDAT
FILE_KYC=/folder1/directory1/KYCDAT
FILE_PEP=/folder1/directory1/PEPDAT
FILE_HRC=/folder1/directory1/HRC
# Source Autosys profile
. /opt/CA/WorkloadAutomationAE/autouser.???/autosys.bash.`hostname`
When the file watcher job runs, it looks for the file under root (/) directory not in the location the variable ${FILE_HISTORY} is set to.
File watcher process details:
FW_ONER+ 12653 29503 0 10:39 ? 00:00:00 /opt/CA/WorkloadAutomationAE/SystemAgent/WA_AGENT/cybspawn.bin 6 /opt/CA/WorkloadAutomationAE/SystemAgent/WA_AGENT 20221212 10390094-1100 WA_AGENT filemon#JavaAgent#tcpip@TS1_SCH 5215838.519347924_1/WAAE_WF0.1/MAIN RUN DOTINTERNAL Data(Command="/opt/CA/WorkloadAutomationAE/SystemAgent/WA_AGENT/filewatcher",Args="18:1_10000_true_60000 62:\"\"\"${FILE_HISTORY}/FILE_HISTORY_${TODAY_YYYYMMDD}.DAT\"\"\" 2:S0 6:CREATE 0: 0: 0: 5:60000 5:false ") Env(AUTOSERV=TS1,AUTO_JOB_NAME="test_fw_var_watch_file",AUTORUN="519347924-1",EWA_ENV_SRC_1=/File_Transfer/profile/atsys_profile) User(fw_owner) Password() MFUser(fw_owner) InternalCommand GblEnv(AUTOSYS="/opt/CA/WorkloadAutomationAE/autosys",AUTOROOT="/opt/CA/WorkloadAutomationAE",EWAGLOBALPROFILE="/etc/auto.profile") MgrEnv(EWAGLOBALPROFILE="/etc/auto.profile") JLErase(TRUE) SpoolErase(TRUE) 5CFF2C9DB7398923733CDF27D9FCFA39DE4322C1 /opt/CA/WorkloadAutomationAE/SystemAgent/WA_AGENT/cybAgent.bin 194c18 /opt/CA/WorkloadAutomationAE/SystemAgent/WA_AGENT/spool WA_AGENT 49154 /opt/CA/WorkloadAutomationAE/SystemAgent/WA_AGENT/security.txt - /usr/bin/sh,/bin/csh,/bin/ksh,/usr/bin/ksh,/bin/sh,/bin/bash,/usr/bin/bash - /bin/sh 256 USER_CONT_NOT_AVAIL - - su - -
FW_ONER+ 12699 12654 0 10:39 ? 00:00:00 /opt/CA/WorkloadAutomationAE/SystemAgent/WA_AGENT/filewatcher 18:1_10000_true_60000 62:/FILE_HISTORY_20221212.DAT 2:S0 6:CREATE 0: 0: 0: 5:60000 5:false
Release : 12.0
This behavior is due to the environment variables are not exported in the profile file.
Because the agent creates multiple wrapper files before initiating the file watcher process, the un-exported variables in the profile script(file) will not be sourced.
Export all the variables in the job profile file and re-trigger the job -
After making the corrections, the aforementioned profile file be like -
# cat /File_Transfer/profile/atsys_profile
#!/usr/bin/bash
# Set yesterday's date in YYYYMMDD format
YDAY_YYYYMMDD=$(date -d"yesterday" +%Y%m%d); export YDAY_YYYYMMDD
# Set today's date in YYYYMMDD format
TODAY_YYYYMMDD=$(date '+%Y%m%d'); export TODAY_YYYYMMDD
# Setup the landing directory for schema, manifest and data files
# for various apps
# Daily extracts locations
export FILE_CUS=/folder1/directory1/CUSDAT
export FILE_ASBCUS=/folder1/directory1/ASBCusScrnDAT
export FILE_ASBPEP=/folder1/directory1/ASBPEPDAT
export FILE_CASEFOLDER=/folder1/directory1/ISAACDAT
export FILE_HISTORY=/folder1/directory1/HISTDAT
export FILE_KYC=/folder1/directory1/KYCDAT
export FILE_PEP=/folder1/directory1/PEPDAT
export FILE_HRC=/folder1/directory1/HRC
# Source Autosys profile
. /opt/CA/WorkloadAutomationAE/autouser.???/autosys.bash.`hostname`
File watcher process details:
FW_ONER+ 12653 29503 0 11:40 ? 00:00:00 /opt/CA/WorkloadAutomationAE/SystemAgent/WA_AGENT/cybspawn.bin 6 /opt/CA/WorkloadAutomationAE/SystemAgent/WA_AGENT 20221212 11400094-1100 WA_AGENT filemon#JavaAgent#tcpip@TS1_SCH 5215838.519347924_1/WAAE_WF0.1/MAIN RUN DOTINTERNAL Data(Command="/opt/CA/WorkloadAutomationAE/SystemAgent/WA_AGENT/filewatcher",Args="18:1_10000_true_60000 62:\"\"\"${FILE_HISTORY}/FILE_HISTORY_${TODAY_YYYYMMDD}.DAT\"\"\" 2:S0 6:CREATE 0: 0: 0: 5:60000 5:false ") Env(AUTOSERV=TS1,AUTO_JOB_NAME="test_fw_var_watch_file",AUTORUN="519347924-1",EWA_ENV_SRC_1=/File_Transfer/profile/atsys_profile) User(fw_owner) Password() MFUser(fw_owner) InternalCommand GblEnv(AUTOSYS="/opt/CA/WorkloadAutomationAE/autosys",AUTOROOT="/opt/CA/WorkloadAutomationAE",EWAGLOBALPROFILE="/etc/auto.profile") MgrEnv(EWAGLOBALPROFILE="/etc/auto.profile") JLErase(TRUE) SpoolErase(TRUE) 5CFF2C9DB7398923733CDF27D9FCFA39DE4322C1 /opt/CA/WorkloadAutomationAE/SystemAgent/WA_AGENT/cybAgent.bin 194c18 /opt/CA/WorkloadAutomationAE/SystemAgent/WA_AGENT/spool WA_AGENT 49154 /opt/CA/WorkloadAutomationAE/SystemAgent/WA_AGENT/security.txt - /usr/bin/sh,/bin/csh,/bin/ksh,/usr/bin/ksh,/bin/sh,/bin/bash,/usr/bin/bash - /bin/sh 256 USER_CONT_NOT_AVAIL - - su - -
FW_ONER+ 12699 12654 0 10:39 ? 00:00:00 /opt/CA/WorkloadAutomationAE/SystemAgent/WA_AGENT/filewatcher 18:1_10000_true_60000 62:/folder1/directory1/HISTDAT/FILE_HISTORY_20221212.DAT 2:S0 6:CREATE 0: 0: 0: 5:60000 5:false