Filewatcher jobs not resolving env variables in the profile file
Article ID: 255994
Updated On:
Products
Issue/Introduction
Autosys File watcher jobs are unable to resolve the variables defined in the job profile file at runtime. Below is the example job w/ the problem -
insert_job: test_fw_var_watch_file job_type: fw
machine: au_cba_nasrem_racoapp_test_buh
owner: fw_owner
max_run_alarm: 60
alarm_if_fail: n
alarm_if_terminated: n
days_of_week: su,mo,tu,we,th,fr,sa
start_times: "03:30"
profile: /File_Transfer/profile/atsys_profile
term_run_time: 300
send_notification: n
watch_file: ${FILE_HISTORY}/FILE_HISTORY_${TODAY_YYYYMMDD}.DAT
watch_file_min_size: 0
watch_interval: 60
The profile file " /File_Transfer/profile/atsys_profile" from the job -
# cat /File_Transfer/profile/atsys_profile
#!/usr/bin/bash
# Set yesterday's date in YYYYMMDD format
YDAY_YYYYMMDD=$(date -d"yesterday" +%Y%m%d); export YDAY_YYYYMMDD
# Set today's date in YYYYMMDD format
TODAY_YYYYMMDD=$(date '+%Y%m%d'); export TODAY_YYYYMMDD
# Setup the landing directory for schema, manifest and data files
# for various apps
# Daily extracts locations
FILE_CUS=/folder1/directory1/CUSDAT
FILE_ASBCUS=/folder1/directory1/ASBCusScrnDAT
FILE_ASBPEP=/folder1/directory1/ASBPEPDAT
FILE_CASEFOLDER=/folder1/directory1/ISAACDAT
FILE_HISTORY=/folder1/directory1/HISTDAT
FILE_KYC=/folder1/directory1/KYCDAT
FILE_PEP=/folder1/directory1/PEPDAT
FILE_HRC=/folder1/directory1/HRC
# Source Autosys profile
. /opt/CA/WorkloadAutomationAE/autouser.???/autosys.bash.`hostname`
When the file watcher job runs, it looks for the file under root (/) directory not in the location the variable ${FILE_HISTORY} is set to.
File watcher process details:
FW_ONER+ 12653 29503 0 10:39 ? 00:00:00 /opt/CA/WorkloadAutomationAE/SystemAgent/WA_AGENT/cybspawn.bin 6 /opt/CA/WorkloadAutomationAE/SystemAgent/WA_AGENT 20221212 10390094-1100 WA_AGENT filemon#JavaAgent#tcpip@TS1_SCH 5215838.519347924_1/WAAE_WF0.1/MAIN RUN DOTINTERNAL Data(Command="/opt/CA/WorkloadAutomationAE/SystemAgent/WA_AGENT/filewatcher",Args="18:1_10000_true_60000 62:\"\"\"${FILE_HISTORY}/FILE_HISTORY_${TODAY_YYYYMMDD}.DAT\"\"\" 2:S0 6:CREATE 0: 0: 0: 5:60000 5:false ") Env(AUTOSERV=TS1,AUTO_JOB_NAME="test_fw_var_watch_file",AUTORUN="519347924-1",EWA_ENV_SRC_1=/File_Transfer/profile/atsys_profile) User(fw_owner) Password() MFUser(fw_owner) InternalCommand GblEnv(AUTOSYS="/opt/CA/WorkloadAutomationAE/autosys",AUTOROOT="/opt/CA/WorkloadAutomationAE",EWAGLOBALPROFILE="/etc/auto.profile") MgrEnv(EWAGLOBALPROFILE="/etc/auto.profile") JLErase(TRUE) SpoolErase(TRUE) 5CFF2C9DB7398923733CDF27D9FCFA39DE4322C1 /opt/CA/WorkloadAutomationAE/SystemAgent/WA_AGENT/cybAgent.bin 194c18 /opt/CA/WorkloadAutomationAE/SystemAgent/WA_AGENT/spool WA_AGENT 49154 /opt/CA/WorkloadAutomationAE/SystemAgent/WA_AGENT/security.txt - /usr/bin/sh,/bin/csh,/bin/ksh,/usr/bin/ksh,/bin/sh,/bin/bash,/usr/bin/bash - /bin/sh 256 USER_CONT_NOT_AVAIL - - su - -
FW_ONER+ 12699 12654 0 10:39 ? 00:00:00 /opt/CA/WorkloadAutomationAE/SystemAgent/WA_AGENT/filewatcher 18:1_10000_true_60000 62:/FILE_HISTORY_20221212.DAT 2:S0 6:CREATE 0: 0: 0: 5:60000 5:false
Environment
Release : 12.0
Cause
This behavior is due to the environment variables are not exported in the profile file.
Because the agent creates multiple wrapper files before initiating the file watcher process, the un-exported variables in the profile script(file) will not be sourced.
Resolution
Export all the variables in the job profile file and re-trigger the job -
After making the corrections, the aforementioned profile file be like -
# cat /File_Transfer/profile/atsys_profile
#!/usr/bin/bash
# Set yesterday's date in YYYYMMDD format
YDAY_YYYYMMDD=$(date -d"yesterday" +%Y%m%d); export YDAY_YYYYMMDD
# Set today's date in YYYYMMDD format
TODAY_YYYYMMDD=$(date '+%Y%m%d'); export TODAY_YYYYMMDD
# Setup the landing directory for schema, manifest and data files
# for various apps
# Daily extracts locations
export FILE_CUS=/folder1/directory1/CUSDAT
export FILE_ASBCUS=/folder1/directory1/ASBCusScrnDAT
export FILE_ASBPEP=/folder1/directory1/ASBPEPDAT
export FILE_CASEFOLDER=/folder1/directory1/ISAACDAT
export FILE_HISTORY=/folder1/directory1/HISTDAT
export FILE_KYC=/folder1/directory1/KYCDAT
export FILE_PEP=/folder1/directory1/PEPDAT
export FILE_HRC=/folder1/directory1/HRC
# Source Autosys profile
. /opt/CA/WorkloadAutomationAE/autouser.???/autosys.bash.`hostname`
File watcher process details:
FW_ONER+ 12653 29503 0 11:40 ? 00:00:00 /opt/CA/WorkloadAutomationAE/SystemAgent/WA_AGENT/cybspawn.bin 6 /opt/CA/WorkloadAutomationAE/SystemAgent/WA_AGENT 20221212 11400094-1100 WA_AGENT filemon#JavaAgent#tcpip@TS1_SCH 5215838.519347924_1/WAAE_WF0.1/MAIN RUN DOTINTERNAL Data(Command="/opt/CA/WorkloadAutomationAE/SystemAgent/WA_AGENT/filewatcher",Args="18:1_10000_true_60000 62:\"\"\"${FILE_HISTORY}/FILE_HISTORY_${TODAY_YYYYMMDD}.DAT\"\"\" 2:S0 6:CREATE 0: 0: 0: 5:60000 5:false ") Env(AUTOSERV=TS1,AUTO_JOB_NAME="test_fw_var_watch_file",AUTORUN="519347924-1",EWA_ENV_SRC_1=/File_Transfer/profile/atsys_profile) User(fw_owner) Password() MFUser(fw_owner) InternalCommand GblEnv(AUTOSYS="/opt/CA/WorkloadAutomationAE/autosys",AUTOROOT="/opt/CA/WorkloadAutomationAE",EWAGLOBALPROFILE="/etc/auto.profile") MgrEnv(EWAGLOBALPROFILE="/etc/auto.profile") JLErase(TRUE) SpoolErase(TRUE) 5CFF2C9DB7398923733CDF27D9FCFA39DE4322C1 /opt/CA/WorkloadAutomationAE/SystemAgent/WA_AGENT/cybAgent.bin 194c18 /opt/CA/WorkloadAutomationAE/SystemAgent/WA_AGENT/spool WA_AGENT 49154 /opt/CA/WorkloadAutomationAE/SystemAgent/WA_AGENT/security.txt - /usr/bin/sh,/bin/csh,/bin/ksh,/usr/bin/ksh,/bin/sh,/bin/bash,/usr/bin/bash - /bin/sh 256 USER_CONT_NOT_AVAIL - - su - -
FW_ONER+ 12699 12654 0 10:39 ? 00:00:00 /opt/CA/WorkloadAutomationAE/SystemAgent/WA_AGENT/filewatcher 18:1_10000_true_60000 62:/folder1/directory1/HISTDAT/FILE_HISTORY_20221212.DAT 2:S0 6:CREATE 0: 0: 0: 5:60000 5:false
Additional Information
Feedback
