csh Script Loops and Job Stucks in RU State on RHEL 8 Server CA System Agent

Products

• CA Workload Automation AE - System Agent (AutoSys)
• CA Workload Automation AE - Business Agents (AutoSys)

Issue/Introduction

Autosys command job with csh script on RHEL 8 server loops and creates multiple processes and the job hungs in "RU" state.

Environment

Release : 11.5 and 12.0 

Component : CA Workload Automation System Agent

Cause

The problem is specific to Red Hat 8, csh and auto.profile. The auto.profile setting the path to include $AUTOSYS/test/bin. The $AUTOSYS/test/bin directory contains a script named "test".  Since the auto.profile script has the following:

# Set a PATH so executables can be found
PATH=".:$AUTOSYS/bin:$AUTOSYS/test/bin:/bin:/usr/bin:/usr/local/bin:/usr/openwin/bin:/usr/bin/X11:/bin:/usr/ucb:/usr/etc:$PATH"

It sets the $AUTOSYS/test/bin directory in the beginning of the PATH so the "test" script is being called instead of /usr/bin/test.  Red Hat 8 has change the /etc/profile.d/colorzgrep.csh script to include the following:

test -f /usr/libexec/grepconf.sh

This did not exist in earlier versions of RedHat.  This script gets executed by /etc/csh.cshrc.  In the /etc/csh.cshrc there is the following section which sources all of the csh scripts in /etc/profile.d.

# Check if we aren't a loginshell and do stuff if we aren't
if (! $?loginsh) then
    if ( -d /etc/profile.d ) then
        set nonomatch
        foreach i ( /etc/profile.d/*.csh )
            if ( -r "$i" ) then
                if ($?prompt) then
                    source "$i"
                else
                    source "$i" >&/dev/null
                endif
            endif
        end
        unset i nonomatch
    endif
endif

Resolution

To identify the problem follow the below instructions.

1. Trigger the problematic job(Let the job run)
2. Run ps -ef | grep csh
3. See if the job is recursively running /bin/csh /opt/CA/WorkloadAutomationAE/autosys/test/bin/test -f /usr/libexec/grepconf.sh
4. Do one of the following:

Rename /opt/CA/WorkloadAutomationAE/autosys/test/bin/test to /opt/CA/WorkloadAutomationAE/autosys/test/bin/as_sample

or

Remove $AUTOSYS/test/bin from the PATH in auto.profile.

Additional Information

This would be addressed in(future release)12.1 Agent version.