How can I extract the program call stack of a core file on Unix/Linux?

Products

• CA Workload Automation AE - Business Agents (AutoSys)
• CA Workload Automation AE - Scheduler (AutoSys)
• Workload Automation Agent

Issue/Introduction

An Autosys process crashed and a core file is generated. How can I extract the program call stack?

Environment

Release: All Supported WAAE Versions

Resolution

  Step #1. Obtain a core file:

1. Usually, when a process experiences a fatal process error, the UNIX operating system generates a core file containing the memory image of the process at the time of the fatal process error. Your UNIX operating system may provide utilities that control whether core files are generated, and if they are, where the generated core files are placed and what names toassign them. Consult your operating system documentation to configure your UNIX machine to generate core files at a known location with a known name to find the core file.
   If your UNIX machine is configured with default values to generate core files, you may find the core file by performing a recursive search for files named "*core*" within the CA Workload Automation AE installation directory (i.e. /opt/CA/WorkloadAutomationAE) of from /

    For example: find / -name “core*”

   If a core file is not generated, consult your operating system documentation for ways to configure your UNIX machine to generate a core file at a known location with a known name and try to reproduce the fatal process error.

   For example, some UNIX systems use the ulimit utility as a way to configure the operating system to generate core files. To use this utility, you may log onto the UNIX machine as the autosys user and execute the UNIX command "ulimit -a". You can verify that the output for the row "coredump" reads "unlimited". If the "coredump" row does not read as "unlimited", set it by running the command "ulimit -c unlimited".

   However, to ensure that a core file is properly generated on your UNIX machine, we recommend that you consult your operating system documentation.

 Step #2: Verify that the core file resulted from running a CA Workload Automation AE binary by executing the following UNIX command:

1. file <complete path of core file>

 Step #3: Extract the process stack information from the core file that is generated.

     Solaris:

        Use the Solaris pstack utility that comes with the Solaris O/S.

     1. pstack <compete path of core file>

  Linux or HP-UX (requires installation of HP Wildebeest Debugger - WDB):

         Open a command prompt, source the shell script to set CA WAAE environments and execute the following commands:

    1. script /tmp/aestack.txt
    2. If the core file resulted from running the Scheduler then the execute following command:

                 gdb $AUTOSYS/bin/event_demon <complete path to core file>

 

            If the core file resulted from running the Application Server then the execute following command:

                gdb $AUTOSYS/bin/as_server <complete path to core file>

     1. At the gdb prompt, type the following gdb commands:

                (gdb) set pagination off [enter]

                (gdb) info threads [enter]

                (gdb) thread apply all bt [enter]

                (gdb) quit [enter]

     1. Type control+D to complete the recording of your console session into file /tmp/aestack.txt.
     2. Upload /tmp/aestack.txt to the case

 AIX:

Open a command prompt, source the shell script to set CA WAAE environments and execute the following commands:

    1. script /tmp/aestack.txt
    2. If the core file resulted from running the Scheduler then execute following command:

                    dbx $AUTOSYS/bin/event_demon <complete path to core file>

             If the core file resulted from running the Application Server then execute following command:

                   dbx $AUTOSYS/bin/as_server <complete path to core file>

    1. At the dbx prompt, type the following dbx commands:

                (dbx) thread [enter]

                     This command outputs a list of all thread numbers in the form of $tN, where N is the thread number.

    1. Cut and paste this output into a text editor.

               Edit the text file to create the following list of commands for all thread numbers:

              thread current 1

              where

              thread current 2

             where

             ...

            thread current N

            where <newline>

    1. Copy these commands and paste into the dbx command prompt.
    2. (dbx) quit [enter]
    3. Type control+D to complete the recording of your console session into file /tmp/aestack.txt.
    4. Upload /tmp/aestack.txt to the case