as_server process goes down frequently

Products

• CA Workload Automation AE - Scheduler (AutoSys)
• Autosys Workload Automation

Issue/Introduction

The Workload Automation AE as _server process core dumps frequently. We see below oracle error message in the $AUTOUSER/out/as_server.$AUTOSERV log file:

ORA-24550: signal received: [si_signo=11] [si_errno=0] [si_code=1] [si_int=0] [si_ptr=(nil)] [si_addr=0x6e]

Environment

Release: ATSYHA99000-11.3.6-Workload Automation AE-High Availability Option
Component:

Workload Automation AE 11.3.x
Oracle - 11.x
Platform - Linux/Unux

Resolution

Workaround: 

In this specific client's case CA's sustaining engineering team reviewed the core file and as_server debug log and found a a large number of threads being used as a result of 11.0 agents still in use in the client's environment.

The recommendation was to:

• Make a back up the existing /etc/init.d/waae_server.$AUTOSERV file
• Modify /etc/init.d/waae_server.$AUTOSERV (which will be active).
• Near the top of the file where AUTOSERV AUTOSYS and AUTOUSER being set,
• Add the following four lines:
  • RECV_TIMEOUT=4
  • MSGSVC_LISTEN_TIMEOUT=4
  • export RECV_TIMEOUT
  • export MSGSVC_LISTEN_TIMEOUT
• Save the file
• Restart the application server.

 

These setting will be used as default values in future AutoSys releases to help avoid thread consumption and longer timeouts.  

RECV_TIMEOUT = 4

In the context of application server this controls how long to wait for a response from 11.3 agent-specific operations like autoping, chase, non-EEM user authentications. It also controls the time the sender threads wait for new send data to arrive in the send queue before exiting.

MSGSVC_LISTEN_TIMEOUT = 4

Controls how long any receiver thread waits for an acknowledgement from an r11 or r11.3 agent to arrive for sent data before timing out the socket connection. This controls the time the receiver thread lingers waiting for the next message before exiting.