Scheduler on a linux machine is down and when tried to restart it gives listener could not be started messages

Products

• CA Workload Automation AE - Business Agents (AutoSys)
• CA Workload Automation AE - Scheduler (AutoSys)
• Workload Automation Agent

Issue/Introduction

Application server fails to start complaining about agent listener port.

CAUAJM_W_10134 CA WAAE Scheduler failed to establish a listener at host [hostname:49200]. Acquiring a new agent listener port...

CAUAJM_W_00002 Listener could not be started on port [0]. Proceeding... [msgSocket::msgSocket(server socket)(): 16: Devic

The following errors and messages are seen in event demon log:

CAUAJM_W_10134 CA WAAE Scheduler failed to establish a listener at host [hostname:49200]. Acquiring a new agent listener port... 

CAUAJM_W_00002 Listener could not be started on port [0]. Proceeding... [msgSocket::msgSocket(server socket)(): 16: Device or resource busy] 

CAUAJM_E_10125 CA WAAE Scheduler failed to establish a listener at host hostname. Exiting. 

CAUAJM_E_10498 CA WAAE scheduler failed to establish the required listeners. 

CAUAJM_I_40245 EVENT: ALARM ALARM: EP_SHUTDOWN 

Environment

CA Workload Automation AE 11.3.6 on linux OS.

Cause

It occurs when Csam is stuck or is in CLOSE_WAIT state because the scheduler fails to catch up to the number of socket connections from the agents.

Resolution

Sometimes a reboot of the machine works for this error. The other solution (for linux OS) is to follow the below procedure:

1. 1.    Stop the scheduler and application server

Unisrvcntr stop waae_server.$AUTOSERV

Unisrvcntr stop waae_sched.$AUTOSERV

2. 2.    Stop SSA

cd $CSAM_SOCKADAPTER/bin

./csampmux stop

3. 3.    Check for csam processes

$ps –ef | grep csam

4. 4.    Look for any processes on WAIT status and clear all of those processes.
5. 5.    Restart CSAM

Cd $CSAM_SOCKADAPTER/bin

./csampmux start

6. 6.    Start scheduler and application server

Unisrvcntr start waae_server.$AUTOSERV

Unisrvcntr staert waae_sched.$AUTOSERV