Description:
Jobs that run on a Linux Remote Agent will intermittently fail with the following error sequence in the event_demon log:
[09:46:15.6380] [2] [linux_agent connected for linux_job] [09:46:15.6380] [2] ## socket read <>, rc=0 [09:46:15.6690] [2] System Restart - Job (linux_job) was unable to start [09:46:15.9190] [1] EVENT: ALARM ALARM: STARTJOBFAIL JOB: linux_job[09:46:15.9190] [1] <Auto Remote process did not start.>
When the job attempts to restart after receiving the error above, it runs successfully. Other jobs are also running at the same time on the same Remote Agent without error.
Solution:
There is a parameter in the Linux xinetd service configuration that can be set called "per_source". This parameter sets the maximum concurrent connections to xinetd from any one IP address. Therefore, this effectively puts a limit on the amount of jobs an AutoSys instance can attempt to start on the LINUX machine at the same time. When the limit is reached, the socket error described above will occur for any further job that attempts to run on the machine. This will continue until the amount of connections from the AutoSys server to the Remote Agent machine drops below the "per_source" limit.
There are two options for setting this parameter:
The parameter can be set in the "defaults" section of the xinetd.conf file. This sets the parameter for all xinetd services configured on the machine. In this example, the per_source limit is set to 10:
defaults { instances = 60 log_type = SYSLOG authpriv log_on_success = HOST PID log_on_failure = HOST cps = 25 30
per_source = 10 }
Alternatively, the parameter can be set within the individual configuration file for the auto_remote service that is created in the xinetd.d directory. There will be a file in that directory that matches the service name assigned to the Remote Agent in the /etc/services file. The parameter can be set in that file so that it only limits connections to the auto_remote service. For example...
# more auto_remote_45_ora # AutoSys Version 4.5 service auto_remote_45_ora { socket_type = stream wait = no user = root server = /sbin/runuser server_args = root - -c /opt/CA/UnicenterAutoSysJM/4.5-ora/autosys/bin/auto_remote disable = no
per_source = 10 }
NOTE: When setting this parameter, consideration should be given to the AutoSys schedule so that it is set high enough to accommodate the load. A value of UNLIMITED is allowed. Also, The above mentioned settings require a refresh or restart of the xinetd process.