Long running Job ends with error "230 Error Timed out waiting for response on client pipe 240"

Products

• CA Automic Applications Manager (AM)

Issue/Introduction

A Job may continue to be in Running status for a period of time while the program launched by the Job is already finished on the application end.

Cause

It is possible that a database call or procedure is taking a long time to return which may is stopping the Job from going from a Running status to a Finished status.

Resolution

While this behavior can be cause by multiple issues, it is best to enable debug and review the RmiServer<timestamp>.log to see if there are any database calls or procedures that are taking a long time to return. This can be done by searching the string "elapsed ms", which is elapsed milliseconds.

For example, if the RMI log is opened in notepad++, all occurrence where database calls or procedures take 100 seconds or more to returned be found by the following:

1. Use Control + F to open Find Window
2. Change Search Mode to Regular Expression
3. Type in search string "elapsed ms \d\d\d\d\d\d" without the double quotes
4. Select "Find All in Current Document"

Note that the "\d" represents a 1 digit number so "\d\d\d\d\d\d" represents a 6 digit number in milliseconds which translates to 100 seconds. Below is an example search result where it took 98944446 milliseconds or about 27 hours to return a database procedure.

08:52:54.510 awserverWorker1: TRN:6129: .D: getProcedureData() awdyn.execute_plsql elapsed ms 98944446

It is recommended that this result is reported to the DBA for review to see why the database call or procedure.