Finding the cause of a hung Datacom application

Products

• Datacom
• Datacom/DB
• Ideal

Issue/Introduction

While running a job that executes an Ideal program, the program reads in data from external tapes. After most of the records have been processed, it appears the program goes into a loop, because this step, which  normally runs for two minutes, now will run for hours until it is cancelled. The job is using CPU and not performing IOs.

Cause

There could be many reasons for a never-ending application, but the most common cause is that bad data causes the program to loop or to branch to an unexpected fall-through location.

Resolution

When it appears that the program has gone into a loop, the first thing to check is if there is a problem with the data being processed in the program. This  is more likely if the program has been running fine for some time, and has not been changed.

In this case, if there is no I/O but there is processing, it sounds like the program is processing a record, and because there is some value that was not expected, the program keeps processing this same record over and over.

One method of troubleshooting is to add some displays of the data to the program and test it. You might be able to reduce the set of external records, or to break the data in to multiple files, but you will have to display every record (or selected fields), and then display other text to know where in the program it is failing.

A common situation is to have either some kind of logic like "Perform this until some total is > something" or "If some counter is > something, go get the next record" and that total or counter field is too small and overflowing, or the data is invalid, and is being picked up as some kind of very large numeric value and that is driving a loop counter.

Whatever the actual data problem, it should be fairly straightforward to track down, though the test runs will likely generate lots of output to get to the point where you hit the problem. once you find the rogue records, you can either modify the data to run through the programs successfully, or you will modify the program to handle the bad data.