Memory Leak when Generating Wide Tables

Products

• CA Test Data Manager (Data Finder / Grid Tools)

Issue/Introduction

I increased Wrapper  of JVM but publish keeps blowing out 

vm 2    | 2020/02/12 21:42:21 | Exception in thread "taskExecutor-1" java.lang.OutOfMemoryError: GC overhead limit exceeded

wrapper  | 2020/02/12 21:42:21 | The JVM has run out of memory.  Restarting JVM.

Test scenario:  Have 1 header trailer and the two table bodies. Created 3 rows for each registered table bodies.

Cause

Was due to table referencing

Environment

Test Data Manager
TDM Web Portal

Resolution

To show there is no memory leak:

Allocated 12 GB memory to Portal. When Portal starts it already takes about 6 GB. This means that you have about 6 GB to use.

We set up the environment same as yours... with 1 header trailer and the two table bodies. Created 3 rows for each registered table bodies.

When the table repeat count was set up to 50,000 (meaning 300,000 entries in generated FD file), the Portal was not able to handle the load as eventually the Garbage collector was running all the time trying to free up memory.

Restarted the Portal and reduced the table repeat count to 25,000. this time the Portal completed the job and generated 150,000 entries in FD file.

We then ran 2 more jobs with 25,000 table repeat count, the Portal did not run out of memory as it was able to free up memory each time with the Garbage collector

Here are the data... note that performance gets worse when it starts processing the second table. We have found out it was due to table referencing.