search cancel

TDM Portal - Masking a huge project always gives a Read Timeout error message

book

Article ID: 242929

calendar_today

Updated On:

Products

CA Test Data Manager (Data Finder / Grid Tools)

Issue/Introduction

Last Friday we tried to "stress test" our DEV infra running a huge masking project but the masking always gave us a Read Timeout error message. We tried masking the same project on 2 different TDM portal but ending with the same error. We tried masking this project 3 times using each time a different value for the "httpClient.ReadTimeout" parameter in the "application.properties" config file. We tried with a value of 900000 which is currently the setting we use on all our TDM portals. We also tried a value of 1200000 and even 3600000. Each time the masking gives a Read Timeout error. This is really not good and needs to be fixed. For now we have to split the project so the masking is ran in multiple waves or we need to mask table by table which is not performant and takes days to complete.

We have attached the logs from 2 different TDM portal for you to review. If you guys needs more information on the project itself to be able to reproduce the issue, please let us know. Also, note that they were no other activities on our other TDM portal to make sure the DB repository server was not too much busy.

Oracle DB server is always consuming a huge amount of RAM causing to go into swapping, which then causes CPU to go over 90% of usage also.

We are planning to add more RAM to the DB server. Then we will retry a stress test to see if this helps.

Environment

Release : 4.10

Component : TDM Web Portal

Resolution

Unfortunately they didn't go high enough - the call needs at least 4620000 milliseconds to complete (yes, almost one hour and half). I'm checking if we can implement some optimization too.

So this needs to be attempted again with a higher timeout value please.  I would go with rhe suggestion plus 50% (6930000 ) for any potential database internal slowness that "might" occur.  The ultimate purpose is to get the job completed.