search cancel

TDM Portal - Data getting mixed up when using a generator to copy data


Article ID: 190667


Updated On:


CA Test Data Manager (Data Finder / Grid Tools)


We are using a Generator in TDM Portal to copy data from our Production environment to Staging, and noticed the data is getting mixed up when publishing to staging. For example, we have account IDs, and each account ID has a number of invoice numbers associated to them. When the problem is seen, we noticed that account 1 has some invoices from account 2, and vice versa. 
There are no error in logs in the logs, so we aren't sure how to troubleshoot this and why it is happening?


Release : 4.6

Component : CA Test Data Manager


The best way to troubleshoot is to enable Portal in debug.

See KB Article 9851 - "TDM Portal: How to enable debugging for TDM Portal"


When examining the logs, we noticed that the data being inserted is different. This can happen if the SQL expressions used by the Data Generator, has different SQL expressions in each of the columns. In order to ensure the data is found in the same order, you must use the exact same SQL expression for each of the columns.

Looking at the SQL expressions in the Generator where the problem was seen, we found there are differences in the expressions, and thus the mixed results returned.

When the generator is run for the first time, TDM builds the internal lists for each of the SQL expressions used. Because the SQL Engine may not return each of the result sets in the exact same order, it is possible for a divergent in the results, thus a mixed, or scrambled set of data. Therefore, anytime you want to ensure a consistent result set between environments, it is best practice to ensure each of the columns uses the exact same SQL expression. This ensures TDM is working from the same internal table, and not from a mixed number of tables.

One way to ensure all columns for a given Generator is using the same SQL expression, open the Generator in Datamaker, and make sure each column has the exact same expression. An easy way to do this is using the  'Copy Column Right' to 'Copy Right All' options.