search cancel

TDM Portal - Data getting mixed up when using a generator to copy data

book

Article ID: 190667

calendar_today

Updated On:

Products

CA Test Data Manager (Data Finder / Grid Tools)

Issue/Introduction

We are using a Generator in TDM Portal to copy data from our Production environment to Staging, and noticed the data is getting mixed up when publishing to staging. For example, we have account IDs, and each account ID has a number of invoice numbers associated to them. When the problem is seen, we noticed that account 1 has some invoices from account 2, and vice versa. 
There are no error in logs in the logs, so we aren't sure how to troubleshoot this and why it is happening?

Environment

Release : 4.6

Component : CA Test Data Manager

Cause

The best way to troubleshoot is to enable Portal in debug.

See KB Article 9851 - "TDM Portal: How to enable debugging for TDM Portal"
https://knowledge.broadcom.com/external/article?articleId=9851

Resolution

When examining the logs, we noticed that the data being inserted is different. This can happen if the SQL expressions used by the Data Generator, has different SQL expressions in each of the columns. In order to ensure the data is found in the same order, you must use the exact same SQL expression for each of the columns.

Looking at the SQL expressions in the Generator where the problem was seen, we found there are differences in the expressions, and thus the mixed results returned.

When the generator is run for the first time, TDM builds the internal lists for each of the SQL expressions used. Because the SQL Engine may not return each of the result sets in the exact same order, it is possible for a divergent in the results, thus a mixed, or scrambled set of data. Therefore, anytime you want to ensure a consistent result set between environments, it is best practice to ensure each of the columns uses the exact same SQL expression. This ensures TDM is working from the same internal table, and not from a mixed number of tables.

One way to ensure all columns for a given Generator is using the same SQL expression, open the Generator in Datamaker, and make sure each column has the exact same expression. An easy way to do this is using the  'Copy Column Right' to 'Copy Right All' options.