FDM: Custom Seed list returns all values as NULL

Products

• CA Test Data Manager (Data Finder / Grid Tools)

Issue/Introduction

When running a masking job with HASHLOV and selecting the custom seed list, we are seeing FDM return all NULL values when loading the custom table. The Scramble database is on SQL Server 2019c, and we used the SQL Server Management Studio (SSMS) to manually import the custom seed list into the gtsrc_reference_data table from an Excel file.

With the masking job in debug (diaglevel=5) we see the following in the log file:

2022-03-29 13:24:47.871 loadSeedData p1=CUSTOM_PERSON_NAMES p2=4 
2022-03-29 13:24:47.871 Total Memory: 958 
2022-03-29 13:24:47.871 Free Memory: 870 
2022-03-29 13:24:47.871 Used Memory: 87 
2022-03-29 13:24:47.871 Max Memory: 3556 
2022-03-29 13:24:47.871 loading seed data category CUSTOM_PERSON_NAMES and column 4 at 2022.03.29 13:24:47.871 UTC 
2022-03-29 13:24:47.902 executing seed table sql:select rd_ref_value,rd_ref_value2,rd_ref_value3,rd_ref_value4,rd_ref_value5,rd_ref_value6,rd_ref_value7,rd_ref_value8,rd_ref_value9,rd_index from dbo.gtsrc_reference_data where rd_ref_id = 'CUSTOM_PERSON_NAMES' order by rd_index 
2022-03-29 13:24:47.918 stepping through seed table cursor... 
2022-03-29 13:24:48.058 Seed Values for:CUSTOM_PERSON_NAMES 
2022-03-29 13:24:48.058 0:null 
2022-03-29 13:24:48.058 1:null 
2022-03-29 13:24:48.058 2:null 
2022-03-29 13:24:48.058 3:null 
2022-03-29 13:24:48.058 4:null 
....
2022-03-29 18:54:09.689 49997:null 
2022-03-29 18:54:09.689 49998:null 
2022-03-29 18:54:09.689 49999:null

The data contained in the Excel file looks like:

rd_ref_id	rd_ref_value	rd_old_value	rd_ref_value2	rd_ref_value3	rd_ref_value4	rd_ref_value5	rd_ref_value6	rd_ref_value7	rd_ref_value8	rd_ref_value9	rd_index
CUSTOM_PERSON_NAMES	Dante		Brown	Smith	Smith, Dante Brown	Dante B Smith	Dante B. Smith	Dante Brown Smith	Dante Smith	[e-mail]	71186
CUSTOM_PERSON_NAMES	Iluminada		Roberts	Johnson	Johnson, Iluminada Roberts	Iluminada R Johnson	Iluminada R. Johnson	Iluminada Roberts Johnson	Iluminada Johnson	[e-mail]	71187
CUSTOM_PERSON_NAMES	Apolonia		Turner	Williams	Williams, Apolonia Turner	Apolonia T Williams	Apolonia T. Williams	Apolonia Turner Williams	Apolonia Williams	[e-mail]	71188

 

Environment

Release : 4.10

Component : Fast Data Masker

Cause

The issue is related to the rd_index values for the 'CUSTOM_PERSON_NAMES' category/group. Looking at the sample of the excel file, the rd_index values start at 71,186. The rd-index value is not a unique index value for the entire table. Instead, each category/group will start at 1 for the group's indexing. By starting the rd_index for this group starts at 71,186, which is greater than 50,000, when loading the table we have NULL values for 1-50,000 because those index values do not exist.

Resolution

To resolve this, modify the rd_index values for 'CUSTOM_PERSON_NAMES' category/group, and start the indexing at 1 - 50,000.

 

Additional Information

The preferred method is to use Datamaker to add your custom tables. Datamaker will handle the indexing for you.

See How to Add a Custom Seed List into TDM Portal for adding a custom seed table through Datamaker.