FDM: Custom Seed list returns all values as NULL
Article ID: 238872
Updated On:
Products
Issue/Introduction
When running a masking job with HASHLOV and selecting the custom seed list, we are seeing FDM return all NULL values when loading the custom table. The Scramble database is on SQL Server 2019c, and we used the SQL Server Management Studio (SSMS) to manually import the custom seed list into the gtsrc_reference_data table from an Excel file.
With the masking job in debug (diaglevel=5) we see the following in the log file:
2022-03-29 13:24:47.871 loadSeedData p1=CUSTOM_PERSON_NAMES p2=4
2022-03-29 13:24:47.871 Total Memory: 958
2022-03-29 13:24:47.871 Free Memory: 870
2022-03-29 13:24:47.871 Used Memory: 87
2022-03-29 13:24:47.871 Max Memory: 3556
2022-03-29 13:24:47.871 loading seed data category CUSTOM_PERSON_NAMES and column 4 at 2022.03.29 13:24:47.871 UTC
2022-03-29 13:24:47.902 executing seed table sql:select rd_ref_value,rd_ref_value2,rd_ref_value3,rd_ref_value4,rd_ref_value5,rd_ref_value6,rd_ref_value7,rd_ref_value8,rd_ref_value9,rd_index from dbo.gtsrc_reference_data where rd_ref_id = 'CUSTOM_PERSON_NAMES' order by rd_index
2022-03-29 13:24:47.918 stepping through seed table cursor...
2022-03-29 13:24:48.058 Seed Values for:CUSTOM_PERSON_NAMES
2022-03-29 13:24:48.058 0:null
2022-03-29 13:24:48.058 1:null
2022-03-29 13:24:48.058 2:null
2022-03-29 13:24:48.058 3:null
2022-03-29 13:24:48.058 4:null
....
2022-03-29 18:54:09.689 49997:null
2022-03-29 18:54:09.689 49998:null
2022-03-29 18:54:09.689 49999:null
The data contained in the Excel file looks like:
| rd_ref_id | rd_ref_value | rd_old_value | rd_ref_value2 | rd_ref_value3 | rd_ref_value4 | rd_ref_value5 | rd_ref_value6 | rd_ref_value7 | rd_ref_value8 | rd_ref_value9 | rd_index |
| CUSTOM_PERSON_NAMES | Dante | Brown | Smith | Smith, Dante Brown | Dante B Smith | Dante B. Smith | Dante Brown Smith | Dante Smith | [email protected] | 71186 | |
| CUSTOM_PERSON_NAMES | Iluminada | Roberts | Johnson | Johnson, Iluminada Roberts | Iluminada R Johnson | Iluminada R. Johnson | Iluminada Roberts Johnson | Iluminada Johnson | [email protected] | 71187 | |
| CUSTOM_PERSON_NAMES | Apolonia | Turner | Williams | Williams, Apolonia Turner | Apolonia T Williams | Apolonia T. Williams | Apolonia Turner Williams | Apolonia Williams | [email protected] | 71188 |
Environment
Release : 4.10
Component : Fast Data Masker
Cause
The issue is related to the rd_index values for the 'CUSTOM_PERSON_NAMES' category/group. Looking at the sample of the excel file, the rd_index values start at 71,186. The rd-index value is not a unique index value for the entire table. Instead, each category/group will start at 1 for the group's indexing. By starting the rd_index for this group starts at 71,186, which is greater than 50,000, when loading the table we have NULL values for 1-50,000 because those index values do not exist.
Resolution
To resolve this, modify the rd_index values for 'CUSTOM_PERSON_NAMES' category/group, and start the indexing at 1 - 50,000.
Additional Information
The preferred method is to use Datamaker to add your custom tables. Datamaker will handle the indexing for you.
See How to Add a Custom Seed List into TDM Portal for adding a custom seed table through Datamaker.
Feedback
