Date field not masked by FDM in Delimited Flat Files
Article ID: 399798
Updated On:
Products
Issue/Introduction
We are trying to mask a date field in a delimited flat file, using the HASHDOB function. The job run successfully, but when looking at audit log and the created scramble (masked) file, we see the original values.
The file we are trying to mask looks like:
DATE
19880519
19720903
20031004
19680322
19610302
19760305
19840613
19910111
20170615
20240325
The definition file looks like:
HEADER=1,TRAILER=0,DELIM=,,DATEQUOTED=,CHARQUOTED=,NUMQUOTED=
DATE
The Audit log shows:
MAPPING FILE ARCHIVED AS: C:/Users/<user>/AppData/Roaming/Grid-Tools/FastDataMasker/backups/DATE_MASK.20250529095001.csv
OPTIONS:
AUDIT=ROW100
AUDITDIR=E:\CATDM-FDM\DATE_TEST\AUDIT
AUDITFILE=DATE_Audit.csv
BLANKSASNULLS=Y
CASEINSENSITIVEHASHLOV=N
EMPTYASNULL=Y
MASKING STARTED AT: 2025.05.29 09:50:01.237 CDT
LINE,MASK COLUMN,FUNCTION,OLDVALUE,NEWVALUE
2,"DATE","HASHDOB","19880519","19880519"
3,"DATE","HASHDOB","19720903","19720903"
4,"DATE","HASHDOB","20031004","20031004"
5,"DATE","HASHDOB","19680322","19680322"
6,"DATE","HASHDOB","19610302","19610302"
7,"DATE","HASHDOB","19760305","19760305"
8,"DATE","HASHDOB","19840613","19840613"
9,"DATE","HASHDOB","19910111","19910111"
10,"DATE","HASHDOB","20170615","20170615"
11,"DATE","HASHDOB","20240325","20240325"
Environment
All Supported release of Fast Data Masker (FDM)
Cause
The date format is missing from the definition file. When masking a flat file in FDM, you must include the date format for all date fields found in the file. If the date format is missing, FDM treats the field as a string. For more information, see Mask Data Stored in Flat Files.
Resolution
Modify the definition file and include the proper date format for all date related fields.
For example, add the data format in the <defention.dm.txt> file:
HEADER=1,TRAILER=0,DELIM=,,DATEQUOTED=,CHARQUOTED=,NUMQUOTED=
INSURED_PERSONS_BIRTH_DATE,"yyyymmdd"
Using the HASHDOB masking function, the job completed successfully. The <date.txt.scramble> file contained the expected values, as shown below:
Note: when configuring the HASHDOB masking function, you must also include the same date format, so the masking function understands how to format the masked data.
- MASKING_AUDIT.csv:
MAPPING FILE ARCHIVED AS: C:/Users/<user>/AppData/Roaming/Grid-Tools/FastDataMasker/backups/DATE_MASKING.20250603100006.csv
OPTIONS:
AUDIT=ALL
AUDITDIR=C:\Users\<user>\Desktop\<directory>\AUDIT
AUDITFILE=MASKING_AUDIT.csv
LOGDIR=C:\Users\<user>\Desktop\<folder>\LOG
LOGFILENAME=MASKING_LOG.txt
MASKING STARTED AT: 2025.06.03 10:00:06.497 CDT
LINE,MASK COLUMN,FUNCTION,OLDVALUE,NEWVALUE
2,"DATE,"yyyymmdd"","HASHDOB","19880519","19880422"
3,"DATE,"yyyymmdd"","HASHDOB","19720903","19720912"
4,"DATE,"yyyymmdd"","HASHDOB","20031004","20031016"
5,"DATE,"yyyymmdd"","HASHDOB","19680322","19680328"
6,"DATE,"yyyymmdd"","HASHDOB","19610302","19610308"
7,"DATE,"yyyymmdd"","HASHDOB","19760305","19760320"
8,"DATE,"yyyymmdd"","HASHDOB","19840613","19840622"
9,"DATE,"yyyymmdd"","HASHDOB","19910111","19910114"
10,"DATE,"yyyymmdd"","HASHDOB","20170615","20170630"
11,"DATE,"yyyymmdd"","HASHDOB","20240325","20240409"
- DATE.txt.scramble:
DATE
19880422
19720912
20031016
19680328
19610308
19760320
19840622
19910114
20170630
20240409
Please modify your definition file, and include the proper date format. Then try running the masking job again.
Feedback
