JSON file masking is not working

Products

• CA Test Data Manager (Data Finder / Grid Tools)

Issue/Introduction

We are trying to mask a JSON File (Converted from a Parquet file) and the file is not read correctly. JSON Body has Multiple components, but none of those nested fields are being read properly by FDM. If we go ahead and mask one of the attributes in the JSON file, it does not error out. But, when we check the results, nothing is being masked.

The JSON file contains four records (nested JSON file), each on their own line.

 

Environment

Release : 4.10

Cause

The reason you're seeing the unexpected behavior is that the JSON file is not in a valid format. 

Resolution

At this time, FDM does not support nested JSON files. When masking a nested JSON file (JSON array), FDM will only mask the first array, and not pick up the remaining arrays in the file.

To make the file a valid JSON file, you need to put the contents of the file inside rectangular/square brackets in order for this to be picked up properly as a JSON array (nested JSON files). The array elements should be separated by commas. For example [{Array1},{Array2},{Array3},{Array4}].

Many customers use a pre-script to correct the JSON format, putting in the required brackets and commas, so that the JSON array is valid.

We also noticed that the body parameter was still reading as a string. To correct that, we had to remove some unnecessary backslashes (\), and quotes (").

It is always a good idea to check the JSON file syntax with a JSON checker/validator before attempting to mask.

A better workaround for JSON arrays, or nested JSON files, is to export the data to a CSV file. If you're not able to export to CSV, then you will need to separate out the JSON array/nested JSON into separate (individual) JSON files.