FDM - Large Table Processing

Article Id: 141316

Status: Published

Updated On: 05-12-2019 12:45

Products:

CA Test Data Manager (Data Finder / Grid Tools)

Issue/Introduction:

Enhancement Request for FastDataMasker Large Table Processing:

US619722 - Multithread large table processing - DB2 Support

Environment:

Release : 4.8

Component : CA Test Data Manager

Resolution:

Provided FastDataMasker-4.8.149.0

We have introduced a new feature in FastDataMasker by improving masking performance.
Very large table masking with primary key or unique index has been improved by adding two new options

LARGETABLESPLITENABLED => Define this parameter to enable large tables processing. Default N
LARGETABLESPLITSIZE => Define the minimal number of rows to start the use large table processing. Default is 1000000.

Large table processing will then be done by generating several blocks with each block containing LARGETABLESPLITSIZE rows to be processed.

The existing option PARALLEL defines the number of threads that can run concurrently to process the blocks.
If the option is not set and LARGETABLESPLITENABLED is enabled then PARALLEL is set to 10.

If there are more blocks than threads then remaining blocks will be queued for processing when a thread becomes idle.