FDM - Large Table Processing

Products

• CA Test Data Manager (Data Finder / Grid Tools)

Issue/Introduction

Enhancement Request for FastDataMasker Large Table Processing:

US619722 - Multithread large table processing - DB2 Support

Environment

Release : 4.8

Component : CA Test Data Manager

Resolution

Provided FastDataMasker-4.8.149.0

We have introduced a new feature in FastDataMasker by improving masking performance.
Very large table masking with primary key or unique index has been improved by adding two new options

LARGETABLESPLITENABLED => Define this parameter to enable large tables processing. Default N
LARGETABLESPLITSIZE => Define the minimal number of rows to start the use large table processing. Default is 1000000.

Large table processing will then be done by generating several blocks with each block containing LARGETABLESPLITSIZE rows to be processed.

The existing option PARALLEL defines the number of threads that can run concurrently to process the blocks.
If the option is not set and LARGETABLESPLITENABLED is enabled then PARALLEL is set to 10.

If there are more blocks than threads then remaining blocks will be queued for processing when a thread becomes idle.