Overriding DFSORT parms loading a very large Datacom area
Article ID: 136915
Updated On:
Products
Issue/Introduction
When using DBUTLTY as it is delivered to load a data area, the key records are built and sorted to load the index area. If the product of the number of records and keys exceeds the SORT product limitation, the load function will fail.
If the sort product is IBM's DFSORT, an error message will be produced: ICE121A - FILE SIZE IS TOO LARGE if the file exceeds these limits:
- EQUALS in effect - 4 294 967 295 records
- VLSHRT in effect - 2 147 483 647 records
- Blockset technique not selected - 2 147 483 647 records
For example, if an area has 585,000,000 records, and 7 keys for each record, the load process will try to sort 4 095 000 000 records, which is greater than the 2 gigarecord -1 limit.
Environment
Release : all
Component : DATACOM/DB
Sort product: IBM DFSORT
Resolution
If the product of records and keys is less than 4 gigarecords - 1 (4294967295), but more than 2 gigarecords -1 (2147483647), you can use the DFSORT override file called DFSPARM.
In this case, you would specify SORT=1 and SORTDFLT=YES to allow DBUTLTY to prepare and call the sort with the expected size of the sortfile, either from the backup file, or from the CXX, if the value is available. However, if the sort filesize cannot be calculated, you can also specify the estimated size in the DFSPARM file. To use this larger limit, you need to specify the EQUALS option in DFSPARM.
Here is an example of a DBUTLTY LOAD statement:
FUNCTION=LOAD
AREA=ABC
DBID=00123
DDNAME=MYBKUP
FORMAT=EXTRACT
OPTIMIZE=YES
OPTION1=(I)
SORT=00000000001
SORTDFLT=YES
SORTWK=15
TABLE=ABC
And a corresponding DFSPARM file:
//DFSPARM DD *
OPTION EQUALS
FILSZ=E4300000000
/*
From the output, here is the calculation provided by SORTDFLT=YES:
-SORTDFLT INFORMATION
TABLE ID BACKUP DIRECTORY KEYS ESTIMATE
ABC 6 0 585,000,000 7 4095,000,000
*DSOP 280,124
TOTAL (SORT ESTIMATE OVERRIDE) 4095,280,124 SORT=04095280124
In this case, the FILSZ parameter was not needed in DFSPARM, but was still provided. DFSPARM settings will override the DFSORT control statements, as shown here:
SORT FILSZ=E04095280124,FIELDS=(1,0044,BI,A),DYNALLOC=(SYSDA,15)
RECORD TYPE=F,LENGTH=0044
DEBUG NOABEND
And then these messages, showing that EQUALS was used, and the FILSZ value was used:
. . .
ICE128I 0 OPTIONS: SIZE=134217728,MAXLIM=1048576,MINLIM=450560,EQUALS=Y,LIST=Y,ERET=RC16 ,MSGDDN=SYSPRINT
. . .
ICE253I 0 RECORDS SORTED - PROCESSED: 4095000000, EXPECTED: 4300000000
. . .
Note that if your calculated filesize is still more than 4 gigarecords - 1, you will need to consider other methods to manage your data (archiving data, partitioning the table, removing some keys, etc.), or if you do not care about preserving the input order for records with duplicate keys, you can use NOEQUALS. Also, if you are running on Z15 and higher you can use sort accelerator (with OPTION ZSORT) which does NOT have that limit.
Additional Information
For more information on the DBUTLTY LOAD function, please refer to the Datacom Core 15.1 documentation, LOAD data area function of DBUTLTY
For more information about the IBM Integrated Accelerator for Z Sort on the z15 processor, please see the z/OS documentation for DFSORT.
For more information on the DFSORT items discussed above, please refer to the following:
z/OS 2.5 DFSORT Application Programming Guide
z/OS 2.4 DFSORT Application Programming Guide
z/OS 2.3 DFSORT Application Programming Guide
z/OS 2.2 DFSORT Application Programming Guide
z/OS 2.1 DFSORT Application Programming Guide
Sections used from search are:
- DFSPARM data set
- ICE121A
- OPTION control statement
- Specification/override of DFSORT options
As always, please contact Broadcom support for Datacom if you have further questions.
Feedback
