Using a large number of datasets with VISION:Results
Article ID: 57013
Updated On:
Products
Issue/Introduction
There are 8 Input and 8 Output standard datasets available in VISION:Results.
When more datasets are needed, EXIT DYLIOU and Dynamic allocation of datasets can be used. However there are other limits when using the EXIT DYLIOU which must be considered. This document summarizes known aspects in this area.
Resolution
Following are known limitations:
A) IEF649I EXCESSIVE NUMBER OF DD STATEMENTS
According to the IBM MVS JCL Reference Guide, a maximum of 3273 DD statements are allowed in single JOB STEP.
B) IEA705I ERROR DURING GETMAIN SYS CODE = 878-10 xxxxxxxx
The allocation of buffers takes place during each OPEN. All VISION:Results allocations are done below 16MB line. The size of each allocated buffer is BLKSIZE x BUFNO (blocksize multiplied by the number of buffers).
C) DYL-628E UNABLE TO ALLOCATE WORK AREA
There are other factors influencing the available WORKAREA size in addition to the allocation of variables in WORKAREA. VISION:Results generates code internally. Such generated L parameters use WORKAREA and unused OUTPUT FILE areas. This means that the available space may range from minimum 32,760 to maximum 294,840 (9 x 32,760) bytes.
When EXIT DYLIOU is specified on the FILE STATEMENT, 16 additional bytes of control parameters are allocated for the dataset record area. The number of FILE statements for OUTPUT not using the EXIT DYLIOU reduces the available space by 32,760 bytes each. The number of FILE statements with EXIT DYLIOU reduces the available space by the LRECL+16 (logical record length plus 16 bytes) which is usually significantly less than 32,760.
The QLF feature has the same effect of reducing the available WORKAREA.
Circumvention methods:
When encountering the above known limits the first step is to review both the VISION:Results and generated VISION:Sixty source code.
- This review may reveal if there are some unused fields.
- All output files should use EXIT DYLIOU.
- The number of needed record areas may not be the same as number of the processed datasets.
- Avoiding use of QLF may save some space.
1) Consider splitting the JOB or STEP
Some problems may disappear by splitting the JOB or STEP. This may not necessarily increase total processing time.
2) Change PARALLEL to SERIAL approach
Serial approach involving Dynamic Allocation enables the user to avoid reaching the above mentioned limits. Dynamic Allocation is discussed in the VISION:Results for z/OS Guide Reference Guide page 317. Usually the SORT is involved either within VISION:Results code or in a separate step.
Advantage:
No need of the DD statement for each dataset.
No need of the record area for each datasets.
The sorted data may significantly decrease processing time during subsequent steps.
Disadvantage:
The quantity of data may prevent use of this method due to the requirement to allocate working datasets for SORT.
Serial approach example:
//TAPEIN DD DISP=SHR,DSN=XXXXXXXX.XXXXXXXY
OPTION STRUCTURED2
FILE TAPEIN STATUS INSTAT FB 2785 COUNT INCOUNT
INREC 2785 1
CELL 5 151
FILE OUTPUTX OUTPUT FROM OUTPUTX STATUS OUT1STA FB 2785
EXIT DYLIOU DYNAM
OUTRECX 2785 1
WORKAREA
OUTJCL 320 1
FILLER 80 1
VALUE '//OUTPUTX DD DCB=(RECFM=FB,LRECL=2785,BLKSIZE=0),'
FILLER 80
VALUE '// DISP=(,CATLG,DELETE),FREE=CLOSE,'
FILLER 80
VALUE '// SPACE=(TRK,(1,1),RLSE),UNIT=DISK,'
FILLER 7 VALUE '// DSN='
OUTDSNY 44 VALUE ' '
FILLER 29 VALUE ' '
WORKAREA
OUTDSNX 44
OUTDSNOLD 44 VALUE 'X1X'
OUTDSN1 44 VALUE 'XXXXXXXX.XXXXXXXX'
. . .
SORT TAPEIN USING CELL
MOVE OUTDSN1 TO OUTDSNX
PERFORM WRITEX TO EX_WRITEX
WRITEX:
IF OUTDSNOLD NE OUTDSNX
IF OUTDSNOLD NE 'X1X'
CLOSE OUTPUTX
ENDIF
MOVE OUTDSNX TO OUTDSNY
ALLOCATE FILE DYNAMICALLY OUTJCL STATUSFLAG JCLSTAT
IF JCLSTAT EQ 'N'
PRINT 'ERROR IN DYNAMICALLY ALLOCATED JCL'
STOP
ENDIF
ENDIF
MOVE INREC TO OUTRECX
WRITE OUTPUTX
MOVE OUTDSNY TO OUTDSNOLD
EX_WRITEX:
3) Use PARM in FILE Statement with EXIT DYLIOU
This method provides information to EXIT DYLIOU by passing parameters. It is the user’s responsibility to fill them properly. The FILE statement has the record length set to 1 so the allocation for each FILE statement is only 17 bytes. The parameter resets the value of the record length to proper value when it is passed to DYLIOU. Standard READ and WRITE statements are used for I/O. An implicit CLOSE of datasets takes place at the end.
More about the EXIT DYLIOU and PARM is described in VISION:Results for z/OS Guide Reference Guide page 131.
Parameters for DYLIOU are:
- THE ADDRESS OF A RECORD AREA SUPPLIED BY THE USER
- THE ADDRESS OF A 1-BYTE STATUS INDICATOR
- SET TO 'E' AT END-OF-FILE (INPUT)
- VALUE OF 'E' (OUTPUT) = CLOSE THE FILE
- THE ADDRESS OF AN AREA DESCRIBING THE FILE:
1-8 - DDNAME
9-10 - RECFM (F , FB, V , VB)
11-12 - LRECL, BINARY
13-14 - BLKSIZE, BINARY, USED ON OUTPUT IF NOT IN JCL
15 - I (INPUT) OR O (OUTPUT)
PARM in FILE statement example:
//OUTPUT1D DD DSN=XXXXXXXX.XXXXXXXX,DISP=(,CATLG,DELETE),
// DCB=(RECFM=FB,LRECL=2785,BLKSIZE=0,BUFNO=2),UNIT=DISK,
// SPACE=(TRK,(1,1),RLSE)
FILE OUTFILE OUTPUT FROM OUTFILE FB 1
EXIT DYLIOU PARM (INRECC, OUTSTAT, OUTDD)
FILE OUTPUT1 OUTPUT FROM OUTPUT1 FB 1 EXIT DYLIOU
WORKAREA
OUTSTAT 1 VALUE ' '
*
OUTDD 8 VALUE 'OUTFILE'
OUTRECFM 2 VALUE 'FB'
OUTLRECL 2 BI VALUE 2785
OUTBLK 2 BI VALUE 0
OUTIOFNC 1 VALUE 'O'
MOVE 'OUTPUTxx' TO OUTDD
WRITE OUTFILE
4) USE direct CALLs to DYLIOU Program
Instead of the standard READ and WRITE statements, CALL statements are used for I/O. Explicit CALL statements to perform the CLOSE of datasets must be coded.
In this case DYLIOU does not know how many different files are to be processed. If the number of files is greater than the default of 175, then the MAXDYLF must be set by DYLINSTL Macro.
The advantage of this method is that no data fields are allocated implicitly.
Brief description of DYLIOU functionality:
DYLIOU allocates a DCB, opens the file, and reads or writes the first record on the first call for any given DDname.
Each subsequent call for the same DDname results in another record being read from, or written to that file.
When end-of-file is sensed on input, an 'E' is moved to the status flag, the file is closed. For any file (input or output), if the status flag is an 'E', the file is closed and its buffer(s) are freed.
If only one address is passed to the routine (x'80' in byte 1) no records are read or written. Instead, all open files are closed, their buffers released, and the storage obtained for the DCBs is freed.
A record count is maintained in the file table, for each file up to 4095.
Direct CALLs to DYLIOU Program example:
//OUT0001 DD DSN=XXXXXXXX.TESTxxxx,DISP=(,CATLG,DELETE),
// DCB=(RECFM=FB,LRECL=80,BLKSIZE=400,BUFNO=2),
// SPACE=(TRK,(1,1),RLSE),UNIT=DISK
WORKAREA
OUTSTATUS 1 VALUE ' '
*
OUTDD 8 VALUE 'OUTXXXXX'
OUTRECFM 2 VALUE 'FB'
OUTLRECL 2 BI VALUE 80
OUTBLK 2 BI VALUE 0
OUTIOFNC 1 VALUE 'O'
*
CLOSEALL 1 VALUE X'80'
MOVE 'OUT0001' TO OUTDD
CALL DYLIOU USING INREC, OUTSTATUS, OUTDD
CALL DYLIOU USING CLOSEALL
Feedback
