Using a large number of datasets with VISION:Results
search cancel

Using a large number of datasets with VISION:Results

book

Article ID: 57013

calendar_today

Updated On:

Products

Vision:Results

Issue/Introduction

There are 8 Input and 8 Output standard datasets available in VISION:Results.

When more datasets are needed, EXIT DYLIOU and Dynamic allocation of datasets can be used. However there are other limits when using the EXIT DYLIOU which must be considered. This document summarizes known aspects in this area.

 

 

Resolution

Following are known limitations:

A) IEF649I EXCESSIVE NUMBER OF DD STATEMENTS

According to the IBM MVS JCL Reference Guide, a maximum of 3273 DD statements are allowed in single JOB STEP.

 

B) IEA705I ERROR DURING GETMAIN SYS CODE = 878-10 xxxxxxxx

The allocation of buffers takes place during each OPEN. All VISION:Results allocations are done below 16MB line. The size of each allocated buffer is BLKSIZE x BUFNO (blocksize multiplied by the number of buffers).

 

C) DYL-628E UNABLE TO ALLOCATE WORK AREA

There are other factors influencing the available WORKAREA size in addition to the allocation of variables in WORKAREA. VISION:Results generates code internally. Such generated L parameters use WORKAREA and unused OUTPUT FILE areas. This means that the available space may range from minimum 32,760 to maximum 294,840 (9 x 32,760) bytes.

When EXIT DYLIOU is specified on the FILE STATEMENT, 16 additional bytes of control parameters are allocated for the dataset record area. The number of FILE statements for OUTPUT not using the EXIT DYLIOU reduces the available space by 32,760 bytes each. The number of FILE statements with EXIT DYLIOU reduces the available space by the LRECL+16 (logical record length plus 16 bytes) which is usually significantly less than 32,760.

The QLF feature has the same effect of reducing the available WORKAREA. 

Circumvention methods:

When encountering the above known limits the first step is to review both the VISION:Results and generated VISION:Sixty source code.

  • This review may reveal if there are some unused fields.
  • All output files should use EXIT DYLIOU.
  • The number of needed record areas may not be the same as number of the processed datasets.
  • Avoiding use of QLF may save some space.

1)     Consider splitting the JOB or STEP

Some problems may disappear by splitting the JOB or STEP. This may not necessarily increase total processing time.

2)     Change PARALLEL to SERIAL approach

Serial approach involving Dynamic Allocation enables the user to avoid reaching the above mentioned limits. Dynamic Allocation is discussed in the VISION:Results for z/OS Guide Reference Guide page 317. Usually the SORT is involved either within VISION:Results code or in a separate step.

Advantage:

No need of the DD statement for each dataset.  

No need of the record area for each datasets.

The sorted data may significantly decrease processing time during subsequent steps.

 

Disadvantage:

The quantity of data may prevent use of this method due to the requirement to allocate working datasets for SORT.

 

Serial approach example:

 

//TAPEIN   DD DISP=SHR,DSN=XXXXXXXX.XXXXXXXY

 

OPTION STRUCTURED2                                   

FILE TAPEIN STATUS INSTAT FB 2785 COUNT INCOUNT      

  INREC  2785    1                                   

  CELL      5  151                                   

 

FILE OUTPUTX OUTPUT FROM OUTPUTX STATUS OUT1STA  FB 2785  

  EXIT DYLIOU DYNAM                                        

  OUTRECX   2785    1                                     

 

WORKAREA                                                            

    OUTJCL 320 1                                                    

    FILLER 80 1                                                      

       VALUE '//OUTPUTX DD DCB=(RECFM=FB,LRECL=2785,BLKSIZE=0),'    

    FILLER 80                                                       

       VALUE '// DISP=(,CATLG,DELETE),FREE=CLOSE,'                  

    FILLER 80                                                       

       VALUE '// SPACE=(TRK,(1,1),RLSE),UNIT=DISK,'                 

    FILLER   7  VALUE '// DSN='                                     

    OUTDSNY 44  VALUE ' '                                            

    FILLER  29  VALUE ' '                                           

WORKAREA

    OUTDSNX   44                                  

    OUTDSNOLD 44 VALUE 'X1X'                      

    OUTDSN1   44 VALUE 'XXXXXXXX.XXXXXXXX'  

 . . .

SORT TAPEIN USING CELL

 

MOVE OUTDSN1  TO OUTDSNX    

PERFORM WRITEX TO EX_WRITEX 

 

WRITEX:

    IF OUTDSNOLD NE OUTDSNX

       IF OUTDSNOLD NE 'X1X'

          CLOSE OUTPUTX

       ENDIF

       MOVE OUTDSNX TO OUTDSNY

       ALLOCATE FILE DYNAMICALLY OUTJCL STATUSFLAG JCLSTAT

       IF JCLSTAT EQ 'N'

          PRINT 'ERROR IN DYNAMICALLY ALLOCATED JCL'

          STOP

       ENDIF

    ENDIF

    MOVE INREC TO OUTRECX

    WRITE OUTPUTX

    MOVE OUTDSNY TO OUTDSNOLD

EX_WRITEX:

 

3)     Use PARM in FILE Statement with EXIT DYLIOU

This method provides information to EXIT DYLIOU by passing parameters. It is the user’s responsibility to fill them properly. The FILE statement has the record length set to 1 so the allocation for each FILE statement is only 17 bytes. The parameter resets the value of the record length to proper value when it is passed to DYLIOU. Standard READ and WRITE statements are used for I/O. An implicit CLOSE of datasets takes place at the end.

More about the EXIT DYLIOU and PARM is described in VISION:Results for z/OS Guide Reference Guide page 131.

 

Parameters for DYLIOU are:

 

  1. THE ADDRESS OF A RECORD AREA SUPPLIED BY THE USER   
  2. THE ADDRESS OF A 1-BYTE STATUS INDICATOR            

    - SET TO 'E' AT END-OF-FILE (INPUT)                 

    - VALUE OF 'E' (OUTPUT) = CLOSE THE FILE            

  1. THE ADDRESS OF AN AREA DESCRIBING THE FILE:         

     1-8  - DDNAME                                      

     9-10 - RECFM (F , FB, V , VB)                      

    11-12 - LRECL, BINARY                                

    13-14 - BLKSIZE, BINARY, USED ON OUTPUT IF NOT IN JCL

      15  - I (INPUT) OR O (OUTPUT)                     

 

PARM in FILE statement example:

 

//OUTPUT1D DD DSN=XXXXXXXX.XXXXXXXX,DISP=(,CATLG,DELETE), 

//            DCB=(RECFM=FB,LRECL=2785,BLKSIZE=0,BUFNO=2),UNIT=DISK, 

//            SPACE=(TRK,(1,1),RLSE)                                 

 

FILE OUTFILE  OUTPUT FROM OUTFILE FB 1              

     EXIT DYLIOU PARM (INRECC, OUTSTAT, OUTDD)      

FILE OUTPUT1  OUTPUT FROM OUTPUT1  FB 1 EXIT DYLIOU 

 

WORKAREA                              

  OUTSTAT   1         VALUE ' '       

*                                     

  OUTDD     8         VALUE 'OUTFILE' 

  OUTRECFM  2         VALUE 'FB'      

  OUTLRECL  2 BI      VALUE 2785      

  OUTBLK    2 BI      VALUE 0         

  OUTIOFNC  1         VALUE 'O'       

 

MOVE 'OUTPUTxx' TO OUTDD

 

WRITE OUTFILE

 

4)     USE direct CALLs to DYLIOU Program

Instead of the standard READ and WRITE statements, CALL statements are used for I/O. Explicit CALL statements to perform the CLOSE of datasets must be coded.

In this case DYLIOU does not know how many different files are to be processed. If the number of files is greater than the default of 175, then the MAXDYLF must be set by DYLINSTL Macro.

The advantage of this method is that no data fields are allocated implicitly.

 

Brief description of DYLIOU functionality:

DYLIOU allocates a DCB, opens the file, and reads or writes the first record on the first call for any given DDname.

Each subsequent call for the same DDname results in another record being read from, or written to that file.

When end-of-file is sensed on input, an 'E' is moved to the status flag, the file is closed.  For any file (input or output), if the status flag is an 'E', the file is closed and its buffer(s) are freed.

If only one address is passed to the routine (x'80' in byte 1) no records are read or written. Instead, all open files are closed, their buffers released, and the storage obtained for the DCBs is freed.

A record count is maintained in the file table, for each file up to 4095.

 

Direct CALLs to DYLIOU Program example:

 

//OUT0001 DD DSN=XXXXXXXX.TESTxxxx,DISP=(,CATLG,DELETE),     

//            DCB=(RECFM=FB,LRECL=80,BLKSIZE=400,BUFNO=2),   

//            SPACE=(TRK,(1,1),RLSE),UNIT=DISK               

 

WORKAREA                                               

  OUTSTATUS 1         VALUE ' '                        

*                                                      

  OUTDD     8         VALUE 'OUTXXXXX'                  

  OUTRECFM  2         VALUE 'FB'                       

  OUTLRECL  2 BI      VALUE 80                         

  OUTBLK    2 BI      VALUE 0                          

  OUTIOFNC  1         VALUE 'O'                        

*                                                       

  CLOSEALL  1         VALUE X'80'                      

 

MOVE 'OUT0001'  TO OUTDD

 

CALL DYLIOU USING INREC, OUTSTATUS, OUTDD  

 

CALL DYLIOU USING CLOSEALL