DB00110W FORCED SPILL, DB00109W BACKOUT WAITING & WAIT E/C DBSTXB ROLBK Status

Products

Datacom Datacom/DB Datacom/AD

Issue/Introduction

Some long-running jobs with many add, update or delete maintenance transactions can cause a log file (LXX) to fill up. In the event the LOG reaches 100% full, or the value of the MUF Startup Option LOGSPILL “d” value is reached, the older Log file entries are “Force Spilled” to the Recovery File (RXX), and a message is issued in the MUF log:

DB00110W - FORCED SPILL  JOB-jjjjjjjj NUMBER-nnnnn  TSN-xxxxxxxx

When using the recommended default setting of “RXXROLLBACK YES” in the MUF Startup Options, a situation could arise where this long-running job fails and needs to rollback to undo the changes. When the rollback encounters a forced spill, a message is written to the MUF log and displayed on the console:

DB00109W - BACKOUT JOB jjjjjjjj NUMBER-nnnnn TSN-xxxxxxxx WAITING TO COMPLETE.

In this scenario, the backout process cannot complete, because all the transactions are not in the LXX, having been SPILLed previously to one or more RXX files. A STATUS command will show the following for this job:

WAIT E/C DBSTXB ROLBK

Environment

Component : CA Datacom/DB

Component : CA Datacom/AD

Resolution

To clear the DB00109W message, execute the following procedure to undo all the changes for this task/jobstep.

Be sure that there has been an LXX Spill to the RXX following the DB00109W message, or manually run a SPILL job.
Note the TSN value in the DB00109W message as shown above.
Execute DBUTLTY with the function RECOVERY OPTION=BACKWARD,TSN=xxxxxxxx,…, specifying all the RXX files from the one preceding the problem job through the one taken in Step 1, above. The files must be listed in the RXX DD concatenation from the oldest to the most recent. Please refer to the job sample below.
Once the RECOVERY completes successfully, the STATUS function should show that the problem job is no longer running. If the DB00109W message has been displayed on the console, and once the recovery is done, this message must be manually cleared.

Due to the complex nature of logging and recovery, and due to the possibility of encountering other problems in this process, this article addresses only the basic operation. If you need to process a Backward Recovery for your system, and have any question about what to do, contact Broadcom Support for assistance.

Here is a sample jobstep for the Backward Recovery:

//DBUTLTY  EXEC PGM=DBUTLTY,REGION=6M
//STEPLIB  DD DSN=CAI.DATACOM.CUSLIB,DISP=SHR     <<< Change these
//         DD DSN=CAI.DATACOM.CAAXLOAD,DISP=SHR
//SYSOUT   DD SYSOUT=*
//SYSPRINT DD SYSOUT=*
//SYSPUNCH DD DUMMY
//SYSUDUMP DD SYSOUT=*                           +--- Change this
//SYSIN    DD  *                                /
RECOVERY OPTION=BACKWARD,MISMATCH=FAIL,TSN=3CA32D1B,SORT=9999999999,
         OPTION1='(I)' 
/*
//RXX  DD  DISP=SHR,DSN=<<oldest RXX file  e.g. RXX.G0123V00>>
//     DD  DISP=SHR,DSN=<<next RXX file    e.g. RXX.G0124V00>>
//     DD  DISP=SHR,DSN=<<next RXX file    e.g. RXX.G0125V00>>
//     DD  . . .

Additional Information

For an overview of the Recovery process, please refer to Knowledge Base article 18722, titled "An overview of CA Datacom Forward and Backward Recovery processing."

For more information about Logging and the SPILL functionality with the RXX, about the Log file becoming full, about a Forced Spill and about the Recovery process, please refer to the links below.

As always, please contact Broadcom support for CA Datacom if you have further questions.