Rebuilding a Data File While Eliminating Corrupt Data
Article ID: 65858
Updated On: 08-27-2018
Products
Issue/Introduction
Although the best solution to deal with corrupt data is to restore a copy of the database taken prior to the corruption, this is simply not always possible. Backups should be taken frequently with the CA Ramis BACKUP utility. This will ensure a good copy of the database is available should a problem occur. So what can you do if you don't have a good backup of a corrupted file? Several methods of determining corruption, salvaging data and rebuilding a file may be appropriate depending upon the file structure and the location of the bad data. This article provides one simple example of how to pinpoint corrupt data and how to rebuild a data file when a backup is unavailable.
Environment
Resolution
Once it has been determined that a file is corrupted, you will want to pinpoint the location of the corrupt data to allow you to retain as much of the good data as possible. Generally, if a file has been corrupted, an error will be encountered when accessing the file. (The error could be an abend or any of the following messages: SY0020, SY0021, SY0025, SY0026, SY0030, SY0130, SY0140, ...)
In this example, let's assume that we have discovered that our sample file PRODINFO is corrupted. PRODINFO is a 2-level file with 8 fields. We can display this information by issuing the command RAMINDEX PRODINFO and reviewing the output.
DESCRIPTION OF CA-RAMIS FILE: PRODINFO PAGE 1 L E V E L SEGMENT LIST FIELDNAME SYNONYM NAME TYPE FACTOR FORMAT ---- --------- ------- --------- ------- ------ 1 PRODNUM PN 001 D 1 A 6 2 PRODUCT PNAME 001 D 1 A 20 3 PRODTYPE PTYPE 001 D 1 A 11 4 DISCOUNT DCLASS 001 D 1 A 2 5 MONTH YM 002 S 12 I 4 6 LISTPRICE LIST 002 S 12 P 10.2 7 STDCOST UCOST 002 S 12 P 10.2 8 PLANQTY PLAN 002 S 12 I 6
The first thing we need to identify is where the corruption exists. To begin, we will issue several TABLEF requests beginning with the first level.
TABLEF
FILE PRODINFO
PRINT PRODNUM AND PRODUCT AND PRODTYPE AND DISCOUNT
END
This will cause CA Ramis to read every record on the first level of the file. If you get a message indicating there is an error, we know the corruption exists on Level 1. Since this is a multiple level file, if no error occurred on Level 1, we would continue to Level 2 and run a similar request referencing the fields on that level.
TABLEF
FILE PRODINFO
PRINT MONTH AND LISTPRICE AND STDCOST AND PLANQTY
END
Once we have identified the Level on which the corruption exists, we can try to isolate the damaged data. We determine what records are affected by including a selection statement (IF or WHERE) in our TABLEF request. Our goal is to be able to test a value greater than and less than the corrupted data. For example, we can see in the RAMINDEX output that PRODNUM is defined as an alpha field with a length of 6. If the corruption exists on Level 1, we might choose to start testing the PRODNUM field at the mid-range of " L "s by using the following request:
TABLEF
FILE PRODINFO
PRINT PRODNUM
IF PRODNUM LT 'L99999'
END
If that runs successfully, we would continue our testing by moving the range of values, perhaps trying the same basic request, with
IF PRODNUM LT 'N99999'
Our goal is to find the range of values that cause the request to fail. Ultimately, we will end up with 2 tests, a GREATER THAN and a LESS THAN test. For our example, let's say when we run the following test, we receive the message SY0020: Incorrect linking address.
TABLEF
FILE PRODINFO
PRINT PRODNUM
IF PRODNUM LT 'U99999'
END
Based on the output from our earlier tests, this tells us the bad data exists in a record with a PRODNUM that is greater than the character string 'N99999', but less than the string 'U99999'. We can continue trying to isolate the data from both directions by testing for a value with both GREATER THAN and LESS THAN tests. In our sample case, let's say we are able to successfully run the following 2 requests:
TABLEF TABLEF FILE PRODINFO FILE PRODINFO PRINT PRODNUM PRINT PRODNUM IF PRODNUM LT 'S99999' IF PRODNUM GT 'TB1111' END END
This has narrowed the data we will be eliminating to ONLY those records with a PRODNUM between S99999 and TB1111. Now that we've identified the corrupted data, what do we do about it? Well, there is nothing we can do to correct data once it has been corrupted. However, we can rebuild the database WITHOUT the bad data, thus minimizing the loss.
The following steps should be followed in order, and done precisely as described here. Before beginning, you will need to define three RAMSAVBx files and two separate RAMBACKU files, according to your operating system requirements. We want to create separate backups: one at the very beginning, which will contain the database with the corrupted data, and one in step 5, after the file with the bad data has been erased. The RAMSAVBx files may use any three unique suffix characters. For this example we will use F, 1 and 2.
For additional information on how to define these files to the host operating system, please refer to the CA Ramis publication Datasets.
| Step 1) | Create a CA Ramis BACKUP of the bad database. BACKUP |
| Step 2) | RDUMP the file description for the damaged file (PRODINFO) from RAMASTER using the RDUMPF command. Notice we are using the suffix "F" on the RDUMP command. Since we will be generating three RAMSAVBx files using three RDUMPxcommands, it is important to remember to use the suffix. This will prevent the multiple RDUMPx activities from over-writing the same dataset.
RDUMPF |
| Step 3) | Perform 2 RDUMPx commands to capture as much data as possible from the damaged PRODINFO file. Again, we are using a suffix, in this case 1 and 2,to ensure separate RAMSAVBx files are being created.
RDUMP1 RDUMP2 FILE PRODINFO FILE PRODINFO IF PRODNUM LT 'S99999' IF PRODNUM GT 'TB1111' END END |
| Step 4) | Use the ERASE * command to remove the file definition and data for the PRODINFO file. Although not required, it is usually a good idea to run a RAMCLEAN utility after deleting data from the database.
ERASE* |
| Step 5) | Using the BACKUP utility, create a second backup copy of the damaged database. This copy will NOT have the bad data included. (Remember this backup should not overwrite the backup taken in Step 1.)
BACKUP |
| Step 6) | Create a new version of the database by issuing the RELOAD command, to reload the backup created in Step 5.
USE 1 |
| Step 7) | Add the file description for PRODINFO into the newly created database:
REVISE |
| Step 8) | Load the data into file PRODINFO. This will be done as 2 steps, since we have 2 separate RAMSAVB files created from Step 3, RAMSAVB1 and RAMSAVB2.
REVISE REVISE INCLUDE FILENAME INCLUDE FILENAME JUSTIFIED JUSTIFIED BINARY BINARY NATURAL FORM NATURAL FORM SEARCH FORWARD SEARCH FORWARD READ RAMSAVB1 READ RAMSAVB2 FILE PRODINFO FILE PRODINFO END END |
| Step 9) | Run the following TABLE request to ensure the file description and data has loaded successfully into the "new" PRODINFO file.
TABLEF |
That's all there is to it. Once you have verified the data in Step 9, you should create another final backup of the newly recreated database. As we mentioned earlier, the very best solution is to be sure to create backups frequently so if a problem does occur, you have a recent backup available.
Additional Information
For information on using Records Management commands, refer to the publication CA Ramis Records Management.
For information on using DMF, refer to the publication CA Ramis Data Maintenance Facility.
All CA Ramis and CA Ramis Report manuals can be found in docops.ca.com
https://docops.ca.com/bookshelves/ca-ramis
https://docops.ca.com/bookshelves/ca-ramis-reporter
Feedback
