When PLOG processing is being considered by a customer interested in Change Data Capture (CDC) against ADABAS, it should be understood from the outset that PLOG processing will be an ongoing activity.  Every transaction that occurs within an ADABAS database will have to be at least cursorily examined to see whether it is a transaction against the file or files of interest. For all transactions selected, each transactional record will  need to be decompressed.

While other processing will be involved in properly handling PLOG data, the decompression processing is particularly resource intensive.  As a result, it is highly advisable for any ADABAS customer who is contemplating doing ongoing PLOG / CDC processing to have an understanding of how long CDC processing might take, and how to schedule this processing into the available batch time windows.

We have prepared this page for the expressed purpose of demonstrating the performance characteristics of the NatCDC / NatCDCSP solution, and we invite you to compare this performance information to other available solutions.

The following execution times were captured at one of our existing NatCDC customers who supplied us with these times from one of their worst / busiest transactional days that they had in recent history.  This was a day where their primary production ADABAS databases created Twenty-two (22) separate PLOG tape cartridges.

These tapes were then processed in the following manner:

1. All Twenty-two (22) tapes were initially processed through the ADASEL utility, and this utility was instructed to capture changes that occurred against four (4) files that required CDC processing and to split the individual transactions from each of these four files into their own respective dataset.  The results from this execution can be seen in the following section entitled ADASEL Execution Results.
    
2. Each of the 4 datasets were then simultaneously processed using NatCDC-generated processing to drop unneeded fields, place the output into a fixed-length format, sort this fixed-length output, produce a “Delta” of the transactional changes for each individual record, and perform any required ADABAS to ASCII data transformations.  The results from these executions can be seen in the following section entitled NatCDC Processing Results.
    

As can be seen in the following results, ADASEL selected 940,363 transactional records that were associated with the 4 source files from the Twenty-two (22) tape cartridge input, and then decompressed all 940,363 transactional records into 15,800 cylinders in 110.26 Minutes.  If we ignore the processing time required to read the Hundreds of Thousands of transactional records that were not of interest to CDC processing (and were therefore bypassed), as well as time lost waiting for possible tape mounts, we can roughly calculate ADASEL’s ability to select and decompress transactional records of interest at being approximately 142 transactional records a second.

For the remaining processing, we can see a range of between 309 records per second to 6,152 records per second. 

With these numbers as a benchmark, this customer was able to process their worst transactional day against the four (4) files of interest in approximately 3.5 hours, with virtually all of this being sequential processing that can allow other ADABAS activities to continue.

ADASEL EXECUTION RESULTS

Execution Time

110:26 MINUTES EXECUTION TIME (Total - Minutes:Seconds)

Disk Activity:

ADAI03 DDASSOR1        13 READS         0 WRITES
ADAI01 DDSIIN      418930 READS
ADAI01 DDEXPA1     615269 WRITES
ADAI01 DDEXPA2     269693 WRITES
ADAI01 DDEXPA3      51537 WRITES
ADAI01 DDEXPA4       3864 WRITES

Disk Space Usage:

DDEXPA1 - 11,340 Cylinders
DDEXPA2 -  4,251 Cylinders
DDEXPA3 -    202 Cylinders
DDEXPA4 -      7 Cylinders

Note:
The DDEXPA1, DDEXPA2, DDEXPA3 and DDEXPA4 references shown above equate to each of the four files being captured.

[Back to Top]

NatCDC Processing Results
Subsequent to the execution of ADASEL, each output file was then concurrently processed through a NatCDC generated JCL stream which performed 3 steps (a step-by-step description of the steps involved can be found on the NatCDC Operation and Architecture Page):

• JOBSTEP1 -Execution of the NatCDC module
  This NatWorks’ utility program drops unwanted fields from the transactional records and places the remaining fields into a fixed length format (PE fields and MU fields are specified with a fixed upper-limit of occurrences).
• JOBSTEP2 - Execution of a System SORT
  The output of NatCDC is processed through a System Sort utility to sort the records into ascending sequence by the first 12 bytes of each record (which corresponds to each record’s ISN and the transaction’s Date / Timestamp).
• JOBSTEP3 - Execution of a NatCDC-generated Natural Program
  The output of the SORT is then processed by a NatQuery generated Natural program which analyzes the records using Break Logic to deduce the transaction type (Store, Update or Delete) and provide the transaction “Delta” and additionally handles the transformation of ADABAS data types into ASCII equivalents.

The above processing, when applied to each of the four files of interest, yielded the following results:

DDEXPA1 - Transaction File Processing

Execution Time:

100.04 MINUTES EXECUTION TIME
       JOBSTEP1     83:42 (Minutes:Seconds)
       JOBSTEP2      3:54 (Minutes:Seconds)
       JOBSTEP3     12:24 (Minutes:Seconds)

Execution Report (Partial):

    TOTAL NUMBER OF RECORDS READ FROM INPUT:      615269
        TOTAL NUMBER OF RECORDS WRITTEN OUT:      292593
               TOTAL UNIQUE RECORDS HANDLED:      285738
              TOTAL NUMBER OF BEFORE IMAGES:      307185
               TOTAL NUMBER OF AFTER IMAGES:      308084

Disk Space Utilization:

       Disk Space Read:  11,240 Cylinders
Intermediate Disk Used:     892 Cylinders
 Final Disk Space Used:     425 Cylinders (LRECL 573)

DDEXPA2 - Transaction File Processing

Execution Time:

87.55 MINUTES EXECUTION TIME
      JOBSTEP1      66:46 (Minutes:Seconds)
      JOBSTEP2       2:59 (Minutes:Seconds)
      JOBSTEP3      18:26 (Minutes:Seconds)

Execution Report (Partial):

    TOTAL NUMBER OF RECORDS READ FROM INPUT:      269693
        TOTAL NUMBER OF RECORDS WRITTEN OUT:      120159
               TOTAL UNIQUE RECORDS HANDLED:      117230
              TOTAL NUMBER OF BEFORE IMAGES:      134567
               TOTAL NUMBER OF AFTER IMAGES:      135117

Disk Space Utilization:

       Disk Space Read:   4,251 Cylinders
Intermediate Disk Used:     295 Cylinders
 Final Disk Space Used:     126 Cylinders (LRECL 249)

DDEXPA3 - Transaction File Processing

Execution Time:

11.94 MINUTES EXECUTION TIME
      JOBSTEP1       3:22 (Minutes:Seconds)
      JOBSTEP2       3:10 (Minutes:Seconds)
      JOBSTEP3       5:24 (Minutes:Seconds)

Execution Report (Partial):

    TOTAL NUMBER OF RECORDS READ FROM INPUT:       51537
        TOTAL NUMBER OF RECORDS WRITTEN OUT:       15243
               TOTAL UNIQUE RECORDS HANDLED:       15183
              TOTAL NUMBER OF BEFORE IMAGES:       21783
               TOTAL NUMBER OF AFTER IMAGES:       29754

Disk Space Utilization:

       Disk Space Read:     202 Cylinders
Intermediate Disk Used:   1,718 Cylinders
 Final Disk Space Used:     509 Cylinders (LRECL 24849)

DDEXPA4 - Transaction File Processing

Execution Time:

1.04 MINUTES EXECUTION TIME
      JOBSTEP1        :20 (Minutes:Seconds)
      JOBSTEP2        :16 (Minutes:Seconds)
      JOBSTEP3        :25 (Minutes:Seconds)

Execution Report (Partial):

    TOTAL NUMBER OF RECORDS READ FROM INPUT:        3864
        TOTAL NUMBER OF RECORDS WRITTEN OUT:        2482
               TOTAL UNIQUE RECORDS HANDLED:        2474
              TOTAL NUMBER OF BEFORE IMAGES:        1385
               TOTAL NUMBER OF AFTER IMAGES:        2479

Disk Space Utilization:

       Disk Space Read:       7 Cylinders
Intermediate Disk Used:      43 Cylinders
 Final Disk Space Used:      28 Cylinders (LRECL 8351)

[Back to Top]
[Back to the NatCDC Information Page]