How Replication Works

The DATABridge Client controls the replication process of DMSII databases and flat files. It initiates connections to DATABridge Server on the host and maintains the state information necessary for resuming replication, in case operations are interrupted. At the heart of the host-based DATABridge system is the DATABridge Engine—a system library that retrieves structural information and data from the DMSII database and passes it to DATABridge Server. When you optionally use Enterprise Server with the DATABridge Client, Enterprise Server takes over much of the functionality of the DATABridge Engine and DATABridge Server.

The DATABridge Support Library, also installed on the host, provides filtering and translation to the DATABridge Server. See Client vs. Host Filtering. Translation is more cost effective using the Client.

After DATABridge Server receives data from the DATABridge Engine, it calls the Support Library to determine if the data should be replicated, and if so, it passes the data to the Support Library for formatting.

Replication involves three discrete phases, as described below. These three phases are tracked for each data set in the ds_mode column of the DATASETS control table as values 0, 1, and 2.

Data extraction

 

This phase (identified by a mode of 0) applies only to data sets that have not yet been cloned. During this phase, the DATABridge Engine sequentially reads all of the records from the data sets being cloned and passes them to the DATABridge Client. Using the appropriate bulk loader utility, the Client populates the relational database tables and creates indexes for the tables.

Fixup

 

During this phase (identified by a mode of 1), the DATABridge Engine processes audit files and passes all of the DMSII updates that occurred while data extraction was taking place to the Client, which updates the relational database. This phase is fully restartable.

The only difference between the Fixup Phase and the Update (or Tracking) Phase is that the Client has to deal with conditions caused by the fact that the tables from which records are extracted are changing as the extraction is taking place. Until the audit file processing gets past the point in the audit trail where the data extraction ends, the Client behaves somewhat differently in order to handle such issues as updates to records that are not in the tables, deletions of records that are not in the tables, and inserts of records that are already in the tables.

Update

 

During this phase (identified by a mode of 2), the Client processes audit files and then passes all of the DMSII database updates to the Client, which updates the relational database. This phase is also referred to as the change tracking phase.

DATABridge uses quiet points to synchronize the replicated database with the DMSII database and ensure accuracy of the data. Quiet points mark the start of a group of updates, which is referred to as a transaction. When the DATABridge Engine reaches the end of the last DMSII audit file (or encounters a program that did a rollback), it usually rolls back the transaction and instructs the Client to roll back updates. The Client stores quiet point information with other state information in a set of control tables, referred to as the Client Control Tables, and uses it to restart the replication process.

If near real-time replication is required, set the use_dbwait configuration parameter to true. This causes the Engine to enter a wait-and-retry loop for a configurable amount of time, instead of returning an audit file unavailable status, which normally occurs when no more audit files are available.