Data Integrity Check Process

1. Purpose of the Data Integrity Check Process:
• The Data Integrity Check Process is responsible for ensuring that the data adheres to predefined constraints and quality checks defined in the Oracle Data Integrator (ODI) model.
• It helps to maintain data accuracy and reliability by verifying that the data conforms to the business rules and constraints set for the source or target datastores.
2. When is the Data Integrity Check Activated?:
• The data integrity check is activated in the following cases:
1. Static Control:
• When a Static Control is triggered (via Studio or a package) on a model, sub-model, or datastore.
• The data in the datastore is checked against the constraints defined in the ODI model.
2. Flow Control:
• When a mapping is executed and Flow Control is activated in the Integration Knowledge Module (IKM).
• The data staged in the integration table (I$) is checked against the constraints of the target datastore as defined in the model.
• Only the constraints that are selected in the mapping are checked during this process.
3. Role of the CKM (Check Knowledge Module):
• The CKM is the component responsible for performing the data integrity checks.
• It can be used in two main contexts:

• Static Control: Checks existing data in the model.
• Flow Control: Checks data flowing through the mapping and staged in the integration table.

• The CKM performs the following tasks:

• Verifies the data quality by applying the constraints defined in the model.
• Removes erroneous records from the checked table, if specified in the control process.

4. Types of Control:
• Static Control:

• Used for checking existing data in the datastore, sub-model, or model.
• The CKM used for the static control is defined in the model.

• Flow Control:

• Used during the data integration process to verify the quality of data flowing through a mapping.
• The CKM for the flow control is specified for the mapping during execution.

5. Key Steps in Data Integrity Check Process:

1.               Activate Static Control:

• A static control is initiated in Studio or through a package.
• The data in the selected datastore or model is checked against predefined constraints.

2.               Activate Flow Control:

• A mapping is executed with flow control enabled in the IKM.
• Data staged in the integration table (I$) is checked against the constraints defined for the target datastore.

3.               CKM Execution:

• The CKM executes data integrity checks based on the constraints defined in the model or mapping.
• The CKM can remove erroneous records if specified.

4.               Data Validation:

• The data is validated against the constraints.
• Invalid or erroneous data may be flagged or removed from the table depending on the configuration.

6. Types of Constraints Checked:
• Constraints that are checked during the Data Integrity Check process may include:

• Null constraints (e.g., ensuring a column is not null).
• Range constraints (e.g., ensuring values fall within a defined range).
• Unique constraints (e.g., ensuring no duplicates exist).
• Foreign key constraints (e.g., ensuring relationships between tables are maintained).
• Any other custom business rules defined in the ODI model.

7. Error Handling:
• If any records fail the integrity check, the CKM can remove them, if configured to do so.
• The CKM may also generate logs or reports that list the failed records for further analysis.

 Summary of the Data Integrity Check Process Steps:

1. Static Control or Flow Control is triggered.
2. The CKM is executed based on the control type (static or flow).
3. The data integrity checks are performed according to predefined constraints.
4. The CKM may remove any erroneous records from the table, as configured.
5. The results of the check are logged, and any violations are reported.