Integration Process Overview

1. Strategy Dependence:
• The integration process heavily relies on the integration strategy chosen.
2. Elements Used in the Integration Process:
• Integration Table (Flow Table):
• A temporary table created to stage data after transformations are made in the staging area.
• Named after the target table, prefixed with "I$."
• Contains extra fields required for the strategy, and data is flagged, transformed, or checked before integration into the target table.
• Source and/or Loading Tables (created by LKM):
• These tables contain the data to be loaded into the integration table or directly into the target table.
• Check Knowledge Module (IKM):
• The IKM may trigger a flow check phase to validate data against constraints defined in the target table.
• Invalid data is removed from the integration table, ensuring only valid data moves forward.
• Mapping and Model Metadata:
• Metadata like Insert, Update, UD1, etc., or specific model metadata (e.g., Slowly Changing Dimension behavior) are used to configure attribute-level behavior during the integration process.
3. Typical Integration Process Workflow:
• Step 1: Create Integration Table (if needed):
• If necessary, a temporary integration table is created.
• For example, an "update flag" is added to the integration table to track whether a row is to be inserted or updated (e.g., flag values such as 'I' for Insert and 'U' for Update).
• Step 2: Load Data into Integration Table:
• Data from source and/or loading tables is loaded into the integration table.
• This step includes executing the necessary transformations (joins, filters, mappings) defined in the staging area.
• Step 3: Transformation in the Integration Table:
• Perform necessary transformations on the integration table to implement the integration strategy.
• For example, comparing the integration table with the target table to set flags for which rows should be updated.
• Step 4: Modify and Load Data into Target Table:
• The data is then modified and loaded from the integration table into the target table, based on the integration strategy (insert, update, etc.).