Integration Process Overview FAQS

1. What is an integration process in data mapping?
• The integration process is the step where data from the source or loading tables is moved into the target datastore. It often involves using a temporary integration table for staging the data after transformation.
2. What is an integration table (flow table)?
• An integration table (or flow table) is a temporary table used to stage data after transformations. It holds the data before it's integrated into the target table, and it can include additional fields for handling the integration strategy.
3. What is the role of the source and/or loading tables?
• The source and/or loading tables, created by the Loading Knowledge Module (LKM), are the starting point for loading data. These tables hold the data that will be moved into the integration table or directly into the target table.
4. What is the Check Knowledge Module (IKM)?
• The IKM performs a flow check to validate the data in the integration table against the constraints in the target table. It ensures only valid data moves forward by removing any invalid entries from the integration table.
5. What is mapping and model metadata in the integration process?
• Mapping and model metadata, such as Insert, Update, or Slowly Changing Dimension (SCD) behavior, are used to configure the behavior of attributes during the integration process. They help to define how data should be processed during the integration.
6. What happens during the integration process?
• The integration process involves several steps:
1. Create an integration table (if necessary) for staging the data.
2. Load data from source and loading tables into the integration table while applying any necessary transformations.
3. Transform the data in the integration table to implement the chosen integration strategy (e.g., flagging records for update or insert).
4. Modify and load data into the target table, ensuring the changes (inserts or updates) are correctly applied.
7. Why is an integration table necessary?
• The integration table is needed to temporarily hold and transform data before it is inserted or updated in the target table. This staging area allows for validation, transformation, and checking of data before final integration.
8. How are invalid records handled in the integration process?
• The IKM checks the data against constraints in the target table. Any invalid records are removed from the integration table to ensure that only valid data is transferred to the target table.
9. Can multiple integration strategies be used?
• Yes, multiple integration strategies can be used depending on the requirements of the data and the target system. Common strategies include Append, Incremental Update, and Slowly Changing Dimension.
10. What is the role of mapping metadata in the integration phase?
• Mapping metadata is used to configure how each attribute of the data is treated during the integration process. This includes deciding whether the data should be inserted, updated, or treated according to other rules like Slowly Changing Dimension (SCD) behavior.