Check Knowledge Module Overview FAQS

1. What is a Knowledge Module (KM)?

•  A Knowledge Module (KM) is a set of procedures or scripts designed to extract, transform, and load (ETL) data. It encapsulates business logic and data processing tasks, including data quality checks and error management.

 

2. What are the two types of tables maintained by Standard CKMs?

•  The two types of tables maintained by Standard CKMs are:
1. SNP_CHECK_TAB: A summary table that stores a high-level overview of errors for each table and constraint.
2. E$_: An error table that contains the specific records rejected by data quality processes.

 

3. Where is the SNP_CHECK_TAB table located?

•  The SNP_CHECK_TAB table is created in the work schema of the default physical schema of the data server.

 

4. How is the SNP_CHECK_TAB table used?

•  SNP_CHECK_TAB is used to analyze the overall data quality of a model. It provides a summary of errors and can be reviewed to get insights into the health of the data processing system.

 

5. What data is stored in the error tables (E$_)?

•  The error tables contain records that were rejected by the data quality checks, including both static and flow controls. These tables hold the actual data that did not meet the required quality standards.

 

6. How can error tables help in troubleshooting data quality issues?

•  Error tables provide detailed information on the specific records that failed validation checks. By reviewing these tables, data quality issues can be identified and corrected. It helps in pinpointing exactly where and why data failed during processing.

 

7. Can the error tables be accessed after data quality checks?

•  Yes, the error tables are available for post-process analysis. These tables are crucial for understanding what went wrong during the data quality process and correcting any issues that arose.

 

8. What role does the E$_ table play in data validation?

•  The E$_ table plays a key role in recording the records that fail data validation checks. This allows for closer inspection and correction of those records, ensuring data integrity.

 

9. Are there any limitations to the data stored in the SNP_CHECK_TAB and error tables?

•  The SNP_CHECK_TAB provides a high-level summary of errors, but it may not contain detailed information about individual records. The error tables contain rejected data but may not explain why they failed; additional analysis or logs might be required to fully understand the issues.

 

10. How frequently should the data quality tables be reviewed?

•  The data quality tables should be reviewed regularly, especially after major data processing tasks. A proactive approach to monitoring errors can help catch issues early and prevent larger problems down the line.

 

1. What is the purpose of dropping and creating the summary table in CKM?

•  The summary table is dropped and recreated to ensure that previous errors or outdated data are removed before starting the current run. The CREATE statement is executed to ensure the table exists, and any errors due to the table already existing are tolerated.

 

2. Why is it necessary to remove the previous run’s summary records from the summary table?

•  Removing previous run’s summary records ensures that only the errors from the current run are stored in the summary table. This helps maintain up-to-date and accurate data quality information.

 

3. What does the "DROP and CREATE" process for the error table do?

•  Dropping and recreating the error table is done to remove any old or outdated error records from prior runs. The CREATE statement ensures that the error table is available for the current run, and any errors due to its prior existence are ignored.

 

4. What happens if an error table already exists during the "CREATE" step?

•  If the error table already exists, the error is tolerated. The CREATE statement will still run to ensure the table is ready for the current run, even if it means overwriting the previous table.

 

5. Why are records rejected for violating primary, alternate, and foreign key constraints?

•  Records are rejected if they violate key constraints because these constraints ensure data integrity. Violating primary keys (uniqueness), alternate keys (alternate identifiers), or foreign keys (relationships between tables) could lead to inconsistencies in the data model.

 

6. What is the role of the "check condition constraint" in rejecting records?

•  A check condition constraint is used to enforce specific rules or conditions on data (e.g., valid ranges for a numerical field or non-null values). Records violating these conditions are rejected to maintain data accuracy.

 

7. What does rejecting records violating mandatory attribute constraints mean?

•  Mandatory attribute constraints ensure that certain attributes must have values. If a record is missing a required attribute or contains a null value for a mandatory field, it is rejected to avoid incomplete data entries.

 

8. Why would rejected records need to be removed from the checked table?

•  Rejected records might need to be removed from the checked table to prevent them from being processed further or causing issues in subsequent steps. Removing them ensures that only valid data is processed and stored.

 

9. How is the summary of detected errors inserted into the summary table?

•  After running the validation and rejection steps, the summary of the detected errors (such as counts of rejected records or types of violations) is inserted into the summary table. This provides a high-level view of the data quality status after the checks.

 

10. Can the CKM process be customized for different data validation needs?

•  Yes, the CKM process can be customized to handle different validation requirements. The constraints and error detection steps can be tailored to fit the specific business rules and data quality standards of an organization.

1. What is the purpose of tagging CKM commands?

•  Tagging CKM commands allows Oracle Data Integrator (ODI) to generate the correct code for each specific type of constraint (primary key, alternate key, foreign key, etc.) during the data integration process. It helps automate the creation of validation checks and ensures data integrity.

 

2. What does the "Primary Key" tag do?

•  The "Primary Key" tag is used to generate the code needed to check the primary key constraint during the data validation process. This ensures that records are unique and do not violate primary key rules.

 

3. When is the "Alternate Key" tag used?

•  The "Alternate Key" tag is used when defining the code for validating alternate key constraints. During code generation, ODI will apply this tag to all alternate key constraints, ensuring that alternate key rules are adhered to.

 

4. What is the function of the "Join" tag?

•  The "Join" tag defines the code needed to check foreign key constraints, which ensure the integrity of relationships between tables. This tag is used during code generation to validate that records in one table correspond to related records in another table via foreign keys.

 

5. What is the purpose of the "Condition" tag?

•  The "Condition" tag defines the code required to check condition constraints, such as range checks or custom validation rules. During code generation, this tag ensures that all specified conditions (e.g., value ranges or formats) are checked for data accuracy.

 

6. How does the "Mandatory" tag work?

•  The "Mandatory" tag is used to enforce constraints that require certain attributes (fields) to be non-null. During code generation, this tag ensures that all mandatory attributes have a value and do not contain null entries.

 

7. What is the purpose of the "Remove Errors" tag?

•  The "Remove Errors" tag is used to define the code necessary to remove rejected records from the checked table. This ensures that only valid records remain in the table and any rejected data is properly cleaned up after validation checks.

 

8. Can these tags be customized?

•  While the tags themselves are predefined for specific constraint types, you can customize the logic associated with each tag to suit your specific business rules or validation needs during the code generation process.

 

9. Do I need to manually apply these tags during CKM creation?

•  No, during the CKM creation process, these tags are typically used automatically by Oracle Data Integrator to generate the appropriate validation logic. However, you may manually adjust or customize the tags depending on your specific data validation requirements.

 

10. How do these tags help in the overall CKM process?

•  These tags automate the generation of validation logic, ensuring that all necessary data quality checks are applied. They help enforce business rules, ensure data integrity, and streamline the CKM process by eliminating the need for manual coding for each type of constraint.