Loading Knowledge Modules (LKM) FAQS

1. What is an LKM?
•  An LKM (Loading Knowledge Module) is a component used to load data from a source server to a target server. It is often used when the source and target servers are not on the same machine, and it facilitates data movement in different types of data integration scenarios.
2. When do I need to use an LKM?
•  You need to use an LKM when the source and target servers are on different machines, and you need to move data between them. It’s also used when certain logic must be executed on the source server, or when data needs to be transferred transparently between systems.
3. What is the difference between the LKM types?
•  
• PERSISTENT: Loads the data into a staging table on the target server, saving a physical copy of the data.
• TRANSPARENT_SOURCE: Allows the target server to access the data on the source server without physically transferring the data.
• TRANSPARENT_TARGET: Allows the source server to access and load data directly into the target server without transferring it.
4. What is the purpose of a staging table in LKM?
•  A staging table is used to temporarily store the data from the source server on the target server (typically when using the PERSISTENT LKM type). It serves as an intermediary step before the final transformation and loading process into the final target datastore.
5. Can I use LKM if my source and target servers are on the same machine?
•  While you can still use an LKM in this case, it is typically not necessary if both the source and target are on the same server. You might prefer other integration techniques, such as direct access or local processing, without needing a separate LKM for this scenario.
6. What happens if the source and target servers are on different platforms or databases?
•  LKMs can handle different platforms and databases by setting up transparent access mechanisms (in TRANSPARENT_SOURCE and TRANSPARENT_TARGET types) or by loading the data into an intermediary staging area (PERSISTENT LKM). This ensures data can be transferred efficiently even between different systems.
7. Can LKM be used for data transformation?
•  While LKMs are primarily for data extraction and loading, they can be used in conjunction with other components, such as Transformation Knowledge Modules (TKM), to implement data transformations as part of the overall mapping process.
8. What is a C$ staging table?
•  A C$ staging table is a temporary table created on the target server during the LKM PERSISTENT type process. This table stores the result set from the source server before further processing, transformation, and loading into the final target datastore.
9. Can I change the LKM type based on my needs?
•  Yes, you can choose the appropriate LKM type based on your specific use case. If you need to load data into a staging area, use PERSISTENT. If you need transparent access to source data, use TRANSPARENT_SOURCE or TRANSPARENT_TARGET.
10. What are the benefits of using LKMs?
•  LKMs provide flexible data movement strategies, especially when working with distributed systems. They allow for efficient data access and transfer, even between different platforms, and can optimize the integration process by using transparent data access or by staging data for further processing.

 

1. What is the purpose of the C$ temporary table?
•  The C$ temporary table is created by the LKM in the staging area to temporarily hold the records that are loaded from the source server. This table serves as an intermediary storage location before further processing or loading the data into the final target.
2. How does the LKM obtain data from the source server?
•  The LKM retrieves data from the source server by executing a SQL SELECT query if the source server is an RDBMS. If the source does not support SQL (e.g., flat files or applications), the LKM reads the data using an appropriate method such as file reading or API execution.
3. What happens if the source server does not support SQL?
•  If the source server does not support SQL, such as with flat files or non-database applications, the LKM uses an alternative method to retrieve the data. This could involve reading files directly or executing APIs to fetch the data from the source system.
4. What kind of data does the LKM transfer?
•  The LKM transfers pre-transformed records from the source server to the target server. These records are either pre-transformed at the source or transformed during the transfer process, depending on the LKM configuration and mapping logic.
5. Why does the LKM load data into the C$ table?
•  The C$ table is used as a temporary staging area for the records before they are processed further. It helps manage and stage the data in a structured way before it's loaded into the final target data store.
6. Can the LKM process data that is already transformed?
•  Yes, the LKM can handle pre-transformed data. It fetches the data from the source and loads it into the C$ table in its transformed state, without needing to transform it again.
7. What types of sources can the LKM work with?
•  The LKM can work with a wide variety of source systems, including RDBMS (using SQL queries), flat files, and applications (using file reading or API methods).
8. Is there a limit to the size of data that can be loaded into the C$ table?
•  The size of the data that can be loaded into the C$ table depends on the resources of the staging area and the database system. The performance and scalability of the LKM process will depend on the database configuration, available storage, and processing power.
9. Can the LKM handle data from multiple source servers?
•  Yes, the LKM can handle data from multiple source servers, provided that the source data is mapped correctly to the target staging table and that the correct access mechanisms are set up.
10. What happens after data is loaded into the C$ table?
•  Once the data is loaded into the C$ table, it can be processed further, typically through additional transformations or moved to the final target data store. The C$ table serves as an intermediary holding area for data to be processed and transformed according to the mapping logic.

 

Frequently Asked Questions (FAQs) about LKM of Type TRANSPARENT_SOURCE

1. What is a TRANSPARENT_SOURCE LKM?
•  A TRANSPARENT_SOURCE LKM creates a transparent access mechanism that allows the target server to directly access the data on the source server without physically transferring the data. This makes the process more efficient and eliminates the need for data movement.
2. Why would I use a TRANSPARENT_SOURCE LKM?
•  You would use a TRANSPARENT_SOURCE LKM when you want to access data from the source server without moving it to the target server. It’s especially useful when data needs to be accessed in real-time or when minimizing the movement of large volumes of data is important.
3. What is the role of the metadata object created by the LKM?
•  The code generation metadata object created by the LKM stores information about the source data and the source mapping logic. This metadata is passed to the target IKM, which uses it to process the data and load it into the target system.
4. How does the transparent access mechanism work?
•  The transparent access mechanism allows the target server to query and access data directly from the source server as if the data were locally available on the target. This is achieved through database links, views, or other techniques depending on the data sources involved.
5. Is data physically moved with a TRANSPARENT_SOURCE LKM?
•  No, data is not physically moved when using a TRANSPARENT_SOURCE LKM. Instead, the target server is given direct access to the source data, allowing it to query and work with the data in place, without the need for transferring it.
6. What does the target IKM do with the metadata object?
•  The target IKM uses the metadata object passed by the LKM to understand the source structure and mapping logic. It then processes the data from the source and loads it into the target datastores, following the transformation and loading rules defined in the mapping.
7. Can the TRANSPARENT_SOURCE LKM be used with non-database sources?
•  The TRANSPARENT_SOURCE LKM is typically used with database sources, where direct access can be achieved via database links or similar techniques. For non-database sources like flat files, transparent access is generally not applicable.
8. Does the TRANSPARENT_SOURCE LKM require any special configuration on the source or target systems?
•  Yes, the source and target systems must be configured to allow for transparent access. This could involve setting up database links, views, or other mechanisms to enable the target server to access the data from the source server without physically moving the data.
9. Can I transform the data while accessing it using the TRANSPARENT_SOURCE LKM?
•  Yes, while the TRANSPARENT_SOURCE LKM provides access to the data, transformations can be applied to the data during or after retrieval, depending on how the target IKM and mapping are configured.
10. What types of data sources can the TRANSPARENT_SOURCE LKM access?
•  The TRANSPARENT_SOURCE LKM typically works with RDBMS (Relational Database Management Systems) where transparent access mechanisms such as database links or views can be established. It’s not designed for flat files or non-database systems.

 

Frequently Asked Questions (FAQs) about LKM of Type TRANSPARENT_TARGET

1. What is a TRANSPARENT_TARGET LKM?
•  A TRANSPARENT_TARGET LKM creates a transparent access mechanism that allows the source server to access and load data directly into the target datastore without physically moving the data between servers.
2. Why would I use a TRANSPARENT_TARGET LKM?
•  You would use a TRANSPARENT_TARGET LKM when you need to load data into the target datastore but want to avoid physically transferring the data between servers. This is efficient when the source can directly access and modify the target.
3. How does the transparent access mechanism work?
•  The transparent access mechanism allows the source server to access the target datastore directly, as if the data were locally available on the source system. This setup removes the need for an intermediary storage location like a staging area.
4. Is data physically moved in the TRANSPARENT_TARGET LKM process?
•  No, data is not physically moved in the TRANSPARENT_TARGET LKM process. Instead, the source server directly writes data to the target datastore using the transparent access mechanism, which enables efficient data transfer without physical movement.
5. How are the pre-transformed records obtained?
•  The TRANSPARENT_TARGET LKM obtains pre-transformed records from the source server by executing the appropriate transformations on the data. These transformations ensure the data is ready for direct loading into the target datastore.
6. Can I use a TRANSPARENT_TARGET LKM with non-database sources?
•  The TRANSPARENT_TARGET LKM is typically used with RDBMS sources. It relies on transparent access mechanisms like database links or views, which may not be applicable for non-database sources (such as flat files or applications).
7. Can I perform transformations during the TRANSPARENT_TARGET LKM process?
•  Yes, transformations can be applied before or during the transfer of data from the source to the target, depending on how the source data is processed and the mapping is set up.
8. What happens if the source and target are on different servers?
•  If the source and target are on different servers, the TRANSPARENT_TARGET LKM enables the source server to access and write data to the target datastore directly, using the transparent access mechanism set up between the two servers.
9. Do I need an LKM if the source and target are on the same server?
•  No, if the source and target are on the same server, no LKM is needed. The data can be transferred directly between the source and target without the need for an intermediary mechanism or separate loading logic.
10. Can multiple LKMs be used in a single mapping?
•  Yes, if a mapping involves datastores from multiple sources, multiple LKMs may be needed to handle data from different servers. This is especially necessary when source datastores are located on different servers.