Using dynamic lookup cache

Applies to:
Informatica PowerCenter

Summary
This article outlines the possibility to eliminate duplicate records by using a dynamic lookup cache.

Author Bio
Author(s): Matthias Urech Company: interface-development.com Created on: August 21, 2009 Matthias Urech, founder and project leader of interface - development.com, has a proven track record of applying data integration solutions for several companies across the industry. He played a key role to implement projects in the area of data integration, data migration, data consolidation and data warehousing. Matthias contributed several articles about data integration best practices for Informatica Technology Network, resulting in nomination as member of developer wall of fame and receiving an award as contributor of the year in 2008.

Table of Contents
Introduction .........................................................................................................................................................2 Analyzing the problem ........................................................................................................................................2 Walkthrough: Using dynamic lookup cache........................................................................................................3
Step 1: Creating Source and Source Qualifier .............................................................................................................3 Step 2: Creating Target................................................................................................................................................4 Step 3: Filter inactive records.......................................................................................................................................4 Step 4: Sorting records for latest change.....................................................................................................................5 Step 5: Lookup target data...........................................................................................................................................6 Step 6: Flagging data for insert or update....................................................................................................................7 Step 7: Creating Workflow and Session.......................................................................................................................8

Disclaimer and Liability notice ............................................................................................................................9

Informatica Technology Network © 2009 Informatica Corporation. All Rights Reserved.

http://technet.informatica.com 1

CHANGE_DATE. The example used in this article is dealing with the issue that a status is stored multiple times in the source system without flagging the actual record. We will analyze the problem and go step by step in building a solution by using dynamic lookup cache and storing only the actual status in the target. We want to get the actual STATUS_ID for each OBJECT_ID. By taking a closer look at the table we already recognize a pattern: • • • Each status has a date stamp Equal date stamps contain a sequential number (attribute CHANGE_NO) An inactive status is marked Figure 1: source table OBJECT_STATUS Given the fact that we know the pattern. http://technet. Analyzing the problem Figure 1 shows example table OBJECT_STATUS.com 2 .Using dynamic lookup cache Introduction This article will dive into the deep blue of using a dynamic lookup cache that inserts and updates rows in the cache as it passes rows to the target. CHANGE_NO Use dynamic lookup cache to determine if current records exists in target Flag record to force insert or update in target Once the solution has been built and ran.informatica. All Rights Reserved. we are able to roughly draw the solution: • • • Sort data by attribute OBJECT_ID. the yellow marked records in Figure 1 are reflecting the result (= unique value for each OBJECT_ID) Informatica Technology Network © 2009 Informatica Corporation.

http://technet.informatica. Sorting records for latest change 5. Creating Workflow and Session Figure 2 illustrates the result of our activities. All Rights Reserved. Filter inactive records 4. The source table has the following data definitions: Figure 3: data definition of Source 3. Creating Source and Source Qualifier 2. Creating Target 3.com 3 . Figure 2: mapping m_USING_DYNAMIC_LOOKUP_CACHE Step 1: Creating Source and Source Qualifier To create Source and Source Qualifier: 1. Add Source and Source Qualifier to your mapping and connect the ports Informatica Technology Network © 2009 Informatica Corporation. Create a new mapping m_USING_DYNAMIC_LOOKUP_CACHE 2. Lookup target data 6.Using dynamic lookup cache Walkthrough: Using dynamic lookup cache In this section you will accomplish the following activities: 1. Create a Source by defining the table definition. Flagging data for insert or update 7.

Using dynamic lookup cache Step 2: Creating Target To create a target: 1. The Target has the following data definitions: Figure 4: data definition of Target 2. Select all ports of the Source Qualifier and drop them into the Filter transformation 3. http://technet. Create a relational Target to stage the output.com 4 . All Rights Reserved.informatica. Add a Filter transformation to your mapping 2. Create physical target in your preferred database by running the create table statement 3. Edit the transformation and specify the values in tab Properties as follows: Figure 5: settings of Filter transformation Informatica Technology Network © 2009 Informatica Corporation. Add the Target to your mapping Step 3: Filter inactive records To filter inactive records: 1.

Edit the transformation. CHANGE_NO 4.informatica.com 5 . Select all ports of the Filter and drop them into the Sorter transformation 3. http://technet. select checkbox Key and specify Ascending as sort order for the following ports: OBJECT_ID. All Rights Reserved. CHANGE_DATE.Using dynamic lookup cache Step 4: Sorting records for latest change To sort records for latest change: 1. Specify other settings according to figure 6: Figure 6: settings of Sorter transformation Informatica Technology Network © 2009 Informatica Corporation. Add as Sorter transformation to your mapping 2.

http://technet. Associate lookup ports with input ports 6. Edit the transformation and select the Condition tab. Select all ports from the Sorter and drop it to the Lookup transformation 3. Add a Lookup Transformation to your mapping. Select the Properties tab and select checkbox Dynamic Lookup Cache and Insert Else Update 5. Enter other settings according to figure 7: Figure 7: settings of Lookup transformation Informatica Technology Network © 2009 Informatica Corporation. Select target OBJECT_STATUS in the popup window and click [OK] button 2. Add a new condition and specify OBJECT_ID as port to be matched 4.com 6 .Using dynamic lookup cache Step 5: Lookup target data To lookup target data: 1. All Rights Reserved.informatica.

All Rights Reserved. The Informatica Server inserts the row into the cache. The Informatica Server updates the row in the cache. Save your mapping m_USING_DYNAMIC_LOOKUP_CACHE Informatica Technology Network © 2009 Informatica Corporation. Add an Update Strategy transformation to your mapping Select port NewLookupRow of the Lookup and drop it to the Update Strategy transformation Select all ports of the Sorter and drop them to the Update Strategy transformation Edit the Update Strategy transformation and select tab Properties Enter the following value in the Update Strategy Expression: IIF(NewLookupRow = 1. 2.Using dynamic lookup cache Step 6: Flagging data for insert or update To flag data for insert or update: 1. 4. http://technet. DD_UPDATE) Note: The Informatica Server assigns a value to the port. depending on the action it performs to the lookup cache: NewLookupRow Value 0 1 2 Table 1: NewLookupRow values Description The Informatica Server inserts the row into the cache. DD_INSERT.informatica. 3. 6. Select all ports of the Update Strategy transformation and connect them to the Target 7. 5.com 7 .

Run the workflow and see the output: Figure 9: output of session workflow wf_USING_DYNAMIC_LOOKUP_CACHE Informatica Technology Network © 2009 Informatica Corporation. http://technet. 3.dat) Select tab Targets. All Rights Reserved. specify the database to connect and select Insert. Perform all other required settings 6. Update Else Insert and Truncate action in the Property section 5. 4.Using dynamic lookup cache Step 7: Creating Workflow and Session To create Workflow and Session: Create a workflow wf_USING_DYNAMIC_LOOKUP_CACHE Create a session for mapping m_USING_DYNAMIC_LOOKUP_CACHE Select tab Sources and specify the source directory and file name (object_status. 2. Save the workflow wf_USING_DYNAMIC_LOOKUP_CACHE 1.com 8 . Figure 8: workflow wf_USING_DYNAMIC_LOOKUP_CACHE 7.informatica.

All Rights Reserved.informatica.Using dynamic lookup cache Disclaimer and Liability notice Informatica offers no guarantees and assumes no responsibility or liability of any type with respect to the content of this software asset. http://technet. including any liability resulting from incompatibility between the content within this asset and the materials and services offered by Informatica. or seek to hold. You agree that you will not hold.com 9 . Informatica responsible or liable with respect to the content of this software asset. Informatica Technology Network © 2009 Informatica Corporation.

Sign up to vote on this title
UsefulNot useful

Master Your Semester with Scribd & The New York Times

Special offer for students: Only $4.99/month.

Master Your Semester with a Special Offer from Scribd & The New York Times

Cancel anytime.