Informatica Scd Type-2 implementation

What is SCD Type-2:

The Type 2 method tracks historical data by creating multiple records for a given natural key in the dimensional tables with separate surrogate keys and/or different version numbers. With Type 2, we have unlimited history preservation as a new record is inserted each time a change is made. Type2 can be achieved in different ways. How to implement SCD Type2 through Informatica: There is number of way to implement SCD Type2 in informatica. Example: Create source table CUST with below query.
CREATE TABLE CUST ( CUST_ID NUMBER, CUST_NM VARCHAR2(250), ADDRESS VARCHAR2(250), CITY VARCHAR2(50), STATE VARCHAR2(50), INSERT_DT DATE, UPDATE_DT DATE);

Insert following data in CUST table. Source Data: CUST_ID CUST_NAME Marion 80001 Atkins 80002 Laura Jones 80003 Jon Freeman ADDRESS 100 Main St. 510 Broadway Ave. 555 6th Ave. CITY Bangalore STATE KA INSER_DT UPDATE_DT 1/7/2011 Hyderabad AP Bangalore KA 1/7/2011 1/7/2011 1/7/2011 1/7/2011 1/7/2011

The Type 2 method tracks historical data by creating multiple records for a given natural key (CUST_ID) in the dimensional tables with separate surrogate keys (PM_PRIMARYKEY). For this create table with below query

Go to Sources > Import from Database. Import the source definition CUST table using the Source Analyzer workspace. CITY VARCHAR2(50). ADDRESS VARCHAR2(250). CUST_NM VARCHAR2(250). So if any change in ADDRESS or CITY or STATE then we need to insert new record and inactivate old record. . PM_PRIMARYKEY is surrogate key which is used to identify each and every record uniquely in target. INACTIVE_DT DATE. UPDATE_DT DATE) Here active_dt and inactive_dt used to indentify history data. SCD Type-2 Over View lkp_CUST_D Flow Flow CUST SQ_CUST exp_FLAG rtr_INS_UPD upd_INSERT CUST_D_INS upd_UPDATE CUST_D_UPD Before implementing we need to identify the attributes needs to be consider for history maintain. ACTIVE_DT DATE. Creation of mapping: Step1: First import source and target to informatica from data base.CREATE TABLE CUST_D ( PM_PRIMARYKEY INTEGER. In this example we will consider if any change in ADDRESS or CITY OR STATE. STATE VARCHAR2(50). CUST_ID NUMBER. The below diagram will explain high level over view of SCD type2 through informatica. INSERT_DT DATE.

Select the CUST table to import and click OK to continue. Assuming that a system DSN is already created for this connection. specify all the necessary details and click Connect. .This opens the Import Tables window.

This mapping would be run daily to capture the historical data in the CUST_D target table.The CUST source definition is created and appears in the workspace. . Click Save to save the source definition in the repository. The source table CUST contains only current data and doesn't have any historical data. Follow the same steps using target Designer to import CUST_D table. The Active and Inactive Date logic would be used for SCD Type 2 mapping.

.Now we have source CUST and target CUST_D tables are available. Mapping creation: Create mapping with name m_SCD_Type_2.

Create lookup table CUST_D Create one input port in lkp_CUST_D table with name in_CUST_D with data type double. And add condition CUST_ID=in_CUST_ID .Now drag CUST table from sources to mapping designer workspace.

Connect CUST_ID from source to in_CUST_ID lookup table .

Change attributes names in expression to identify source and lookup attributes as shown in diagram. CITY. ADDRESS.Create expression transformation and drag CUST_ID. . STATE from lkp_CUST_D table to expression. CUST_NM. ADDRESS. In the same why drag CUST_NM. STATE from Source qualifier to expression. CITY.

INACTIVE_DT as INACTIVE_DT. CUST_D. CUST_D. CUST_D. This you can do it in lookup sql override. SELECT CUST_D. CUST_D.STATE as STATE.UPDATE_DT as UPDATE_DT. CUST_D.PM_PRIMARYKEY as PM_PRIMARYKEY.INSERT_DT as INSERT_DT. CUST_D. CUST_D.CITY as CITY.CUST_NM as CUST_NM.ADDRESS as ADDRESS. CUST_D.CUST_ID as CUST_ID FROM CUST_D WHERE INACTIVE_DT IS NOT NULL .ACTIVE_DT as ACTIVE_DT.In lookup transformation apply filter to retrieve only active records. CUST_D.

Out_FALG-. INSERT_DT. UPDATE_DT attributes in target.> which is used to flag record for insert new/insert update or update/Inactivate record. . Assign SYSDATE for this attribute. INACTIVE_DT.Now create two output ports in expression transformation. Out_DUMMY_DATE-.> which is used to populate ACTIVE_DT.

Are there any changes in attributes from source and Target? For this if PM_PRIMARYKEY is not null means the CUST_ID coming from source present in target.IIF(lkp_CUST_NM!=src_CUST_NM OR lkp_ADDRESS!=src_ADDRESS OR lkp_CITY!=src_CITY OR lkp_STATE!=src_STATE.1. If record coming from source present in target and no changes between source and target attributes then filter those records. IIF(ISNULL(lkp_PM_PRIMARYKEY). Now we need to validate is there any changes in CUST_NM.3)) . This type of records needs to be inserted in target directily. and STATE between source and target data. CITY.To flag a record we need to check three conditions Source record present in target or not.2. Write this condition in out_FLAG expression editor. If any change in data then need to insert this record in target at the same time need to inactivate record present in target. For this if PM_PRIMARYKEY which is coming from source is null that means the record with CUST_ID coming from source not present in target. ADDRESS.

src_CUST_NM. there is no difference between attributes. src_ADDRESS. Now create router transformation and drag the following attributes from expression to router transformation. out_DUMMY_DT and out_FLAG. Lkp_PM_PRIMARYKEY. If any record flagged as 2 that means it’s already exists in target and there is a difference in attributes from source and target. src_CUST_ID. If any record flagged as 3 that means this record present in source and target. src_CITY. . src_STATE.From the above condition if any record flagged as 1 that means it new record which is coming from source and this is not available in target.

In insert group will pass both new and changed records for insert. Update group only to pass the record which we need to inactivate records.Create two groups in router one for insert and another one for update. .

Connect the flowing attributes from INSERT group to Target src_CUST_ID.Up to this mapping will be like below Now connect INSERT group from router to Target. src_CUST_NM. Connect out_DUMMY_DT field from router to INSERT_DT.out_DUMMAY_DT to respective fields in target. . src_CITY. src_STATE. Create sequence transformation and connect nextval from seq transformation to target PM_PRIMARYKEY attribute in INSERT pipe line as shown in below screen. UPDATE_DT and ACTIVE_DT attributes. src_ADDRESS.

out_DUMMAY_DT from UPDATE group of router transformation and connect to update strategy transformation.Now drag lkp_PM_PRIMARYKEY. Connect same to target as shown in above screen shot. .

Now in update strategy transformation set the property as DD_UPDATE. . With this you have completed SCD Type2 mapping. Finally the mapping will be look like below.

Create work flow with session for this mapping and assign source and target relational connections. .

Now we will go with data example. CITY Bangalore STATE INSERT_DT UPDATE_DT KA 7/1/2011 7/1/2011 7/1/2011 7/1/2011 7/1/2011 7/1/2011 Hyderabad AP Bangalore KA Assuming this is the first time we are running mapping so there won’t be any data in target. 555 6th Ave. . CUST_ID CUST_NM Marion 80001 Atkins 80002 Laura Jones 80003 Jon Freeman ADDRESS 100 Main St. Data available in CUST table source on 1st Jul. 510 Broadway Ave.

STATE changed from previous day to today. Changed data in source looks like below. After this run assume that data changed in source on 2nd. From this you can identify for over period of time what is the active record for particular customer.If you are running job on 2nd Jul data in target looks like below So for all this records inactive date is null. CUST_ID CUST_NM Marion 80001 Atkins 80002 Laura Jones 80003 Jon Freeman 80004 Veeru ADDRESS 100 Main St. And another record which is changed record 80003. 510 Broadway Ave. So update date changed from 1St Jul to 2nd Jul. One is new record which is with CUST_ID 80004.HSR Layout CITY Bangalore STATE INSERT_DT UPDATE_DT KA 7/1/2011 7/1/2011 7/1/2011 7/2/2011 7/1/2011 7/1/2011 7/2/2011 7/2/2011 Hyderabad AP Hyderabad AP Bangalore KA In the above data for first two records there are no changes after last refresh so there is now change in update date. If you run same job on 3rd Jul. In target today two records get inserted. Means all are active records. . So in target for CUST_ID 80003 we have two records one is inactivated (PM_PRIMARYKEY=3) and another one is active record (PM_PRIMARYKEY=5). then target data looks like below. Hope this will help you to understand SCD Type 2 logic implementation in Informatica. 555 6th Ave. 90. But for record with CUST_ID 80003 CITY.

Sign up to vote on this title
UsefulNot useful