You are on page 1of 143

Important Concepts in Informatica8.

6
Debugger
. To debug a mapping, you configure and run the Debugger from within the Mapping Designer. The Debugger uses a
session to run the mapping on the Integration Service. When you run the Debugger, it pauses at breakpoints and you
can view and edit transformation output data.
You might want to run the Debugger in the following situations:

Before you run a session. After you save a mapping, you can run some initial tests with a debug session before
you create and configure a session in the Workflow Manager.
After you run a session. If a session fails or if you receive unexpected results in the target, you can run the
Debugger against the session. You might also want to run the Debugger against a session if you want to debug
the mapping using the configured session properties.

Debugger Session Types:
You can select three different debugger session types when you configure the Debugger. The Debugger runs a
workflow for each session type. You can choose from the following Debugger session types when you configure the
Debugger:

Use an existing non-reusable session. The Debugger uses existing source, target, and session configuration
properties. When you run the Debugger, the Integration Service runs the non-reusable session and the existing
workflow. The Debugger does not suspend on error.
Use an existing reusable session. The Debugger uses existing source, target, and session configuration
properties. When you run the Debugger, the Integration Service runs a debug instance of the reusable session
And creates and runs a debug workflow for the session.
Create a debug session instance. You can configure source, target, and session configuration properties
through the Debugger Wizard. When you run the Debugger, the Integration Service runs a debug instance of the
debug workflow and creates and runs a debug workflow for the session.

Debug Process
To debug a mapping, complete the following steps:
1. Create breakpoints. Create breakpoints in a mapping where you want the Integration Service to evaluate data and
error conditions.
2. Configure the Debugger. Use the Debugger Wizard to configure the Debugger for the mapping. Select the session
type the Integration Service uses when it runs the Debugger. When you create a debug session, you configure a subset
of session properties within the Debugger Wizard, such as source and target location. You can also choose to load or
discard target data.
3. Run the Debugger. Run the Debugger from within the Mapping Designer. When you run the Debugger, the
Designer connects to the Integration Service. The Integration Service initializes the Debugger and runs the debugging
session and workflow. The Integration Service reads the breakpoints and pauses the Debugger
when the breakpoints evaluate to true.
4. Monitor the Debugger. While you run the Debugger, you can monitor the target data, transformation and mapplet
output data, the debug log, and the session log. When you run the Debugger, the Designer displays the following
windows:


Debug log. View messages from the Debugger.
Target window. View target data.
Instance window. View transformation data.

5. Modify data and breakpoints. When the Debugger pauses, you can modify data and see the effect on
transformations, mapplets, and targets as the data moves through the pipeline. You can also modify breakpoint
information.
The Designer saves mapping breakpoint and Debugger information in the workspace files. You can copy breakpoint
information and the Debugger configuration to another mapping. If you want to run the Debugger from another Power
Center Client machine, you can copy the breakpoint information and the Debugger configuration to the other Power
Center Client machine.
Running the Debugger:
When you complete the Debugger Wizard, the Integration Service starts the session and initializes the Debugger. After
initialization, the Debugger moves in and out of running and paused states based on breakpoints and commands that
you issue from the Mapping Designer. The Debugger can be in one of the following states:


Initializing. The Designer connects to the Integration Service.
Running. The Integration Service processes the data.
Paused. The Integration Service encounters a break and pauses the Debugger.

Note: To enable multiple users to debug the same mapping at the same time, each user must configure different port
numbers in the Tools > Options > Debug tab.
The Debugger does not use the high availability functionality.

.

you can monitor the following information:     Session status. The session log displays in the Session Log tab. Breakpoints. Debug indicators on transformations help you follow breakpoints and data flow. you can view transformation data and row information in the Instance window. Data movement. . When the Debugger pauses. Notifications tab. View target data for each target in the mapping.Monitoring the Debugger : When you run the Debugger. Output window. Displays messages from the Repository Service. Target window. Instance window. The Mapping Designer displays windows and debug indicators that help you monitor the session:        Debug indicators. Session Log tab. Monitor data as it moves through transformations. Monitor the status of the session. The Integration Service writes messages to the following tabs in the Output window: Debugger tab. The debug log displays in the Debugger tab. Monitor target data on a row-by-row basis. Monitor data that meets breakpoint conditions. Target data.

Lookup transformation. NewLookupRow port for a Lookup transformation configured to use a dynamic cache. Custom transformation. Sequence Generator transformation. you might want to change the transformation output data to see the effect on subsequent transformations or targets in the data flow. Rank transformation. Router transformation.While you monitor the Debugger. Generated Keys and Generated Column ID ports. RANKINDEX port. All output ports. you cannot change data associated with the following:    Mapplets that are not selected for debugging Input or input/output ports Output ports when the Debugger pauses on an error breakpoint MAPPING PARAMETERS & VARIABLES . You might also want to edit or add more breakpoint information to monitor the session more closely. Restrictions You cannot change data for the following output ports:        Normalizer transformation. Java transformation. CURRVAL and NEXTVAL ports. Ports in output groups other than the current output group. Ports in output groups other than the current output group. Additionally.

first we declare the mapping parameter or variable for use in each mapplet or mapping. Then. 2004. We can override a saved value with the parameter file. We might use a mapping variable to perform an incremental read of the source. mapping variables are values that can change between sessions. and in the Expression Editor of reusable transformations. We can also use parameters in a source qualifier filter. it reads only rows from August 2. Example: When we want to extract records of a particular month during ETL process. The next time it runs the session. user-defined join. we have a source table containing time stamped transactions and we want to evaluate the transactions on a daily basis.    After we create a parameter. If we set the initial value of $$IncludeDateTime to 8/1/2004. MAPPING PARAMETERS   A mapping parameter represents a constant value that we can define before running a session. We can then use the parameter in any expression in the mapplet or mapping. In the source qualifier. When we use a mapping parameter or variable in a mapping. For example. the Integration Service sets $$IncludeDateTime to 8/2/2004. the first time the Integration Service runs the session. A mapping parameter retains the same value throughout the entire session. During the session. Used in following transformations:     Expression Filter Router Update Strategy Initial and Default Value: . such as: TIMESTAMP = $$IncludeDateTime In the mapping. use a variable function to set the variable value to increment one day each time the session runs. it appears in the Expression Editor. it reads only rows dated 8/1/2004.Mapping parameters and variables represent values in mappings and mapplets. MAPPING VARIABLES     Unlike mapping parameters. The Integration Service saves the latest value of a mapping variable to the repository at the end of each successful session. It saves 8/2/2004 to the repository at the end of the session. Instead of manually entering a session override to filter source data each time we run the session. we will create a Mapping Parameter of data type and use it in query to compare it with the timestamp field in SQL override. we define a value for the mapping parameter or variable before we run the session. we can create a mapping variable. We can also clear all saved values for the session in the Workflow Manager. create a filter to read only rows whose transaction date equals $ $IncludeDateTime. or extract override. $ $IncludeDateTime.

the Integration Service uses a default value based on the data type of the parameter or variable. delete. Value in parameter file Value saved in the repository Initial value Default value Current Value: The current value is the value of the variable as the session progresses. the current value of a variable is the same as the start value. Aggregation types are:    Count: Integer and small integer data types are valid only. the start value of the variable is saved to the repository. we need to configure the Data type and aggregation type for the variable. It ignores rows marked for update. When a session fails to complete. or reject. It ignores rows marked for update. The IS uses the aggregate type of a Mapping variable to determine the final current value of the mapping variable. the Integration Service does not update the value of the variable in the repository. Note: If a variable function is not used to calculate the current value of a mapping variable. . When a session starts. SetMinVariable: Sets the variable to the minimum value of a group of values.When we declare a mapping parameter or variable in a mapping or a mapplet. delete. or reject. SetMaxVariable: Sets the variable to the maximum value of a group of values. Min: All transformation data types except binary data type are valid. Variable Data type and Aggregation Type When we declare a mapping variable in a mapping. 4. Variable Functions Variable functions determine how the Integration Service calculates the current value of a mapping variable in a pipeline. we can enter an initial value. and we did not declare an initial value for the parameter or variable. Max: All transformation data types except binary data type are valid. When the Integration Service needs an initial value. The final current value for a variable is saved to the repository at the end of a successful session. 3. Aggregation type set to Min. Aggregation type set to Max. The Integration Service looks for the start value in the following order: 1. Data ->Default Value Numeric ->0 String ->Empty String Date time ->1/1/1 Variable Values: Start value and current value of a mapping variable Start Value: The start value is the value of the variable at the start of the session. 2.

3. Example: Use of Mapping of Mapping Parameters and Variables      EMP will be source table. -or. Ex: m_mp_mv_example 4. Enter name. click Mapplet > Parameters and Variables. DEPTNO. Click Mapping-> Create-> Give name. Create a target table MP_MV_EXAMPLE having columns: EMPNO. and subtracts one when the row is Marked for deletion. Drag EMP and target table. MIN_VAR. Aggregation type set to Count. Select Aggregation type for mapping variables. it compares the final current value of the variable to the start value of the variable. Select Type and Data type. COMM and DEPTNO to Expression.In the Mapplet Designer. SAL. At the end of a session. Create Parameter $$Bonus and Give initial value as 200. SetVariable: Sets the variable to the configured value. 5. Based on the aggregate type of the variable. Give Initial Value. 2. Transformation -> Create -> Select Expression for list -> Create –> Done. ENAME. Create shortcuts as necessary. 4. It ignores rows marked for update or reject. 8. Creating Mapping Parameters and Variables 1. Create variable $$var_count of COUNT aggregation type and initial value 0.SetCountVariable: Increments the variable value by one. Creating Mapping 1. Open the folder where we want to create parameter or variable. . COUNT is visible when datatype is INT or SMALLINT. ENAME. 6. Create variable $$var_set of MAX aggregation type. it saves a final value to the repository. 5. TOTAL_SAL. Create variable $$var_max of MAX aggregation type and initial value 1500. 6. HIREDATE. Click the add button. 10. 2. click Mappings > Parameters and Variables. COUNT_VAR and SET_VAR. In the Mapping Designer. Click ok. It adds one to the variable value when a row is marked for insertion. 7. 9. Click Tools -> Mapping Designer. Drag EMPNO. TOTAL_SAL = SAL+ COMM + $$BONUS (Bonus is mapping parameter that changes every month) SET_VAR: We will be added one month to the HIREDATE of every employee. 3. Do not remove $$ from name. MAX_VAR. Open folder where we want to create the mapping. 11. Create variable $$var_min of MIN aggregation type and initial value 1500.

Select $$var_max from variable tab and SAL from ports tab as shown below. Create 5 output ports out_ TOTAL_SAL. 15. Open Expression editor for out_max_var. out_MAX_VAR. Select the variable function SETMAXVARIABLE from left side pane. select variable tab and select the parameter from mapping parameter. out_MIN_VAR. out_COUNT_VAR and out_SET_VAR.SETMAXVARIABLE($$var_max. 13.12.SAL) . SAL + COMM + $$Bonus 14. To add $$BONUS to it. Open expression editor for TOTAL_SAL. Do the same as we did earlier for SAL+ COMM.

Validate. Click OK. Expression Transformation below: 21. . Open Expression editor for out_set_var and write the following expression: SETVARIABLE($$var_set.1)). Link all ports from expression to target and Validate Mapping and Save it. Validate the expression.17. Open Expression editor for out_count_var and write the following expression: SETCOUNTVARIABLE($$var_count).ADD_TO_DATE(HIREDATE. 19. 20. See mapping picture on next page. 22.SAL).'MM'. 18. Open Expression editor for out_min_var and write the following expression: SETMINVARIABLE($$var_min. Validate the expression.

worklet. or session to which we want to assign parameters or variables. Run workflow and see result. We can create multiple parameter files and change the file we use for a session or workflow.    Make session and workflow. worklet. Session parameter: Defines a value that can change from session to session. such as a database connection or file name. We can create a parameter file using a text editor such as WordPad or Notepad. A parameter file contains the following types of parameters and variables:     Workflow variable: References values and records information in a workflow. Worklet variable: References values and records information in a worklet. but we cannot use workflow variables from the parent workflow in a worklet. workflow. Sample Parameter File for Our example: . The heading identifies the Integration Service. Give connection information for source and target table. Parameter files provide flexibility to change these variables each time we run a workflow or session. Integration Service process. Use predefined worklet variables in a parent workflow. or session. Enter the parameter file name and directory in the workflow or session properties. Mapping parameter and Mapping variable USING A PARAMETER FILE Parameter files contain several sections preceded by a heading.PARAMETER FILE     A parameter file is a list of parameters and associated values for a workflow.

4. Click the Properties tab and open the General Options settings. 2. SCHEDULERS .ST:s_m_MP_MV_Example] $$Bonus=1000 $$var_max=500 $$var_min=1200 $$var_count=0 CONFIGURING PARAMTER FILE We can specify the parameter file name and directory in the workflow or session properties. Open a Workflow in the Workflow Manager. 4. Click OK. Click Workflows > Edit.txt or $PMSourceFileDir\Para_File. Open a session in the Workflow Manager.txt [Practice. Click OK. 3. Enter the parameter directory and name in the Parameter Filename field. 3.In the parameter file. Example: D:\Files\Para_File. Create a text file in notepad with name Para_File. 2. Click the Properties tab. folder and session names are case sensitive. To enter a parameter file in the session properties: 1.txt 5. 5. Enter the parameter directory and name in the Parameter Filename field. To enter a parameter file in the workflow properties: 1.

enter a name for the scheduler. the workflow runs on demand. we must edit them and replace the missing scheduler. Run on Demand 2. Run Continuously 3. To make the workflows valid. The Workflow Manager marks a workflow invalid if we delete the scheduler associated with the workflow. By default. and end options for the schedule. In the General tab. If we choose a different Integration Service for the workflow or restart the Integration Service. Scheduler can be non-reusable or reusable. 4. all workflows that use the deleted scheduler becomes invalid. The Integration Service does not run the workflow if: The prior workflow run fails. Configure the scheduler settings in the Scheduler tab. or we can manually start a workflow. There are 3 run options: 1. 5. 2. We remove the workflow from the schedule The Integration Service is running in safe mode Creating a Reusable Scheduler    For each folder. When we delete a reusable scheduler. Click Add to add a new scheduler. Open the folder where we want to create the scheduler. The Integration Service runs a scheduled workflow as configured. If we change schedule settings.          A scheduler is a repository object that contains a set of schedule settings. schedule options. Use a reusable scheduler so we do not need to configure the same set of scheduling settings in each workflow. Run on Server initialization . repeat at a given time or interval.We can schedule a workflow to run continuously. Steps: 1. 6. it reschedules all workflows. the Integration Service removes workflows from the schedule. Configuring Scheduler Settings Configure the Schedule tab of the scheduler to set run options. Click Apply and OK. 3. click Workflows > Schedulers. start options. If we delete a folder. the Integration Service reschedules the workflow according to the new settings. In the Workflow Designer. We can change the schedule settings by editing the scheduler. the Workflow Manager lets us create reusable schedulers so we can reuse the same set of scheduling settings for workflows in the folder.

Run every: Run the workflow at regular intervals. The Integration Service then starts the next run of the workflow according to settings in Schedule Options. 3. Customized Repeat: Integration Service runs the workflow on the dates and times specified in the Repeat dialog box. Schedule options for Run on Server initialization:    Run Once: To run the workflow just once. The Integration Service then starts the next run of the workflow as soon as it finishes the previous run. Run on Demand: Integration Service runs the workflow when we start the workflow manually. Run Continuously: Integration Service runs the workflow as soon as the service initializes. 2. as configured. Run on Server initialization Integration Service runs the workflow as soon as the service is initialized. .1. Start options for Run on Server initialization: · Start Date · Start Time End options for Run on Server initialization:  End on: IS stops scheduling the workflow in the selected date.

Click the right side of the Scheduler field to edit scheduling settings for the non. we must 5. To reschedule a workflow on its original schedule. In the Workflow Designer. 9. Note: If we do not have a reusable scheduler in the folder. Browser dialog box. right-click the workflow in the Navigator window and choose Schedule Workflow. If we select Reusable. Click Workflows > Edit. 6. In the Scheduler tab. Click Ok. Points to Ponder :   To remove a workflow from its schedule. create one before we choose Reusable. Select Reusable if we want to select an existing reusable scheduler for the workflow. right-click the workflow in the Navigator window and choose Unscheduled Workflow. Creating a Non-Reusable Scheduler 1. 4.   End After: IS stops scheduling the workflow after the set number of workflow runs.reusable scheduler 7. 3. . Forever: IS schedules the workflow as long as the workflow does not fail. choose Non-reusable. open the workflow. 2. choose a reusable scheduler from the Scheduler 8.

Performance Tuning Overview .

You can also use the Pushdown Optimization Viewer to view the messages related to pushdown optimization. If you tune all the bottlenecks. transformation logic. The source or target database executes the SQL queries to process the transformations. and time the session both before and after the change. The amount of transformation logic you can push to the database depends on the database. first identify a performance bottleneck. The Integration Service processes all transformation logic that it cannot push to a database. change one variable at a time. eliminate it. Optimize the grid deployments. the Integration Service translates the transformation logic into SQL queries and sends the SQL queries to the database. Adding partitions can improve performance by utilizing more of the system hardware while processing the session. and then identify the next performance bottleneck until you are satisfied with the session performance. you can further optimize session performance by increasing the number of pipeline partitions in the session. When you run a session configured for pushdown optimization. Enables the Integration Service to read source data efficiently. 2. 4. Optimize the target. The following figure shows a mapping containing transformation logic that can be pushed to the source database: . 5. Enables the Integration Service to transform and move data efficiently. You can use the test load option to run sessions when you tune session performance. Enables the Integration Service to run on a grid with optimal performance. Use the Pushdown Optimization Viewer to preview the SQL statements and mapping logic that the Integration Service can push to the source or target database. Enables Power Center service processes to run more quickly. Optimize the system. Enables the Integration Service and Repository Service to function optimally. Enables the Integration Service to write to the targets efficiently. Complete the following tasks to improve session performance: 1. Enables the Integration Service to run the session more quickly. Optimize the transformation.The goal of performance tuning is to optimize session performance by eliminating performance bottlenecks. 6. 8. Enables the Integration Service to process transformations in a mapping efficiently. you might want to return to the original configuration. Because determining the best way to improve performance can be complex. Optimize the Power Center components. 3. Optimize the mapping. If session performance does not improve. Optimize the session. 7. PUSH DOWN OPTIMISATION You can push transformation logic to the source or target database using pushdown optimization. To tune session performance. and mapping and session configuration. Optimize the source.

ITEM_DESC FROM ITEMS2 ITEMS The Integration Service generates an INSERT SELECT statement to retrieve the ID. It generates an INSERT. an underscore. It concatenates the store number 5419. To push the transformation logic to the database. DELETE. Running Source-Side Pushdown Optimization Sessions When you run a session configured for source-side pushdown optimization. name. If the Integration Service cannot push all transformation logic to the database. it performs both sourceside and target-side pushdown optimization. and the original ITEM ID to get the new item ID. the Integration Service analyzes the mapping from the target to the source or until it reaches an upstream transformation it cannot push to the target database. ITEM_DESC) SELECT CAST((CASE WHEN 5419 IS NULL THEN '' ELSE 5419 END) + '_' + (CASE WHEN ITEMS. and description values from the source table. The Integration Service pushes as much transformation logic as possible to the target database. The Integration Service attempts to push all transformation logic to the target database.ITEM_ID IS NULL THEN '' ELSE ITEMS. Full pushdown optimization. the Integration Service generates the following SQL statement: INSERT INTO T_ITEMS(ITEM_ID. or UPDATE statement based on the transformation logic for each transformation it can push to the target database. Target-side pushdown optimization. Pushdown Optimization Types You can configure the following types of pushdown optimization:    Source-side pushdown optimization. ITEM_NAME. create new item IDs. and ITEM_DESC columns in the target table. ITEMS. and insert the values into the ITEM_ID. Running Target-Side Pushdown Optimization Sessions When you run a session configured for target-side pushdown optimization. ITEMS.ITEM_NAME. The Integration Service processes the transformation logic up to the . The Integration Service generates and executes a SELECT statement based on the transformation logic for each transformation it can push to the database. Then. ITEM_NAME. it reads the results of this SQL query and processes the remaining transformations.ITEM_ID END) AS INTEGER). The Integration Service pushes as much transformation logic as possible to the source database. the Integration Service analyzes the mapping from the source to the target or until it reaches a downstream transformation it cannot push to the source database.This mapping contains an Expression transformation that creates an item ID based on the store number 5419 and the item ID from the source.

Then. A long transaction increases the likelihood of an unexpected event. but not SYSTIMESTAMP (‘MS’) or SYSTIMESTAMP (‘US’). consider using source-side or target-side pushdown optimization. To minimize database performance issues for long transactions. . When you push TO_CHAR (DATE) or TO_DATE () to Netezza. not the Integration Service. you cannot push the function to a Teradata database. hours. When you run a session configured for full pushdown optimization. If the format is different. Running Full Pushdown Optimization Sessions To use full pushdown optimization. Oracle trims the date to the second. When you push SYSDATE or SYSTIMESTAMP to the database. When you run a session with large quantities of data and full pushdown optimization. When you push LAST_DAY () to Oracle. the database server returns the timestamp in the time zone of the database server. Oracle returns the date up to the second. the source and target databases must be in the same relational database management system. but the Integration Service treats the argument (' ') as spaces. Consider the following database performance issues when you generate a long transaction:    A long transaction uses more database resources.US format. the Integration Service analyzes the mapping from the source to the target or until it reaches a downstream transformation it cannot push to the target database. and you specify the format for SYSTIMESTAMP. A long transaction locks the database for longer periods of time. the database ignores the format and returns the complete time stamp.point that it can push the transformation logic to the database. If the input date contains sub seconds. When you push LTRIM. If you push SYSTIMESTAMP to an IBM DB2 or a Sybase database. it executes the generated SQL on the Target database. minutes. the Integration Service does not push the function to Netezza. the database server must run a long transaction. the database treats the argument (' ') as NULL. Rules and Guidelines for Functions in Pushdown Optimization Use the following rules and guidelines when pushing functions to a database:         If you use ADD_TO_DATE in transformation logic to change days. It generates and executes SQL statements against the source or target based on the transformation logic it can push to the database. or SOUNDEX to a database. You can push SYSTIMESTAMP (‘SS’) to a Netezza database. An IBM DB2 database and the Integration Service produce different results for STDDEV and VARIANCE. or seconds. RTRIM. IBM DB2 uses a different algorithm than other databases to calculate STDDEV and VARIANCE. This reduces database concurrency and increases the likelihood of deadlock. dates with sub second precision must be in the YYYY-MM-DD HH24: MI: SS.

PERFORMANCE TUNING OF LOOKUP TRANSFORMATIONS
Lookup transformations are used to lookup a set of values in another table.Lookups slows down the performance.
1. To improve performance, cache the lookup tables. Informatica can cache all the lookup and reference tables; this
makes operations run very fast. (Meaning of cache is given in point 2 of this section and the procedure for determining
the optimum cache size is given at the end of this document.)
2. Even after caching, the performance can be further improved by minimizing the size of the lookup cache. Reduce the
number of cached rows by using a sql override with a restriction.
Cache: Cache stores data in memory so that Informatica does not have to read the table each time it is referenced. This
reduces the time taken by the process to a large extent. Cache is automatically generated by Informatica depending on
the marked lookup ports or by a user defined sql query.
Example for caching by a user defined query: Suppose we need to lookup records where employee_id=eno.
‘employee_id’ is from the lookup table, EMPLOYEE_TABLE and ‘eno’ is the
input that comes from the from the source table, SUPPORT_TABLE.
We put the following sql query override in Lookup Transform
‘select employee_id from EMPLOYEE_TABLE’
If there are 50,000 employee_id, then size of the lookup cache will be 50,000.
Instead of the above query, we put the following:‘select emp employee_id from EMPLOYEE_TABLE e, SUPPORT_TABLE s
where e. employee_id=s.eno’
If there are 1000 eno, then the size of the lookup cache will be only 1000.But here the performance gain will happen
only if the number of records in SUPPORT_TABLE is not huge. Our concern is to make the size of the cache as less as
possible.
3. In lookup tables, delete all unused columns and keep only the fields that are used in the mapping.
4. If possible, replace lookups by joiner transformation or single source qualifier.Joiner transformation takes more time
than source qualifier transformation.
5. If lookup transformation specifies several conditions, then place conditions that use equality operator ‘=’ first in the
conditions that appear in the conditions tab.
6. In the sql override query of the lookup table, there will be an ORDER BY clause. Remove it if not needed or put
fewer column names in the ORDER BY list.
7. Do not use caching in the following cases: -Source is small and lookup table is large.
-If lookup is done on the primary key of the lookup table.

8. Cache the lookup table columns definitely in the following case: -If lookup table is small and source is large.
9. If lookup data is static, use persistent cache. Persistent caches help to save and reuse cache files. If several sessions
in the same job use the same lookup table, then using persistent cache will help the sessions to reuse cache files. In case
of static lookups, cache files will be built from memory cache instead of from the database, which will improve the
performance.
10. If source is huge and lookup table is also huge, then also use persistent cache.
11. If target table is the lookup table, then use dynamic cache. The Informatica server updates the lookup cache as it
passes rows to the target.
12. Use only the lookups you want in the mapping. Too many lookups inside a mapping will slow down the session.
13. If lookup table has a lot of data, then it will take too long to cache or fit in memory. So move those fields to source
qualifier and then join with the main table.
14. If there are several lookups with the same data set, then share the caches.
15. If we are going to return only 1 row, then use unconnected lookup.
16. All data are read into cache in the order the fields are listed in lookup ports. If we have an index that is even
partially in this order, the loading of these lookups can be speeded up.
17. If the table that we use for look up has an index (or if we have privilege to add index to the table in the database, do
so), then the performance would increase both for cached and un cached lookups.
Optimizing the Bottleneck’s
1. If the source is a flat file, ensure that the flat file is local to the Informatica server. If source is a relational table,
then try not to use synonyms or aliases.
2. If the source is a flat file, reduce the number of bytes (By default it is 1024 bytes per line) the Informatica reads
per line. If we do this, we can decrease the Line Sequential Buffer Length setting of the session properties.
3. If possible, give a conditional query in the source qualifier so that the records are filtered off as soon as possible
in the process.
4. In the source qualifier, if the query has ORDER BY or GROUP BY, then create an index on the source table and
order by the index field of the source table.
PERFORMANCE TUNING OF TARGETS
If the target is a flat file, ensure that the flat file is local to the Informatica server. If target is a relational table, then try
not to use synonyms or aliases.
1. Use bulk load whenever possible.
2. Increase the commit level.
3. Drop constraints and indexes of the table before loading.
PERFORMANCE TUNING OF MAPPINGS
Mapping helps to channel the flow of data from source to target with all the transformations in between. Mapping is the
skeleton of Informatica loading process.
1. Avoid executing major sql queries from mapplets or mappings.

2. Use optimized queries when we are using them.
3. Reduce the number of transformations in the mapping. Active transformations like rank, joiner, filter,
aggregator etc should be used as less as possible.
4. Remove all the unnecessary links between the transformations from mapping.
5. If a single mapping contains many targets, then dividing them into separate mappings can improve performance.
6. If we need to use a single source more than once in a mapping, then keep only one source and source qualifier
in the mapping. Then create different data flows as required into different targets or same target.
7. If a session joins many source tables in one source qualifier, then an optimizing query will improve
performance.
8. In the sql query that Informatica generates, ORDERBY will be present. Remove the ORDER BY clause if not
needed or at least reduce the number of column names in that list. For better performance it is best to order by
the index field of that table.
9. Combine the mappings that use same set of source data.
10. On a mapping, field with the same information should be given the same type and length throughout the
mapping. Otherwise time will be spent on field conversions.
11. Instead of doing complex calculation in query, use an expression transformer and do the calculation in the
mapping.
12. If data is passing through multiple staging areas, removing the staging area will increase performance.
13. Stored procedures reduce performance. Try to keep the stored procedures simple in the mappings.
14. Unnecessary data type conversions should be avoided since the data type conversions impact performance.
15. Transformation errors result in performance degradation. Try running the mapping after removing all
transformations. If it is taking significantly less time than with the transformations, then we have to fine-tune
the transformation.
16. Keep database interactions as less as possible.
PERFORMANCE TUNING OF SESSIONS
A session specifies the location from where the data is to be taken, where the transformations are done and where the
data is to be loaded. It has various properties that help us to schedule and run the job in the way we want.
1. Partition the session: This creates many connections to the source and target, and loads data in parallel
pipelines. Each pipeline will be independent of the other. But the performance of the session will not improve if
the number of records is less. Also the performance will not improve if it does updates and deletes. So session
partitioning should be used only if the volume of data is huge and the job is mainly insertion of data.
2. Run the sessions in parallel rather than serial to gain time, if they are independent of each other.
3. Drop constraints and indexes before we run session. Rebuild them after the session run completes. Dropping
can be done in pre session script and Rebuilding in post session script. But if data is too much, dropping indexes
and then rebuilding them etc. will be not possible. In such cases, stage all data, pre-create the index, use a
transportable table space and then load into database.
4. Use bulk loading, external loading etc. Bulk loading can be used only if the table does not have an index.
5. In a session we have options to ‘Treat rows as ‘Data Driven, Insert, Update and Delete’. If update strategies are
used, then we have to keep it as ‘Data Driven’. But when the session does only insertion of rows into target
table, it has to be kept as ‘Insert’ to improve performance.
6. Increase the database commit level (The point at which the Informatica server is set to commit data to the target
table. For e.g. commit level can be set at every every 50,000 records)
7. By avoiding built in functions as much as possible, we can improve the performance. E.g. For concatenation,
the operator ‘||’ is faster than the function CONCAT (). So use operators instead of functions, where possible.

then the target table is the bottleneck. stored procedures. sessions etc. without the test filter. network. Identify bottleneck in Target If the target is a relational table. Preference should be in the opposite order. mapping and session. 2. use delimited strings in case the source flat files or use varchar data type. Manipulating high precision data types will slow down Informatica server. Synonyms and aliases slow down the performance. put a filter transformation in the mapping. The performance of the Informatica server is related to network connections. session should take less time. So all records will be filtered off and none will proceed to other parts of the mapping. Localize all source and target tables. views. Total Time = time taken by source So if source was fine. If the time taken now is very much less than the time taken for the session to load to table.database or network. Identify bottleneck in Source If source is a relational table. then substitute it with a flat file and run the session. Identify bottleneck in Transformation . reduce the performance to a big extent in this order.Data generally moves across a network at less than 1 MB per second. Still if the session takes near equal time as former case. then there is a source bottleneck. IS_NUMBER (). 3. just after source qualifier. and rtrim reduce the performance. After identifying the bottleneck. sequences etc. make the condition of filter to FALSE. If the source and target are flat files.In original case. target. stored procedures and queries used in Informatica should be tuned well. transformations. IFF (). ltrim. Optimize target databases. then they should be present in the system in which the Informatica server is present. we have to identify the bottleneck first. String functions like substring. IDENTIFICATION OF BOTTLENECKS Performance of Informatica is dependant on the performance of its several components like database. then in the latter case. It is best to identify performance issue in components in the order source. apply the tuning mechanisms in whichever way they are applicable to the project. total time taken is as follows:Total Time = time taken by (source + transformations + target load) Now because of filter. 8.The functions like IS_SPACES (). 9. transformations. So avoid network connections. whereas a local disk moves data five to twenty times faster. So disable ‘high precision’. DECODE () etc. DATABASE OPTIMISATION To gain the best Informatica performance. mapping. 4. 10. target. In the sources. Thus network connections often affect on session performance. mappings. Increase the network packet size. Bottleneck may be present in source. transformations. Try not to connect across synonyms. session. To tune the performance of Informatica. the database tables. 1.

000000] secs. then that part is the bottleneck. then the transformation is the bottleneck. If the session run takes equal time with and without this test filter. Session logs contain thread summary records like the following:MASTER> PETL_24018 Thread [READER_1_1_1] created for the read stage of partition point [SQ_test_all_text_data] has completed: Total Run Time =[11.703201] secs. If busy percentage is 100. Busy Percentage = [100. Total Idle Time = [0. all the performance related information would appear in the log created by the session.Remove the transformation from the mapping and run it. Identify bottleneck in sessions We can use the session log to identify whether the source.764368] secs.000000]. . Note the time taken.Then put the transformation back and run the mapping again. target or transformations are the performance bottleneck. Busy Percentage =[18. But removal of transformation for testing can be a pain for the developer since that might require further changes for the session to get into the ‘working mode’. Total Idle Time = [9. So we can put filter with the FALSE condition just after the transformation and run the session. If the time taken now is significantly more than previous time. Once the ‘Collect Performance Data’ option (In session ‘Properties’ tab) is enabled.560945] secs.304876].then transformation is the bottleneck. MASTER> PETL_24019 Thread [TRANSF_1_1_1_1] created for the transformation stage of partition point [SQ_test_all_text_data] has completed: Total Run Time = [11. Basically we have to rely on thread statistics to identify the cause of performance issues.

We can run the Session tasks sequentially or concurrently. We can create reusable tasks in the Task Developer. The Power Center Server creates several files and in-memory caches depending on the transformations and options used in the session. EMAIL TASK   The Workflow Manager provides an Email task that allows us to send email during a workflow. . Created by Administrator usually and we just drag and use it in our mapping. depending on our needs. Types of tasks: Task Type Tool where task can be created Reusable or not Session Task Developer Yes Email Workflow Designer Yes Command Worklet Designer Yes Event-Raise Workflow Designer No Event-Wait Worklet Designer No Timer No Decision No Assignment No Control No SESSION TASK     A session is a set of instructions that tells the Power Center Server how and when to move data from sources to targets. To run a session. we must first create a workflow to contain the Session task. We can run as many sessions in a workflow as we need.WORKING WITH TASKS –Part 1 17 The Workflow Manager contains many types of tasks to help you build workflows and worklets.

Steps: 1. Example: To send an email when a session completes: Steps: 1. Double-click the Email task in the workspace. Pre. Enter the subject of the email in the Email Subject field. Validate workflow and Repository -> Save   We can also drag the email task and use as per need. or archive target files. Edit Session task and go to Components tab. 5. choose Tasks-Create. Command: COPY D:\sample. Example: to copy a file sample. . 7. Select an Email task and enter a name for the task. 7. 2. 2. This is done in COMPONENTS TAB of a session. Click OK twice to save your changes. Click the Properties tab. Select the Value and Type option as we did in Email task. COMMAND TASK The Command task allows us to specify one or more shell commands in UNIX or DOS commands in Windows to run during the workflow. In Value. we can specify shell commands in the Command task to delete reject files. For example. We can set the option to send email on success or failure in components tab of a session task. The Edit Tasks dialog box appears. 6. 8. In the Task Developer or Workflow Designer. We can run it in Pre-Session Command or Post Session Success Command or Post Session Failure Command. 2. In Type select reusable or Non-reusable.or post-session shell command for a Session task. Click Done. 3. Enter the fully qualified email address of the mail recipient in the Email User Name field. Click the Open button in the Email Text field to open the Email Editor. Ways of using command task: 1. In the Task Developer or Workflow Designer. copy a file. 9. 3.txt E:\ in windows Steps for creating command task: 1. choose Tasks-Create. select the email task to be used. 6. 4. you can leave this field blank. Standalone Command task: We can use a Command task anywhere in the workflow or worklet to run shell commands. See On Success Email Option there and configure it.txt from D drive to E. 4.and post-session shell command: We can call a Command task as the pre. Click Create. Or. 8. Click Apply -> Ok. Create a workflow wf_sample_email Drag any session task to workspace. 5.

Then click done. Workflow -> Create -> Give name and click ok. User-defined event: A user-defined event is a sequence of tasks in the Workflow. click the Edit button to open the Command Editor. 9. Start is displayed. Go to commands tab. Steps to create the workflow using command task: 1. Create a task using the above steps to copy a file in Task Developer. Repeat steps 5-9 to add more commands in the task. click the Add button to add a command. Open any workflow where we want to create an event. Types of Events:   Pre-defined event: A pre-defined event is a file-watch event. 6. 7. 10.2. 4. Open Workflow Designer. 2. We create events and then raise them as per need. Link Start to Session task and Session to Command Task. Double-click the Command task. 3. 3. 5. 4. 7. enter a name for the new command. 5. Select Command Task for the task type. 8. This event Waits for a specified file to arrive at a given location. Enter a name for the Command task. Click Create. . Double click link between Session and Command and give condition in editor as $S_M_FILTER_EXAMPLE.Status=SUCCEEDED Workflow-> Validate Repository –> Save WORKING WITH EVENT TASKS We can define events in the workflow to specify the sequence of task execution. In the Commands tab. Drag session say s_m_Filter_example and command task. In the Command field. 6. Steps for creating User Defined Event: 1. Click OK. 8. Enter only one command in the Command Editor. 11. In the Name field. Click OK to close the Command Editor.

Click create and done. Workflow validate and Repository Save. 5. give directory and filename to watch. Example1: Use an event wait task and make sure that session s_filter_example runs when abc. Task -> Create -> Select Event Wait. Click Workflow-> Edit -> Events tab. Workflow -> Create -> Give name wf_event_wait_file_watch -> Click ok. 3. 2.2. Drag s_filter_example to workspace and link it to event wait task. 4. 3. Types of Events Tasks:   EVENT RAISE: Event-Raise task represents a user-defined event. Validate the workflow and Save it. Link Start to Event Wait task.txt file is present in D:\FILES folder.tct 7. EVENT WAIT: Event-Wait task waits for a file watcher event or user defined event to occur before executing the next session in the workflow. Steps for creating workflow: 1. Right click on event wait task and click EDIT -> EVENTS tab. Capture this event in event wait task and run session S_M_TOTAL_SAL_EXAMPLE Steps for creating workflow: . 6. Click Apply -> Ok. Example: D:\FILES\abc. Select Pre Defined option there. In the blank space. Give name. Example 2: Raise a user defined event when session s_m_filter_example succeeds. Click to Add button to add events and give the names as per need. We use this task to raise a user defined event. 4.

Workflow -> Edit -> Events Tab and add events EVENT1 there. WORKING WITH TASKS –Part 2 .1. Reactions: You can follow any responses to this entry through the RSS 2. 6.Status=SUCCEEDED 8. Link EW_WAIT to START task. Apply -> OK. 13. Drag S_M_TOTAL_SAL_EXAMPLE and link it to EW_WAIT.Link ER_Example to s_m_filter_example. 2. Click link between ER_Example and s_m_filter_example and give the condition $S_M_FILTER_EXAMPLE. Click Tasks -> Create -> Select EVENT RAISE from list. 5. 16. 7. 12. Right click EW_WAIT -> EDIT-> EVENTS tab. Give name ER_Example. 3. Repository -> Save. Give name EW_WAIT. 4. Run workflow and see. 10. 14. Select User Defined there. Apply -> OK. Select the Event1 by clicking Browse Events button. Drag s_m_filter_example and link it to START task. Click Tasks -> Create -> Select EVENT WAIT from list. Click Create and then done. Mapping -> Validate 15. 11. Workflow -> Create -> Give name wf_event_wait_event_raise -> Click ok. Right click ER_Example -> EDIT -> Properties Tab -> Open Value for User Defined Event and Select EVENT1 from the list displayed. Click Create and then done. 9.0 feed. You can leave a response.

Link TIMER_Example to START task. Apply -> OK. Click Tasks -> Create -> Select TIMER from list. Workflow-> Validate and Repository -> Save. DECISION TASK . 5. Select Relative Time Option and Give 1 min and Select ‘From start time of this task’ Option. 2. Relative time: We instruct the Power Center Server to wait for a specified period of time after the Timer task. Workflow -> Create -> Give name wf_timer_task_example -> Click ok.TIMER TASK The Timer task allows us to specify the period of time to wait before the Power Center Server runs the next task in the workflow. Right click TIMER_Example-> EDIT -> TIMER tab. the parent workflow. Drag s_m_filter_example and link it to TIMER_Example. Give name TIMER_Example. 6. 3. The next task in workflow will run as per the date and time specified. The Timer task has two types of settings:   Absolute time: We specify the exact date and time or we can choose a user-defined workflow variable to specify the exact time. 8. 4. Click Create and then done. Example: Run session s_m_filter_example relative to 1 min after the timer task. 7. Steps for creating workflow: 1. or the top-level workflow starts.

Status = SUCCEEDED 7. Give name DECISION_Example. Click Tasks -> Create -> Select DECISION from list. Drag s_m_filter_example and S_M_TOTAL_SAL_EXAMPLE to workspace and link both of them to START task. The Decision task has a pre-defined variable called $Decision_task_name. 6. 4. CONTROL TASK . 12. 11. Click Create and then done. Validate the condition -> Click Apply -> OK. We can specify one decision condition per Decision task. 3. Validate and click OK. If any of s_m_filter_example or S_M_TOTAL_SAL_EXAMPLE fails then S_m_sample_mapping_EMP should run. Run workflow and see the result.    The Decision task allows us to enter a condition that determines the execution of the workflow. 5. Drag command task and S_m_sample_mapping_EMP task to workspace and link them to DECISION_Example task. Double click link between Command task and DECISION_Example and give the condition: $DECISION_Example. Workflow -> Create -> Give name wf_decision_task_example -> Click ok. Link DECISION_Example to both s_m_filter_example and S_M_TOTAL_SAL_EXAMPLE. Example: Command Task should run only if either s_m_filter_example or S_M_TOTAL_SAL_EXAMPLE succeeds.condition that represents the result of the decision condition.Condition = 1. similar to a link condition. Open the Expression editor by clicking the VALUE section of Decision Name attribute and enter the following condition: $S_M_FILTER_EXAMPLE. Default is AND. 10. Right click DECISION_Example-> EDIT -> GENERAL tab. Workflow Validate and repository Save. Double click link between S_m_sample_mapping_EMP & DECISION_Example & give the condition: $DECISION_Example. Steps for creating workflow: 1. 8. Apply and click OK. The Power Center Server evaluates the condition in the Decision task and sets the pre-defined condition variable to True (1) or False (0). 9. Set ‘Treat Input Links As’ to OR. Validate & click OK. 2. Now edit decision task again and go to PROPERTIES Tab.Status = SUCCEEDED OR $S_M_TOTAL_SAL_EXAMPLE.Condition = 0.

Give name cntr_task. Right click cntr_task-> EDIT -> GENERAL tab. Set ‘Treat Input Links As’ to OR. Default is AND. Repeat above step for remaining 2 sessions also. Control Option Description Fail Me Fails the control task. Run workflow and see the result. A parent workflow or worklet is the workflow or worklet that contains the Control task. 8. 9. Double click link between cntr_task and any session say s_m_filter_example and give the condition: $S_M_FILTER_EXAMPLE. Fail Top-Level WF Stop Top-Level WF Abort TopLevel WF Fails the workflow that is running. Fail Parent Marks the status of the WF or worklet that contains the Control task as failed. Click Apply and OK. then Abort the top level workflow. Workflow -> Create -> Give name wf_control_task_example -> Click ok. 3. We give the condition to the link connected to Control Task. 5. 6.Status = SUCCEEDED.   We can use the Control task to stop. Steps for creating workflow: 1. Aborts the workflow that is running. . Stops the workflow that is running. 12. Abort Parent Aborts the WF or worklet that contains the Control task. abort. Click Tasks -> Create -> Select CONTROL from list. 2. Stop Parent Stops the WF or worklet that contains the Control task. 4. Link all sessions to the control task cntr_task. or fail the top-level workflow or the parent workflow based on an input link condition. Example: Drag any 3 sessions and if anyone fails. Go to PROPERTIES tab of cntr_task and select the value ‘Fail top level 10. Workflow’ for Control Option. Drag any 3 sessions to workspace and link all of them to START task. 7. Click Create and then done. 11. Workflow Validate and repository Save.

2. Enter the value or expression you want to assign. 7. until a salesperson is transferred from one regional office to another. Creating sales reports seems simple enough. Enter a name for the Assignment task. her totals will look much stronger than the other . Select the variable for which you want to assign a value. 12. Click the Edit button in the Expression field to open the Expression Editor. rather than changing on a time-based. 10. See Workflow variable topic to add user defined variables. 9. Click Create. Click the Open button in the User Defined Variables field. regular schedule For example. that might give misleading information. you may have a dimension in your database that tracks the sales records of your company's salespeople. How do you record such a change in your sales dimension? You could sum or average the sales by salesperson. Select Assignment Task for the task type. If the salesperson that was transferred used to work in a hot market where sales were easy. Steps to create Assignment Task: 1. Click OK. Then click Done. To use an Assignment task in the workflow. Edit Workflow and add user defined variables. 11. Click OK. 3. Open any workflow where we want to use Assignment task. first create and add the Assignment task to the workflow. Choose Tasks-Create. Then configure the Assignment task to assign values or expressions to userdefined variables. Double-click the Assignment task to open the Edit Task dialog box. but if you use that to compare the performance of salesmen. 4. Repeat steps 7-10 to add more variable assignments as necessary. 6. SCD – Type 1 17 Slowly Changing Dimensions (SCDs) are dimensions that have data that changes slowly.ASSIGNMENT TASK      The Assignment task allows us to assign a value to a user-defined workflow variable. click Add to add an assignment. On the Expressions tab. and now works in a market where sales are infrequent. 5. We cannot assign values to pre-defined workflow. 8.

and therefore does not track historical data at all. (Assuming you won't ever need to know how it used to be misspelled in the past. However. Now imagine that this supplier moves their headquarters to Illinois.salespeople in her new region. for example. The updated table would simply overwrite this record: Supplier_Key Supplier_Code Supplier_Name Supplier_State 123 ABC Acme Supply Co IL The obvious disadvantage to this method of managing SCDs is that there is no historical record kept in the data warehouse. You can't tell if your suppliers are tending to move to the Midwest. such as the spelling of a name. even if they are just as good. Technically. But an advantage to Type 1 SCDs is that they are very easy to maintain. Or you could create a second salesperson record and treat the transferred person as a new sales person. but that creates problems also. Explanation with an Example: Source Table: (01-01-11) Target Table: (01-01-11) Emp no Ename Sal 101 A 1000 102 B 2000 103 C 3000 Emp no Ename Sal 101 A 1000 102 B 2000 103 C 3000 . Dealing with these issues involves SCD management methodologies: Type 1: The Type 1 methodology overwrites old data with new data. This is most appropriate when correcting certain types of data errors. since the table will be unique by the natural key (Supplier_Code). the surrogate key is not necessary. Supplier_Code is the natural key and Supplier_Key is a surrogate key.) Here is an example of a database table that keeps supplier information: Supplier_Key Supplier_Code Supplier_Name Supplier_State 123 ABC Acme Supply Co CA In this example. the joins will perform better on an integer than on a character string.

The snap shot of the connections using different kinds of transformations are shown below. Step 2: Design the mapping and apply the necessary transformation. emp_target2.The necessity of the lookup transformation is illustrated using the above source and target table. Step 1: Is to import Source Table and Target table. .      Create a table by name emp_source with three columns as shown above in oracle. Source Table: (01-02-11) Target Table: (01-02-11) Emp no Ename Sal Empno Ename Sal 101 A 1000 101 A 1000 102 B 2500 102 B 2500 103 C 3000 103 C 3000 104 D 4000 104 D 4000  In the second Month we have one more employee added up to the table with the Ename D and salary of the Employee is changed to the 2500 instead of 2000. In the same way as above create two target tables with the names emp_target1. Go to the targets Menu and click on generate and execute to confirm the creation of the target tables. Import the source from the source analyzer.

Here in this transformation we are about to use four kinds of transformations namely Lookup transformation,
Expression Transformation, Filter Transformation, Update Transformation. Necessity and the usage of all the
transformations will be discussed in detail below.

Look up Transformation: The purpose of this transformation is to determine whether to insert, Delete, Update or
reject the rows in to target table.




The first thing that we are goanna do is to create a look up transformation and connect the Empno from the
source qualifier to the transformation.
The snapshot of choosing the Target table is shown below.

What Lookup transformation does in our mapping is it looks in to the target table (emp_table) and compares it
with the Source Qualifier and determines whether to insert, update, delete or reject rows.
In the Ports tab we should add a new column and name it as empno1 and this is column for which we are gonna
connect from the Source Qualifier.
The Input Port for the first column should be unchked where as the other ports like Output and lookup box
should be checked. For the newly created column only input and output boxes should be checked.
In the Properties tab (i) Lookup table name ->Emp_Target.

(ii)Look up Policy on Multiple Mismatch -> use First Value.
(iii) Connection Information ->Oracle.

In the Conditions tab (i) Click on Add a new condition

(ii)Lookup Table Column should be Empno, Transformation port should be Empno1 and Operator should ‘=’.

Expression Transformation: After we are done with the Lookup Transformation we are using an expression
transformation to check whether we need to insert the records the same records or we need to update the records. The
steps to create an Expression Transformation are shown below.

Drag all the columns from both the source and the look up transformation and drop them all on to the
Expression transformation.
Now double click on the Transformation and go to the Ports tab and create two new columns and name it as
insert and update. Both these columns are gonna be our output data so we need to have check mark only in front
of the Output check box.
The Snap shot for the Edit transformation window is shown below.

The condition that we want to parse through our output data are listed below.


Input à IsNull(EMPNO1)
Output à iif(Not isnull (EMPNO1) and Decode(SAL,SAL1,1,0)=0,1,0) .

We are all done here .Click on apply and then OK.

Filter Transformation: we are gonna have two filter transformations one to insert and other to update.


Connect the Insert column from the expression transformation to the insert column in the first filter
transformation and in the same way we are gonna connect the update column in the expression transformation
to the update column in the second filter.
Later now connect the Empno, Ename, Sal from the expression transformation to both filter transformation.
If there is no change in input data then filter transformation 1 forwards the complete input to update strategy
transformation 1 and same output is gonna appear in the target table.


If there is any change in input data then filter transformation 2 forwards the complete input to the update
strategy transformation 2 then it is gonna forward the updated input to the target table.
Go to the Properties tab on the Edit transformation

(i) The value for the filter condition 1 is Insert.
(ii) The value for the filter condition 1 is Update.

The Closer view of the filter Connection is shown below.

Update Strategy Transformation: Determines whether to insert, delete, update or reject the rows.



Drag the respective Empno, Ename and Sal from the filter transformations and drop them on the respective
Update Strategy Transformation.
Now go to the Properties tab and the value for the update strategy expression is 0 (on the 1st update
transformation).
Now go to the Properties tab and the value for the update strategy expression is 1 (on the 2nd update
transformation).
We are all set here finally connect the outputs of the update transformations to the target table.

Step 3: Create the task and Run the work flow.


Don’t check the truncate table option.
Change Bulk to the Normal.
Run the work flow from task.

Step 4: Preview the Output in the target table.

We are gonna use the SCD-2 style to extract and load the records in to target table.  The thing to be noticed here is if there is any update in the salary of any employee then the history of that employee is displayed with the current date as the start date and the previous date as the end date. Source Table: (01-01-11) . There is a new employee added and one change in the records in the month (01-02-2010).Type 2 SCD 2 (Complete): Let us drive the point home using a simple scenario.. For eg..(01-01-2010) we are provided with an source table with the three columns and three rows in it like (EMpno.Sal). in the current month ie.Ename.

Emp no Ename Sal 101 A 1000 102 B 2000 103 C 3000 Target Table: (01-01-11) Skey Emp no Ename Sal S-date E-date Ver Flag 100 101 A 1000 01-01-10 Null 1 1 200 102 B 2000 01-01-10 Null 1 1 300 103 C 3000 01-01-10 Null 1 1 Source Table: (01-02-11) Emp no Ename Sal 101 A 1000 102 B 2500 103 C 3000 104 D 4000 Target Table: (01-02-11) Skey Emp no Ename Sal S-date E-date Ver Flag 100 101 A 1000 01-02-10 Null 1 1 200 102 B 2000 01-02-10 Null 1 1 300 103 C 3000 01-02-10 Null 1 1 201 102 B 2500 01-02-10 01-01-10 2 0 400 104 D 4000 01-02-10 1 1 Null In the second Month we have one more employee added up to the table with the Ename D and salary of the Employee is changed to the 2500 instead of 2000. Import the source from the source analyzer. . Step 1: Is to import Source Table and Target table.   Create a table by name emp_source with three columns as shown above in oracle.

 Here in this transformation we are about to use four kinds of transformations namely Lookup transformation (1).E_Date). The snapshot of choosing the Target table is shown below. Necessity and the usage of all the transformations will be discussed in detail below.  In The Target Table we are goanna add five columns (Skey. Look up Transformation: The purpose of this transformation is to Lookup on the target table and to compare the same with the Source using the Lookup Condition.   The first thing that we are gonna do is to create a look up transformation and connect the Empno from the source qualifier to the transformation. S_date . Sequence Generator. Filter Transformation (2).   Drag the Target table twice on to the mapping designer to facilitate insert or update process. Step 2: Design the mapping and apply the necessary transformation. Version. Go to the targets Menu and click on generate and execute to confirm the creation of the target tables. Flag. Expression Transformation (3). . The snap shot of the connections using different kinds of transformations are shown below.

 In the Conditions tab (i) Click on Add a new condition (ii)Lookup Table Column should be Empno. We specify the condition here whether to insert or to update the table. In the Properties tab (i) Lookup table name ->Emp_Target. Expression Transformation: After we are done with the Lookup Transformation we are using an expression transformation to find whether the data on the source table matches with the target table. Transformation port should be Empno1 and Operator should ‘=’. Now double click on the Transformation and go to the Ports tab and create two new columns and name it as insert and update. Both these columns are goanna be our output data so we need to have unchecked input check box. The steps to create an Expression Transformation are shown below. (iii) Connection Information ->Oracle. .    Drag all the columns from both the source and the look up transformation and drop them all on to the Expression transformation. (ii)Look up Policy on Multiple Mismatch -> use Last Value. The Input Port for only the Empno1 should be checked.   Drag the Empno column from the Source Qualifier to the Lookup Transformation. The Snap shot for the Edit transformation window is shown below.

0) . If there is any change in input data then filter transformation 2 forwards the complete input to the Exp 2 then it is gonna forward the updated input to the target table.1. Insert : IsNull(EmpNO1) Update: iif(Not isnull (Skey) and Decode(SAL.SAL1. .  The closer view of the connections from the expression to the filter is shown below.  We are all done here . The condition that we want to parse through our output data are listed below. (ii) The value for the filter condition 2 is Update.1. Go to the Properties tab on the Edit transformation (i) The value for the filter condition 1 is Insert.    If there is no change in input data then filter transformation 1 forwards the complete input to Exp 1 and same output is goanna appear in the target table.0)=0.Click on apply and then OK. Filter Transformation: We need two filter transformations the purpose the first filter is to filter out the records which we are goanna insert and the next is vice versa.

Sequence Generator: We use this to generate an incremental cycle of sequential range of number. Expression Transformation: . Connect the output of the sequence transformation to the Exp 1.The purpose of this in our mapping is to increment the skey in the bandwidth of 100.   We are gonna have a sequence generator and the purpose of the sequence generator is to increment the values of the skey in the multiples of 100 (bandwidth of 100).

F    Drag all the columns from the filter 2 to the Exp 2. Version is also made as output and expression parsed through it is 1. Now add a new column as N_skey and the expression for it is gonna be Skey+1. Flag is also made as output and expression parsed through it is 1. Else the there is no modification done on the target table.Exp 1: It updates the target table with the skey values. Point to be noticed here is skey gets multiplied by 100 and a new row is generated if there is any new EMP added to the list. Now add a new column as N_skey and the expression for it is gonna be Nextval1*100. .      Drag all the columns from the filter 1 to the Exp 1. Both the S_date and E_date is gonna be sysdate. We are goanna make the s-date as the o/p and the expression for it is sysdate. Exp 2: If same employee is found with any updates in his records then Skey gets added by 1 and version changes to the next higher number.

Update Strategy: This is place from where the update instruction is set on the target table. . Create the task and run the work flow. Step 3: Create the task and Run the work flow.Exp 3: If any record of in the source table gets updated then we make it only as the output.  The update strategy expression is set to 1. Change Bulk to the Normal. Run the work flow from task.     Don’t check the truncate table option.  If change is found then we are gonna update the E_Date to S_Date.

Step 4: Preview the Output in the target table. .

and we are goanna use skey as the Primary key here.26 SCD –Type 3: This Method has limited history preservation. Source table: (01-01-2011) Empno Ename Sal 101 A 1000 102 B 2000 103 C 3000 Target Table: (01-01-2011) Empno Ename C-sal P-sal 101 A 1000 - 102 B 2000 - 103 C 3000 - Source Table: (01-02-2011) Empno Ename Sal .

101 A 1000 102 B 4566 103 C 3000 Target Table (01-02-2011): Empno Ename C-sal P-sal 101 A 1000 - 102 B 4566 Null 103 C 3000 - 102 B 4544 4566 So hope u got what I’m trying to do with the above tables: Step 1: Initially in the mapping designer I’m goanna create a mapping as below. Look up Transformation: The look Transformation looks the target table and compares the same with the source table.Stuff’s logically. expression. And in this mapping I’m using lookup. insert. and delete the data from being loaded in to the target table. Prior to this Look up transformation has to look at the target table. update strategy to drive the purpose. Add two Ports and Rename them as Insert. .   Drag all the ports from the Source Qualifier and Look up in to Expression. Step 2: here we are goanna see the purpose and usage of all the transformations that we have used in the above mapping. Based on the Look up condition it decides whether we need to update. Update. Finally specify that connection Information (Oracle) and look up policy on multiple mismatches (use last value) in the Properties tab.    As usually we are goanna connect Empno column from the Source Qualifier and connect it to look up transformation. filter. Next to this we are goanna specify the look up condition empno =empno1. Expression Transformation: We are using the Expression Transformation to separate out the Insert-stuff’s and Update. Explanation of each and every Transformation is given below.

Filter 1:   Drag the Insert and other three ports which came from source qualifier in to the Expression in to first filter. In the Properties tab specify the condition as the 1 or dd_update. In the Properties tab specify the Filter condition as update.Curr_Sal. In the Properties tab specify the Filter condition as Insert.1. Update Strategy 2: This is intended to update in to the target table. Update Strategy 1: This is intended to insert in to the target table.  In the Properties tab specify the condition as the 0 or dd_insert.0)=0. These two ports are goanna be just output ports.  Drag all the ports except the insert from the first filter in to this. Specify the below conditions in the Expression editor for the ports respectively.   Drag all the ports except the update from the second filter in to this. Update Strategy: Finally we need the update strategy to insert or to update in to the target table.0) Filter Transformation: We are goanna use two filter Transformation to filter out the data physically in to two separate sections one for insert and the other for the update process to happen. Insert: isnull(ENO1 ) Update: iif(not isnull(ENO1) and decode(SAL.1. Filter 2:   Drag the update and other four ports which came from Look up in to the Expression in to Second filter. .

Step 4: Observe the output it would same as the second target table .Finally connect both the update strategy in to two instances of the target. Step 3: Create a session for this mapping and Run the work flow.

the Integration Service processes the entire source. rather than forcing it to process the entire source and recalculate the same data each time you run the session. You can capture those incremental changes because you have added a filter condition to the mapping that removes pre-existing data from the flow of data. If processing the incrementally changed source alters more than half the existing target. For each input record. Use incremental aggregation when the changes do not significantly change the target.Consider using incremental aggregation in the following circumstances:   You can capture new source data. In this case.Incremental Aggregation: When we enable the session option-> Incremental Aggregation the Integration Service performs incremental aggregation. and saves the incremental change. drop the table and recreate the target with complete source data. When the session runs with incremental aggregation enabled for the first time on March 1. the Integration Service checks historical information in the index file for a corresponding group. you filter out all the records except those time-stamped March 2. On March 2. The Integration Service then processes the new data and updates the target accordingly. It saves modified aggregate data in the index and data files to be used as historical data the next time you run the session. the index file and the data file. You then enable incremental aggregation. the Integration Service stores aggregate data from that session run in two files. Incremental changes do not significantly change the target. Note: Do not use incremental aggregation if the mapping contains percentile or median functions. When using incremental aggregation. As a result. the Integration Service performs the aggregate operation incrementally. the Integration Service does not store incremental aggregation values for percentile and median functions in disk caches. . Integration Service Processing for Incremental Aggregation (i)The first time you run an incremental aggregation session. If it does not find a corresponding group. the session may not benefit from using incremental aggregation. (ii)Each subsequent time you run the session with incremental aggregation. Use incremental aggregation when you can capture new source data each time you run the session. you use the incremental source changes in the session. you use the entire source. you might have a session using a source that receives new data every day. If it finds a corresponding group. using the aggregate data for that group. configure the Integration Service to overwrite existing aggregate data with new aggregate data. The Integration Service creates the files in the cache directory specified in the Aggregator transformation properties. (iii)When writing to the target. when you run the session again. This allows the Integration Service to read and store the necessary aggregate data. you can configure the session to process those changes. For example. At the end of the session. the Integration Service creates a new group and saves the record data. This allows the Integration Service to update the target incrementally. it passes source data through the mapping and uses historical cache data to perform aggregation calculations incrementally. If the source changes incrementally and you can capture changes. (iv) If the source changes significantly and you want the Integration Service to continue saving aggregate data for future incremental changes. the Integration Service applies the changes to the existing target. Use a Stored Procedure or Filter transformation to process new data. The Integration Service uses system memory to process these functions in addition to the cache memory you configure in the session properties. you apply captured changes in the source to aggregate calculations in a session.

Delete cache files. The Integration Service creates new aggregate data. by using the process variable for all sessions using incremental aggregation. $PMCacheDir.Each subsequent time you run a session with incremental aggregation. the Integration Service creates a backup of the incremental aggregation files. The cache directory for the Aggregator transformation must contain enough disk space for two sets of the files. When the Integration Service rebuilds incremental aggregation files. you can easily change the cache directory when necessary by changing $PMCacheDir. Configuring the Mapping Before enabling incremental aggregation.     The index and data files grow in proportion to the source data. Note: To protect the incremental aggregation files from file corruption or disk failure. However. Then. Configuring the Session Use the following guidelines when you configure the session for incremental aggregation: (i) Verify the location where you want to store the aggregate files. In a grid. you need to configure both mapping and session properties:   Implement mapping logic or filter to remove pre-existing data. decide where you want the files stored. Be sure the cache directory has enough disk space to store historical data for the session. When you run multiple sessions with incremental aggregation. You can use a Filter or Stored Procedure transformation in the mapping to remove pre-existing source data during a session. in the Workflow Manager. the Integration Service creates one set of cache files for each partition. you must capture changes in source data. Configure the session for incremental aggregation and verify that the file directory has enough disk space for the aggregate files. Changing the cache directory without moving the files causes the Integration Service to reinitialize the aggregate cache and gather new aggregate data. (v)When you partition a session that uses incremental aggregation. When an Integration Service rebuilds incremental aggregation files. Change the configured path or directory for the aggregate files without moving the files to the new location. You can enter session-specific directories for the index and data files. Configure the session to reinitialize the aggregate cache. Preparing for Incremental Aggregation: When you use incremental aggregation. Decrease the number of partitions. . the data in the previous files is lost. Move the aggregate files without correcting the configured path or directory for the files in the session properties. Integration Services rebuild incremental aggregation files they cannot find. when you perform one of the following tasks:       Save a new version of the mapping. enter the appropriate directory for the process variable. it loses aggregate history. periodically back up the files. instead of using historical data.

the Workflow Manager displays a warning indicating the Integration Service overwrites the existing cache and a reminder to clear this option after running the session. If not. If you choose to reinitialize the cache.   You can configure the session for incremental aggregation in the Performance settings on the Properties tab. If it did. use a Variable in a Decision task to determine whether the previous task ran properly. you can stop the workflow. Use the following types of workflow variables: . For example.(ii) Verify the incremental aggregation settings in the session properties. You can also configure the session to reinitialize the aggregate cache. Workflow Variables You can create and use variables in a workflow to reference values and record information. you can run the next task.

Timer tasks. The Workflow Manager lists built-in variables under the Built-in node in the Expression Editor. The Workflow Manager lists task-specific variables under the task name in the Expression Editor. If the task fails. Use the following types of predefined variables:   Task-specific variables. Links connect each workflow task. Built-in variables. Decision tasks determine how the Integration Service runs a workflow. you can increment a user-defined counter variable by setting the variable to its current value plus 1. Use the following keywords to write expressions for user-defined and predefined workflow variables:        AND OR NOT TRUE FALSE NULL SYSDATE Predefined Workflow Variables: Each workflow contains a set of predefined variables that you use to evaluate workflow and task conditions. Links. The Workflow Manager provides predefined workflow variables for tasks within a workflow. Use workflow variables in links to create branches in the workflow. Use workflow variables when you configure the following types of tasks: Assignment tasks. Timer tasks specify when the Integration Service begins to run the next task in the workflow. and another link to follow when the decision condition evaluates to false.      Predefined workflow variables. use the Status variable to run a second session only if the first session completes successfully. For example. Integration Service Name. after a Decision task. Use an Assignment task to assign a value to a user-defined workflow variable. Use built-in variables in a workflow to return run-time or system information such as folder name. system date. User-defined workflow variables. For Example. Decision tasks. Use task-specific variables in a link condition to control the path the Integration Service takes when running the workflow. For example. you can create one link to follow when the decision condition evaluates to true. Integer . Task-Specific Variables Condition Description Task Data type Types Evaluation result of decision Decision condition expression. the Workflow Manager keeps the condition set to null. The Workflow Manager provides a set of task-specific variables for each task in the workflow. Use a user-defined date/time variable to specify the time the Integration Service starts to run the next task. You create user-defined workflow variables when you create a workflow. or workflow start time.

Integer ErrorMsg Last error message for the All tasks associated task. Precision is to the second. Nstring First Error Code Error code for the first error Session message in the session.EndTime > TO_DATE('11/10/2004 08:13:25') ErrorCode Last error code for the associated All tasks task. the Integration Service sets ErrorCode to 0 when the task completes.ErrorMsg = 'PETL_24013 Session run completed with failure Variables of type Nstring can have a maximum length of 600 characters.If there is no error. Note: You might use this variable when a task consistently fails with this final error message. If there is no error. Note: You might use this variable when a task consistently fails with this final error message.Sample syntax: $Dec_TaskStatus. Sample syntax: Integer . the Integration Service sets ErrorMsg to an empty string when the task completes. Sample syntax: $s_item_summary. If there is no error. the Integration Service sets FirstErrorCode to 0 when the session completes. Sample syntax: $s_item_summary.ErrorCode = 24013. Sample syntax: $s_item_summary.Condition = <TRUE | FALSE | NULL | any integer> End Time Date and time the associated task All tasks Date/Time ended.

$s_item_summary.SUCCEEDED Use these key words when writing expressions to evaluate the status of the previous task. Nstring PrevTaskStatus Status of the previous task in the All workflow that the Integration Tasks Service ran.ABORTED 2.PrevTaskStatus = FAILED Integer SrcFailedRows Total number of rows the Session Integration Service failed to read from the source. Sample syntax: $s_dist_loc. Sample syntax: . Sample syntax: $s_item_summary. the Integration Service sets FirstErrorMsg to an empty string when the task completes.If there is no error.FAILED 3.FirstErrorCode = 7086 FirstErrorMsg First error message in the Session session. Statuses include: 1. Sample syntax: $s_dist_loc.SrcFailedRows = 0 Integer SrcSuccessRows Total number of rows successfully Session read from the sources.FirstErrorMsg = 'TE_7086 Tscrubber: Debug info… Failed to evalWrapUp'Variables of type Nstring can have a maximum length of 600 characters. Precision is to the second. Sample syntax: $Dec_TaskStatus.STOPPED 4.SrcSuccessRows > 2500 Integer StartTime Date and time the associated task All Task Date/Time started.

SUCCEEDED Use these key words when writing expressions to evaluate the status of the current task.STOPPED .ABORTED . you create a workflow to load data to an orders database nightly. When you create a variable in a workflow. You also need to load a subset of this data to headquarters periodically. Statuses include: .DISABLED .TgtFailedRows = 0 Integer TgtSuccessRows Total number of rows successfully Session written to the target. Create separate sessions to update the local database and the one at headquarters. it is valid only in that workflow. Sample syntax: $s_dist_loc.STARTED .FAILED . Sample syntax: $s_dist_loc.$s_item_summary. For example. Sample syntax: $s_dist_loc. Use user-defined variables when you need to make a workflow decision based on criteria you specify. You can edit and delete user-defined workflow variables. Sample syntax: $s_dist_loc.NOTSTARTED .StartTime > TO_DATE('11/10/2004 08:13:25') Status Status of the previous task in the All Task workflow. .TotalTransErrors = 5 Integer User-Defined Workflow Variables: You can create variables within a workflow.TgtSuccessRows > 0 Integer TotalTransErrors Total number of transformation Session errors. Use the variable in tasks within that workflow.Status = SUCCEEDED Integer TgtFailedRows Total number of rows the Session Integration Service failed to write to the target. every tenth time you update the local orders database.

Create a persistent workflow variable. The single dollar sign is reserved for predefined workflow variables Data type Data type of the variable. In the Workflow Designer. You can select from the following data types: . The session that updates the database at headquarters runs every 10th time the workflow runs. Place a Decision task after the session that updates the local orders database. 3. 3. Click Add. 5. 4. to represent the number of times the workflow has run. 2.Date/Time .Use a user-defined variable to determine when to run the session that updates the orders database at headquarters. the session that updates the local database runs every time the workflow runs. $$WorkflowCount. Add a Start task and both sessions to the workflow. To create a workflow variable: 1. Use the modulus (MOD) function to do this. Link it to the Assignment task when the decision condition evaluates to false. 2. To configure user-defined workflow variables. Create an Assignment task to increment the $$WorkflowCount variable by one. complete the following steps: 1. Link the Decision task to the session that updates the database at headquarters when the decision condition evaluates to true. create a new workflow or edit an existing one. When you configure workflow variables using conditions. The correct format is $ $VariableName. Workflow variable names are not case sensitive. 4. Select the Variables tab. Enter the information in the following table and click OK: Field Description Name Variable name. Creating User-Defined Workflow Variables : You can create workflow variables for a workflow in the workflow properties.Set up the decision condition to check to see if the number of workflow runs is evenly divisible by 10. Do not use a single dollar sign ($) for a user-defined workflow variable.

Description Description associated with the .MM/DD/YYYY HH24:MI:SS. The Integration Service ignores extra spaces.MM/DD/YYYY .MM/DD/YYYY HH24:MI:SS.MM/DD/RR HH24:MI:SS. Enable this option if you want the value of the variable retained from one execution of the workflow to the next.US .).. period (. Is Null Whether the default value of the variable is null.Nstring Persistent Whether the variable is persistent. Default Value Default value of the variable. slash (/).MM/DD/YYYY HH24:MI:SS . You cannot use one.Integer . Variables of type Nstring can have a maximum length of 600 characters.Double .MM/DD/RR HH24:MI:SS. The Integration Service uses this value for the variable during sessions if you do not set a value for the variable in the parameter file and there is no value stored in the repository.MM/DD/RR HH24:MI . and space.NS You can use the following separators: dash (-). backslash (\).MM/DD/RR HH24:MI:SS. Variables of type Date/Time can have the following formats: .MM/DD/RR . colon (:).MS .MM/DD/RR HH24:MI:SS . enable this option. If the default value is null.MS .or three-digit values for year or the “HH12” format for hour.NS .US .MM/DD/YYYY HH24:MI:SS.MM/DD/YYYY HH24:MI .

The Integration Service cannot enforce constraint-based loading for these tables. the Integration Service reverts to normal loading for those tables. Define the same target type for all targets in the session properties. 6. Similarly. you must verify the tables are in the same target connection group. Key Relationships: When target tables have no key relationships. and target. when target tables have circular key relationships. It reverts to a normal load. then the foreign key table. The third pipeline contains a source. you have one target containing a primary key and a foreign key related to the primary key in a second target. Target Connection Groups: The Integration Service enforces constraint-based loading for targets in the same target connection group. Related target tables must have the same active source. . When you select this option. the Integration Service reverts to normal loading for both targets. but loads all other targets in the session using constraint-based loading when possible. To validate the default value of the new workflow variable. the Integration Service reverts to a normal load. the Integration Service performs constraint-based loading: loading the primary key table first. a mapping contains three distinct pipelines. Target connection groups. you can specify constraint-based loading for a session. the Integration Service does not perform constraint-based loading. Define the same database connection name for all targets in the session properties. the Integration Service cannot enforce constraint-based loading when you run the workflow. Target tables must have key relationships. click the Validate button. Key relationships. Choose normal mode for the target load type for all targets in the session properties. Use this option when you insert into the target. For example. Click Apply to save the new workflow variable.variable. and two targets. For every row generated by an active source. the Integration Service loads the corresponding transformed row first to the primary key table. To verify that all targets are in the same target connection group. 5. Active Source: When target tables receive rows from different active sources. Normalizer. The second target also contains a foreign key that references the primary key in the first target. If the tables with the primary key-foreign key relationship are in different target connection groups. You cannot use updates with constraint based loading. Treat rows as insert. The first two contain a source. Since these two targets share a single active source (the Normalizer). complete the following tasks:      Verify all targets are in the same target load order group and receive data from the same active source. Constraint-based loading depends on the following requirements:     Active source. then to any foreign key tables. For example. Use the default partition properties and do not add partitions or partition points. the Integration Service orders the target load on a row-by-row basis. Since these two targets receive data from different active sources. If you want to specify constraint-based loading for multiple targets that receive data from the same active source. Constraint-Based Loading: In the Workflow Manager. Targets must be in one target connection group. source qualifier.

Treat Rows as Insert: Use constraint-based loading when the session option Treat Source Rows As is set to insert. transformations. split the mapping using one of the following options:   Load primary key table in one mapping and dependent tables in another mapping. target T_1 has a primary key. . T_2 and T_3 contain foreign keys referencing the T1 primary key. and targets linked together in a mapping. Target load ordering defines the order the Integration Service reads the sources in each target load order group in the mapping. You might get inconsistent data if you select a different Treat Source Rows As option and you configure the session for constraint-based loading. Constraint-based loading does not affect the target load ordering of the mapping. Perform inserts in one mapping and updates in another mapping. Constraint based loading establishes the order in which the Integration Service loads individual targets within a set of targets receiving data from a single source qualifier. When the mapping contains Update Strategy transformations and you need to load data to a primary key table first. Example The following mapping is configured to perform constraint-based loading: In the first pipeline. Use constraint-based loading to load the primary table. A target load order group is a collection of source qualifiers. T_3 has a primary key that T_4 references as a foreign key.

the Integration Service loads rows to the tables in the following order:   T_5 T_6 T_1. The Integration Service loads T_4 last. T_5 and T_6 are in another target connection group together if you use the same database connection for each target and you use the default partition properties. In the Advanced settings. T_1 2. the Integration Service begins reading source B. the Aggregator AGGTRANS. 3. .Since these tables receive records from a single active source. select Constraint Based Load Ordering. the Integration Service orders the target load on a row-by-row basis. T_2 and T_3 (in no particular order) 3. and you use the default partition properties. choose Insert for the Treat Source Rows As property. they are not loaded in any particular order. Enabling Constraint-Based Loading: When you enable constraint-based loading. but since T_2 and T_3 have no dependencies. T_3. the Integration Service loads rows to the target in the following order: 1. The Integration Service then loads T_2 and T_3. To enable constraint-based loading: 1. and T_4 are in one target connection group if you use the same database connection for each target. the Integration Service reverts to a normal load for both targets. If there are no key relationships between T_5 and T_6. because it has a foreign key that references a primary key in T_3.After loading the first set of targets. If T_6 has a foreign key that references a primary key in T_5. Click OK. The Integration Service includes T_5 and T_6 in a different target connection group because they are in a different target load order group from the first four targets. Click the Config Object tab. since T_5 and T_6 receive data from a single active source. T_4 The Integration Service loads T_1 first because it has no foreign key dependencies and contains a primary key referenced by T_2 and T_3. T_2. In the General Options settings of the Properties tab. 2. SQ_A.

Filter rows when the Power Center Server reads source data: If we Include a filter condition. you might use a Custom query to perform aggregate calculations.Different Transformations SOUCE QUALIFIER TRANSFORMATION : : Active and Connected Transformation. Default transformation except in case of XML or COBOL files. the Power Center Server adds an ORDER BY clause to the default SQL query. Create a custom query to issue a special SELECT statement for the Power Center Server to read source data: For example.the Power Center Server adds a SELECT DISTINCT statement to the default SQL query. the Power Center Server replaces the join information Specified by the metadata in the SQL query. SAMPLE MAPPING TO BE MADE: . Select only distinct values from the source: If we choose Select Distinct. the Power Center Server adds a WHERE clause to the Default query. Specify an outer join rather than the default inner join: If we include a User-defined join. The entire above are possible in Properties Tab of Source Qualifier t/f. Tasks performed by Source Qualifier:        Join data originating from the same source database: We can join two or more tables with primary key-foreign key relationships by linking the sources to one Source Qualifier transformation. It is only transformation that is not reusable. Specify sorted ports: If we specify a number for sorted ports.    The Source Qualifier transformation represents the rows that the Power Center Server reads when it runs a session.

Target. 4. Create shortcuts in your folder as needed.   Source will be EMP and DEPT tables. Click Tools -> Mapping Designer. Open folder where we want to create the mapping. DEPT. Click Mapping-> Create-> Give mapping name. Creating Mapping: 1. 2. Ex: m_SQ_example Drag EMP. 3. . Create target table as showed in Picture above.

we override the customized SQL query in the Source Qualifier transformation. 7.5. 2. then the data will be sorted by column that is at the top in SQ. Make sure to give connection information for all tables. move ENAME to top. In the Mapping Designer. SQ PROPERTIES TAB 1) SOURCE FILTER: We can enter a source filter to reduce the number of rows the Power Center Server queries. 4. the Power Center Server adds the ports to the ORDER BY clause in the default query.SAL)2000 Click OK. 5. Click Apply -> OK 11. If we change it to 2. Edit SQ -> Properties Tab -> Open User defined Join -> Give Join condition EMP.   If we want to sort as per ENAME. Select the Properties tab. 10. Enter any number instead of zero for Number of Sorted ports. Click the Open button in the Source Filter field. then data will be sorted by top two columns. 4.DEPTNO. 3. 6. In the Mapping Designer. Validate the mapping. 2. Right Click SQ_DEPT and Select Delete from the mapping. Click OK. 3. 9. Right Click SQ_EMP and Select Delete from the mapping. enter the filter. Steps: 1. In the SQL Editor Dialog box. Mapping -> Validate 12. Select EMP and DEPT both. Click Apply -> Click OK. 2) NUMBER OF SORTED PORTS: When we use sorted ports. Repository -> Save   Create Session and Workflow as described earlier. Select the Properties tab. . open a Source Qualifier transformation. By default it is 0. Now refresh session and save the changes. Example: EMP.DEPTNO=DEPT. Example: DEPTNO in above figure. Link all as shown in above picture. Click Transformation -> Create -> Select Source Qualifier from List -> Give Name -> Click Create 8. Steps: 1. Run the Workflow and see the data in target table. If we change it to 1. Now run the workflow and see output. open a Source Qualifier transformation. Save it. Note: When we enter a source filter in the session properties.

It runs post-session SQL commands against the source database after it writes to the target. we only enter the contents of the WHERE clause. However. 3) SELECT DISTINCT: If we want the Power Center Server to select unique values from a source. Enter the syntax for the join.Validate the mapping. To use full outer join. Validate the mapping.  Steps: 1. The SQL Editor Dialog Box appears. Open the Source Qualifier transformation. Use a semi-colon (. Click OK -> Again Ok. Save it. Now run the workflow and see output. 4. Just check the option in Properties tab to enable it. Now run the workflow and see output. we can use the Select Distinct option. Now refresh session and save the changes. We can specify equi join. Save it. we need to write SQL Query. left outer join and right outer join only.) to separate multiple statements.DEPTNO=EMP.    5) USER DEFINED JOINS Entering a user-defined join is similar to entering a custom SQL query. and click the Properties tab. 2.DEPTNO Left Outer Join {EMP LEFT OUTER JOIN DEPT ON DEPT. Click the Open button in the User Defined Join field. 3. Now refresh session and save the changes.DEPTNO=EMP. We Cannot specify full outer join. not the entire query.  4) PRE and POST SQL Commands The Power Center Server runs pre-session SQL commands against the source database before it reads the source.DEPTNO} Right Outer Join {EMP RIGHT OUTER JOIN DEPT ON . Join Type Syntax Equi Join DEPT.

the Power Center Server reads only the columns that are connected to another Transformation. the Power Center Server generates a query for each Source Qualifier transformation when it runs a session. 3.DEPTNO=EMP. The SQL Editor displays. Click Cancel to exit. In other words. 4. 2. Click Generate SQL.DEPTNO FROM EMP Viewing the Default Query 1. The default query is a SELECT statement for each source column used in the mapping. Open the Source Qualifier transformation. and click the Properties tab. In mapping above. Open SQL Query.DEPT. .SAL. we are passing only SAL and DEPTNO from SQ_EMP to Aggregator transformation.DEPTNO} 6) SQL QUERY For relational sources. 5. Default query generated will be:  SELECT EMP. EMP. The SQL Editor displays the default query the Power Center Server uses to Select source data.

DEPT. Example: As in our case. Before entering the query. EMP.DEPTNO.DEPTNO FROM EMP FULL OUTER JOIN DEPT ON DEPT. then it will not work. DEPT.Note: If we do not cancel the SQL query. We can join as many tables in one query as Required if all are in same database. EMP.SAL. Also always connect to the database and validate the SQL in SQL query editor.DEPTNO. Important Points:  When creating a custom SQL query. DNAME is second in our SQ mapping. SELECT DEPT. EMP.DNAME. then it won’t work in Informatica too. We can enter an SQL statement supported by our source database. Informatica will always use the custom query only. SELECT statement have name DNAME first. Make sure to test the query in database first before using it in SQL Query. It is very handy and used in most of the projects.ENAME.DNAME    Once we have written a custom query like above.EMPNO. EMP. then this query will Always be used to fetch data from database. So when we write SQL Query.DEPTNO=EMP.LOC. the SELECT statement must list the port names in the order in which they appear in the transformation. We can enter more conditions and write More complex SQL. DNAME second and so on.DEPTNO WHERE SAL>2000  We also added WHERE clause. connect all the input and output ports we want to use in the mapping. Example: DEPTNO is top column. We can write any query. we used WHERE SAL>2000. If query is not running in database.JOB. DEPT. we can’t use full outer join in user defined join. Now if we use Source Filter and give condition SAL) 1000 or any other.COMM. the Power Center Server overrides the default query with the custom SQL query. In our example. we can write SQL query for FULL OUTER JOIN: SELECT DEPT. EMP. . EMP.

Creating Mapping: 1. Click Mapping -> Validate 15. Open folder where we want to create the mapping. We can filter rows in a mapping with the Filter transformation. Pass ports from SQ_EMP to Filter Transformation. 2. create the target table Filter_Example. We pass all the rows from a source transformation through the Filter transformation. Repository -> Save . then don’t Import. FILTER TRANSFORMATION Active and connected transformation. Give name and click Create. 12. 9. Ex: m_filter_example 4. 14. Keep all fields as in EMP table. 3. and then click the Open button. Now click done. and then enter a Filter condition for the transformation. 6. Edit Filter Transformation. Drag EMP from source in mapping. In shared folder. 5. 13. Create the necessary shortcuts in the folder. Example: to filter records where SAL>2000    Import the source table EMP in Shared folder. Click Transformation -> Create -> Select Filter from list. Go to Properties Tab 8. Click Mapping -> Create -> Give mapping name. Click Tools -> Mapping Designer. The Expression Editor appears. 11. Enter the filter condition you want to apply. Click OK -> Click Apply -> Click Ok. 7. Click Validate to check the syntax of the conditions you entered. Now connect the ports from Filter to target table. If it is already there. 10. All ports in a Filter transformation are input/output and only rows that meet the condition pass through the Filter Transformation. Click the Value section of the Filter condition.

use the following condition: IIF (ISNULL (FIRST_NAME). 2. For example. How to filter out rows with null values? To filter out rows containing null values or spaces. if we want to filter out rows that Contain NULLs in the FIRST_NAME port. the return value is FALSE and the row should be discarded. use the ISNULL and IS_SPACES Functions to test the value of the port. FALSE. Run the workflow and see the data in target table. Otherwise. Performance tuning: Filter transformation is used to filter off unwanted fields based on conditions we Specify.Create Session and Workflow as described earlier. Use conditional filters and keep the filter condition simple. involving TRUE/FALSE or 1/0 . If elimination of unwanted data can be done by source qualifier instead of filter. TRUE) This condition states that if the FIRST_NAME port is NULL. 1. the row passes through to the next Transformation. 3. Use filter transformation as close to source as possible so that unwanted data gets Eliminated sooner.Then eliminate them at Source Qualifier itself.

then don’t import. we might need to adjust employee salaries. Use the Expression transformation to calculate values in a single row before we write to the target. we need salary and commission. Example: Calculating Total Salary of an Employee    Import the source table EMP in Shared folder. Subtraction. Create the necessary shortcuts in the folder. We can also use the Expression transformation to test conditional statements before we output the results to target tables or other transformations. create the target table Emp_Total_SAL. Uppercase conversion. Add Total_SAL port to store the calculation. Division. Keep all ports as in EMP table except Sal and Comm in target table. Decode There are 3 types of ports in Expression Transformation:    Input Output Variable: Used to store any temporary calculation. Multiplication. In shared folder. Use the Expression transformation to perform any non-aggregate calculations. Then. We can enter multiple expressions in a single Expression transformation. . Calculating Values : To use the Expression transformation to calculate values for a single row. Example: Addition. For example. we must include the following ports:   Input or input/output ports for each value used in the calculation: For example: To calculate Total Salary. concatenate first and last names. The return value for the output port needs to match the return value of the expression. lowercase conversion etc. Example: IF. If it is already there. Concat. We can create any number of output ports in the transformation. Output port for the expression: We enter one expression for each output port. or convert strings to numbers.EXPRESSION TRANSFORMATION  Passive and connected transformation.

Edit Expression Transformation. Open folder where we want to create the mapping. 2. As we do not want Sal and Comm in target. Click Transformation -> Create -> Select Expression from list. 14. Click OK -> Click Apply -> Click Ok. 11. Check the expression syntax by clicking Validate. . Click Mapping -> Validate 15. remove check from output port for both columns. Ex: m_totalsal Drag EMP from source in mapping. Now click done. Run the workflow and see the data in target table.Creating Mapping: 1. Click Tools -> Mapping Designer. Enter expression SAL + COMM. 12. Repository -> Save Create Session and Workflow as described earlier. Give name and click Create. 10. 8. 7. 3. Now create a new port out_Total_SAL. Link ports from SQ_EMP to Expression Transformation. You can select SAL and COMM from Ports tab in expression editor. Click Mapping -> Create -> Give mapping name. 4. 13. Click the small button that appears in the Expression section of the dialog box and enter the expression in the Expression Editor. 9. 5. Now connect the ports from Expression to target table. 6. Make it as output port only.

Select COMM port. . 1. Use operators instead of functions. we can use ABORT function to abort the session if COMM is null. Click OK to refresh the mapping. Now apply changes. Now open your mapping and expression transformation. Syntax: ABORT(‘Any message here’) Make sure to double click the session after doing any changes in mapping. In Default Value give 0. Syntax: ERROR(‘Any message here’) Similarly.As COMM is null. 2. It will prompt that mapping has changed. Run workflow after validating and saving the workflow. Performance tuning : Expression transformation is used to perform simple calculations and also to do Source lookups. Validate Mapping and Save. Run the workflow and see the result again. Total_SAL will be null in most cases. Then we need to use only this variable for all computations. Refresh the session and validate workflow again. then Make that expression as a variable. Minimize the usage of string functions. If we use a complex expression multiple times in the expression transformer. Now use ERROR in Default value of COMM to skip rows where COMM is null. 3.

A Router transformation is similar to a Filter transformation because both transformations allow you to use a condition to test data. input and output ports. Example: If we want to keep employees of France.Reactions:  ROUTER TRANSFORMATION Active and connected transformation. and properties that we configure in the Designer. a Router transformation tests data for one or more conditions And gives you the option to route rows of data that do not meet any of the conditions to a default output group. A Filter transformation tests data for one condition and drops the rows of data that do not meet the Condition. A Router transformation consists of input and output groups. US in 3 different tables. Mapping A uses three Filter transformations while Mapping B produces the same result with one Router transformation. then we can use 3 Filter transformations or 1 Router transformation. group filter conditions. India. However. .

We cannot modify or delete Output ports or their properties.Working with Groups A Router transformation has the following types of groups:   Input: The Group that gets the input ports. Output: User Defined Groups and Default Group. A user-defined group consists of output ports and a group filter Condition. Create one user-defined group for each condition that we want to specify. User-Defined Groups: We create a user-defined group to test a condition based on incoming data. . We can create and edit user-defined groups on the Groups tab with the Designer.

9. Now click done. and then click the Add button to create a user-defined Group. 6. Example: Filtering employees of Department 10 to EMP_10. Go to Groups Tab 8. Drag EMP from source in mapping. Ex: m_router_example 4. 5. Click Mapping-> Create-> Give mapping name. Structure should be same as EMP table. Click Validate to check the syntax of the conditions you entered. Click the Group Filter Condition field to open the Expression Editor.. Edit Router Transformation. Pass ports from SQ_EMP to Router Transformation. 3. Click Transformation -> Create -> Select Router from list. Create 3 target tables EMP_10. Ex: DEPTNO=10 11. Open folder where we want to create the mapping. Department 20 to EMP_20 and rest to EMP_REST    Source is EMP Table. This group does not have a group filter condition associated with it. Creating Mapping: 1. Create the shortcuts in your folder. 7. . 2. Click Tools -> Mapping Designer.The Default Group: The Designer creates the default group after we create one new user-defined group. Enter a group filter condition. EMP_20 and EMP_REST in shared folder. If all of the conditions evaluate to FALSE. The Designer does not allow us to edit or delete the default group. the IS passes the row to the default group. 10. Click the Groups tab. Give name and Click Create. The default group is created automatically.

Make sure to give connection information for all 3 target tables.12. Create another group for EMP_20. Run the Workflow and see the data in target table. 16. Click OK -> Click Apply -> Click Ok. Sample Mapping: . Repository -> Save   Create Session and Workflow as described earlier. The rest of the records not matching the above two conditions will be passed to DEFAULT group. Click Mapping -> Validate 17. See sample mapping 14. Now connect the ports from router to target tables. 15. Condition: DEPTNO=20 13.

Rejected data is in Default Group of router. .Difference between Router and Filter : We cannot pass rejected data forward in filter but we can pass it in router.

All data passing through the Sorter transformation is sorted according to a sort key. Sorter Transformation Properties 1. . Sorter Cache Size: The Power Center Server uses the Sorter Cache Size property to determine the maximum amount of memory it can allocate to perform the sort operation. 4. Sorter will sort data according to all the ports when it is selected. Case Sensitive: The Case Sensitive property determines whether the Power Center Server considers case when sorting data. Distinct: Check this option if we want to remove duplicates. We can also configure the Sorter transformation for case-sensitive sorting. configure Sorter cache size with a value less than or equal to the amount of available physical RAM on the Power Center Server machine. Informatica recommends allocating at least 8 MB of physical memory to sort data using the Sorter transformation.     We can specify any amount between 1 MB and 4 GB for the Sorter cache size. The Power Center Server passes all incoming data into the Sorter transformation Before it performs the sort operation. For best performance. The sort key is one or more ports that we want to use as the sort criteria. We also configure sort criteria the Power Center Server applies to all sort key ports and the system resources it allocates to perform the sort operation. we specify one or more ports as a sort key and configure each sort key port to sort in ascending or descending order. When we enable the Case Sensitive property. 3. and specify whether the output rows should be distinct. the Power Center Server sorts uppercase characters higher than lowercase characters. If it cannot allocate enough memory. The Sorter transformation contains only input/output ports. Work Directory Directory Power Center Server uses to create temporary files while it sorts data. When we create a Sorter transformation in a mapping. We can sort data in ascending or descending order according to a specified sort key.SORTER TRANSFORMATION     Connected and Active Transformation The Sorter transformation allows us to sort data. 2. the Power Center Server fails the Session.

Example: Sorting data of EMP by ENAME


Source is EMP table.
Create a target table EMP_SORTER_EXAMPLE in target designer. Structure same as EMP table.
Create the shortcuts in your folder.

Creating Mapping:
1. Open folder where we want to create the mapping.
2. Click Tools -> Mapping Designer.
3. Click Mapping-> Create-> Give mapping name. Ex: m_sorter_example
4. Drag EMP from source in mapping.
5. Click Transformation -> Create -> Select Sorter from list. Give name and click Create. Now click done.
6. Pass ports from SQ_EMP to Sorter Transformation.
7. Edit Sorter Transformation. Go to Ports Tab
8. Select ENAME as sort key. CHECK mark on KEY in front of ENAME.
9. Click Properties Tab and Select Properties as needed.
10. Click Apply -> Ok.
11. Drag target table now.
12. Connect the output ports from Sorter to target table.
13. Click Mapping -> Validate
14. Repository -> Save

Create Session and Workflow as described earlier. Run the Workflow and see the data in target table.
Make sure to give connection information for all tables.

Sample Sorter Mapping :

Performance Tuning:
Sorter transformation is used to sort the input data.
1. While using the sorter transformation, configure sorter cache size to be larger than the input data size.
2. Configure the sorter cache size setting to be larger than the input data size while Using sorter transformation.
3. At the sorter transformation, use hash auto keys partitioning or hash user keys Partitioning.

RANK TRANSFORMATION

Active and connected transformation

The Rank transformation allows us to select only the top or bottom rank of data. It Allows us to select a group of top or
bottom values, not just one value.
During the session, the Power Center Server caches input data until it can perform The rank calculations.
Rank Transformation Properties :





Cache Directory where cache will be made.
Top/Bottom Rank as per need
Number of Ranks Ex: 1, 2 or any number
Case Sensitive Comparison can be checked if needed
Rank Data Cache Size can be set
Rank Index Cache Size can be set

Ports in a Rank Transformation :
Ports

Number
Required

Description

I

1 Minimum

Port to receive data from
another transformation.

O

1 Minimum

Port we want to pass to
other transformation.

V

not needed

can use to store values or
calculations to use in an
expression.

R

Only 1

Rank port. Rank is
calculated according to
it. The Rank port is an
input/output port. We
must link the Rank port
to another
transformation.
Example: Total Salary

Rank Index
The Designer automatically creates a RANKINDEX port for each Rank transformation. The Power Center Server uses
the Rank Index port to store the ranking position for Each row in a group.

Ex: m_rank_example 4. Go to Ports Tab. Repository -> Save   Create Session and Workflow as described earlier.For example. 3. Check R type in front of TOTAL_SAL. Defining Groups Rank transformation allows us to group information. if we create a Rank transformation that ranks the top five salaried employees. Click Tools -> Mapping Designer. Make sure to give connection information for all tables. Edit Rank Transformation. 8. the rank index numbers the employees from 1 to 5. Click Transformation -> Create -> Select RANK from list. Click Mapping-> Create-> Give mapping name. Structure should be same as EMP table. we can define a group for Department. For example: If we want to select the top 3 salaried employees of each Department. Open folder where we want to create the mapping. Pass ports from Expression to Rank Transformation. Create an EXPRESSION transformation to calculate TOTAL_SAL. . Create a target table EMP_RANK_EXAMPLE in target designer. Click Mapping -> Validate 16. We can pass the rank index to another transformation in the mapping or directly to a target. 1) Example: Finding Top 5 Salaried Employees    EMP will be source table. We cannot Group By on port which is also Rank Port. 2) Example: Finding Top 2 Salaried Employees for every DEPARTMENT   Open the mapping made above. 13. We cannot delete or edit it. Click Apply -> Ok. 15. Select Group By for DEPTNO. 11. 5.    By defining groups. 12. Select TOTAL_SAL as rank port. Drag target table now. Run the Workflow and see the data in target table. Go to Ports Tab 9. Drag EMP from source in mapping. Connect the output ports from Rank to target table. Click Properties Tab and Select Properties as needed. 6.    The RANKINDEX is an output port only. Give name and click Create. Just add one more port Rank_Index to store RANK INDEX. Click the Group By for needed port. 7. Top in Top/Bottom and Number of Ranks as 5. We define a group in Ports tab. Create the shortcuts in your folder. Creating Mapping: 1. Edit Rank Transformation. 2. 14. we create one set of ranked rows for each group. 10. Now click done.

Click Apply -> Ok.   All Variable ports if there. Save the changed and run workflow to see the new result. When 6th row is read. If the input row out-ranks a Stored row. Example: Power Center caches the first 5 rows if we are finding top 5 salaried Employees. A transaction is the set of rows bound by commit or roll back rows.  All Group By Columns are in RANK INDEX CACHE. RANK CACHE Sample Rank Mapping When the Power Center Server runs a session with a Rank transformation. 1) RANK INDEX CACHE: The index cache holds group information from the group by ports. 30 etc. Set Number of Ranks as 2. it compares it with 5 rows in cache and places it in Cache is needed. Mapping -> Validate and Repository Save. 20. then this cache stores values 10. Refresh the session by double clicking. To reduce the data cache size. the Power Center Server replaces the stored row with the input row. You can define a . Example: All ports except DEPTNO In our mapping example Transaction Control transformation Power Center lets you control commit and roll back transactions based on a set of rows that pass through a Transaction Control transformation. DEPTNO 2) RANK DATA CACHE: It holds row data until the Power Center Server completes the ranking and is Generally larger than the index cache. All ports going out from RANK Transformations are stored in RANK DATA CACHE.   Go to Properties tab. If we are Using Group By on DEPTNO. connect Only the necessary input/output ports to subsequent transformations. Ex. it compares an input row with rows in the data cache. Rank Port.

TC_ROLLBACK_BEFORE. you define transaction control at the following levels:   Within a mapping. begins a new transaction. you can choose to commit. or continue without any transaction changes. roll back. You can dynamically name each target flat file. When the Integration Service issues a commit or roll back based on the return value of the expression. such as employee ID or order entry date. The current row is in the committed transaction. and writes the current row to the target. you can configure the following properties:   Transaction control expression Tracing level Enter the transaction control expression in the Transaction Control Condition field. The Integration Service does not perform any transaction change for this row. and begins a new transaction. or makes no transaction changes to the row. rolls back. The return value determines whether the Integration Service commits. If the transaction control expression evaluates to a value other than commit. The Integration Service evaluates the condition on a row-by-row basis. value1. The Integration Service commits the transaction. Based on the return value of the expression. You might want to define transactions based on a group of rows ordered on a common key. commits the transaction. The Integration Service writes the current row to the target. You define transactions using an expression in a Transaction Control transformation. Within a mapping. Within a session. the Integration Service fails the session. it commits all rows in the transaction to the target or targets. roll back. Use the following syntax for the expression: IIF (condition. it rolls back all rows in the transaction from the target or targets. If the mapping has a flat file target you can generate an output file each time the Integration Service starts a new transaction. Use the following built-in variables in the Expression Editor when you create a transaction control expression:      TC_CONTINUE_TRANSACTION. TC_COMMIT_BEFORE. the Integration Service evaluates the expression for each row that enters the transformation. When the Integration Service evaluates a roll back row. The Integration Service rolls back the current transaction. You can choose to commit or roll back a transaction if the Integration Service fails to transform or write any row to the target. you configure it for user-defined commit.transaction based on a varying number of input rows. When it evaluates a commit row. value2) The expression contains values that represent actions the Integration Service performs based on the return value of the condition. you use the Transaction Control transformation to define a transaction. The Integration Service writes the current row to the target. or continue. TC_COMMIT_AFTER. In Power Center. When you run the session. rolls back the transaction. When you configure a session. TC_ROLLBACK_AFTER. The current row is in the new transaction. The current row is in the rolled back transaction. The transaction control expression uses the IIF function to test each row against the condition. and begins a new transaction. Properties Tab On the Properties tab. begins a new transaction. . The current row is in the new transaction. it begins a new transaction. and writes the current row to the target. This is the default value of the expression.

A Transaction Control transformation may be effective for one target and ineffective for another target. Example to Transaction Control: Step 1: Design the mapping. and you choose to append or create a new document on commit. If each target is connected to an effective Transaction Control transformation.Mapping Guidelines and Validation Use the following rules and guidelines when you create a mapping with a Transaction Control transformation:          If the mapping includes an XML target. XML. Step 2: Creating a Transaction Control Transformation. Either all targets or none of the targets in the mapping should be connected to an effective Transaction Control transformation. You can connect multiple targets to a single Transaction Control transformation. or dynamic MQSeries targets are ineffective for those targets. If you use a dynamic Lookup transformation and a Transaction Control transformation in the same mapping. the mapping is valid. You can connect only one effective Transaction Control transformation to a target. You must connect each target instance to a Transaction Control transformation. the input groups must receive data from the same transaction control point. Transaction Control transformations connected to any target other than relational. a rolled-back transaction might result in unsynchronized target data. . You cannot place a Transaction Control transformation in a pipeline branch that starts with a Sequence Generator transformation.

Go to the Properties tab and click on the down arrow to get in to the expression editor window. Click Done.       In the Mapping Designer. Open the Edit Transformations dialog box. Step 3: Create the task and the work flow.TC_COMMIT_BEFORE. Select the Properties tab. Select the Transaction Control transformation. Enter a description for the transformation. Enter the transaction control expression that defines the commit and roll back behavior.) select the below things from the built in functions. Enter a name for the transformation. . IIF (EMPNO=7654. Click OK. and select the Ports tab.TC_CONTINUE_TRANSACTION)    Connect all the columns from the transformation to the target table and save the mapping. Select the Metadata Extensions tab. Drag the ports into the transformation. Later go to the Variables tab and Type IIF(EMpno=7654. click Transformation > Create.[ The naming convention for Transaction Control transformations is TC_TransformationName]. Create or edit metadata extensions for the Transaction Control transformation. Step 4: Preview the output in the target table. Click Create.

Status codes: Status codes provide error handling for the IS during a workflow. Pre-load of the Target: Before the session sends data to the target. Post-load of the Target: After the session sends data to the target. Database administrators create stored procedures to Automate tasks that are too complicated for standard SQL statements. Post-load of the Source: After the session retrieves data from the source. Data Passes Between IS and Stored Procedure One of the most useful features of stored procedures is the ability to send data to the stored procedure. . the stored procedure runs. Use of Stored Procedure in mapping:     Check the status of a target database before loading data into it. A Stored Procedure transformation is an important tool for populating and Maintaining databases. Drop and recreate indexes. Connected stored procedures run only in normal mode. The IS uses it to determine whether to continue running the session or stop. Perform a specialized calculation. This is useful for verifying the existence of tables or performing joins of data in a temporary table. Specifying when the Stored Procedure Runs Normal: The stored procedure runs where the transformation exists in the mapping on a row-by-row basis. the stored procedure runs. The stored procedure issues a status code that notifies whether or not the stored procedure completed successfully. We pass some input to procedure and it returns some calculated values. Mostly used for this in projects. This is useful for removing temporary tables. There are three types of data that pass between the Integration Service and the stored procedure: Input/output parameters: Parameters we give as input and the parameters returned from Stored Procedure. Return values: Value returned by Stored Procedure if any.   STORED PROCEDURE T/F Passive Transformation Connected and Unconnected Transformation Stored procedures are stored and run within the database. We cannot see this value. Determine if enough space exists in a database. Pre-load of the Source: Before the session retrieves data from the source. the stored procedure runs. the stored procedure runs. This is useful for dropping indexes or disabling constraints. and receive data from the stored procedure. This is useful for recreating indexes on the database.

max_sal out number) As Begin Select max(Sal) into max_sal from EMP where deptno=in_deptno. End. If connected. MIN_SAL. Create a target table SP_CONN_EXAMPLE with fields DEPTNO.avg_sal. Determine whether to use the transformation as connected or unconnected. 5. max_sal out number.   DEPT will be source table. 2. MIN. Connected Stored Procedure T/F Example: To give input as DEPTNO from DEPT table and find the MAX.min(sal). AVG and SUM of SAL from EMP table. 4.avg(sal). .Using a Stored Procedure in a Mapping : 1. or configure it to run from an expression in another transformation. If unconnected. / 1. AVG_SAL & SUM_SAL. min_sal out number. Create the stored procedure in the database. avg_sal out number. map the appropriate input and output ports. / CREATE OR REPLACE procedure sp_unconn_1_value(in_deptno in number. 2. sum_sal out number) As Begin select max(Sal). Stored Procedures: Connect to Source database and create the stored procedures given below: CREATE OR REPLACE procedure sp_agg (in_deptno in number. Creating Mapping: 1. End. 6. Write Stored Procedure in Database first and Create shortcuts as needed. Configure the session.or post-session.sum(sal) into max_sal. Import or create the Stored Procedure transformation.sum_sal from emp where deptno=in_deptno group by deptno. 3. Click Tools -> Mapping Designer. MAX_SAL. either configure the stored procedure to run pre. Open folder where we want to create the mapping.min_sal.

Drag DEPTNO from SQ_DEPT to the stored procedure input port and also to DEPTNO port of target. Repository -> Save . Mapping -> Validate 9. Connect the ports from procedure to target as shown below: 8. 6. Ex: m_SP_CONN_EXAMPLE 4. Drag DEPT and Target table. 5. Transformation -> Import Stored Procedure -> Give Database Connection -> Connect -> Select the procedure sp_agg from the list. 7. Click Mapping-> Create-> Give name.3.

. Creating Mapping: 1. Give connection information for Stored Procedure also.or post-session: Runs before or after a session. Write Stored Procedure in Database first and Create shortcuts as needed. Assign the output value to the system variable PROC_RESULT. Give connection information for all tables. Unconnected Stored Procedure T/F : An unconnected Stored Procedure transformation is not directly connected to the flow of data through the mapping.    DEPT will be source table. Run workflow and see the result in table. 2.    Create Session and then workflow. Instead. (See Later) Example 1: DEPTNO as input and get MAX of Sal as output. Create a target table with fields DEPTNO and MAX_SAL of decimal data type. Click Tools -> Mapping Designer. the stored procedure runs either:   From an expression: Called from an expression transformation. Method of returning the value of output parameters to a port:   Assign the output value to a local variable. Open folder where we want to create the mapping. 2. Pre.

you must create variables for each output parameter. 6. Example: DEPTNO as Input and MAX_SAL. Repository Save. . MIN_SAL. Create four variable ports in expression VAR_MAX_SAL. VAR_AVG_SAL and iVAR_SUM_SAL. Stored Procedure has been imported.     Create Session and then workflow. Open Expression editor and call the stored procedure as below:Click OK and connect the port from expression to target as in mapping below: 10. 9. Example: DEPTNO as Input and MAX Sal as output : :SP. 5. Give connection information for all tables. Edit expression and create an output port OUT_MAX_SAL of decimal data type. Click Mapping-> Create-> Give name. Ex: m_sp_unconn_1_value 4. 8.3. T/F -> Create Expression T/F. Run workflow and see the result in table.PROC_RESULT)  If the stored procedure returns multiple output parameters. PROC_RESULT use:  If the stored procedure returns a single output parameter or a return value. Pass DEPTNO from SQ_DEPT to Expression T/F. Drag DEPT and Target table. we the reserved variable PROC_RESULT as the output variable. 7.VAR_MIN_SAL. Give connection information for Stored Procedure also. Click OK.SP_UNCONN_1_VALUE(DEPTNO. AVG_SAL and SUM_SAL as output then: 1. Mapping -> Validate 11. Transformation -> Import Stored Procedure -> Give Database Connection -> Connect -> Select the procedure sp_unconn_1_value from the list.

:SP.VAR_MIN_SAL.2. MIN_SAL. OUT_MIN_SAL. PROC_RESULT) Example 2: DEPTNO as Input and MAX_SAL. AVG_SAL and SUM_SAL as O/P Stored Procedure to drop index in Pre Load of Target Stored Procedure to create index in Post Load of Target   DEPT will be source table. Stored procedures are given below to drop and create index on target. AVG_SAL & SUM_SAL. MIN_SAL. VAR_MAX_SAL. Create four output ports in expression OUT_MAX_SAL. MAX_SAL. Create a target table SP_UNCONN_EXAMPLE with fields DEPTNO.Make sure to create target table first. Call the procedure in last variable port says VAR_SUM_SAL. 3. OUT_AVG_SAL and OUT_SUM_SAL. VAR_AVG_SAL. Write Stored Procedure in Database first and Create shortcuts as needed. Stored Procedures to be created in next example in Target Database: Create or replace procedure CREATE_INDEX As Begin .SP_AGG (DEPTNO.

Drag DEPT and Target table. Click Mapping-> Create-> Give name. 7. Edit Expression and create 4 variable ports and 4 output ports as shown below: . 6. / Create or replace procedure DROP_INDEX As Begin Execute immediate 'drop index unconn_dept'. 8. Click Tools -> Mapping Designer. 5. End. 3. Click OK.Execute immediate 'create index unconn_dept on SP_UNCONN_EXAMPLE(DEPTNO)'. T/F -> Create Expression T/F. Transformation -> Import Stored Procedure -> Give Database Connection -> Connect -> Select the procedure sp_agg from the list. Stored Procedure has been imported. Pass DEPTNO from SQ_DEPT to Expression T/F. / Creating Mapping: 1. 2. Ex: m_sp_unconn_1_value 4. Open folder where we want to create the mapping. End.

Repository -> Save  Create Session and then workflow. 12. . Edit DROP_INDEX -> Properties Tab -> Select Target Pre Load as Stored Procedure Type and in call text write drop_index. Call the procedure in last variable port VAR_SUM_SAL. Connect to target table as needed. 13. Click Apply -> Ok. Click OK. 10. PROC_RESULT) 11. VAR_MAX_SAL. Mapping -> Validate 17.9. Click Apply -> Ok. 16. :SP. VAR_MIN_SAL. Transformation -> Import Stored Procedure -> Give Database Connection for target -> Connect -> Select the procedure CREATE_INDEX and DROP_INDEX from the list. 15. 14. VAR_AVG_SAL. Edit CREATE_INDEX -> Properties Tab -> Select Target Post Load as Stored Procedure Type and in call text write create_index. Click Apply and Ok.SP_AGG (DEPTNO.

The transformation returns one row for each input row. You pass a script name to the transformation with each input row. if there is no INDEX on target table. The type of database the SQL transformation connects to. Connection type. The SQL transformation executes a query that you define in a query editor. DROP_INDEX will fail and Session will also fail. you configure the following options: Mode:-The SQL transformation runs in one of the following modes:      Script mode. Passive or active transformation. The SQL transformation processes the query and returns rows and database errors. To include multiple query statements in a script. Database type. You pass each script file name from the source to the SQL transformation Script Name port. The SQL transformation runs ANSI SQL scripts that are externally located. You can output multiple rows when the query has a SELECT statement. you can separate them with a semicolon. Script Mode An SQL transformation running in script mode runs SQL scripts from text files. You can change the locale of the script. . The SQL transformation outputs one row for each input row. You can configure it as a passive transformation when you create the transformation. Give connection information for Stored Procedures also. The script code page defaults to the locale of the operating system. When you create an SQL transformation. Run workflow and see the result in table. This is because. you create a passive transformation. The Integration Service must have read permissions on the directory that contains the script. You can pass strings or parameters to the query to define dynamic queries or change the selection parameters. Also make sure that you execute the procedure CREATE_INDEX on database before using them in mapping.    Give connection information for all tables. When you configure the transformation to run in script mode. The transformation processes external SQL scripts or SQL queries that you create in an SQL editor. The SQL transformation is an active transformation by default. You can use mapping variables or parameters in the script file name. Query mode. The script file must be accessible by the Integration Service. SQL Transformation: You can pass the database connection information to the SQL transformation as input data at run time. Rules and Guidelines for Script Mode Use the following rules and guidelines for an SQL transformation that runs in script mode:      You can use a static or dynamic database connection with script mode. Pass database connection information to the SQL transformation or use a connection object. The script file name contains the complete path to the script file. The output row contains results of the query and any database error.

the transformation returns data to the SQL Error port. but you can use query parameters to change the data. A script cannot accept run-time arguments. SQL Transformation Properties After you create the SQL transformation. the input ports must be string data types. You can use string substitution instead of parameter binding in a query. When the number of output ports is less than the number of columns in the SELECT clause. The query statement does not change. Rules and Guidelines for Query Mode Use the following rules and guidelines when you configure the SQL transformation to run in query mode:          The number and the order of the output ports must match the number and order of the fields in the query SELECT clause. You can create the following types of SQL queries in the SQL transformation:   Static SQL query. The Integration Service generates a row error when the data types do not match. The native data type of an output port in the transformation must match the data type of the corresponding column in the database. you can define ports and set attributes in the following transformation tabs:  Ports. or DELETE clause. the extra ports receive a NULL value. the Integration Service generates a row error. However. You can change the query statements and the data. The SQL transformation in script mode does not output more than one row of data for each input row. The Integration Service prepares a query for each input row. You cannot use nested scripts where the SQL script calls another SQL script. When the number of output ports is more than the number of columns in the SELECT clause. Query Mode    When you configure the SQL transformation to run in query mode. When the SQL query contains an INSERT. When an SQL transformation runs in query mode. You pass strings or parameters to the query from the transformation input ports to change the query statement or the query data. When the SQL query contains a SELECT statement and the transformation has a pass-through port. the transformation returns data to the pass-through port whether or not the query returns database data. You cannot use scripting languages such as Oracle PL/SQL or Microsoft/Sybase T-SQL in the script. and the Num Rows Affected port when it is enabled. Dynamic SQL query. it executes an SQL query that you define in the transformation. You cannot use the pass-through port to return data from a SELECT query. you create an active transformation. the pass-through ports. . The Integration Service prepares the query once and runs the query for all input rows. Displays the transformation ports and attributes that you create on the SQL Ports tab. UPDATE. If you add output ports the ports receive NULL data values. The SQL transformation returns a row with NULL data in the output ports.    The Integration Service ignores the output of any SELECT statement you include in the SQL script. You cannot add the "_output" suffix to output port names that you create.

SQL Ports. SQL transformation ports and attributes. If this property is blank. Properties Tab Configure the SQL transformation general properties on the Properties tab. You must copy all DLLs or shared libraries to the runtime location or to the environment variable defined on the Integration Service node. SQL Settings. Attributes unique to the SQL transformation. they display on the Ports tab. When you define ports on the SQL Ports tab.   Properties. the . Note: You cannot update the columns on the Ports tab. the Integration Service uses the environment variable defined on the Integration Service node to locate the DLL or shared library. SQL transformation general properties. Tracing Level Sets the amount of detail included in the session log when you run a session containing this transformation. The following table describes the SQL transformation properties: Property Description Run Time Location Enter a path relative to the Integration Service node that runs the SQL transformation session. shared library. Some transformation properties do not apply to the SQL transformation or are not configurable. The Integration Service fails to load the procedure when it cannot locate the DLL. or a referenced file. When you configure the SQL transformation tracing level to Verbose Data.

but the Integration Service must run all partitions in the pipeline on the same node. The transformation and other transformations in the same pipeline are limited to one partition. . Use the following options: . Default is No. such as data cleansing. . Choose Locally when different partitions of the transformation must share objects in memory. You might choose No if the transformation processes all the input data together. The transformation can be partitioned.Integration Service writes each SQL query it prepares to the session log. You can enable this property for query mode SQL transformations. The transformation cannot be partitioned. Transformation Scope The method in which the . Default is disabled.No. and the Integration Service can distribute each partition to different nodes. Update Strategy The transformation defines the update strategy for output rows.Locally.Across Grid. The transformation can be partitioned. Is Partition able Multiple partitions in a pipeline can use this transformation.

All Input Set transaction scope to transaction when you use transaction control in static query mode.Always.Integration Service applies the transformation logic to incoming data.Never. Use the following options: .Based On Input Order. . Default is Never.Transaction .Row . The order of the output data is consistent between session runs even if the order of the input data is inconsistent between session runs. Enable this property for query mode . Default is Row for script mode transformations. . .Default is All Input for query mode transformations. The order of the output data is inconsistent between session runs. Generate Transaction The transformation generates transaction rows. Output is Repeatable Indicates if the order of the output data is consistent between session runs. The output order is consistent between session runs when the input data order is consistent between session runs.

After which we are goanna get a wizard with three subsequent windows and follow the on screen instructions to complete the process of importing the source. Import the source (second notepad) using the source->import from the file. Requires Single Thread Per Partition Indicates if the Integration Service processes each partition of a procedure with one thread.    Create a Notepad and in it create a table by name bikes with three columns and three records in it. Inside the Notepad just type in (C:\bikes. Output is Deterministic The transformation generate consistent output data between session runs.txt) and save it. Default is enabled. Create Mapping : Step 1: Creating a flat file and importing the source from the flat file.SQL transformations that commit data in an SQL query. Enable this property to perform recovery on sessions that use this transformation. . Default is disabled. Create one more notepad and name it as path for the bikes.

Now apply the SQL transformation. Snapshot for the above discussed things is given below. .Step 2: Importing the target and applying the transformation. In the first window when you apply the SQL transformation we should select the script mode. In the same way as specified above go to the targets->import from file and select an empty notepad under the name targetforbikes (this is one more blank notepad which we should create and save under the above specified name in the C :\). Connect the SQ to the ScriptName under inputs and connect the other two fields to the output correspondingly. We are all set here.     Create two columns in the target table under the name report and error.

Example 1: To create 4 records of every employee in EMP table.    EMP will be source table. The Normalizer transformation normalizes records from COBOL and relational sources. allowing us to organize the data. Structure same as EMP and datatype of HIREDATE as VARCHAR2. Create shortcuts as necessary. Step 4: Preview the output data on the target table.Step 3: Design the work flow and run it. Use a Normalizer transformation instead of the Source Qualifier transformation when we normalize a COBOL source. . We can also use the Normalizer transformation with relational sources to create multiple rows from a single row of data. Create target table Normalizer_Multiple_Records.     NORMALIZER TRANSFORMATION Active and Connected Transformation. Go to the mappings tab and click on the Source on the left hand pane to specify the path for the output file.   Create the task and the work flow using the naming conventions.

7. 5. Edit Normalizer and Normalizer Tab. 4. 14. done. 9. 11. Transformation-> Create-> Select Normalizer-> Give name. 15. 12.Creating Mapping : 1. Click create. Example 2: To break rows into columns Source: Roll_Number Name ENG HINDI MATHS 100 Amit 78 76 90 101 Rahul 76 78 87 102 Jessie 65 98 79 Target : Roll_Number Name Marks . 10. 6. Give connection information for source and target table. Run workflow and see result. Normalizer doesn’t have DATETIME datatype. As EMPNO identifies source records and we want 4 records of every employee. Columns equal to columns in EMP table and datatype also same. Ex: m_ Normalizer_Multiple_Records Drag EMP and Target table. Repository -> Save    Make session and workflow. 8. Transformation->Create->Select Expression-> Give name. Pass all ports from SQ_EMP to Expression transformation. Click Mapping-> Create-> Give name. give OCCUR for EMPNO as 4. Edit Normalizer and Normalizer Tab. Not Possible. 13. Click Apply and then OK. Connect ports from Expression to Normalizer. Add columns. 2. Mapping -> Validate 17. Create output port out_hdate and do the conversion. 3. Try dragging ports from Expression to Normalizer. Click Tools -> Mapping Designer. Add link as shown in mapping below: 16. create & done. So convert HIREDATE to char in expression t/f. Open folder where we want to create the mapping.

Connect accordingly and connect to target. Create Mapping as before. give Occurs as 3 for Marks in Normalizer tab. Rank Transformation Properties :       Cache Directory where cache will be made. During the session. not just one value. the Power Center Server caches input data until it can perform The rank calculations. create only 3 ports Roll_Number. Validate and Save Make Session and workflow and Run it. Name and Marks as there are 3 columns in target table. Also as we have 3 marks in source. Import it and create target table. Give Source File Directory and Source File name for source flat file in source properties in mapping tab of session. 2 or any number Case Sensitive Comparison can be checked if needed Rank Data Cache Size can be set Rank Index Cache Size can be set . It Allows us to select a group of top or bottom values.  Active and connected transformation       RANK TRANSFORMATION The Rank transformation allows us to select only the top or bottom rank of data. Top/Bottom Rank as per need Number of Ranks Ex: 1.100 Amit 78 100 Amit 76 100 Amit 90 101 Rahul 76 101 Rahul 78 101 Rahul 87 102 Jessie 65 102 Jessie 98 102 Jessie 79  Make source as a flat file. In Normalizer tab. See the result.

We define a group in Ports tab. Click the Group By for needed port. R Only 1 Rank port. 1) Example: Finding Top 5 Salaried Employees . For example: If we want to select the top 3 salaried employees of each Department.Ports in a Rank Transformation : Ports Number Required Description I 1 Minimum Port to receive data from another transformation. Defining Groups Rank transformation allows us to group information. The Power Center Server uses the Rank Index port to store the ranking position for Each row in a group. the rank index numbers the employees from 1 to 5. We cannot Group By on port which is also Rank Port. Example: Total Salary Rank Index The Designer automatically creates a RANKINDEX port for each Rank transformation. O 1 Minimum Port we want to pass to other transformation. V not needed can use to store values or calculations to use in an expression. We cannot delete or edit it. The Rank port is an input/output port.    The RANKINDEX is an output port only. We can pass the rank index to another transformation in the mapping or directly to a target. For example. we can define a group for Department. We must link the Rank port to another transformation.    By defining groups. if we create a Rank transformation that ranks the top five salaried employees. we create one set of ranked rows for each group. Rank is calculated according to it.

Drag EMP from source in mapping. Go to Ports Tab. Now click done. 14. Mapping -> Validate and Repository Save. the Power Center Server replaces the stored row with the input row. Save the changed and run workflow to see the new result. Make sure to give connection information for all tables. Creating Mapping: 1. Click Transformation -> Create -> Select RANK from list. Structure should be same as EMP table. Ex: m_rank_example 4. Click Apply -> Ok. Pass ports from Expression to Rank Transformation. Check R type in front of TOTAL_SAL. Drag target table now. Connect the output ports from Rank to target table. Set Number of Ranks as 2. Create a target table EMP_RANK_EXAMPLE in target designer. Refresh the session by double clicking. Click Mapping-> Create-> Give mapping name. Open folder where we want to create the mapping. Run the Workflow and see the data in target table. Go to Properties tab. Create an EXPRESSION transformation to calculate TOTAL_SAL. Create the shortcuts in your folder. 6. Edit Rank Transformation. Edit Rank Transformation. If the input row out-ranks a Stored row. Click Apply -> Ok. 12. 13. Repository -> Save   Create Session and Workflow as described earlier. it compares an input row with rows in the data cache. 2. Click Mapping -> Validate 16.   EMP will be source table. 15. Just add one more port Rank_Index to store RANK INDEX. RANK CACHE Sample Rank Mapping When the Power Center Server runs a session with a Rank transformation. . 2) Example: Finding Top 2 Salaried Employees for every DEPARTMENT      Open the mapping made above. 5. 8. 11. Go to Ports Tab 9. 10. Select Group By for DEPTNO. Click Tools -> Mapping Designer. Top in Top/Bottom and Number of Ranks as 5. Give name and click Create. 7. Select TOTAL_SAL as rank port. 3. Click Properties Tab and Select Properties as needed.

then we can use 3 Filter transformations or 1 Router transformation. US in 3 different tables. DEPTNO 2) RANK DATA CACHE: It holds row data until the Power Center Server completes the ranking and is Generally larger than the index cache. India. Example: All ports except DEPTNO In our mapping example. 20.  All Group By Columns are in RANK INDEX CACHE.  ROUTER TRANSFORMATION Active and connected transformation. . If we are Using Group By on DEPTNO. a Router transformation tests data for one or more conditions And gives you the option to route rows of data that do not meet any of the conditions to a default output group. Rank Port.Example: Power Center caches the first 5 rows if we are finding top 5 salaried Employees. 1) RANK INDEX CACHE: The index cache holds group information from the group by ports. Mapping A uses three Filter transformations while Mapping B produces the same result with one Router transformation. 30 etc. To reduce the data cache size. All ports going out from RANK Transformations are stored in RANK DATA CACHE. it compares it with 5 rows in cache and places it in Cache is needed. then this cache stores values 10. connect Only the necessary input/output ports to subsequent transformations. Example: If we want to keep employees of France.  A Router transformation is similar to a Filter transformation because both transformations allow you to use a condition to test data. However.  All Variable ports if there. Ex. A Filter transformation tests data for one condition and drops the rows of data that do not meet the Condition. When 6th row is read.

A Router transformation consists of input and output groups. group filter conditions. Working with Groups A Router transformation has the following types of groups:   Input: The Group that gets the input ports. . and properties that we configure in the Designer. We cannot modify or delete Output ports or their properties. input and output ports. Output: User Defined Groups and Default Group.

Open folder where we want to create the mapping. We can create and edit user-defined groups on the Groups tab with the Designer. If all of the conditions evaluate to FALSE. Click the Groups tab. The Designer does not allow us to edit or delete the default group. 3.. Create the shortcuts in your folder. 10. Click Transformation -> Create -> Select Router from list. 7. Now click done. Enter a group filter condition. Create 3 target tables EMP_10.User-Defined Groups: We create a user-defined group to test a condition based on incoming data. Ex: m_router_example 4. Click Mapping-> Create-> Give mapping name. Click Tools -> Mapping Designer. 2. A user-defined group consists of output ports and a group filter Condition. Give name and Click Create. Click the Group Filter Condition field to open the Expression Editor. Department 20 to EMP_20 and rest to EMP_REST    Source is EMP Table. Click Validate to check the syntax of the conditions you entered. 9. and then click the Add button to create a user-defined Group. The Default Group: The Designer creates the default group after we create one new user-defined group. Create one user-defined group for each condition that we want to specify. the IS passes the row to the default group. . Edit Router Transformation. 6. This group does not have a group filter condition associated with it. Creating Mapping: 1. The default group is created automatically. Structure should be same as EMP table. EMP_20 and EMP_REST in shared folder. Example: Filtering employees of Department 10 to EMP_10. Pass ports from SQ_EMP to Router Transformation. Go to Groups Tab 8. Ex: DEPTNO=10 11. 5. Drag EMP from source in mapping.

Click OK -> Click Apply -> Click Ok. Condition: DEPTNO=20 13. Repository -> Save   Create Session and Workflow as described earlier. Click Mapping -> Validate 17. 15. Run the Workflow and see the data in target table. 16. The rest of the records not matching the above two conditions will be passed to DEFAULT group. See sample mapping 14. Create another group for EMP_20. Sample Mapping: . Make sure to give connection information for all 3 target tables.12. Now connect the ports from router to target tables.

Rejected data is in Default Group of router.Difference between Router and Filter : We cannot pass rejected data forward in filter but we can pass it in router. .

Connected and Unconnected Lookup Connected Lookup Unconnected Lookup . Update slowly changing dimension tables: Most important use. Cached or Un cached Lookup: We can check the option in Properties Tab to Cache to lookup or not. The Power Center Server queries the lookup source based on the lookup ports in the transformation. Pass the result of the lookup to other transformations and a target. LOOKUP TYPES We can configure the Lookup transformation to perform the following types of lookups:    Connected or Unconnected Relational or Flat File Cached or Un cached Relational Lookup: When we create a Lookup transformation using a relational table as a lookup source. We can use multiple Lookup transformations in a mapping. We can use the Lookup transformation to perform following:    Get a related value: EMP has DEPTNO but DNAME is not there. the Designer invokes the Flat File Wizard. Use a Lookup transformation in a mapping to look up data in a flat file or a relational table. Dynamic lookup is connected. or we can import it. When we import a flat file lookup source. By default. We will write Lookup Override query.     LOOKUP TRANSFORMATION Passive Transformation Can be Connected or Unconnected. It compares Lookup transformation port values to lookup source column values based on the lookup condition.   We can override the default SQL statement if we want to add a WHERE clause or query multiple tables. we can use any flat file definition in the repository. Perform a calculation: We want only those Employees who’s SAL > Average (SAL). view. We can import a lookup definition from any flat file or relational database to which both the PowerCenter Client and Server can connect. lookup is cached. Flat File Lookup: When we use a flat file for a lookup source. 1. or synonym. we can connect to the lookup source using ODBC and import the table definition as the structure for the Lookup transformation. We can use a Lookup transformation to determine whether rows already exist in the target. We use Lookup to get DNAME from DEPT table based on Lookup Condition. We can use a dynamic lookup cache with relational lookups.

Receives input values directly from the pipeline. Cache includes all lookup columns used in the mapping. If there is a match for If there is a match for the the lookup condition. lookup condition. Receives input values from the result of a :LKP expression in another transformation. the Power the Power Center Server Center Server returns NULL. returns the default value for all output ports. We can use a dynamic or static cache. Pass one output value to another transformation. lookup/output ports. Cache includes all lookup/output ports in the lookup condition and the lookup/return port. .the Power the Power Center Server Center Server returns the result returns the result of the of the lookup condition into the lookup condition for all return port. Supports user-defined default values Does not support user-defined default values. lookup condition. We can use a static cache. If there is no match for If there is no match for the the lookup condition. Pass multiple output values to another transformation.

LOOKUP T/F COMPONENTS Define the following components when we configure a Lookup transformation in a mapping: Lookup source Ports Properties Condition     1. or we can join multiple tables in the same database using a lookup SQL override in Properties Tab. Usually ports used for Join condition are Input .    2. When we create a Lookup t/f. Lookup Source: We can use a flat file or a relational table for a lookup source.2 . Ports: Ports I Lookup Number Type Needed Connected Minimum 1 Unconnected Description Input port to Lookup. we can import the lookup source from the following locations: Any relational source or target definition in the repository Any flat file source or target definition in the repository Any table or file that both the Power Center Server and Client machine can connect to The lookup table can be a single table.

O Connected Unconnected Minimum 1 Ports going to another transformation from Lookup. The 1 Designer Unconnected automatically Designates each column in the lookup source as a lookup (L) and output port (O). 3.ports. Relational Determines what happens when the Lookup transformation finds multiple rows . Lookup Caching Enabled Flat File. Lookup Policy on Multiple Match Flat File. R Unconnected 1 Only Return port. Properties Tab Options Lookup Type Description Lookup SQL Override Relational Overrides the default SQL statement to query the lookup table. Lookup Table Name Relational Specifies the name of the table from which the transformation looks up and caches values. L Connected Minimum Lookup port. Use only in unconnected Lookup t/f only. Relational Indicates whether the Power Center Server caches lookup values during the session.

Relational Whether to use Dynamic Cache or not. Insert Else Update Relational Use only with dynamic caching enabled. Source Type Flat File. Lookup Cache Directory Name Flat File. Relational Displays the lookup condition you set in the Condition tab. Relational Use only with persistent lookup . Options: Use First Value or Use Last Value or Use Any Value or Report Error Lookup Condition Flat File. Lookup Data Cache Size Flat File. Recache From Lookup Source Flat File. Connection Information Relational Specifies the database containing the lookup table. Lookup Cache Persistent Flat File. Relational Data Cache Size Lookup Index Cache Size Flat File. Relational Location where cache is build. Dynamic Lookup Cache Flat File.that match the lookup condition. Relational To rebuild cache if cache source changes and we are using Persistent Cache. Relational Lookup is from a database or flat file. Applies to rows entering the Lookup transformation with the row type of insert. Relational Index Cache Size Cache File Name Prefix Flat File. Relational Whether to use Persistent Cache or not.

cache. represented by lookup ports. >. We can use = operator in case of Dynamic Cache. >=. place the conditions with an equal sign first to optimize lookup performance. <. The Power Center Server matches null values. not an OR. The Power Center Server uses the lookup condition to test incoming values. <=. 2. We compare transformation input values with values in the lookup source or cache. =. . Specifies the file name prefix to use with persistent lookup cache files. Example: IN_DEPTNO = DEPTNO In_DNAME = 'DELHI' Tip: If we include more than one lookup condition. The Power Center Server fails the session when it encounters multiple keys for a Lookup transformation configured to use a dynamic cache. The input value must meet all conditions for the lookup to return a value. != Operators can be used. When we enter multiple conditions. Note: 1. Some other properties for Flat Files are:       Date time Format Thousand Separator Decimal Separator Case-Sensitive String Comparison Null Ordering Sorted Input 4: Condition Tab We enter the Lookup Condition. the Power Center Server evaluates each condition as an AND.       The data types in a condition must match.

Edit Lookup and rename it to IN_DEPTNO in ports tab. The Following screen is displayed. Click Ok. Now Pass DEPTNO from SQ_EMP to this Lookup. DEPTNO from SQ_EMP will be named as DEPTNO1. 6. Create a target table CONN_Lookup_EXAMPLE in target designer. As DEPT is the Source definition. 7.3. . Click Tools -> Mapping Designer. 9. DEPT will be LOOKUP table. 10. Connect all fields from SQ_EMP to target except DNAME and LOC. Ex: m_CONN_LOOKUP_EXAMPLE 4. Creating Mapping: 1. Table should contain all ports of EMP table plus DNAME and LOC as shown below. Transformation-> Create -> Select LOOKUP from list. 5. Create the shortcuts in your folder. 3. Now go to CONDITION tab and add CONDITION. Give name and click Create. Click Mapping-> Create-> Give name. 2. Drag EMP and Target table. 8. click Source and then Select DEPT. Open folder where we want to create the mapping. 11. Connected Lookup Transformation Example: To create a connected Lookup Transformation    EMP will be source table.

13. Mapping -> Validate 14. We are not passing IN_DEPTNO and DEPTNO to any other transformation from LOOKUP.DEPTNO = IN_DEPTNO and Click Apply and then OK. Repository -> Save    Create Session and Workflow as described earlier.There is no use of Return Port in Connected Lookup. Add input ports. We write an expression using the :LKP reference qualifier to call the lookup within another transformation. 3. Run the workflow and see the data in target table. Call the lookup from another transformation. We use Connected Lookup when we need to return more than one column from Lookup table. . 2. Make sure to give connection information for all tables. Add the lookup condition. Example: To create a unconnected Lookup Transformation  EMP will be source table. Designate a return value. SEE PROPERTY TAB FOR ADVANCED SETTINGS 4. we can edit the lookup transformation and remove the OUTPUT check from them. Unconnected Lookup Transformation An unconnected Lookup transformation is separate from the pipeline in the mapping. Make sure to give connection for LOOKUP Table also. 4. Steps to configure Unconnected Lookup: 1. DEPT will be LOOKUP table. Link the mapping as shown below: 12.

Table should contain all ports of EMP table plus DNAME as shown below. Now Transformation-> Create -> Select EXPRESSION from list. 5. Then Click Done. Create the shortcuts in your folder. Now Edit the Lookup Transformation. Open folder where we want to create the mapping. 9. create a port IN_DEPTNO ports tab. 11. Pass all ports from SQ_EMP to EXPRESSION transformation. Follow the steps as in Connected above to create Lookup on DEPT table. Creating Mapping: 1. Transformation-> Create -> Select LOOKUP from list. 6. . 7. Go to Ports tab. Ex: m_UNCONN_LOOKUP_EXAMPLE 4. 8. Give name and click Create. Click Mapping-> Create-> Give name. Click Tools -> Mapping Designer. 12. 3. Connect all fields from EXPRESSION to target except DNAME. 13. Drag EMP and Target table. 10. Check on R to make it. 2.  Create a target table UNCONN_Lookup_EXAMPLE in target designer. Click Ok. Designate DNAME as Return Port. As DEPTNO is common in source and Lookup. Give name and click Create. Make it Input port only and Give Datatype same as DEPTNO.

16. Open the Expression editor and call Lookup as given below: We double click Unconn in bottom of Functions tab and as we need only DEPTNO. 17. we pass only DEPTNO as input. Validate the call in Expression editor and Click OK. . 15. 18. Run the workflow and see the data in target table.  Create Session and Workflow as described earlier.14. Repository Save. Now we need to call this Lookup from Expression Transformation. DEPTNO = IN_DEPTNO and Click Apply and then OK. Edit Expression t/f and create a new output port out_DNAME of data type as DNAME. Now add a condition in Condition Tab. Mapping -> Validate 19.

For an unconnected Lookup transformation. use a Lookup transformation with a dynamic cache. It caches the lookup file or table and Looks up values in the cache for each row that comes into the transformation. 5. The IS dynamically inserts or updates data in the lookup cache and passes data to the target. Static Cache By default. eliminating the time Required to read the lookup table. We set the Cache type in Lookup Properties. The IS saves and reuses cache files from session to session. 4. the IS always caches the lookup source. Lookup Cache Files 1. Persistent Cache If the lookup table does not change between sessions. The Integration Service also creates cache files by default in the $PMCacheDir. The Integration Service (IS) builds a cache in memory when it processes the first row of data in a cached Lookup transformation. IS releases cache memory and deletes the cache files. Recache from Source . Lookup Index Cache:  Stores data for the columns used in the lookup condition. When session completes. not including ports used in the lookup condition.   If we use a flat file lookup.The IS does not update the cache while it processes the Lookup transformation. Lookup Data Cache:   For a connected Lookup transformation. Types of Lookup Caches: 1. Dynamic Cache To cache a target table or flat file source and insert new rows or update existing rows in the cache. If the data does not fit in the memory cache. 2. stores data from the return port. No good for performance if table is huge. Target table is also our lookup table. 2.  Make sure to give connection information for all tables. Lookup Caches We can configure a Lookup transformation to cache the lookup table. Make sure to give connection for LOOKUP Table also. 3. we can configure the Lookup transformation to use a persistent lookup cache. stores data for the connected output ports. the IS stores the overflow values in the cache files. the IS creates a static cache.

You can share static and dynamic named caches. 1.If Lookup table has changed. When we configure the session to build concurrent caches. the IS does not wait for the first row to enter the Lookup transformation before it creates caches. Building Lookup Caches Sequentially: 2. it builds multiple caches concurrently. the IS creates caches as the source rows enter the Lookup transformation.If the persistent cache is not synchronized with the lookup table. we can Configure the Lookup transformation to rebuild the lookup cache. Instead. Building Lookup Caches Concurrently: . Named cache: Use a persistent named cache when we want to share a cache file across mappings or share a dynamic and a static cache. we can use this to rebuild the lookup cache. You can only share static unnamed caches. Building Connected Lookup Caches We can configure the session to build caches sequentially or concurrently. 5.   When we build sequential caches. the IS shares the cache by default. Shared Cache   Unnamed cache: When Lookup transformations in a mapping have compatible caching structures. The caching structures must match or be compatible with a named cache.

So we create a Sequence 1. When we want to Maintain history. . To configure the session to create concurrent caches Edit Session -> In Config Object Tab-> Additional Concurrent Pipelines for Lookup Cache Creation -> Give a value here (Auto By Default) Note: The IS builds caches for unconnected Lookups sequentially only    SEQUENCE GENERATOR T/F Passive and Connected Transformation. We use this sequence as the key.2.4 and so on. or cycle through a sequential range of numbers. The Sequence Generator transformation generates numeric values. Use the Sequence Generator to create unique primary key values. Sequence Generator Ports : The Sequence Generator transformation provides two output ports: NEXTVAL and CURRVAL. then we need a key other than Primary Key to uniquely identify the record. So we use Surrogate key as Primary key and not EMPNO.3. replace missing primary keys. We use it to generate Surrogate Key in DWH environment mostly. Example: If EMPNO is the key. we can keep only one record in target and can’t maintain history.

only then Sequence for table 2 will be generated. Likewise. If we connect the CURRVAL port without connecting the NEXTVAL port.     We typically only connect the CURRVAL port when the NEXTVAL port is Already connected to a downstream transformation. We can optimize performance by connecting only the NEXTVAL port in a Mapping.  We cannot edit or delete these ports. . the Integration Service processes one row in each block. we cannot add ports to the transformation. When table 1 has been loaded. Sequence in Table 1 will be generated first. the Integration Service passes a constant value for each row. we might connect NEXTVAL to two target tables in a mapping to generate unique primary key values. NEXTVAL: Use the NEXTVAL port to generate sequence numbers by connecting it to a Transformation or target. CURRVAL: CURRVAL is NEXTVAL plus the Increment By value. For example. when we connect the CURRVAL port in a Sequence Generator Transformation.

Structure same as EMP. Create shortcuts as needed. Validate Mapping 9. Add two more ports NEXT_VALUE and CURR_VALUE to the target table. Create a target EMP_SEQ_GEN_EXAMPLE in shared folder. Click Mapping-> Create-> Give name. Click Tools -> Mapping Designer. 6. 5. Run workflow and see the result in table. Creating Mapping: 1. Transformation -> Create -> Select Sequence Generator for list -> Create -> Done 7. Give connection information for all tables. 3. Ex: m_seq_gen_example 4.Example: To use Sequence Generator transformation    EMP will be source. Connect all ports from SQ_EMP to target table. 8. Repository -> Save    Create Session and then workflow. Connect NEXT_VAL and CURR_VAL from Sequence Generator to target. Drag EMP and Target table. Open folder where we want to create the mapping. 2. Sequence Generator Properties: Setting Required/Optional Description Start Value Required Start value of the .

Increment By Required Difference between two consecutive values from the NEXTVAL port.generated sequence that we want IS to use if we use Cycle option. last value of sequence during session is saved to repository. Next time the sequence is started from the valued saved. the value must be greater than or equal to the start value and less the end value. If selected. the Integration Service .If cycle option used. Reset Optional By default. Default is 0. the Integration Service cycles through the sequence range. Cycle Optional If selected. Current Value Optional First value in the sequence. Ex: Start Value:1 End Value 10 Sequence will be from 1-10 and again start from 1. End Value Optional Maximum value the Integration Service generates.

1-10 1-7. give Start Value as 1. If we connect just CURR_VAL only. There are 17 records in source. cycle option there. 0 1 2 3 – To make above sequence as 1-10 1-20. In this case session will fail.    SORTER TRANSFORMATION Connected and Active Transformation The Sorter transformation allows us to sort data. Session runs. cycle option there. There are 17 records in source. Start value is 1. sequence will start from 8. no cycle option. 7 will be saved in repository.generates values based on the original current value for each session. If Current value is 1 and end value 10. Sequence: 1 2 – 10. Start value is used along with Cycle option only. If we run session again. We can sort data in ascending or descending order according to a specified sort key. If Current value is 1 and end value 10. Start value is 0. Use reset option if you want to start sequence from CURR_VAL every time. the value will be same for all records.        . Points to Ponder:  If Current value is 1 and end value 10. There are 17 records in source.

Sorter Transformation Properties 1. 2. When we create a Sorter transformation in a mapping. We can also configure the Sorter transformation for case-sensitive sorting. For best performance. the Power Center Server sorts uppercase characters higher than lowercase characters. and specify whether the output rows should be distinct. 3. . Informatica recommends allocating at least 8 MB of physical memory to sort data using the Sorter transformation. configure Sorter cache size with a value less than or equal to the amount of available physical RAM on the Power Center Server machine. Distinct: Check this option if we want to remove duplicates. The Power Center Server passes all incoming data into the Sorter transformation Before it performs the sort operation. Work Directory Directory Power Center Server uses to create temporary files while it sorts data. 4. Case Sensitive: The Case Sensitive property determines whether the Power Center Server considers case when sorting data.     We can specify any amount between 1 MB and 4 GB for the Sorter cache size. All data passing through the Sorter transformation is sorted according to a sort key. When we enable the Case Sensitive property. The Sorter transformation contains only input/output ports. The sort key is one or more ports that we want to use as the sort criteria. Sorter will sort data according to all the ports when it is selected. the Power Center Server fails the Session. We also configure sort criteria the Power Center Server applies to all sort key ports and the system resources it allocates to perform the sort operation. Example: Sorting data of EMP by ENAME  Source is EMP table. we specify one or more ports as a sort key and configure each sort key port to sort in ascending or descending order. Sorter Cache Size: The Power Center Server uses the Sorter Cache Size property to determine the maximum amount of memory it can allocate to perform the sort operation. If it cannot allocate enough memory.

Sample Sorter Mapping : Performance Tuning: Sorter transformation is used to sort the input data. 13. Structure same as EMP table. 1. Configure the sorter cache size setting to be larger than the input data size while Using sorter transformation. Creating Mapping: 1. Repository -> Save   Create Session and Workflow as described earlier.  Create a target table EMP_SORTER_EXAMPLE in target designer. 3. 12. Click Mapping -> Validate 14. 9. Click Apply -> Ok. Open folder where we want to create the mapping. Drag EMP from source in mapping. Click Transformation -> Create -> Select Sorter from list. Edit Sorter Transformation. 5. At the sorter transformation. Create the shortcuts in your folder. 10. Pass ports from SQ_EMP to Sorter Transformation. 2. CHECK mark on KEY in front of ENAME. Ex: m_sorter_example 4. use hash auto keys partitioning or hash user keys Partitioning. Click Properties Tab and Select Properties as needed. Connect the output ports from Sorter to target table. Go to Ports Tab 8. Click Tools -> Mapping Designer. 11. . configure sorter cache size to be larger than the input data size. Drag target table now. Give name and click Create. 6. 2. Now click done. 7. While using the sorter transformation. Make sure to give connection information for all tables. 3. Click Mapping-> Create-> Give mapping name. Select ENAME as sort key. Run the Workflow and see the data in target table.

The SQL transformation outputs one row for each input row. The transformation processes external SQL scripts or SQL queries that you create in an SQL editor. The type of database the SQL transformation connects to. Database type.SQL Transformation: You can pass the database connection information to the SQL transformation as input data at run time. The SQL transformation is an active transformation by default. Passive or active transformation. The SQL transformation executes a query that you define in a query editor. you configure the following options: Mode:-The SQL transformation runs in one of the following modes:     Script mode. You can pass strings or parameters to the query to define dynamic queries or change the selection parameters. You pass a script name to the transformation with each input row. Query mode. You can output multiple rows when the query has a SELECT statement. . You can configure it as a passive transformation when you create the transformation. When you create an SQL transformation. The SQL transformation runs ANSI SQL scripts that are externally located. The SQL transformation processes the query and returns rows and database errors.

Query Mode    When you configure the SQL transformation to run in query mode. The Integration Service prepares a query for each input row. You can change the query statements and the data. You pass each script file name from the source to the SQL transformation Script Name port. The script file must be accessible by the Integration Service. The transformation returns one row for each input row. Dynamic SQL query. Pass database connection information to the SQL transformation or use a connection object. The Integration Service prepares the query once and runs the query for all input rows. You cannot use nested scripts where the SQL script calls another SQL script. The Integration Service generates a row error when the data types do not match. Rules and Guidelines for Query Mode Use the following rules and guidelines when you configure the SQL transformation to run in query mode:   The number and the order of the output ports must match the number and order of the fields in the query SELECT clause. The SQL transformation in script mode does not output more than one row of data for each input row. you create an active transformation. The native data type of an output port in the transformation must match the data type of the corresponding column in the database. The output row contains results of the query and any database error. it executes an SQL query that you define in the transformation. When you configure the transformation to run in script mode. You can change the locale of the script. To include multiple query statements in a script. You can create the following types of SQL queries in the SQL transformation:   Static SQL query. you create a passive transformation. The query statement does not change. You cannot use scripting languages such as Oracle PL/SQL or Microsoft/Sybase T-SQL in the script. The script file name contains the complete path to the script file. Rules and Guidelines for Script Mode Use the following rules and guidelines for an SQL transformation that runs in script mode:          You can use a static or dynamic database connection with script mode. Connection type. A script cannot accept run-time arguments. but you can use query parameters to change the data. The script code page defaults to the locale of the operating system. The Integration Service ignores the output of any SELECT statement you include in the SQL script. you can separate them with a semicolon. Script Mode An SQL transformation running in script mode runs SQL scripts from text files. The Integration Service must have read permissions on the directory that contains the script. When an SQL transformation runs in query mode. You pass strings or parameters to the query from the transformation input ports to change the query statement or the query data. . You can use mapping variables or parameters in the script file name.

When the SQL query contains a SELECT statement and the transformation has a pass-through port. Properties Tab Configure the SQL transformation general properties on the Properties tab. Some transformation properties do not apply to the SQL transformation or are not configurable. the transformation returns data to the SQL Error port. and the Num Rows Affected port when it is enabled. The following table describes the SQL transformation properties: Property Description Run Time Location Enter a path relative to the Integration Service node that runs the SQL transformation session. SQL transformation ports and attributes. Note: You cannot update the columns on the Ports tab. The SQL transformation returns a row with NULL data in the output ports. the Integration Service generates a row error.       When the SQL query contains an INSERT. the extra ports receive a NULL value. If you add output ports the ports receive NULL data values. SQL Ports. You cannot use the pass-through port to return data from a SELECT query. When the number of output ports is more than the number of columns in the SELECT clause. If this property is blank. SQL transformation general properties. SQL Settings. When the number of output ports is less than the number of columns in the SELECT clause. or DELETE clause. When you define ports on the SQL Ports tab. Attributes unique to the SQL transformation. . You cannot add the "_output" suffix to output port names that you create. SQL Transformation Properties After you create the SQL transformation. they display on the Ports tab. UPDATE. However. the input ports must be string data types. you can define ports and set attributes in the following transformation tabs:     Ports. the transformation returns data to the pass-through port whether or not the query returns database data. the Integration Service uses the environment variable defined on the Integration Service node to locate the DLL or shared library. the pass-through ports. Displays the transformation ports and attributes that you create on the SQL Ports tab. You can use string substitution instead of parameter binding in a query. Properties.

such as data cleansing.You must copy all DLLs or shared libraries to the runtime location or to the environment variable defined on the Integration Service node. or a referenced file.Locally. shared library. You might choose No if the transformation processes all the input data together. When you configure the SQL transformation tracing level to Verbose Data. Tracing Level Sets the amount of detail included in the session log when you run a session containing this transformation. Is Partition able Multiple partitions in a pipeline can use this transformation. The transformation can be partitioned. Use the following options: . . but the Integration Service must run all partitions in the pipeline on the same node. The transformation and other transformations in the same pipeline are limited to one partition. The Integration Service fails to load the procedure when it cannot locate the DLL. the Integration Service writes each SQL query it prepares to the session log. Choose Locally when different partitions of the .No. The transformation cannot be partitioned.

Transaction . Default is Row for script mode transformations. The order of the output data is inconsistent .All Input Set transaction scope to transaction when you use transaction control in static query mode. Default is No.Never.Default is All Input for query mode transformations. The transformation can be partitioned. Default is disabled. Output is Repeatable Indicates if the order of the output data is consistent between session runs. and the Integration Service can distribute each partition to different nodes. Transformation Scope The method in which the Integration Service applies the transformation logic to incoming data. . Use the following options: . Update Strategy The transformation defines the update strategy for output rows.Row . .Across Grid. You can enable this property for query mode SQL transformations.transformation must share objects in memory.

Enable this property to perform recovery on sessions that use this transformation. Create Mapping : Step 1: Creating a flat file and importing the source from the flat file. The order of the output data is consistent between session runs even if the order of the input data is inconsistent between session runs. The output order is consistent between session runs when the input data order is consistent between session runs. Generate Transaction The transformation generates transaction rows. Default is Never. Enable this property for query mode SQL transformations that commit data in an SQL query. Inside the Notepad just type in (C:\bikes. Default is disabled.Always.txt) and save it.   Create a Notepad and in it create a table by name bikes with three columns and three records in it.between session runs. . Output is Deterministic The transformation generate consistent output data between session runs. Default is enabled. Requires Single Thread Per Partition Indicates if the Integration Service processes each partition of a procedure with one thread.Based On Input Order. . . Create one more notepad and name it as path for the bikes.

 Import the source (second notepad) using the source->import from the file. We are all set here. Snapshot for the above discussed things is given below. In the same way as specified above go to the targets->import from file and select an empty notepad under the name targetforbikes (this is one more blank notepad which we should create and save under the above specified name in the C :\). In the first window when you apply the SQL transformation we should select the script mode. Connect the SQ to the ScriptName under inputs and connect the other two fields to the output correspondingly.     Create two columns in the target table under the name report and error. Step 2: Importing the target and applying the transformation. Now apply the SQL transformation. . After which we are goanna get a wizard with three subsequent windows and follow the on screen instructions to complete the process of importing the source.

One row is for old and one for new. What if we want to update. In DWH.  Update Strategy Transformation Active and Connected Transformation Till now. This way we maintain the historical data. delete or reject the source record as per business need. Go to the mappings tab and click on the Source on the left hand pane to specify the path for the output file. delete or reject rows coming from source based on some condition? Example: If Address of a CUSTOMER changes. we have only inserted rows in our target tables.Step 3: Design the work flow and run it. we create a Lookup on target table to determine whether a row already exists or not. Step 4: Preview the output data on the target table. we can update the old address or keep both old and new address.   Create the task and the work flow using the naming conventions. Then we insert. update. Update Strategy is used with Lookup Transformation. .

Update Strategy within a session: When we configure a session.In Power Center. we can instruct the IS to either treat all rows in the same way or use instructions coded into the session mapping to flag rows for different database operations. Flagging Rows within a Mapping . insert it. Update as Insert. Delete: Select this option to delete a row from a table. Session Configuration: Edit Session -> Properties -> Treat Source Rows as: (Insert. Inset each row flagged for update. Delete. Update. Within a Mapping 1. and Data Driven). we set the update strategy at two different levels: 1. Update each row flagged for update if it exists in the target table. Update: We have the following options in this situation:    Update as Update. Within a session 2. Otherwise. Specifying Operations for Individual Target Tables: You can set the following update strategy options: Insert: Select this option to insert a row into a target table. Update else Insert. Update the row if it exists. 2. Truncate table: Select this option to truncate the target table before loading data. Insert is default.

DD_UPDATE ) Or IIF( ( ENTRY_DATE > APPLY_DATE). DD means DATA DRIVEN Forwarding Rejected Rows: We can configure the Update Strategy transformation to either pass rejected rows to the next transformation or drop them. Use Update Strategy transformation as less as possible in the mapping. DD_REJECT. direct filtering etc. Performance tuning: 1. Create Update Strategy Transformation Pass all ports needed to it. 3. instead use direct mapping. we use the Update Strategy transformation to flag rows for insert. the update strategy expression uses the IIF or DECODE function from the transformation language to test each row to see if it meets a particular condition. 3. update. 2. 2. or reject. Steps: 1. IIF( ( ENTRY_DATE > APPLY_DATE).Within a mapping. Set the Expression in Properties Tab. . 2 )   The above expression is written in Properties Tab of Update Strategy T/f. Do not use update strategy transformation if we just want to insert into target table. delete. 4. Connect to other transformations or target. Operation Constant Numeric Value INSERT DD_INSERT 0 UPDATE DD_UPDATE 1 DELETE DD_DELETE 2 REJECT DD_REJECT 3 Update Strategy Expressions: Frequently. For updating or deleting rows from the target table we can use Update Strategy transformation itself.