Informatica Pushdown Tips

Applies to:
Informatica with Pushdown feature (version 8x)

Summary
Pushdown Optimization is for better performance. Along with enabling this Pushdown property, certain guidelines should be followed while developing the mapping. This document discusses about the guidelines, tuning tips and practices to be followed to achieve the Pushdown for simple/complex mappings.

Author Bio
Author(s): Manimala Rajangam Company: Merill Lynch Created on: April 27, 2010 Manimala Rajangam, Merill Lynch(acquired by Bank Of America), Analyst. Playing the role of lead developer in Informatica.

Informatica Technology Network 2010 Informatica Corporation. All Rights Reserved.

http://technet.informatica.com 1

Informatica Pushdown Tips

Table of Contents
Purpose of the document ................................................................................................................................ 3 Document Usage .............................................................................................................................................. 3 Details of Document ......................................................................................................................................... 3 Steps to be followed ......................................................................................................................................... 3 Mapping level Steps: ................................................................................................................................... 3 Session level Steps: ................................................................................................................................... 4 Workflow level Steps:................................................................................................................................. 6 Session Log information .................................................................................................................................. 6 Benefits:............................................................................................................................................................. 6 Limitations:........................................................................................................................................................ 6

Informatica Technology Network 2010 Informatica Corporation. All Rights Reserved.

http://technet.informatica.com 2

Informatica Pushdown Tips

Purpose of the document This document provides a brief overview of Push down optimization in Informatica. It describes the operational features and capabilities of this optimization technique. Also, the ways to tune the mapping for better performance is discussed.

Document Usage The target audience of this document is: Informatica Architects / Informatica Developer.

Details of Document

Preface
Pushdown is a specific feature which is available in Informatica to optimize the load. If the source, target and other tables (which are used as lookup) are in the same database then pushdown is the better option to load your target in spite of normal Relational reader/writer which takes more time to read/write records. There are few steps to be followed for better performance which is given in detail below: Steps to be followed Mapping level Steps: Create a mapping with a compatible source, lookup and target connections. Push down can be of source-side, target-side and Full. Better performance is achieved while using Full pushdown.

Reason: Full pushdown is possible only when the connections used for source, lookup and target are same (in turn which means the tables should be in the same database). Else we have to opt for either To Source or To Target. Optimizer queries will not be the same for To Source, To Target and Full pushdown. Incase if the lookup table or source table should be restricted with filter condition, best way is to use SQL Override query or Lookup override query with that filter condition which will give better performance in spite of using the source filter or lookup filter.

Reason: Unconnected lookups will always be outer joined with source in the push down query. So it always creates a view for that unconnected lookup. But for source filter, a separate view will not be created if the filter condition is given in the source filter. For better performance, the filter condition should be given along with the SQL override query. View will be created based on the query in SQL override. If the same filter is given in source filter or in any filter transformation in the mapping, the filter will be appended to the pushdown query. But if they were given in the SQL

Informatica Technology Network 2010 Informatica Corporation. All Rights Reserved.

http://technet.informatica.com 3

Informatica Pushdown Tips

Override a separate view will be created after filtering those records. So that the number of records read will reduced. Grant select access to all the lookup tables if they are in different schema in the same database. Else Informatica will fail to create a view when it tries to create a view on a table in other schema. Expression functions in the variable ports cannot be pushed, so always the expression functions or the lookup call can be given in the output ports. For the nested variables one or more expression transformations should be used. For example: Let us say emp_name is available in the source for which we have to lookup Employee table to get the emp_id and then pass this emp_id to get dept_id from department table. This can be achieved in a single expression transformation if we do not use Pushdown. Here the emp_id value should be assigned to a variable port and then do a lookup to department table to retrieve dept_id but the variable port cannot be pushed to a query while using push down optimization. Two expression transformations should be used in this case. Where first expression transformation should have a lookup on employee table and the value returned should be assigned to an output port out_emp_id. And in the second expression transformation, this out_emp_id can be passed to the lookup on department table where the returned value can be assigned to another output port in second expression transformation. Not all expression functions can be pushed by the pushdown optimization. Please refer the Informatica help PDFs to know the functions which are supported for each database. Avoid two lookup conditions in the single output port

For example:
out_emp_id=iif ( isnull( :LKP.LKP_EMPLOYEE(emp_name)),DEFAULT, :LKP.LKP_EMPLOYEE(emp_name))

This could error out pushdown while configuring the pushdown for FULL.We cannot view the reason also. So whenever the pushdown preview fails it shows the query conversion is blocked in the mapping.

Informatica Technology Network 2010 Informatica Corporation. All Rights Reserved.

http://technet.informatica.com 4

Informatica Pushdown Tips

Session level Steps: Create a session, select the pushdown type.

Uncheck the Pre 85 timestamp compatibility session property, if its checked while creating a session. If the source have SQL override or the mapping uses lookup, enable the option for the view creation by pushdown (shown above). If sequence is used, enable the option for the sequence creation (shown above). The views or the sequence will be created in the same schema, which is configured for source and target.

After the run the views will be dropped. Preview the pushdown optimization after configuring the session for push down

Informatica Technology Network 2010 Informatica Corporation. All Rights Reserved.

http://technet.informatica.com 5

Informatica Pushdown Tips

Workflow level Steps: Push down can be configured for both ASCII and UNICODE Integration services. Session Log information The session log will not have the transformations details for the processed records. The session will fail even if there is a single failure while loading the target table, even if the stop on error session property is set as 0. Debugger will not work when a session is configured for push down. Only the total number of records that were read and processed will be available. The busy percentages for the transformations will not be available. Benefits

Better performance gain. Storage space can be reduced. As all temporary views will be created and dropped after the run.

Limitations Not all transformations can be pushed Not all expression functions can be pushed. Pushdown will work only if all the lookup, source and target connections are in the same database. If the target is partitioned along with push down, the committed partitions cannot be rolled back incase of errors. Bad records cannot be identified while using push down unlike the normal run.(like the truncate or overflow error) CPU skew might go up. So its always better to check the cost of the query created after previewing the pushdown query.

Informatica Technology Network 2010 Informatica Corporation. All Rights Reserved.

http://technet.informatica.com 6

Informatica Pushdown Tips

Conclusion
Performance gain can be highly achieved by Pushdown Optimization. Compare the mapping with and without pushdown to measure the gain.

Informatica Technology Network 2010 Informatica Corporation. All Rights Reserved.

http://technet.informatica.com 7

Informatica Pushdown Tips

Disclaimer and Liability notice

Informatica offers no guarantees and assumes no responsibility or liability of any type with respect to the content of this software asset, including any liability resulting from incompatibility between the content within this asset and the materials and services offered by Informatica. You agree that you will not hold, or seek to hold, Informatica responsible or liable with respect to the content of this software asset.

Informatica Technology Network 2010 Informatica Corporation. All Rights Reserved.

http://technet.informatica.com 8