Testing-in-Informatica | Data Warehouse | Databases

Testing Approaches using Informatica PowerCenter

Applies to:
Informatica PowerCenter

Summary
This article briefs about different kinds of testing approaches using Informatica PowerCenter.

Author Bio
Author(s): Sukumar Balasubramanian Company: CIBC Created on: March 23, 2010 Sukumar Balasubramanian is an experienced Informatica ETL Consultant working with CIBC, Canada. He has good exposure to Data Integration/Data Warehousing Projects. He is also a key contributor in Informatica-l group of ittoolbox.

Informatica Technology Network © 2009 Informatica Corporation. All Rights Reserved.

http://technet.informatica.com 1

...................................................................... http://technet................................................................... 5 Granularity .................................................................................................informatica................................................................................................................................................................................................................................................................................................................................................................................................ 8 Disclaimer and Liability notice ............ 3 Unit Testing .................Testing Approaches using Informatica PowerCenter Table of Contents Introduction ............................. 3 Quantitative Testing .......... 4 Dimensional Analysis ............................................................................................................. 6 Testing Processes ... 6 Unsecured .......................................................................................................... 8 Informatica PowerCenter Testing ............................................................................................................................................................................................................... 4 Data Quality Validation...................................................................................................................................................................................................................................................................................................................................................................................................................................................................... 10 Informatica Technology Network © 2009 Informatica Corporation................................. 5 Other validations ......................................................................................................................................................................................................................................................................... 4 Integration Testing ...................................................................................................................................................................................... 6 Secured ............................................................................................................ 3 Qualitative Testing ........... 6 Testing Architecture ........................................................................................................com 2 ...................................................................................................................................... 4 Statistical Analysis .......................................................................................... 6 Informatica Data Subset.................................................... 4 Count Validation ................. All Rights Reserved.............................. 5 UAT (User Acceptance Test) ...........................................................

Quantitative Testing Validate your Source and Target a) Ensure that your connectors are configured properly.com 3 . a) Have customized SQL queries to check the source/targets and here we will perform the Record Count Verification. b) Analyze the rejections and build a process to handle those rejections. c) Use the session and workflow logs to capture the load statistics. b) If you are using flat file make sure have enough read/write permission on the file share. All Rights Reserved. This requires a clear business requirement from the business on how to handle the data rejections. Testing Architecture & Processes Testing tools available with Informatica Testing facilities available with Informatica PowerCenter Unit Testing Unit testing can be broadly classified into 2 categories.informatica. Informatica Technology Network © 2009 Informatica Corporation. b) Check the Read and Write counters. Integration and UAT. d) You need to document all the load timing information. Analyze the Load Time a) Execute the session and review the session statistics. Performance Improvement a) Network Performance b) Session Performance c) Database Performance d) Analyze and if required define the Informatica and DB partitioning requirements. Analyze the success rows and rejections.Testing Approaches using Informatica PowerCenter Introduction This article briefs about the following: Different Testing Types like Unit Testing. http://technet. How long it takes to perform the load. Do we need to reload or reject and inform etc? Discussions are required and appropriate process must be developed. c) You need to document all the connector information.

documents. 3. Informatica Technology Network © 2009 Informatica Corporation. or dollars.Testing Approaches using Informatica PowerCenter Qualitative Testing Analyze & validate your transformation business rules. or simple record counts Hash totals: This is a technique for improving data accuracy. g) Ensure that appropriate dimension lookup’s have been used and your development is in sync with your business requirements. 5. Testing the rejected records that don’t fulfil transformation rules. More of functional testing. 4. Error log generation. Integration testing will involve following 1. control totals can be compared by the system with manually entered or otherwise calculated control totals using the data fields such as quantities.g. line items. http://technet. Initial loading of records on data warehouse. part number. social security number. Dimensional Analysis Data integrity between the various source tables and relationships. Statistical Analysis Validation for various calculations. e) You need review field by field from source to target and ensure that the required transformation logic is applied. All Rights Reserved. These totals have no significance other than for internal system control purposes. Limit checks: The program tests specified data fields against defined high or low value limits (e. Integration Testing After unit testing is complete. Sequence of ETLs jobs in batch.. such as account number. quantities or dollars) for acceptability before further processing.com 4 . it should form the basis of starting integration testing. Integration Testing would cover End-to-End Testing for DWH. f) If you are making changes to existing mappings make use of the data lineage feature available with Informatica PowerCenter. fields for which it would logically be meaningless to construct a total).. The coverage of the tests would include the below: Count Validation Record Count Verification: DWH backend/Reporting queries against source and target as an initial check. whereby totals are obtained on identifier fields (i. or employee number. This will help you to find the consequences of altering or deleting a port from existing mapping. Integration testing should test out initial and incremental loading of the data warehouse. Incremental loading of records at a later date to verify the newly inserted or updated data.e. 2.informatica. Control totals: To ensure accuracy in data entry and processing.

http://technet. All Rights Reserved.informatica. You cannot perform a test load on sessions using XML sources.and post-session functions. Format checks: These are used to determine that data are entered in the proper mode. Usually. Sign test: This is a test for a numeric data field containing a designation of an algebraic sign. The proper mode in each case depends on the data field definition. Instead you use the Enable Test Load feature available in Informatica PowerCenter. and division). Enter the number of source rows you want the Integration Service to test load.. For example. but rolls back the data when the session completes. multiplication. If you configure the session for bulk mode. Data Quality Validation Check for missing data. The Integration Service writes data to relational targets. addition. You can perform a test load for relational targets when you configure a session for normal mode. Overflow checks: This is a limit check based on the capacity of a data field or data file area to accept data. Enter the number of source rows you want to test in the Number of Rows to Test field. Field-by-Field data verification can be done to check the consistency of source and target data.Testing Approaches using Informatica PowerCenter o o When you validate the calculations you don’t require loading the entire rows into target and validating it. the Integration Service does not write data to the targets. Informatica Technology Network © 2009 Informatica Corporation. The Integration Service generates all session files. the first digit is the one lost.g. a social security number in the United States should have nine digits Granularity Validate at the lowest granular level possible Other validations Audit Trails. and performs all pre. Note: Based on your project and business needs you might have additional testing requirements. which can be used to denote.. This programming technique can be used to detect the truncation of a financial or quantity data field value after computation (e. For all other target types. for example. negatives and consistency.com 5 . the session fails. Error Logs and Validity checks. Size test: This test can be used to test the full size of the data field. debits or credits for financial data fields. the Integration Service reads and transforms data without writing to targets. as numeric or alphabetical characters. such as flat file and SAP BW. With a test load. + or . as if running the full session. Property Enable Test Load Number of Rows to Test Description You can configure the Integration Service to perform a test load. within designated fields of information. The Integration Service reads the number you configure for the test load. Transaction Logs.

you have implement a change requirement where by you need 1 year worth of data from production. Secured In this case production data will be always masked before they are available in the DEV environment. effort.com 6 . Unsecured 2. Informatica Data Subset is a flexible enterprise data growth solution that automates the process of creating smaller. a developer can see the production data as it is. separating out only functionally related data. All Rights Reserved. Secured Unsecured Even now many organizations go for unsecured testing architecture because it requires little budget and less maintenance. In this case you will develop a mapping to read the data from production warehouse and load into development and proceed with the development. IT organizations can dramatically reduce the amount of time. Meaning. and disk space necessary to support nonproduction systems.Testing Approaches using Informatica PowerCenter UAT (User Acceptance Test) In this phase you will involve the user to test the end results and ensure that business is satisfied with the quality of the data.informatica. Some organizations will perform data masking before bringing the data from production to UAT or Development environment. complex databases. Informatica Technology Network © 2009 Informatica Corporation. Any changes to the business requirement will follow the change management process and eventually those changes have to follow the SDLC process. With referentially intact subsets of production data. Informatica Data Subset Informatica also provides a tool called “Informatica Data Subset” The following is an overview copied from Informatica web site about the Data subset product. targeted databases from large. Assume that you have sales data warehouse. Informatica Data Subset helps IT organizations untangle complex transactional systems. http://technet. Testing Architecture From my perspective there are two broad categories of testing architecture 1.

com 7 . instead of coding by hand—which is expensive.Testing Approaches using Informatica PowerCenter Informatica Data Subset is ideal for: Optimize Development. and training systems. resource intensive.informatica. testing. smaller targeted data subsets for development. production-like data in training systems • • • • • • Support Corporate Divestitures and Reorganizations • • • Untangle complex operational systems and separate data along business lines to quickly build the divested organization’s system Accelerate the provisioning of new systems by using only data that’s relevant to the divested organization Decrease the cost and time of data divestiture with no reimplementation costs Reduce the Total Cost of Storage Ownership • Dramatically increase an IT team’s productivity by reusing a comprehensive list of data objects for data selection and updating processes across multiple projects. while maintaining full data integrity Quickly build and update nonproduction systems with a small subset of production data and replicate current subsets of nonproduction copies faster Simplify test data management and shrink the footprint of nonproduction systems to significantly reduce IT infrastructure and maintenance costs Reduce application and upgrade deployment risks by properly testing configuration updates with upto-date. Testing. and time consuming Accelerate application delivery by decreasing R&D cycle time and streamlining test data management Improve the reliability of application delivery by ensuring IT teams have ready access to updated quality production data Lower administration costs by centrally managing data growth solutions across all packaged and custom applications Substantially accelerate time to value for subsets of packaged applications Decrease maintenance costs by eliminating custom code and scripting • • • • • Informatica Technology Network © 2009 Informatica Corporation. realistic data before introducing them into production Easily customize provisioning rules to meet each organization’s changing business requirements Lower training costs by standardizing on one approach and one infrastructure Train employees effectively using reliable. and Training Systems • Dramatically accelerate development and test cycles and reduce storage costs by creating fully functional. All Rights Reserved. http://technet.

Testing Approaches using Informatica PowerCenter Testing Processes Concentrate on the following for any testing requirements that you have: 1. such as flat file and SAP BW. Refer informatica documentation to know more about debugger tool. All Rights Reserved. In any organization we will have parallel activities going on. but rolls back the data when the session completes. Like BA’s want to test a functionality in UAT for which they need data from Production. Data Load Matrix Project Team QA Development Request Description Priority HIGH LOW Data Refresh Cycle Monthly Daily Request Type One-Off Regular 4. http://technet. Property Description Enable You can configure the Integration Service to perform a test load. Developer wants to perform a unit testing for which he/she needs data from Production. Defining data selection criteria and data masking rules 5. as if running the full session. You cannot perform a test load on sessions using XML sources. You can perform a test load for relational targets when you configure a session for Informatica Technology Network © 2009 Informatica Corporation. For the above such requests we need to create a data load matrix and prioritize their needs. Test Load Options – Relational Targets. the Integration Service reads and transforms data without writing to targets. 2. Auditing and security Informatica PowerCenter Testing Debugger: Very useful tool for debugging a valid mapping to gain troubleshooting information about data and error conditions.com 8 . The Integration Service generates all session files. the Integration Service does not write data to the targets. and performs all preand post-session functions. Identify the sensitive information and develop processes to protect it 3. Enter the number of source rows you want to test in the Number of Rows to Test field. The Integration Service writes data to relational targets.informatica. Testing and Validation 5. Understanding data requirements for test and development. For all other target types. Understanding the application data & business requirement. Test Load With a test load.

Configure the Integration Service to fail over in safe mode and troubleshoot errors when you migrate or test a production environment configured for high availability. You can export the mapping to an XML file and edit the repository connection information before sending the XML file. If you configure the session for bulk mode.informatica. Run the Integration Service in safe mode to test a development environment before migrating to production Troubleshoot the Integration Service. http://technet. Performance Testing for identifying the following bottlenecks: o o o o o Target Source Mapping Session System Use the following methods to identify performance bottlenecks: Run test sessions. the session fails.Testing Approaches using Informatica PowerCenter normal mode. you can correct the error that caused the Integration Service to fail over.com 9 . Share metadata. Analyze performance details. The third party can import the mapping from the XML file and analyze the metadata. For example. Analyze thread statistics to determine the optimal number of partition points. I/O waits. Informatica Technology Network © 2009 Informatica Corporation. you want to send a mapping to someone else for testing or analysis. and paging to identify system bottlenecks. You can configure a test session to read from a flat file source or to write to a flat file target to identify source and target bottlenecks. Syntax Testing: Test your customized queries using your source qualifier before executing the session. Monitor system performance. Test Running the Integration Service in Safe Mode o o Test a development environment. You can also use the Workflow Monitor to view system resource usage. Analyze thread statistics. After the Integration Service fails over in safe mode. but you do not want to disclose repository connection information for security reasons. such as performance counters. to determine where session performance decreases. Rows to The Integration Service reads the number you configure for the test load. You can use system monitoring tools to view the percentage of CPU use. All Rights Reserved. Number of Enter the number of source rows you want the Integration Service to test load. Analyze performance details. You can share metadata with a third party. Use PowerCenter conditional filter in the Source Qualifier to improve performance.

You agree that you will not hold. Informatica responsible or liable with respect to the content of this software asset. including any liability resulting from incompatibility between the content within this asset and the materials and services offered by Informatica. http://technet. Informatica Technology Network © 2009 Informatica Corporation.Testing Approaches using Informatica PowerCenter Disclaimer and Liability notice Informatica offers no guarantees and assumes no responsibility or liability of any type with respect to the content of this software asset.informatica.com 10 . or seek to hold. All Rights Reserved.

Sign up to vote on this title
UsefulNot useful