Professional Documents
Culture Documents
1
Basics
Education Services
Version PC7B-20041208
1
Introduction
Course Objectives
3
About Informatica
Founded in 1993
Leader in enterprise solution products
Headquarters in Redwood City, CA
Public company since April 1999 (INFA)
2000+ customers, including over 80% of Fortune 100
Strategic partnerships with IBM, HP, Accenture, SAP,
and many others
Worldwide distributorship
4
Informatica Products
• Exams A, B, C, D plus
• Exam E: Enablement Technologies
7
Extract, Transform and Load
8
PowerCenter Client Tools
Console
Manage repository: Build ETL Build and start Monitor Administer repositories on
• Connections mappings workflows to and start a Repository Server:
• Folders run mappings workflows • Create/upgrade/delete
• Objects • Configuration
• Users and groups • Start/stop
• Backup/restore
9
PowerCenter 7 Architecture
Informatica Server
Native Native
Sources Targets
TCP/IP
Repository
Heterogeneous Server Heterogeneous
Sources Targets
TCP/IP Repository
Agent
Native
Repository Designer Workflow Workflow Rep Server
Manager Manager Monitor Administrative
Repository
Console
Not Shown: Client ODBC connections from Designer to sources and targets for metadata
10
Distributed Architecture and Platforms
The following components can be distributed across a
network of host computers:
− Clients Tools
− PowerCenter Servers
− Repository Servers
− Repository Databases
− Sources and Targets
Platforms:
− Client tools run on Windows
− Servers run on AIX, HP-UX, Solaris, Redhat Linux, Windows
− Repositories on any major RDBMS
11
Design and Execution Process
12
Demonstration
13
Source Object Definitions
Source Object Definitions
15
Methods of Analyzing Sources
Source Analyzer
Import from:
− Relational database
− Flat file
− XML object
Repository Create manually
Server
TCP/IP
Repository Agent
Native
Repository
DEF
16
Analyzing Relational Database Sources
Source Analyzer Relational DB Source
ODBC • Table
• View
• Synonym
DEF
Repository
Server
TCP/IP
Repository Agent
Native
Repository
DEF
17
Analyzing Relational Database Sources
Editing Source Definition Properties
18
Analyzing Flat File Sources
Source Analyzer • Mapped Drive Flat File
• NFS Mount
• Local Directory DEF
• Fixed Width
• Delimited
Repository
Server
TCP/IP
Repository Agent
Native
Repository
DEF
19
Flat File Wizard
Three-step
wizard
Columns can
be renamed
within wizard
Text, Numeric
and Datetime
datatypes are
supported
Wizard
‘guesses’
datatype
20
Flat File Source Properties
21
Analyzing XML Sources
Source Analyzer • Mapped Drive XML Schema (XSD),
• NFS Mounting DTD or XML File
• Local Directory
DEF
Repository
Server DATA
TCP/IP
Repository Agent
Native
Repository
DEF
22
Data Previewer
Preview data in
• Relational database sources
• Flat file sources
• Relational database targets
• Flat file targets
23
Using Data Previewer in Source Analyzer
Data Preview Example
From Source Analyzer,
select Source drop down
menu, then Preview Data
24
Using Data Previewer in Source Analyzer
Data Preview Results
Data
Display
View up
to 500
rows
25
Metadata Extensions
26
Metadata Extensions
27
Example – Metadata Extension for a Source
28
Target Object Definitions
Target Object Definitions
30
Creating Target Definitions
31
Import Definition from Relational Database
Can obtain existing object definitions from a database
system catalog or data dictionary
Relational DB
Warehouse
Designer ODBC
•Table
•View
Repository DEF •Synonym
Server
TCP/IP
Repository Agent
Native
Repository
DEF
32
Import Definition from XML Object
Can infer existing object definitions from a database
system catalog or data dictionary
• Mapped Drive DTD, XML Schema or
Warehouse • NFS Mounting
Designer • Local Directory XML File
DEF
Repository
Server
D
TCP/IP ATA
Repository Agent
Native
Repository
DEF
33
Creating Target Automatically from Source
Drag-and-drop
a Source
Definition into
the
Warehouse
Designer
Workspace
34
Target Definition Properties
35
Heterogeneous Targets
36
Heterogeneous Targets
37
Definition: Heterogeneous Targets
38
Step One: Identify Different Target Types
Oracle table
39
Step Two: Different Database Connections
40
Target Type Override (Conversion)
41
Lab 1 – Define Sources and Targets
42
Transformation Basic Concepts
43
Transformations Objects Used in This Class
Source Qualifier: reads data from flat file & relational sources
Expression: performs row-level calculations
Filter: drops rows conditionally
Sorter: sorts data
Aggregator: performs aggregate calculations
Joiner: joins heterogeneous sources
Lookup: looks up values and passes them to other objects
Update Strategy: tags rows for insert, update, delete, reject
Router: splits rows conditionally
Sequence Generator: generates unique ID values
44
Other Transformation Objects
Normalizer: normalizes records from relational or VSAM sources
Rank: filters the top or bottom range of records
Union: merges data from multiple pipelines into one pipeline
Transaction Control: allows user-defined commits
Stored Procedure: calls a database stored procedure
External Procedure : calls compiled code for each row
Custom: calls compiled code for multiple rows
Midstream XML Parser: reads XML from database table or message queue
Midstream XML Generator: writes XML to database table or message queue
45
Transformation Views
A transformation has
three views:
Iconized – shows the
transformation in relation
to the rest of the
mapping
Normal – shows the flow
of data through the
transformation
Edit – shows
transformation ports
(= table columns)
and properties;
allows editing
46
Expression Transformation
Ports
• Mixed
• Variables allowed
Create expression in an
output or variable port
Usage
• Perform majority of Click here to invoke the
data manipulation Expression Editor
47
Expression Editor
An expression formula is a calculation or conditional statement for a
specific port in a transformation
Performs calculation based on ports, functions, operators, variables,
constants and return values from other transformations
48
Expression Validation
49
Informatica Functions – Character
50
Informatica Functions – Conversion
TO_CHAR (numeric)
TO_DATE Conversion Functions
TO_DECIMAL
TO_FLOAT Used to convert datatypes
TO_INTEGER
51
Informatica Functions – Data Cleansing
INSTR Used to process data during data
IS_DATE
IS_NUMBER
cleansing
IS_SPACES METAPHONE and SOUNDEX create
ISNULL
LTRIM
indexes based on English
METAPHONE pronunciation (2 different standards)
REPLACECHR
REPLACESTR
RTRIM
SOUNDEX
SUBSTR
TO_CHAR
TO_DATE
TO_DECIMAL
TO_FLOAT
TO_INTEGER
52
Informatica Functions – Date
53
Informatica Functions – Numerical and Scientific
54
Informatica Functions – Special and Test
Special Functions
55
Variable Ports
56
Variable Ports (cont’d)
57
Default Values – Two Usages
For input and I/O ports, default values are used to replace null
values
For output ports, default values are used to handle transformation
calculation errors (not-null handling)
Selected
port Validate the
default
value
expression
Default
value for the
selected
port ISNULL function
is not required
58
Informatica Datatypes
NATIVE DATATYPES TRANSFORMATION DATATYPES
Specific to the source and target PowerCenter internal datatypes
database types
Display in source and target tables Display in transformations within
within Mapping Designer Mapping Designer
Native Transformation Native
Transformation datatypes allow mix and match of source and target database types
When connecting ports, native and transformation datatypes must be compatible
(or must be explicitly converted)
59
Datatype Conversions within PowerCenter
For further information, see the PowerCenter Client Help > Index
> port-to-port data conversion
60
Mappings
Mappings
Mapping validation
62
Mapping Designer
Transformation Toolbar
Mapping List
Iconized Mapping
63
Source Qualifier Transformation
Represents the source record set queried by the
Server. Mandatory in Mappings using relational or
flat file sources
Ports
• All input/output
Usage
• Convert datatypes
• For relational sources:
Modify SQL statement
User Defined Join
Source Filter
Sorted ports
Select DISTINCT
Pre/Post SQL
64
Source Qualifier Properties
65
Pre-SQL and Post-SQL Rules
66
Mapping Validation
67
Connection Validation
68
Mapping Validation
Mappings must:
• Be valid for a Session to run
• Be end-to-end complete and contain valid expressions
• Pass all data flow rules
Mappings are always validated when saved; can be validated
without being saved
Output Window displays reason for invalidity
69
Lab 2 – Create a Mapping
70
Workflows
Workflows
By the end of this section, you will be familiar with:
The Workflow Manager GUI interface
Creating and configuring Workflows
Workflow properties
Workflow components
Workflow tasks
72
Workflow Manager Interface
Task
Workflow
Tool Bar
Designer
Tools
Navigator
Window
Workspac
e
Output
Status Window
Bar
73
Workflow Manager Tools
Workflow Designer
• Maps the execution order and dependencies of Sessions,
Tasks and Worklets, for the Informatica Server
Task Developer
• Create Session, Shell Command and Email tasks
• Tasks created in the Task Developer are reusable
Worklet Designer
• Creates objects that represent a set of tasks
• Worklet objects are reusable
74
Workflow Structure
A Workflow is set of instructions for the Informatica
Server to perform data transformation and load
Combines the logic of Session Tasks, other types of
Tasks and Worklets
The simplest Workflow is composed of a Start Task, a
Link and one other Task
Link
Start Session
Task Task
75
Reusable Tasks
Reusable Tasks
77
Reusable Tasks
78
Reusable Tasks in a Workflow
Reusable
Non-reusable
79
Command Task
80
Command Task
81
Command Task (cont’d)
82
Command Task (cont’d)
Add Cmd
Remove Cmd
83
Email Task
84
Email Task (cont’d)
85
Non-Reusable Tasks
Non-Reusable Tasks
Decision
Assignment
Timer
Control
Event Wait
Event Raise
87
Decision Task
Specifies a condition to be evaluated in the Workflow
88
Assignment Task
General Tab
Expressions Tab
89
Timer Task
Waits for a specified period of time to execute the
next Task
General Tab Timer Tab
• Absolute Time
• Datetime Variable
• Relative Time
90
Control Task
General
Tab
91
Event Wait Task
92
Event Wait Task (cont’d)
General Tab
Properties Tab
93
Event Wait Task (cont’d)
Events Tab
94
Event Raise Task
Represents the location of a user-defined event
The Event Raise Task triggers the user-defined event when the
Informatica Server executes the Event Raise Task
95
Session Task
96
Additional Workflow Tasks
Eight additional Tasks are available in the Workflow Designer (covered
later)
• Command
• Email
• Decision
• Assignment
• Timer
• Control
• Event Wait
• Event Raise
97
Sample Workflow
Session 1
Command
Task
(required)
98
Sequential and Concurrent Workflows
Sequential
Concurrent Combined
Note: Although only session tasks are shown, can be any tasks
99
Creating a Workflow
Customize
Workflow name
Select a
Server
100
Workflow Properties
Customize Workflow
Properties
May be reusable or
non-reusable
Select a Workflow
Schedule (optional)
101
Workflow Scheduler
102
Workflow Metadata Extensions
103
Workflow Links
Required to connect Workflow Tasks
Can be used to create branches in a Workflow
All links are executed – unless a link condition is used which
makes a link false
Link 1 Link 3
Link 2
104
Conditional Links
Optional link
condition
‘$taskname.STATUS’
is a pre-defined
task variable
105
Workflow Variables 1
Used in decision tasks and conditional links – edit task or link:
Pre-defined variables User-defined variables
(see separate slide)
Task-specific
variables
Built-in system
variables
106
Workflow Variables 2
User-defined variables are set in Workflow properties,
Variables tab – can persist across sessions
Can be reset in
an Assignment task
107
Workflow Summary
108
Session Tasks
Session Tasks
110
Creating a Session Task
Created to execute the logic of a mapping (one
mapping only)
Session Tasks can be created in the Task Developer
(reusable) or Workflow Developer (Workflow-specific)
To create a Session Task
• Select the Session button from the Task Toolbar
111
Session Task – Properties and Parameters
Properties Tab
Session Task
Session
parameter
Parameter file
112
Session Task – Setting Source Properties
Mapping Tab
Session Task
Select
source
instance
Set
connection
Set
properties
113
Session Task – Setting Target Properties
Mapping Tab
Session Task
Select target
instance
Set connection
Set properties
114
Monitoring Workflows
Monitoring Workflows
116
Workflow Monitor
The Workflow Monitor is the tool for monitoring
Workflows and Tasks
Choose between two views:
• Gantt chart
• Task view
117
Monitoring Current and Past Workflows
118
Monitoring Operations
Perform operations in the Workflow Monitor
• Stop, Abort, or Restart a Task, Workflow or Worklet
• Resume a suspended Workflow after a failed Task is
corrected
• Reschedule or Unschedule a Workflow
119
Monitoring in Task View
Start Completion
Task Server Workflow Worklet Time Time
120
Filtering in Task View
Monitoring filters
can be set using
drop down menus.
Minimizes items
displayed in
Task View
121
Filter Toolbar
122
Truncating Workflow Monitor Logs
Workflow Monitor
Repository Manager
Repository Manager’s
Truncate Log option
clears the Workflow
Monitor logs
123
Lab 3 – Create and Run a Workflow
124
Lab 4 – Features and Techniques I
125
Debugger
Debugger
127
Debugger Features
Initialize variables
128
Debugger Interface
Edit
Breakpoints
Debugger Mode
indicator
Solid yellow
arrow is current
transformation
indicator
Flashing
yellow
SQL
indicator
129
Set Breakpoints
1. Edit breakpoint
2. Choose global or
specific transformation
3. Choose to break on
data condition or error.
Optionally skip rows.
4. Add breakpoint(s)
5. Add data conditions
131
Lab 5 – The Debugger
132
Filter Transformation
Filter Transformation
Ports
• All input / output
Usage
• Filter rows from
input flow
134
Lab 6 – Flat File Wizard and Filter
Transformation
135
Sorter Transformation
Sorter Transformation
137
Sorter Transformation
Sorts data from any source, at any point in a data flow
Sort Keys
Ports
• Input/Output
• Define one or more
sort keys
• Define sort order for
each key
Example of Usage
• Sort data before
Aggregator to improve
performance
Sort Order
138
Sorter Properties
139
Aggregator Transformation
Aggregator Transformation
141
Aggregator Transformation
Ports
• Mixed I/O ports allowed
• Variable ports allowed
• Group By allowed
Create expressions in
variable and output ports
Usage
• Standard aggregations
142
Aggregate Expressions
Aggregate
functions are
supported only in
the Aggregator
Transformation
Conditional Aggregate
expressions are supported: Conditional SUM format: SUM(value, condition)
143
Aggregator Functions
Return summary values for non-null data
AVG in selected ports
COUNT
FIRST Use only in Aggregator transformations
LAST
MAX
Use in output ports only
MEDIAN
MIN
Calculate a single value (and row) for all
PERCENTILE records in a group
STDDEV
SUM
Only one aggregate function can be
VARIANCE nested within an aggregate function
Conditional statements can be used with
these functions
144
Aggregator Properties
Instructs the
Aggregator to
expect the data
to be sorted
Set Aggregator
cache sizes for
Informatica Server
machine
145
Sorted Data
146
Aggregating Unsorted Data
147
Aggregating Sorted Data
148
Data Flow Rules – Terminology
Passive transformation
• Operates on one row of data at a time AND
• Cannot change the number of rows on the data flow
• Example: Expression transformation
Active transformation
• Can operate on groups of data rows AND/OR
• Can change the number of rows on the data flow
• Examples: Aggregator, Filter, Source Qualifier
149
Data Flow Rules
Each Source Qualifier starts a single data stream (data flow)
Transformations can send rows to more than one
transformation (split one data flow into multiple pipelines)
Two or more data flows can meet only if they originate from
a common active transformation
ALLOWED DISALLOWED
Passive Active
T T T T
Example holds true with Normalizer instead of Source Qualifier.
Exceptions are: Mapplet Input and sorted Joiner transformations
150
Joiner Transformation
Joiner Transformation
152
When to Join in Source Qualifier
If you can perform a join on the source database, then
you can configure it in the Source Qualifier
The SQL that the Source Qualifier generates, default or
custom, executes on the source database at runtime
Example: homogeneous join – 2 database tables in same
database
153
When You Cannot Join in Source Qualifier
154
Joiner Transformation
Active Transformation
Ports
• All input or input / output
• “M” denotes port comes
from master source
Examples
• Join two flat files
• Join two tables from
different databases
• Join a flat file with a
relational table
155
Joiner Conditions
156
Joiner Properties
Join types:
• Normal (inner)
• Master outer
• Detail outer
• Full outer
Set Joiner
Caches
Joiner can accept sorted data (configure the join condition to use the
sort origin ports)
157
Nested Joins
158
Mid-Mapping Join (Unsorted)
159
Lab 7 – Heterogeneous Join, Aggregator, and
Sorter
160
Lookup Transformation
Lookup Transformation
162
How a Lookup Transformation Works
For each mapping row, one or more port values are looked up in a
database table or flat file
If a match is found, one or more table values are returned to the
mapping. If no match is found, NULL is returned
Lookup value(s)
Lookup transformation
Return value(s)
163
Lookup Transformation
Looks up values in a database table or flat file and
provides data to other components in a mapping
Ports
• Mixed
• “L” denotes Lookup port
• “R” denotes port used as a
return value (unconnected
Lookup only – see later)
Specify the Lookup Condition
Usage
• Get related values
• Verify if records exists or if
data has changed
164
Lookup Conditions
165
Lookup Properties
Lookup
table name
Lookup condition
Native database
connection object name
Source type:
Database or Flat File
166
Lookup Properties cont’d
Policy on multiple
match:
• Use first value
• Use last value
• Report error
167
Lookup Caching
Caching can significantly impact performance
Cached
• Lookup table data is cached locally on the Server
• Mapping rows are looked up against the cache
• Only one SQL SELECT is needed
Uncached
• Each Mapping row needs one SQL SELECT
Rule Of Thumb: Cache if the number (and size) of records in
the Lookup table is small relative to the number of mapping
rows requiring the lookup
168
Persistent Caches
169
Lookup Caching Properties
Override Lookup
SQL option
Toggle
caching
Cache
directory
170
Lookup Caching Properties (cont’d)
Make cache
persistent
Set Lookup
cache sizes
Reload
persistent
cache
171
Lab 8 – Basic Lookup
172
Target Options
Target Options
174
Setting Default Target Load Type
175
Target Properties
Edit Tasks: Mappings Tab
Session Task
Select target
instance
Target load type
Row loading
operations
Error handling
176
WHERE Clause for Update and Delete
PowerCenter uses the primary keys defined in the
Warehouse Designer to determine the appropriate SQL
WHERE clause for updates and deletes
Update SQL
• UPDATE <target> SET <col> = <value>
WHERE <primary key> = <pkvalue>
• The only columns updated are those which have values linked
to them
• All other columns in the target are unchanged
• The WHERE clause can be overridden via Update Override
Delete SQL
• DELETE from <target> WHERE <primary key> = <pkvalue>
SQL statement used will appear in the Session log file
177
Constraint-based Loading
PK1
FK1 PK2
FK2
178
Setting Constraint-based Loading
179
Constraint-based Loading – Terminology
Active transformation
• Can operate on groups of data rows and/or
can change the number of rows on the data flow
• Examples: Source Qualifier, Aggregator, Joiner, Sorter, Filter
Active source
• Active transformation that generates rows
• Cannot match an output row with a distinct input row
• Examples: Source Qualifier, Aggregator, Joiner, Sorter
• (The Filter is NOT an active source)
Active group
• Group of targets in a mapping being fed by the same active
source
180
Constraint-Based Loading – Restrictions
Cannot have two active groups
PK1 Example 1
With only one Active source,
FK1 rows for Targets1, 2, and 3 will
PK2 be loaded properly and maintain
referential integrity
FK2
PK1 Example 2
With two Active sources, it is not
possible to control whether rows
FK1 for Target3 will be loaded before
PK2
or after those for Target2
FK2
181
Lab 9 – Deleting Rows
182
Update Strategy
Transformation
Update Strategy Transformation
Used to specify how each individual row will be used to
update target tables (insert, update, delete, reject)
Ports
• All input / output
• Specify the Update
Strategy Expression –
IIF or DECODE logic
determines how to
handle the record
Example
• Updating Slowly
Changing Dimensions
184
Update Strategy Expressions
185
Lab 10 – Data Driven Operations
186
Lab 11 – Incremental Update
187
Lab 12 – Features and Techniques II
188
Router Transformation
Router Transformation
Ports
• All input/output
• Specify filter conditions
for each Group
Usage
• Link source data in one
pass to multiple filter
conditions
190
Router Groups
Input group (always one)
User-defined groups
Each group has one condition
ALL group conditions are evaluated
for EACH row
One row can pass multiple
conditions
Unlinked Group outputs are
ignored
Default group (always one) can
capture rows that fail all Group
conditions
191
Router Transformation in a Mapping
192
Lab 13 – Router
193
Sequence Generator
Transformation
Sequence Generator Transformation
Ports
• Two predefined output
ports, NEXTVAL and
CURRVAL
• No input ports allowed
Usage
• Generate sequence
numbers
• Shareable across mappings
195
Sequence Generator Properties
Number of
cached values
196
Mapping Parameters and
Variables
Mapping Parameters and Variables
198
System Variables
Provides current datetime on the
SYSDATE
Informatica Server machine
• Not a static value
Returns the system date value on the
SESSSTARTTIME
Informatica Server
• Used with any function that accepts
transformation date/time datatypes
• Not to be used in a SQL override
• Has a constant value
$$$SessStartTime
Returns the system date value as a
string. Uses system clock on machine
hosting Informatica Server
• Format of the string is database type
dependent
• Used in SQL override
• Has a constant value
199
Mapping Parameters and Variables
200
Mapping Parameters and Variables
Sample declarations
Set datatype
User-defined
names Set
aggregation
type
Set optional
initial value
204
Parameters & Variables – Initialization Priority
1. Parameter file
2. Repository value
4. Default value
205
Unconnected Lookups
Unconnected Lookups
207
Unconnected Lookup
Physically unconnected from other transformations – NO data flow
arrows leading to or from an unconnected Lookup
Lookup data is called from the point in the Mapping that needs it
Lookup function can be set within any transformation that supports
expressions
208
Unconnected Lookup Technique
Use lookup lookup function within a conditional statement
Row keys
Condition (passed to Lookup)
IIF ( ISNULL(customer_id),:lkp.MYLOOKUP(order_no))
Lookup function
209
Unconnected Lookup Advantage
Data lookup is performed only for those rows which
require it. Substantial performance can be gained
Condition Lookup
(true for 2 percent of all (called only when condition is
rows) true)
Net savings = 490,000 lookups
210
Unconnected Lookup Functionality
Must check a
Return port in the
Ports tab, else
fails at runtime
211
Connected versus Unconnected Lookups
CONNECTED LOOKUP UNCONNECTED LOOKUP
Part of the mapping data flow Separate from the mapping data
flow
Returns multiple values (by Returns one value - by checking
linking output ports to another the Return (R) port option for the
transformation) output port that provides the
return value
Executed for every record Only executed when the lookup
passing through the function is called
transformation
More visible, shows where the Less visible, as the lookup is
lookup values are used called from an expression within
another transformation
Default values are used Default values are ignored
212
Lab 14 – Straight Load
213
Lab 15 – Conditional Lookup
214
Mapplets
Mapplets
216
Mapplet Designer
Mapplet
Input and Output
Transformation
Mapplet Designer Tool Icons
Mapplet Output
Transformation
217
Mapplet Advantages
218
A Mapplet Used in a Mapping
219
The “Detail” Inside the Mapplet
220
Unsupported Transformations
221
Mapplet Source Options
Internal Sources
• One or more Source definitions / Source Qualifiers
within the Mapplet
External Sources
Mapplet contains a Mapplet Input transformation
• Receives data from the Mapping it is used in
Mixed Sources
• Mapplet contains one or more of either of a Mapplet Input
transformation AND one or more Source Qualifiers
• Receives data from the Mapping it is used in, AND from
the Mapplet
222
Mapplet Input Transformation
Use for data sources outside a Mapplet
Passive Transformation
Connected
Ports
• Output ports only • Connecting the
Usage same port to more
Only those ports than one
connected from an transformation is
Input transformation disallowed
to another Transformation • Pass to an
transformation Expression
will display in the transformation
Transformation
resulting Mapplet first
223
Data Source Outside a Mapplet
Source data is defined
OUTSIDE the Mapplet logic
Mapplet Input
Transformation
224
Data Source Inside a Mapplet
Source data is defined Source
WITHIN the Mapplet logic Qualifier
• No Input transformation
is required (or allowed)
• Use a Source Qualifier
instead
225
Mapplet Output Transformation
Use to contain the results of a Mapplet pipeline. Multiple
Output transformations are allowed.
Passive Transformation
Connected
Ports
• Input ports only
Usage
• Only those ports connected to
an Output transformation (from
another transformation) will
display in the resulting Mapplet
• One (or more) Mapplet Output
transformations are required in
every Mapplet
226
Mapplet with Multiple Output Groups
227
Unmapped Mapplet Output Groups
Warning: An unlinked
Mapplet Output Group
may invalidate the
mapping
228
Active and Passive Mapplets
229
Using Active and Passive Mapplets
Multiple Passive
Mapplets can populate
Passive the same target
instance
230
Mapplet Parameters and Variables
231
Lab 17 – Mapplets
232
Reusable Transformations
Reusable Transformations
234
Transformation Developer
Make a
transformation
reusable from
the outset,
or
test it in a
Reusable mapping first
transformations
235
Reusable Transformations
Define once, reuse many times
Reusable Transformations
• Can be a copy or a shortcut
• Edit Ports only in Transformation Developer
• Can edit Properties in the mapping
• Instances dynamically inherit changes
• Caution: changing reusable transformations can invalidate
mappings
236
Promoting a Transformation to Reusable
Check the
Make reusable box
(irreversible)
237
Copying Reusable Transformations
238
Lab 18 – Reusable Transformations
239
Session-Level Error Logging
Error Logging Objectives
241
Error Types
Transformation error
− Data row has only passed partway through the mapping
transformation logic
− An error occurs within a transformation
Data reject
− Data row is fully transformed according to the mapping
logic
− Due to a data issue, it cannot be written to the target
− A data reject can be forced by an Update Strategy
242
Error Logging Off/On
243
Setting Error Log Options
In Session task
244
Error Logging Off – Specifying Reject Files
In Session task
245
Error Logging Off – Transformation Errors
X
X
246
Error Logging Off – Data Rejects
Conditions causing data to be rejected include:
• Target database constraint violations, out-of-space errors, log
space errors, null values not accepted
• Data-driven records, containing value ‘3’ or DD_REJECT
(the reject has been forced by an Update Strategy)
• Target table properties ‘reject truncated/overflowed rows’
First column:
Sample reject file
0=INSERT →0,D,1313,D,Regulator System,D,Air Regulators,D,250.00,D,150.00,D
1=UPDATE→1,D,1314,D,Second Stage Regulator,D,Air Regulators,D,365.00,D,265.00,D
2=DELETE →2,D,1390,D,First Stage Regulator,D,Air Regulators,D,170.00,D,70.00,D
3=REJECT →3,D,2341,D,Depth/Pressure Gauge,D,Small Instruments,D,105.00,D,5.00,D
247
Log Row Data
Logs:
Session metadata
For errors on input, logs row data for I and I/O ports
For errors on output, logs row data for I/O and O ports
248
Logging Errors to a Relational Database 1
Relational
Database Log
Settings
249
Logging Errors to a Relational Database 2
250
Error Logging to a Flat File 1
251
Logging Errors to a Flat File 2
Format: Session metadata followed by de-normalized error information
Sample session metadata
**********************************************************************
Repository GID: 510e6f02-8733-11d7-9db7-00e01823c14d
Repository: RowErrorLogging
Folder: ErrorLogging
Workflow: w_unitTests
Session: s_customers
Mapping: m_customers
Workflow Run ID: 6079
Worklet Run ID: 0
Session Instance ID: 806
Session Start Time: 10/19/2003 11:24:16
Session Start Time (UTC): 1066587856
**********************************************************************
Row data format
Transformation || Transformation Mapplet Name || Transformation Group || Partition
Index || Transformation Row ID || Error Sequence || Error Timestamp || Error UTC
Time || Error Code || Error Message || Error Type || Transformation Data || Source
Mapplet Name || Source Name || Source Row ID || Source Row Type || Source Data
252
Log Source Row Data 1
253
Log Source Row Data 2
Source row logging is not available downstream of an
Aggregator, Joiner, Sorter (where output rows are not
uniquely correlated with input rows)
Source row logging Source row logging
available not available
254
Workflow Configuration
Workflow Configuration Objectives
Reusable Schedules
256
Workflow Server Connections
257
Workflow Server Connections
Configure Server data access connections in the Workflow Manager
Used in Session Tasks
(Native Databases)
(MQ Series)
(File Transfer Protocol file)
(Custom)
(External Database Loaders)
258
Relational Connections (Native )
Create a relational [database] connection
− Instructions to the Server to locate relational tables
− Used in Session Tasks
259
Relational Connection Properties
Define native
relational database
connection
User Name/Password
Database connectivity
information
Rollback Segment
assignment (optional)
260
FTP Connection
Create an FTP connection
− Instructions to the Server to ftp flat files
− Used in Session Tasks
261
External Loader Connection
Create an External Loader connection
− Instructs the Server to invoke an external database loader
− Used in Session Tasks
262
Reusable Workflow Schedules
263
Reusable Workflow Schedules
Set up reusable schedules to associate with multiple Workflows
− Defined at folder level
− Must have the Workflow Designer tool open
264
Reusable Workflow Schedules
265
Reusable Session Configurations
266
Session Configuration
267
Session Configuration (cont’d)
268
Session Configuration (cont’d)
269
Session Task – Config Object
270
Session Task – Config Object Attributes
Attributes
may be
overridden
within the
Session task
271
Worklets
Worklets
Worklets
Node
Tasks in a Worklet
274
Using a Reusable Worklet in a Workflow
Worklet
used in a
Workflo
w
275
Non-Reusable Worklet
• Create worklet task in
Workflow Designer
• Workspace switches to
Worklet Designer
NOTE: Worklet
shows only under
Workflows node
276
Lab 21 – Reusable Worklet and Decision Task
277
Lab 22 – Event Wait with Pre-Defined Event
278
Lab 23 – User-Defined Event, Event Raise,
and Event Wait
279
Parameters and Variables
Review
280
Types of Parameters and Variables
TYPE HOW DEFINED WHERE USED EXAMPLES
Mapping/ Mapping/mapplet Transformation port $$LastUpdateTime
Mapplet properties. Reset by expressions $$MaxValue
Variables variable functions.
Mappings Mapping/ Mapping/mapplet Transformation port $$FixedCosts
& Mapplets Mapplet properties. Constant for expressions $$DiscountRate
Parameters session.
System Built-in, pre-defined. Transformation port SYSDATE
Variables expressions, SESSSTARTIME
Workflow decision WORKFLOWSTARTTIME
tasks and conditional
links.
Task Built-in, pre-defined. Workflow decision $session1. Status
Workflows Variables tasks and conditional $session1.ErrorCode
& Worklets links
Workflow/ Workflow or worklet Workflow decision $$NewStartTime
Worklet properties. Reset in tasks and conditional
Variables Assignment tasks. links
Session Parameter file. Session properties $DBConnectionORCL
Parameters Constant for session. $InputFile1
281
PowerCenter 7.1 Options and
Data Access Products
282
PowerCenter 7.1 Options
Allows export/import of metadata to or from business
Metadata Exchange with BI intelligence tools like Business Objects and Cognos
Profile wizards, rules definitions, profile results tables,
Data Profiling and standard reports
283
Virtual Classes
284
Data Access – PowerExchange
Provides access to all critical enterprise data systems,
including mainframe, midrange relational databases,
and file-based systems
Offers batch, change capture and real-time options
285
Data Access – PowerCenter Connect
286
287