Welcome to Scribd, the world's digital library. Read, publish, and share books and documents. See more
Download
Standard view
Full view
of .
Look up keyword or section
Like this
4Activity

Table Of Contents

Data Integrator Designer Guide
Welcome
Overview of this document
Audience and assumptions
More Data Integrator product documentation 1
More Data Integrator product documentation
About this chapter
Creating a Data Integrator repository
Associating the repository with a Job Server
Entering repository information
Version restrictions
Oracle login
Microsoft SQL Server login
IBM DB2 login
Sybase ASE login
Resetting users
Data Integrator objects
Reusable objects
Single-use objects
Object hierarchy
Designer window
Menu bar
Project menu
Edit menu
View menu
Tools menu
Debug menu
Validation menu
Window menu
Toolbar 3
Help menu
Toolbar
Project area 3
Project area
Tool palette
Workspace
Moving objects in the workspace area
Viewing and changing object properties
Creating descriptions
Creating annotations
Saving and deleting objects
Searching for objects
General and environment options
Designer — Environment
Designer — General
Designer — Graphics
Designer — Central Repository Connections
Data — General
Job Server — Environment
Job Server — General
Projects
Objects that make up a project
Creating new projects
Opening existing projects
Saving projects
Jobs
Creating jobs
Naming conventions for objects in jobs
Datastores
What are datastores?
Database datastores
Mainframe interface
Defining a database datastore
Changing a datastore definition
Browsing metadata through a database datastore
Importing metadata through a database datastore
Imported table information
Imported stored function and procedure information
Ways of importing metadata
Reimporting objects
Memory datastores
Memory table target options
Persistent cache datastores
Linked datastores
Adapter datastores
DB1
Defining an adapter datastore
Browsing metadata through an adapter datastore
Importing metadata through an adapter datastore
Creating and managing multiple datastore configurations
Definitions
Why use multiple datastore configurations?
Creating a new configuration
Adding a datastore alias
Portability solutions
Migration between environments
Multiple instances
OEM deployment
Multi-user development
Job portability tips
Renaming table and function owner
Defining a system configuration
What are file formats?
File format editor 6
File format editor
Creating file formats
Creating a new file format
Modeling a file format on a sample file
Replicating and renaming file formats
Creating a file format from an existing flat table schema
Editing file formats
File format features
Reading multiple files at one time
Identifying source file names
Number formats
Ignoring rows with specified markers
Date formats at the field level
Error handling for flat-file sources
WL_GetKeyValue function
chapter
Data Flows
What is a data flow?
Naming data flows
Data flow example
Steps in a data flow
Data flows as steps in work flows
Intermediate data sets in a data flow
Passing parameters to data flows 7
Operation codes
Passing parameters to data flows
Creating and defining data flows
Source and target objects
Source objects
Target objects
Adding source or target objects to data flows
Template tables
Transforms
Transform editors
Adding transforms to data flows
Query transform overview
Adding a Query transform to a data flow
Query editor
Data flow execution
Push down operations to the database server
Distributed data flow execution
Load balancing
Caches
Audit Data Flow Overview
What is a work flow?
Steps in a work flow
Order of execution in work flows 8
Order of execution in work flows
Example of a work flow
Creating work flows 8
Creating work flows
Conditionals
While loops
Design considerations
Defining a while loop
Using a while loop with View Data
Try/catch blocks
Categories of available exceptions
Scripts
Debugging scripts using the print function
Nested Data
What is nested data?
Representing hierarchical data 9
Representing hierarchical data
Formatting XML documents
Importing XML Schemas
Importing XML schemas
Importing abstract types
Importing substitution groups
Specifying source options for XML files
Reading multiple XML files at one time
Mapping optional schemas
Using Document Type Definitions (DTDs)
Generating DTDs and XML Schemas from an NRDM schema
Operations on nested data
Overview of nested data and the Query transform
FROM clause construction
Nesting columns
Using correlated columns in nested data
Distinct rows and nested data
Grouping values across nested schemas
Unnesting nested data
How transforms handle nested data
XML extraction and parsing for columns
Sample Scenarios
Overview
Request-response message processing
What is a real-time job?
Real-time versus batch
Messages
Real-time job examples
Creating real-time jobs
Real-time job models
Single data flow model
Multiple data flow model
Using real-time job models
Creating a real-time job
Real-time source and target objects
Secondary sources and targets
Transactional loading of tables
Design tips for data flows in real-time jobs
Testing real-time jobs
Executing a real-time job in test mode
Using an XML file target
Building blocks for real-time jobs
Supplementing message data
Branching data flow based on a data cache value
Calling application functions
Designing real-time applications 10
Designing real-time applications
Reducing queries requiring back-office application access
Messages from real-time jobs to adapter instances
Real-time service invoked by an adapter instance
Example of when to use embedded data flows 11
Example of when to use embedded data flows
Creating embedded data flows
Using the Make Embedded Data Flow option
Creating embedded data flows from existing flows
Using embedded data flows
Testing embedded data flows
Troubleshooting embedded data flows
The Variables and Parameters window
The Variables and Parameters window opens
Using local variables and parameters
Parameters
Passing values into data flows
Defining local variables
Defining parameters
Using global variables
Creating global variables
Viewing global variables
Setting global variable values
Local and global variable rules 12
Local and global variable rules
Naming
Replicating jobs and work flows
Importing and exporting
Environment variables
Setting file names at run-time using variables
Overview of Data Integrator job execution
Preparing for job execution 13
Preparing for job execution
Validating jobs and job components
Ensuring that the Job Server is running
Setting job execution options
Executing jobs as immediate tasks
Monitor tab
Log tab
Debugging execution errors
Using Data Integrator logs
Examining trace logs
Examining monitor logs
Examining error logs
Examining target data
Changing Job Server options
Chapter overview
Using the Data Profiler
Data sources that you can profile
Connecting to the profiler server
Profiler statistics
Column profile
Basic profiling
Detailed profiling
Relationship profile
Executing a profiler task
Submitting column profiler tasks
Submitting relationship profiler tasks
Monitoring profiler tasks using the Designer
Viewing the profiler results
Viewing column profile data
Viewing relationship profile data
Using View Data to determine data quality
Data tab
Profile tab
Relationship Profile or Column Profile tab
Using the Validation transform
Analyze column profile
Define validation rule based on column profile
Using Auditing
Auditing objects in a data flow
Accessing the Audit window
Defining audit points, rules, and action on failure
Guidelines to choose audit points
Auditing embedded data flows
Enabling auditing in an embedded data flow
Audit points not visible outside of the embedded data flow
Resolving invalid audit labels
Viewing audit results
Job Monitor Log
Job Error Log
Metadata Reports
Data Cleansing with Data Integrator Data Quality
Overview of Data Integrator Data Quality architecture
Overview of steps to use Data Integrator Data Quality 14
Data Quality Terms and Definitions
Overview of steps to use Data Integrator Data Quality
Creating a Data Quality datastore
Importing Data Quality Projects 14
Importing Data Quality Projects
Using the Data Quality transform
Mapping input fields from the data flow to the project
Creating custom projects
Data Quality blueprints for Data Integrator
Using View Where Used
From the object library
From the workspace
Using View Data
Accessing View Data
Viewing data in the workspace
View Data properties
Filtering
Sorting
View Data tool bar options
View Data tabs
Column Profile tab
Using the interactive debugger
Before starting the interactive debugger
Changing the interactive debugger port
Starting and stopping the interactive debugger
Windows
Filters and Breakpoints window
Menu options and tool bar
Viewing data passed by transforms
Push-down optimizer
Comparing Objects
Overview of the Difference Viewer window
To change the color scheme
Navigating through differences
Calculating usage dependencies 15
Calculating usage dependencies
Metadata exchange
Importing metadata files into Data Integrator
Exporting metadata files from Data Integrator
Creating Business Objects universes 16
Creating Business Objects universes
Mappings between repository and universe metadata
Attributes that support metadata exchange 16
Attributes that support metadata exchange
Recovery Mechanisms
Recovering from unsuccessful job execution
Automatically recovering jobs
Enabling automated recovery
Marking recovery units
Running in recovery mode
Ensuring proper execution path
Using try/catch blocks with automatic recovery
Ensuring that data is not duplicated in targets
Using preload SQL to allow re-executable data flows
Manually recovering jobs using status tables
Processing data with problems
Using overflow files
Filtering missing or bad values
Overview of CDC for Oracle databases
Setting up Oracle CDC
CDC datastores
Importing CDC data from Oracle
Viewing an imported CDC table
Configuring an Oracle CDC source
Creating a data flow with an Oracle CDC source
Maintaining CDC tables and subscriptions
Limitations
Using CDC with DB2 sources
Guaranteed delivery
Setting up DB2
Setting up Data Integrator
CDC Services
Importing CDC data from DB2
Configuring a DB2 CDC source
Using CDC with Attunity mainframe sources
Setting up Attunity CDC
Importing mainframe CDC data
Configuring a mainframe CDC source
Using mainframe check-points
Using CDC with Microsoft SQL Server databases
Overview of CDC for SQL Server databases
Setting up SQL Replication Server for CDC
Overlaps
Overlap avoidance
Overlap reconciliation
Presampling
Types of timestamps
Create-only timestamps
Update-only timestamps
Create and update timestamps
Timestamp-based CDC examples
Preserving generated keys
Using the lookup function
Comparing tables
Preserving history
Additional job design tips
Header and detail synchronization
Using CDC for targets 18
Capturing physical deletions
Using CDC for targets
Administrator
SNMP support
About the Data Integrator SNMP agent
Job Server, SNMP agent, and NMS application architecture
About SNMP Agent’s Management Information Base (MIB)
About an NMS application
Configuring Data Integrator to support an NMS application
SNMP configuration parameters
Job Servers for SNMP
System Variables
Access Control, v1/v2c
Access Control, v3
Traps
Troubleshooting
Index
0 of .
Results for:
No results containing your search query
P. 1
Di Designer Guide

Di Designer Guide

Ratings: (0)|Views: 500|Likes:
Published by Massimo Bellucci

More info:

Published by: Massimo Bellucci on Aug 30, 2010
Copyright:Attribution Non-commercial

Availability:

Read on Scribd mobile: iPhone, iPad and Android.
download as PDF, TXT or read online from Scribd
See more
See less

09/21/2012

pdf

text

original

You're Reading a Free Preview
Pages 12 to 44 are not shown in this preview.
You're Reading a Free Preview
Pages 56 to 155 are not shown in this preview.
You're Reading a Free Preview
Pages 167 to 467 are not shown in this preview.
You're Reading a Free Preview
Pages 479 to 516 are not shown in this preview.
You're Reading a Free Preview
Pages 528 to 583 are not shown in this preview.

Activity (4)

You've already reviewed this. Edit your review.
1 hundred reads
1 thousand reads
Johnny Zhong liked this
vipin.s.tomar liked this

You're Reading a Free Preview

Download
scribd
/*********** DO NOT ALTER ANYTHING BELOW THIS LINE ! ************/ var s_code=s.t();if(s_code)document.write(s_code)//-->