Intersecting with Other Information Server Tools

DataStage Essentials v8.5

Copyright IBM Corporation 2011 Course materials may not be reproduced in whole or in part without the prior written permission of IBM.

5.4

Unit objectives
After completing this unit, you should be able to: Intersect with FastTrack Intersect with Metadata Workbench

Copyright IBM Corporation 2011

FastTrack

Copyright IBM Corporation 2011

What is FastTrack?
Collaboration tool
Designed for business analysts whose job it is to translate business requirements into technical specifications for developers

The business analysts deliverable is a Mapping Specification for an ETL (for example, DataStage) job Prototype DataStage jobs can be generated from these Mapping Specifications
Prototype because it is expected that a DataStage developer will need to complete and/or edit the generated jobs

DataStage jobs can be reverse engineered

Mapping specifications can be generated from DataStage jobs
Provide documentation

Copyright IBM Corporation 2011

Mapping specifications
Mapping specifications:
Define mappings between source and target columns Filter data from source tables
Select a row from a source table if and only if it satisfies a specified condition
For example, Customer area is West coast

Joins
Merge columns from multiple source rows into a single output row

Lookups Switches
Choose between alternate targets

For each target column, specify a value to map to it

Objects you can map to target columns include:
Source columns Business rules Functions Lookup return values
Copyright IBM Corporation 2011

Inside FastTrack: Mapping specification

Source to target mappings Shared Repository Metadata

Copyright IBM Corporation 2011

Creating a filtering constraint

Constraint

Copyright IBM Corporation 2011

Switch constraints

Switch constraints

Target links

Copyright IBM Corporation 2011

Generated DataStage job

Source

Targets

Filtering constraints, mappings, and derivations

Copyright IBM Corporation 2011

Switch constraints

Inside the Filtering Transformer

Implements:
Column mappings Derivations Filtering constraint
Constraint

Column mappings and derivations

Copyright IBM Corporation 2011

Inside the Switch Transformer

Constraint

Constraint

Copyright IBM Corporation 2011

Generating mapping specifications from DataStage jobs

Mappings specifications can be generated from DataStage jobs:
Use to capture changes made to previously generated DataStage jobs back into the Mapping Specification Use to document existing DataStage jobs

Restrictions:
Not all DataStage designs are supported The metadata of sources and targets of existing DataStage jobs must be captured into the shared Information Server Repository
By default, DataStage Table Definitions are local to DataStage

Copyright IBM Corporation 2011

Creating a shared table for a relational table

Table definition Shared Table Creation Wizard

Specify table metadata

Copyright IBM Corporation 2011

Creating a shared table for a sequential file

Table definition Shared Table Creation Wizard

Specify table metadata

Copyright IBM Corporation 2011

Metadata Workbench

Copyright IBM Corporation 2011

Workbench feature overview

ADMINISTER
Manage Integration Assets to enable in-depth analysis: Assign security roles Link together multiple viewpoints of design assets from ETL, business, BI and modeling with operational metadata Edit names and descriptions of Integration Assets Access runtime information to enrich reporting Import export manager for 3rd party integration

EXPLORE
Explore key Integration Assets: Jobs, Reports, Databases, Models, Terms, Stewards, Systems, Specifications, Quality Rules Easy navigation of key Integration Assets Simple and advanced search Integrated cross-view of Information Server and 3rd party assets Graphical view of Asset Relationships Robust query builder

ANALYZE
Analyze dependencies and relationships between key Integration assets, Business Intelligence Reports and data models: Trace data movement to and from databases, jobs and reports for full lineage Understand business meaning of columns, tables, and other assets Assess the impact of change across Integration assets Extend analyses to business assets outside of IS

Copyright IBM Corporation 2011

Exploring in Workbench
Explore metadata assets including:
Jobs, reports, databases, files, tables, columns, terms, stewards, servers

Simple and advanced search capabilities Robust query capabilities

Return repeatable sets of information

Multiple ways to search:

By asset class, name, property

Save results in reports

Various formats are supported

View graphs of asset relationships Create and edit descriptions of assets

Copyright IBM Corporation 2011

Data Lineage and Impact Analysis

Shows the flow of data through a series of assets Types of flow:
Where does the data go from a selected asset? Where does the data come from?

Can depict the flow through different types of assets

For example, DataStage job to a database table For example, from a BI report to the true data source

Captured relationships include:

Operational data
Operational lineage shows the order of activities within a run of a job

Design data User-defined data

You can save in reports with text and graphical format

Copyright IBM Corporation 2011

Extended Data Mappings

Allows a business to extend the trace and document data flow beyond the Information Server DataStage jobs and other assets
Extend to source applications that flow into DataStage data flows Extend to target applications that DataStage data flows into

Enables Search and Analysis to cover other business assets that intersect with Information Server assets Business Lineage
Applies to a subset of assets of interest to business analysts Administrators can exclude assets that business analysts arent interested in Business names of the assets can be provided

Copyright IBM Corporation 2011

Opening Metadata Workbench

Accessed by means of a Web Browser:
Internet Explorer Firefox

Browser URL: http://edserver:9080/workbench

IS host name followed by port number and /workbench

Log in as a Metadata Workbench User or Administrator:

Assigned by an IS administrator in the IS Web Console Workbench Administrator can access all Workbench functionality including, for example, the Automated Services for associating IS assets Workbench User access limited to Explore and Analysis Workbench functionality

Copyright IBM Corporation 2011

Metadata Workbench Home

Display assets

Search for assets

Query for assets

Copyright IBM Corporation 2011

Exploring Metadata Assets

View job

Drill down to job assets

Analyze

Copyright IBM Corporation 2011

Data lineage: Column level

Click Book icon for info about column Database column

Top shows both the design and operational flow of the data through stage columns across jobs Bottom shows flow into a Database Table column
Copyright IBM Corporation 2011

Impact Analysis dependency path

Dependent assets: Jobs, Tables

Click i for information

Blue line (operational metadata) Orange line (design metadata)

Copyright IBM Corporation 2011

Information about dependent assets

Checkpoint
1. What is a mapping specification? 2. What are mapping specifications used for? 3. What functionality does Metadata Workbench provide for exploring metadata assets? 4. What functionality does Metadata Workbench provide for analysis of metadata assets?

Copyright IBM Corporation 2011

Checkpoint solutions
1. What is a mapping specification?
A mapping specification provides the specifications for an ETL job, for example, a DataStage job. It includes column mappings, derivations, constraints, and source and target specifications.

2. What are mapping specifications used for?

Document a DataStage job. DataStage jobs can be generated from them.

3. What functionality does Metadata Workbench provide for exploring metadata assets?
Display metadata assets and drill down into their component assets. Search and Query assets.

4. What functionality does Metadata Workbench provide for analysis of metadata assets?
Impact analysis and data lineage at both the design and operational level.
Copyright IBM Corporation 2011

Unit summary
Having completed this unit, you should be able to: Intersect with FastTrack Intersect with Metadata Workbench

Copyright IBM Corporation 2011