Release 3i June 15, 2001 Part No. A90361-01 Oracle Warehouse Builder User s Guide, Release 3i Part No. A90361-01 Copyright 1996, 2001, Oracle Corporation. All rights reserved. Authors: Patrick Terhune, Shirinne Alison Contributing Authors: Kavita Nayar, Yu Gong Contributors: Thomas Lau, Edwin Meijer, Jooseph Zheng, Barry Mosbrucher, Thomas OShaughnassy, Dan Gallagher, Josephine Ho, David Allen, Krishna Behara, Gary Hostetler, JaimeSingson, Robert Velisar, Robert Paul, Joseph Klein, Josef Hasenberger, Jim Lorenz, Jean-Pierre Djicks, Olaf Fermum, Shauna OBoyle, Jeffrey Hutchins, Debra Phelan, Elina Sternik, John Potter, Barry Mosbrucher, Adrian Scott, Ning Lin, Lefty Leverenz, James Jonas, Brian Jeffries, Arun Manchanda, Richard Whittington, Ramesh Uppala. The Programs (which include both the software and documentation) contain proprietary information of Oracle Corporation; they are provided under a license agreement containing restrictions on use and disclosure and are also protected by copyright, patent, and other intellectual and industrial property laws. Reverse engineering, disassembly, or decompilation of the Programs is prohibited. The information contained in this document is subject to change without notice. If you find any problems in the documentation, please report them to us in writing. Oracle Corporation does not warrant that this document is error free. Except as may be expressly permitted in your license agreement for these Programs, no part of these Programs may be reproduced or transmitted in any form or by any means, electronic or mechanical, for any purpose, without the express written permission of Oracle Corporation. If the Programs are delivered to the U.S. Government or anyone licensing or using the programs on behalf of the U.S. Government, the following notice is applicable: Restricted Rights Notice Programs delivered subject to the DOD FAR Supplement are "commercial computer software" and use, duplication, and disclosure of the Programs, including documentation, shall be subject to the licensing restrictions set forth in the applicable Oracle license agreement. Otherwise, Programs delivered subject to the Federal Acquisition Regulations are "restricted computer software" and use, duplication, and disclosure of the Programs shall be subject to the restrictions in FAR 52.227-19, Commercial Computer Software - Restricted Rights (June, 1987). Oracle Corporation, 500 Oracle Parkway, Redwood City, CA 94065. The Programs are not intended for use in any nuclear, aviation, mass transit, medical, or other inherently dangerous applications. It shall be the licensee's responsibility to take all appropriate fail-safe, backup, redundancy, and other measures to ensure the safe use of such applications if the Programs are used for such purposes, and Oracle Corporation disclaims liability for any damages caused by such use of the Programs. Oracle is a registered trademark, and of Oracle Corporation. Other names may be trademarks of their respective owners. iii Contents Send Us Your Comments................................................................................................................. xix Preface.......................................................................................................................................................... xxi 1 Overview The Developmental Phases............................................................................................................... 1-2 The Definition Phase .................................................................................................................... 1-3 Defining A Warehouse ......................................................................................................... 1-3 Defining The Data Sources and Targets............................................................................. 1-3 Defining ETL Mappings ....................................................................................................... 1-4 The Generation Phase .................................................................................................................. 1-5 The Load And Manage Phase..................................................................................................... 1-6 Taking The Next Step......................................................................................................................... 1-6 2 Getting Started on a Project Understanding The OWB Project .................................................................................................... 2-2 About Warehouse Builder Projects ............................................................................................ 2-2 Module Contents ................................................................................................................... 2-3 Object Names ......................................................................................................................... 2-3 About Multiple-User Access ....................................................................................................... 2-4 Read/ Write Mode ................................................................................................................. 2-4 Read Only Mode.................................................................................................................... 2-5 Lock Synchronization ........................................................................................................... 2-6 Creating A Project......................................................................................................................... 2-7 iv Logging On To A Repository...................................................................................................... 2-8 Using The New Project Wizard ................................................................................................ 2-10 Commit ................................................................................................................................. 2-10 About The OWB Console................................................................................................................ 2-11 Operational Environments ........................................................................................................ 2-12 Project Environment ........................................................................................................... 2-12 Transformation Environment ............................................................................................ 2-14 Administration Environment ............................................................................................ 2-15 The Utilities Button .................................................................................................................... 2-15 Utility Property Sheet ......................................................................................................... 2-16 The Console Tool Bar ................................................................................................................. 2-18 Property Icon and Property Sheets ................................................................................... 2-18 The Commit Icon ................................................................................................................. 2-19 About Naming Policies.................................................................................................................... 2-20 Setting The Naming Mode ........................................................................................................ 2-21 Logical Name Mode............................................................................................................ 2-21 Physical Name Mode .......................................................................................................... 2-23 Using The Warehouse Builder Wizards ....................................................................................... 2-23 Using Navigation and Help ............................................................................................... 2-24 Searching Navigation Trees ............................................................................................................ 2-25 Taking The Next Step....................................................................................................................... 2-27 3 The Target Modules About Star Schemas ........................................................................................................................... 3-2 About Dimension Tables ............................................................................................................. 3-2 About Fact Tables ......................................................................................................................... 3-3 Linking Facts to Dimensions ............................................................................................... 3-3 Physical Characteristics ........................................................................................................ 3-3 About Materialized Views .......................................................................................................... 3-4 Requirements for Query Rewrite ........................................................................................ 3-4 About Conventional Views ......................................................................................................... 3-5 Creating a Warehouse Module ......................................................................................................... 3-5 Displaying the Warehouse Module ......................................................................................... 3-10 Verifying the Module.......................................................................................................... 3-10 Inspecting the Module ........................................................................................................ 3-10 v Creating Dimension Definitions ................................................................................................... 3-11 Defining Rules for Dimension Objects .................................................................................... 3-12 About Levels and Hierarchies ........................................................................................... 3-12 About Unique Key Constraints ......................................................................................... 3-12 About Mixed Levels of Aggregation ................................................................................ 3-13 Creating A Dimension Definition ............................................................................................ 3-14 Updating Dimension Definitions ............................................................................................. 3-21 Using The Dimension Editor ............................................................................................. 3-21 Using The Property Sheets................................................................................................. 3-22 Creating Attribute Sets ....................................................................................................... 3-29 Using the New Time Dimension Wizard ................................................................................ 3-33 About the New Time Dimension Wizard ........................................................................ 3-34 Creating a Time Dimension Definition ................................................................................... 3-34 Defining a Sequence Object ....................................................................................................... 3-37 Creating Fact Table Definitions ..................................................................................................... 3-37 Creating a Definition for a Fact Table...................................................................................... 3-38 Updating a Fact Table Definition ............................................................................................. 3-41 Using the Fact Editor .......................................................................................................... 3-41 Importing Definitions ................................................................................................................ 3-45 Creating Materialized View Definitions...................................................................................... 3-46 Creating A Materialized View Definition ............................................................................... 3-47 Updating a Materialized View Definition .............................................................................. 3-51 Renaming a Materialized View ......................................................................................... 3-51 Creating Conventional View Definitions .................................................................................... 3-52 Creating A View Definition ...................................................................................................... 3-52 Updating a View Definition ...................................................................................................... 3-56 Renaming A View ............................................................................................................... 3-57 Adding Transformations ................................................................................................................. 3-58 About Transformations ............................................................................................................ 3-58 About Transformation Parameters .......................................................................................... 3-59 About Oracle Transformation Libraries .................................................................................. 3-59 The Global-Shared Library ................................................................................................ 3-60 The Oracle Library .............................................................................................................. 3-60 Accessing Transformation Libraries................................................................................. 3-60 Creating Transformation Categories ................................................................................ 3-60 vi Defining Custom Transforms ................................................................................................... 3-61 Editing Transformation Properties .......................................................................................... 3-64 Importing PL/ SQL Packages.................................................................................................... 3-64 Defining Business Areas ................................................................................................................. 3-69 About Business Areas ................................................................................................................ 3-70 Creating a Definition for a Business Area............................................................................... 3-71 Infrastructure Requirements .......................................................................................................... 3-73 Committing Your Changes.............................................................................................................. 3-74 Taking the Next Step ........................................................................................................................ 3-74 4 The Data Source Modules Creating Source Modules .................................................................................................................. 4-2 About OWB Software Integrators .............................................................................................. 4-2 Configuring Connection Information for Database Sources.................................................. 4-2 Using The New Database Link Dialog ............................................................................... 4-4 Creating A Database Link Pointing to an Oracle System................................................ 4-6 Creating A Link Pointing to an IBM DB2 System ............................................................ 4-6 Creating A Source Module for Database Definitions.............................................................. 4-7 Updating A Source Module Definition ................................................................................... 4-11 Creating A Source Module for an Oracle Designer Repository .......................................... 4-12 Creating A Source Module for Flat Files ................................................................................. 4-13 Updating A Flat File Source Module ....................................................................................... 4-16 Importing Definitions From Database Sources .......................................................................... 4-16 Importing Definitions from an Oracle Database.................................................................... 4-17 Re-Importing Definitions from an Oracle Database.............................................................. 4-21 Updating Oracle Database Source Definitions....................................................................... 4-25 Creating Definitions for Flat File Sources ................................................................................... 4-28 Using The Flat File Sample Wizard ......................................................................................... 4-29 Creating a Definition for a Fixed-Length File ........................................................................ 4-31 Using The Source Module Editor ...................................................................................... 4-36 Defining Multiple Records in a Fixed-Length File ......................................................... 4-36 Updating a Fixed-Length File Definition ................................................................................ 4-39 Updating a Multiple-Record Fixed-Length Definition .................................................. 4-40 Specifying Logical Records ................................................................................................ 4-41 Creating a Definition for a Delimited File .............................................................................. 4-41 vii Defining Multiple Record Types in a Delimited File ..................................................... 4-44 Updating a Delimited File Definition ...................................................................................... 4-47 Updating a Single-Record Delimited File Definition ..................................................... 4-47 Updating a Multiple-Record Delimited File Definition................................................. 4-47 Importing Definitions for Pure Extract and Pure Integrate...................................................... 4-49 Importing Definitions from an Export File ............................................................................. 4-49 Infrastructure Requirements .......................................................................................................... 4-51 Oracle Heterogeneous Services ................................................................................................ 4-51 Transparent Gateway ......................................................................................................... 4-51 Generic Connectivity .......................................................................................................... 4-51 Flat File Sources .......................................................................................................................... 4-52 Committing Your Changes.............................................................................................................. 4-52 Taking The Next Step....................................................................................................................... 4-53 5 The Source to Target Mappings About Mappings ................................................................................................................................. 5-2 About Rows ................................................................................................................................... 5-2 About Row-Sets ............................................................................................................................ 5-2 About Operators ........................................................................................................................... 5-3 About Operator Properties.......................................................................................................... 5-3 About Attributes And Attribute Groups .................................................................................. 5-4 About Display Sets ....................................................................................................................... 5-4 About Binding Mapping Operators........................................................................................... 5-5 About Mapping Operator Names .............................................................................................. 5-6 About Reconciliation.................................................................................................................... 5-6 When To Use Inbound Reconciliation................................................................................ 5-7 When to use Outbound Reconciliation .............................................................................. 5-8 About The Mapping Editor ......................................................................................................... 5-8 About Expressions...................................................................................................................... 5-12 About Mapping Generation ...................................................................................................... 5-13 Defining Mappings .......................................................................................................................... 5-13 Creating a Mapping Warehouse Object .................................................................................. 5-14 Selecting A Source Operator ..................................................................................................... 5-17 Selecting a Target Operator....................................................................................................... 5-20 Selecting A Data Flow Operator ............................................................................................... 5-23 viii Defining The Data Flow Connections ..................................................................................... 5-24 Configuring A Mapping ............................................................................................................ 5-26 Setting The Load Type........................................................................................................ 5-26 Configuring Physical Properties For a Mapping ............................................................ 5-28 Validating the Mapping............................................................................................................. 5-29 Generating the Mapping ................................................................................................................. 5-31 Editing Mapping Operator Attributes .......................................................................................... 5-33 Adding or Removing Operator Attribute Groups................................................................. 5-33 Editing Operator Attributes ...................................................................................................... 5-33 Adding Or Removing Attributes ...................................................................................... 5-34 Renaming Attributes........................................................................................................... 5-35 Creating New Attributes ........................................................................................................... 5-35 Reconciling Your Mapping ....................................................................................................... 5-36 Reconciling Inbound Operators ........................................................................................ 5-36 Reconciling Outbound Operators ..................................................................................... 5-37 Committing Changes To The Mapping ................................................................................... 5-39 Taking The Next Step....................................................................................................................... 5-40 6 Mapping Operators and Transformations Using Expression Builder.................................................................................................................. 6-2 About Expression Builder ........................................................................................................... 6-2 Creating Expression Components.............................................................................................. 6-4 Adding Data Flow Operators To A Mapping................................................................................ 6-6 Filtering Data................................................................................................................................. 6-6 Ordering Data ............................................................................................................................. 6-10 Splitting Data............................................................................................................................... 6-14 De-Duplicating Data .................................................................................................................. 6-19 Aggregating Data ....................................................................................................................... 6-21 Joining Data Sources .................................................................................................................. 6-25 Adding Sequences ...................................................................................................................... 6-30 Adding External Processes........................................................................................................ 6-34 Adding Data Generators ........................................................................................................... 6-36 Setting a Column to the Datafile Record Number.......................................................... 6-37 Setting a Column to the Current Date.............................................................................. 6-37 Setting a Column to a Unique Sequence Number .......................................................... 6-37 ix Adding Expressions ................................................................................................................... 6-40 Adding Constants....................................................................................................................... 6-44 Adding Transformations ........................................................................................................... 6-49 Adding Mapping Input Parameters ........................................................................................ 6-54 Adding Mapping Output Parameters ..................................................................................... 6-59 Adding A Pre-Mapping Process .............................................................................................. 6-63 Adding A Post-Mapping Process............................................................................................. 6-68 Taking The Next Step....................................................................................................................... 6-71 7 Mapping Configuration and Code Generation Reconciling Mapping Operators With Repository Objects ....................................................... 7-2 Reconciling Mappings ................................................................................................................. 7-2 More About Naming Mapping Objects ............................................................................. 7-3 Using Inbound Reconciliation ............................................................................................. 7-5 Inbound Reconciliation Strategies ...................................................................................... 7-5 Using Outbound Reconciliation ........................................................................................ 7-10 Configuring The Mapping For Deployment............................................................................... 7-13 Configuring Attribute Properties ............................................................................................. 7-13 Delete: Use For Matching ................................................................................................... 7-14 Insert: Use For Loading ...................................................................................................... 7-14 Update: Operation............................................................................................................... 7-14 Update: Use For Matching ................................................................................................. 7-15 Update: Use For Loading ................................................................................................... 7-15 Configuring The Mapping Operator Physical Properties .................................................... 7-15 Setting The Database Link or Schema Information ........................................................ 7-17 Setting Partition Exchange Loading ................................................................................. 7-18 Adding Optimizing Hints .................................................................................................. 7-22 Setting Constraint Handling.............................................................................................. 7-24 Defining Code Generation Strategies For The Mapping ......................................................... 7-24 Viewing The Generated Code for a Mapping ........................................................................ 7-24 Using the Code Viewer....................................................................................................... 7-26 Configuring Loading Step Strategies....................................................................................... 7-27 Setting The Default Audit Level ....................................................................................... 7-28 Setting the Default Purge Group....................................................................................... 7-28 Setting Maximum Number of Errors ............................................................................... 7-28 x Setting Commit Frequency ................................................................................................ 7-28 Setting The Default Operating Mode ............................................................................... 7-28 Setting Bulk Processing ...................................................................................................... 7-30 Setting the Analyze Statistics Percentage ........................................................................ 7-30 Setting A SQL*Loader Step ............................................................................................................ 7-30 Setting Load Operator Physical Parameters ........................................................................... 7-32 Setting Enable Constraints ................................................................................................. 7-32 Setting The Exceptions Table............................................................................................. 7-32 Setting SQL*Loader Parameters ........................................................................................ 7-32 Setting Step Parameters ............................................................................................................. 7-34 Setting An External Process Step................................................................................................... 7-37 Setting OS Executable/ Pure Integrate..................................................................................... 7-37 Setting Pure Extract .................................................................................................................... 7-37 Taking The Next Step....................................................................................................................... 7-37 8 Configure, Generate and Deploy Configuring a Physical Instance ...................................................................................................... 8-2 Configuring Existing Logical Objects ........................................................................................ 8-3 Configuring New Physical Objects ............................................................................................ 8-4 Configuring Warehouse Modules.............................................................................................. 8-7 Database Links ....................................................................................................................... 8-7 Generation Preferences......................................................................................................... 8-8 Oracle Enterprise Manager Registration............................................................................ 8-8 Run Time Directories ............................................................................................................ 8-9 Generation Target Directories ........................................................................................... 8-10 Identification ........................................................................................................................ 8-11 Configuring Dimensions ........................................................................................................... 8-11 Performance and Parallel Parameters .............................................................................. 8-12 Partition Parameters ........................................................................................................... 8-13 Storage Space ....................................................................................................................... 8-13 Generation Options for Time Dimension ........................................................................ 8-13 Generation Options ............................................................................................................. 8-13 Identification ........................................................................................................................ 8-14 Configuring Facts ....................................................................................................................... 8-14 Performance and Parallel Parameters .............................................................................. 8-15 xi Partition Parameters ........................................................................................................... 8-16 Storage Space ....................................................................................................................... 8-16 Identification ........................................................................................................................ 8-16 Configuring Materialized Views .............................................................................................. 8-16 Materialized View Parameters .......................................................................................... 8-18 Configuring Sequences .............................................................................................................. 8-18 Configuring Tables ..................................................................................................................... 8-19 Indexes .................................................................................................................................. 8-20 Partitions............................................................................................................................... 8-21 Configuring Views ..................................................................................................................... 8-23 Validating Definitions ..................................................................................................................... 8-24 Rationale for Validation............................................................................................................. 8-24 Valid and Invalid Definitions .................................................................................................. 8-25 Valid Definition ................................................................................................................... 8-25 Invalid Definition ................................................................................................................ 8-25 Validating a Set of Definitions .................................................................................................. 8-26 Generating Scripts ............................................................................................................................ 8-28 The Generated Scripts ................................................................................................................ 8-30 Database Links ..................................................................................................................... 8-30 Dimensions........................................................................................................................... 8-30 Views and Materialized Views.......................................................................................... 8-31 Mappings .............................................................................................................................. 8-32 Sequences.............................................................................................................................. 8-33 Tables .................................................................................................................................... 8-33 Post Generation Tasks................................................................................................................ 8-33 The Schema Objects Tab. .................................................................................................... 8-34 The Transforms Tab. ........................................................................................................... 8-35 Deploying Scripts ............................................................................................................................. 8-37 Upgrading a Warehouse Physical Instance.................................................................................. 8-41 OWB and the OEM Change Management Pack (OEM/ CM) .............................................. 8-42 Generating and Executing Upgrade Scripts .......................................................................... 8-43 Taking the Next Step ........................................................................................................................ 8-48 9 Periodic Loads and Updates Loading and Updating Jobs .............................................................................................................. 9-2 xii Scheduling Jobs ............................................................................................................................. 9-2 Types of Jobs .......................................................................................................................... 9-2 Scheduling with Oracle Enterprise Manager................................................................................ 9-3 Deploying and Registering the Jobs .......................................................................................... 9-4 Registering Job Scripts .......................................................................................................... 9-4 Verifying OEM Registration ................................................................................................ 9-5 Scheduling the Jobs ...................................................................................................................... 9-7 Scheduling .............................................................................................................................. 9-7 Submitting the Job ................................................................................................................. 9-7 Submitting Jobs with Parameters........................................................................................ 9-8 Managing Dependencies Using Oracle Workflow .................................................................... 9-10 Deploying to the Oracle Workflow Server ............................................................................. 9-11 Using the Workflow Deployment Wizard....................................................................... 9-11 Defining the Workflow Process................................................................................................ 9-17 Running the Process................................................................................................................... 9-20 Running the Process Immediately .................................................................................... 9-21 Scheduling the Workflow Process .................................................................................... 9-23 Viewing the Runtime Audit ........................................................................................................... 9-23 Setting the Audit Level ....................................................................................................... 9-24 Using the OWB Runtime Audit Viewer .................................................................................. 9-24 Viewing By Date Range...................................................................................................... 9-26 Searching the Navigation Tree .......................................................................................... 9-27 Refreshing the Audit Viewer ............................................................................................. 9-28 Purging Runtime Entries.................................................................................................... 9-28 Understanding the Audit Viewer ............................................................................................ 9-29 Viewing the Job Audit ........................................................................................................ 9-30 Viewing the Job Instance Audit ........................................................................................ 9-30 Viewing the Task Audit ..................................................................................................... 9-32 Viewing the Task Details Audit ........................................................................................ 9-33 Viewing the Errors .............................................................................................................. 9-36 Viewing the Error Details................................................................................................... 9-38 Taking the Next Step ....................................................................................................................... 9-38 10 Administration Repository Administration ............................................................................................................. 10-2 xiii The OWB Metadata Loader ............................................................................................................ 10-2 Metadata Export ......................................................................................................................... 10-3 The Export File..................................................................................................................... 10-3 Exporting Metadata ............................................................................................................ 10-5 Metadata Import ......................................................................................................................... 10-8 Importing Metadata .......................................................................................................... 10-11 Metadata Loader Command Line Utility.............................................................................. 10-17 Directive Keywords for the Export Utility .................................................................... 10-17 Export a Project .................................................................................................................. 10-19 Directive Keywords for the Import Utility .................................................................... 10-20 Import Selected Modules ................................................................................................. 10-21 Validation Rules Governing Import ............................................................................... 10-21 The OWB Transfer Wizard............................................................................................................ 10-22 The OWB Transfer Wizard Overview ................................................................................... 10-22 Transfer Considerations .......................................................................................................... 10-23 Importing Metadata into Oracle Warehouse Builder.......................................................... 10-23 Launching the OWB Transfer Wizard for Import ........................................................ 10-24 Importing Metadata using the OWB Transfer Wizard ................................................ 10-25 Exporting Metadata from Oracle Warehouse Builder ........................................................ 10-29 Launching the OWB Transfer Wizard for Export......................................................... 10-29 Exporting Metadata from OWB to a Target .................................................................. 10-31 The Archive/Restore Utility.......................................................................................................... 10-35 Setting up Preferences.............................................................................................................. 10-37 Archiving a Project ................................................................................................................... 10-39 Restoring a Project .................................................................................................................... 10-41 Reports Administration................................................................................................................. 10-45 Using the Administration Pages .................................................................................................. 10-45 Registering an OWB Repository.................................................................................................. 10-48 Administer Database Links..................................................................................................... 10-51 Unregistering a Repository ..................................................................................................... 10-54 User Access....................................................................................................................................... 10-54 Understanding Roles................................................................................................................ 10-54 Granting or Revoking Access ................................................................................................. 10-55 Assigning a Custom Report to a Role.................................................................................... 10-56 Purging Stale User Information.............................................................................................. 10-57 xiv Registering a Custom Report........................................................................................................ 10-57 11 OWB Metadata Reports Viewing OWB Reports..................................................................................................................... 11-1 Viewing Reports from OWB.............................................................................................. 11-2 Viewing Reports from the OWB Browser........................................................................ 11-5 Using the OWB Browser.................................................................................................................. 11-7 Browsing Your Repository ........................................................................................................ 11-7 Header ................................................................................................................................... 11-8 Item Definition ..................................................................................................................... 11-9 Item Details ........................................................................................................................ 11-10 Browsing Your Favorites ......................................................................................................... 11-14 Header ................................................................................................................................. 11-15 Filter..................................................................................................................................... 11-15 Navigation Favorites......................................................................................................... 11-16 Reports Favorites............................................................................................................... 11-16 Customizing Your Favorites Page .................................................................................. 11-17 The Favorites Customize Display Options Page .......................................................... 11-18 Available OWB Reports................................................................................................................. 11-19 Summary Reports.............................................................................................................. 11-19 Detailed Reports ................................................................................................................ 11-20 Implementation Reports................................................................................................... 11-22 Lineage and Impact Analysis Reports............................................................................ 11-23 Lineage and Impact Analysis Diagrams ........................................................................ 11-24 Creating Custom Reports .............................................................................................................. 11-26 Creating Custom Reports for the OWB Browser ................................................................. 11-26 Creating a Custom Report in Oracle Portal ................................................................... 11-26 Registering the Custom Report with the OWB Browser ............................................. 11-27 Adding a Custom Report to an OWB Browser Role .................................................... 11-28 Viewing the Custom Report from the OWB Browser .................................................. 11-28 Creating Other Custom Reports............................................................................................. 11-28 A Reserved Words Reserved Words................................................................................................................................... A-2 xv B The Mapper User Interface The Mapping Editor........................................................................................................................... B-1 The Mapping Menu Bar ............................................................................................................... B-1 The Mapping Toolbar .................................................................................................................. B-4 The Object Palette ......................................................................................................................... B-5 The Operator Property Inspector ............................................................................................... B-6 Using the Mouse in the Mapping Editor................................................................................... B-7 Left Mouse Click Operations ............................................................................................... B-7 Tool Tips ................................................................................................................................. B-8 Right Mouse-Click Operations ............................................................................................ B-8 Keyboard Operations.......................................................................................................... B-10 Other Dialogs .................................................................................................................................... B-11 The Add Operator Dialog ......................................................................................................... B-11 The Add/ Remove Attributes Dialog....................................................................................... B-12 The (Inbound and Outbound) Reconcile Dialog.................................................................... B-13 The Physical Properties Inspector ............................................................................................ B-13 C OWB Public View Tables Management ........................................................................................................................................ C-1 Data Model .................................................................................................................................... C-3 Implementation Model .............................................................................................................. C-10 Flat File/ Record Model ............................................................................................................. C-11 Impact Analysis/ Data Lineage Model .................................................................................... C-14 Classification Model ................................................................................................................... C-17 Expression & Function Model .................................................................................................. C-18 Configuration & Deployment Model ...................................................................................... C-20 Transformation/ Mapping Model ............................................................................................ C-20 Runtime Public Views................................................................................................................ C-23 D Using The XML Toolkit Retrieving Data From Sources .................................................................................................... D-1 Storing Data In Targets ................................................................................................................ D-2 Using Runtime Controls .............................................................................................................. D-2 How to Call the XML Toolkit ..................................................................................................... D-2 xvi Typical Control Files .................................................................................................................... D-4 XML Documents Stored in Files ................................................................................................. D-5 XML Documents Stored as Other Objects.............................................................................. D-11 Document Stored as a CLOB............................................................................................ D-11 Document Stored As An Object-Based Advanced Queue............................................ D-13 Document Stored at URL .................................................................................................. D-15 Control File Element Names and Attributes ........................................................................ D-15 Source Elements .................................................................................................................. D-15 Target Elements .................................................................................................................. D-18 Runtime Control ................................................................................................................. D-18 Document Type Definition for the Control File .................................................................... D-19 Reference Materials ................................................................................................................... D-21 E Batch Services Overview .............................................................................................................................................. E-1 Batch Service Interface....................................................................................................................... E-1 Open and Close Repository Connection ................................................................................... E-2 Projects ........................................................................................................................................... E-2 Applications................................................................................................................................... E-3 Metadata Import ........................................................................................................................... E-3 Validation....................................................................................................................................... E-4 Generation...................................................................................................................................... E-4 Deployment ................................................................................................................................... E-5 Deployment into a File System............................................................................................ E-5 Deployment in a Database ................................................................................................... E-5 Batch Service Command Line Feature ............................................................................................ E-5 Validate .......................................................................................................................................... E-6 Generate ......................................................................................................................................... E-7 Deploy into a File System ............................................................................................................ E-7 Deploy into a Database ................................................................................................................ E-7 Metadata Loader Import .............................................................................................................. E-8 Type 1 ...................................................................................................................................... E-8 Type 2 ...................................................................................................................................... E-8 Type 3 ...................................................................................................................................... E-9 Using Batch Services Through a Parameter File ...................................................................... E-9 xvii Parameter file ......................................................................................................................... E-9 Parameter file command line............................................................................................. E-10 Batch Service Command Line Scripts...................................................................................... E-10 UNIX..................................................................................................................................... E-10 DOS........................................................................................................................................ E-10 Batch Service Command Line Output ..................................................................................... E-10 Validation Examples ........................................................................................................... E-11 Metadata Import Examples................................................................................................ E-12 Generation Example............................................................................................................ E-14 Deployment into File System Examples .......................................................................... E-14 Deployment in to a Database Example ............................................................................ E-14 F Performance Enhancements Overview .............................................................................................................................................. F-1 Overhead-Free INSERT ............................................................................................................... F-3 Fast Index Creation ...................................................................................................................... F-3 Fast Constraint Maintenance ...................................................................................................... F-4 Error Handling.............................................................................................................................. F-4 Minimum Target Locking ........................................................................................................... F-4 Parallel Direct-Path INSERT....................................................................................................... F-4 When to Use Partition Exchange Loading...................................................................................... F-5 Using Partition Exchange Loading .................................................................................................. F-7 Configuring the Mapping ........................................................................................................... F-8 Configuring the Target ................................................................................................................ F-9 Create All Partitions.............................................................................................................. F-9 Create All Indexes Using the LOCAL option.................................................................. F-13 Specify the USING INDEX Option for Primary/ Unique Keys .................................... F-13 Restrictions......................................................................................................................................... F-14 Restriction 1: Allow One Date Partition Key.......................................................................... F-15 Restriction 2: Allow Natural Calendar System Only ............................................................ F-15 Restriction 3: All Data Partitions Must be in the Same Tablespace..................................... F-15 Restriction 4: All Index Partitions Must be in the Same Tablespace................................... F-15 Performance Comparisons .............................................................................................................. F-15 The Internal Workings ..................................................................................................................... F-16 Function Example 1: GET_PN ........................................................................................................ F-18 xviii Function Example 2: GET_VC........................................................................................................ F-19 G OWB Bridges: Transfer Parameters and Considerations Transfer Parameters ........................................................................................................................... G-1 Transfer Considerations.................................................................................................................... G-6 Creating Business Areas in OWB ................................................................................................... G-6 Importing Metadata from an Object Management Group (OMG) CWM Standard System........ G-9 Importing Metadata from Computer Associates ERwin 3.5.1................................................. G-10 Importing Metadata from Powersoft PowerDesigner 6.0 ........................................................ G-10 Exporting Metadata to Oracle Discoverer 3.1 and 4i ................................................................. G-11 Configuring OWB for Dimensional Reuse............................................................................. G-12 Defining dimensions and facts in OWB.......................................................................... G-13 Defining the Dummy Tables............................................................................................. G-13 Hiding Data Prior to Transfer .................................................................................................. G-14 Importing Transferred Data into Discoverer.............................................................................. G-16 Dimensional Reuse Naming Conventions in Discoverer ............................................. G-16 Exporting Metadata to an Object Management Group (OMG) CWM Standard System.. G-18 Exporting Metadata to Oracle Express ......................................................................................... G-18 OSA Configuration............................................................................................................. G-19 Index Glossary xix Send Us Your Comments Oracle Warehouse Builder Users Guide, Release 3i Part No. A87327-01 Oracle Corporation welcomes your comments and suggestions on the quality and usefulness of this publication. Your input is an important part of the information used for revision. Did you find any errors? Is the information clearly presented? Do you need more information? If so, where? Are the examples correct? Do you need more examples? What features did you like most about this manual? If you find any errors or have any other suggestions for improvement, please indicate the chapter, section, and page number (if available). You can send comments to us in the following ways: Electronic mail - dwhdoc_us@oracle.com Postal service: Oracle Corporation MS 2op713 500 Oracle Parkway Redwood Shores, California 94065 U.S.A. If you would like a reply, please give your name, address, and telephone number below.
If you have problems with the software, please contact your local Oracle World Wide Support Center. xx xxi Preface Oracle Warehouse Builder 3i is a comprehensive toolset designed for practitioners who develop and implement data warehouses. This guide contains instructions that show you how to use OWB to: s Create a definition of a data warehouse. s Configure the definitions for a physical instance of the logical warehouse. s Validate the set of definitions and their configurations. s Generate a set of scripts to create and populate the physical database instance. s Deploy and initially load the physical instance. s Periodically refresh the physical instance with generated scripts. Audience This guide is intended for data warehouse practitioners which includes: s Warehouse Architects, Designers, and Developers s Data Analysts and those who develop extract, transform, and load routines s Developers of large-scale products based on data warehouses s Warehouse Administrators s System Administrators s Other MIS professionals In order to successfully use the information in this guide, you need to be conversant in SQL and PL/ SQL. Additionally, you need to be comfortable with concepts of xxii Relational Database Management Systems and Data Warehouse design. Also, you need to be at least familiar with Oracles relational database software products such as Oracle8i/ 9i, SQL*Plus, SQL*Loader, Oracle Enterprise Manager, and Oracle Workflow. How This Guide Is Organized The Oracle Warehouse Builder Builder 3i Users Guide contains eleven chapters and six appendices. Chapter 1, "Overview", introduces you to Oracle Warehouse Builder and briefly describes the design and implementation process for a data warehouse. It briefly describes the three developmental phases for most data warehouse implementation projects, and it describes how the chapters in this guide are patterned after these phases. Chapter 2, "Getting Started on a Project", shows you how to get started on a data warehouse project using Oracle Warehouse Builder. This chapter introduces the OWB repository, shows you how to log on to the repository using the OWB client, how to get started on an actual project and how Builder handles multiple users on the same module. Chapter 3, "The Target Modules", shows you how to describe a dimensional model using Oracle Warehouse Builder. This chapter also shows you how to modify these definitions by directly editing their property sheets or using a OWB editor. Chapter 4, "The Data Source Modules" shows you how to create definitions for data sources using OWB wizards. The chapter describes how to import the definitions from Oracle and non-Oracle database systems, and Designer repositories. It shows you how to import data from a flat file source and create formats for the files. This chapter also shows you how to modify definitions, generate and display diagrams for the source objects, and print diagrams. Chapter 5, "The Source to Target Mappings", is the first of three chapters that describe mapping data providers to data recipients. This chapter introduces you to the basic concepts of mapping how data providers load data into data recipients when you deploy a data warehouse. The chapter then shows you how to create logical definitions that map source operator attributes to target operator attributes and how to set the logical properties for those attributes. Chapter 6, "Mapping Operators and Transformations", is the second of three chapters that describe mapping data providers to data recipients. This chapter describes the various tools available for transforming data as it moves from source to target. The chapter describes the Oracle Transformation Library, the Global xxiii Shared (transformation) Libraries, Expression Builder, custom transformations you define for specific needs, and SQL operations such as filters, splitters, and aggregation functions. Chapter 7, "Mapping Configuration and Code Generation", is the third and final chapter describing mapping. The chapter describes how to configure your mappings physical properties and how to define code generation strategies for optimal performance during deployment. Chapter 8, "Configure, Generate and Deploy" shows you how to configure and validate the definitions for a specific instance. The chapter then describes how to generate and deploy the scripts that extract, transform and load data into your data warehouse. These scripts create and populate a data warehouse with dimensions, facts, views, and materialized views. Chapter 9, "Periodic Loads and Updates", shows you how to load your warehouse with data. It then describes how to register job scripts with Oracle Enterprise Manager to schedule periodic data loading into the warehouse. Chapter 10, "Administration" shows you how to perform the various administrative tasks on your warehouse using Warehouse Builder utilities. Specifically, it shows you how to create, and drop OWB repositories using the Builder Repository Assistant; how to install or drop a OWB Runtime Catalog using the Builder Runtime Assistant; how to display audit statistics and error messages stored in the Runtime tables using the Builder Runtime Audit Viewer; how to export definitions from a OWB repository and Import definitions from a OWB Metadata file using the Builder Metadata Loader utility. Chapter 11, "OWB Metadata Reports", describes how to use Oracle Portal to view a your reports as well as create your own reports on that metadata. Appendix A, "Reserved Words" lists the words reserved for Warehouse Builder. Appendix B, "The Mapper User Interface", describes how to define a staging mapping that you can use to perform intermediate data transformations prior to loading the data into your warehouse. Appendix C, "OWB Public View Tables", lists the OWB Public Views thant can be accessed via SQL. These views can be used to extend the reporting capabilities of OWB. Appendix D, "Using The XML Toolkit", describes how to retrieve and store data from XML documents in a target schema. xxiv Conventions Oracle8i refers to Oracle8i Version 8.1.7. The SQL*Plus interface to Oracle8i may be referred to as SQL. This interface is the Oracle8i/9i implementation of the SQL standard ANSI X3.135-1992, ISO 9075:1992, commonly referred to as the ANSI/ ISO SQL standard or SQL92. In the examples, an implied carriage return occurs at the end of each line, unless otherwise noted. You must press the Return key at the end of a line of input. The following conventions are also used in this manual: Related Documents For more information, see the following manuals: s Oracle Warehouse Builder 3i Installation Guide s Oracle Warehouse Builder 3i Release Notes s Oracle8i/9i Data Warehousing Guide Oracle provides additional information sources, including other documentation, training, and support services that can enhance your understanding and knowledge of Oracle Warehouse Builder on the World Wide Web. s For more information on Oracle Warehouse Builder technical support, you can contact Oracle World Wide Support services at www.oracle.com/ support Convention Meaning . . . Vertical ellipsis points in an example mean that information not directly related to the example has been omitted. ... Horizontal ellipsis points in statements or commands mean that parts of the statement or command not directly related to the example have been omitted. boldface text Boldface type in text indicates a term defined in the text, the glossary, or in both locations. May also refer to interface buttons and links. [] Brackets enclose optional clauses from which you can choose one or none. xxv s For the latest information on, and downloads of, software and documentation changes and updates to Oracle Warehouse Builder, visit MetaLink at http:/ / metalink.us.oracle.com s You can order other Oracle documentation by using the World Wide Web at http:/ / oraclestore.oracle.com xxvi Overview 1-1 1 Overview Oracle Warehouse Builder is a comprehensive and powerful tool that supports the definition and maintenance of a large scale logical model of a complete data warehouse and the deployment of a physical instance of the data warehouse model. This chapter describes how the Oracle Warehouse Builder 3i User s Guide matches the process of using OWB to design, implement and load data warehouses. In this guide you will find instruction on how to use OWB to: s Create a set of definitions for a logical warehouse schema, its data sources, and the operations that map, transform, and populate a warehouse. s Configure a physical instance of the logical warehouse, validate the set of definitions for the instance, generate a set of scripts that create and load a physical instance, and deploy a physical instance to an Oracle8i/ 9i database system. s Populate and refresh the data warehouse, and set up Oracle Enterprise Manager and Oracle Workflow to manage the warehouse operation. Archive and restore projects; enable projects to be maintained through source control systems so that they can be maintained throughout the development lifecycle. The Developmental Phases 1-2 Oracle Warehouse Builder 3i Users Guide The Developmental Phases This guide is organized according to three developmental phases in the data warehouse building process: definition of the logical warehouse; configuration, validation, and deployment of a physical warehouse instance; and implementation of load and refresh operations. Although data warehouse deployment projects typically develop in a non-linear and iterative fashion, three phases are commonly recognized: 1. Definition Phase During the Definition phase, you create numerous logical definitions that describe a warehouse and its sources. These definitions formally describe: s The warehouse schema s A business structure s Data sources and targets s Mappings of the extract, transform, and load (ETL) operations 2. Generation Phase A physical instance of the warehouse is described by configuring a set of logical definitions. The configured logical definitions are validated and then used to generate a set of scripts to create and manage the physical instance: s DDL scripts to create the warehouse and intermediate schema objects. s PL/ SQL, SQL*Loader, and Tcl scripts to extract data, map and transform the data, and then load it into the physical instance. Once generated, OWB to runs the scripts that deploy the logical definitions as physical objects to the physical instance. The scripts also deploy to a file system where they can be used in conjunction with Oracle Enterprise Manager and Oracle Workflow to schedule and manage load and refresh jobs. 3. Load and Manage Phase After the generated scripts have been deployed to a file system, the infrastructure for load and refresh jobs is implemented: s Job dependencies are described as an Oracle Workflow process s Oracle Enterprise Manager is used to schedule individual jobs or an Oracle Workflow process. The Developmental Phases Overview 1-3 Refer to the Oracle8i Data Warehousing Guide for information on data warehouse design and data warehousing strategies. The Definition Phase During this phase, you create numerous logical definitions to describe your data warehouse. These logical definitions describe a data warehouse schema, your data sources, optional intermediate schemas, and the target warehouse. Defining A Warehouse You begin to build your data warehouse by defining a Project. The project typically contains the set of formal definitions that define a logical warehouse. These definitionsreferred to as Modules in OWBcorrespond to either a source or a target data warehouse. OWB enables you to define a logical warehouse and then configure its physical implementation parameters. Warehouse Builder then validates the definitions for a physical instance and generates the scripts to create objects for the instance. Chapter 2, "Getting Started on a Project" describes how to use OWB to define a warehouse project and modules. Defining The Data Sources and Targets As you create a dimensional model, developers start assessing sources of data for the warehouse. You can use the Import Metadata Wizard to import definitions of data sources from: s Database Sources including Oracle Designer repositories s Flat File Sources s Pre-packaged Applications This Warehouse Builder wizard imports logical definitions and stores them in a repository structure called a Source Module. During a later stage, these source definitions are used in the design, development, and implementation of routines that extract, transform, and load data into target warehouse schemas. Chapter 3, "The Target Modules," shows you how to create, modify, validate, and display the formal definitions of a dimensional model using the Warehouse Module Editor. OWB can display and print the model graphically or in tabular formats. One tabular format displays the models logical properties (Property Sheets). The Developmental Phases 1-4 Oracle Warehouse Builder 3i Users Guide Chapter 4, "The Data Source Modules" shows you how to create, modify, and display definitions for data sources using the Source Module Editor. The chapter describes how to create definitions for: s Relational databases (Oracle, SQL Server, IBM DB2) s Flat Files (fixed or delimited formats, etc) s Oracle Designer Repositories s Pre-packaged Applications Defining ETL Mappings After the target schema for the logical warehouse and its data sources have been logically defined, you can create logical definitions that describe how to extract, transform, and load data. These definitions are called Mappings and are stored in the warehouse module that defines the target warehouse. Three separate chapters describe how to define map: Chapter 5 introduces the concepts of mapping and how to define a basic mapping; Chapter 6 shows you how to create definitions that transform data as it moves from a source to a target, and Chapter 7 shows you how to configure the properties of the mapping. Chapter 5, "The Source to Target Mappings," describes the basic concepts behind the mapper. It then shows you how to create definitions that map source object attributes to target object attributes. It then shows you how to configure the attributes of the mapping objects and how to validate, reconcile, and edit the code generated by the mapper. Chapter 6, "Mapping Operators and Transformations," shows you how to add operators and transformations at specific points within the code that implements a mapping to modify the data before it is loaded into a target table; chapter six describes how to build PL/ SQL operations using Expression Builder. The chapter also describes how to use the Oracle Transformation Library to create custom transforms for a specific need and store it in a Global Shared Library within your project. And finally, this chapter shows you how to configure the logical properties of the mapping objects. Chapter 7, "Mapping Configuration and Code Generation", describes how to prepare your mappings for deployment. The chapter describes how to configure properties for code generation and how to set up strategies for the code generation. This chapter marks the end of the definition phase. The Developmental Phases Overview 1-5 The Generation Phase Once youve defined your logical definitions, you can configure Warehouse Builder to create and load a physical instance of the warehouse. This phase takes four steps. 1. Configure a set of definitions for a physical instance 2. Validate the set of definitions 3. Generate scripts from the set of definitions 4. Deploy objects to the physical instance and/ or deploy the scripts to a file Warehouse Builder generates the following kinds of scripts from a configured set of definitions: s DDL scripts that create objects for the physical instance s PL/ SQL and SQL*Loader routines s Tcl scripts Chapter 8, "Configure, Generate and Deploy" describes the tasks you need to perform to generate these scripts. Specifically, the chapter describes s Configuring The Logical Definitions The configuration parameters determine how an object is deployed, the processing characteristics of selected objects, location of deployed scripts, the physical properties of warehouse objects, and many other properties. s Validating The Logical Definitions After you configure a physical instance, you validate the set of definitions before Warehouse Builder generates the scripts that implement that instance. The Validate utility detects script errors such as foreign key references, invalid object references in mappings, and data type mismatches. s Generating The Deployment Scripts After you have a validated the set of configured definitions describing a physical instance, you generate a set of scripts that is used to deploy objects to a physical instance and another that is used to load and refresh jobs. After Warehouse Builder generates the scripts for a physical instance, you can immediately deploy the database objects (database links, tables, dimensions, facts, materialized views, synonyms, and PL/ SQL packages) to one or more instances. You can also deploy the scripts that create the objects to a file system, Taking The Next Step 1-6 Oracle Warehouse Builder 3i Users Guide and then deploy the objects by manually executing the deployed scripts. And finally, you can deploy the scripts to a set of directories. These directories include DDL, PLS, Tcl, SQL*Loader control file directory, and others. After deployment, you can use the Workflow Deploy Wizard to register the Tcl scripts that define load and refresh jobs to an Oracle Enterprise Manager repository. You can also register the scripts with an Oracle Workflow server to manage job dependencies such as the dependence of fact tables on its dimensions. The Load And Manage Phase The warehouse objects have been deployed to a physical instance and the jobs that initially load or periodically refresh the instance have been deployed to a directory and registered with Oracle Enterprise Manager. Chapter 9, "Periodic Loads and Updates" shows you how to set up and schedule jobs that initially load and periodically refresh a data warehouse. Jobs that load new data or refresh existing data in a data warehouse must be run in a strict sequence to insure that all foreign key references can be satisfied. Chapter 10, "Administration" shows you how to complete administrative tasks using Oracle Warehouse Builder utilities. This chapter discusses the Repository Assistant, the Run Time Audit Viewer, the Metadata Loader Utility. It also describes the Archive/ Restore facility which allows versioning control throughout the development lifecycle and provides for smooth interaction with source control systems used in development. Chapter 11, "OWB Metadata Reports", describes how to use Oracle Portal to view the metadata your repository. OWB contains a browser that you use to navigate the metadata and use Portal to see. You can also build customized reports to suit your OLAP needs using Oracle Portal. Taking The Next Step The next chapter, "Getting Started on a Project," shows you how to log onto a OWB repository with the Oracle Warehouse Builder client, create a repository structure called a Project, and customize the OWB console. After you create a Project, you can start creating definitions for that warehouse. Getting Started on a Project 2-1 2 Getting Started on a Project This chapter shows you how to get started on a design and implementation project using Warehouse Builder. To get started, you must first create a place within OWBs repository where you can store and manage the definitions that formally describe the initial logical warehouse. OWB stores these definitions in a repository structure called an OWB Project. This chapter summarizes the contents and structure of a OWB Project, and then shows you, step by step, how to log on to a OWB repository and create a Project. The chapter closes with a few topics that will help you get off to a smoother start: a description of the policies that govern business and physical names, a discussion of how to move about OWB wizards, and how to personalize the OWB console so that it meets your own specific needs. This chapter includes the following sections: s Understanding The OWB Project s Creating A Project s About The OWB Console s About Naming Policies s Using The Warehouse Builder Wizards s Searching Navigation Trees s Taking The Next Step Understanding The OWB Project 2-2 Oracle Warehouse Builder 3i Users Guide Understanding The OWB Project A Warehouse Builder Project is a high-level repository structure that stores and organizes the formal descriptions that define a data warehouse. These definitions include descriptions of data sources, target warehouse objects, mappings of source data to various targets, transformation operations, and configuration parameters. Warehouse Builder generates and stores numerous scripts in a Project. As a warehouse project evolves, developers use OWB to configure and validate warehouse definitions and then use OWB to generate scripts to create and load the data warehouse. OWB also deploys scripts to a file system so the operations staff can run the daily operation using Oracle Enterprise Manager and Oracle Workflow. About Warehouse Builder Projects At the highest level, Warehouse Builder Projects are subdivided into containers called "modules." There are two kinds of modules: source modules and warehouse modules. Modules can contain various object types. For example, warehouse modules are subdivided into dimensions, facts, materialized views, and other warehouse objects. The definitions for the objects occur on the lowest level of the Project hierarchy. Figure 21 maps the structure of a OWB Project. Figure 21 The Warehouse Builder Project Structure Projects Object Types Modules Definitions Understanding The OWB Project Getting Started on a Project 2-3 Module Contents OWB Projects are divided into source modules and warehouse modules. s Source Modules contain definitions of: Relational database objects Flat files Proprietary structures s Warehouse Modules contain definitions of: Dimensions Facts Relational database tables Mappings Transformation operations Materialized Views Views Sequences Scripts generated by OWB Object Names Using business names, developers can make a warehouse schema resemble the business being modeled. The OWB technical term for business names is logical names. However, before you can generate the DDL for a named object all the logical names used to define that object must be translated into physical names. These physical names must conform to the syntax rules defined in the Oracle8i/9i SQL Reference. OWB maintains a logical and a physical name for each object stored in its repository. When you create an object and give it a logical name, OWB automatically creates a legal physical name for the object. And, vice-versa, when you create an object and give it a physical name, OWB validates the physical name and automatically creates a logical counterpart. Thus, no translation is required when DDL or another kind of script must be generated for an object. Naming Mode You can run the OWB client in either the logical or physical name mode, but to create or modify logical or long descriptive names, you must configure Understanding The OWB Project 2-4 Oracle Warehouse Builder 3i Users Guide the OWB client to run in the Logical Name modeby default, OWB starts in Physical Names mode. To set OWB to operate in the logical naming mode, open the Preferences window and click the appropriate check boxes on the Naming panel. For specific information on how to set these preferences, see "About Naming Policies" on page 2-20. About Multiple-User Access Before logging on to Warehouse Builder, you should know about the multiple-user access feature. The OWB Multi-User Access feature ensures only one user has write privileges to a warehouse object while simultaneously enabling other users to maintain read-only privileges to that same object or module. The lock is maintained for the duration of any transaction started by the write-privileged user. The lock is released when the user commits (or performs a rollback) and then closes all editors associated with the warehouse object. Write locks occur whenever you access data warehouse object for editing purposes. Acquiring a write lock prevents other users from editing the object while you are active in that object. You will acquire a Folder-in-Use Lock whenever you access any object within any OWB-related hierarchy. This lock prevents other users from deleting any parent object in the hierarchy as long as you are active in the hierarchy. For instance, you cannot delete a module from the Logical Tree in the OWB Console while other users are active in that module because those users hold a folder-in-use lock. Read/Write Mode To enter Read/ Write mode for and object (and lock the object), you either: Note: OWB Projects do not appear in any SQL (no code is generated), so their names need not conform to the syntax rules for physical names except their length must not exceed 30 characters. Note: If you generate code at any time during your Warehouse OWB Session, Multi-User Locking feature will remain in effect until after you commit your changes. Understanding The OWB Project Getting Started on a Project 2-5 s open an editor, property sheet or dialog; by default, the user interface attempts to access existing OWB metadata objects in Read/ Write mode. s transition to another editor that is available in Read/ Write mode from one currently open in Read/ Write mode. All editor(s) associated with it will display READ/ WRITE in the status bar of the editor. Property sheets or dialogs will indicate in the title bar that the object is in read/ write mode. You can freely edit the object in the property sheet or dialog, using the provided controls. Figure 22 The Read-Write Indicator In An Object Title Bar To end the Read/ Write access mode for an object (and unlock the object for others) and access the object in Read-Only, you can either s Close all of the objects editor(s) and commit/ rollback current transaction. s Exit OWB. Read Only Mode If OWB cannot open the editor, property sheet or dialog in read/ write mode, then it is because the object is locked by another user. You will know if you are activating a locked warehouse object or module if the object editor s status bar displays the text LOCKED BY [SESSION ID]. If you attempt to open a locked object, OWB will alert you and prompt you either to cancel the request or access the object in Read-Only mode. Understanding The OWB Project 2-6 Oracle Warehouse Builder 3i Users Guide Figure 23 The Locked Object Alert If you choose to continue in Read-Only mode, the editor will indicate in the title bar that the object is Locked and will show the ID of the user currently locking the object. You can move objects around on the editor canvas, minimize and maximize containers. However, you cannot edit the object in Read-Only mode; any attempt to modify the object will cause the following dialog to appear: Figure 24 A Read-Only Alert For any property sheet activated in Read-Only access mode, the OK button will be disabled. Lock Synchronization Whenever an OWB data warehouse object is locked by a user, the changes can be visible to all users with Read-Only access to that object. The Read-Only users will see the changes on a locked object as it happens by using the Synchronize feature To invoke synchronization, select View > Synchronize in the Warehouse Module Editor. Understanding The OWB Project Getting Started on a Project 2-7 Figure 25 The Synchronize Menu Choice OWB includes two types of synchronization options: s Automatic SynchronizationOWB automatically synchronizes upon accessing an object in Read-Write mode. s Tree SynchronizationYou may also choose explicitly to synchronize OWB from the trees by selecting View > Synchronize. This method picks up objects that have been added or removed that do not show up with automatic synchronization. Creating A Project To create a Warehouse Builder Project, you must log on to a repository, switch to the Administration environment, and then open the Create New Project wizard. After you supply a name and description for the Project, the wizard creates your Project in the OWB repository. A OWB repository is a collection of database objects installed into an Oracle8i/9i schema that has been initialized by the Repository Assistant with the following: s The Oracle Library of transformation operations s A Global Shared Library s A Project shell named My Project Understanding The OWB Project 2-8 Oracle Warehouse Builder 3i Users Guide The project shell is required for the initial logon sequence (you always log on to a specific project) and can be deleted after you create your own projects. For additional information on how to create and initialize a repository, see "Repository Administration" on page 10-2 Logging On To A Repository The following instructions show you how to start the OWB client, log on to a repository and then create a project. To log on to a repository: 1. Select Start > Oracle > OWB Client OWB returns the logon dialog box. 2. Enter the logon information: s Name and password of the repositorys owner . s Host Name s Port Number s Oracle SID This is the standard information required to connect with an Oracle8i/9i database instance. 3. Click Logon. OWB returns the Project dialog box. This repository is empty and contains only the shell Project "My Project." When a repository contains multiple Projects, you can select Note: You cannot delete the currently active project. Understanding The OWB Project Getting Started on a Project 2-9 a specific project from the drop-down list. This list includes all projects in the repository. 4. Click OK. OWB returns its console in a window. This window remains open and available during the entire session. The above illustration shows OWBs console in its initial state (Project environment). In this state the console: s Operates in the Project environment s Displays the navigation tree for the Project selected during the logon sequence When you switch to another Project within the repository (Project > Switch Project) or to a different context (Transformation Library or Administration), the console Note: If the Oracle8i/9i database instance is inactive, OWB returns a connection error message. Navigation Tree Menus Toolbar Administration Project (default) Transformation Library ENVIRONMENTS Utilities Understanding The OWB Project 2-10 Oracle Warehouse Builder 3i Users Guide displays the navigation tree for that Project or environment and resets the drop-down lists under each menu item. For additional information on different environments, see "Operational Environments" on page 2-12. Using The New Project Wizard To create a Project, you must switch to the Administration environment from the Warehouse Builder console. To create a new project: 1. Click the icon for the Administration environment. OWB points to the Administration navigation tree and updates the menu items to reflect this new context. 2. Click the plus sign next ADMINISTRATION. OWB displays the navigation trees Projects and Integrators branches. 3. Right-click Projects > Create Project OWB returns the welcome page for the New Project wizard. 4. Click Next. OWB returns the Name page. This page contains a text box for the new Projects name and a text area for a brief description. 5. Enter the Projects name, a concise description, and then click Finish. OWB creates a Project and inserts its name in the PROJECTS branch. Commit OWB does not commit a newly created Project to the repository until you click the Commit icon on the toolbar, or exit OWB and click Yes on the pop-up Commit Confirmation panel . You now have a place to store and modify all the definitions for the data warehouse. OWB also uses this space to validate these definitions and generate scripts that create and load your warehouse. About The OWB Console Getting Started on a Project 2-11 The next section familiarizes you with the console and its different operational contexts. About The OWB Console The OWB console has six major components: s Set of menu selections s Toolbar s Wizard button s Icons to switch context s Utilities button s Window with a navigation tree When you switch to another mode of operation the icons all remain constant, but the menu selections and navigation tree change. The illustration below shows the console in its initial state: the Project context. About The OWB Console 2-12 Oracle Warehouse Builder 3i Users Guide Figure 26 The OWB Console Operational Environments The OWB console operates in three environments: s Project (initial environment) s Transformation Library s Administration When the OWB client starts, the console begins in the Project environment and displays the navigation tree for the project selected during the logon sequence. To change environments during a session, click one of the context icons. Project Environment When you are in the Project environment, you can: Navigation Tree Menus Toolbar Administration Project (default) Wizard button Transform Library Utilities button About The OWB Console Getting Started on a Project 2-13 s Create and modify definitions for repository objects such as dimensions, facts, relational tables, materialized views, sequences, and views s Create, modify, and import definitions for data sources such as relational schema objects and flat files s Create and modify transformation operations s Create and modify mappings to direct the flow of data from sources to transformations and on to target schemas (warehouse or temporary staging areas) s Configure and validate definitions s Generate, deploy, and run scripts based on the definitions to create, load, and operate target warehouses or staging area schemas s Export metadata for an entire Project, its modules, or individual definitions Warehouse developers operate mostly with the OWB console in this environment. The navigation tree includes: Before you can create any of these definitions, whether for data sources or warehouse objects, you must first create the modules where you can store, modify, and manage the definitions. Each chapter in this guide shows you how to create the Table 21 Modules in a Typical Warehouse Project Module Name Type Contents SRC_ Source Module Definitions for objects. SRC_ Source Module Definitions for flat flies STG_ Warehouse Module Definitions for the intermediate work area TRG_ Warehouse Module Definitions for the target schema About The OWB Console 2-14 Oracle Warehouse Builder 3i Users Guide required containers before it shows you how to create the definitions themselves. For example, Chapter 3, "The Target Modules," shows you how to create warehouse module before it shows you how to create and modify the definitions that formally describe the star schema. At any time during the development process, you can configure and validate the set of working definitions. Thus, incremental corrections can be made as the project moves forward and, should large scale corrections be necessary, OWB can do most of the work. Once a set of valid definitions have been created, the team can generate scripts to: s Create an instance of a warehouse s Extract and transform data for that instance s Perform initial and incremental loads s Maintain the data warehouse using Oracle Enterprise Manager and Oracle Workflow The development team can then turn around, reconfigure the set of definitions, validate the reconfigured set, and then generate another set of scripts for a different version of the warehouse. Transformation Environment You can create transformation libraries and transformations within these libraries when the console is in this environment. You can also modify the properties of warehouse and source modules. The illustration shows the fully expanded navigation tree for the Transformation Library. The tree provides access to the built-in Oracle Library as well as the transformation operations created by developers for your warehouse. About The OWB Console Getting Started on a Project 2-15 Administration Environment When you are in the Administrative environment, you can: s Create, modify, and display the properties of a Project s Import metadata from a OWB export file The illustration shows the Administration tree fully expanded. The tree contains three Projects and the Integrators supplied with OWB. The Integrators can retrieve metadata from applications based on relational databases, flat files, and proprietary systems. This guide shows you how to use the Flat File and Oracle DB Integrators to extract data from applications (see Chapter 4, "The Data Source Modules", for more information). The Utilities Button You can run a utility directly from the OWB console using the Utilities icon. For example, to run SQL*PLUS from the OWB console: 1. Click the Utilities icon. OWB expands the icon into the Utilities drawer. 2. Click the icon of a utility that occurs in the drawer (SQL*PLUS). OWB calls the utility. The Utilities Drawer is pre-configured with five utilities: s SQL*PLUS s Developer Reports Runtime s Oracle Enterprise Manager About The OWB Console 2-16 Oracle Warehouse Builder 3i Users Guide s Oracle Workflow s CWM Bridge Most likely, you will need to update the configurations of these icons so that they reference the utilities as they are installed at your site. Utility Property Sheet To update or delete an existing icon or add a new one, you must call the OWB Consoles Preferences sheet. This sheet actually includes three sheets that are addressed by tabs: OEM, Browse, and Archive/ Restore. You can call this set of property sheets from the Consoles toolbar. To call the Consoles Preferences sheet: 1. Select Project > Preferences OWB returns its Preferences Sheet 2. Select Utility on the tab bar. OWB returns the configuration panel for configured utilities. The following instructions describe how to add a new utility, update the configuration of an existing utility, and delete an existing utility. Add a Utility To add a utility to the Utilities Drawer: 1. Click Utility on the tab bar of the Preferences Sheet. 2. Select the name of a tool. OWB returns its configuration information. 3. Enter the information. s Name of the utility. s Location of the utility. Click Browse to search directories. s Location of the icon for the utility. Click Browse to search icon folders. s Description of the utility. 4. Click Add. About The OWB Console Getting Started on a Project 2-17 Update a Utility To update the configuration of an existing utility: 1. Click Utility on the tab bar of the Preferences window. 2. Select the utilitys name from the Contents box (e.g., SQL*PLUS). OWB returns its configuration information. 3. Modify the configuration details: s Name of the utility. s Location of the utility. Click Browse to search for program folders and files. s Location of the icon for the utility. Click Browse to search for icon folders and files. s Description of the utility. 4. Click Update. OWB stores the changes in the repository. Remove a Utility To remove a utility from the Utilities Drawer: 1. Select the name of an existing tool. OWB highlights the name of the selected utility and returns its configuration information. 2. Click Remove OWB removes the utility from the Utility Drawer. About The OWB Console 2-18 Oracle Warehouse Builder 3i Users Guide The Console Tool Bar From the console tool bar, you can: s Commit changes to the repository s Search the repository for an object s Look at the properties of an object s Call the Help system You can also perform these actions from drop down menus. The function and usage of the Find and Help buttons are self-explanatory. Property Icon and Property Sheets The function and usage of the Property icon on the Console toolbar is always the same: return the property sheets for the selected object. Each object stored in a OWB Project has its own property sheet, and this property sheet defines the object. You can resize the property sheet to fit your window. Property sheets vary by object. The property sheet for a Project contains two sheets but the property sheet for a warehouse module contains three, and that for a Dimension contains five. About The OWB Console Getting Started on a Project 2-19 Example 21 Property Sheet for a Dimension Edit a Definition Definitions for objects such as dimensions, tables, facts, and mappings are created manually using a wizard or imported from an external source. After a definition has been created for an object and stored in a OWB Project, you can update the definition by editing its property sheet. The Commit Icon The Commit icon is important for this reason: When you create or modify any part of your warehouse, OWB does not automatically commit the change. You must explicitly commit changes. You can commit changes at any time during a OWB session or defer the operation until you terminate your session. To commit changes during a session, make the console the active window and click the Commit icon on the tool bar. OWB returns a Commit Confirmation dialog box. Click OK to commit the changes. If you have not committed and exited OWB, the commit confirmation dialog box appear. Click Yes to commit any changes you have made. About Naming Policies 2-20 Oracle Warehouse Builder 3i Users Guide About Naming Policies Warehouse developers often use long descriptive names for dimensions, fact tables, materialized views, and other objects to promote communication. Using business names, developer s can make a warehouse schema resemble the business being modeled. The OWB technical term for business names is "logical names." Before you can generate DDL for a named object, all of the logical names for that object must be translated into physical names so that they conform to the syntax rules for basic elements as defined in the Oracle8i SQL Reference manual. Additionally, selected physical names must be unique within their category: s Module names must be unique within a OWB Project s Warehouse object names must be unique within a warehouse module. This includes the names of: Dimensions Facts Mappings Materialized Views Sequence Names Views s Transformation names must be unique within a transformation category. OWB maintains a logical and a physical name for each object stored in its repository. When you create an object and give it a logical name, OWB automatically creates a legal physical name for the object. And vice-versa, when you create an object and give it a physical name, OWB validates the physical name and automatically creates its logical counterpart. Thus, no translation is required when DDL or another kind of script must be generated for an object. OWB remembers your Naming preferences from session to session provided you always use the OWB client from the same Windows NT workstation. The Naming mode preference set for a specific host is stored in the Preference.properties file. Hence, if you work on a repository from different workstations, the Naming mode may vary. Note: OWB Projects are not deployed (no code is generated), so their names need not conform to the syntax rules for physical names except their length must not exceed 30 characters. About Naming Policies Getting Started on a Project 2-21 Setting The Naming Mode To create a logical name for an object or rename an existing object, the OWB client must be in the Logical Name mode. Similarly, to create a physical name for an object or rename an existing object, you must put the OWB client in the Physical Name mode. The default naming mode for the client is Physical Name mode.The default naming preferences for the OWB client are: s Mode: Physical Name Mode s Propagation: Propagate physical name to logical name You can change these from the consoles Preferences sheets. The following procedure describes how to set the naming mode using the Preference sheets. There are two icons for the name and propagation modes located in the lower right corner of the various editors. These icons indicate the mode in which you are currently active. To set the name mode: 1. Open the Warehouse Builder Console and select the Project Environment. 2. Select Project > Preferences. OWB returns the Preferences window. 3. Select the Names tab card. OWB returns the Names panel. You can select either physical or logical name modes and you can select how the names propagate. You can switch Naming mode at any time during a session. Logical Name Mode You can create a logical name for an object or change the logical name of an existing object when OWB is in the Logical name mode. OWB editors, wizards, and property sheets also display the logical names of objects when in this mode. About Naming Policies 2-22 Oracle Warehouse Builder 3i Users Guide A logical name must conform to these rules: s The length of a name cannot exceed 200 characters. s The name must be unique within its category (for example, the same column name can be used in different tables but not the same table). s All source modules reflect the case of the imported source and are subject to the double-quotes rules as defined in the Oracle8i SQL Reference. s Copy operations from a source to a target in a mapping does not propagate case. When you create a logical name, OWB automatically generates a legal physical name that resembles the logical name. If you create a logical name that duplicates an existing physical name, OWB appends an underscore and a numeral to the logical name to avoid a name collision. When OWB is in the Logical name mode and you change the logical name of an object, you can choose whether to propagate this change to the objects physical name. To propagate the changes, check Propagate logical name to physical name. Using The Warehouse Builder Wizards Getting Started on a Project 2-23 Physical Name Mode You can create a physical name for an object or change the physical name of an existing object when OWB is in the Physical name mode. OWB editors, wizards, and property sheets display physical names of objects when in this mode. Physical names are automatically converted to upper case. A physical name must contain no more than 28 characters (two characters are reserved for Warehouse Builder) and conform with the basic syntax rules for schema objects defined by the rules set down in the Oracle8i SQL Reference. OWB enforces these syntax rules when in is in the Physical name mode: you cannot enter an invalid physical name. For example, you cannot enter a duplicate name, a name with too many characters, or misuse reserved words. When OWB is in the Physical name mode and you change the physical name of an object, you can choose whether to propagate this change to its logical counterpart. To propagate the change, check Propagate physical to logical name. Using The Warehouse Builder Wizards Oracle Warehouse Builder provides numerous wizards to simplify the design, creation, and implementation process. These wizards can be invoked in a number of ways but mostly by right-clicking on the object type and selecting Create from the popup menu. Each wizard is divided into one or more pages. The first page is always the welcome page. This page lists the number of pages in a specific wizard and describes the task performed by each page. Using The Warehouse Builder Wizards 2-24 Oracle Warehouse Builder 3i Users Guide
You can configure individual wizards so they do not return their welcome pages using the check box. To re-enable this behavior, at a later time, you must open the Preferences sheets and click the button "Display Welcome page on all wizards." Using Navigation and Help To navigate through individual pages of a wizard, click the Back and Next buttons. When you define a warehouse object, you will often need to traverse these pages to make corrections or additions. Click Cancel to exit without saving your work. For help regarding a specific topic within a wizard, click the Help button at the bottom of the wizard page for context-sensitive help instead of the general help button on the tool bar. Searching Navigation Trees Getting Started on a Project 2-25 The wizard pages solicit information in a variety of ways: down-arrow lists, text-boxes, check boxes, radio buttons for various functions including Add, Update, and Remove buttons. You can move around a specific page using a mouse or using the Tab key. When you tab to the last text box, you must then use the CTL-Tab combination to move the cursor down to and across the control buttons. Searching Navigation Trees Definitions and other objects stored in a OWB repository are usually organized by a navigation tree.You can click on the tab cards or press the tab key to switch between the tab sheets of different trees: press the tab key, and press the arrow key to switch between pages. Searching Navigation Trees 2-26 Oracle Warehouse Builder 3i Users Guide Definitions stored in a warehouse module, for example, are organized by the navigation tree displayed by the Warehouse Module Editor. Or, Projects defined within a repository are organized by the Project navigation tree displayed by the OWB console. To find an object in the repository, you can manually search for the object or use the Find button to call the Object Find Dialog panel. To manually search for an object, expand each branch of the navigation tree until you find it (click the plus sign). To use the Object Find Dialog panel, click the Find button and describe your search. You can include the complete name of an object or use the * wildcard character to match zero or more characters. You can attach the wildcard character as a prefix or suffix to any substring. The wildcard character matches any character. Taking The Next Step Getting Started on a Project 2-27 Taking The Next Step This chapter summarized the preliminary steps in setting up a Warehouse Builder: 1. Logon to a OWB repository. 2. Create a OWB Project where you can store and manage all the definitions that formally describe your data warehouse. 3. Review the operation of the OWB console. 4. Familiarize yourself with naming conventions. The next chapter, "The Target Modules," shows you how to create a complete set of definitions for a dimensional model that serves as the target schema for the Global Computing Companys data warehouse. OWB Project: Repository structure where warehouse developers store and modify the formal descriptions that define a data warehouse, and where OWB stores generated scripts used to create and load the warehouse. Taking The Next Step 2-28 Oracle Warehouse Builder 3i Users Guide The Target Modules 3-1 3 The Target Modules This chapter describes how to define a dimensional model using Oracle Warehouse Builder. The definitions for this logical model are created using a OWB wizard or imported from external sources. This chapter also shows you how to modify these definitions by directly editing their property sheets or using a OWB editor. The OWB editors can also display and print diagrams as well as detailed reports about the logical warehouses objects. An introductory section summarizes a dimensional model, that serves as the target warehouse for this guide. The next section shows you how to create a place to store these definitions into a warehouse module. The remaining sections show you how to create and modify each of the definitions required to formally describe a data warehouse model. This chapter includes the following sections: s About Star Schemas s Creating a Warehouse Module s Creating Dimension Definitions s Creating Fact Table Definitions s Creating Materialized View Definitions s Creating Conventional View Definitions s Infrastructure Requirements s Committing Your Changes s Taking the Next Step About Star Schemas 3-2 Oracle Warehouse Builder 3i Users Guide About Star Schemas A typical dimensional model used as the target schema for a logical data warehouse may contain a fact table, dimensions, materialized views, and a conventional view. This schema, known as a Star Schema, is central to the concept of Data Warehouse design. The following illustration describes a fact table linked to a set of dimensions over foreign key constraints. For background and specific information on star schemas and dimensional models, see the Oracle8i/9i Data Warehousing Guide. About Dimension Tables Dimension tables contain descriptive details that give meaning to the billions of numbers that populate fact tables. Dimensions provide the context for what would otherwise be faceless numbers and describe the fastest access path to sets of those numbers. Query performance can be further improved in many cases because users often analyze data by drilling down known hierarchies using slice-and-dice techniques. These hierarchies may include a temporal hierarchy Year > Quarter > Month > Day and the brand hierarchy Category > Brand > Product. Oracle8i/9i can then exploit defined hierarchies by rewriting queries to fetch data from summary rather than detail tables. The rewritten queries run much faster. Typical dimension tables usually possess the following characteristics: s A single column primary key populated with values called "warehouse keys." About Star Schemas The Target Modules 3-3 s Warehouse keys provide administrative control over the dimension, support techniques that preserve dimension history, and reduce the size of fact tables. s One or more hierarchies that are explicitly defined as dimension objects. Hierarchies defined by a Create Dimension statement maximize the number of query rewrites by the Oracle8i/9i server. This chapter shows you how to create definitions for dimension objects and their underlying tables using the New Dimension Wizard. This chapter also shows you how to modify the definitions of dimension objects and their underlying table by editing their respective property sheets or using the Dimension Editor. About Fact Tables Fact tables contain measures and links to a set of dimension tables. For the most part, a fact tables measures are additive. Common additive measures include dollars, units, and cost. Some fact tables, however, include non-additive and semi-additive measures such as margins, averages, and balances. Event fact tables, on the other hand, contain no measures but only links to a set of dimension tables. Linking Facts to Dimensions Fact tables are linked to a set of dimension tables over foreign key constraints. These constraints are critical in a data warehousing environment where data integrity is paramount: the constraints enforce referential integrity during the daily operation of the data warehouse. Referential integrity constraints can impose a significant performance penalty during the load of data into the warehouse because they can incur an enormous number of look-up operations. Developers can avoid this penalty by verifying the truth of the constraint during the extraction and transformation process and disabling the constraint during the load process. The constraints are then enabled after the load. When dimensions are designed with warehouse keys, the fact table row length is usually reduced significantly because warehouse keys are usually much shorter than their natural counterparts. The result is improved query performance. Physical Characteristics Like dimension tables, fact tables vary by practitioner and project, but typical fact table has two characteristics: s A primary key defined on the set of foreign key reference columns or, in the case of a data list, on an artificial key or a set of warehouse key columns. When About Star Schemas 3-4 Oracle Warehouse Builder 3i Users Guide the fact table is a data list, the foreign key reference columns do not uniquely identify each row in the fact table. s A set of foreign key reference columns that link the table with its dimensions. When you create a definition for a fact table, you must define its measures and its foreign key references. To define a foreign key reference, you must include the name of the referenced dimension and its primary key column. Consequently, definitions for dimension tables should be created before those for fact tables. This chapter shows you how to create a definition for a fact using the New Fact Wizard and modify the definition by editing its property sheet and using the Fact Editor. About Materialized Views Materialized views are summary tables used by the Oracle8i/9i server to rewrite queries so they execute faster. The basic idea is simple: a query that retrieves product totals from a monthly summary table will run much faster than a query that retrieves data from a detail table and then calculates the monthly subtotals. The first query retrieves far fewer rows and performs fewer calculations; consequently, the first query executes much faster. The Oracle8i/9i server makes this technique viable because end-users need not know whether the materialized views exist nor how to rewrite queries: the server automatically does this work transparently. Moreover, the Oracle8i/9i server can refresh materialized views as the detail data changes. After proper dimensional design, materialized views are the next best way to boost query performance, and this chapter shows you how to create the definition for materialized view using the New Materialized View Wizard. This chapter also shows you how to modify these definitions using the Materialized View Editor. Requirements for Query Rewrite To take advantage of query rewrite, you must create the materialized views and enable the query rewrite feature. To maximize the number of query rewrites, create dimension and fact tables so they include the physical characteristics of dimensions and facts as outlined on page 3-2 and on page 3-3. See the Oracle8i/9i Data Warehousing Guide, Release 2 for more information about query rewrites. The Oracle8i/9i server can rewrite and speed up the execution of a large number of queries using these two materialized view provided the dimension and fact tables have the physical characteristics of classic dimensions and facts. Appendix B, The Mapper User Interface includes the DDL for a materialized view. Creating a Warehouse Module The Target Modules 3-5 About Conventional Views Conventional views simplify and secure data within a data warehouse without requiring additional physical storage space. For example, a view could calculate and return standard deviations for brand summaries rather than a list of details. This simplifies access for users. Another view could restrict access to the sales values of specific products. This kind of view simplifies management of the data warehouses for the Database Administrator. The procedure for creating a definition for a view is similar to creating a materialized view. The objects themselves, however, differ considerably. A materialized view occupies storage space; a view does not. A materialized view speeds queries; a view has a different purpose. Materialized views are principally designed for the server and users need not know about their existence; views are typically advertised to users. This chapter describes how to create a view definition using the New View Wizard. This chapter also shows you how to modify the definition by editing its property sheet or using the View Editor. Before you create definitions for a star schema, you first create a warehouse module in the Warehouse Builder repository where the definitions can be stored. The next section shows you how to create such a module. You should create schema objects in the following order: 1. Dimensions 2. A fact table 3. Views Typically, dimensions are created first and objects that reference the dimensions (facts and views) second. You could create the definition for the fact table first by omitting the foreign key reference columns and add them later after the dimensions have been defined. You should also note that when a fact table relies on the server to verify foreign key references, this same order must be followed. Creating a Warehouse Module Before you create definitions for a schemas dimensions, fact table, and materialized views, you must first create a place where OWB can store them. OWB stores definitions for target schemas in warehouse modules, and this section shows you how to create one for the warehouse schema. Creating a Warehouse Module 3-6 Oracle Warehouse Builder 3i Users Guide To create a warehouse module: 1. Click the navigation trees plus sign. OWB expands the navigation tree. 2. Select MODULES > Create Module OWB returns the New Module Wizards welcome page. 3. Click Next. The wizard returns its Name and Other Information page which solicits: s Name of the module s Status of the module s Type of the module A module can be Data Source or Warehouse Target. s Description of the module 4. Enter the requested information. 5. Click Next. OWB returns the Target page. This page requests: s Application s Application Version or System Type s Integrator used to access other database dictionaries for definitions Select the Application, Type, and Integrator using the drop-down list activated by down-arrow buttons. 6. Click Next. OWB returns the Connection Information page. This page solicits information required to create a database link. OWB can use this link to import definitions from another schema or from an Oracle Designer repository. Creating a Warehouse Module The Target Modules 3-7 If you intend to import definitions from another schema or an Oracle Designer repository, then you need to supply the connection information. Otherwise, you can simply skip this page. The following steps show you how to crate a link to an Oracle Designer repository: 7. Select the button for the Oracle Designer Repository. 8. If a link to the Designer repository already exists in the OWB repository, click the down-arrow, and select its name from the drop-down list. Otherwise, you must provide the necessary link information. Check for a Designer Repository. Creating a Warehouse Module 3-8 Oracle Warehouse Builder 3i Users Guide To create a new database link: 1. Click New DB Link OWB returns the New Database Link window. Enter the following: * A name for the new link. Enter an SQL*Net connect string or complete host information. This connect string must be defined before you can test the connection. * Enter a user name and password for the target schema: StarMart. 2. Click Create and Test. OWB attempts to connect with the Designer Repository using the connection information and returns a status message in the message box below the Create and Test button. 3. Click OK. OWB stores the database link in the repository and its list of available database links. 4. Click the down-arrow and select the database link from the drop-down list. The link has been created, tested, and you can now import definitions from the Oracle Designer Repository. To import definitions from an Oracle8i/9i data dictionary, select Oracle Data Dictionary instead of Oracle Designer Repository in step six above. 9. Click Next. OWB returns the Finish page. This page summarizes the information you entered on each of the wizards pages. Note: A OWB warehouse module can connect only with an Oracle8i/9i database instance. Creating a Warehouse Module The Target Modules 3-9 If the information displayed on the summary page is incorrect, navigate back through the wizard pages using the Back button to make corrections. 10. Click Finish. OWB creates the module in the active project and inserts its name in the MODULES branch of the navigation tree. OWB does not automatically commit your work to the repository, that is your responsibility. You can commit your work immediately or defer this action until you exit a OWB session. To commit your work immediately, click the Commit icon on the toolbar. OWB returns a Commit Confirmation panel: click Yes. You can also defer this action until you exit a OWB session. On exit, OWB always returns an Exit Confirmation panel: click Yes to commit your work. Note: OWB returns a Warning panel when the character set of the database instance differs from the character set of the host for the OWB client. Creating a Warehouse Module 3-10 Oracle Warehouse Builder 3i Users Guide Displaying the Warehouse Module You can now verify that OWB created the warehouse module and then use the Warehouse Module Editor to inspect its initial structure. Verifying the Module To verify that OWB created a warehouse module, fully expand the navigation tree using the Warehouse Module Editor. The name of the warehouse module will appear in the navigation tree. The navigation tree shows the modules within the Project. Typically, the modules are a target warehouse and source modules. OWB represents these modules using different icons: s Warehouse modules with a warehouse icon. s Source modules for relational database sources with a disk icon s Source modules for files with a file icon. To open a module with an editor, double-click the modules name. Inspecting the Module You can use the Warehouse Module Editor to inspect the structure of the empty module or create and store objects in it. To open the Warehouse Module Editor: 1. Expand the MODULES branch. OWB lists all of the projects modules. 2. Double-click the warehouse modules name. OWB opens a new window for the Warehouse Module Editor, and displays its navigation tree. The navigation tree in this window is the starting point for the remaining instructions in this chapter. Creating Dimension Definitions The Target Modules 3-11 Creating Dimension Definitions This section shows you how to create and update a complete definition for a dimension. Actually, two definitions are associated with each dimension: one describes the dimension object and the other describes the underlying table. You create definitions for most dimensions using the New Dimension Wizard. When the dimension is a time dimension, however, you can speed the process by using the New Time Dimension Wizard instead. This section describes how to create definitions for a dimension using the New Dimension Wizard and a Days dimension using the New Time Dimension Wizard. This section also shows you how to modify the definitions using their respective property sheets. When you create a definition for a dimension, you enter detailed information about the dimension into a series of pages for the New Dimension Wizard. Most of this information is about the dimension object. Table 31 summarizes the kind of information solicited by each page of the wizard (both wizards have the same set of pages). Defining a dimension table begins from the Navigation Tree in the Warehouse Module Editor. To display the navigation tree: 1. Open the Project 2. Fully expand its navigation tree 3. Double-click the modules name in the expanded tree Table 31 The New Dimension Wizard Pages Page Details Example 1 Name of the dimension table Days 2 Levels of aggregation YearL, QuarterL, MonthL, WeekL, DayL 3 Attributes within each level MonthL: day_of_month, month_desc 4 Name of each hierarchy Calendar, Fiscal 5 Order of levels within each hierarchy YearL > QuarterL > MonthL Creating Dimension Definitions 3-12 Oracle Warehouse Builder 3i Users Guide Defining Rules for Dimension Objects When you create a definition for a dimension, you create two definitions: one for a dimension object and the other for a dimension table. Dimension objects are subject to a few rules which are described in this preliminary section. About Levels and Hierarchies Dimension objects consist of a set of levels and a set of hierarchies defined over those levels. The levels represent "levels of aggregation" and hierarchies describe parent-child relationships among a set of levels. For example, city is the child of state because you can aggregate city values into a state value. For example, a typical calendar dimension could contain five levels and two hierarchies could be defined on these levels: H1: YearL > QuarterL > MonthL > WeekL > DayL H2: YearL > WeekL > DayL The hierarchies are described from parent to child, so that Year is the parent of Quarter, Quarter the parent of Month, and so forth. About Unique Key Constraints When you create a definition for a hierarchy, OWB creates an identifier key for each level of the hierarchy and a unique key constraint on the lowest level. OWB uses the identifier keys during the generation phase to build a DDL script to create the dimension object. Subsequently, when you create a foreign key reference constraint on a fact table that points to a dimension, OWB shows the unique key constraint as well as the other identifier keys as candidate for the referenced column. A fact table can reference, however, only the lowest level of a hierarchy: namely, the only level within the hierarchy with a unique key constraint. Select any other level and the definition is invalid. Creating Dimension Definitions The Target Modules 3-13 About Mixed Levels of Aggregation An application can require two hierarchies that start at different levels of aggregation. For example, three hierarchies where: H1: YearL > QuarterL > MonthL > WeekL > DayL H2: YearL > WeekL > DayL H3: YearL > QuarterL > MonthLowL To model this mixed case using OWB: s The dimension table must contain the additional column MonthLow. s The MonthLow column must be populated with unique values. s A separate MonthLow level must be defined for the dimension. For this set of hierarchies, OWB generates six level identifiers and two unique key constraints. One unique constraint is defined on the Days column and the other on the MonthLow column. Because DayL and MonthLowL are at the bottoms of their respective hierarchies, they can serve as targets of a foreign key references. Denormalized Dimensions OWB generates a dimension as a single denormalized table with a set of levels and hierarchies defined on that table. Each level can have an arbitrary number of columns. Not valid as targets for a foreign key reference Creating Dimension Definitions 3-14 Oracle Warehouse Builder 3i Users Guide Creating A Dimension Definition To create definitions for this dimension, you need several specific details about the table and its dimension object. Specifically, you need to name the Dimension table and define a primary key constraint on its warehouse key column. When you define each column, Oracle recommends setting the constraint to NOT NULL to prevent inconsistent result sets and to maximize the number of query rewrites. You also define the dimension hierarchy and its levels of aggregation. Table 33 describes an example dimension table with each level of aggregation, a prefix for each level, and the attributes defined on each level. The levels occur in parent to child order: class is the parent of family, and family the parent of product. Table 32 OWB Rules for Dimension Objects Rule Description Denormalized A generated dimension object is defined on a single table. OWB does not currently support the definition of a dimension object on a set of normalized tables. Functional Dependence Child values must uniquely determine their parents value. For example, a city determines a state, a month determines a quarter, and a product determines a brand. These relationships must obtain in the physical data else queries can return incorrect result sets. Unique Key Generation OWB generates a unique key constraint only on the lowest level of a hierarchy. Foreign Key References A table can reference only the lowest level of a hierarchy (where the unique key constraint is always defined). Table 33 An Example of a Dimension Object Level Prefix Attribute Data Type Description class cl class_id number Level identifier or key class_desc varchar(20) Description of product class family fa family_id number Level identifier or key family_desc varchar(20) Description of product family product pd prod_WH number Base level or warehouse key Creating Dimension Definitions The Target Modules 3-15 To create a dimension definition, open the warehouse module editor, select the warehouse module and then fully expand its navigation tree. To create a dimension definition 1. Right-click, then select Dimensions > Create Dimension. OWB returns the welcome page for the New Dimension Wizard. 2. Click Next. The wizard returns its Name page. 3. Provide information as required, including: s The name of the underlying table. The wizard automatically generates a unique name for the dimension object. s A prefix. Every hierarchy is anchored on a base level of aggregation, and the wizard always generates a Unique Key constraint on the column that serves as the key column for the base level. The wizard uses the table prefix to generate a unique name for the UK constraint. If you leave the prefix blank, OWB uses the tables name as the default. s A description of the dimension. item_desc varchar(35) Description of the product product_upc varchar(11) Universal product code (natural key) item_source varchar(30) Supplier for product packaging varchar(20) Packaging for the product Table 33 An Example of a Dimension Object Level Prefix Attribute Data Type Description Creating Dimension Definitions 3-16 Oracle Warehouse Builder 3i Users Guide The description field is optional but should be used to fully document warehouse objects. 4. To name the dimension: enter the dimensions name, a prefix , and a useful but concise description. 5. Click Next. The wizard returns its Levels page. Use this page to explicitly name levels of aggregation known to exist in the actual data. These levels are used in a subsequent page to explicitly define the dimensions hierarchies. To define a level of aggregation, enter the following information: 1. The name of the level: Product, Family, or Class 2. A prefix for the level: pro, fa, or cl (the default is the name of the level) 3. A brief description of the level 4. After you finish entering the information for a level, click Add. The wizard stores the information about the level and clears the text boxes. Continue this process until you have defined each level of aggregation. Prefixes are useful because they: Note: Dimensions contain at least one level of aggregation. You can define a dummy or default level of aggregation to satisfy this requirement and include additional levels as required. Creating Dimension Definitions The Target Modules 3-17 s Reduce the number of attributes you need to enter manually: The wizard automatically generates an ID attribute for each level and assigns it the name levelprefix_ID. s Allow you great flexibility in attribute names: you can reuse attribute names. This is a common practice when you build dimensions for higher levels of aggregation. 6. Click Next. The wizard returns its Level Attributes page. Use this page to completely define each levels attributes. A level can have one or more attributes, and the wizard always generates an ID attribute for each level. The ID attribute for a level is special: it serves to identify the level, or you could say the attribute is the levels logical key column. This attribute is used in the CREATE DIMENSION statement to define the level, and the defined level is used in the statements DETERMINES clause to specify all the other columns within that level (dependent columns). See the Oracle8i/9i SQL Reference and the Oracle8i/9i Data Warehousing Guide for a complete discussion. You can use the default name prefix_ID or change ID to something else using the Update button. The following procedure shows you how to define an attribute, including its aggregation level, data type, and data type parameters. Note: The dimension prefix is used to form the names of level unique keys. Once a unique key name is generated, changing the dimension prefix does not change the names of existing levels because fact foreign keys may already refer to the generated level. If you change a dimension prefix, however, any unique key name generated subsequently will use the new prefix. Creating Dimension Definitions 3-18 Oracle Warehouse Builder 3i Users Guide To define an attribute: 1. Click the down-arrow to select a level of aggregation. 2. Enter a name for the attribute. 3. Select a data type for the attribute from the drop-down list under Data Type. OWB supports the following Oracle8i/9i data types: * CHAR * DATE * FLOAT * NUMBER * VARCHAR * VARCHAR2 4. Enter the precision, length, or scale as appropriate for the data type and your application. 5. Click Add or Update. The wizard records this information in the warehouse module and clears all of the text boxes. You can now define another attribute for the active level, or you can change to another level and define its attributes. Continue this process until you have defined all the attributes for each level. The following procedure shows you how to rename the ID column and set the Hierarchical levels. Creating Dimension Definitions The Target Modules 3-19 To rename the ID column: 1. Select ID in the Level Attributes text box s Enter a new name in the Name text box s Enter a brief description s Click Update. 2. Click Next. The wizard returns its Hierarchies page. 3. Define the hierarchy. s Enter a name and prefix for each hierarchy. s Enter a concise description of the hierarchy. 4. Click Next. The wizard returns its Levels Relationships page. Creating Dimension Definitions 3-20 Oracle Warehouse Builder 3i Users Guide Use this page to order the levels that participate within each hierarchy. 5. Define the levels within a hierarchy: s Click the down-arrow and select a hierarchy from the drop-down list. s Move the names of levels that participate in a selected hierarchy from Available Levels to Selected Levels. s Order the levels so they reflect the parent to child order. 6. Click Next. The wizard returns its Finish page. Verify the description. 7. Click Finish. The wizard creates a definition for the dimension, stores the definition in the warehouse module, and inserts its name in the navigation tree. To confirm this action, fully expand the Warehouse Module editor s navigation tree. The name of the dimension now occurs under DIMENSIONS. The wizard always generates a unique key constraint for a dimension table, and this constraint is always on the ID column that represents the dimensions base level of aggregation. Dimensional designs often call for a PK (primary key) rather than a UK constraint. After you complete a definition for a dimension, you can change the Creating Dimension Definitions The Target Modules 3-21 UK to a PK constraint. See "Changing Key Constraints" on page 3-44 for more information on how to change the generated UK constraint to a PK constraint. Updating Dimension Definitions You can update the definition for a dimension object with the Dimension Editor or by editing entries in the definitions property sheet. Using The Dimension Editor To update a dimension object, right-click its name in the navigation tree and select Edit from the pop-up list. OWB returns the Dimension Editor in a separate window. Creating Dimension Definitions 3-22 Oracle Warehouse Builder 3i Users Guide This editor displays its tool palette and the dimension object. To add an element to the dimension object, drop an icon from the tool palette onto a one of the dimension elements. The tool palette contains three icons: Attribute, Level, and Hierarchy. To add an attribute to a level: 1. Fully expand the Level where you want to add the new attribute. 2. Drop the Attribute icon onto the Level. OWB adds a dummy attribute (attribute1) in the level with data type of Number. 3. Enter a name for the attribute. 4. To declare a different data type, double-click the attributes name. The Dimension Editor returns the dimension objects property sheet. 5. Select the Level Attributes tab. The Dimension Editor returns the Level Attributes sheet. 6. Change the data type using the drop-arrow list. After you select a data type from the drop-down arrow list, you may need to change the Length, Scale, or Precision depending on the data type selected. To print the diagram, click the Print icon on the Dimension Editor s toolbar. Using The Property Sheets You can update the definition for a dimension table by editing entries in its property sheet. While you cannot edit a dimension tables definition using the Dimension Editor, you must open the Dimension Editor before you can open the property sheet for its definition.Each definition stored in a OWB repository has an associated Property Sheet, and you can update the definition directly by editing its property sheet. Creating Dimension Definitions The Target Modules 3-23 To display the property sheet for a dimension object from either the Warehouse Module Editor or the Dimension Editor: Select Edit > Properties or click the Properties icon You can also display the sheet from the Warehouse Module Editor s navigation tree: Select Products > Properties In all three cases, OWB displays the property sheet for the dimension object: The property sheet for a dimension object has five tabs: s Name s Levels s Level Attributes s Hierarchies s Level Relationships You can edit this sheet, delete items, or use an Add button to include new items. To display the property sheet for a dimension table: 1. Open the Warehouse Module Editor and fully expand the branch that contains the names of dimensions. Creating Dimension Definitions 3-24 Oracle Warehouse Builder 3i Users Guide 2. Select the dimension name from the navigation tree. 3. Select Edit > Edit Object from the menu. OWB returns the Dimension Editor. 4. Select Edit > Table Properties from the menu. OWB returns the dimension tables Property Sheet. The property sheet for the dimension table contains four tabs: s Name s Columns s Constraints s Attribute Sets Creating Dimension Definitions The Target Modules 3-25 The following examples show you how to order columns within a tables definition, change a UK to a PK constraint, create a check constraint, and create attribute sets. Ordering the Columns By default, the column names within a tables property sheet are ordered alphabetically. You can reorder a column by selecting it in the gray box to the left of the column name and then drag the row to the up or down the list depending on where you want the column to be in the physical order. After re-ordering the column names, the names with position numbers occur first in ascending order followed by the remaining names in alphabetic order. The order of column names in the property sheet is significant because this ordering is propagated to the DDL script that OWB generates to create the table. The default ordering could result in problems when an application is sensitive to the order of columns in a table. Also, you can order the columns so that a OWB generated concatenated key (for a fact table) corresponds to expected query usage. This ordering can greatly influence query performance. To assign a position number: 1. Open the tables Property Sheet. 2. Select the Columns tab. OWB returns the information on all columns defined in the table. Creating Dimension Definitions 3-26 Oracle Warehouse Builder 3i Users Guide 3. Enter numbers in the Position column. The editor orders the columns with position numbers first (increasing value) and the columns without position numbers second (alphabetic). You can order the following warehouse objects: s Dimensions s Facts s Tables s Views (conventional and materialized) To change a position number, edit the property sheet. Changing a Constraint OWB automatically generates a UK constraint for each Level defined within a dimension. The generated constraint is always on the column defined as the Levels ID column, and except for the lowest-level, these constraints are purely "logical constraints" which are not used when DDL is generated to create the table. You can edit these constraints but you cannot remove them. If you attempt to remove one of these constraints, OWB returns an error message. Figure 31 Generated UK Constraints When you declare a PK or UK constraint, you cannot later change it to a FK or CK constraint: you must drop the constraint and create a new one. Similarly, if you Creating Dimension Definitions The Target Modules 3-27 create an FK constraint, you cannot change it to any other kind of constraint: you must drop the constraint and create a new one. OWB automatically generates a UK rather than a PK constraint on the column that defines the lowest level of aggregation for a dimension table. To change a constraint type: 1. Open the tables Property Sheet. 2. Select the Constraints tab. OWB returns the information on all constraints defined in the dimension table. 3. Click the drop-down arrow under the Type column and select Primary Key. 4. Change UK in the constraint name to PK. Note that the editor only allows you to select a UK or PK constraint. 5. Change the name of the constraint to reflect this change. 6. Click OK. OWB updates the definition of the underlying table. Note: You cannot remove a generated UK constraint or column for a dimensions Level ID. Creating Dimension Definitions 3-28 Oracle Warehouse Builder 3i Users Guide Adding Check Constraints You can add a check constraint to any of a tables columns. These constraints typically enforce business rules on values stored in a dimension, fact, or table but can also be used to define a not null constraint. However, using the Add Check Constraints feature may slow your load performance. To add a check constraint: 1. Open the dimension tables Property Sheet and select the Constraints tab. OWB returns information on all the constraints defined in the dimension table. 2. Click the Add button. OWB inserts a row in the sheets top pane. 3. Enter a name for the constraint, select Check Constraint, and enter a condition. 4. Click OK. OWB stores the CHECK constraint in the property sheet. A CHECK integrity constraint requires that a condition be true or unknown for every row of the table. If a statement causes the condition to evaluate to false, then the statement is rolled back. The condition of a CHECK constraint has the following limitations: s The condition must be a Boolean expression that can be evaluated using the values in the row being inserted or updated. Creating Dimension Definitions The Target Modules 3-29 s The condition cannot contain subqueries or sequences. s The condition cannot include the SYSDATE, UID, USER, or USERENV SQL functions. s The condition cannot contain the pseudocolumns LEVEL, PRIOR, or ROWNUM. For additional information on CHECK constraints, see the Oracle8i/9i Application Developers Guide - Fundamentals. Creating Attribute Sets Each table in the warehouse module has a predefined attribute set consisting of all the tables columns and another attribute set for each constraint. You can create additional named attribute sets containing any number of the tables columns in the order you specify. A table can have three types of attribute sets: s Predefined: attribute sets generated by OWB. You cannot modify or delete these attribute sets, but OWB will delete an attribute set for a user-defined constraint if you delete that constraint. s User-defined: optional attribute sets that you can create, modify, or delete in the Table Properties sheet. s Bridge-type: an optional attribute set that can be transferred across a bridge for viewing in another software program. You can create, modify, or delete a bridge-type attribute set in the Table Properties sheet. A table can have only one bridge-type attribute set. All three types of attribute sets can be used in mappings. To add a named attribute set: 1. Open the dimension tables Property Sheet and select the Attribute Sets tab. Notes: s The column name referenced in the condition of the CHECK constraint must exactly match a physical name defined for the table in the tables Property Sheet. s OWB does not check the syntax of the condition. This can result in problems during the code generation phase when OWB generates a Create Table statement to deploy the table. Creating Dimension Definitions 3-30 Oracle Warehouse Builder 3i Users Guide OWB returns information on all the attribute sets defined in the dimension table. 2. Click the Add button. OWB inserts a row in the sheets top pane. Creating Dimension Definitions The Target Modules 3-31 3. Enter a name for the attribute set, select USER_DEFINED or BRIDGE_TYPE, and enter a description in the appropriator columns. 4. In the Table Properties sheets bottom pane, click the checkbox for each column (attribute) you want to include. The order in which you select the columns determines their initial order in the attribute set. (This order appears after you click OK to create the attribute set and then reopen the Table Properties sheet.) To change this order, drag and drop the columns to new locations. Creating Dimension Definitions 3-32 Oracle Warehouse Builder 3i Users Guide 5. If you selected BRIDGE_TYPE, click the Advanced... button to specify additional properties of the bridge-type attribute set. (The Advanced... button is enabled only for a bridge-type attribute set.) OWB returns the Advanced Attribute Set Properties sheet. Note: When you view an attribute set that was created previously, the attributes that are not selected appear after the selected attributes in their original order. Changing the order of non-selected attributes has no effect after you close the Table Properties sheet. Creating Dimension Definitions The Target Modules 3-33 For each attribute in the bridge-type attribute set, specify the properties: s Aggregation: Select SUM, MIN, MAX, AVG, COUNT, or DETAIL for no aggregation. The default is MIN. s Position: Select DATA POINT, PAGE, SIDE, TOP, or TOP/ SIDE. The default is PAGE. s Item Class: Check for TRUE or no check for FALSE. The default is FALSE. s Heading: Enter the heading text. s Format: Enter the text for the format field. s Custom: Enter the text for the custom field. You can use this field as an escape extra attribute. After you have specified the properties for each attribute, click OK. 6. Click OK. Using the New Time Dimension Wizard OWB uses the definition created by the Time Dimension Wizard to generate an SQL insert statement that can be used to populate the dimension. This eliminates the need for a data source, the definition of a mapping of the source to the dimension, and load routines to extract and load data. To set the start and end dates for the statement that generates the data, you must configure the dimension (see "Configuring Dimensions" on page 8-11 for more information). This section describes how to use the New Time Dimension Wizard. how to generate and execute an insert statement that populates the dimension with values. Creating Dimension Definitions 3-34 Oracle Warehouse Builder 3i Users Guide With the New Time Dimension Wizard, you enter information similar to that in the New Dimension Wizard (see Table 31 on page 3-11). About the New Time Dimension Wizard The Time Dimension Wizard uses predefined names and prefixes for levels, hierarchies, and attributes to speed up the process. The attributes also have predefined data types. When you call the wizard, you simply check off the elements your dimension requires. You can also define custom hierarchies within the wizard. After you create the definition, you can edit its property sheet to tailor the definition to your own specific needs. Creating a Time Dimension Definition The following procedure describes how to set up a time dimension definition. From the Warehouse Module Editor, open the appropriate warehouse module and then fully expand its navigation tree. 1. Select Dimensions > Create Time Dimension. OWB returns the welcome page for the New Time Dimension Wizard. 2. Click Next. The wizard returns its Name page. This page solicits: s The name of the underlying table. The wizard automatically generates a unique name for the object. s A prefix. s A description of the dimension. The role of the prefix is described on page 3-16. 3. Click Next. Note: Time dimensions differ considerably and many designs depend on multiple time dimensions. The New Time Dimension Wizard covers a limited number of cases. If the wizard doesnt meet your needs closely, use the New Dimension Wizard instead. Creating Dimension Definitions The Target Modules 3-35 The wizard returns its Levels page. This page contains several predefined levels of aggregation and a prefix for each level. 4. Check the levels of aggregation required to support your dimensional model. 5. Click Next. The wizard returns the Level Attributes page. This page contains a set of predefined attributes for each level. s Click the down-arrow and select a level of aggregation. s Check the required attributes for that level. Continue this process until attribute sets are selected for each level. 6. Click Next. The wizard returns the Hierarchies page. This page contains a set of predefined attributes for each level. down-arrow Creating Dimension Definitions 3-36 Oracle Warehouse Builder 3i Users Guide 7. Check the predefined hierarchies required by the time dimension. 8. Add custom hierarchies required by the time dimension. To add a custom hierarchy, click Add, name the hierarchy, and briefly describe the hierarchy. After the definition has been created, you can change the name by editing its property sheet. 9. Click Next. The wizard returns the Level Relationships page. Use this page to define the levels for each custom hierarchy. order the levels within each hierarchy. To define the levels, move the levels name from the Available Levels box to the Selected Levels box. The wizard automatically orders the levels. 10. Click Next. The wizard returns the Finish page. Verify the levels, attributes, and hierarchies and make corrections to individual elements (use the Back button to navigate). 11. Click Finish. The wizard stores the definition in the warehouse module. Creating Fact Table Definitions The Target Modules 3-37 The New Time Dimension Wizard generates the attributes you select plus additional attributes and constraints. In addition to the selected attributes, the wizard generates: s An ID attribute for each level s A Smart_key attribute for the base level s A UK constraint on the ID attribute for each level, which you can modify but not remove (see the discussion in "Changing a Constraint" on page 3-26) s A PK constraint on the Smart_key attribute You can modify the names of these elements and change the constraint type. The predefined names for the levels, attributes, and hierarchies in the Time dimension differ from those in other dimensions. You can change these names quickly by editing the dimension objects property sheet. For additional information on editing a dimension object, see the discussion on page 3-21. Defining a Sequence Object A sequence object populates the warehouse key column for a dimension. To define a sequence object, you use the New Sequence Wizard to name the sequence. You can configure these settings for use in creating ETL mappings. These settings are described in "Adding Sequences" on page 6-30. To define a sequence object: 1. Select SEQUENCE > Create Sequence. 2. Enter a name and description for the sequence in the appropriate fields. 3. Click Finish. Creating Fact Table Definitions This section shows you how to create and update a definition for a fact table. You create a definition for a fact table using the New Fact Table Wizard, and you update the definition by editing its property sheet. You can also import definitions for tables from another database source or an Oracle Designer Repository. When you create a definition for a fact table, you use the New Fact Table Wizard to enter detailed information about the table using a series of pages. This information includes details regarding foreign key references, measures, and the data types of all Creating Fact Table Definitions 3-38 Oracle Warehouse Builder 3i Users Guide the tables columns. Table 34 summarizes the kind of information to enter into each wizard page..
To create a fact table definition, open the warehouse module editor, select the warehouse module and then fully expand its navigation tree. To display this navigation tree: 1. Open a project 2. Fully expand its navigation tree 3. Double-click the modules name in the expanded tree. Creating a Definition for a Fact Table This section describes how to create a fact table definition. From the Warehouse Module Editor, open the warehouse module and expand its navigation tree. To create a fact table definition: 1. Select FACTS > Create Fact. OWB returns the welcome page for the New Fact Wizard. 2. Click Next. The wizard returns its Name page. Into the appropriate fields, enter s The name of the fact table s A description of the fact table The description field is optional but should be used to fully document warehouse objects. 3. Enter the fact tables name and a description into the appropriate fields. Table 34 New Fact Table Wizard Pages Page Details Examples 1 Name of the fact table Sales, Telephone Calls, Poll Answers 2 Define Foreign Keys Days, Products, Customers, Channels 3 Define Measures Dollars, Units, Cost, Margin Creating Fact Table Definitions The Target Modules 3-39 4. Click Next. The wizard returns the Define Foreign Keys page. Use this page to explicitly define each of the tables foreign key references. To define a foreign key reference: 1. Click the first down-arrow to select the name of a dimension (Days). 2. Click the next down-arrow to select the base or warehouse key Level (dayL). 3. Click the third down-arrow to select the Primary Key column constraint defined on the dimension. 4. Click Add. The wizard generates a name for the foreign key reference constraint and inserts a line in the text box that lists the tables foreign keys. Repeat this four-step procedure to define a foreign key constraint for each dimension referenced by the fact table. Always choose the lowest level of aggregation for the foreign key reference target. Note: s The Fact Wizard displays the name for each generated PK constraint on a dimensions level columns. Only the lowest level PK constraint is an actual physical constraint. s You cannot modify the name or data type of the foreign key reference columns. You can only do this by editing the definition for the referenced table. s If you add a column to a PK or UK constraint on a dimension, you must also update the fact table foreign key references. Creating Fact Table Definitions 3-40 Oracle Warehouse Builder 3i Users Guide You can modify the name of the generated foreign key reference constraint to a more meaningful one: select the generated name and type over it. The name must be unique within the Project. 5. Check the box at the bottom of the page that says "Create segmented unique key automatically from foreign keys." 6. Click Next. The wizard returns the Define Measures page. Use this page to define each of the fact tables measures. To add a measure: s Click Add. s Enter its name. s Set its data type. Continue these steps until all of the fact tables measures have been defined. 7. Click Next. The wizard returns the Attribute Sets page. Create attributes as described in "Creating Attribute Sets" on page 3-29. 8. Click Next after you have created Attribute Sets. The wizard returns the Finish page. This page describes the fact table. Verify the description and click Back if you need to modify any of the elements. 9. Click Finish. Creating Fact Table Definitions The Target Modules 3-41 The wizard creates a definition for a fact table, stores it in the warehouse module, and inserts its name in the warehouse modules navigation tree. You can confirm this by fully expanding the editor s navigation tree. The name of the dimension occurs under the FACTS branch. Updating a Fact Table Definition A fact objectlike a dimension objecthas two property sheets: one for the Fact Object and another for the Fact Table. You can update a fact objects properties by editing entries in these property sheets. In addition, you can add foreign key references or measures to a fact object using the Fact Editor. You can also use to Fact Editor to display a fact objects property sheets as well as all the property sheets of the dimensions it references. Using the Fact Editor The Fact Editor displays a diagram of a fact object and its dimensions. To edit the Sales fact table using this editor, right-click the name of the fact table in the Warehouse Module Editor s navigation tree and then select Edit from the pop-up menu. OWB returns the Fact Editor in a new window. This editor returns its tool palette and a diagram of the fact table and its dimensions. To print the diagram, click the printer icon on the editor s tool bar. Creating Fact Table Definitions 3-42 Oracle Warehouse Builder 3i Users Guide The fact table diagram is a live display. For example, drop one of the icons from the tool palette on the fact table, and the editor returns the welcome page for the Add Measure or Foreign Key References Wizard. To edit a Fact tables properties, right-click on the table object in the Fact Editor or select Fact > Fact Properties from the Fact Editor menu. The editor returns a pop-up list that contains several selections. Select Properties from the pop-up list and the editor returns a property sheet that describes the objects properties. These properties include the objects name and description, its foreign key references, its measures, and its attribute sets. Click the Foreign Keys tab to display a list of all the objects foreign key references. From these sheets, you can: s Change the name and description of the object. s Add or Remove a foreign key reference constraint. Creating Fact Table Definitions The Target Modules 3-43 This panel shows all of the UK constraints defined on a dimension: the base level of aggregation and each higher level of aggregation. OWB generates DDL only for the constraint defined on the base level of aggregation. s Change the name of a foreign key reference constraint. s Add, Remove, or edit a measure (name, data type, and description). You edit the tables property sheet if you want to change its physical properties. Select Table Properties from the pop-up list and the editor returns the property sheet that gives a view of the underlying fact table. The property sheet describes the fact table according to its constraints and columns. From these sheets, you can: s Change the name and description of the object. Creating Fact Table Definitions 3-44 Oracle Warehouse Builder 3i Users Guide s Add a new column, edit the name, data type, column position, and description of an existing column, or Remove a column. For a description of the Position column, see discussion on page 3-25. s Add, Remove, or edit a foreign key reference constraint. s Add, Remove, or edit Attribute Sets If you click a dimension instead of the fact, the Fact Editor returns a pop-up list specific to dimensions. From this list, you can display the property sheets for the selected dimension and also call the Dimension Editor. Changing Key Constraints The Oracle8i/9i server can automatically refresh materialized views provided that a materialized view log file is created for the underlying fact table. You cannot build the log file, however, unless the fact table has a PK constraint. To change the composite UK constraint defined on a fact table to a PK constraint, display the Constraints property sheet for the fact table. From this sheet: Note: You can change a UK constraint to a PK constraint or vice-versa; however, you cannot change an FK constraint to another kind of constraint. You must first remove the constraint and create a new one. Creating Fact Table Definitions The Target Modules 3-45 s Select the generated segmented key name. s Select Primary Key from the down-arrow for the Type column. s Change the name of the constraint to reflect its new property. s Click OK. Importing Definitions If a warehouse module is configured for connection with an Oracle8i/9i database instance or with an Oracle Designer Repository, you can import definitions for tables, Views and Sequences using the Import Metadata Wizard. To import definitions from an Oracle8i/9i database or an Oracle Designer Repository: 1. Configure the warehouse module to connect with the source. 2. Open the warehouse module. 3. Select Module > Import from the Warehouse Module Editor s toolbar. OWB returns the welcome page for the Import Metadata Wizard. 4. Click Next. The wizard returns its Filter Information page. Creating Materialized View Definitions 3-46 Oracle Warehouse Builder 3i Users Guide See "To import definitions from an Oracle database application:" on page 4-17 for a discussion of wizards subsequent pages. Creating Materialized View Definitions This section shows you how to create and update definitions for materialized views. You create a definition for a materialized view using the New Materialized View Wizard. You modify a views definition by editing its property sheet. Both views summarize sales by date. Together these views support a wide variety of query rewrites that speed up query execution. The query rewrites depend on the hierarchies and not null constraints defined on the referenced dimensions. Designer Repository Notes: You cannot import: s Dimension Objects. s Position will be renumbered as 10, 20, 30, . Creating Materialized View Definitions The Target Modules 3-47 When you create a definition for a materialized view, the New Materialized View Wizard solicits detailed information about the view through a series of pages. This information includes details regarding foreign key references, columns, and their data types. Table 35 summarizes the information solicited by each page. The starting point for each example is the navigation tree for the warehouse module. To display this navigation tree: 1. Open the project 2. Fully expand its navigation tree 3. Double-click the modules name in the expanded tree. For more details, see the example on page 3-6. Creating A Materialized View Definition Table 36 describes the columns and foreign key references defined for the materialized view. This view also has a composite primary key constraint defined on its foreign key reference columns.
To Create a Materialized View Definition: Table 35 New Materialized View Wizard Pages Page Details Examples 1 Name and description Customer_Summaries, Product_Summaries 2 Aliases for each column Total_Sales, Total_Cost, Total_Units 3 Query that defines the view Customer_Summaries, Product_Summaries 4 Constraints on the view Days, Products, Customers, Channels Table 36 An Example Materialized View Alias Data Type Description Constraint Name day_WH number(38) FKR to Days dimension dl_day_PK product_WH number(38) FKR to Products dimension po_product_PK sales number(36, 2) Daily subtotal cost number(36, 2) Daily subtotal units number(38) Daily subtotal Creating Materialized View Definitions 3-48 Oracle Warehouse Builder 3i Users Guide 1. Open the warehouse module and expand its navigation tree. 2. Select Materialized View > Create Materialized View. OWB returns the welcome page for the wizard. 3. Click Next. The wizard returns its Name page. Enter s The name of the view s A description of the view The description field is optional but should be used to fully document warehouse objects. 4. Click Next. The wizard returns its Columns page. Use this page to explicitly define each of the materialized views columns (or an alias for each column). To define a column: 1. Click Add. 2. Enter the columns name. 3. Set the columns data type. Repeat this procedure for each column. 5. Click Next. The wizard returns its optional Query Text page. When you create the definition for a materialized view, you have two options; you can: s Enter the text for a query that defines the view. s Leave the Query Text box empty. Then later in the development (during the mapping stage) you can create a mapping that defines the necessary query. The following example includes the query text, and Chapter 6, "Mapping Operators and Transformations" shows you how to create a mapping to define the query text. Creating Materialized View Definitions The Target Modules 3-49 Click Next. Usage Notes: s OWB does not generate code for a view if query text is not included in its Property Sheet nor if it has no columns defined. s OWB generates a Create Materialized View statement to deploy the view even if its syntax is invalid: OWB does not check the syntax of the select statement used to define a view. s Manually entered queries are automatically completed with a semi-colonadding one will cause a syntax error. Creating Materialized View Definitions 3-50 Oracle Warehouse Builder 3i Users Guide The wizard returns its Define Constraints page. To define a foreign key constraint: 1. Click Add adjacent to the upper box and enter the constraints name. 2. Select the constraint type: Foreign Key. 3. Select the constraints: * Referenced Application (defined within a Warehouse Module) * Referenced table * Referenced key 4. Click Add adjacent to the lower text box and then select the column local to the view (click the down-arrow) that makes the foreign key reference. Continue these steps until all of the foreign key constraints have been defined. To define a PK or UK constraint: 1. Click Add adjacent to the upper box, then enter the constraints name and select its type. 2. Click Add adjacent to the lower text box and then select the local columns that participate in the constraint. Continue these steps until all of the PK and UK constraints have been defined. Creating Materialized View Definitions The Target Modules 3-51 6. Click Next. The wizard returns its Finish page. 7. Verify the description, and if you need to modify the definition, click Back. 8. Click Finish. The wizard creates a definition for the materialized view, stores this definition in the warehouse module, and inserts its name in the warehouse modules navigation tree. Confirm this by fully expanding the editor s navigation tree. The name of the materialized view now occurs under the MATERIALIZED VIEWS subtree. 9. Double-click the name of the materialized view to display a diagram of the materialized view. Updating a Materialized View Definition To update the materialized view, right-click its name in the navigation tree. OWB returns a pop-up menu. You can select Edit or Properties from the pop-up menu. Select Edit from this the pop-up menu and OWB returns the Materialized View Editor in a new window. This editor diagrams the materialized view and its references. Select Properties from the pop-up menu and OWB returns the materialized views property sheet. You can modify the views definition by editing the property sheet. For examples on editing a definition from its property sheet, see "Updating Dimension Definitions" on page 3-21. Renaming a Materialized View You can rename a materialized view or other warehouse object. For example, to rename a materialized view, right-click the views name and select Rename. OWB highlights the name and you can type over the old name. Creating Conventional View Definitions 3-52 Oracle Warehouse Builder 3i Users Guide Creating Conventional View Definitions This section describes how to create and update definitions for conventional views. You create a definition for a conventional view using the New View Wizard. You update the views definition by editing its property sheet. When you create a definition for a view you use the New View Wizard to enter detailed information about the view using a series of pages. This information includes details regarding foreign key references, columns, and data types of each column. Table 37 summarizes the kind of information solicited by each wizard page. The starting point for each example is the navigation tree for the warehouse module. To display this navigation tree: 1. Open the project module. 2. Fully expand its navigation tree 3. Double-click the modules name in the expanded tree. For more details, see the example on page 3-6. Creating A View Definition A view summarizes the daily sales of each product manufactured by Global Computing Company. The view contains data only for products that are within the Desk Top family. Table 36 describes the columns and foreign key references defined for an example view. This view also has a composite primary key constraint defined on its foreign key reference columns. Table 37 The New View Wizard Pages Page Details Examples 1 Name and description Customer_Summaries, Product_Summaries 2 Aliases for each column Total_Sales, Total_Cost, Total_Units 3 Query that defines the view Customer_Summaries, Product_Summaries 4 Constraints on the view Days, Products, Customers, Channels Creating Conventional View Definitions The Target Modules 3-53
Although this view has similar structure to a materialized view, the views differ as follows: s Function: A view restricts access to a single family of products; the materialized view makes queries run faster. s Visibility: A view is visible to a class of users; the materialized view and its operation are transparent to all users. s Physical Storage: A view occupies no storage space; the materialized view does. Although a view differs from a materialized view, the procedure to create a views is the same. To Create the Definition 1. Open the warehouse module and expand its navigation tree. 2. Select View > Create View. OWB returns the welcome page for the wizard. 3. Click Next. OWB returns the wizards Name page which solicits: s The name of the view s A description of the view Complete the description box to document the view. 4. Click Next. Table 38 An Example of a Conventional View Alias Data Type Description Constraint Name day_WH number(38) FKR to Days dimension dl_day_PK product_WH number(38) FKR to Products dimension po_product_PK sales number(36, 2) Daily subtotal cost number(36, 2) Daily subtotal units number(38) Daily subtotal Creating Conventional View Definitions 3-54 Oracle Warehouse Builder 3i Users Guide The wizard returns its Columns page. Use this page to explicitly define each of the materialized views columns (or an alias for each column). To define a column: 1. Click Add. 2. Enter the columns name. 3. Enter the columns data type. Repeat this procedure for each column. 5. Click Next. The wizard returns its optional Query Text page. When you create the definition for a view, you have two options. You can: s Enter the text for a query that defines the view in the Query Text box. s Leave the Query Text box empty. Then, during the mapping stage, you can create a mapping that defines the necessary query. The following example shows you how to include the text; Chapter 6, "Mapping Operators and Transformations" shows you how to create a mapping to define the query text. Creating Conventional View Definitions The Target Modules 3-55 6. Click Next. The wizard returns its Define Constraints page. Use this page to define "logical constraints" for a view. These constraints may be useful when the view serves as a data source within a mapping. For example, if you use a view as a data source within a mapping, then the Mapping Editor can use the logical foreign key constraints to include its referenced dimensions as secondary sources in the mapping. Usage Notes: s OWB does not generate code for a view if its query text is not included in its Property Sheet nor if it has no columns defined. s OWB generates a Create View statement to deploy the view even if its syntax is invalid: OWB does not check the syntax of the select statement used to define a view. Creating Conventional View Definitions 3-56 Oracle Warehouse Builder 3i Users Guide Follow the procedure described on page 3-50 that shows you how to define foreign key reference constraints on a View. 7. Click Next. OWB returns the Finish page. 8. Verify the description, and if you need to modify the definition, click Back. 9. Click Finish. OWB creates a definition for the view, stores the definition in the warehouse module, and inserts its name in the warehouse modules navigation tree. Confirm this by fully expanding the editor s navigation tree. The name of the materialized view now occurs under the VIEWS subtree. 10. Double-click the name of the materialized view to display a diagram of the materialized view. Updating a View Definition To update the view, right-click its name in the navigation tree. OWB returns a pop-up menu. You can select Edit or Properties from the pop-up menu. Creating Conventional View Definitions The Target Modules 3-57 Select Edit from this the pop-up menu and OWB returns the View Editor in a new window. This editor diagrams the view and its references. Select Properties from the pop-up menu and OWB returns the views property sheet. You can modify the views definition from the property sheet by editing existing entries. For information on editing a definition from its property sheet, see "Updating Dimension Definitions" on page 3-21. Renaming A View You can rename a view or other warehouse object. To rename a view, right-click the views name and select Rename. OWB highlights the name and you can type over the old name. Click to change displayed position of the key columns. Click to change the displayed sort order. Neither of these actions alter the actual position in the generated DDL. Adding Transformations 3-58 Oracle Warehouse Builder 3i Users Guide Adding Transformations Transformations are pre-built PL/ SQL functions, procedures, package functions, and package procedures. Custom transforms are used to define an operation outside of the Oracle Library or a group of operations. You create a custom transform using the New Transform Wizard. The following sections describe how to create transformation categories and create definitions for transformations using the libraries or Expression Builder. About Transformations A transformation is a black box operation that changes the values of the data passing through it. It consists of one input parameter group and at least one output parameter group. The input or the output parameters of a transform is all the parameters in all its input or output parameter groups. A transform takes one row of data from its input parameters and produces one row of data to its output parameters. In other words, the cardinality of the output parameter groups is the same as that of the input parameter group. OWB supports the following transformation types: s User Transform PackageThis category is created by the user and contains package functions and procedures that are defined by the user. s Predefined TransformsThese categories exist in the Oracle Library and consist of a mix of built-in and seeded functions and functions and procedures. s Standalone FunctionsThis category is called FUNCTIONS and is automatically created in every warehouse module. This category contains any standalone functions used as transformations. These functions may be defined by the user or imported from a database. s Standalone ProceduresThis category is called PROCEDURES and is automatically created in every warehouse module. This category contains any standalone procedures used as transformations. These procedures may be defined by the user or imported from a database. Transform: Accepts source data as input, transforms the data, and returns the result to a column in a target object. Adding Transformations The Target Modules 3-59 s Imported PackageThis category is created by importing a PL/ SQL package. The package body may be modified. The package header (function/ procedure signatures) may not be modified. The package may be viewed in the transformation category property sheet. A function transform takes 0-n input parameters and produces a result value. A procedure transform takes 0-n input parameters and produces 0-n output parameters. About Transformation Parameters Most transformations have parameters. The input parameter specifies a source of data, the output specifies a result. More complex transformations often have multiple input, input/ output, and output parameters. About Oracle Transformation Libraries Each time you create a source or warehouse module, OWB creates a Transformation Library for that module containing transform operations. This library always contains the standard Oracle Library and an additional library for each module defined within the repository. Definitions for custom transforms are stored in a warehouse modules Transformation Categories branch, a branch that can be subdivided into one or more categories using the New Transformation Category wizard. These libraries are used to store definitions for custom transforms. Transformation Libraries consist of the following types: s The Global Shared Librarya collection of re-usable Transformations categorized as functions and procedures defined within your repository. s The Oracle Librarya collection of pre-defined functions from which you can define procedures for your Global Shared Library. Every time you create a target warehouse module, the transformations associated with the module will be considered an additional transformation library accessible only to that module. Transformations associated with mappings within the target module should be added to the warehouse modules Transformation Categories branch. Transformations that are to be shared between projects should be added to the Transformation Categories branch in the Global-Shared-Library. Adding Transformations 3-60 Oracle Warehouse Builder 3i Users Guide The Global-Shared Library Custom transforms are used to define an operation outside of the Oracle Library or a group of operations. You create a custom transform much like you would a new table for a mapping. Definitions for custom transforms are stored in a warehouse modules Global Shared Library in the Transformations branch, a branch that can be subdivided into one or more categories in the Module Editor. The Global-Shared Library houses those transformations that are shared across a repository. The default categories are s FunctionsThis category stores standalone functions. s ProceduresThis category stores standalone Procedures. The Oracle Library The Oracle Library includes a set of standard transforms organized into five categories. Expand the Oracle Library branch to display the categories. s Administration s Character s Conversion s Date s Numeric s Other s XML Accessing Transformation Libraries You can access the Transformation Libraries either from Expression Builder, the Add Transformation dialog, or from the New Transformation Wizard. You can also access Transformation Libraries from the navigation tree in the OWB Console. The methods of accessing the libraries are described in the following task descriptions. Creating Transformation Categories Definitions for custom transforms reside within a warehouse module, and you can create Transformation Categories to organize these definitions into packages. The following steps show you how to create transformation categories within the warehouse module using the New Transformation Category Wizard. To create a transformation category: Adding Transformations The Target Modules 3-61 1. Expand the navigation tree for the active warehouse module. 2. Select Transformation Categories > Create Transformation Category OWB returns the New Transformation Category Wizard welcome page. 3. Click Next. The wizard returns the Name page. 4. Enter a name and description for the category. A category name can have from 2 to 40 alphanumeric characters but no spaces. 5. Click Finish. The wizard inserts the categorys name the modules navigation tree. Defining Custom Transforms You can create new transformations using the New Transformation Wizard. Note by creating a new transformation , you are actually creating a custom transformation that requires code entry on your behalf.. You will need to be familiar with SQL and/ or PL/ SQL in order to complete this task. To define a custom Transform: 1. Open the Transformation node on the Navigation Tree in the Warehouse Module Editor 2. Select a transformation category as appropriate for the type of transformation you want to create. 3. Right-click on the category and select Create Transformation from the popup menu. OWB opens the New Transformation Wizard. 4. Enter a name in the Name field and a description documenting what the transform does. 5. Select a transform type from the Type pull down list. 6. Click Next. Adding Transformations 3-62 Oracle Warehouse Builder 3i Users Guide Figure 32 The Transformation Name Page 7. Define the Parameters for the transform on the Parameters page. s Click Add. s Enter a name for the Parameter into the Name column. s Specify the type; the order; whether it is an Input, Output, or Input/ Output parameter; and whether the parameter is required. You must specify values for each parameter. Adding Transformations The Target Modules 3-63 Figure 33 The Transformation Parameter Page 8. Click Next. 9. Enter the PL/ SQL code for the parameter on the Transformation Implementation page. Adding Transformations 3-64 Oracle Warehouse Builder 3i Users Guide Figure 34 The Transformation Implementation Page After values have been specified for each parameter, if necessary, use the Back button to navigate the wizard pages make corrections. 10. Click Finish. Editing Transformation Properties You can update the definition of a transform by editing its property sheet. The following procedure assumes you are in the Module Editor. Edit these property sheets carefully. If you change the name of an input parameter, then you will probably need to reflect this change in implementation syntax. Importing PL/SQL Packages Using the Import Wizard, you can import PL/ SQL functions, procedures, and packages into an OWB project.When OWB generates a script for the extract and load job, it will generate the added constraint within the PL/SQL routine that Adding Transformations The Target Modules 3-65 implements the mapping. At runtime, you can accept the default value or supply a different one. When you submit the job using Oracle Enterprise Manager, you can modify the runtime parameter value then. The following steps describe how to import PL/ SQL packages from other sources into OWB. To import a PL/SQL function, procedure, or package: 1. Select Module > Import... from the Warehouse Module Editor menu. Figure 35 The Import Selection On The Warehouse Module Editor Menu OWB returns the Import Metadata Wizard Welcome page. Click Next. 2. Select PL/SQL Transformation from the Object Type portion of the Filter Information window. Adding Transformations 3-66 Oracle Warehouse Builder 3i Users Guide Figure 36 The PL/SQL Transformation Selection 3. Click Next. The Import Metadata Wizard returns the Object Selection Page. Adding Transformations The Target Modules 3-67 Figure 37 The Object Selection Page 4. From the Object Selection page, select a function, procedure, or package from the database source tree within the available objects window. Move the object(s) to the selected objects side using either the > to move a single object, or >> to move multiple objects. 5. Click Next. The Import Metadata Wizard returns the Summary and Import page. Adding Transformations 3-68 Oracle Warehouse Builder 3i Users Guide Figure 38
6. The Summary Import window displays information regarding what it is importing. You may click Back to revise your selections. 7. Click Finish to import. The imported PL/ SQL information appears in the Module Editor under Transformation Categories. Expand imported function and package to see the Transformation information. Defining Business Areas The Target Modules 3-69 Figure 39 You may now use the imported transformations. You can proceed to generate the code and deploy. A few things to take note of when using the imported PL/ SQL: s You can edit, save, and deploy the imported PL/ SQL functions and procedures s You cannot edit imported PL/ SQL packages s Wrapped PL/ SQL objects are not readable s Imported packages may be viewed/ modified in the category property sheet. s You may edit the imported package body but not the imported package specification Defining Business Areas A business area is a logical grouping of data that satisfies specific data requirements for a particular business purpose. You can define one or more business areas within a warehouse module. These areas define "shortcuts" to a subset of the modules objects. The business tree displays a warehouse modules business areas, which contain links to the actual data in the logical tree. Defining Business Areas 3-70 Oracle Warehouse Builder 3i Users Guide Business areas are most useful when the logical warehouse contains a large number of objects. You should define business areas when you need to continually examine natural subsets of the warehouses objects or export subsets of objects to a decision support tool such as Oracle Discoverer or Oracle Express. For example, you could define a subset of warehouse objects called Products that contains links only those objects that pertain to product sales and another area called Customers that pertains to customer sales. Oracle Discoverer and Oracle Express use business areas to provide their users with access to the precise data they need for ad hoc queries, decision support, and presentation of results. You can use the Oracle CWM bridges to export business areas from OWB to Discoverer and Express. About Business Areas You can create multiple business areas, and these areas can share the same links. For example, the Products and the Customers business areas both include the Days dimension and the Sales fact tables. When you create a business area, you are creating links to existing objects and organizing them into identifiable subsets. Thus, for a given warehouse module, there is only one Sales table, one Days dimension table, and one Product_ Summaries materialized view although their names can occur in the navigation tree for a warehouse module and one or more business areas. After you create a business area, you can update its definitions using the standard Oracle Warehouse Builder editors, such as the Dimension, Fact, Table, and Mapping editors. You can also update an objects definitions by editing its property sheet. For additional information on updating definitions, see previous examples in this chapter. Note: When you create a business area, you do not create any new objectsthe business area only organizes existing objects into identifiable subsets or a more friendly view. Defining Business Areas The Target Modules 3-71 Creating a Definition for a Business Area To create a definition for a subset of warehouse objects in a business area: 1. Open the warehouse module. 2. Click the Business Tree tab. OWB returns the navigation tree for the business areas. 3. Right-click on the warehouse module and select Create Business Area from the drop-down menu. (Alternatively, use the keyboard command Ctrl + N.) OWB returns the Business Area Dialog panel. This panel contains three areas: s Text box for the business areas name. The name can be a logical or physical name. s Text box for the business areas description. s Names for each link to an object within the warehouse module and a corresponding check box. Defining Business Areas 3-72 Oracle Warehouse Builder 3i Users Guide Use the text boxes to name and describe the business area and then select the check box for each object that is to be linked within the business area. 4. Enter a name and description, and select the objects to include within the business area. You can select the check box for a type to select all objects of that type. 5. Click OK. OWB creates the business area and stores it in the warehouse modules business tree. Infrastructure Requirements The Target Modules 3-73 Infrastructure Requirements Before you can deploy a physical instance of a logical data warehouse, you need complete the following tasks: 1. Create a user name for the owner of the physical schema in an Oracle8i/9i database instance. 2. Create a set of tablespaces for the physical database objects. These must be defined before you can complete a configuration of a physical instance. 3. Deploy the OWB Runtime Libraries to the target schema. The first two tasks are Oracle8i/9i administration tasks, but the third is specific to OWB and is completed using the OWB Runtime Assistant. See the Oracle Warehouse Builder 3i Installation Guide for more information on the OWB Runtime Assistant. Committing Your Changes 3-74 Oracle Warehouse Builder 3i Users Guide Committing Your Changes OWB does not automatically commit your additions or changes to the repository, that is your responsibility. You can immediately commit your work or defer this action until you exit a OWB session. To commit your work immediately, click the Commit icon on the toolbar. OWB returns a Commit Confirmation panel: click Yes. You can also defer this action until you exit a OWB session. On exit, OWB always returns an Exit Confirmation panel: click Yes to commit your work. Taking the Next Step This chapter described how to create logical definitions for a data warehouse project using Oracle Warehouse Builder wizards and how to edit these definitions by using various a Warehouse Builder editors and property sheets. You are now ready to create the next set of definitions. These definition will describe data sources used to populate the data warehouse project. Chapter 4, "The Data Source Modules" will describe how to create definitions for database and flat file sources and how to create source modules in the OWB repository to store the definitions. Note: If you generate code at any time during your Warehouse OWB Session, Multi-User Locking feature will remain in effect until after you commit your changes. Warehouse Module: A place within a OWB project that organizes and stores definitions for a target warehouse schema. The Data Source Modules 4-1 4 The Data Source Modules This chapter describes how to create definitions for data sources using Oracle Warehouse Builder wizards. To create a set of definitions for a database source, you import the definitions from the source. You can import definitions from Oracle and non-Oracle database systems, and from Designer repositories. OWB enables you to define a source file structure by sampling data from flat files. This chapter also shows you how to modify definitions, generate and display diagrams for the source objects, and print diagrams. OWB stores definitions for data sources in containers called source modules. The first section of this chapter describes how to create source modules for different kinds of data sources. The subsequent sections describe how to import or create definitions for sources using OWB wizards. The closing section reviews the infrastructure requirements that must be satisfied before you can implement a physical instance of the logical warehouse. This chapter includes the following sections: s Creating Source Modules s Importing Definitions From Database Sources s Creating Definitions for Flat File Sources s Importing Definitions for Pure Extract and Pure Integrate s Infrastructure Requirements s Committing Your Changes s Taking The Next Step Creating Source Modules 4-2 Oracle Warehouse Builder 3i Users Guide Creating Source Modules OWB stores and manages the definitions for data sources in source modules. You create a source module for each source type. You can base your sources on a database system (Oracle or non-Oracle) or a set of operating system files. In addition, you can create a source module for an Oracle Designer repository. You create a source module using the New Module Wizard. Specifically, you can create source modules for: s A Database Application (Oracle, Oracle Designer Repository, and non-Oracle) s A Flat File Application When you create a source module, you must supply the wizard with the applications type (database or flat file) and the version or system type of its host. From this information, OWB determines which one of its software integrators can read data from the specified source. You must also supply information that allows OWB to access data sources. OWB accesses database sources over database links, which you must create. OWB accesses flat files over shared directories, which you must configure. You can update the definition of a source module at any time by editing its property sheet. About OWB Software Integrators OWB uses software integrators to read definitions and extract data from source systems. The New Module Wizard determines the correct integrator for a specific case from the sources type and its hosts version or system type. Configuring Connection Information for Database Sources When you create a source module for a database source, you must create a database link in the OWB repository that points to the source. OWB uses the link to access the sources data dictionary. You can create a database link by selecting the link name from the drop down menu and verifying the link owner, user name and connect string. You can also use the New Database Link dialog to create a new database link. The New Database Link Source Module: A place within a OWB Project that organizes and stores definitions for relational database sources and flat files. Creating Source Modules The Data Source Modules 4-3 dialog can be accessed from the wizards Connection Information page. Database links raise security issues and database administrators usually prefer to create and control them manually. The following examples show you how to create database links for Oracle and non-Oracle systems. Check only for a non-Oracle system Creating Source Modules 4-4 Oracle Warehouse Builder 3i Users Guide Using The New Database Link Dialog When you create a new database link, you must specify a name for the link, connection information, and a user name and password for the database system. You can specify the connection information as a connect string or specify individual values for a connect strings parameters. For more information on database links and connect strings, see Oracle8i/9i Distributed Database Systems Release 2 (8.1.7) and Oracle8i/9i SQL Reference Release 2 (8.1.7). After you create a database link, you can edit the link information in a module property sheet. These sheets differ according to whether the source is based on an Parameter Description DB Link Name A database link name can be a maximum of 128 bytes and can include periods (.) and the "at" sign (@). SQL*Net Connect String A connect string for the database system. Non-Oracle Database: If the system is a non-Oracle system, specify this by including (HS=OK) within the connect_data clause. Host Name Information Alternate specification of values for database link parameters Host Name Port Number Oracle SID Heterogeneous Services Alias for the IP address of the host machine. Configured port for the Oracle Listener. SID for an Oracle Instance or an Oracle Transparent Gateway. Check only if the link points to a non-Oracle database system. This includes systems accessed via ODBC, OLE DB or an Oracle Transparent Gateway. User Name & Password User name and password for the database system. Case sensitive names and passwords need to be double-quoted. Non-Oracle Database Systems: s If you specify an SQL*Net connect string, it must include (HS=OK) within its connect_data clause. s If you specify values for individual connect string parameters, check the box labeled "Use for heterogeneous services." Creating Source Modules The Data Source Modules 4-5 Oracle or non-Oracle system. To access this sheet, see the discussion in "Update a Source Definition" on page 4-26. After you create and test a database link, OWB stores the links properties in its repository. It then displays a Connection information window in which you can edit your Schema, select a different database link or define a new one, and select a source from which you can import your metadata. A database link does not have a property sheet of its own, but you can display a links properties by opening the property sheet of a source module that references the link. Also, you must add a schema using the Change Schema button. For information on opening a source modules property sheet, see "Update a Source Definition" on page 4-26. The Connections Finish window for a source module contains information about the source module and the database link, and the information about a link that references an Oracle system differs slightly from one that points to a non-Oracle system. Creating Source Modules 4-6 Oracle Warehouse Builder 3i Users Guide Creating A Database Link Pointing to an Oracle System The Connections window for a source module contains the following information about a database link: s Owner s User name to access the application s Name of SQL*Net connect string If individual parameter values were specified instead of the name of a connect string, then those values would be displayed. The remaining information pertains to the source module that uses the link. Creating A Link Pointing to an IBM DB2 System When the database link points to a heterogeneous services agent, the property sheet contains an additional box that describes the specific gateway agent. Specifically, the sheet contains: s Name s User name to access the application s Name of SQL*Net connect string s Gateway type If the name of a connect string was specified when the link was created, the name of the link would be displayed instead of the individual parameter values (DB2appnc). Table 41 summarizes application types and their corresponding integrators.
Warning: The init<sid>.ora parameter OPEN_LINKS of an Oracle database determines the number of simultaneously open links during a session. If the value is too small, the Oracle8i/9i instance returns an ORA-2020 error. Table 41 Applications and Corresponding Software Integrators Type of Source Application Application Version or System Type Integrator Reason For Use Oracle Database Oracle Database 7.3, 8.0, 8i, 9i OWB Integrator for Oracle DB & Apps 2.0 Use to access an Oracle database Creating Source Modules The Data Source Modules 4-7 Creating A Source Module for Database Definitions This section describes how to create a source module that connects with an application based on a database system. You can configure a module for an Oracle system, a non-Oracle database, or an Oracle Designer repository. The discussion shows how to implement these alternative configurations. To create the source module: 1. Select MODULES > Create Module from the OWB Console menu or right-click MODULES and select Create Module from the popup menu. OWB returns the welcome page for the New Module Wizard. 2. Click Next. The wizard returns the Name page. Non-Oracle Database Oracle Generic Gateway Connectivity OWB Integrator for Oracle DB & Apps 2.0 Use to access a non-Oracle database Flat File System Generic File System OWB Integrator for Flat Files Use to access flat files Table 41 Applications and Corresponding Software Integrators (Cont.) Type of Source Application Application Version or System Type Integrator Reason For Use Creating Source Modules 4-8 Oracle Warehouse Builder 3i Users Guide On the Name page, enter: s Name of the module s Status of the module Specify the module as Development, Quality Assurance, or Production. This specification is for descriptive purposes only. s Module Type Modules can be Data Sources or Warehouse Targets. Data Source modules contain definitions of data sources. Warehouse Target modules contain a wide variety of definitions such as definitions for warehouse objects, mappings, transformation operations, PL/ SQL packages, configuration parameters, and a set of generated scripts. s Description Provide information into the appropriate locations in the wizard. Creating Source Modules The Data Source Modules 4-9 3. Click Next. The wizard returns the Data Source Information page. This page contains three down-arrow lists: s Application Type s An Application Version or System Type s Oracle OWB Integrator You select the Application Type and the Application Version or System Type, and the wizard determines the correct software integrator. 4. Click Next. The wizard returns the Connections Information page. Use this page to specify: s A source as an Oracle Data Dictionary or an Oracle Designer Repository s A database link s The schema owner s name Click New DB Link to create the link if it not does not exist in the OWB repository (see the discussion of database links in "Configuring Connection Information for Database Sources" on page 4-2). Non-Oracle Database: Select Oracle Generic Gateway Connectivity for the Database Version. Application Type The wizard selects the corresponding Oracle Integrator Application Version or System Type Creating Source Modules 4-10 Oracle Warehouse Builder 3i Users Guide 5. Select the name of the database link from the down-arrow list. The wizard returns the link information and the name of the schemas owner. 6. To change schema owner s name, click Change Schema. The wizard returns a list of users. Creating Source Modules The Data Source Modules 4-11 7. Select a schema and then click OK. The wizard updates the schema owner s name in the Connections Page. 8. Click Next. The wizard returns its Finish page. This page summarizes the information you entered on each of the wizard pages. Verify the information. 9. Click Finish. The wizard creates the source module and inserts its name into the project navigation tree. Updating A Source Module Definition You can update the definition of a source module by editing its property sheet. The property sheet for a module (Figure 41) contains three tabs: s Name Describes the modules usage (Data Source or Warehouse Target) and function. s Data Source Describes the Application Type, Application Version or System Type, and the Oracle OWB Integrator used to access the source. s Connection Describes the configured connection. You can edit active entries in the sheet which include the connection information. Creating Source Modules 4-12 Oracle Warehouse Builder 3i Users Guide To display the property sheet for a source module, fully expand the navigation tree for the warehouse project, select the modules name, and then click the Property icon in the console toolbar. Figure 41 Property Sheet for a Source Module Creating A Source Module for an Oracle Designer Repository You can create a source module that connects with an Oracle Designer repository. When the definitions for an application are stored and managed in an Oracle Designer repository, you can reduce the amount of time you need to connect with the application itself. Creating Source Modules The Data Source Modules 4-13 A single property on the New Module Wizards Connections page determines whether a module can import definitions from an Oracle Database or from an Oracle Designer Repository. Check the button and select a database link that connects with the host where the Oracle Designer Repository resides. Otherwise, create the source module using the procedure described for the previous example. Creating A Source Module for Flat Files A project may need to extract data from flat files. The following section describes how to create a source module for a flat file source. Before you can create the source module, the files must reside in a directory that is visible to the machine that hosts the OWB client. The files must be local to the host machine or mounted on the machine. For additional information on the infrastructure requirements, see the discussion in "Flat File Sources" on page 4-52. To create the source module 1. Fully expand the navigation tree for the warehouse project 2. Select MODULES > Create Module from the OWB Console menu or right-click MODULES and select Create Module from the popup menu. OWB returns the Welcome page for the New Module Wizard. 3. Click Next. The wizard returns the Name page. Check for a Designer Repository. Creating Source Modules 4-14 Oracle Warehouse Builder 3i Users Guide On this page, you enter: s Module name: s Module status: s Module type: s Description This information is described in greater detail on page 4-7. 4. Enter the requested information and click Next. The wizard returns the Data Source Information page. Creating Source Modules The Data Source Modules 4-15 This page contains three down-arrow lists that requests: s Application Type s An Application Version or System Type s Oracle OWB Integrator When you select the Application Type as Generic File Based Application, the wizard determines the remaining two selections. The Oracle Warehouse Builder integrator for flat files reads files that have fixed-length or delimited single record formats. Fixed-length files can contain logical records that have multiple physical records. Multiple record types can be classified in either fixed-length or delimited files. 5. Click Next. The wizard returns the Connection Information page. This page contains two text boxes, Drive and Directory. Use these text boxes to specify the full path name of the source directory. To specify the source directory: a. Enter the name of the drive that contains the source directory. This drive must be mapped to the workstation that hosts the OWB client. b. Click Browse. The wizard returns the Flat File Directory Chooser panel which displays a navigation tree for the drive specified in the first text box. c. Select the name of the directory that contains the flat file and click OK. The wizard inserts the directory name in the Directory text box. d. Click Next. Importing Definitions From Database Sources 4-16 Oracle Warehouse Builder 3i Users Guide The wizard returns the Finish page which summarizes the information you entered. Verify this information before you proceed. e. Click Finish. The wizard creates the flat file module and inserts its name in the navigation tree for the warehouse project. To verify this, expand the Project navigation tree. The modules name now occurs in the MODULES branch. Updating A Flat File Source Module Each definition stored in a OWB repository has an associated Property Sheet, and you can update the definition by editing this property sheet. To display the property sheet for a module, right-click the modules name and then select Properties. OWB returns the modules property sheet. For additional information on property sheets for source modules, see the discussion in "Updating A Source Module Definition" on page 4-11. Importing Definitions From Database Sources This describes how to import definitions from a database application and store them in a source module. You follow this same procedure to import definitions from an Oracle Designer repository or a non-Oracle database. The only difference is the configuration of their respective source modules. You can also reconcile your data definitions after changing your repository without having to delete and re-import all your definitions. Importing Definitions From Database Sources The Data Source Modules 4-17 Figure 42 An Example Database Application The diagram in Figure 42 was generated using the Source Module Editor. After importing definitions into a source module, you can display and print this diagram using the Source Module Editor. To call this editor, fully expand the Project navigation tree and double-click the modules name. Importing Definitions from an Oracle Database You use the Import Metadata Wizard to Import metadata from an Oracle database The following procedures describe this task. To import definitions from an Oracle database application: 1. Fully expand the navigation tree for the warehouse project. 2. Select the source module to which you want to import metadata. 3. Right-click <Module> > Import OWB returns the welcome page for the Import Metadata Wizard. Importing Definitions From Database Sources 4-18 Oracle Warehouse Builder 3i Users Guide 4. Click Next. The wizard returns the Filter Information page. Use this page to limit the search of the sources data dictionary. You can limit a search to: s Tables, views, sequences, or PL/ SQL procedures or packages s Search pattern (for example, a warehouse project name followed by a %) s Maximum number of objects Check the boxes for target objects; enter a search name, and set a max value. 5. Click Next. OWB connects with the source over a database link, reads its data dictionary, retrieves the list of names that meet the search condition, and then returns the Object Selection page. Importing Definitions From Database Sources The Data Source Modules 4-19 The Object Selection page contains an Available Objects box that includes a navigation tree and a Selected Objects box that is blank. To import definitions, expand the navigation tree and then use the arrow keys to move the names from the Available to the Selected box. If you are re-importing definitions, any previously imported objects appear in bold face. You can move a few or all of the names from the Available to the Selected Objects box: a. To move all the names, click the double-arrow icon. b. To move all names for objects of a specific type, select the name of the type and click the arrow icon. c. To move one name, select the name and click the arrow. d. To move one name and the names of objects it references, select the name and check One Level. e. To move a single name and names of the objects it references directly or indirectly, select its name and check All Levels. Verify the names in the selected list. 6. Click Next. Importing Definitions From Database Sources 4-20 Oracle Warehouse Builder 3i Users Guide The wizard returns the Summary and Import page. This page summarizes your selections in a spreadsheet listing the name, type of object, and the action that was performed on the data: whether it was reconciled or created. Verify the contents of this page and add descriptions for each of the objects. 7. Click Finish. OWB stores the definitions in the warehouse module and inserts their names in the modules navigation tree. Importing Definitions From Database Sources The Data Source Modules 4-21 Re-Importing Definitions from an Oracle Database Re-importing your source database definitions enables you bring in changes added to the source since your original import. You do not have to remove the original definitions from the repository. You are also given advanced options that allow you to preserve some of the changes you have made to the objects since the original import. This includes any new objects, foreign keys, relationships, and descriptions you may have created. To Re-Import Definitions: 1. Select a module. 2. Select Import... from the Project menu or from the right-click menu. The Import Metadata Wizard opens and the Welcome page displays. 3. Click Next. The Filter Information page displays. Use this page to select the object types you want to import. You must select the same settings used in the original import to insure that the same objects are re-imported for reconciliation. Importing Definitions From Database Sources 4-22 Oracle Warehouse Builder 3i Users Guide 4. Click Next. The Object Selection page displays. The objects that were originally imported display in bold text. Use the arrow buttons to select the objects that you originally imported. 5. Click Next. Objects Already Exist in the
Repository Importing Definitions From Database Sources The Data Source Modules 4-23 The Summary and Import page displays. The action Reconcile is displayed for the objects you are re-importing. 6. Select Advanced Reconcile Options... to select from a set of advanced reconciliation options. Note: It is possible that a new object has been created in the source that is related to an object you are re-importing. If that is the case, the wizard requires that you import the object(s) at the same time. The action Create displays for those objects. Importing Definitions From Database Sources 4-24 Oracle Warehouse Builder 3i Users Guide The Advanced Reconcile Options dialog displays. There are three options: s Preserve repository added constraints s Preserve existing descriptions s Preserve repository added columns After selecting your options, click OK. 7. Click Finish. OWB reconciles and creates objects as directed. When this is complete, an Import Results report displays. Note: By default, all options are checked. Uncheck if you want these repository objects replaced instead of preserved. Importing Definitions From Database Sources The Data Source Modules 4-25 This report gives details about the actions performed by OWB for each component of each object. There are two additional features available to you: s SaveClick Save to save the report. Make sure to use a naming convention that is specific to the re-import. s UndoClick Undo to undo all changes to your OWB repository. 8. Click OK if you want to proceed. Updating Oracle Database Source Definitions You can update source definitions, diagram individual definitions and their references, and print the diagrams using the Source Module Editor. To call this editor, fully expand the Project navigation tree and double-click the modules name. Importing Definitions From Database Sources 4-26 Oracle Warehouse Builder 3i Users Guide Update a Source Definition You can update a source definition by editing entries in its Property Sheet. To display a definitions property sheet, select its name in the navigation tree and then select Properties from the pop-up list. To update an existing entry, select the entry and enter the new information. Some entries have down-arrow lists that limit the range of selections. For example, when Importing Definitions From Database Sources The Data Source Modules 4-27 you change the data type of a column, you must select an entry from a down-arrow list. You can also add new entries or remove existing ones. Update the Connection You can update the connection information for a data source by changing its link (click the down-arrow and select from the drop-down list). When you change the connection information, OWB returns a warning message that you may compromise the existing definitions in the source module. To change the connection, click OK. Diagram a Source Definition You can display a diagram for a definition and its references using the Source Module Editor. To display a diagram for a definition, select its name in the navigation tree and then select Edit from the pop-up list. The Source Module Editor displays a diagram of the definition. Creating Definitions for Flat File Sources 4-28 Oracle Warehouse Builder 3i Users Guide Each object in the diagram has a toolbar which you can use to sort the column names. The toolbar is divided into three rectangles: to sort the column names click one of the rectangles: s Left rectangle determines the position and order of the primary key columns. s Middle rectangle sorts all the column names. s Right rectangle determines the position and order of foreign key columns. The sort order is only for display purposes and has no bearing on the ordering of column names within the definition. Print a Source Definition To print a diagram of a source definition, display the diagram and then click the Print icon on the editor s toolbar. Creating Definitions for Flat File Sources This section shows you how to create a format definition for a flat file using the Flat File Sample Wizard. Using the wizard you can: s Create format definitions for delimited and fixed-length files s Create format definitions for logical records within a fixed-length file that consists of one or more physical records within a file s Identify and create format definitions for multiple record types within a file After you create and store a format definition, you can use it to describe other flat files that have the same properties. Toolbar Creating Definitions for Flat File Sources The Data Source Modules 4-29 Using The Flat File Sample Wizard When you create a format definition, the Flat File Sample Wizard opens the file, returns a sample of data, and requests detailed information about the files format. The Flat File Sample Wizard is structured as shown in Table 4-4. Character set: OWBs default NLS character set is the same as its host. If it differs from the source files character set, the data sample might be unintelligible. You can direct OWB to display the data sample in the sources native character set by using the down-arrow selection list. For complete information on NLS character sets, see the Oracle8i/9i National Language Support Guide Release 2 (8.1.6). Physical record length: The length of a fixed-format record can be specified as length in bytes or set to use the operating system terminator. The length specification results in greater efficiency. Logical records: OWB can manage a source file whose logical record consists of multiple physical records. In this case, the number of physical records can be fixed or vary. If fixed, you must specify the number of physical records per logical record; otherwise, you must specify a continuation character at the end or beginning of Table 42 Flat File Sample Wizard Pages Page Name Information Required Setup File format: Fixed-length or delimited Field delimiter and enclosure characters for delimited format Terminator or physical record length for fixed-length format Character set Record Organization Single or multiple record types Number of rows to sample and rows to skip before sampling Logical record definition (only available for fixed-length files that have a single record type) Record Types (only appears for multiple record types) Column positions specifying the record type Name of each record type Column Definition (only appears for fixed-length files) Width of each column in a fixed-length file Properties Name, type, mask, NULLIF/ DEFAULTIF, field length Header row Creating Definitions for Flat File Sources 4-30 Oracle Warehouse Builder 3i Users Guide each physical record within a logical record. See the example in "Specifying Logical Records" on page 4-41. Multiple record types: The Flat File Wizard can interpret a source file which contains a variety of record types. You must specify the column within the source file that contains unique record types. You then scan the column to identify unique record type values, then define the characteristics of that record type. The record types can be renamed to be more meaningful. Field type: Describes the data type of the field for the SQL*Loader. OWB supports the following set of portable data types: s CHAR s DATE s DECIMAL EXTERNAL s FLOAT EXTERNAL s INTEGER EXTERNAL s ZONED EXTERNAL The native numeric data is a number in character form: it is not a binary representation. Thus, the numeric data types are identical to CHAR except for their behavior with respect to the DEFAULTIF and NULLIF constraints. See discussion below on field constraints. You can represent FLOAT EXTERNAL data either in scientific or regular notation. The representations "5.33" and "533E-2" are both valid. For complete information on SQL*Loader field and data types, refer to the Oracle8i/9i Utilities Guide, Release 2 (8.1.6). Field mask: The SQL*Loader uses "dd-mon-yy" as its default date mask. You can override this default by entering a valid date mask when you describe the file. For example, should the input data have the format "DD-Mon-YYYY" rather than the SQL*Loader default, you can enter the true format as a mask. NULLIF/DEFAULTIF conditions: You can override the default action of the SQL*Loader by placing a DEFAULTIF or NULLIF condition on a field. s When a numeric or DATE field contains all blanks, SQL*Loader rejects the entire record. To override this action, include a DEFAULTIF = BLANKS condition on the field. When SQL*Loader evaluates this condition, it sets the field to zeros and loads the record. Creating Definitions for Flat File Sources The Data Source Modules 4-31 s When a character field contains all blanks, you can direct SQL*Loader to mark the column as null rather than storing the blanks by including a NULLIF = BLANKS condition on the field. When you describe the field using the Flat File Sample Wizard, you can optionally choose one of these constraints. Creating a Definition for a Fixed-Length File The following procedure describes how to create a definition for a file using the Import Metadata and Flat File Sample Wizards. Each logical record of this file consists of a single physical record. The location of the file was configured in the warehouse source module. To create a definition for a flat file format: 1. Select GCCFILES > Import OWB returns the welcome page for the Import Metadata Wizard. 2. Click Next. The wizard returns the Filter Information page. Use this page to filter file names. 3. Click Next. The wizard returns the Object Selection page. This page displays two window panes: Available and Selected Objects. 4. Move the name of the file to be described from the Available to the Selected Objects window pane. 5. Click Next. Creating Definitions for Flat File Sources 4-32 Oracle Warehouse Builder 3i Users Guide The wizard returns the Summary and Import page. The left-most column of this page contains a status ball which can be red or green. If green, then OWB already has a definition of the files formatproceed to step 7; if red, then you must create a format for the file using the Flat File Sample Wizard. 6. Create a format: s Select a file that has a red status ball. s Click the Sample button at the bottom of the Summary and Import page. s The wizard returns the welcome page for the Flat File Sample Wizard. s Click Next. The wizard opens the file, reads a sample of data, and returns the File Setup page. This page displays the sample of data in a template with a few initial values set for the files global properties. s Verify and select the global properties: * File format: Fixed Length * Record ends at: <CR> by default. * Record size can be specified by length * NLS Character set: WE8MSWIN1252 Creating Definitions for Flat File Sources The Data Source Modules 4-33 s Click Next. The wizard returns the Record Organization page. Use this page to specify whether the file contains single or multiple record types, or if it requires a logical record structure. You may also select how many rows of the file to sample. s Click Next. The wizard returns the Column Definition page. Use this page to specify the column widths. Note: If OWBs character set differs from the sources, the sample might not be readable. If so, select the sources character set from the down-arrow list and OWB translates the sample. Creating Definitions for Flat File Sources 4-34 Oracle Warehouse Builder 3i Users Guide Define a column using one of two methods: Locate where the column ends in the sample and click that position on the ruler. The wizard returns a red tick mark on top of the ruler and marks the boundary with a red line. Specify the column width in the Field Widths space. If you make a mistake, double-click the tick mark and the wizard erases the boundary marker. Use the vertical and horizontal scroll bars to navigate. s Click Next. The wizard returns the Properties page. This page describes each field of the logical record just as you marked it off in the previous step. s Use the Properties page to completely describe each of the logical records fields. Name can be changed to reflect a more useful field name. If your flat file supplies or con- tains field names, select the check box Use the first record as the field names to automatically transfer header information. Creating Definitions for Flat File Sources The Data Source Modules 4-35 Type describes the source to SQL*Loader. To specify type, click the fields data type and select a type from the drop-down list. Mask overrides the default for DATE formats. See discussion on page 4-28. NULLIF or DEFAULTIF overrides the default SQL*Loader behavior for fields that contain all blanks. Specify DEFAULTIF or NULLIF and OWB generates a corresponding DEFAULTIF=BLANKS or NULLIF=BLANKS condition. See discussion on page 4-28. Length specifies the length of the field. For more background information on these fields as well as the names of a few reference materials, see the discussion on page 4-28. s Click Next. OWB returns the Summary page. Verify that the definition is correct. If not, click the Back button to navigate the wizard pages. s Click Finish. OWB exits the Flat File Sample Wizard and returns to the Summary and Object page of the Import Object Wizard. The Summary and Object page has been updated: s The status ball is green. s The File Structure Name column now has an entry (product_fix). If you call this wizard at a later time with a file that has the same format as the one you have just set, then you can select this entry from the Same As field (fourth column) instead of creating a new definition with the Flat File Sample Wizard. 7. Click Finish. Creating Definitions for Flat File Sources 4-36 Oracle Warehouse Builder 3i Users Guide OWB creates a definition for file, stores the definition in the source module , and inserts the formats name in the source modules navigation tree. Using The Source Module Editor The source module now contains a definition for a format. To display the contents of this source module, double-click its name in the Project navigation tree. OWB returns a window for the Source Module Editor. Fully expand the navigation tree to view its contents. Defining Multiple Records in a Fixed-Length File You may have a variety of different record types within a file which can be defined and formatted using the Flat File Wizard. To define multiple record types, do the following: 1. Call the Import Metadata Wizard, select a file to describe, and then click Sample to call the Flat File Sample Wizard (Steps 1 - 5 on page 4-31). The wizard returns the welcome page for the Flat File Sample Wizard. 2. Click Next. The wizard returns the Setup page. 3. Select Fixed Length records. 4. Select a way to specify where each fixed-length record ends. s If you select Records end at, specify the code that terminated each record. s If you select Record length (characters), specify the number of characters in each record. 5. Optionally select a different character set. 6. Click Next. The wizard returns the Record Organization page. 7. Select Multiple record types and specify the number of rows to sample. Optionally specify the number of rows to skip before sampling. 8. Click Next. The wizard returns the Record Types page. Creating Definitions for Flat File Sources The Data Source Modules 4-37 9. Identify the column or columns that identify the record type in the file by using the Record Type begins at position field and the And ends at position field. In the following example, the first column defines the record type, so the first column begins in position 0 and ends in position 1.
10. Click Scan for Record Type Values. A list of distinct type values appears with the default record names RECORD_1, RECORD_2, and so on. You can edit the record names, select a different record name for any type value, and add or delete type values. 11. Click Next. The wizard returns the Column Definition page. 12. Select a record name and use the ruler to specify the length of a field, or specify the field length in numbers in Field Widths. The Field Width settings you define refer to the positions that contain the record type value. When you have adjusted the information for each column, either select another record name and define its columns or click Next to go to the Properties page. Creating Definitions for Flat File Sources 4-38 Oracle Warehouse Builder 3i Users Guide
You now need to define the data characteristics for each field, and adjust masking and constraints. 13. Adjust the data type, mask, and constraints (NULLIF, DEFAULTIF) as needed for each record type. 14. Click Next. The wizard returns the Summary page. 15. The Summary panel shows what is to be imported by the wizard. The source module now contains a definition for the MEmpascii_dat file format and within it, definitions for the individual records. Creating Definitions for Flat File Sources The Data Source Modules 4-39 Updating a Fixed-Length File Definition You can update the definition of the file format by editing its property sheet. To update a single-record fixed-length definition: 1. Select the file definition in the navigation tree. 2. Right-click <file name> > properties. OWB returns the definitions property sheet (General tab). You can edit the name and description of the definition. You can also change the global properties ascribed to the file: the physical record size and the number of physical records per logical record. 3. Select Structure. OWB returns the definitions Structure sheet. Using this sheet, you can: s Edit a field name, data type, mask, constraint, and description s Add a field mask s Add a NULLIF condition s Add a DEFAULTIF condition s Add or delete a field The tabular format of this sheet is useful when you map a field from a data source to a target column. Creating Definitions for Flat File Sources 4-40 Oracle Warehouse Builder 3i Users Guide 4. After completing your changes, click OK. Updating a Multiple-Record Fixed-Length Definition You can update the record definitions within a multiple record file format by editing its property sheet. To display the property sheet for a format: 1. Select <file name> > Properties. OWB returns the definitions property sheet (General tab). You can edit the name and description of the definition. You can also change the global properties ascribed to the file: the physical record size and the number of physical records per logical record. 2. Select Record. OWB returns the definitions Record sheet. You can edit the record type information, for example, you can delete a record type or add a new one. 3. Select Structure. OWB returns the definitions Structure sheet. Using this sheet, you can: s Select a record type from the Record Name drop-down list s Edit a field name, data type, mask, constraint, and description s Add a field mask s Add a NULLIF condition s Add a DEFAULTIF condition s Add or delete a field The tabular format of this sheet is useful when you map a field from a data source to a target column. 4. After completing your changes, click OK. Creating Definitions for Flat File Sources The Data Source Modules 4-41 Specifying Logical Records When the logical record for a source file contains multiple physical records, you must specify that the logical record contains multiple physical records and describe the assembly method. Only fixed-length files with a single record type can have logical records that contain multiple physical records. To describe multiple physical records: 1. Call the Import Metadata Wizard, select a file to describe, and then click Sample to call the Flat File Sample Wizard (Steps 1 - 5 on page 4-31). The wizard returns the welcome page for the Flat File Sample wizard. 2. Click Next. The wizard returns the Setup page. Use this page to define the files global properties. Also, note that the sample of data in the lower panel: some fields have @ as their final character while others do not. 3. Check the Fixed Length button and complete the other selections as required. 4. Click Next. The wizard returns the Record Organization page. 5. Select the button that describes how the logical record is assembled: s Fixed number of physical records per logical record. s Variable number of physical records with a continuation character at the end of each physical record that signifies the record belongs with the next physical record (@). s Variable number of physical records with a continuation character at the beginning of each physical record that signifies the record belongs with the previous physical record. The wizard updates the display of the logical record in the lower panel to reflect your selection. The default selection is one physical record per logical record. 6. You can now define the breaks for each field, click Next, and complete the definition for the format. Creating a Definition for a Delimited File The following procedure describes how to create a definition for a file using the Import Metadata and Flat File Sample Wizards. The fields in this file are delimited Creating Definitions for Flat File Sources 4-42 Oracle Warehouse Builder 3i Users Guide by a comma and enclosed with double quotes. The location of the file was configured in a warehouse source module. The task begins by calling the Import Metadata Wizard from the navigation tree for the active project. To create a definition for a delimited file format: 1. Select the warehouse module 2. Right-click <Module Name> > Import OWB returns the welcome page for the Import Metadata Wizard. 3. Click Next. The wizard returns the Filter Information page which you can use to filter the file names. 4. Click Next. The wizard returns the Object Selection page. This page displays two window panes: Available and Selected Objects. The Available pane displays the tree of the directory configured for the source module; the other pane is empty. 5. Move the files name from the Available to the Selected Objects pane using the single arrow key. 6. Click Next. The wizard returns the Summary and Import page. The left-most column of this page contains a status ball that can be red or green: if green, then OWB already has a definition of the files format; otherwise, you must create a definition for the file using the Flat File Sample Wizard. The ball in the present case is red and you cannot describe the file with the format, so you must create a definition for this files format. To create a format: a. Click the Sample button at the bottom of the page. The wizard returns the welcome page for the Flat File Sample Wizard. Creating Definitions for Flat File Sources The Data Source Modules 4-43 b. Click Next. The wizard opens the file, reads a sample of data, and returns the Setup page. This page displays a sample of data in a template with a few initial values set for the files global properties. s The Field Delimiter default is the comma (,). s The Enclosures defaults are double quotation marks (") for both the left and right enclosures. c. Verify and select the file format (Delimited) and an NLS character set. See the discussion on NLS character sets on page 4-29. Use the text box for the left and right enclosure characters to exclude these characters from the data. You can enter an enclosure character in the text box or select one from the drop-down list. d. Click Next. The wizard returns the Record Organization page. e. Specify single record type and the number of rows to sample. (You cannot enable logical record support for a delimited format file.) For a file that has multiple record types, see "Defining Multiple Record Types in a Delimited File" on page 4-44. f. Click Next. The wizard returns the Properties page. Use this page to completely describe each field. See the detailed discussion for each field property in Step n on page 4-34. Note: In this example, the first record represents the field names. g. Click Next. The wizard returns the Summary page. Verify that the format definition is correct; if not, navigate the wizard pages by clicking the Back button and correct the definition. h. Click Finish. The Flat File Sample Wizard exists and then returns to the Summary and Object page of the Import Object Wizard. The Summary and Object page has been updated: its status ball is now green and the File Structure Name column now has an entry (channel_csv). Creating Definitions for Flat File Sources 4-44 Oracle Warehouse Builder 3i Users Guide You can call this wizard at a later time and use the channel_csv format to describe any flat that has the properties described by this format. Instead of sampling the new file you can select this format from the Same As field. 7. Click Finish. OWB creates a definition for file, stores it in the source module, and inserts its name in the source modules navigation tree. To display the navigation tree for the source module, double-click the modules name in the Project tree.
OWB returns the Source Module Editor. Fully expand the editor s tree and look under the FILES branch. Defining Multiple Record Types in a Delimited File When a flat file contains several different types of records within it, you can use the scanning feature within the Flat File Sampling Wizard to search and label record types. To associate multiple record types within a flat file, do the following: 1. Call the Import Metadata Wizard, select a file to describe, and then click Sample to call the Flat File Sample Wizard (Steps 1 - 5 on page 4-31). The wizard returns the welcome page for the Flat File Sample Wizard. 2. Click Next. The wizard returns the Setup page. Use this page to define the files global properties such as whether records are delimited by a character or space. 3. Click Next. Creating Definitions for Flat File Sources The Data Source Modules 4-45 Define whether the file contains single or multiple record types. . 4. Click Next. Selecting multiple record types brings up the Record Types page: 5. You should now identify the column that contains definitive unique record information. For delimited files, the program assumes the column to scan is the first column of the record unless you specify a different column. Click Scan for Record Type Values: all unique values appear. Records identified are named RECORD_1, RECORD_2, and so on, but you can rename them to more meaningful names by simply typing the new name in the field. For example, change Es record name to employee and Ps record name to payroll, as shown in the following figure. Creating Definitions for Flat File Sources 4-46 Oracle Warehouse Builder 3i Users Guide When you select a record type in the list, the lower panel shows data only for that record type. 6. Click Next. The wizard returns the Properties page: 7. For each record type, select its record name and adjust the data types, mask, and constraint information. See the detailed discussion for each field property on page 4-34. 8. Select the remaining record type and adjust the date types and other information for that record type. When you have the record type definition in the proper structure, select Next. The Wizard processes your records. 9. The Summary presents all record information to be imported. 10. Click Finish to import the file or click Back to return to a previous page to make further changes. Creating Definitions for Flat File Sources The Data Source Modules 4-47 Updating a Delimited File Definition You can update the definition of the file format by editing its property sheet. Updating a Single-Record Delimited File Definition To display the property sheet for the flat file named channel_csv: Right-click <file name> > Properties OWB returns the definitions property sheet (General tab). You can edit the name and description of the definition. You can also change the global properties ascribed to the file: the delimit and enclosure characters. This information, now at your finger tips, is quite useful when you create definitions for the mapping and transformation operations. Updating a Multiple-Record Delimited File Definition You can update the record definitions within a multiple-record file format by editing its property sheet. To display the property sheet for a format: 1. Right-click <file name> > Properties. OWB returns the definitions property sheet (General tab). You can edit the name and description of the definition. You can also change the global properties ascribed to the file: the physical record size and the number of physical records per logical record. 2. Select Record. Creating Definitions for Flat File Sources 4-48 Oracle Warehouse Builder 3i Users Guide OWB returns the definitions Record sheet. You can edit the record type information, for example, you can delete a record type or add a new one. 3. Select Structure.
Using this sheet, you can: s Select a record type from the Record Name drop-down list s Edit a field name, data type, mask, constraint, and description s Add a field mask s Add a NULLIF condition s Add a DEFAULTIF condition s Add or delete a field The tabular format of this sheet is useful when you map a field from a data source to a target column. 4. Click OK. Importing Definitions for Pure Extract and Pure Integrate The Data Source Modules 4-49 Importing Definitions for Pure Extract and Pure Integrate You can import definitions that describe source data that can be extracted using Pure Extract and Pure Integrate. These imported definitions can then be the subject of mappings defined within your project. The following procedures describe how to import definitions using the Oracle Warehouse OWB Metadata Load Utility. These definitions are imported from an export file that was created with Pure Integrate. You can also export definitions with Pure Extract and import them into a OWB Project. For a complete description of how to use the Oracle Warehouse OWB Metadata Load Utility, refer to "The OWB Metadata Loader" on page 10-2. Importing Definitions from an Export File You can import definitions into a Project from an export file using the OWB client or the OWB Metadata Load Utility. This section shows you how to import definitions using the OWB client. The export file was created with Pure Integrate and contains information about a Banking source. To import the exported file: 1. Activate the OWB console and switch to the Administration environment. 2. Select Administration > MetaData Import OWB returns the Metadata Import Confirmation Window. This window contains two radio buttons: Commit or Rollback to previous save point. 3. Select one of the radio buttons, and then click OK. OWB returns the Metadata Import Utility Window which requests information about the operation: File Names: The complete file name of the exported metadata and a log file. Use the Browse buttons to search the host machine for the file names. OWB reads and processes the exported metadata and writes status and diagnostic information in the log file. Operation: You can Replace existing definitions, Add new definitions without replacing existing definitions, or you can Add new definitions and update existing definitions. Importing Definitions for Pure Extract and Pure Integrate 4-50 Oracle Warehouse Builder 3i Users Guide Search by: select the Logical Names or Physical Names button to determine how the Import Utility searches the repository. For a complete discussion of how the utility operates, see "The OWB Metadata Loader" on page 10-2. Character Set: The host machine for the OWB client determines the default character set. Use the down-arrow list to select the character set of the exported file if it differs from the clients character set. 4. Click Scan for a summary of the exported files contents, and then click Import. OWB reports the progress of the operation in a Progress panel, and after the operation completes OWB returns a Metadata Import Results panel. 5. Click View Log File to verify the results of the operation. If the Import operation was successful, you can now view the definitions imported into the OWB repository. You can now create definitions for mappings that reference the source module definitions. Also, each warehouse module now contains a Pure Integrate mapping that can be deployed as an Oracle Workflow function. For additional information on defining dependencies among workflow functions and scheduling workflow jobs, see "Scheduling Jobs" on page 9-2. Infrastructure Requirements The Data Source Modules 4-51 Infrastructure Requirements When a source module points to a non-Oracle system or a UNIX file, the network infrastructure must include components that allow OWB to access the source. When the source is a non-Oracle system, OWB uses Oracle8i/9i Heterogeneous Services. At minimum these services must be configured, and in most cases an Oracle Transparent Gateway agent must be installed on the sources host. When the source is based on UNIX flat files, OWB uses a Network File System (NFS) connection to read the files. To implement this link, third-party software must be installed on the OWB client and an NFS server started on the application host. The following sections summarize these infrastructure components. Oracle Heterogeneous Services OWB communicates with non-Oracle systems using Oracle8i/9i Heterogeneous Services and a complimentary agent. Heterogeneous Services makes a non-Oracle system appear as a remote Oracle database server. The agent can be an Oracle Transparent Gateway or the generic connectivity agent included with Oracle8i/9i. Transparent Gateway OWB can communicate with a non-Oracle system using a transparent gateway agent which is a system-specific source. For example, if a OWB source module defines an source based on a Sybase system, then the agent is a Sybase-specific transparent gateway. You must install and configure this agent to support the communication between the two systems. Generic Connectivity OWB can also communicate with a non-Oracle system using generic connectivity provided that the non-Oracle system supports the ODBC or OLE DB protocols. In this case, you do not need to purchase a separate transparent gateway; you can use the generic connectivity agent included with the Oracle8i/9i database server. You must still create and customize an initialization file for your generic connectivity agent. Generic connectivity is intended for low-end data integration solutions and the transfer of data is subject to the rules of specific ODBC or OLE DB drivers installed on the client system. For additional information on distributed processing systems, see the Oracle8i Distributed Database Systems Release 2 (8.1.6) guide. Committing Your Changes 4-52 Oracle Warehouse Builder 3i Users Guide Flat File Sources OWB can retrieve a data sample from any flat file source that is visible to its Windows NT host. This includes files located in local and remote directories. When the files directory resides on a UNIX machine, the directory must be mapped to the OWB clients host, and in this case you must install third-party software on the host to support a Network File System (NFS) connection. Several of these tools are available from third-party sources. The general steps required to map a UNIX directory to a OWB host are similar regardless of the third-party software: 1. Set up the UNIX host as an NFS server. 2. Declare the source directory on the UNIX machine as sharable. 3. Install the third-party NFS-software package on the OWB host. 4. Map the UNIX directory to the OWB host. When you create a source module using the New Module Wizard, you can now point the module to the NFS directory using the Browse button or entering the complete path name to the directory . Committing Your Changes OWB does not automatically commit your additions or changes to the repository, that is your responsibility. You can immediately commit your work or defer this action until you exit a OWB session. To commit your work immediately, click the Commit icon on the toolbar. OWB returns a Commit Confirmation panel: click Yes. You can also defer this action until you exit a OWB session. On exit, OWB always returns an Exit Confirmation panel: click Yes to commit your work. Note: If you generate code at any time during your Warehouse OWB Session, Multi-User Locking feature will remain in effect until after you commit your changes. Taking The Next Step The Data Source Modules 4-53 Taking The Next Step This chapter showed you how to create definitions for all the sources required to populate the and how to update these definitions by editing their property sheets. The chapter also showed you how to import definitions from an Oracle Designer repository or a non-Oracle system, and an SAP source system. You are now ready to create definitions that map data from the data sources to the target schema. When you create the mapping definitions you can also create definitions for operations that transform the source data. Taking The Next Step 4-54 Oracle Warehouse Builder 3i Users Guide The Source to Target Mappings 5-1 5 The Source to Target Mappings This is the first of three chapters that discuss mapping sources to targets. This chapter describes how to create and update logical definitions that map the source object attributes you defined in the previous two chapters to target object attributes. The next chapter describes how to add operators and transformations to a mapping. And the following chapter describes how to configure the mappings logical and physical properties as well as describing how to reconcile mapping operators with repository objects. Creating a mapping with OWB involves five steps: 1. Define the mapping. 2. Add operators that reference metadata objects for query and altering of the data. 3. Reconcile the mapping operators with the repository objects they represent. 4. Configure the mapping. 5. Generate the mapping. In addition to creating and updating a mapping, this chapter describes the concepts behind the Oracle Warehouse Builder Mapping Editor. This chapter includes the following sections: s About Mappings s Defining Mappings s Generating the Mapping s Taking The Next Step About Mappings 5-2 Oracle Warehouse Builder 3i Users Guide About Mappings You need to have an understanding of the concepts behind the OWB Mapping Editor in order to give you the foundation you will need to build mappings. This section introduces those concepts. When you create a mapping, you define the Extraction, Transformation, and Loading (ETL) operations from a data warehouse source object to a data warehouse target object. Mappings are defined in a warehouse module and as such can be configured, generated and deployed like other objects. The mapping and transformation definitions all reside in the warehouse module that defines the target objects. . In Oracle Warehouse Builder, you define mappings using the Mapping Editor, Property Inspector, and Expression Builder Graphical User Interfaces. The mapping GUIs make it easy to create mapping definitions that extract and transform data and then load it into target tables. For a description of the Mapping Editor Interface, see "About The Mapping Editor" on page 5-8. For a description of Expression Builder, see "About Expression Builder" on page 6-2. About Rows A row (or record) is the basic unit for the processing of data in any mapping. A row has a structure and is defined by attributes, where each attribute is given a name and datatype (such as CHAR, NUMBER, etc), and length, scale, and precision. About Row-Sets A row-set is defined as rows of structured data brought into or emerging from an operator (as defined in the next section), within mapping. Any set of zero or more rows may comprise a row-set. A mapping defines how row-sets are to be extracted from a source (such as a table, or file), transformed and loaded into a target, (such as a table, dimension or fact using operators. The number of rows in a row-set is known as the cardinality of that row-set. Mapping Definition: A mapping definition formally describes a series of operations that together pulls data from sources, transforms that data as necessary, and loads it into target tables. About Mappings The Source to Target Mappings 5-3 About Operators An operator is a self-contained "box" that manipulates data in a mapping. An operator defines how the input row-sets are manipulated to produce the output row-sets and the cardinality of its input row-sets is potentially different from that of its output row-sets. Several operators are available in OWB, each with its own purpose for processing row-sets. Mappings provide a unified interface that connects these different operators that have certain properties. This helps you to visualize the logical flow of the data from the sources to the targets and the operations you perform on the data during the ETL process. Hence, operators are key to the Oracle Warehouse Builder mapping paradigm. You define and edit operators using the Mapping Editor and Property Sheets. The Mapping Editor contains an Object Palette that visually represents the operators. The operator types available within the Mapping Editor include: s Extract OperatorsMapping Table, Mapping View, Mapping Materialized View, Mapping Sequence, Mapping Fact, Mapping Dimension, Mapping Flat File etc. s Load OperatorsMapping Table, Mapping Materialized View, Mapping Dimension, Mapping Fact, etc. s Standard OperatorsAggregator, Pre- and Post-Mapping Processes, Filter, Joiner, Splitter, Sorter, Deduplicator (Distinct), etc. s TransformationsMapping Transformation, Expressions, Constants. s External ProcessPure*Integrate, Pure*Extract, custom processes. You define and edit operators using the Mapping Editor canvas, Operator Property inspectors, and the object palette, all of which are described in "About The Mapping Editor" on page 5-8. About Operator Properties Each operator has a very generic mapping purpose, but to use them in maps you can give them very specific instructions. You do this by altering the Operator Properties. Each operator type has a different set of properties it supports. These properties can exist at 3 levels: s Operator level properties that effect the overall operator and what it does s Group levelproperties that effect attributes within the group s Attribute levelproperties that effect only the attribute or its value About Mappings 5-4 Oracle Warehouse Builder 3i Users Guide The operator properties can be viewed and altered by the operator s Property Inspector dialog. About Attributes And Attribute Groups Each operator is a self-contained box that can have inputs, outputs, and/ or input/ outputs. These inputs and outputs are called Attributes. Each attribute is organized within an attribute group. The group type will determine what type each of the attributes within the group are (input, output, etc). Each type of operator limits the types of attribute group or attribute groups it can supportfor example, a joiner operator can have only one output group. An attribute group represents a row-set of certain cardinality. It is important to understand that row-sets of different cardinality cannot be mixed arbitrarily. In other words, an attribute from one output attribute group and an attribute from another output attribute group cannot be connected to the same input attribute group unless the two output attribute groups are of the same cardinality. For example, the output attribute of an transform and an attribute from a source table can be connected to the same target table if the transform derives its data from the same source table since a transform does not change the cardinality of the input row-set. The names of attributes and attribute groups are logical; you can change the names of attributes and attribute groups. They are often the same name as the attributes of the operator that is mapped into the operator. However, it is not a requirement that the attribute name match the input data source name. Since the operator is self-contained and it can be remapped at any timethe attribute names are independent of the attributes of the operator to which they are connected. This protects any expression or use of the attribute within the operator. You can consider an operator attribute as a placeholder for data. Each attribute has a data type and size, precision and scale if appropriate. These attribute items are considered properties of the attribute. Attribute Groups have properties as well. The complete list of supported properties for Attributes and Attribute Groups varies by operator. The values for the properties can be set and changed by the Property Inspector. About Display Sets A Display Set represents graphically the attributes for a particular attribute group of a selected operator. If an attribute group contains more than one display set, then you can select a different display set from a list located on the View menu. You can define display sets for an attribute group in the mapping editor, or they can be About Mappings The Source to Target Mappings 5-5 inherited from the attribute sets of a warehouse object that are defined during the source and target definition phase of data warehouse design. See "Creating Attribute Sets" on page 3-29 for information on attribute sets. You can see only one display set at a time for an attribute group. By default, the Mapping canvas will always show the display sets with all the attributes in the respective attribute groups. You can also edit display setsas well as define new display setswithin an operator using the Mapping Editor. See "Reconciling Your Mapping" on page 5-36 for more information on editing operator attributes. Table 51 describes the default attribute sets for Display Sets. Note, the ALL display set is read-only. About Binding Mapping Operators Mapping operators are independent from the repository objects to which they refer. The process of linking a mapping operator to a repository object is known as binding. Within the Mapping Editor are several methods of binding a mapping operator to a repository object. Some of these methods relate to the original creationafter either the mapping operator or the repository object has been changed. See (Page 5-xx) Some mapping operators describe repository object of various types. Their internal definitions can be created from an existing repository object or, in some cases, be used to create a repository object. The extract, load and transformations operators fall in to this category. Whenever you create a new extract or load operator in the Mapping Editor, you can choose to bind it to an existing repository object or you can leave it unbound and create the attributes yourself (Transformations are always bound). When creating mapping operators that are derived (inbound reconciled) from repository objects (e.g., a table, a function, a dimension, etc.), the operator's attributes and display sets will inherit the repository object's attributes and attribute sets. You may also Table 51 The Default Attribute Sets Attribute Set Description All This is the default attribute set and includes all attributes. Hierarchies For every Dimension with hierarchies defined, the system will automatically create for each hierarchy in the Dimension a display set containing all the level attributes in that hierarchy (named after the hierarchy name)." About Mappings 5-6 Oracle Warehouse Builder 3i Users Guide manually reconcile (outbound) created mapping objects to create repository objects. Any mapping object whose attributes and attribute sets have been reconciled with a repository objects attributes is referred to as being bound to that repository object. Bound operators represent actual tables, functions, or procedures via their reference to a repository object. The repository object is the primary definition and all of the OWB import, generation, and deployment tools can be used to maintain the integrity of the definition. Editing of bound operators is limited to changing the logical name and adding new attributes (which can be outbound reconciled to update the repository object definition). Deleting a bound operator has no effect on the repository object. If you choose to leave the operator unbound, it will generate code but that code will have no repository object upon which to affect. You need to bind the mapping operator to a repository object in order for the code to affect the repository object. Once you bind a mapping object to a repository object, it cannot be unbound. See "Reconciling Your Mapping" on page 5-36 for more information on reconciling mapping operators with repository objects. About Mapping Operator Names All names visible in the mapping editor diagram are logical names irrespective of the user preference settings described in chapter two. You use them as descriptors of the repository objects they represent, and you can edit them. For those components that refer to repository objects, the Bound Name property is used to specify the name used for code generation. If a mapping operator or attribute is currently bound ( and has been reconciled), the repository object is considered to be the primary definition and all edits to the physical (bound) name must be done on it. The bound name of the mapping object will be read only. On the other hand, if an object or attribute is not yet bound, you may edit the bound name within the mapping environment in preparation for eventual reconciliation to a repository object. About Reconciliation The loose coupling of mapping operators to repository objects is one of the key mapping concepts in Warehouse Builder. It gives you the flexibility to evolve mappings and repository objects independently. Only when you are satisfied with your changes to repository object definitions do you need to update your mappings to reflect these new object definitions. About Mappings The Source to Target Mappings 5-7 The process of keeping mapping operators and repository objects in step is known as reconciliation. Do not confuse reconciliation with the term synchronization, which is used to describe the process of ensuring that you are up to date with changes made by other users in a multi-user environment. To characterize the two terms as used in Warehouse Builder, synchronization is a necessary activity (akin to saving a file frequently) to keep users in step, whereas reconciliation is a deliberate act which can be planned to occur at significant milestones in the warehouse development. See "About Multiple-User Access" on page 2-4 for more information. You can use reconciliation to control the phases of the warehouse development process or to allow proper planning of warehouse maintenance. The reconciliation feature provides great flexibility to accommodate different development methodologies; here are two examples: s If you are driving the warehouse development process from your mappings then you can use outbound reconciliation to create new repository objectstables, views and materialized views. s You can prototype your mappings using tables and when you are satisfied with the transformation logic switch to other object types for the production mappingsfor example views, materialized views, factsby using inbound reconciliation. When To Use Inbound Reconciliation Inbound reconciliation updates the mapping operator to reflect changes in the selected repository object; in other words the mapping operator is the target for the reconciliation. You would use inbound reconciliation for any of the following reasons: s To capture structural changes previously applied to the selected repository object and propagate these to the mapping operator. s To capture attribute name changes previously applied to the selected repository object and propagate these to the mapping operator. s To capture attribute data type changes previously applied to the selected repository object and propagate these to the mapping operator. s To associate the mapping operator to a different repository object of the same type; for example you might do this if you are migrating mappings from one version of a data warehouse to a newer version and you maintain separate different object definitions for each version About Mappings 5-8 Oracle Warehouse Builder 3i Users Guide s To associate the mapping operator to a repository object of a different type; for example you might do this if you want to provide access to the underlying source data through views rather than go directly against the base tables. Inbound reconciliation has no impact on the linking of other mapping operators in the mapping to repository objects; OWB preserves these links. When to use Outbound Reconciliation Outbound reconciliation causes the selected repository object to be updated to reflect changes in the mapping operator; in other words the mapping operator is the source for the reconciliation. You would use outbound reconciliation for any of the following reasons: s To capture structural changes applied to the mapping operator and propagate these to the currently linked repository object. s To capture attribute logical name changes applied to the mapping operator and propagate these to the currently linked repository object. s To capture attribute data type changes applied to the mapping operator and propagate these to the currently linked repository object. s To create a new repository object in the same warehouse module or create or replace a different repository object in a different module. This will create a repository object of the same type as the mapping operator; you might do this if you are defining your definitions for staging tables as a product of building your mappings. You cannot create a repository object of a different type using outbound reconciliation. Outbound reconciliation has no impact on the linking of other mapping operators in the mapping to repository objects; OWB preserves these links. See "Reconciling Mappings" on page 7-2 for addition information about reconciliation. About The Mapping Editor The Mapping Editor is the OWB graphical user interface for defining mappings. This editor provides an easy to use drag-and-drop interface for defining what you want to transform and map. The mapping editor is based on an operator-mapping concept. The operator-mapping concept requires you to add an operator for each operation you want to perform. A series of operations define the map. About Mappings The Source to Target Mappings 5-9 For example, you can create a mapping to s Add Mapping Table operator to read from a source table s Add Filter Operator to only query specific rows s Add Mapping Table operator to write to a target table Each instance of the operators you use in a map can have its own unique name. The Mapping Editor generates a default name which you can change at any time. This name is a logical name and has nothing to do with the physical objects associated to it. The operators used in the map only refer to the metadata objects. This reference is called Binding. The binding is a loose bindingin other words, any changes to the metadata sources or targets that are referred to in the map will not be reflected in the map until you reconcile the mapping operator with the repository object. See "About Binding Mapping Operators" on page 5-5 for more information on binding. Depending on the position of the attribute where you start your mapping, the attribute of the source and target attributes (expanded, collapsed, minimized), and the position you release the mouse-button will determine the type of data flow connections made in the mapping. s You can drag a line from an output attribute group to an input attribute group of a downstream operator. s You can drag a line from an output attribute to an input attribute of a downstream operator. s You can copy all selected attributes from the source node and add them to the selected target attribute group (if any) by dragging a line from the source to the target. If the mouse button is released over an invalid target, no data flow connection will be established. You cannot create the following links: s link an output attribute to an output attribute s link an input attribute to an input attribute s link attributes in the same operator (simplest case of circular dependency) s link to the same input attribute twice Several components comprise the Mapping Editor: About Mappings 5-10 Oracle Warehouse Builder 3i Users Guide s The Mapping Menu BarThe menu provides access to all features of the mapping editor, including those commonly done by using the mouse (for accessibility purposes). s The Mapping Tool BarYou can execute certain commands using the Mapping Toolbar rather than the main menu. Figure 51 The Mapping Editor Toolbar and Menu s The Object PaletteYou can add Operators to a Mapping by dragging the corresponding icon from the Object Palette to the Mapping Editor Canvas. Figure 52 The Object Palette s The CanvasThe white space on the Mapping Editor is called the Canvas. You use the canvas to perform most of your mapping activities. You use the canvas to model your extract, transform, and load (ETL) operations. The canvas is where you graphically add and remove objects as well as edit the operators of a mapping and draw the associations between them. About Mappings The Source to Target Mappings 5-11 Figure 53 The Mapping Canvas Showing Mapping Operators You can select multiple mapping operators or data flow connections either by SHIFT or CONTROL mouse clicks dragging a box around components s Operator Property Inspector You edit the properties of a mapping operator, attribute, or attribute group using the Operator Property inspector. The content and organization of properties in the Operator Property inspector is different for each operator, attribute, or attribute group; each operator, attribute, or attribute group may have its unique set of properties and organization, but attributes will have a set of common properties (such as datatype, name, etc.). The Property Inspector is a non-modal dialogonce opened, you can select another operator, attribute, or attribute group and the content of the property inspector will display the properties for the selected object. Only one property inspector can be active at a time. About Mappings 5-12 Oracle Warehouse Builder 3i Users Guide Figure 54 The Property Inspector You can launch the Operator Property inspector by s clicking the title bar of an operator s selecting an operator, attribute group or attribute and clicking on the menu item Edit > Properties s right-clicking on the header, attribute group or attribute of an Operator and selecting Properties If no operator, attribute group or attribute is currently selected while the Property inspector is open, the Operator Property inspector will show an empty list. About Expressions At various points in defining a map, you may need to add an expression to get the map to do what you want it to do (for example, joiner condition or a splitter condition). Expressions consist of values of operator properties, and they are set and altered through the Property Inspector using the Expression Builder. Each operator will determine what kind of expressions it supports. The Expression Builder contains a validate feature that enables you to check to ensure if the expressions you created are correct. The expressions are limited to the inputs of the operator and to any transformations available in the project. This limitation helps protect the expression from becoming invalid because of change external to the operator. Defining Mappings The Source to Target Mappings 5-13 For a description of Expression Builder, see "About Expression Builder" on page 6-2. About Mapping Generation Once you have completed your mapping definition, it is time for you to run it. Oracle Warehouse Builder will generate code in order to perform the map that you defined. It can generate a number of different languages PL/ SQL for executing within the database, SQL Loader for flat files, and TCL for use with Oracle Enterprise Manager. It will select the appropriate language based on the operators and objects you used in your map. To run your mapping, you need to perform the following tasks: s Validate your mapping This is not a required task but it is highly recommended. This task will attempt to catch anything that will prevent the mapping from deploying or running. The validation task will provide a list of errors and warnings that you should correct prior to running the map. s Configuration This task will define the physical characteristics of the map. You can tune your map to take advantage of specific Oracle features and you can change runtime parameters of the mapping before you generate it. s Generate This task will produce the code in the appropriate language to perform the mapping. s Deploy This task will save install the code in a file or install it to the database where it is to be executed. s Run You can now execute the map. Defining Mappings The remainder of this chapter describes how to define a basic mapping. Specifically, this section will describe how to s Map data from a flat file to a table Defining Mappings 5-14 Oracle Warehouse Builder 3i Users Guide s Add an operator that enriches or complements the data flowing to the table from the flat file. This section describes the following steps in defining a new mapping: 1. Create a mapping object in your warehouse module using the New Mapping Wizard. 2. Define data sources, data targets, operators, and transformations. 3. Configure their attributes and their properties. 4. Graphically define the data flow connections between the operators attributes. 5. Validate and reconcile the mapping and generate the code. Creating a Mapping Warehouse Object You begin the Mapping process by defining a mapping object in the Mapping node in the Warehouse Module Editor. To create a mapping object: 1. Open a Warehouse Module using the Warehouse Module Editor. 2. Select Mappings in the Logical Tree Defining Mappings The Source to Target Mappings 5-15 Figure 55 The Warehouse Module Editor 3. Right-click Mappings > Create Mapping. OWB returns the New Mapping Wizard Welcome dialog. 4. Click Next. 5. Enter a name for the mapping and a description in the appropriate fields on the Name dialog. See About Naming Policies on page 2-20 for information on naming. Defining Mappings 5-16 Oracle Warehouse Builder 3i Users Guide Figure 56 The Name Dialog in the New Mapping Wizard 6. Click Next. The New Mapping Wizard returns the Finish dialog. You can use the Finish dialog to verify the name of your new mapping. 7. Click Finish. OWB stores the definition for the mapping and inserts its name in the warehouse modules navigation tree. You have created a container that will hold the mapping operators that define the ETL operation from a data warehouse source object to a data warehouse target object. The Mapping Editor automatically opens. Defining Mappings The Source to Target Mappings 5-17 Figure 57 The Mapping Editor The next section describes how to add the mapping operators. Selecting A Source Operator You use the Mapping Editor to define the ETL operations, first of which is selecting sources and targets for your mapping. The following instructions assume you have kept the Mapping Editor open. If you closed the mapping editor after creating a new mapping you can re-open it either s from the logical tree in the Warehouse Module Editor by selecting a mapping under the MAPPINGS node and selecting Edit > Edit Object from the menu or s from the logical tree in the Warehouse Module Editor by selecting and double-clicking a mapping in the MAPPING node. s from the logical tree in the Warehouse Module Editor, by selecting a mapping, right-clicking it, and selecting Edit... from the pop-up menu. If you still have it open, then you are ready to select operators for a mapping on the canvas using one of two methods: Defining Mappings 5-18 Oracle Warehouse Builder 3i Users Guide s dragging the operator type icon from the Object Palette and dropping it onto the canvas. s selecting Edit > Add and selecting the operator from the menu list. The following instructions describe how to select a Mapping Flat File operator as data source for a mapping. A File operator is used as the row-set generator for reading from a flat file. A File operator may be connected to a downstream filter operator, expression operator, or transformation operator. The row-set generated from the file operator must ultimately arrive at a table, view, dimension, or fact operator. A file operator will generate a SQL*Loader mapping. The following instructions assume that you already have an open Mapping Editor. To define a flat file source operator: 1. Drag a Mapping Flat File icon from the Object Palette and drop it onto the canvas. OWB opens the Add File dialog. 2. Select one of the four options: s Create unbound file with no attributesenter a name for the new mapping file you are creating. s Create new repository file and bindselect the warehouse module where you want to create the file. s Import file into repository and bindselect the module from which you are importing the file. s Select from existing repository file and bindeither type the prefix to search for the flat file or select from the displayed list of flat files within the selected module. Defining Mappings The Source to Target Mappings 5-19 Figure 58 The Add File Dialog 3. For this case, select a flat file from an existing repository file and bind and the click OK. Once youve made your selection, OWB places a Mapping Flat File operator on the canvas which reflects the names and attributes of the flat file to which it is now bound Defining Mappings 5-20 Oracle Warehouse Builder 3i Users Guide Figure 59 The Mapping Editor Showing A Mapping Flat File Operator You are now ready to define a target operator for your mapping. Selecting a Target Operator The following instructions describe how to select a Mapping Table operator as a data target for a mapping. You can use the Mapping Table operator as a data source as well, but these instructions use it as a target. The following instructions assume that you already have an open Mapping Editor. To define a mapping table target operator: 1. Drag a Mapping Table icon from the Object Palette and drop it onto the canvas. OWB opens the Add Table dialog. 2. Select one of the four options: s Create unbound table with no attributesenter a name for the new mapping table you are creating. s Create new repository table and bindselect the warehouse module where you want to create the table. Defining Mappings The Source to Target Mappings 5-21 If you make this selection and then click OK, OWB returns the New Table Wizard. Follow the wizard steps to create a new table. s Import table into repository and bindselect the module from which you are importing the table. If you make this selection and click OK and have already defined a database link for the module from which you are importing the table, then OWB returns the Database Link Information dialog. Enter the appropriate information into the dialog. If you have not defined a database link, then OWB returns the New Database Link Information dialog. See "Using The New Database Link Dialog" on page 4-4 for information on how to define a new database link. s Select from existing repository table and bindeither type the prefix to search for the table or select from the displayed list of tables within the selected module. Defining Mappings 5-22 Oracle Warehouse Builder 3i Users Guide Figure 510 The Add Table Dialog 3. For this case, choose Select from an existing repository table and bind and then click OK. Once youve made your selection, OWB places a Mapping Table operator on the canvas which reflects the names and attributes of the Table to which it is now bound. Defining Mappings The Source to Target Mappings 5-23 Figure 511 The Mapping Editor That Includes A Mapping Table Operator A Mapping Table operator will then appear on the canvas. Selecting A Data Flow Operator You can add operators that will act upon your source data as it flows to the data targets. OWB includes a variety of operators on the Object Palette, or you can design your own using the Oracle Transformation Library or Expression Builder. See Chapter 6, "Mapping Operators and Transformations" for more information on creating custom operators. A commonly used Data Flow operator is a Mapping Data Generator. This operator is used to add a pseudo column to the source metadata before reaching the target. To add a Data Generator operator to a mapping, drag a Data Generator icon from the object palette and drop it on the canvas. OWB adds a Mapping Data Generator Operator onto the Mapping Editor canvas. Defining Mappings 5-24 Oracle Warehouse Builder 3i Users Guide Figure 512 The Mapping Editor Including A Data Generator There is no need to bind it to any object in the OWB repository since this build-in Operator already has a predefined set of attributes. You are now ready to connect your operators. Defining The Data Flow Connections To connect Mapping Operators, you draw lines from output attributes or output attribute groups to input attributes or groups between the operators. These lines are Data Flow Connections; these lines graphically represent how the data flows from a source to a target. To connect operators: 1. Click and hold down the mouse button while the pointer is positioned over an output attribute. Defining Mappings The Source to Target Mappings 5-25 2. Drag the mouse away from the output attribute and toward the input attribute to which you want data to flow. As you drag the mouse, a line appears on the Mapping Editor canvas to indicate a connection. 3. Release the mouse over the input attribute. You have now created a data flow connection between a data source and data target. Repeat steps one through three until you have created all the data flow connection appropriate for your situation. Figure 513 The Connected Operators In A Mapping This defines a mapping. Note that to complete a mapping, you may also need to include transformations in addition to those describedtransformations such as pre and postmapping processes and mapping transformations. Chapter 6, Defining Mappings 5-26 Oracle Warehouse Builder 3i Users Guide "Mapping Operators and Transformations" shows you how to create definitions for these and other custom transformations. Before you can actually generate a mapping, you may need to alter settings for attributes in the data target. The following section describes how to do this. If not, then skip the following section and proceed to the next section that describes how to reconcile and validate your mapping. Configuring A Mapping In mapping a source operator to a target operator, you may need to edit certain characteristics in your mapping. To customize your mapping, you can provide additional configuration information. Configuration can either be an intrinsic part of your ETL design or be of a more physical nature, for instance, based on how and where you want to deploy your Mapping. The following section addresses the more commonly used settings that require configuration. See "Configuring The Mapping For Deployment" on page 7-13 for more complete information on configuring a mapping. Setting The Load Type OWB generates code to extract data from source operators and then loads it to target operators. For each load operator, you define a load type which determines the type of load code that OWB generates. The generated code can have one of these load types: s INSERTThis type of operation specifies that the incoming row sets will be inserted into the data target. s UPDATEThis type of operation specifies that the incoming row sets will be used to update existing rows in the data target. s INSERT/ UPDATEThis type of operation specifies that for each incoming row, an insert operation will first be performed. If the insert fails, an update operation occurs. s UPDATE/ INSERTThis type of operation specifies that for each incoming row, an update operation will first be performed. s DELETEThis type of operation specifies that the incoming rowsets will be used to determine which of the rows at the target get deleted. Defining Mappings The Source to Target Mappings 5-27 s TRUNCATE/ INSERTThis type of operation specifies that the data target will first be truncated before the incoming rowset is inserted into the data target. s DELETE/ INSERTThis type of operation specifies that all the rows in the data target will first be deleted before the incoming rowset is inserted into the data target. s CHECK/ INSERTThis type of operation specifies that the data target will first be checked if it contains any rows. If not, the incoming rowsets will be inserted into the data target. s NONEno operation will be performed on the data target. This setting might be useful for test runs; all transformations and extraction will be run but will have no effect on the target. For more specific information on setting target characteristics, see "Configuring Attribute Properties" on page 7-13. To define the load type: 1. Right-click the header of the target operator. 2. Select Operator Properties. OWB returns the Table Properties for the operator. 3. Click Data Entity Parameters to expand the properties list. 4. Click the Loading Type row on the right-hand column. 5. Select a loading operation from the pulldown list. Warning: If you choose this option, the procedure cannot be rolled back even if the execution of the mapping fails. Defining Mappings 5-28 Oracle Warehouse Builder 3i Users Guide Figure 514 The Loading Types Property Inspector Configuring Physical Properties For a Mapping Mappings will require you to configure their physical properties. For example, you may also need to define the physical location of the file from which we will read the records. Configuring this kind of physical property for a Mapping can be done using the Configuration Properties dialog. See "Configuring The Mapping For Deployment" on page 7-13 for more information. The following instructions will lead you through the steps to accomplish physical logical configuration. To Configure the Physical Properties for a mapping: 1. Select a Mapping object from the Warehouse Editor Navigation Tree. 2. Right Click the mapping you want to configure 3. Select Configure.. from the popup menu. OWB launches the Configuration Properties dialog. Defining Mappings The Source to Target Mappings 5-29 Figure 515 The Configuration Properties Dialog 4. Select the Steps node and expand the child nodes until you see the property you need to set. 5. Click the close window button to save the setting. Validating the Mapping To ensure you have not made mistakes in defining your mapping, you can validate it. The validation procedures verify foreign key references, object references, data type (mis)matches, and other properties in your mapping. See "Validating Definitions" on page 8-24 for more information on validation. To validate a mapping: 1. Select a Mapping on the Mapping Node in the Warehouse Module Editor. 2. Select Module > Validate... on the main menu. The results of a validation are reported in a Validation Results window. Defining Mappings 5-30 Oracle Warehouse Builder 3i Users Guide Figure 516 The Validation Results Window 3. Click on the Details button to see the full message that appears in the Validation Message Field. Generating the Mapping The Source to Target Mappings 5-31 Figure 517 The Details Window If you have errors, the Validation Details window will often include suggestions for correcting them. You can open an editor to make these corrections by clicking Edit in the Validation Results Window. Once you have reconciled the operators and validated the mapping, you are ready to generate the SQL code defined by the mapping. The next section describes how to view and edit that code. Generating the Mapping When you generate the mapping, you are implementing the SQL code that performs the DML and DDL commands necessary to move the data from the sources to the targets defined in your mapping. Once you have fully defined a mapping, configured it, and successfully validated it, the code generator creates the code that will actually implement you ETL design. generating the code for a mapping can be done using two methods: s From the Module Editorthis method will create scripts with the generated code that the user can deploy and run. Generating the Mapping 5-32 Oracle Warehouse Builder 3i Users Guide s From the Mapping Editorthis method will generate code that the user can inspect from a code viewer. To generate code from the mapping editor, select Mapping > Generate from the Mapping Editor menu. OWB will then generate the code for the mapping. When the code generation completes, OWB returns the Code Viewer. Figure 518 The Generated Code For A Mapping For a Mapping that extracts records from a flat file and loads them as rows into a target table, the generated code results in a SQL*Loader control-file. The control-file reflects a Mapping definition from a flat file to a table source and includes a date stamp: Editing Mapping Operator Attributes The Source to Target Mappings 5-33 Editing Mapping Operator Attributes An operator s properties play a critical role in the mapping process. Therefore, you will need to become familiar with editing mapping operator properties and attributes using the Mapping editor. Adding or Removing Operator Attribute Groups When you create a new attribute group, you may do so directly in the mapping editor by right-clicking the name header and choosing the Add/Remove Groups option. Figure 519 The Adding/Remove Groups Dialog You cannot convert a input group to an output group or vise-versa. This feature works only for Splitters and Joiners. See "Adding Data Flow Operators To A Mapping" on page 6-6 for more information. Editing Operator Attributes For each mapping operator, you can add, delete and rename Attributes and Attribute Groups. Note: Several properties and parameter settings involve PL/ SQL or SQL*Loader expressions. When you edit or create these settings, the Expression Builder appears automatically for you to construct these expressions. See "Using Expression Builder" on page 6-2 for information on using Expression Builder. Editing Mapping Operator Attributes 5-34 Oracle Warehouse Builder 3i Users Guide Adding Or Removing Attributes You can add attributes to, or remove attributes from, an operator from the Mapping Editor canvas. The following procedure starts with an open Mapping Editor containing mapping operators. To add attributes to an Operator 1. Select a mapping operator. 2. Click Edit > Add/Remove Attribute or right-click the operator s display set > Add/Remove Attribute or right-click on the attribute group of the operator and select Add/Remove Attribute from the popup menu. 3. OWB returns the Add/Remove Attribute dialog. Figure 520 The Add/Remove Attribute dialog 4. Enter the new attribute name in the Add/Remove Attributes dialog. 5. Click Add. 6. Click OK. See "Creating New Attributes" on page 5-35 for an alternate method to add attributes. To remove attributes from an operator 1. Select an attribute group in a mapping operator. 2. Click Edit > Add/Remove Attribute or right-click the operator s display set > Add/Remove Attribute or right-click on the attribute group of the operator and select Add/Remove Attribute from the popup menu. Editing Mapping Operator Attributes The Source to Target Mappings 5-35 3. OWB returns the Add/Remove Attribute dialog. 4. Select the attribute you want to remove in the Add/Remove Attributes dialog. 5. Click Delete. Renaming Attributes To rename an operator, attribute or attribute group, select the appropriate item on the Mapping Editor canvas and s Select Edit > Rename from the menu or right-click an attribute and select Rename from the popup menu. s Specify the new name of the operator in the Rename dialog box. Figure 521 The Rename Attribute Dialog s Click OK. You can also right mouse-click on the operator and select Rename from the popup menu. Creating New Attributes The section "Adding Or Removing Attributes" on page 5-34 described how to change an operator s properties using the Operator Property inspector. Another method to create operator attributes is to use the connecting lines. You can create a new attribute for an operator within the Mapping Editor by s Dragging and dropping the attribute group header to the operator that you want the attributes to belong. This is a copy operation. Any name conflict will be resolved by auto-generated names. The drag begins in the input/ output indicator. Editing Mapping Operator Attributes 5-36 Oracle Warehouse Builder 3i Users Guide s You can create a new unconnected attribute by right clicking the attribute header and choosing Add/Remove Attribute. Reconciling Your Mapping Once you have added or changed attributes and/ or attribute groups or have changed the repository object to which the mapping operator is bound, you should reconcile the mapping operators with their corresponding repository objects. Reconciliation enables you to synchronize mapping operators with repository objects. You can reconcile mapping operator attributes with repository object attributes using the Inbound Reconciliation dialog. Reconciliation is an optional task that you perform only if you have changed attributes in a repository object. Using this mechanism, you can: s capture structural changes applied to the associated object in the repository and propagate these to the mapping operator. s associate the mapping operator to a repository object of the same type s associate the mapping operator to a repository object of a different type. You can do all this while still preserving the connections with other mapping operators. See "About Reconciliation" on page 5-6 and "Reconciling Mappings" on page 7-2 for more information about reconciliation. Reconciling Inbound Operators You use the Inbound reconcile operator dialog to bind a mapping operator to a repository object. You can reconcile inbound an operator by name, position, or object identifier match. You can also use a combination of the above three methods. Using this mechanism, you can: s capture structural changes applied to the mapping operator and propagate these to the associated object in the repository. s create a new repository object that inherits the structure of the mapping operator and binds to it. You can do all this while still preserving the connections with other mapping operators. See "About Reconciliation" on page 5-6 and "Reconciling Mappings" on page 7-2 for more information about reconciliation. To start inbound reconciliation, select the mapping operator on the canvas and: s Select Edit > Reconcile Inbound menu item from the menu bar or Editing Mapping Operator Attributes The Source to Target Mappings 5-37 s Right mouse-click the header of the mapping operator and select the Inbound Reconcile menu item from the popup menu. OWB launches Inbound Reconcile Operator dialog. Figure 522 The Inbound Reconcile Operator Dialog The Inbound Reconcile operator dialog enables you to select the object in the repository to reconcile the mapping operator with and then select the strategy for matching the attributes of the mapping operator with the selected repository object. Note that operators of a certain origin (table, file) can be reconciled with a different repository object type (view, etc.) See "Reconciling Mapping Operators With Repository Objects" on page 7-2 for more information on Reconcile Strategies. Reconciling Outbound Operators To start Outbound Reconcile, you select an operator on the canvas and s Select Edit > Reconcile Outbound menu item from the menu bar or s Right mouse click on the header of the Operator and select Reconcile Outbound menu item from the popup menu. Editing Mapping Operator Attributes 5-38 Oracle Warehouse Builder 3i Users Guide OWB will open the Outbound Reconcile Operator dialog. Figure 523 The Outbound Reconciliation Dialog In the Outbound reconcile operator dialog you bind a mapping operator to a repository object by choosing: s Create a new object if the repository object does not already exist in that module, or s Reconcile with an existing object; in this case you need to select the appropriate Reconcile Strategy. In addition to reconciling the attributes of an operator, the reconciliation process may also update the logical properties of an operator. Once you have edited your attributes and reconciled the mapping, you can now generate the Mapping. Select Mapping > Generate from the Mapping Editor menu. OWB will then generate the code for the mapping. When the code generation completes, OWB returns the Code Viewer. See "Defining Code Generation Strategies For The Mapping" on page 7-24 for more information on generating code. Editing Mapping Operator Attributes The Source to Target Mappings 5-39 Figure 524 The Mapping Editor After Editing Attributes Committing Changes To The Mapping OWB does not commit your changes to the repository until you click the Commit icon on the tool bar, or exit OWB and click Yes on the pop-up Commit confirmation panel (Builder always returns this panel on exit). Note: If you generate code at any time during your OWB Session, Multi-User Locking feature will remain in effect until after you commit your changes. Taking The Next Step 5-40 Oracle Warehouse Builder 3i Users Guide Taking The Next Step This chapter described the concepts behind the OWB Mapper and how to create a mapping that will generate the code necessary to move data from a source to a target. It described how to define and update attribute and attribute group properties for mapping operators. This chapter also included basic information regarding how to validate the code and view the code in order to edit it manually. Chapter 6, "Mapping Operators and Transformations" shows you how to add more sophisticated flow operators to a mapping, how to access transforms in the Oracle Library, create custom transformations, and import PL/ SQL procedures that will enhance how your data flows from sources to targets. Mapping Operators and Transformations 6-1 6 Mapping Operators and Transformations This chapter is the second of three chapters that discusses mapping by describing advanced mapping features. Specifically, this chapter describes how to add Data Flow operators that transform and manipulate data as it moves from a source to a target. This chapter describes how to construct transform expressions using specific icons on the Object Palette. It also describes how to use Expression Builder to create transforms using standard transformations selected from the Oracle Library as well as defining custom transformations.. This chapter includes the following sections: s Using Expression Builder s Adding Data Flow Operators To A Mapping Using Expression Builder 6-2 Oracle Warehouse Builder 3i Users Guide Using Expression Builder Mapping operations may require you to add simple SQL expressions to transform your data. The Expression Builder provides an easy to use editor that allows quick construction of expressions by using a drag and drop interface. These expressions can then be verified to be syntactically and semantically correct before leaving the editor. Figure 61 The Expression Builder Interface The following two sections describe Expression Builder and how to use it to build data transform expressions. About Expression Builder An expression component transforms data that flows through it. In order to specify transforms, you need to create SQL expressions for each output parameter defined in an expression component. For example, these expressions may be snippets of Using Expression Builder Mapping Operators and Transformations 6-3 SQL that are used in-line as part of a bigger SQL statement. With Expression Builder, you can drag and drop pre-defined SQL expressions selected from the GUI. These expressions enable you to use s Any input to the operator in which the expression is defined. s Transformations or predefined functions from the Oracle Transform Libraries as well as from Custom Transforms Expression Builder contains the following fields: s The "expression type" title will be dynamically populated based on the expression type. Allowed expressions types include: Filter Condition Join Condition Split Condition Constant Data Generator Output Attribute Output Attribute in Expression Having Clause for Aggregation Update Target Condition Delete Target Condition s The tree items list which displays two tab cards: Inputscontains a list of input attributes to the operator that you are defining the expression in. Transformationscontains a list of the Oracle Transformation Library, the Global Shared Library, and a Custom Transformation Library available within the active module s The description field that displays relevant information about the selected functions. s The expression field within which you build expressions. You create your expressions by typing them manually into the Expression field. You can also create them by dragging them from the predefined functions available from the tree list into the Expression field. Or you may create them by clicking keypad operators available below the Expression field. Using Expression Builder 6-4 Oracle Warehouse Builder 3i Users Guide s A set of Operator buttons from which you can select accepted operators. The available operators vary by the expression type you are building. s A drop-down list of available SQL clauses that are appropriate for the active expression type. s The validate button which you use to validate the current transformation expression in the expression builder. The validation return status will be displayed in the message field located directly below the operator buttons. Creating Expression Components Expression Builder launches from the Operator Properties inspector if a parameter requires an expression. Expression Builder displays an Inputs tab card containing a set of parameters that are appropriate for the type of operator for which you are building and expression. It also displays a Transformation tab card that provides access to the transformation libraries. You can choose from a list of available transforms and parameters in the Oracle Transformation Libraries navigation treesee "About Oracle Transformation Libraries" on page 3-59 for more information. Using Expression Builder Mapping Operators and Transformations 6-5 Figure 62 The Expression Builder Interface Showing Transformation Libraries Expression Builder plays a critical role in defining operators such as Expression, Constant, and Transformation all of which are described in the next section. Therefore, to effectively create definitions for operators, you need a working knowledge of SQL and PL/ SQL; some of the operators described in the next section require you to write SQL and PL/ SQL statements appropriate for your situation. After creating an expression , you can use Expression Builder to evaluate it by clicking the Validate button after you have entered your expression. Validation w ensures that all mapping objects referred to by the expression have affiliated repository objects. The validation system uses the design repository database to syntactically validate the expression. The deployment database could be different than the design repository and thus not accept the expression. As such, the validations your expressions are not guaranteed to be correct. If this occurs, the expression errors can only be found at deployment time Adding Data Flow Operators To A Mapping 6-6 Oracle Warehouse Builder 3i Users Guide Adding Data Flow Operators To A Mapping This section describes how to add and edit flow operators to a mapping. The flow operators enable you to control what data moves from sources to targets. They also enable you to change your data as appropriate once it reaches the data recipient. In many cases, some of these operators may also change the data values themselves. The following instructions describe how to add various flow operators to your mapping. While flow operators differ in how you set them up in a mapping, all have the following common steps: 1. Drag and drop an icon from the Object palette to the Mapping Editor Canvas. 2. Define attributes, procedures, and/ or functions. 3. Connect the Operator to a data provider and/ or to a data recipient. Most flow operators require you to enter a condition statement into a field located in the operator s Property Inspector. You use Expression Builder to do this. See "About Expression Builder" on page 6-2 for detailed information on Expression Builder Filtering Data You can conditionally filter out rows from a row-set using the Filter operator. The Filter operator filters data from a source to a target by placing a WHERE clause in the code represented by the mapping. You connect the Filter operator to an operator that is upstream in the Mapping, perform some filtering based on some condition and pass a subset of rows to any down-stream operator that's connected to it. The Filter operator can have only one INOUT attribute group. This group can be connected to both a source and target row-set. It then produces a row-set that is a filtered subset of the source row-set, based on a boolean filter condition expression. The Filter operator contains the following property: s Filter Condition (operator level)The boolean condition which determines which rows will be passed on to the output row-set. To filter data: 1. If you have not already done so, define a source operator as described in "Selecting A Source Operator" on page 5-17. 2. Drop the Filter icon onto the Canvas. Adding Data Flow Operators To A Mapping Mapping Operators and Transformations 6-7 Figure 63 The Filter Icon 3. Connect the source attributes appropriate for your mapping to the Filter attribute group labeled INOUTGRP1. Figure 64 The Mapping Editor With A Filter Operator 4. Open the Filter Property Inspector by right-clicking the operator header and selecting Operator Properties... from the popup menu. Adding Data Flow Operators To A Mapping 6-8 Oracle Warehouse Builder 3i Users Guide Figure 65 The Filter Property Inspector 5. Click the field to the right of the Filter Condition property and click the ... button to open Expression Builder. OWB returns the Expression Builder window. 6. Create the correct expression in Expression Builder. For the example, create the expression shown in Figure 66 by double-clicking on the appropriate inputs from the INPUTS tab, and using the appropriate operator buttons. You may also type in the expression free form. Adding Data Flow Operators To A Mapping Mapping Operators and Transformations 6-9 Figure 66 The Expression Builder Showing A Filter Condition 7. Click OK in Expression Builder and close the Filter Property inspector. 8. If you have not already done so, define a target operator as described in "Selecting A Source Operator" on page 5-17 and connect the filter outputs to the target INOUT attribute group 9. Link the appropriate data provider attributes to the Filter and then to the appropriate data target attributes. 10. Select Mapping > Generate Mapping from the Mapping Editor menu. OWB returns the Code Viewer window. Adding Data Flow Operators To A Mapping 6-10 Oracle Warehouse Builder 3i Users Guide Figure 67 The Code Viewer Window 11. Examine the generated code for this map in the code viewer dialog box. You will see the filter condition expression generated as a "WHERE" clause (for set-based view mode). The filter input names in the original filter condition have been replaced by actual column names from the source table, qualified by the source table alias. Ordering Data You can produce a sorted row-set through the Sorter operator. The Sorter allows you to specify on which input attributes the sorting is performed and whether the sorting is performed in ascending or descending order. The Sorter operator has one input/ output attribute group. This group can be connected from any output or input/ output attribute group and to any target input Adding Data Flow Operators To A Mapping Mapping Operators and Transformations 6-11 or input/ output attribute group. OWB sorts provided data moving to a data recipient by placing an ORDER BY clause in the code represented by the mapping. The Sorter operator contains the following properties: s Order By Expression (operator level)This property is an ordered list of attributes in the input/ output attribute group, specifying that sorting is performed in the same order as the ordered attribute list. Ascending or descending sorting can be set for each attribute participating in the sort. To order data: 1. Drag a source operator from the Object Palette onto the Mapping Editor Canvas. 2. Drag the Mapping Sorter operator from the Object Palette onto the Mapping Editor Canvas. Figure 68 The Sorter Icon 3. Drag a target operator from the Object Palette onto the Mapping Editor Canvas. 4. Draw a line from the input/ output attribute group of the source operator to the input/ output attribute group INOUTGRP1 of the Mapping Sorter. Adding Data Flow Operators To A Mapping 6-12 Oracle Warehouse Builder 3i Users Guide Figure 69 The Mapping Editor With A Sorter Operator 5. Open the Sorter Property Inspector by right-clicking the operator header. 6. Click the field to the right of the Order By Expression property. Adding Data Flow Operators To A Mapping Mapping Operators and Transformations 6-13 Figure 610 The Sort Condition Property 7. Click the ... button to open the Order By Expression window. Adding Data Flow Operators To A Mapping 6-14 Oracle Warehouse Builder 3i Users Guide Figure 611 The Sort By Expression Window 8. Select the attributes you want to sort and define whether they will ascend or descend clicking the appropriate button. 9. Click OK. Splitting Data You can use the Splitter operator to split a single input row-set into several output row-sets. The Splitter uses a boolean split condition as a filter to produce each output row-set. Each output rowset will have a cardinality less than or equal to the input cardinality. The effect of the splitter is similar to having several Filter operators connected to a single upstream source. The Splitter operator creates an output group called REMAINING_ROWS. If you use this group, it will produce a row-set containing all input rows not included in any of the other output groups.You may delete this output group if you want, but you cannot edit it. The Splitter Operator contains the following properties: Adding Data Flow Operators To A Mapping Mapping Operators and Transformations 6-15 s Split Condition (output attribute groups)The text expression template for the Split Condition. For code generation, the actual source columns are substituted for the input attribute names in the expression template. Generally, the expression should be a legal SQL expression which could be used in a "WHERE" clause. See the "languages" section in chapter 7 for a fuller description of the syntax rules for expressions in various contexts. s Data Type (attributes)The data type of the attribute. s Precision (attributes)The precision of the attribute (used for numeric type attributes only). s Scale (attributes)The scale of the attribute (used for numeric type attributes only). s Length (attributes)The length of the attributes (used for string-type attributes only). The following instructions describe how to split data from a single source to three targets. To split data from one source to several targets: 1. Drag a source operator from the Object Palette onto the Mapping Editor Canvas. 2. Drag the Splitter operator from the Object Palette onto the Mapping Editor Canvas. Figure 612 The Splitter Icon 3. Drag a target operator from the Object Palette onto the Mapping Editor Canvas. 4. Drag a second target operator from the Object Palette onto the Mapping Editor Canvas. 5. Drag a third target operator from the Object Palette onto the Mapping Editor Canvas. 6. Connect the attribute group from INOUTGRP1 of the source to the INGRP1 of the Splitter. Adding Data Flow Operators To A Mapping 6-16 Oracle Warehouse Builder 3i Users Guide Figure 613 The Mapping Editor With A Splitter Operator All output attributes are created automatically, and output attribute data types will be automatically set to match the data type of the corresponding input 7. Open the Splitter Property Inspector by right-clicking a Group header. 8. Click the field to the right of the Split Condition property. Adding Data Flow Operators To A Mapping Mapping Operators and Transformations 6-17 Figure 614 The Split Condition Property Inspector 9. Click the ... button immediately to the right of the Split Condition property to open Expression Builder. 10. Enter a Split Condition in the Expression Builder Editor. Figure 615 The Expression Builder Showing A Split Condition 11. Click Validate to validate the Split Condition expression. 12. Click OK. Adding Data Flow Operators To A Mapping 6-18 Oracle Warehouse Builder 3i Users Guide 13. Close the Attribute Group Property inspector. 14. Repeat the previous 5 steps except select the appropriate OUTGRP on the Splitter. 15. Connect OUTGRP1on the Splitter to INOUTGRP1 in the first target. 16. Repeat the previous step except connect the appropriate OUTGRP on the Splitter to the appropriate INOUTGRP on the targets. 17. Select Mapping > Generate > Mapping from the Mapping Editor menu. OWB opens the Mapping Code Viewer. Figure 616 The Mapping Code Viewer 18. Examine the generated code for this map in the code viewer dialog box. You should see the Split Conditions in the WHERE clauses of the SELECT queries for the insert DML (There will be a separate query for each target). Adding Data Flow Operators To A Mapping Mapping Operators and Transformations 6-19 De-Duplicating Data OWB enables you to remove duplicate data from a source to a target by placing a DISTINCT clause in the select code represented by the mapping. You use the Deduplicator to perform this operation. To eliminate duplicate data: 1. Drag a source operator from the Object Palette onto the Mapping Editor Canvas. 2. Drag a target operator from the Object Palette onto the Mapping Editor Canvas. 3. Drag the Deduplicator operator from the Object Palette onto the Mapping Editor Canvas. Figure 617 The Deduplicator Icon 4. Map the attributes in the source operator to the input/ output attribute group INOUTGRP1 of the deduplicator operator. 5. Map the attributes in the input/ output attribute group INOUTGRP1 of the deduplicator operator to the attributes of the target operator. Adding Data Flow Operators To A Mapping 6-20 Oracle Warehouse Builder 3i Users Guide Figure 618 The Mapping Editor With A Deduplicator 6. Click the input/ output attribute group INOUTGRP1 of the target operator. 7. Select menu item Mapping > Generate > Intermediate Result from the Mapping Editor menu. OWB returns the Code Viewer window showing DISTINCT in the select statement. Adding Data Flow Operators To A Mapping Mapping Operators and Transformations 6-21 Figure 619 The Code Viewer Showing Distinct In the Code Aggregating Data The Aggregator performs data aggregations, such as SUM or AVG, and provides an output rowset with aggregated data. Each Aggregator operator shares a GROUP BY and HAVING clause, so each output parameter in the output parameter group has the same cardinality. This operator delivers an aggregated row-set derived from the input row-set; the number of rows in the output row-set is less than or equal to the number of input rows. The Aggregator operator has one input attribute group and one output attribute group. The source row-set can be connected to input attribute group and the aggregator operator produces the corresponding aggregated row-set in its output attribute group. The Aggregator operator contains the following properties: s Group By Clause (operator level)This property defines how the incoming row-set is grouped to return a single summary row for each group. This consists of an ordered list of attributes in the input attribute group specifying how this grouping is performed. All output attributes with no aggregation function applied (that is Function in the Expression property for those attributes is NONE ) is necessary to be included in this clause. s Having Clause (operator level)This property is a boolean condition restricting the groups of rows returned in the output attribute group to Adding Data Flow Operators To A Mapping 6-22 Oracle Warehouse Builder 3i Users Guide those groups for which this condition is true. If this clause is not specified, all the summary rows for all the groups will be returned in the output attribute group. This clause can refer to any attributes or an expression (possibly with aggregation functions) of the attributes in the input attribute group. s Expression (attribute level)This property defines the aggregation functions to be performed on the attribute where this expression is specified. If no aggregation function is necessary, NONE should be specified for the Function of this property. To add an aggregator to a mapping: 1. Drag a source operator from the Object Palette onto the Mapping Editor Canvas. 2. Drag a target operator from the Object Palette onto the Mapping Editor Canvas. 3. Drag an Aggregator operator from the Object Palette onto the Mapping Editor Canvas. Figure 620 The Aggregator Icon On The Object Palette 4. Map the attributes from the source operator appropriate for your situation to the attribute group INGRP1 of the aggregator operator. 5. Right-click on the attribute group header OUTGRP1 of the aggregator operator and select Add/Remove Attributes from the popup menu. OWB opens the Add/ Remove Attributes dialog. 6. Add the appropriate attributes. See "Adding or Removing Operator Attribute Groups" on page 5-33 for information on this dialog. 7. Right-click on an attribute in the Aggregator operator and select Attribute Properties from the popup menu. OWB opens the Aggregator Properties Inspector. Adding Data Flow Operators To A Mapping Mapping Operators and Transformations 6-23 Figure 621 The Aggregator Properties Inspector 8. Edit the Expression property by clicking the button to the right of that property. OWB opens the Expression dialog. Select a function and a parameter from the drop down lists. Figure 622 The Expression Dialog 9. Click OK. 10. Repeat steps 7-9 as appropriate for additional attributes on the Aggregator operator. 11. Click the title bar of the aggregator operator in the canvas. The property inspector will be refreshed with the properties for the operator the aggregator operator. 12. Edit the property called Group By Clause by clicking the button to the right of that property. OWB opens the Group By Clause window. Adding Data Flow Operators To A Mapping 6-24 Oracle Warehouse Builder 3i Users Guide Figure 623 The Group By Clause Window 13. Verify that your properties appear in the list of GROUP BY parameters. 14. Click OK. 15. Edit the property called Having Clause by clicking the button to the right of that property. OWB opens Expression Builder. Use Expression Builder to create a sum expressionor example, sum(INGRP1.OR_TOTAL) > 10000. Figure 624 The Expression Builder Showing A Sum Statement Adding Data Flow Operators To A Mapping Mapping Operators and Transformations 6-25 16. Map the attributes you edited from the attribute group OUTGRP1 in the aggregator operator to the attributes in the target. Figure 625 Joining Data Sources You can use the Joiner operator to join together multiple row-sets from different sources, which may have different cardinalities, and produce a single output row-set. The Joiner operator is associated with each output attribute group for a data provider. It is a boolean condition expression that relates column values in each source row-set to at least one other row-set(s).
Tip: You can have intervening operators between the data source operator and the Joiner (such as an expression operator), but an arbitrarily complex and non-performing SQL or PL/ SQL statement may be generated. Therefore, a Joiner should immediately follow the data source operators to be joined in the mapping. Adding Data Flow Operators To A Mapping 6-26 Oracle Warehouse Builder 3i Users Guide If the input row-sets are related by foreign key relationships in the repository, OWB will use these foreign key relationships to form a default join condition. You can use this default condition as it is or you can modify it. If the sources are not related by foreign key relationships, then you must define a join condition. The join condition property for any output parameter group can be left blanka blank condition will not generate a validation error in the expression builder. However, by leaving the join condition blank, the Joiner operator brings in all of the columns of the source it references despite of the number of columns actually being mapped which can increase the overhead in the ETL process. Therefore, you may want to create a complete join condition for the first input attribute group but leave the rest blank. The advantage of doing it the first way is that an input attribute group could be removed and the final join condition would still be correct. You may define a join condition to relate columns that are not unique or foreign keys in the source table. However, the performance of the generated query will be severely degraded. For this case, it might be better to create an index on the desired column in the source row-set before attempting to run the generated map. The join condition may contain other relational operators. Again, it is best if the related operand columns are both indexed in the sources. However, most real-life applications would have no need for relational operators other than = . The join condition may contain transform function calls, but if it does the performance of the generated query may be severely degraded. For this case it would be better to create a stage table with a column for the transformed column, populate the stage table with another map, then join the stage table with the other sources.ee Appendix F, "Performance Enhancements" for more information on the value of staging tables. The join condition may contain arithmetic operators to form a special join condition (for example, LINES.PO_OR_ORDER_ID + 10 = ORD.ORDER_ID). However, this will also degrade performance because all values from the source table have to be read and transformed to find the joined rows for a given row. Therefore, it would be better to create a stage table with the transformed data before joining. See Appendix F, "Performance Enhancements" for more information on the value of staging tables. The join condition expression can not contain any aggregation functions (SUM, etc). If a join condition expression contains aggregation functions, compile errors will occur when deploying the generated code for the map. You may add an unlimited number of input groups to the Joiner. However, only one output group is allowed. Adding Data Flow Operators To A Mapping Mapping Operators and Transformations 6-27 The Joiner operator contains the following properties: s Join Condition (operator level)The text expression template for the Join Condition. For code generation, the actual source columns are substituted for the input attribute names in the expression template. Generally, the expression should be a legal SQL expression which could be used in a WHERE clause. s Data Type (attributes)The data type of the attribute. s Precision (attributes)The precision of the attribute (used for numeric type attributes only). s Scale (attributes)The scale of the attribute (used for numeric type attributes only). s Length (attributes)The length of the attributes (used for string-type attributes only). To add a Joiner to a mapping: 1. Drag a first source operator from the Object Palette and drop it onto the Mapping Editor Canvas. 2. Drag a second source operator from the Object Palette and drop it onto the Mapping Editor Canvas. 3. Drag a target operator from the Object Palette and drop it onto the Mapping Editor Canvas. 4. Drag the Joiner operator from the Object Palette and drop it onto the Mapping Editor Canvas. Figure 626 The Joiner Icon 5. Connect the INOUTGRP1 attribute group from the first source to the INGRP1 group of the Joiner operator. OWB automatically create all of the output attributes. The output attribute data types are set to match the data type of their corresponding input data types. Adding Data Flow Operators To A Mapping 6-28 Oracle Warehouse Builder 3i Users Guide 6. Connect the INOUTGRP1 attribute group from the second source operator to the INGRP2 group of the Joiner operator. Figure 627 The Mapping Editor Showing A Joiner Operator 7. Right-click the title bar of the Joiner operator and select the Operator Properties from the popup menu. Tip: Rename the input group on the Joiner operator to more accurately reflect its function. For the example, right click on INGRP1 on the Joiner operator and select menu Rename from the popup menu. Enter a new name in the Rename dialog and click OK Adding Data Flow Operators To A Mapping Mapping Operators and Transformations 6-29 Figure 628 The Join Operator Property Inspector 8. Click the button immediately to the right of the Join Condition property. OWB opens the Expression Builder indicating the default join condition for what you have mapped. Adding Data Flow Operators To A Mapping 6-30 Oracle Warehouse Builder 3i Users Guide Figure 629 The Expression Builder Showing A Join Condition 9. Click Validate. OWB returns the results in the Validation Results window in Expression Builder. 10. Click OK. 11. Close the Joiner Operator Property inspector. 12. Select Mapping > Generate > Mapping from the Mapping Editor menu. OWB opens the Code Viewer dialog which displays the join condition in the WHERE clause of the SELECT query for the insert DML. Adding Sequences A sequence operator provides numeric data which increments each time it is referenced. The operator contains a display set named OUTGRP which contains two Adding Data Flow Operators To A Mapping Mapping Operators and Transformations 6-31 parameters called CURRVAL and NEXTVAL and no inputs. You determine which parameter to use as the output of the sequence operator. You can generate a row-set of consecutively incremented values using the Sequence operator. You can connect a Sequence operator to a target operator input or to the inputs of many other types of down-stream operators. The Sequence operator is one of the exceptions to the rule that all attributes of an input attribute group must be connected to attributes from the same output group. You may combine the sequence outputs with outputs from other up-stream operators . You can bind a Sequence operator to a repository sequence in one of the repository modules, but you must reconcile it to that object. The repository sequence must then be generated and deployed before the map is deployed or the generated package for the map will have errors. Sequences are incremented even if rows are not selected. If you want a sequence to start from the last number, then you should not run your SQL package in Set Based or in Set Based With Failover operating modes. See Setting The Default Operating Mode on page 7-28 for more information on configuring mode settings. The Sequence operator contains the following property: s Bound Name (operator level)The name of the physical sequence that will be used in the generated code. If the sequence has been reconciled from a repository component, the Bound Name will be equal to the physical name of the repository sequence. To add a Sequence to a mapping: 1. Drag a source operator from the Object Palette and drop it onto the Mapping Editor canvas. 2. Drag a target operator from the Object Palette and drop it onto the Mapping Editor canvas. 3. Drag the Sequence icon from the Object Palette and drop it onto the Mapping Editor Canvas. Figure 630 The Sequence Operator Icon Adding Data Flow Operators To A Mapping 6-32 Oracle Warehouse Builder 3i Users Guide Builder returns the Add Sequence dialog where you can select one of the four options: s If you select Create new repository sequence and bind, then select the warehouse module where you want to the sequence to reside and click OK. Builder returns the New Sequence Wizard. Follow the wizard steps to create a new sequence. s If you select Import sequence into repository and bind, then select the source module from where you are importing the table and click OK. Builder returns the Database Link Information dialog. Click New DB Link to establish connection with your source database and click OK. Builder returns the Import Object Wizard. s If you choose Select from existing repository sequence and bind, you can either type the prefix to search for a sequence or select from the displayed list of sequences within the selected module and click OK. Figure 631 The Add Sequence Dialog Adding Data Flow Operators To A Mapping Mapping Operators and Transformations 6-33 4. Drag the source operator from the palette onto the map. . 5. Drag the sequence operator from the palette onto the map. 6. Connect the appropriate source attributes to the appropriate target operator attributes. 7. Connect the sequence to the appropriate attribute in the target. Figure 632 The Mapping Editor Showing A Sequence Operator 8. Select Mapping > Generate > Mapping from the Module Editor menu. OWB opens the Code Viewer dialog displaying the sequence in the SELECT list for the insert DML. Adding Data Flow Operators To A Mapping 6-34 Oracle Warehouse Builder 3i Users Guide Figure 633 The Code Viewer Showing A Sequenced Load Adding External Processes The external process operator provides the means to represent a process not defined by OWB in order to incorporate it into OWBs data flow. The external process operator comprises a complete, self-contained operation and requires no inputs or outputs. During code generation, OWB generates a job control code (TCL script) for external process operators that can be deployed as part of the OWB workflow. Three types of processes are available in OWB : s OS Executable s Pure Integrate s Pure Extract Adding Data Flow Operators To A Mapping Mapping Operators and Transformations 6-35 External Processes are self-contained; you may not use an external process with other mapping operators. To add an External Process operator to a mapping: 1. Drag the External Process icon from the Object Palette and drop it onto the Mapping Editor Canvas. Figure 634 The External Process Icon 2. Right-click the External Process operator header OWB returns the External Process Properties inspector. Figure 635 The External Process Property Inspector 3. Click field . Adding Data Flow Operators To A Mapping 6-36 Oracle Warehouse Builder 3i Users Guide 4. Select an External Process from the pull down list to the right of the External Process property 5. Select the map that contains the External process from the navigation tree in the Warehouse Module Editor. 6. Click Configure. OWB returns the Configuration Properties inspector. Figure 636 The External Process Configuration Properties Inspector 7. Expand the Steps property and specify the process directory location and input parameters or, in case of Pure Extract, the status file name. Adding Data Generators You use a Data Generator operator to provide pseudo-column information. It allows for downstream mapping of information such as record number, system date, and sequence values. It also provides a place to enter constant information. The Data Adding Data Flow Operators To A Mapping Mapping Operators and Transformations 6-37 Generator operator connects the mapping in OWB to SQL*Loader to generate the data stored in the database record, rather than reading it from a datafile. The following functions are available: s RECNUM s SYSDATE s SEQUENCE It is possible to for OWB to generate data by specifying only sequences, record numbers, system dates, and constants as field specifications. SQL*Loader inserts as many records as are specified by the LOAD keyword. Setting a Column to the Datafile Record Number Map the RECNUM keyword to an attribute to set that attribute to the number of the logical record from which that record was loaded. Records are counted sequentially from the beginning of the first datafile, starting with record 1. RECNUM is incremented as each logical record is assembled. Thus it increments for records that are discarded, skipped, rejected, or loaded. For example, if you use the option SKIP=10, the first record loaded has a RECNUM of 11. Setting a Column to the Current Date A column specified with SYSDATE gets the current system date, as defined by the SQL language SYSDATE function. The target column must be of type CHAR or DATE. If the column is of type CHAR, then the date is loaded in the form dd-mon-yy. After the load, it can be accessed only in that form. If the system date is loaded into a DATE column, then it can be accessed in a variety of forms that include the time and the date. A new system date/ time is used for each array of records inserted in a conventional path load and for each block of records loaded during a direct path load. Setting a Column to a Unique Sequence Number The SEQUENCE keyword ensures a unique value for a particular column. SEQUENCE increments for each record that is loaded or rejected. It does not increment for records that are discarded or skipped. Adding Data Flow Operators To A Mapping 6-38 Oracle Warehouse Builder 3i Users Guide The combination of column name and the SEQUENCE function is a complete column specification. If a record is rejected (that is, it has a format error or causes an Oracle error), the generated sequence numbers are not reshuffled to mask this. For example, if four rows are assigned sequence numbers 10, 12, 14, and 16 in a particular column, and the row with 12 is rejected; the three rows inserted are numbered 10, 14, and 16, not 10, 12, 14. This allows the sequence of inserts to be preserved despite data errors. When you correct the rejected data and reinsert it, you can manually set the columns to agree with the sequence. The Data Generator operator has only one output group. The Data Generator operator has predefined attributes corresponding to Record Number, System Date, and a typical Sequence. Modification of these attributes is not recommended but you can create new attributes. You must ensure the value entered for the expression is valid SQL*Loader syntax in the context used. There can be at most one Data Generator operator for a given map. The Data Generator operator is only valid for an SQL*Loader mapping. The Data Generator contains the following properties: s Expression (common attribute property)expression to use when this attribute is mapped. It is the responsibility of the user to ensure this expression contains valid SQL*Loader syntax for where the attribute is used. To add a Data Generator to a mapping: Table 61 The Sequence Value Options Value Description column_name The name of the column in the database to which to assign the sequence. SEQUENCE Use the SEQUENCE keyword to specify the value for a column. integer Specifies the specific sequence number to begin with. COUNT The sequence starts with the number of records already in the table plus the increment. MAX The sequence starts with the current maximum value for the column plus the increment. incr The value that the sequence number is to increment after a record is loaded or rejected. Adding Data Flow Operators To A Mapping Mapping Operators and Transformations 6-39 1. Drag the Data Generator icon from the Object Palette and drop it onto the Mapping Editor Canvas. Figure 637 The Data Generator Icon 2. Right-click the RECNUM attribute. 3. Select Attribute Properties... from the popup menu. OWB returns the RECNUM Attribute Properties inspector. Figure 638 The RECNUM Attribute Properties Inspector 4. Click on field to the right of Expression. 5. Click the ... button located on the right side of the field. OWB returns the Expression Builder. 6. Use Expression Builder to define an Expression for RENUM. 7. Repeat steps two through six for the SYS_DATE and SEQUENCE attributes if necessary. Adding Data Flow Operators To A Mapping 6-40 Oracle Warehouse Builder 3i Users Guide Adding Expressions The expression operator enables you to write SQL expressions, each of which defines a non-procedural algorithm for one output parameter of the operator. The expression text can contain combinations of input parameter names, variable names, and library functions. You use the Expression operator to transform the column value data of rows within a row-set using SQL-type expressions, while preserving the cardinality of the input row-set. To create these expressions, you open the Attribute Property inspector for the output attribute and open Expression Builder. You may have only one input attribute group and one output attribute group in the Expression operator, both of which are created automatically when you drop it on the Mapping Editor canvas. The output expressions for this operator can not contain any aggregation functions. To use aggregation functions, use the Aggregator operator. The Expression operator contains the following properties: s Data Type (attributes)The data type of the attribute. s Precision (attributesThe precision of the attribute (used for numeric type attributes only). s Scale (attributes)The scale of the attribute (used for numeric type attributes only). s Length (attributesThe length of the attributes (used for string-type attributes only). s Expression (output attributes)The text expression template for the output attribute. For code generation, the actual source columns are substituted for the input attribute names in the expression template. To add an expression operator: 1. Drag source operator from the Object Palette and drop it onto the Mapping Editor Canvas. 2. Drag a target operator from the Object Palette and drop it onto the Mapping Editor Canvas. 3. Drag the Expression operator from the Object Palette and drop it onto the Mapping Editor Canvas. Adding Data Flow Operators To A Mapping Mapping Operators and Transformations 6-41 Figure 639 The Expression Icon 4. Connect the appropriate source attributes to the appropriate target table attributes. 5. Connect the appropriate source attributes to the INGRP of the Transformation Operator. This automatically creates the input attributes. 6. Create an output attribute named OUTPUT1 in the Expression operator output attribute group. s Right-click OUTGRP. s Select Add/Remove Attributes from the popup menu. s Click Add. s Click OK. 7. Select the OUTPUT1 attribute, right-click it, and select the Attribute Properties from the popup menu. Adding Data Flow Operators To A Mapping 6-42 Oracle Warehouse Builder 3i Users Guide Figure 640 The Attribute Properties Inspector 8. Open Expression Builder to edit the expression for the attribute. s Click on the button immediately to the right of the Expression property. s Create an expression by using the Inputs tree, the expression operator buttons, or type in the expression manually. Adding Data Flow Operators To A Mapping Mapping Operators and Transformations 6-43 Figure 641 The Expression Builder For Defining An Expression 9. Click Validate. You should see a message in the Validation results window reading Validation Successful!. 10. Click OK to close Expression Builder. 11. Close the attribute property editor. 12. Connect OUTPUT1 in the Expression output attribute to the appropriate target attribute. Adding Data Flow Operators To A Mapping 6-44 Oracle Warehouse Builder 3i Users Guide Figure 642 The Mapping Editor Showing An Expression Operator 13. Select Mapping > Generate > Mapping from the Mapping Editor main menu. OWB opens the Code Viewer window displaying the expression. Adding Constants You can define constants for use in the mapping through the constant operator. These constants are initialized at the beginning of the execution of the mapping and these values can be used as inputs to pre-mapping process, post-mapping process, mapping output parameter and other operators in the data flow. The constant operator can have only one output attribute group. This group can be connected to any target input or input/ output attribute group. If the constant defined is of data type VARCHAR or VARCHAR2, its expression needs to be a valid Adding Data Flow Operators To A Mapping Mapping Operators and Transformations 6-45 SQL expression returning a value of data type VARCHAR or VARCHAR2. In the case for a simple constant string literal, it needs to be within single quotes. The Constant operator contains the following properties: s Expression (attribute level)This property defines the constant value represented by the attribute. It needs to be a valid SQL or PL/ SQL expression returning a value that is of the same data type of the attribute. To add a constant operator: 1. Drag source operator from the Object Palette and drop it onto the Mapping Editor Canvas. 2. Drag a target operator from the Object Palette and drop it onto the Mapping Editor Canvas. 3. Drag the Constant icon from the Object Palette and drop it onto the Mapping Editor Canvas. Figure 643 The Constant Icon 4. Connect the appropriate source attributes to the appropriate target table attributes. 5. Connect the appropriate source attributes to the INGRP of the Transformation Operator. This automatically creates the input attributes. 6. Create an output attribute named OUTPUT1 in the Expression operator output attribute group. s Right-click OUTGRP. s Select Add/Remove Attributes from the popup menu. s Click Add. s Click OK. 7. Select the OUTPUT1 attribute, right-click it, and select the Attribute Properties from the popup menu. Adding Data Flow Operators To A Mapping 6-46 Oracle Warehouse Builder 3i Users Guide Figure 644 The Attribute Properties Inspector 8. Open Expression Builder to edit the expression for the attribute. s Click on the button immediately to the right of the Expression property. s Create an expression by using the Inputs tree, the expression operator buttons, or type in the expression manually. Adding Data Flow Operators To A Mapping Mapping Operators and Transformations 6-47 Figure 645 The Expression Builder For Defining A Constant 9. Click Validate. You should see a message in the Validation results window reading Validation Successful!. 10. Click OK to close Expression Builder. 11. Close the attribute property editor. 12. Connect OUTPUT1 in the Constant output attribute to the appropriate target attribute. Adding Data Flow Operators To A Mapping 6-48 Oracle Warehouse Builder 3i Users Guide Figure 646 The Mapping Editor Showing Constant Operator 13. Select Mapping > Generate > Mapping from the Mapping Editor main menu. OWB opens the Code Viewer window displaying the constant. Adding Data Flow Operators To A Mapping Mapping Operators and Transformations 6-49 Figure 647 The Code Viewer For A Constant Adding Transformations You use the Transformation Operator to transform the column value data of rows within a row-set using a PL/ SQL function, while preserving the cardinality of the input row-set. The Transformation Operator must be bound to a function or procedure contained by one of the modules in the repository. The inputs and outputs of the Transformation Operator will correspond to the input and output parameters of the bound repository function or procedure. Also, if the Transformation Operator is bound to a function, a "result" output is added to the operator that corresponds to the result of the function. The bound function or procedure must be generated and deployed before the mapping can be deployed, unless the function or procedure already exists in the repository. Adding Data Flow Operators To A Mapping 6-50 Oracle Warehouse Builder 3i Users Guide When you place a transformation operator onto the Mapping Editor Canvas, a selection dialog opens displaying a tree of function libraries, categories, functions and procedures. You pick a given function or procedure from the list and the operator appears on the Mapping Editor Canvas with the input and output parameters predefined. OWB provides a set of pre-defined PL/ SQL library functions that already exist in the installed run-time schema, any of which can be selected as a bound function when dropping a Transformation Operator on a mapping. In addition, you may choose a function or procedure from the Global Shared Library. The Transformation operator contains the following properties: s Function Call (operator level, read-only)This is the text template for the function call that will be generated by the code generator for the function call, with the operator attribute names listed as the calling parameters. For the actual call, the operator attribute names will be replaced with the actual source or target columns that are connected to the attributes. s Function Name (operator level, read-only)The name of the function (or procedure) to which this operator is bound. s Procedure (operator level, read-only)A boolean value indicating, if true, that the bound transform is a procedure rather than a function and as such, it gives no returned value s Data Type (attributes, read-only)Indicates the data type of the input, output, or result parameter of the bound function which corresponds to the given attribute. s Default Value (input attributes, read-only)The default value (blank if none) for the given attribute. s Optional Input (input attributes, read-only)A boolean value indicating, if true, that the given attribute is optional. If the attribute is optional, it does not need to be connected in the mapping. s Function Return (output attributes, read-only)A boolean value indicating, if true, that the given output attribute is the result attribute for the function. The result attribute is normally named result, but this property exists to avoid confusion in case there is another output named result, or the user changes the name of the result output. To add a transformation operator: 1. Drag a source operator from the Object Palette onto the Mapping Editor canvas. Adding Data Flow Operators To A Mapping Mapping Operators and Transformations 6-51 2. Drag a target operator from the Object Palette onto the Mapping Editor canvas. 3. Drag a Transformation operator from the Object Palette onto the Mapping Editor canvas. Figure 648 The Transformation Operator Icon The editor returns the Add Transformation page. Figure 649 The Add Transformation Dialog Adding Data Flow Operators To A Mapping 6-52 Oracle Warehouse Builder 3i Users Guide 4. Select one of the three options by clicking it. See "Adding Sequences" on page 6-30 for information about each option. s Create a new transformation repository and bind s Import existing transformation into repository and bind s Select from existing repository transformation and bind This selection contains a search text box and a directory tree for all the transforms stored in the OWB repository. 5. Click on the appropriate node to locate a Transformation function or procedure. 6. Double-click on a transformation to select it. OWB returns to the Mapping Editor displaying a transformation operator Figure 650 The Transformation Operator 7. Connect the appropriate source attributes to the appropriate target table attributes for your application. 8. Connect the appropriate source attributes to the inputs of the Transformation Operator. 9. (Optional step) Select one of the inputs, right-click it, and select the Attribute Properties from the popup menu. OWB opens the Attribute Properties inspector. Adding Data Flow Operators To A Mapping Mapping Operators and Transformations 6-53 Figure 651 The Attribute Properties Inspector s Select an input attribute. s Notice that the Optional Parameter property is true. If it reads true, then you need not connect the input parameter. s Close the attribute property inspector. 10. Connect the Transformation operator output attributes to the pirate target attributes. Adding Data Flow Operators To A Mapping 6-54 Oracle Warehouse Builder 3i Users Guide Figure 652 The Mapping Editor Showing A Transformation Operator 11. Select Mapping > Generate > Mapping from the Mapping Editor main menu. OWB opens the Code Viewer window displaying the transformation. Adding Mapping Input Parameters An input parameter operator enables you to pass parameter values into a mapping. These values can then be used to affect the actions of a mapping. The input parameter has a cardinality of one and effectively creates a single row-set that can be combined with another row-set as input to a downstream operator. The names of the input attributes become the names of the mapping output parameters. The parameters can be used by connecting the attributes of the mapping input parameters operator within the mapping editor. There can be at Adding Data Flow Operators To A Mapping Mapping Operators and Transformations 6-55 most one mapping input parameter operator and one mapping output parameter in a given mapping. The default value for the mapping parameter for code generation of the PL/ SQL "main" function will appear in the DEFAULT clause following the function parameter declarations in the generated "main function definition in the PL/ SQL package. For example, if a mapping parameter named param1 with data type VARCHAR2 is defined, with a default value of HELLO, the generated main function in the PL/ SQL package will appear as: ... param1 IN VARCHAR2 DEFAULT HELLO ... Or, if a mapping parameter output named "param1" with data type VARCHAR2 is defined the generated main function in the PL/ SQL package will appear as: ... param1 IN VARCHAR2 ... The Mapping Input operator contains the following properties: s Default Value (attribute levelThe character string value which, if specified, will be place in the generated code as the default value for the specified attribute: i.e. if the value entered is 1-JUN-2001 then the generated code would contain DEFAULT 1-JUN-2001 s Data Type (common attribute level)Specifies the data type for this input parameter The following instructions assume you have already mapped a source operator to a target operator. To add an input parameter to a mapping: 1. Drag a Mapping Input operator from the Object Palette onto the Mapping Editor Canvas. Figure 653 The Mapping Input Operator 2. Add an attribute appropriate for your mapping to the Mapping Input operator: s Right-click the MAP_INPUT attribute group. Adding Data Flow Operators To A Mapping 6-56 Oracle Warehouse Builder 3i Users Guide s Select Add/ Remove Attributes from the popup menu. s Add a name for the attribute in the Add/ Remove Attributes dialog. s Click OK. Figure 654 The Add/Remove Attributes Dialog 3. Change the Data Type for the new attribute as appropriate for your application by using the attribute properties Inspector: s Right-click the new property. s Select Attribute Properties... from the popup menu. s Select a new property from the pulldown list. Adding Data Flow Operators To A Mapping Mapping Operators and Transformations 6-57 Figure 655 The Attribute Properties Inspector 4. Map the MAP_INPUT attribute to the INOUTGRP1 of the appropriate target operator. Adding Data Flow Operators To A Mapping 6-58 Oracle Warehouse Builder 3i Users Guide Figure 656 The Mapping Editor Showing An Input Parameter Operator 5. If necessary, edit target attributes condition to match the new attribute in the Input Mapping operator. 6. Select Mapping > Generate > Mapping from the Mapping Editor main menu. OWB opens the Code Viewer window displaying the main procedure entry point to the mapping containing the parameter. Adding Data Flow Operators To A Mapping Mapping Operators and Transformations 6-59 Figure 657 The Code Viewer Showing A Mapping Input Parameter Adding Mapping Output Parameters You can use a Mapping Output Parameter operator to pass values out of a mapping. The Output Parameter operator must have exactly one Input attribute group. There can be at most one Output Parameter operator for a given map. Only attributes not associated with an actual row set may be mapped into an Output Parameter, i.e. Constant, Input Parameter, output from a Pre Process, or output from a Post Process. An Output Parameter operator is not valid for an SQL*Loader mapping. The default value for the mapping parameter for code generation of the PL/ SQL main function will appear in the DEFAULT clause following the function parameter declarations in the generated main function definition in the PL/ SQL package. For example, if a mapping parameter named param1 with data type VARCHAR2 is Adding Data Flow Operators To A Mapping 6-60 Oracle Warehouse Builder 3i Users Guide defined, with a default value of HELLO, the generated main function in the PL/ SQL package will appear as: ... param1 OUT VARCHAR2 DEFAULT HELLO ... Or, if a mapping parameter output named param1 with data type VARCHAR2 is defined the generated main function in the PL/ SQL package will appear as: ... param1 OUT VARCHAR2 ... The Mapping Output Parameter operator contains the following properties: s Data Type (common attribute level)Specifies the data type for this output parameter s Bound Name (common attribute level)Specifies the actual physical name for this output parameter The following instructions assume you have already created a mapping. To add an Output parameter to a mapping: 1. Drag a Mapping Output operator from the Object Palette onto the Mapping Editor Canvas. Figure 658 The Mapping Output Operator 2. Add an output attribute to the MAP_OUTPUT operator. s Right-click the MAP_OUTPUT attribute group. s Select Add/ Remove Attributes from the popup menu. s Add a name for the attribute in the Add/ Remove Attributes dialog. s Click OK. Adding Data Flow Operators To A Mapping Mapping Operators and Transformations 6-61 Figure 659 The Add/Remove Attributes Dialog 3. Change the data type of the new attribute to as appropriate for your application by using the Attribute Properties inspector. s Right-click the new property. s Select Attribute Properties... from the popup menu. s Select a new property from the pulldown list. Figure 660 The Attribute Properties Inspector Adding Data Flow Operators To A Mapping 6-62 Oracle Warehouse Builder 3i Users Guide 4. Connect the MAP_OUTPUT attribute to the INOUTGRP1 of the appropriate target operator. Figure 661 The Mapping Editor Showing An Output Parameter Operator 5. Select Mapping > Generate > Mapping from the Mapping Editor main menu. OWB opens the Code Viewer window displaying the main procedure exit point to the mapping containing the parameter. Adding Data Flow Operators To A Mapping Mapping Operators and Transformations 6-63 Figure 662 The Code Viewer Window Showing An Output Parameter Adding A Pre-Mapping Process The Pre-Mapping Process operator is a specific kind of operator. It calls a function or procedure (whose metadata is defined in OWB) prior to the start of map execution. Connections are restricted to constants or to output attributes of a Mapping Input Parameter operator. The output parameter group provides the connection point for the returned value (if implemented via a function) and the output parameters of the function or procedure. There are no restrictions on the connection of these output attributes When you drop a Pre-Mapping Process operator onto the Mapping Editor Canvas a dialog box opens in which a tree of all the available libraries, categories, functions and procedures. You chose a function or procedure from the tree, and the operator appears with all the correct input and output parameters predefined. Adding Data Flow Operators To A Mapping 6-64 Oracle Warehouse Builder 3i Users Guide The Pre-Mapping Process operator has attribute groups corresponding to the number and direction of the parameters associated with the PL/ SQL procedure or function selected. This list of groups and attributes are not modifiable through any other means than reconciliation. There can be at most one Pre-Mapping Process operator for a given map. Only constants, mapping input parameters, and output from a Pre-Mapping Process may be mapped into a Post Process operator. To add a pre-mapping process operator to a mapping: 1. Drag a Pre-Mapping Process operator from the Object Palette onto the Mapping Editor canvas. Figure 663 The Pre-Mapping Process Operator 2. Select the appropriate procedure from the selection list. Notes: s Only components that have a "dummy" source can be input to Triggers (both pre & post). This means, Constants, and Mapping Input Parameters can be input to both Pre & Post Triggers. s Mapping Input Parameters work like Constants, they can be mapped into anything that a constant could. s Mapping Output Parameters cannot be mapped TO anything, but can be mapped FROM either Constants, Mapping Input Parameters, or the Output of a Pre or Post Map Trigger (including the return value). s Violation of these rules is reported at generation time. Adding Data Flow Operators To A Mapping Mapping Operators and Transformations 6-65 Figure 664 The Pre-Mapping Process Selection List 3. Connect the output attribute of the Pre-Mapping Process operator to the INOUTGRP1 of the appropriate target operator. Adding Data Flow Operators To A Mapping 6-66 Oracle Warehouse Builder 3i Users Guide Figure 665 The Mapping Editor Showing A Pre-Mapping Process Operator 4. You may choose to rename the operator. s Right-click the operator header. s Select Rename from the popup menu s Enter a new name s Click OK. Adding Data Flow Operators To A Mapping Mapping Operators and Transformations 6-67 Figure 666 The Rename Component Dialog 5. Select Mapping > Generate > Mapping from the Mapping Editor main menu. OWB opens the Code Viewer window displaying the pre-mapping process. Figure 667 The Pre-Mapping Process Code Viewer Window Adding Data Flow Operators To A Mapping 6-68 Oracle Warehouse Builder 3i Users Guide Adding A Post-Mapping Process The Post-Mapping Process operator is a specific kind of operator. It calls a function or procedure (whose metadata is defined in OWB) after the end of a map execution. Connections are restricted to constants or to output attributes of a Mapping Output Parameter operator. The output parameter group provides the connection point for the returned value (if implemented via a function) and the output parameters of the function or procedure. There are no restrictions on the connection of these output attributes The Post-Mapping Process operator has attribute groups corresponding to the number and direction of the parameters associated with the PL/ SQL procedure or function selected. This list of groups and attributes are not modifiable through any other means than reconciliation. There can be at most one Post-Mapping Process operator for a given map. Only constants, mapping input parameters, and output from a Post-Mapping Process may be mapped into a Post-Mapping Process operator. The Post-Mapping Process operator is not valid for an SQL*Loader mapping. To add a post-mapping process operator to a mapping: 1. Drag a Post-Mapping Process operator from the Object Palette onto the Mapping Editor canvas. Notes: s Only components that have a "dummy" source can be input to Triggers (both pre & post). This means, Constants, and Mapping Input Parameters can be input to both Pre & Post Triggers. s Mapping Input Parameters work like Constants, they can be mapped into anything that a constant could. s Mapping Output Parameters cannot be mapped TO anything, but can be mapped FROM either Constants, Mapping Input Parameters, or the Output of a Pre or Post Map Trigger (including the return value). s Violation of these rules is reported at generation time. Adding Data Flow Operators To A Mapping Mapping Operators and Transformations 6-69 Figure 668 The Post-Mapping Process Operator 2. Select the appropriate procedure from the selection list. Figure 669 The Add Transformation Window Used For Selecting a Post-Process Mapping Operator 3. Connect the output attribute of the Pre-Mapping Process operator to the INOUTGRP1 of the appropriate target operator. Adding Data Flow Operators To A Mapping 6-70 Oracle Warehouse Builder 3i Users Guide Figure 670 The Mapping Editor Showing A Pre-Mapping Process Operator 4. You may choose to rename the operator. s Right-click the operator header. s Select Rename from the popup menu s Enter a new name s Click OK. Figure 671 The Rename Component Dialog Taking The Next Step Mapping Operators and Transformations 6-71 5. Select Mapping > Generate > Mapping from the Mapping Editor main menu. OWB opens the Code Viewer window displaying the post-mapping process. Taking The Next Step This chapter described how to add flow operators to a mapping and transform data as it moves from source to target. It also described how you can use standard transformations selected from the Oracle Library, custom transformations for a specific need, pre-packaged SQL operations, or imported PL/ SQL procedures. This chapter marks the end of the definition phase and the beginning of the configuration for generation phase for building a warehouse. The next chapter describes how to reconcile a mapping, validate the mapping, configure the physical properties for a mapping, and define code generation strategies. Taking The Next Step 6-72 Oracle Warehouse Builder 3i Users Guide Mapping Configuration and Code Generation 7-1 7 Mapping Configuration and Code Generation The previous two chapters described how to define a mapping, how to add operators, transformations, and external processes, how to edit a mapping, and how a mapping generates the code necessary to load your data warehouse. This chapter describes the tasks necessary to configure the physical properties of a mapping, and how to manage code generation. It also provides an in depth discussion of reconciling mapping operators with repository objects, and mapping validation. This chapter contains the following sections: s Reconciling Mapping Operators With Repository Objects s Configuring The Mapping For Deployment s Defining Code Generation Strategies For The Mapping s Setting A SQL*Loader Step s Setting An External Process Step s Taking The Next Step Reconciling Mapping Operators With Repository Objects 7-2 Oracle Warehouse Builder 3i Users Guide Reconciling Mapping Operators With Repository Objects Once you have selected the source and target objects, configured their properties, and linked the operators, you may need to reconcile the mapping objects with their corresponding repository objects. After reconciliation, you are ready to configure the physical properties of your mapping. Reconciling Mappings Mapping operators are independent from the repository objects to which they refer. The process of linking a mapping operator to a repository object is known as binding. Binding a mapping operator links it to a repository object both by unique object identifier and by physical name at the object and attribute levels. If you look at the Bound Name property of the mapping operator or the Bound Name property of any of its attributes you will see that these are the physical names of the corresponding repository object and its attributes. Additionally you can identify the repository object to which a mapping operator is bound by looking at the tool tip for the mapping operator; the part in parentheses gives the qualified physical name of the repository object. The logical names used in the mapping for both mapping operators and attributes do not need to be the same as the bound names. If you set your Project Preferences to Logical name mode without propagation to physical names or if you define your mapping operator and attribute names in the Mapping Editor then the logical names will generally be different. See Within the Mapping Editor there are several methods of binding a mapping operator to a repository object. Some of these methods relate to the original creation of the mapping operator and some relate to reconciliationinbound or outboundafter either the mapping operator or the repository object has been changed. You reconcile mapping operator attributes with repository attributes using the Inbound Reconciliation dialog. The Inbound Reconcile Operator Window Reconciling Mapping Operators With Repository Objects Mapping Configuration and Code Generation 7-3 Using this mechanism, you can: s capture structural changes applied to the associated object in the repository and propagate these to the mapping object. s associate the mapping object to a repository object of the same type s associate the mapping object to a repository object of a different type. You can do all this while still preserving the connections with other mapping objects. For example, a mapping table operator can be reconciled with new and removed columns for a table in the repository from which it was derived. More About Naming Mapping Objects Generally speaking, all interaction with objects inside the mapping environment use the logical name. The physical name, which is used for code generation, is available as the (logical) property Bound Name in those cases where the user manages the name. For objects brought in to the mapping environment, the reconciliation process will maintain the sanctity of the original physical names and attempt to do so with the logical name. Objects created or renamed manually inside the mapping will present the user with any validation issues and not attempt any Reconciling Mapping Operators With Repository Objects 7-4 Oracle Warehouse Builder 3i Users Guide sort of automatic name fixing. Physical names (Bound Name ) edited inside the mapping will be in upper case. Logical names are used as the displayed identifier of all components, groups, attributes, and display sets in the mapping environment. Logical names are always case sensitive. Multiple mapping objects with the same logical name may be placed on the map as long as their internal constraints allow it. Mapping objects that can be reconciled must have unique logical names. In the event that a new reconcilable mapping object would end up with the same name as an existing mapping object, a new unique name will be created. For example, if creating a mapping table Employee is attempted from two different installed modules, the automatic correction algorithm described later will be used to create an alternate logical name for the second table. Group, attribute, and display set names must be unique within the containing element. For manual processes (create/ rename) the user will be required to enter a correct name. For batch processes (reconciliation/ system creation/ copy and map), this may mean that the automatic correction algorithm described in the detail section will be used to create an alternate logical name. Physical names (or Bound Names) are used by the generation framework to produce the mapping code. Physical names will be case preserved during the reconciliation process, but will be in upper case if created or modified in the mapping environment.This is specifically consistent with the behavior if the integrator framework, i.e., import preserves case, edit does not. Multiple mapping objects with the same physical name may be placed on the map as long as their internal constraints allow it. [See the individual operator descriptions later in this chapter.] Group, attribute, and display set names must be unique within the containing element. For "copy and map", the automatic algorithm described later will be used to create a compliant physical name. For all other cases, this may mean that namespace validation will fail and the user must take corrective action. The name correction algorithm (when enabled) is: 1. Truncate the name to ensure the total length of the proposed name and any suffix does not exceed the maximum allowed length. 2. (Physical) Replace illegal characters by underscore. 3. (Physical) Trim leading & trailing spaces. 4. (Physical) Prefix proposed name with A if 1st character is illegal. 5. Add_n to the proposed name, where n is the first number in ascending order that would make the resulting name unique in its domain. 6. Repeat the first steps until both criteria are met. Reconciling Mapping Operators With Repository Objects Mapping Configuration and Code Generation 7-5 For Transformation Functions, a special name correction is always active: s (Physical) Prefix reserved word names with X_ Using Inbound Reconciliation You can reconcile an inbound operator by name, position, or object identifier match. You can also use a combination of the above three methods. The most common use of inbound reconciliation is to update a mapping operator to align with changes in its referenced repository object. The other less common options for inbound reconcile are discussed later. To start reconciliation, select the mapping operator on the canvas and: s Select Edit > Reconcile menu item from the menu bar or s Right mouse-click the header of the mapping operator and select the Reconcile menu item from the popup menu. OWB will then launch the Reconcile dialog box. The Reconcile dialog box enables you to select the object based on the references internally maintained that points to the corresponding repository column/ field from where the mapping attribute derives its definition. Note that operators of a certain origin (table, file) can be reconciled with a different repository object type (view, etc.) Inbound Reconciliation Strategies Reconciliation strategies are instructions to the inbound (and outbound) reconciliation process on how to match the mapping operator with the repository object to achieve the desired binding. There are three matching options that can be used individually or in conjunction by clicking one or more checkboxes. At least one checkbox must be checked to perform reconciliation. Matching By Object Identifier This strategy uses the unique object identifiers to determine the correlation between the mapping operator attributes and the columns / fields / parameters of the selected repository object. On inbound reconciliation, attributes of the mapping operator that could not be matched with those of the repository object are removed. This may happen if an attribute was added to the mapping operator and not reconciled or a column was removed from the repository object before reconciliation. When an attribute is removed, any incoming or outgoing mapping line connected to that attribute will be removed from the canvas. Attributes of the selected repository object that could not be matched with those of the mapping Reconciling Mapping Operators With Repository Objects 7-6 Oracle Warehouse Builder 3i Users Guide operator are added as new attributes at the end of the mapping operator. Mapping lines for matched attributes are preserved. Use Match by object identifier if you want to keep your mapping operators in step with changes to the bound repository object and you want to maintain separate logical names for your mapping operator attributes regardless of changes to physical names in the repository object. Match by object identifier is not available if you want to reconcile to a different repository object and this check box is grayed out when you select a different object in the tree. In this case the following two strategies are available. Figure 71 Inbound Reconcile Window Using Match By Object Identifier Matching By Physical (Bound) Name This strategy uses matching between the bound names of the mapping operator attributes and the physical names of the columns / fields / parameters of the repository object to perform the reconciliation. Matching is case sensitive. On inbound reconciliation, attributes of the mapping operator that could not be matched with those of the repository object are removed. (When an attribute is removed, any incoming or outgoing mapping line connected to that attribute will Reconciling Mapping Operators With Repository Objects Mapping Configuration and Code Generation 7-7 be removed from the canvas.) Attributes of the selected repository object that could not be matched with those of the mapping operator are added as new attributes to the mapping operator. Mapping lines for matched attributes are preserved. Bound names are read-only after you have bound a mapping operator to a repository object so it is not possible to manipulate these to achieve a different match result in inbound reconciliation. Use Match by physical name as a strategy for reconciliation with the current bound repository object if you want to maintain equivalence of physical names and logical names in your mapping operators. This aids clarity when comparing mappings with generated code but has the limitations associated with physical names. Note that use of this reconciliation strategy will treat the rename of a column in the repository object as if the column was deleted and a new column inserted. Therefore mapping lines for renamed attributes will be removed. Use Match by physical name as a strategy for inbound reconciliation with a different repository object: s if there have been any deletions or insertions of columns / fields / parameters in the repository object which would change the structure of the mapping operator. or s if you want to maintain physical names in your mapping operators and you use the same names consistently across objects Reconciling Mapping Operators With Repository Objects 7-8 Oracle Warehouse Builder 3i Users Guide Figure 72 Inbound Reconcile Using Match By Name Matching By Position This strategy matches mapping operator attributes with columns / fields / parameters of the selected repository object by position; that is the first attribute of the mapping operator is reconciled with the first attribute of the repository object, the second with the second, and so on. If the mapping operator has more attributes than the repository object then the excess attributes are removed from the mapping operator (When an attribute is removed, any incoming or outgoing mapping line connected to that attribute will be removed from the canvas ). If the selected repository object has more attributes than the mapping operator then these are added as new attributes at the end of the mapping operator. Mapping lines for existing attributes in the mapping operator are preserved. As a strategy for inbound reconciliation with the current bound repository object Match by position behaves in a similar way to Match by object identifier under normal circumstances. Only if the repository object has undergone a lot of changes involving deletion and recreation of existing attributes will there be a discernible difference. Your choice will depend on which is more important to you: s preserving the mapping operator s relationship with other mapping operatorsuse Match by object identifier, or Reconciling Mapping Operators With Repository Objects Mapping Configuration and Code Generation 7-9 s keeping the mapping operator structure in line with the repository object structureuse Match by position. Use Match by position as a strategy for reconciliation with a different repository object if you want to preserve the logical names of your mapping operator attributes. This strategy is most effective when the only changes to the repository object are the addition of extra columns / fields / parameters at the end of the object. Otherwise you may need to perform some editing of the mapping to re-connect mapping lines to the relevant attributes. Figure 73 Inbound Reconcile Using Match By position When none of the "match by" checkboxes are checked, the reconcile process will replace the attributes of the operator with an exact copy of the column/ fields of the selected repository object. When you select multiple checkboxes, the following priority order in types of matching will be used 1. Match by object identifier 2. Match by position, Reconciling Mapping Operators With Repository Objects 7-10 Oracle Warehouse Builder 3i Users Guide 3. Match by name. Reconciliation is supported for the following operator types: In addition to reconciling the attributes of an Operation, the reconciliation process may also update additional logical properties of an Operator; for example for a Flat File operator, information about the character set, filename will be refreshed as well. Using Outbound Reconciliation There are two main uses of outbound reconciliation. The first is to create a new repository object with the same structure and properties as a mapping operator created in the Mapping Editor. For example, used in conjunction with the copy and map feature this is a very quick way of creating repository definitions of staging tables. The second use of outbound reconciliation is the inverse of that of inbound reconciliation; that is to update an existing repository object in line with changes in the mapping operator. You can create new mapping objects or update existing objects in the repository and propagate the structure from mapping operators to the repository objects by using the Outbound Reconciliation dialog. Using this dial you can create/ update object in Table 71 The Operator Objects Reconciled With Repository Objects Operator Object Repository Object Mapping Tables Table, Views, Materialized Views, Sequences, Files, Dimensions and Facts. Mapping Views Table, Views, Materialized Views, Sequences, Files, Dimensions and Facts. Mapping Materialized Views Table, Views, Materialized Views, Sequences, Files, Dimensions and Facts. Mapping Sequences Table, Views, Materialized Views, Sequences, Files, Dimensions and Facts. Mapping Flat Files Table, Views, Materialized Views, Sequences, Files, Dimensions and Facts. Mapping Dimensions Table, Views, Materialized Views, Sequences, Files, Dimensions and Facts. Mapping Facts Table, Views, Materialized Views, Sequences, Files, Dimensions and Facts. Mapping Transformations Transformations only (not for OWB 3i). Reconciling Mapping Operators With Repository Objects Mapping Configuration and Code Generation 7-11 the repository whose structure (columns, data types, etc.) are derived from the mapping operator. s Automatically create a new object in the repository that matches the structure of the Mapping Operator. s Propagate changes in the attributes of a Mapping Operator to the repository object from which it was originally derived. s Copy and map the attributes of one Operator with a second Operator and propagate these changes to the repository object associated with the second Operator. s Perform a create-like function to create a new Operator in the Mapping Editor and a new associated object in the repository. To start Outbound Reconcile, you select an eligible operator on the canvas and s Select Edit > Reconcile Outbound menu item from the menu bar or s Right mouse click on the header of the Operator and select Reconcile Outbound menu item from the popup menu. OWB will then launch the Outbound Reconcile dialog box. Reconciling Mapping Operators With Repository Objects 7-12 Oracle Warehouse Builder 3i Users Guide Figure 74 The Outbound Reconciliation dialog The following Mapping Operators are eligible for Outbound Reconcile: Table 72 The Outbound Reconcile Operators Operator Type Notes Table Creates or updates an table object in the repository. Attributes and data-type information are copied as columns of the table. Constraint properties are not copied. View Creates or updates an view object in the repository. Attributes and data-type information are copied as columns of the view. Materialized View Creates or updates an materialized view object in the repository. Attributes and data-type information are copied as columns of the materialized view. Constraint properties are not copied. Transformation Creates or updates a function object in the repository Input attributes and data-type information are copied as input parameters of the function. Output attribute and data-type information is copied as return specification for the function. Configuring The Mapping For Deployment Mapping Configuration and Code Generation 7-13 You can use the outbound reconcile feature to create new objects or update existing objects in the repository that are derived from a mapping operator. After you have created a mapping operator, you can copy and map its attributes into a new operator. You can then create a new corresponding repository object that inherits the same properties. Select an operator on the mapping canvas and choose Edit > Reconcile Outbound and 1. Define a mapping flat file operator and its attributes on the Mapping Editor Canvas. 2. Create a new unbound mapping table operator with no attributes. 3. Copy and map all the attributes of the mapping flat file Operator to the Mapping Table. You can do this by s left-clicking and dragging the mapping lines across to the new operator, or s right-clicking the Operator Group and selecting Link INOUTGRP1 to > <Operator name> > INOUTGRP1. To create a new table in the repository associated with this Mapping Table, 1. select the Mapping Table and choose Edit > Reconcile Outbound. 2. Select Create a new table and its location in the repository in the Outbound Reconcile dialog 3. Click OK. A replica table with identical properties and attributes as the Mapping Table is created in the repository. Configuring The Mapping For Deployment In order for the code to deploy the mapping you need to configure the mappings Physical Properties. These properties include configuring each operator s physical properties as well as the steps necessary for code deployment. The following sections describe those settings in the Mapper that must be set to complete the definition phase of building a warehouse. Configuring Attribute Properties Before you configure the physical properties for a mapping, you may need to configure how your source attributes behave as they load into the mapping targets. Configuring The Mapping For Deployment 7-14 Oracle Warehouse Builder 3i Users Guide The following mapping objects contain attribute properties that may require changing prior to loading: s Tables s Facts s Dimensions s Views s Materialized Views The following settings are useful whenever a surrogate key (for instance, a Sequence operator) is used on a target in a map. Delete: Use For Matching This property sets OWB to delete a data targets row only if there is a match between the source attribute a mapped target attribute. If a match is found, then a delete occurs on the row. By Setting this property to Yes (default), the attribute will be used as a matching attribute. Note: The Use Constraints will override this setting. The See Insert: Use For Loading This property prevents the select attributes data from moving to a target even though it is mapped to do so. By selecting Yes (default), the data will reach the mapped target. Update: Operation When a matching row is located and an update operation is performed on the target, different computations can be done between the data of the source attribute and the target attribute on the matched row before the resulting value is stored onto the target. You can specify one of the following target conditions: = target:= source += target:= source + target -= target:= target - source =- target:= source - target ||= target:= target||source Configuring The Mapping For Deployment Mapping Configuration and Code Generation 7-15 Update: Use For Matching This property sets OWB to update a data targets row only if there is a match between the source attribute a mapped target attribute. If a match is found, then an update occurs on the row. By Setting this property to Yes (default), the attribute will be used as a matching attribute. If you use this setting, then all the key columns must be mapped if there is only one unique key defined on the target entity. Note: The Use Constraints will override this setting. Update: Use For Loading This property prevents the selected attributes data from moving to a target even though it is mapped to do so. By selecting Yes (default), the data will reach the mapped target attribute. After youve completed these settings, you are ready to set the physical properties of the mapping. Configuring The Mapping Operator Physical Properties The Operator Properties are located in the Property Configuration inspector. You open the Property Configuration inspector by selecting an mapping object in the module tree and s Select Module > Configure from the menu bar s Right-click the Mapping on the Warehouse Module Editor and select Configure from the popup menu. =|| target:= source||target Note: The rules of UPDATE are the following: If all columns of a UK are not mapped - UK is not used in construction of match condition. If not columns of UK are mapped - OWB return an error If column (not key column) is not mapped - it would not be used in loading Configuring The Mapping For Deployment 7-16 Oracle Warehouse Builder 3i Users Guide Figure 75 The Configure... Popup Menu Item for Mappings OWB returns the Configuration Properties Inspector. From here, you select the property that you want to set. Figure 76 The Configuration Properties Inspector Configuring The Mapping For Deployment Mapping Configuration and Code Generation 7-17 The Configuration Properties inspector groups the physical properties of a mapping in different groups: Setting The Database Link or Schema Information You can configure your mapping for Schema and Remote Schema Access. You can also link the mapping to a particular schema by clicking on the field to the right of Schema and entering a name. When you specify a schema by name (i.e. Schema property value is not null), the PL/ SQL and SQL generation handlers for the Extract Operators, will consider the property value when referring to the object name in the generated From Clause. Remote Schema Access in OWB mappings is defined as accessing objects from a particular schema in a remote database using Net8 database links. Database Links define the physical connection information for remote access (hostname, ports, etc), that subsequently can be referenced by the Remote Access property of an Extract Operator. The mapping configuration enables you to link the mapping to a database link in the warehouse module. This is done by module nameyou actually create the database link itself when you configure the module for deployment. See "Configuring Connection Information for Database Sources" on page 4-2 for information on setting database links for modules. To define Database Links for a mapping: 1. From the Configure Properties inspector, click Operators > <Mapping Operator> > Access Specification. 2. Click the right-hand field for Database Link. 3. Select a Module from the pulldown list. Configuring The Mapping For Deployment 7-18 Oracle Warehouse Builder 3i Users Guide Figure 77 The Configure Properties Inspector Showing The Database Link List The Mapping is now associated to the database link for the module you selected. Setting Partition Exchange Loading Depending upon your situation, Partition Exchange Loading can increase the speed at which data loads into tables. Partition Exchange Loading leverages Oracle server s partitioning capability to increase OWBs loading performance. This is especially useful in cases such as s The target table is partitioned. s The target table holds a large amount of historical data s Incremental insert into only one empty partition s The target is partitioned by a DATE column The target can be table, dimension, or fact. In order to make PEL useful, the target table must be partitioned and it holds a large amount of historical data. The PEL technique is best suited for situations where relatively small amounts of data need to be loaded into a target that has already cumulated much larger amount of historical data Also, PEL is useful whenever all the new data must be loaded into the same partition in a target table. If the data Configuring The Mapping For Deployment Mapping Configuration and Code Generation 7-19 warehouse collection is scheduled on a regular basis, this condition can normally be satisfied. One reason not to use PEL is if the insertion is accompanied by simultaneous index and constraint maintenance. Therefore, using the PEL method to load data into a non-empty partition will go slower than if the target partition is empty. The PEL technique performs loading of new data by exchanging them into a target table as a partition. What gets exchanged are merely identities. The table that holds the new data takes over the identity of one empty partition from the target table; at the same time, this empty partition assumes the identity of the source table. The whole exchange process is a pure DDL operation, and involves no data movement. PEL configuration requires two phases: configuring the mapping itself and then the target (table, dimension, or fact). By default, the PEL method is disabled on all mappings. The following instructions describe how to configure a mapping and then a target. To configure the mapping for Partition Exchange Loading: 1. From the Configure Properties inspector, click Operators > <Mapping Operator> > Partition Exchange Loading. Configuring The Mapping For Deployment 7-20 Oracle Warehouse Builder 3i Users Guide Figure 78 The Mapping Properties Inspector Showing Partition Exchange Loading 2. Click the right-hand field for Use PEL. 3. Select True. Once the PEL is enabled, the OWB code generator will generate different kind of batch processing code. 4. Click the right-hand field for Partition Granularity. This property tells OWB PL/ SQL code generator how the target table is partitioned. There are seven levels of partition granularity you can choose from. 5. Select the granularity level appropriate for the mapping operator. You configure the target once the mapping is configured. In configuring the target, the following three sub-areas related to the target need be configured: s The target must be correctly partitionedall of the partitions must be created before the PEL method can be used.. Configuring The Mapping For Deployment Mapping Configuration and Code Generation 7-21 s All indexes on the target must be local indexes. s All primary and unique constraints on the target must be created by the USING INDEX option. To configure the target: 1. Create All Partitions. s To create partitions for a table, dimension, or fact, use its property inspector. See s All partition names must follow a strict naming convention. A strict partition naming convention enables the PEL to automatically construct the correct partition name based on freshly loaded data. The partition name must contain information of all the time levels leading to the intended granularity. s After all the partitions are added with correct names, the partitions must be further configured for their VALUE LESS THAN properties. Using the name and the VALUE LESS THAN property, OWB will generate a DDL script for creating the partitioned table. In essence, the VALUE LESS THAN property defines what the partition will contain; and the partition name defines how the partition will look like. Figuring out correct value for the VALUE LESS THAN property may prove equally error-prone as figuring out the correct partition name. 2. Create All Indexes Using the LOCAL option. s Using only local indexes is one of the most important reasons for the PEL s performance benefit. Local indexes require that all indexes be partitioned in the same way as the table. When the temporary table is swapped into the target table using the PEL, so are the identities of index segments. s There is no need to manipulate actual index blocks. The Index Type parameter is configured to UNIQUE and the Local Index parameter is configured to true. s Finally, if an index is created as a local index, the partition key column must be the leading column of the index, or the index cannot be created. 3. Set Primary/ Unique Keys Use to USING INDEX option. This step specifies that all primary and unique key constraints are created with the USING INDEX option. By using the USING INDEX option, a constraint will not trigger automatic index creation when it is being added to the table. Instead, the server will look among existing indexes for one with same column Configuring The Mapping For Deployment 7-22 Oracle Warehouse Builder 3i Users Guide list as that of the constraint. This implies that each primary or unique key constraint must be backed by a user-defined unique local index. The index that supports this option was created in step 2. Adding Optimizing Hints Application developers often develop insights into their data. For example, they may know that a query runs much faster if a set of tables is joined in one order rather than another. OWB can incorporate these insights into the generated SQL code packages as SQL Optimizer Hints. To define hints for a mapping: 1. From the Configuration Properties inspector, click Operators > Hints. 2. Click the right-hand field in either an Extraction Hint or a Loading Hint. 3. Click the ... button. OWB opens the Extraction or the Loading Hint window. Configuring The Mapping For Deployment Mapping Configuration and Code Generation 7-23 Figure 79 The Extraction Hint Window 4. Select a hint from the pulldown list. 5. Click Add. The hint appears in the Existing Hints field. Enter additional text as appropriate for your situation into the Extra Text column. 6. Click OK. The editor includes the hint in the mapping definition. For detailed information on optimizer hints and how to use them, see the Oracle8i Designing and Tuning for Performance guide. Defining Code Generation Strategies For The Mapping 7-24 Oracle Warehouse Builder 3i Users Guide Setting Constraint Handling Setting this parameter to false will disable all referential constraints to and from the target table. This speeds loading by omitting constraint checking by the database. After all the data is loaded, the constraints will be automatically enabled by OWB. OWB will not guarantee referential integrity when using this option. If the data does not comply with the constraints, the constraint will become invalid. In such cases, you will have to manually resolve the datas referential integrity. To set constraint handling: 1. From the Configure Properties inspector, click Operators > <Mapping Operator> > Constraint Management. 2. Click the right-hand field for Enable Constraints. 3. Select True or False. 4. Click the right-hand field for Exceptions Table. 5. Enter the appropriate name into the field. Defining Code Generation Strategies For The Mapping OWB implements a mapping by generating a PL/ SQL package based upon a mapping. A code generation strategy is the mechanism in a mapping which determines the implementation language for each operator after considering the limitations of the implementation languages. For example, a Filter Operator can be generated either in SQL as a WHERE clause in the SQL statement, or in PL/ SQL as an IF statement inside a Fetch loop. The following sections describe the Code Generator and several different code generation strategies for mapping. Viewing The Generated Code for a Mapping The code generator can be invoked from both the Module Editor and the Mapping Editor. The major distinction between the two is that the code generated from within the Mapping Editor is not persisted in the repository and excludes auditing code and code that may be necessary for deployment.Code can be generated for a mapping as for any child-object of a warehouse module.. Note: If you generate code at any time during your Warehouse Builder Session, Multi-User Locking feature will remain in effect until after you commit your changes. Defining Code Generation Strategies For The Mapping Mapping Configuration and Code Generation 7-25 There are two kinds of code-generation in the Mapping Editor: s Mapping You can generate simplified code from the mapping editor that demonstrate logically how data extraction, transformation and load are performed. The generated code will not include support for auditing or bulk-processing. Code generation for a mapping selected in the Mapping editor can be opened as follows s Select the Mapping > Generate > Mapping menu item from the menu bar or s Right mouse click and select the Generate > Mapping menu item from the popup menu s Intermediate Results This type of code generation is intended to give the resultant mapping code up to a certain attribute group in the mapping. There are three different aspects of viewing the intermediate code result for an attribute group, namely Incoming, Outgoing and Loading code.Intermediate Results can be launched from the mapping editor by selecting an Attribute Group for an operator and s Clicking Mapping > Generate > Intermediate Results from the menu bar or s Right mouse-click on the canvas and selecting Generate > Intermediate Results from the popup menu. Once started, code will generate for all upstream operators up to the current selected Attribute Group. Once the code is generated, the Code Viewer launches so that you can inspect the code that comes into an input attribute group, what code goes out of an output attribute group and what load code is generated for a terminating input group. The code generator will only generate code to handle the specified loading type as specified in the data target operator. For example, if you choose INSERT/ UPDATE as the loading type, only INSERT and UPDATE code will appear in the generated code. Note: Note, where applicable the system may generate code for multiple strategies. For example, if a step can be implemented in set-based and row-based operating mode, the code viewer will include both modes. Defining Code Generation Strategies For The Mapping 7-26 Oracle Warehouse Builder 3i Users Guide Using the Code Viewer You can view the generated code for a mapping from the Mapping Editor. Note that code also can be generated from within the Module Editor. The major distinction between the two is that the code generated from within the Mapping Editor is not persisted in the repository and excludes code that may be necessary to deploy the code. Viewing Intermediate Results There are three different aspects of viewing the intermediate code result, namely s Incoming s Outgoing s Loading You can also choose among any of the aspects when viewing the code. At any particular moment, not all the aspects are possible. For example, an output attribute group does not have the Input intermediate code result. To select an aspect, s Click View > Aspect from the menu bar of the Code Viewer. Figure 710 The Intermediate Results Viewer The Code Viewer for intermediate code-generation is non-modalyou can return to the mapping canvas and select any attribute group to see their intermediate code. The code viewer will refresh with the code text generated for the selected attribute group. Defining Code Generation Strategies For The Mapping Mapping Configuration and Code Generation 7-27 Viewing Code From The Module Editor Code can be generated for a mapping as for any child-object of a warehouse module. Code generation for a mapping selected in the Module editor can be initiated as follows: s Select the module and then Module > Generate from the Module Editor menu or s Right-click on the module and select Generatefrom the popup menu. Select Generate Create Scripts from the Generation Mode dialog box. The generated code will contain all possible strategies that can be generated for that particular mapping. For PL/ SQL mappings, a runtime parameter will determine which operating mode to use when running the package. . Configuring Loading Step Strategies A mapping step is the result of the implicit mapping graph analysis performed on a mapping. Mapping steps are automatically created and maintained; you cannot add, delete or rename them. A step can be one of the following three types: PL/ SQL, SQL*Loader, External Process. Each type will have its related physical configuration parameters which the user can configure via the Configuration Property inspector. Code can be generated for mapping steps in its associated language and physical configuration parameters are used in the generation process. The Step properties encompass the runtime parameters. Note: If a view is part of mapping, OWB does not generate a DDL statement for view. OWB generates regular package with assumption that view is updatable. No error checking is done to verify if view is updatable or not. You must ensure that view complies with the definition of updatable view before including this view into the map (see 8i SQL Reference for the definition of updatable view). OWB Validation or Generation does not have any special treatment for view. View or Materialized view are treated the same way as regular tables Defining Code Generation Strategies For The Mapping 7-28 Oracle Warehouse Builder 3i Users Guide Setting The Default Audit Level This is the audit level that will be used when executing the package. Audit levels dictate the amount of audit information that will be captured in the runtime schema when the package is run. The audit level settings are described in the following table.. Setting the Default Purge Group This is the purge group used when executing the package. Each audit record in the runtime schema will be assigned to the purge group specified. Setting Maximum Number of Errors This is the number for maximum errors reached that will be used when generating/ executing the package. When executing this package, the execution will terminate when the number of errors reached is greater than the maximum number of errors value. Setting Commit Frequency This is the commit frequency that will be used when executing the package. Data will be committed to the database after processing the number of rows specified in this parameter. Setting The Default Operating Mode The Default Operating Mode setting involves specifying how code will be generated from the mapping. The next section discusses how to set the default code generation mode. Table 73 The Default Operation Levels Setting Description None No auditing information will be recorded in runtime. Statistics Statistical auditing information will be recorded in runtime. Error Details Statistical plus error information will be recorded in runtime. Complete All auditing information will be recorded in runtime. Defining Code Generation Strategies For The Mapping Mapping Configuration and Code Generation 7-29 You use this setting for the operating mode used for running the generated code package. Set Based The Set Based option enables OWB to insert all of the data in a single SQL command. This option depends upon a correctly designed mapping. This strategy starts from the beginning of the mapping and assigns SQL as the implementation language to each operator unless an operator does not support SQL as the implementation language. All the operators from that point forward will have PL/ SQL as the implementation language. This strategy is very similar to Batch but can handle procedures in mapping as well. Row Based This strategy starts from the beginning of the mapping and assigns SQL as the implementation language to each operator up till a point that all the operators from that point forward supports PL/ SQL as the implementation language. You use this strategy to give maximal amount of auditing or debugging information to the user since it is likely that the expressions or transforms in the mapping will be implemented in PL/ SQL. Row Based (Target Only) This strategy is similar to Default except the implementation language of the data target operators will always be PL/ SQL. When this strategy is used with fail-over, it allows the error rows to be stored into the audit table when data are being loaded into the target. This is mainly used when the source data is clean but there may be Table 74 The Default Operating Modes Setting Description Set based Execute set based (batch) processing - uses SQL as the implementation language. Row based Execute row based processing - uses PL/ SQL as the implementation language. Row based (target only) Execute row based processing - uses PL/ SQL as the implementation language for the data target. Set based fail over to row based Execute set based (batch) processing, but switch to row based if error occurs. Set based fail over to row based (target only) Execute set based (batch) processing, but switch to row based (target only) if error occurs Setting A SQL*Loader Step 7-30 Oracle Warehouse Builder 3i Users Guide errors when loading into the targets. Examples are foreign key constraint violation when loading fact with missing primary key at the dimension, or incompatible implicit data type conversion due to the limited length, scale or precision at the data targets. Set Based Fail Over To Row Based The mapping will first be executed in set based mode. If there is any error occurred, the execution will fail over to row based. If this operating mode is not supported (because it cannot be implemented by the generator), a runtime exception will occur if you attempt to run the mapping in this mode. Set Based Fail Over To Row Based (Target Only) The mapping will first be executed in set based mode. If any error occurred, the execution will fail over to row based (target only). If such operating mode is not supported (because it cannot be implemented by the generator), a runtime exception will occur if you attempt to run the mapping in this mode. Setting Bulk Processing The Bulk Processing properties enable you to load incremental amounts of rows from a source to a target. Setting this to false means that PL/ SQL will process the data one row at a time. Setting it to TRUE means PL/ SQL will process the data grouped by the amount of rows specified by the Bulk Size property. Setting the Analyze Statistics Percentage This is the percentage of rows to estimate when gathering statistics on the target tables. After data is loaded into the target tables, statistics used for cost-based optimization are gathered on each target table. You can set this parameter to the percentage of rows in each target table that will be used for this analysis. Setting A SQL*Loader Step Utilizing Oracle Warehouse Builder s flexible configuration framework, a SQL*Loader mapping can be configured to suit your specific physical requirements. The values chosen during configuration will directly affect the content of the generated SQL*Loader and the runtime control files. SQL*Loader provides two methods for loading data: s Conventional Path Load Setting A SQL*Loader Step Mapping Configuration and Code Generation 7-31 s Direct Path Load A conventional path load executes SQL INSERT statement(s) to populate table(s) in an Oracle database. A direct path load eliminates much of the Oracle database overhead by formatting Oracle data blocks and writing the data blocks directly to the database files. A direct load, therefore, does not compete with other users for database resources so it can usually load data at near disk speed. Certain considerations, such as restrictions, security and backup implications, are inherent to each method of access to database files. Refer to the Oracle 8i Utilities Manual for further information. When designing and implementing a mapping to extract data from a Flat File via SQL*Loader, there are several places where you can choose values for properties to affect the SQL*Loader script generated. These places allow you the flexibility to ensure the generated SQL*Loader script matches your specific logical and physical requirements. First, from a logical perspective, each load operator in a map has an operator property called Loading Types. The value contained by this property affects how the SQL*Loader INTO TABLE clause for that specific load operator will be generated. Modification of this property is accomplished by opening the mapping diagram, selecting a specific load operator, and then choosing Operator Properties. Next, there are two locations where you have the opportunity to supply physical configuration information to affect the generated SQL*Loader output. One is the configuration of the physical properties for a load operator. The other is by setting the values in the physical step configuration. Each will be described below. Table 75 The Loading Types And INTO TABLE Relationship Loading Types INTO TABLE INSERT/ UPDATE APPEND DELETE/ INSERT REPLACE TRUNCATE/ INSERT TRUNCATE CHECK/ INSERT INSERT NONE INSERT Setting A SQL*Loader Step 7-32 Oracle Warehouse Builder 3i Users Guide Setting Load Operator Physical Parameters Each load operator has physical configuration parameters which you can utilize to cause generation of specific SQL*Loader output. For more specifics on the meaning and significance of each SQL*Loader option and clause, please refer to the Oracle 8i Utilities Manual. Each of these properties described below affect how the INTO TABLE clause is generated. Setting Enable Constraints If the Enable Constraints property is set to the value TRUE then the REENABLE DISABLED_CONSTRAINTS clause is generated. This option automatically re-enables integrity constraints at the end of a direct-path load. Setting The Exceptions Table If the Exceptions Table property is set to a non-blank value then the EXCEPTIONS table clause is generated. This option specifies a table that must exist when SQL*Loader is run. It will be used to insert the Reweds of all rows that have violated one of the integrity constraints when constraint re-enabling is processed during a direct path load. Setting SQL*Loader Parameters The SQL*Loader Parameters properties enable you to define the SQL*Loader options appropriate for your mapping. Refer to the Oracle 8i Utilities guide for more specific information on SQL*Loader. The following parameters are available in the configuration properties inspector: s PARTITION NAME indicates that the load is a partition-level load. Partition-level loading lets you load one or more specified partitions or subpartitions within a table. Full database, user, and transportable tablespace mode loading do not support partition-level loading; only table mode loading does. Because incremental loading (incremental, cumulative, and complete) can be done only in full database mode, partition-level loading cannot be specified for incremental loads. In all modes, partitioned data is loaded in a format such that partitions or subpartitions can be loaded selectively. s SORTED INDEXES identify the indexes on which the data is presorted. This clause is allowed only for direct path loads. Generally, you specify only one index in the SORTED INDEXES clause because data that is sorted for one index is not usually in the right order for another index. When the data is in the same order for multiple indexes, however, all indexes can be specified at once. All Setting A SQL*Loader Step Mapping Configuration and Code Generation 7-33 indexes listed in the SORTED INDEXES clause must be created before you start the direct path load. s SINGLEROW is intended for use during a direct path load with APPEND on systems with limited memory, or when loading a small number of records into a large table. This option inserts each index entry directly into the index, one record at a time. By default, SQL*Loader does not use SINGLEROW to append records to a table. Instead, index entries are put into a separate, temporary storage area and merged with the original index at the end of the load. This method achieves better performance and produces an optimal index, but it requires extra storage space. During the merge, the original index, the new index, and the space for new entries all simultaneously occupy storage space. With the SINGLEROW option, storage space is not required for new index entries or for a new index. The resulting index may not be as optimal as a freshly sorted one, but it takes less space to produce. It also takes more time because additional UNDO information is generated for each index insert. This option is suggested for use when either of the following situations exists: s Available storage is limited s The number of records to be loaded is small compared to the size of the table (a ratio of 1:20, or less, is recommended) s TRAILING NULLCOLS tells SQL*Loader to treat any relatively positioned columns that are not present in the record as null columns. s RECORDS TO SKIP invokes the SKIP command in SQL*Loader. SKIP specifies the number of logical records from the beginning of the file that should not be loaded. By default, no records are skipped. This parameter continues loads that have been interrupted for some reason. It is used for all conventional loads, for single-table direct loads, and for multiple-table direct loads when the same number of records were loaded into each table. It is not used for multiple-table direct loads when a different number of records were loaded into each table. s FILES specifies the names of the export files to import. The default extension is .dmp. Because Export supports multiple export files (see the following description of the FILESIZE parameter), you may need to specify multiple filenames to be imported. You need not be the user who exported the export files; however, you must have read access to the files. If you were not the exporter of the export files, you must also have the IMP_FULL_DATABASE role granted to you. Setting A SQL*Loader Step 7-34 Oracle Warehouse Builder 3i Users Guide Setting Step Parameters The parameters described in this section affect the specific output of the generated SQL*Loader scripts common heading section. This section includes the setting of type of loading to be performed, options which determine how the load is to be performed, and the specification of the physical files from which to load data. For more specifics on the meaning and significance of each SQL*Loader option and clause, please refer to the Oracle 8i Utilities Manual. The SQL*Loader Step has the following Configuration Parameters: Parameters Affecting Script Type s Operation RecoverableControls the output of the RECOVERABLE clause. True indicates that the load is recoverable. False indicates that the load is not recoverable and records are not recorded in the redo log. s Continue LoadIf this parameter is set to TRUE, then the generated SQL*Loader script will be CONTINUE_LOAD. Otherwise, the SQL*Loader script will be LOAD. CONTINUE_LOAD is used when a direct load of multiple tables was discontinued and needs to be restarted. It is used in conjunction with the operator-level SKIP option. s Nls CharactersetSpecifies the character set to be placed in the CHARACTERSET clause Parameters Affecting the OPTIONS clause: s Direct ModeSpecifies the value of the DIRECT option as either = TRUE or = FALSE. True indicates that a direct path load will be done. False indicates that a conventional load will be done. s Perform Parallel LoadSpecifies the value of the PARALLEL option as either = TRUE or = FALSE. True indicates that direct loads can operate in multiple concurrent sessions. s Errors AllowedIf the value specified is greater than 0, then the ERRORS = n option will be generated. SQL*Loader will terminate the load at the first consistent point after this error limit is reached. s Records To SkipIf the value specified is greater than 0, then the SKIP = n option will be generated. This value indicates the number of logical records from the beginning of the file that should not be loaded. If the value is not specified, no records will be skipped. Setting A SQL*Loader Step Mapping Configuration and Code Generation 7-35 s Records To LoadIf the value specified is greater than 0, then the LOAD = n option will be generated. This value specifies the maximum number of records to load. If a value is not specified all of the records will be loaded. s Rows Per CommitIf the value specified is greater than 0, then the ROWS = n option will be generated. For direct path loads, the value identifies the number of rows to read from the source before a data save is done. For conventional path loads, the value specifies the number of rows in the bind array. s Bind SizeIf the value specified is greater than 0, then the BINDSIZE = n option will be generated. The value indicates the maximum size in bytes of the bind array. s Read SizeIf the value specified is greater than 0, then the READSIZE = n option will be generated. The value is used to specify the size of the read buffer. s Read BuffersIf the value specified is greater than 0, then the READBUFFERS n clause will be generated. READBUFFERS specifies the number of buffers to use during a direct path load. Do not specify a value for READBUFFERS unless it becomes necessary, as indicated by ORA-2374. s FileIf this parameter is set to a non-blank value, then the FILE= option will be generated. s Preserve BlanksIf this parameter is set to the value of true, then the PRESERVE BLANKS clause will be generated. PRESERVE BLANKS retains leading whitespace when optional enclosure delimiters are not present. It also leaves trailing whitespace intact when fields are specified with a predetermined size. Parameters Affecting the INFILE clause(s): The Data Files configuration parameter allows you to specify the characteristics of the physical files to be loaded. Using this section you have the ability to specify the name of the file to loaded, the bad file name, discard file name, and the maximum number of discards allowed for each of the files to be loaded. Note: The value specified will be enclosed in single quotes in the generated code. FILE specifies the database file from which the temporary segments are allocated for the entire session. Setting A SQL*Loader Step 7-36 Oracle Warehouse Builder 3i Users Guide The initial values for these parameters will be set from the properties of the flat file used in the mapping. s Data File NameThe full physical filename of a data file to be loaded. The value entered will be placed in the INFILE clause. You should enter the complete path name for the file in the syntax of the operating system on which the SQL*Loader script will be deployed. Note: the value specified will be enclosed in single quotes in the generated code. s Bad File NameThe full physical filename of a bad file that SQL*Loader will create to store records that cause errors during insert or are improperly formatted. The value entered will be placed in the BADFILE clause. You should enter the complete path name for the file in the syntax of the operating system on which the SQL*Loader script will be deployed. Note: the value specified will be enclosed in single quotes in the generated code. s Discard File NameThe full physical filename of a discard that SQL*Loader will create to store records that are neither inserted into a table nor rejected. The value entered will be placed in the DISCARDFILE clause. You should enter the complete path name for the file in the syntax of the operating system on which the SQL*Loader script will be deployed. Note: the value specified will be enclosed in single quotes in the generated code. s Discard MaxThe value entered, if greater than 0, will be used to generate the DISCARD n clause. It specifies the number of discards that will terminate the load. Only the following operators support generation of SQL*Loader: Extract Operators: s Mapping File s Data Generator Transformation Operators: s Filter s Expression s Transformation Load Operators: s Mapping Table s Mapping View Taking The Next Step Mapping Configuration and Code Generation 7-37 s Mapping Dimension s Mapping Fact Strategy: Oracle Warehouse Builder will examine each operator in the mapping to determine if it supports SQL*Loader generation. Only if all of the operators support SQL*Loader generation will SQL*Loader be a valid generation Setting An External Process Step This section describes the setting of Step Parameters associated with an External Process Step. Setting OS Executable/Pure Integrate Setting the Executable LocationThis is the directory path and name of the executable. Setting Command Line ParametersThis is the string of parameters to be passed to the executable. Setting Pure Extract Setting the FTP DirectoryThis is the directory for performing FTP. Setting the Status File NameThis is the filename which contains the execution status of the Pure Extract process. Taking The Next Step This chapter described how to transform data as it moves from source to target. You can use standard transformations selected from the Oracle Library, custom transformations for a specific need, or pre-packaged SQL operations. This chapter marks the end of the definition phase and the beginning of the configuration for generation phase for building a warehouse. The next chapter describes how to reconcile a mapping, validate the mapping, configure the physical properties for a mapping, and define code generation strategies. Taking The Next Step 7-38 Oracle Warehouse Builder 3i Users Guide Configure, Generate and Deploy 8-1 8 Configure, Generate and Deploy The definitions stored in an OWB repository describe a logical model of a data warehouse and its mappings. This model can be physically realized in any number of ways depending on how these definitions are configured. So far, in Chapter 7, you have seen how the mappings are configured. This Chapter shows you how to configure the rest of the definitions for a specific instance, validate the definitions, and generate and deploy the scripts that create and populate the instance. Once you have populated a physical instance, you may later need to drop, reconfigure, rename, and upgrade the objects in this initial warehouse. This chapter also describes how OWB helps you make these changes. This chapter includes the following sections: s Configuring a Physical Instance s Validating Definitions s Generating Scripts s Deploying Scripts s Upgrading a Warehouse Physical Instance s Taking the Next Step Configuring a Physical Instance 8-2 Oracle Warehouse Builder 3i Users Guide Configuring a Physical Instance Before you can create the physical instance of your data warehouse you must first specify the physical characteristics. This is called configuration. Configuration entails defining configuration properties for each definable warehouse parameter. Table 81 summarizes the parameters you can configure for each object deployed to the physical warehouse. Table 81 Physical Characteristics In order to configure your data warehouse you must use the Configuration Properties inspector. The Properties inspectors are specific to the type of object that is being configured. Make sure you go through and configure each object you are planning to deploy. The Properties inspectors allow you to look at the current parameter settings and modify them according to your needs. Warehouse Object Parameters Warehouse Module Database link definitions, generation preferences, the job name to register with OEM, runtime and generation target directory names, and general warehouse module identification properties Dimension Indexes, partitions, partition keys, constraints, columns, performance, parallel, partition parameters, table space used to store, generation options for time dimensions, generation options, and identification Fact Indexes, partitions, partition keys, constraints, columns, performance, parallel, partition parameters, table space used to store, and identification Mapping These parameters are covered in Chapter 7. Materialized View Indexes, partitions, partition keys, performance, parallel, partition parameters, materialized view parameters, table space used to store, and identification Sequence Sequence Parameters and Identification Table Indexes, partitions, partition keys, constraints, columns, performance, parallel, partition parameters, table space used to store, and identification Transformation Categories Identification View Identification Configuring a Physical Instance Configure, Generate and Deploy 8-3 There are two kinds of configurations: s Configuration of existing logical objects s Configuration of new physical objects Configuring Existing Logical Objects To configure existing logical objects, you specify which objects are to be physically deployed and describe each objects physical characteristics. An objects physical characteristics are determined by a set of parameters stored in its Configuration Properties inspector. Most parameters have default values. To configure existing objects: 1. Double-click the name of the warehouse module. The Warehouse Module Editor displays with the Logical Tree on top. All of the objects listed in the Logical Tree have Configuration Properties inspectors. 2. Highlight an item in the tree and select Edit > Configure.... The Properties inspector displays with its categories of configuration parameters rolled up. The warehouse module Properties inspector is displayed in the example below. It has seven configuration parameter categories. 3. Expand each category to display their parameters. 4. Choose the parameter you want to configure and click in the space to the right of the parameter s name to be able to edit the parameter value. Configuring a Physical Instance 8-4 Oracle Warehouse Builder 3i Users Guide Depending on the parameter, you can either select an option from a drop-down list, or type in a value. Use the parameter descriptions starting on page 8-7 to help define your parameters. The descriptions are separated into objects and go in the order that they display in the Warehouse Module Editor. Configuring New Physical Objects Physical objects such as database links, indexes, partitions, and partition keys cannot be created when you are designing the logical warehouse. If you want these objects created in your database when you deploy, you must first create the objects and then configure them. These objects can only be created in relation to certain logical objects. Database links can only be created in the warehouse module Configuration Properties inspector, and indexes, partitions and partition keys can only be created in the dimension, fact, table, and materialized view Properties inspectors. To create and configure new objects: 1. Open the Warehouse Module Editor, select the object to which you want to add a new physical object and select Edit > Configure... . The Properties inspector displays with its categories of configuration parameters. The physical object names are listed at the top. The physical objects are listed on the top of the Configuration Properties inspector. Configuring a Physical Instance Configure, Generate and Deploy 8-5 2. Choose the object you want to create and click in the space to the right of the objects name. The ... button appears. This button allows you to add and delete objects. 3. Click the ... button. A pop-up panel for the object displays. The button appears. Configuring a Physical Instance 8-6 Oracle Warehouse Builder 3i Users Guide 4. Type in the name of the object you want to create and click Add. You can continue on adding new objects by typing in new names and clicking Add. When you are done click OK. 5. Expand the object type name to display the new object. 6. Expand the new object name and categories to display the new objects configuration parameters. 7. Choose the parameter you want to configure and click in the space to the right of the parameter s name to be able to edit the parameter value. Depending on the parameter, you can either select an option from a drop-down list, or type in a value. Use the parameter descriptions starting on page 8-7 to help define your parameters. The descriptions are separated into objects and go in the order that they display in the Warehouse Module Editor. Configuring a Physical Instance Configure, Generate and Deploy 8-7 Configuring Warehouse Modules To configure a Warehouse Module: 1. Open the Warehouse Module Editor, select the Warehouse Module and select Edit > Configure.... The Configuration Properties inspector displays with seven categories of configuration parameters. 2. Expand to display the parameters for each category. The following sections discuss each of the categories and their corresponding parameters. Use these as a reference. Database Links Target schemas pull data from database sources using PL/ SQL packages, and these packages rely on database links to connect with the sources. Thus, a definition for a database link must be created, configured, and deployed for each database source. The link is deployed in the target schema. Each time you create a mapping that pulls data from a database source, OWB verifies that a definition for the required database link exists. If the definition does not exist, OWB creates one from the connection information stored in mappings source module. Thus, the default parameter values for a database link correspond with the connection information in a source module. When you deploy a physical instance, you must verify that the database links generated by OWB point to the correct sources. For example, if a test application Configuring a Physical Instance 8-8 Oracle Warehouse Builder 3i Users Guide system was used to develop the logical warehouse but the physical instances use real data stored in production systems, then you must reconfigure the definitions to reflect this change. OWB stores definitions for database links in the warehouse module that pulls the data but are visible in the Properties inspector. You can specify the host machine using and SQL*Net connect string or provide the host name, port, and the ID of the database instance. Generation Preferences This category defines preferences for the generated scripts. The End of Line parameter that you set is dependent up on the platform to which you are deploying your warehouse. For example, if you are deploying to a UNIX platform, use \ n; and if you are deploying to an NT platform, use \ r\ n. Oracle Enterprise Manager Registration This parameter is used to configure OWB so you can schedule jobs and manage their dependencies using Oracle Enterprise Manager and Oracle Workflow. Note: If your source and target tables are in the same instance, you can choose to link directly to the schema. You set this up in the mapping configuration. This will improve performance. When the source module references an Oracle Designer repository, you must configure the database link so it points to the actual database and not the Designer repository. Configuring a Physical Instance Configure, Generate and Deploy 8-9 Run Time Directories This category defines a set of directories used at run time by SQL*Loader and a directory and host for the OWB Work Flow Queue Listener. OWB uses the set of subdirectories as parameters for the generated TCL scripts that can be registered with Oracle Enterprise Manager. After the generated scripts have been registered and deployed to a file system, you can schedule SQL*Loader jobs using Oracle Enterprise Manager. Table 82 Parameter for Oracle Enterprise Manager Parameter Default Description OEM Job Name _job Suffix append to the name of the mapping used to generate the script. Note: SQL*Loader: the TCL scripts must be executed on the machine that hosts the runtime schema. There is only one TCL script to execute per mapping. Table 83 Run Time Directories Parameter Default Description Log Directory log\ Log directory for the SQL*Loader. Archive Directory archive\ Not currently used. Receive Directory receive\ Not currently used. Input Directory input\ Not currently used. Invalid Directory invalid\ Directory for Loader error and rejected records. Work Directory work\ Not currently used. Sort Directory sort\ Not currently used. Queue Listener Directory workflow\ Location of OWB Work Flow Queue Listener. Queue Listener Host localhost Machine name for the Work Flow Queue Listener. Configuring a Physical Instance 8-10 Oracle Warehouse Builder 3i Users Guide Generation Target Directories This category defines the subdirectories within the Top Directory where OWB stores generated scripts for a target schema. Note: OWB generates the subdirectories in the default Top Directory. If you configure a Top Directory different from the default, then you must create the subdirectories. Table 84 Parameters Target Directory Parameter Default Description TCL Directory tcl\ Location where TCL scripts that can be registered with Oracle Enterprise Manager are stored. These are used to schedule and run initial and incremental loader jobs. DDL Directory ddl\ Location where scripts that create database objects for the target schema are stored. DDL Extension .ddl File name extension for DDL scripts. DDL Spool Directory ddl\log\ Location where DDL scripts are buffered during script generation processing. LIB Directory .lib\ Owner s password LIB Extension .lib Location where scripts that generate user-defined funtions and procedures are stored. LIB Spool Directory lib\log\ Suffix append to mappings name PL/ SQL Directory pls\ Location where PL/ SQL scripts are stored. PL/ SQL Extension .pls File name extension for PL/ SQL scripts. PL/ SQL Run Parameter File _run.ini Suffix for the parameter script in a PL/ SQL job. PL/ SQL Spool Directory pls\log\ Location where PL/ SQL scripts are buffered during script generation processing. Loader Directory ctl\ Location where control files are stored. Loader Extension .ctl Suffix for the loader scripts. Loader Run Parameter File _run.ini Suffix for the parameter initialization file. Configuring a Physical Instance Configure, Generate and Deploy 8-11 Identification The Top Directory parameter specifies the location where OWB stores generated scripts. The value for the parameter must terminate with appropriate file separator for the OS that OWB client is running on, on NT this would be the "\ ", and on UNIX this would be the "/ " (backslash character). The directory contains several subdirectories which are configured in the Generation Target Directories category, and it resides on or must be mapped directly to the machine that hosts the target warehouse. Configuring Dimensions OWB always generates two scripts for a dimension: one to create the dimension object and the other to create the underlying table. Additional scripts are generated when indexes are defined on the dimension table, or when the definition was generated by the Time Dimension Wizard. In the latter case, OWB generates an insert statement that loads the dimension. The configuration parameters for a dimension specify how these scripts are generated and whether the dimension is deployed. To configure a Dimension: 1. Open the Warehouse Module Editor, select the Dimension and select Edit > Configure.... Pure Extract Run Parameter File _run.ini Suffix for the parameter script in a P*E job. Pure Integrate Run Parameter File _run.ini Suffix for the parameter script in a P*E job. Note: OWB generates the subdirectories only when the default Top Directory is used. If you configure a different Top Directory, then you must create the directories and its subdirectories. Table 84 Parameters Target Directory Parameter Default Description Configuring a Physical Instance 8-12 Oracle Warehouse Builder 3i Users Guide The Configuration Properties inspector displays with seven categories of configuration parameters. 2. Expand to display the parameters for each category. The following sections discuss each of the categories and their corresponding parameters. Use these as a reference. For information on creating indexes, partitions, and partition keys see "Configuring Tables" on page 8-19. Performance and Parallel Parameters Use these to influence query and load performance. Table 85 Performance and Parallel Parameters Parameter Default Description Log to Redo logging Recovery requirements for a data warehouse are usually less strenuous than for a transaction system. Set to nologging to improve performance. Configuring a Physical Instance Configure, Generate and Deploy 8-13 For additional information on the package OWB uses in the analyze script, see the Oracle8i Designing and Tuning and Performance and PL/SQL Users Guide and Reference guides. Partition Parameters When you partition a table, you specify a partition key and method (Hash or Range). The Partition Parameters apply when you create hash partitions. For more information see "Partitions" on page 8-21. Storage Space A dimension table can reside in one or more tablespaces. Use the Tablespace parameter to specify the name of each tablespace. Generation Options for Time Dimension When you create a definition for a dimension using the Time Dimension Wizard, OWB generates an insert statement to load the underlying table. The Start and End Date parameters determine the range of dates. Generation Options This category contains parameters that determine whether to generate scripts that create a dimension object and its underlying table. Analyze 99 Value represents the sample size as a percentage of total rows. When set to a nonzero value, OWB generates a DDL script to analyze the table. OWB stores the generated script in the \ DDL directory as tablename_ANALYSE and the script can be run at the discretion of the warehouse administrator. To improve performance, use a high number for a large dimension table and a low number for a small dimension table. Parallel parallel Enables parallel processing when the table is created. If you are using a single CPU or a non-Oracle database, set to nonparallel to improve performance. Table 85 Performance and Parallel Parameters Parameter Default Description Configuring a Physical Instance 8-14 Oracle Warehouse Builder 3i Users Guide Identification The Top Directory parameter specifies the location where OWB stores generated scripts. The value for the parameter must terminate with appropriate file separator for the OS that OWB client is running on, on NT this would be the "\ ", and on UNIX this would be the "/ " (backslash character). The directory contains several subdirectories which are configured in the Generation Target Directories category, and it resides on or must be mapped directly to the machine that hosts the target warehouse. Configuring Facts To configure a Fact: 1. Open the Warehouse Module Editor, select the Fact and select Edit > Configure.... Table 86 Generation Options Parameter Default Description Generate Table true Generate a script to create the dimension table. Generate Dimension true Generate script to create the dimension object. Set this parameter to true only when the warehouse requires the hierarchy and level information. For example, this information should be created for an instance that relies on materialized views that reference this dimension. Note: OWB generates the subdirectories only when the default Top Directory is used. If you configure a different Top Directory, then you must create the directories and its subdirectories. Configuring a Physical Instance Configure, Generate and Deploy 8-15 The Configuration Properties inspector displays the ten categories of parameters. 2. Expand to display the parameters for each category. The following sections discuss each of the categories and their corresponding parameters. Use these as a reference. For information on creating indexes, partitions, and partition keys see "Configuring Tables" on page 8-19. Performance and Parallel Parameters Use these to influence query and load performance. Table 87 Performance and Parallel Parameters Parameter Default Description Log to Redo logging Recovery requirements for a data warehouse are usually less strenuous than for a transaction system. Analyze 99 Value represents the sample size as a percentage of total rows. When set to a nonzero value, OWB generates a DDL script to analyze the table. OWB stores the generated script in the \ DDL directory as tablename_ANALYSE and the script can be run at the discretion of the warehouse administrator. Parallel parallel Enables parallel processing when the table is created. Also enables parallel processing or access. If you are using a non-Oracle database, set to nonparallel to improve performance. Configuring a Physical Instance 8-16 Oracle Warehouse Builder 3i Users Guide For additional information on the package OWB uses in the analyze script, see the Oracle8i Designing and Tuning and Performance and PL/SQL Users Guide and Reference guides. Partition Parameters When you partition a table, you specify a partition key and method (Hash or Range). The Partition Parameters apply when you create hash partitions. For more information see "Partitions" on page 8-21. Storage Space A fact table can reside in one or more tablespaces. Use the Tablespace parameter to specify the name of each tablespace. A fact can also be partitioned. Identification The Top Directory parameter specifies the location where OWB stores generated scripts. The value for the parameter must terminate with appropriate file separator for the OS that OWB client is running on, on NT this would be the "\ ", and on UNIX this would be the "/ " (backslash character). The directory contains several subdirectories which are configured in the Generation Target Directories category, and it resides on or must be mapped directly to the machine that hosts the target warehouse. Configuring Materialized Views OWB generates one script to create a materialized view and another for its foreign key reference constraints. This is the same generation process used for views and tables. Additional scripts are generated when indexes are defined on the view Note: Partition Exchange Loading requires different settings. If you are using Partition Exchange Loading, refer to "Setting Partition Exchange Loading" on page 7-18. Note: OWB generates the subdirectories only when the default Top Directory is used. If you configure a different Top Directory, then you must create the directories and its subdirectories. Configuring a Physical Instance Configure, Generate and Deploy 8-17 To configure a Materialized View: 1. Open the Warehouse Module Editor, select the Materialized View and select Edit > Configure.... The Configuration Properties inspector displays. 2. Expand to display the parameters for each category. The Configuration Properties inspector for a materialized view contains six categories, and each category, except for one, corresponds with a dimension table category. For a discussion of those categories common to dimensions and materialized views, see the discussion on page 8-11. The lone exception, Materialized View Parameters, is described in detail on the following page. Usage Notes: s OWB does not generate code for a view if its query text is not included in its Properties inspector nor if it has no columns defined. s OWB generates a Create Materialized View statement to deploy the view even if its syntax is invalid: OWB does not check the syntax of the select statement used to define a view. Configuring a Physical Instance 8-18 Oracle Warehouse Builder 3i Users Guide Materialized View Parameters These determine how Oracle8i manages the view: Configuring Sequences OWB generates a script for each sequence object. Each sequence object has a start and increment parameter. Both parameters are numeric. To configure a Sequence: 1. Open the Warehouse Module Editor, select the Sequence and select Edit > Configure.... Parameter Default Description Build Immediate Immediate populates the view when it is created. Deferred delays population until the next refresh operation. Refresh Complete Complete specifies the complete refresh method implemented by executing the views query. Fast specifies the incremental refresh method which refreshes the view according to changes that have occurred to the master tables. Force specifies that when a refresh occurs, Oracle8i performs a fast refresh if possible or a complete refresh otherwise. For more information on restrictions to the refresh parameter, see the Oracle 8i documentation. Query Rewrite Enable Enable marks the view eligible for query rewrite. Disable marks the view ineligible for query rewrite. Base Tables Blank Type in the exact name of the base tables separated by commas. If the tables were created in Preserve Case, enclose table names with quotes. Configuring a Physical Instance Configure, Generate and Deploy 8-19 The Configuration Properties inspector displays. 2. Expand to display the parameters for each category. Configuring Tables OWB generates multiple scripts to create a table depending on how the table is created. One creates the table and there are separate scripts for each constraint that is defined on the table, there is also an ANALYZE TABLE.. script generated. Additional scripts are generated when indexes are defined on the table. To configure a Table: 1. Open the Warehouse Module Editor, select the Table and select Edit > Configure.... The Configuration Properties inspector displays. Configuring a Physical Instance 8-20 Oracle Warehouse Builder 3i Users Guide 2. Expand to display the parameters for each category. The configuration parameters for a table specify how these scripts are generated and whether the table is deployed. The Configuration Properties inspector for a table contains five categories, and are a subset of the categories defined for a dimension. The discussion on page 8-11 describes each of these parameters. The following section describes the configuration of indexes, and partitions that you want created on a table when it is deployed.
Indexes Indexes are created on tables to speed up query processing. They are often created on large materialized views for the same reason. You can create definitions for single and multiple column indexes. After you create a definition for an index, you can then generate either a BITMAP, UNIQUE, or a non-specific index. Note: Indexes, and Partitions are purely physical objects. They are not created during the logical modeling. They must be created in the Configuration Properties inspector before you can configure their parameters. See "Configuring New Physical Objects" on page 8-4 for more information. Table 88 Index Parameters Parameter Default Description Index Columns None The columns in the index. Configuring a Physical Instance Configure, Generate and Deploy 8-21 Bitmap indexes are created on a fact tables foreign key reference columns to take advantage of the Oracle8i Star Transformation algorithm. Composite indexes are created on a subset of a tables foreign key reference columns. This more application-specific index is created to take advantage of the Oracle8i star optimization algorithms and works well when user queries mostly constrain a subset of a fact tables dimensions. For a complete discussion of indexing strategies, see the Oracle8i Data Warehousing and Oracle8i Designing and Tuning for Performance Release 3 (8.1.7 or 9i) guides. Partitions Partitions are created on fact tables to improve query and load performance and to simplify the management of physical storage. Transportable tablespaces are often introduced for performance reasons and to transport data across Oracle8i databases. A transportable tablespace is the fastest means to move a large volume of data between two Oracle8i databases. Dimension tables and materialized views also occupy storage space and are often candidates for intelligent partitioning. Log to Redo Logging Indicates whether the creation of the index will be logged in the redo log file. Set to nologging to improve performance. Parallel Parallel Select parallel if you want Oracle to select a degree of parallelism. If you are using a non-Oracle database, set to nonparallel to improve performance. Tablespace Users Specifies the tablespace to hold the index in Index Type Bitmap The type of index: bitmap or unique Local Index true Allows you to specify that the index is partitioned on the same columns, with the same number of partitions and the same partition bounds as the table. Name blank The name of the index Previous Deployment Name blank Previous name of the index when it was last deployed. This is used during warehouse upgrade Deployable true Specifies if this index is deployable. Table 88 Index Parameters Parameter Default Description Configuring a Physical Instance 8-22 Oracle Warehouse Builder 3i Users Guide You can create and configure definitions for partitions directly from the Configuration Properties inspector. A table can be partitioned using hash, range partitions, or a combination of range and hash partitions. Table 89 Partition Parameters Parameter Default Description Value Less Than Specify the non inclusive upper bound for the current partition. The entry is an ordered list of literal values corresponding to column_list in the partition_by_range_ clause. You can substitute the keyword MAXVALUE for any literal in value_list. MAVALUE specifies a maximum value that will always sort higher than any other value, including NULL. Tablespace The value specified is the actual physical attribute of the segment associated with the table. For partitioned tables, the value specified for TABLESPACE is the default physical attribute of the segments associated with all partitions specified for the table. Name blank Name of the partition Previous Deployment Name blank Previous name of the partition when it was last deployed. This is used during warehouse upgrade Deployable true Specifies if this partition is deployable. Configuring a Physical Instance Configure, Generate and Deploy 8-23 Table 810 Partition Key Parameters Hash Partitions To create a hash partition, specify a hash key, number of partitions, and tablespace names. Oracle partitions the storage space and stores rows according to a hash algorithm. Range Partitions To create a set of range partition, you specify a range key, names for the partitions, and a set of tablespaces. Oracle8i then stores rows in a partition according to the specified ranges. Range with Hash SubPartitions You can create a set of range partitions that have hashed subpartitions. For this case, create a range partition key, a set of partitions, and then create a hash partition key to specify the number of logical partitions. Configuring Views OWB generates a script for each view defined in a warehouse module. You can configure whether to deploy specific views or not. To configure a View: 1. Open the Warehouse Module Editor, select the View and select Edit > Configure.... Parameter Default Description Type Can be of any built-in data type except ROWID, LONG, or LOB. Name blank Name of the partition key. Previous Deployment Name blank Previous name of the partition key when it was last deployed. This is used during warehouse upgrade Deployable true Specifies if this partition key is deployable. Note: If you are using Partition Exchange Loading, please refer to the Oracle 8i documentation to set up range partitions. Validating Definitions 8-24 Oracle Warehouse Builder 3i Users Guide The Configuration Properties inspector displays. 2. Expand to display the parameters for each category. Validating Definitions After you configure a set of definitions for a physical instance, you need to validate the set before you generate and deploy objects and scripts to a physical instance. This section describes why validation is necessary and how to validate a selected set of definitions. Rationale for Validation A set of configured definitions need to be validated before scripts are generated for several reasons. These are some of the main reasons: s Malformed definitions As a flexible design tool, OWB allows you to create definitions that can be malformed. For example, you could add a foreign key constraints to a fact table that does not as of yet reference a column in a dimension table. OWB stores this definition in the warehouse module but the definition will fail the validation process until the reference is satisfied. Usage Notes: s OWB does not generate code for a view if its query text is not included in its Properties inspector nor if it has no columns defined. s OWB generates a Create View statement to deploy the view even if its syntax is invalid: OWB does not check the syntax of the select statement used to define a view. Validating Definitions Configure, Generate and Deploy 8-25 s Data type mismatches OWB allows you to create definitions that allow data type mismatches but the validation process traps these cases and returns an warning message. s Metadata Import Utility The Import Utility can include definitions that are malformed for a number of reasons. For more information, see "Metadata Import" on page 10-8. Valid and Invalid Definitions During the validation process, OWB assigns a Warning or Error code to each malformed expression within an individual definition. Valid Definition A definition is Valid if the validation process returns no Error codes. The definition can include one or more Warning codes. Invalid Definition A definition is Invalid if the validation process returns at least one Error code. The definition can also include one or more Warning codes. The definitions for the following objects are all invalid because their definitions contain at least one error code. Validating Definitions 8-26 Oracle Warehouse Builder 3i Users Guide Validating a Set of Definitions You can validate a single definition, a set of definitions, or all the definitions stored in a warehouse module. To validate a set of definitions: 1. Open the Warehouse Module Editor, and select an object or group of objects in the Logical Tree to validate. 2. Select Module > Validate from the toolbar menu. OWB validates the selected objects definitions and returns the results in a Validation window. There are two panes in this window: Selected Objects and Validation Messages. Selected Objects This part of the window lists summary information for each object selected for validation. The summary information includes: s Names of the objects selected for the validation s Date and time of the last validation s Status: Valid or Invalid s Date and time when the object was last updated. Validating Definitions Configure, Generate and Deploy 8-27 There are also three buttons: s Validate: This allows you to re-validate after you have made some changes. s View: This displays the validation results in a text editor. This can be useful if you want to save these results to a file. s Refresh: This allows you to refresh your validation results. This can be useful if you are running validations in more that one window. Validation Messages This part of the window lists individual violations for the highlighted object: s Names of the object that violates a validation rule s Validation Code: Error - This code displays if the script will cause code generation or deployment failure. Warning - This code displays if the script may cause failure during generation, deployment, or runtime Success - This code displays if the script will not cause any known problems. s Validation message that describes the nature of the violation There are two buttons: s Details: This allows you to view additional details about the object you have highlighted in the Validation Details window. s Edit: This allows you to go directly into an Editor in OWB and edit the object you have highlighted. For example, if you have a table highlighted and you click Edit, the Table Editor opens with the highlighted table displayed. Generating Scripts 8-28 Oracle Warehouse Builder 3i Users Guide Generating Scripts Once you have a validated a set of definitions for a configured physical instance, you can generate the scripts required to create and populate the instance. During this phase, OWB generates scripts for the selected warehouse module. OWB generates several kinds of scripts for a target warehouse: s DDL scripts to create or drop database objects s SQL*Loader control files to extract and transport data from file sources s TCL scripts to schedule and manage jobs (registered with Enterprise Manager) When you generate objects, it is important to note that a single script is generated for each physical object you want to create. For example, there is one script for each index you are creating. This can be extremely useful if you need to re-deploy a single object at a later time. This allows you to not have to invest the resources into deploying the entire warehouse all over again. The following discussion summarizes the actual scripts generated for each object in the warehouse module. To generate scripts for a set of objects in a warehouse module: 1. Open the module and select a set of warehouse objects. As with validation, you can select an individual object, a set of objects, or all the objects in the module. 2. Select ModuleGenerate. Note: OWB returns a Validation Results window when you validate a definition. To view these results at a later time you can select ViewValidation Messages. OWB returns the same results in an equivalent window labeled Validation Status. Generating Scripts Configure, Generate and Deploy 8-29 OWB displays a dialog box. 3. Select what kind of script is to be generated and click OK. OWB generates the scripts and returns the Generated Results window.
Note: OWB returns a Generated Results window when you generate a script to create an object. To view these results at a later time you can select ViewGenerated Scripts. OWB returns the same results in an equivalent window labeled Generated Scripts. Generating Scripts 8-30 Oracle Warehouse Builder 3i Users Guide The Generated Scripts OWB generates a variety of scripts that are used to implement, populate, and run a configured instance of the data warehouse described by definitions stored in a warehouse module. The following discussion summarizes the scripts generated for each kind of warehouse object. These summaries correspond with the branches of a warehouse modules physical navigation tree. Database Links A data warehouse pulls data from its database sources using database links. The mechanism is straightforward: s OWB generates a script for a database link to the source database. s The database link is deployed in the target schema. s The PL/ SQL scripts that pull data from a source database reference the source tables using the deployed synonyms. For example, OWB generates the following script to create a database link to the GCCAPPS source. ALTER SESSION ENABLE PARALLEL DDL; CREATE DATABASE LINK GCCAPPS CONNECT TO gccapps IDENTIFIED BY manager USING gccnc; This database link is deployed to the physical instance by executing the script. Dimensions OWB can generate multiple ddl scripts for each dimension depending on whether indexes are configured. The scripts can include a ddl script for: s The underlying table and its configured partitions s The hierarchies and levels defined on the table. s The indexes and index partitions configured for the table. Generating Scripts Configure, Generate and Deploy 8-31 After generating the ddl, OWB returns the Generation Results window. The Generated Results window contains a list of scripts that create three different kinds of objects for the Days dimension: s A Table s A Dimension s An Index You can view any of the generated scripts by selecting the dimensions name and then clicking the View button. The example below illustrates the code generated to add hierarchies onto an existing table: ALTER SESSION ENABLE PARALLEL DDL; CREATE FORCE DIMENSION customers_DIM LEVEL customer IS CUSTOMERS.cs_cust_WH LEVEL account IS CUSTOMERS.at_account_ID LEVEL segment IS CUSTOMERS.sg_segment_ID HIERARCHY customer_sums ( customer CHILD OF account CHILD OF segment ) ATTRIBUTE customer DETERMINES (cs_loctype,cs_ship_to_num) ATTRIBUTE account DETERMINES (at_account_desc) ATTRIBUTE segment DETERMINES (sg_segment_desc); Views and Materialized Views OWB can generate multiple ddl scripts for each materialized view depending on whether indexes are configured for the view. The scripts can include ddl scripts for: s The materialized view and its configured partitions. s The foreign key reference constraints. s The indexes and index partitions configured for the table. s The partitions configured for the table. Generating Scripts 8-32 Oracle Warehouse Builder 3i Users Guide After generating the ddl, OWB returns the Generated Results window. You can view the generated code and deploy the materialized view from this window. If the view you are generating is part of a mapping, OWB does not generate a ddl script for the view. OWB generates regular package with assumption that the view is updatable. No error checking is done to verify if view is updatable or not. You must make sure that the view meets the definition of an updatable view before including this view into the map. See the Oracle 8i SQL Reference for the definition of updatable view. Views and materialized views are generated in the same way as regular tables Mappings OWB generates multiple scripts to implement each PL/ SQL mapping: PLS scripts create the PL/ SQL packages that pull data from database sources, perform transformation operations, and load data into the physical instance of the data warehouse. DDL scripts create database links. TCL scripts that are registered with Oracle Enterprise Manager and Oracle Workflow to support job scheduling as well as to capture and log audit and error information. Run the Packages When OWB returns the Generated Results window for a mapping, it enables the Run button. Thus, after you deploy the mapping to the target schema, you could push this button to load data into the target warehouse. See Chapter 9, "Periodic Loads and Updates" for information on how to schedule and monitor database loading using Oracle Enterprise Manager and Oracle Workflow, and see Chapter 10, "Administration" for information on analyzing audit and error messages using the Builder Runtime Audit Viewer. Notes: When you generate scripts for a view: s OWB does not generate code for the view if a query script in its definition has an error. OWB does not check the query script for errors when you define the view. s If a view references dimensions, then you must deploy the dimensions before you deploy the view. Generating Scripts Configure, Generate and Deploy 8-33 Sequences OWB generates a single DDL script for each sequence defined by the warehouse module. Each script creates a sequence object in the physical instance. Tables OWB generates multiple DDL scripts for each table which include DDL scripts for: s The table and its configured partitions s Constraints on the table s The indexes and index partitions configured for the table s The partitions configured for the table. After generating the DDL, OWB returns the Generated Results window. You can view the generated code and deploy the dimensions from this window. Post Generation Tasks After the scripts have been generated, the Generated Scripts window displays with two tabs: s The Schema Objects Tab s The Transforms Tab Note: Fact tables always reference dimensions over foreign key reference columns. You must deploy the referenced dimensions to the physical instance before you deploy the fact table. Generating Scripts 8-34 Oracle Warehouse Builder 3i Users Guide The Schema Objects Tab. The Schema Objects tab displays objects that you can deploy to create your warehouse. Use the Ctrl button to select multiple objects. You can select from these actions: s View Code: Use this button to view and edit the code generated by OWB. Select the objects name and then click View Code. OWB returns the generated code in a code editor. You can choose to view or edit the code. If you edit the code, you will be prompted to save your changes when you close the window. s Deploy...: This deploys the selected objects to your target data warehouse. See the descrption under the Transforms tab for more information. s Save as File: This allows you to save the code to a file in order to use another tool to deploy. See the descrption under the Transforms tab for more information Generating Scripts Configure, Generate and Deploy 8-35 The Transforms Tab. The Transforms tab displays mappings that you can deploy to create your warehouse. Use the Ctrl button to select multiple objects. Select an object and choose from these actions: s View Code: Use this button to view and edit the code generated by OWB. Select the objects name and then click View Code. OWB returns the generated code in a code editor. You can choose to view or edit the code. If you edit the code, you will be prompted to save your changes when you close the window. s Deploy...: Use this button to deploy selected object to a physical instance of the warehouse. This is the point in time when your data warehouse is physically created. In order to create your data warehouse, you must select all the necessary generated scripts and then click Deploy... .You must select the runtime database you are deploying to each time you deploy. This allows you to be able to configure once and deploy to many instances. OWB allows for flexible deployment. This means that you have the option of selecting which scripts you want to deploy to your warehouse. Typically, you would deploy all scripts when you initially create your warehouse. However, if you decide to create additional objects, such as indexes at a later time, you have the option of generating and deploying those scripts without having to re-deploy your entire warehouse. Generating Scripts 8-36 Oracle Warehouse Builder 3i Users Guide s Save as File: This allows you to save the code to a file in order to use another tool to deploy. Choose this option when you want to deploy a script that creates an object to a subdirectory of the Top Directory. The Top Directory is an operating system directory that is configured for the warehouse module. OWB writes the generated script to a subdirectory that corresponds with the scripts language as defined in the Generation Results or Generated Scripts inspector. Specifically, the script is written to the file: \Top Directory\ddl\database_link_name.ddl For example, the generated scripts for the Customers dimension are written to the DDL subdirectory in the following form: e:\target\ddl\Customers.ddl e:\target\ddl\Customers_DIM.ddl Where the Top Directory was configured as e:\target. s Run: Use this button to run a deployed PL/ SQL package that implements a mapping. The usage of this button is intended for testing PL/ SQL load packages with a relatively small data samples. Initial loads and periodic refreshes of a physical instance should be scheduled as jobs. s OEM Register: Use this button to register the selected TCL scripts with Oracle Enterprise Manager. OWB generates one TCL scripts for each mapping. Oracle Enterprise Manager and Oracle Workflow uses the TCL script to initiate, execute, and terminate the capture of audit statistics and error messages. If you plan to schedule jobs using Oracle Enterprise Manager or Oracle Workflow, you must register the TCL script with Oracle Enterprise Manager. Note: This function will run the last deployed package that is currently in the database. If you have made changes, you must first re-deploy, before running the scripts. Deploying Scripts Configure, Generate and Deploy 8-37 Deploying Scripts To create and populate the physical instance of the data warehouse, you run the generated scripts to deploy objects in the target schema, deploy the scripts to an operating system directory, register the scripts with Oracle Enterprise Manager and Oracle Workflow, and then schedule jobs to initially load the warehouse. To deploy generated job scripts: 1. Select a module and open the Warehouse Module Editor. 2. Select View > Generated Scripts. Notes: s If you modify the configuration parameters in any way, you must regenerate the scripts, redeploy all of them, and then reregister them with OEM. s If you attempt to register a job with Enterprise Manager that already exists in its Job Library, OWB returns a VDJ-2003 error. You must remove the job from Job Library before you can successfully register a new version. Deploying Scripts 8-38 Oracle Warehouse Builder 3i Users Guide The Generated Scripts window displays with the Schema Objects tab on top. 3. Select the tab that contains the object or mapping you want to deploy and click Deploy.... You can select multiple objects by pressing Ctrl while you select. The Connection Information dialog displays. 4. Specify the runtime connection information and click OK. Deploying Scripts Configure, Generate and Deploy 8-39 The Database Deployment dialog displays. Use this to verify the objects you are deploying. 5. Click Create to continue with the deployment. The deployment process runs and writes the scripts to the file system configured as Top Directory in the warehouse module. Once the process is Deploying Scripts 8-40 Oracle Warehouse Builder 3i Users Guide complete, the Database Deployment dialog displays again, only this time, the results of the deployment are displayed in the lower half of the window.
6. Click Close. The scripts have now been deployed to the database. Note: To change the suffix appended to each mappings name (JOB), see the discussion on configuring a warehouse module in the section "Generation Target Directories" on page 1-27. Note: When you schedule and execute jobs using Oracle Enterprise Manager or Oracle Workflow, you must deploy the following TCL scripts generated by OWB to the Top Directory: s SQL*Loader mapping s PL/ SQL mapping s External OS Command mapping s Pure Extract mapping s Pure Integrate mapping Upgrading a Warehouse Physical Instance Configure, Generate and Deploy 8-41 Upgrading a Warehouse Physical Instance Once you have populated a physical instance, you may later need to drop, reconfigure, rename, and upgrade the objects in this instance to reflect changes in your environment. OWB enables you to directly propagate incremental changes in your logical warehouse design to your physical instance, without having to drop objects or lose existing data. For example, you may have tables containing data in your physical instance. If you modify the definition of these tables within OWB by adding an index, changing a constraint, or renaming a column, you can directly reconcile these changes with the tables in your physical instance using the warehouse upgrade feature. Your changes are applied in a manner that preserves the existing rows of data within the tables in the physical instance. Figure 81 Warehouse Physical Instance Upgrade OWB utilizes the Change Management (CM) Pack to analyze the existing contents of a target instance and synchronize them with changes in the OWB warehouse design. Next, OWB provides an Impact Report generated by CM detailing the plan for the upgrade; CM also generates necessary SQL and DDL scripts to manage this transition. Before executing the scripts, you should review the Impact Report to make an informed choice on whether to perform the upgrade or take the appropriate action. Generate Code Impact Report Warehouse Analysis
Warehouse Repository Change Metadata Warehouse Metadata Analyzed Upgrade Plan Generated SQL and DDL Scripts Generated for Upgrade Execute Upgrade
Scripts Upgrading a Warehouse Physical Instance 8-42 Oracle Warehouse Builder 3i Users Guide You can create, add, update, and rename the following Warehouse objects: a The warehouse upgrade feature does not currently support deleting objects. To delete an object, you can select the Drop objects first checkbox in the Generation Mode dialog. This will create DROP scripts for the objects selected. OWB and the OEM Change Management Pack (OEM/CM) Before you choose to upgrade a warehouse instance, OWB analyzes the difference between your warehouse design and the target warehouse instance. For this purpose, OWB utilizes OEM/ CM to compare OWB metadata against objects in the target warehouse. For example, the change can be as simple as adding a column to a table, or it can be as complex as adding a column, removing a column, adding an index and removing a partition all at once . OWB passes information on to CM in terms of what the target table should look like. CM processes this information and generates the scripts necessary for creating the table as well as creating a report which lists, in detail, the impact the changes will have. OWB retrieves the impact report generated by CM. An SQL summary of the changes that will take place is provided by CM. CM then generates a TCL script to deploy the changes. Note: The Change Management Pack only comes with Oracle Enterprise Manager 2.2 and Oracle 8.1.7 and higher. Dimension Table Materialized View Constraint (UK, FK, and check constraint) Dimension Object Materialized View log Indexes Hierarchy Table Database link Fact Table View Note: The upgrade feature does not work if you deploy a table with Float data type because the OEM Change Management Pack does not support it. If you want to upgrade tables with FLOAT data type, the NUMBER data type can be used to achieve the same results. Upgrading a Warehouse Physical Instance Configure, Generate and Deploy 8-43 For OWB to utilize the OEM Change Management Pack, you must have: s Oracle 8.1.7 database running. s OWB 3i client installed in the same Oracle home as the Oracle 8.1.7 client. s OEM Change Management 2.2 installed in the same oracle home as Oracle 8.1.7 client, with OMS 2.2 running on an accessible host. Additionally, you need to configure the OEM Preferences panel within the OWB Preferences window. To configure the OEM Preferences panel 1. Select Project > Preferences. OWB returns the OWB Preferences window containing seven tabs. 2. Select the OEM tab. 3. Enter the following connection information: s The OEM Domain - This should be the same machine that you used during the OEM installation. For more information, please refer to the OWB Installation Guide. s The OEM User s The OEM Password 4. Click OK. OWB is now configured to utilize the OEM Change Management Pack technology. Generating and Executing Upgrade Scripts To update scripts for a set of objects in a warehouse module, open the warehouse module editor and select a set of definitions you want to upgrade. As with validation, you can select an individual definition, a set of definitions, or all the modules definitions (See discussion "Generating Scripts" on page 8-28). To generate and execute upgrade scripts 1. Right-click any warehouse module or objects under any warehouse module in the Logical Tree, and select Module/ObjectGenerate. OWB returns the Generation Mode dialog. Upgrading a Warehouse Physical Instance 8-44 Oracle Warehouse Builder 3i Users Guide 2. Select the radio button next to Generate upgrade scripts. 3. Click OK. OWB returns the Connection Information dialog. 4. Enter the username, password, machine name, port number and SID for the target runtime schema you want to upgrade. 5. Click OK. OWB returns the Generation Progress panel indicating the progress of the analyses and script generation for the selected objects. Once the process is complete, OWB returns the Generation Results dialog. 6. The Generation Results dialog displays an Impact Report listing all objects to be upgraded using the newly generated scripts. Note: There is only one Impact Report generated for all the objects selected for upgrade. Upgrading a Warehouse Physical Instance Configure, Generate and Deploy 8-45
From this dialog, you can view the Impact Report and Generation Summary information and perform the following actions: s View Code Click this button to view a SQL summary of the changes that will take place. The summary is shown in a separate window and is read-only. If you view the generated code, you will see hints that contain previously deployed object names. This is included for informational purposes only. For example, if you are changing the name of the object from DIM to DIM1, DIM will be in the hint. This basically informs you that the table DIM1 that you are creating, was named DIM when it was created last. s Upgrade Use this button to upgrade the selected objects at the physical instance of the target warehouse. s Save File s Print File Upgrading a Warehouse Physical Instance 8-46 Oracle Warehouse Builder 3i Users Guide 7. If there are no errors and your schema objects can be readily upgraded from the information supplied, the Upgrade button is activated. Click Upgrade to run the scripts. OWB returns the Upgrade dialog. 8. The Upgrade dialog confirms script that will upgrade the objects that were selected. At this point, you must click Execute to execute the script. 9. When the script completes, you can either click Commit to save the changes you have made, or Rollback to erase the changes and stop the upgrade. The physical instance in your target warehouse is upgraded. Note: If the Upgrade button is deactivated, look for error messages within the Impact Report. Upgrading a Warehouse Physical Instance Configure, Generate and Deploy 8-47 10. Close the window. Note: Since dimensions are not supported by CM, when a dimension is upgraded, OWB will drop and create the dimensions. The drop and create statements appear along with the summary scripts. These statements are executed only after you commit and close the Upgrade window. Taking the Next Step 8-48 Oracle Warehouse Builder 3i Users Guide Taking the Next Step This chapter showed you how to implement a physical instance of a logical warehouse. The following tasks were described: s Configure a physical instance of the logical warehouse defined by a OWB warehouse module. s Validate a set of configured definitions s Generate a complete set of scripts to create, populate, and run a physical instance of the logical warehouse. s Deploy the warehouse objects to a physical instance. s Test the PL/ SQL packages deployed to a physical instance. s Register TCL scripts generated by OWB with Oracle Enterprise Manager. s Move, or upgrade, a physical instance to a new version of OWB. The physical instance of the data warehouse is now ready to be loaded with data. The next chapter, "Periodic Loads and Updates", shows you how to use Oracle Enterprise Manager and Oracle Workflow to schedule jobs that initially load data into the physical instance and periodically refresh the instance. Periodic Loads and Updates 9-1 9 Periodic Loads and Updates This chapter shows you how to set up and schedule jobs that initially load and periodically refresh a data warehouse. You can enforce the job processing sequence manually or through automation. This chapter illustrates both methods. This chapter covers the following topics: s Loading and Updating Jobs s Scheduling with Oracle Enterprise Manager s Managing Dependencies Using Oracle Workflow s Viewing the Runtime Audit s Taking the Next Step Loading and Updating Jobs 9-2 Oracle Warehouse Builder 3i Users Guide Loading and Updating Jobs Jobs that load new data or update existing data in a data warehouse must be run in a strict sequence to insure that all foreign key references can be satisfied. Generally, this means that referenced tables must be loaded before the tables making the references can be loaded. The most obvious case is a fact table and its dimensions. The dimensions must be loaded before the fact table can be loaded. A less obvious case is a materialized view, its underlying fact table, and the dimensions it references. The materialized view cannot be refreshed until the fact table and the referenced dimensions have been loaded. To insure that warehouse load and refresh jobs are correctly sequenced, you can schedule the jobs in batches, one batch at a time. After the first batch finishes, you check its completion status before you schedule the second batch, and so on. Scheduling Jobs There are two main ways for you to schedule your deployed scripts: Manual Scheduling After you generate and deploy your scripts you are given the option of registering them as jobs in Oracle Enterprise Manager (OEM). You can then schedule a job, or sequence of jobs. OEM allows you to manually schedule each job within a batch and check the completion status of each job in the batch before scheduling the next batch. For more information about scheduling using OEM, see "Scheduling with Oracle Enterprise Manager" on page 9-3. Automated Scheduling After setting up your jobs in OEM, you can proceed to automate those jobs based on a process you define. Oracle Workflow allows you to automate the entire schedule for a set of jobs by defining a process. You can then schedule the process rather than the individual jobs. The Oracle Workflow server manages the process so that jobs run in the proper sequence. If an exception occurs, the Workflow server terminates the process. This minimizes rather than eliminates manual intervention. For more information about scheduling using Workflow, see "Managing Dependencies Using Oracle Workflow" on page 9-10. Types of Jobs These are the different types of jobs listed in the OEM: s Loader Mappings extract and load data from flat file sources. s PL/ SQL Mappings extract, transform, and load data from database sources. Scheduling with Oracle Enterprise Manager Periodic Loads and Updates 9-3 s Pure*Extract and Pure*Integrate Mappings are imported from an exported file. s Generic Mappings allow operating system scripts to be executed. The Loader and PL/ SQL scripts were generated and deployed to the target warehouse. The following discussion shows you how to deploy generated job scripts (TCL) associated with these mappings to a file system configured in the warehouse module. The file system is configured as the Top Directory and the subdirectories are configured as Target Generation Directories. You can deploy and register any kind of mapping script to Enterprise Manager s Job Library including Pure Integrate Mappings. Scheduling with Oracle Enterprise Manager To schedule jobs with Oracle Enterprise Manager, make sure you have completed the follow steps: s Configured the target schemas warehouse module. s Created a Windows NT user name. s Configured a set of Preferred Credentials for Enterprise Manager. s Started services for Oracle Agents and Enterprise Manager on the machine that hosts the target schema. If you have not, please see the Installation Guide for more information. The next steps in scheduling jobs with OEM are deployment and registration. Note: If you modify a configuration value in a warehouse module or a mapping definition after the scripts have been deployed to the Generation Directories or registered with Oracle Enterprise Manager, you must: s Re-generate the scripts. s Re-deploy the scripts to the Generation Directories. s Re-register the scripts with Oracle Enterprise Manager. You will need to remove the existing scripts from the Job Library. These three steps should be standard operating practice even if the configuration change seems to be a minor one. Scheduling with Oracle Enterprise Manager 9-4 Oracle Warehouse Builder 3i Users Guide Deploying and Registering the Jobs After the warehouse module for the target warehouse schema has been configured and Enterprise Manager has been supplied with a set of preferred credentials, you can deploy the generated scripts and register them with Enterprise Manager. The deployment and registration takes three steps: 1. Deploy the job scripts to the file system configured in the warehouse module. 2. Register the generated scripts with Enterprise Manager. 3. Verify that the scripts have been registered in Enterprise Manager s Job Library. Registering Job Scripts To register generated job scripts: The deployment process runs and writes the scripts to the file system configured as Top Directory in the warehouse module. Once the process is complete, the Database Deployment dialog displays again, only this time, the results of the deployment are displayed in the lower half of the window. Scheduling with Oracle Enterprise Manager Periodic Loads and Updates 9-5
4. Click Close. The scripts have now been deployed to the database. 5. Select the deployed scripts and click Save as File. The scripts must be saved or you will get an error message if you attempt to register them. 6. Select the objects you want to register with OEM and click OEM Register. You must select the TCL language jobs to register with the OEM. OWB connects with Enterprise Manager, registers the scripts necessary to run the load jobs, and returns a confirmation list. Verifying OEM Registration To verify registration, start the Enterprise Manager Console. Then, from the tool bar: 1. Select Job > Job Library Note: To change the suffix appended to each mappings name (JOB), see the discussion on configuring a warehouse module in the section "Generation Target Directories" on page 1-27. Scheduling with Oracle Enterprise Manager 9-6 Oracle Warehouse Builder 3i Users Guide Enterprise Manager returns a list of jobs currently registered in its library. You can select a specific job and set up its schedule directly from this window. Examples in the next section show you how. Note: You cannot register a job with Oracle Enterprise Manager if that job already exists in Enterprise Manager s Job Library. To update an existing job, remove the job from the Job Library first. Scheduling with Oracle Enterprise Manager Periodic Loads and Updates 9-7 Scheduling the Jobs Now that all the job scripts that initially load the warehouse have been registered, you can schedule the jobs and check their runtime status using Oracle Enterprise Manager. You schedule a job directly from the Enterprise Managers Job Library Window and check its status from the Enterprise Managers Job pane. Scheduling To schedule a job, select the jobs name, click Edit. Enterprise Manager returns a multi-page Edit Job window. Select the Schedule tab. Enterprise Manager returns the Schedule page. Specify the schedule and close the window. You can submit the job from this panel now or later. Submitting the Job Once the job has been submitted, Enterprise Manager places an entry in the Jobs window. This Window has two panes: Active and History. You can track the status Scheduling with Oracle Enterprise Manager 9-8 Oracle Warehouse Builder 3i Users Guide of a submitted job (Submitted or Running) from the Active pane. After the job has completed, you can check its status in the History pane (Completed or Failed). If you right-click the jobs name from the History pane, Enterprise Manager returns a pop-up list that allows you view more details regarding the job, remove the job from the history log, or create another job like the selected job. The Enterprise Managers log contains summary information about a jobs execution, but in those cases when the job fails to terminate normally, you need more detailed information. The OWB Runtime Audit Viewer provides that type of information. For more information, see "Viewing the Runtime Audit" on page 9-23. Submitting Jobs with Parameters The default value is stored in the mappings configuration parameters. If you open the parameter panel for this mapping, you can modify the value. This action takes effect only if you re-generate the script. The default value occurs in one of the TCL scripts generated by OWB. You can modify the parameter s value before you submit the job through Oracle Enterprise Manager. The following example shows you how to modify a parameter s value when you submit the job. Note: The above discussion barely introduces you to Oracle Enterprise Manager. For an in-depth discussion of its advanced scheduling capabilities, refer to the online reference materials within the Oracle Enterprise Manager product. Scheduling with Oracle Enterprise Manager Periodic Loads and Updates 9-9 Running the TCL Script To modify a parameter s value for a job using Enterprise Manager, open the Job Library from the toolbar, select the job name (SALES_ MAPPING_job), and then click Edit. OWB returns the Job Editor window. Select the parameters tab. OWB returns the Parameters panel. Select the TCL script and locate the parameter in the Parameters text box. Modify the parameter in the text box and then submit the job. Managing Dependencies Using Oracle Workflow 9-10 Oracle Warehouse Builder 3i Users Guide Managing Dependencies Using Oracle Workflow To automate the scheduling of an entire set of jobs, you can define an Oracle Workflow process. Then, you schedule the process rather than the individual jobs, and the Oracle Workflow server manages the jobs so that they run in the proper sequence. If an exception occurs, the Workflow server terminates the process. This approach minimizes manual intervention. To define a process that sequences a set of jobs defined within a warehouse module, follow this general procedure: 1. Configure, validate, and generate the mappings. 2. Deploy the generated mappings to the Generation Target Directories and the PL/ SQL packages to the target database. 3. Register the generated mappings with Oracle Enterprise Manager. 4. Define the Workflow Queue Listener Directory and Host for the warehouse module. These are the configuration parameters on the warehouse module that allow you to specify the host and the location for the OWB Workflow Queue Listener to execute. 5. Deploy the mappings as functions identified by an Item Type to the Oracle Workflow server using the Workflow Deployment Wizard. If you plan to schedule the Workflow process with Enterprise Manager, deploy the process to the Enterprise Managers Job Library. 6. Define the workflow process using Oracle Workflow Builder. 7. Schedule the Workflow Process using Oracle Workflow Monitor or with Oracle Enterprise Manager. The following section describes how to define an Oracle Workflow process for the jobs that initially load the warehouse. Note: This example assumes that: s You are familiar with the terminology of Oracle Workflow Products. s The Oracle Workflow server, Oracle Workflow Builder, and Oracle Workflow Monitor have been installed and configured. s The OWB Workflow Queue Listener Directory and Queue Listener Host parameters have been configured. Managing Dependencies Using Oracle Workflow Periodic Loads and Updates 9-11 Deploying to the Oracle Workflow Server After you have deployed the mappings to the Generation Target Directories and the PL/ SQL packages to the target database, registered the TCL scripts for individual jobs with Oracle Enterprise Manager, you can deploy the mappings to an Oracle Workflow server using the Workflow Deployment Wizard. Using the Workflow Deployment Wizard To use the Workflow Deployment Wizard: 1. Select Tool > Wizard > Workflow Deployment Wizard from the OWB console, or expand the wizard drawer and click the icon for the Workflow Deploy Wizard. The Workflow Deployment Wizard Welcome page displays. 2. Click Next. Managing Dependencies Using Oracle Workflow 9-12 Oracle Warehouse Builder 3i Users Guide The Workflow Login page displays. 3. Specify the following connection information: s User name s Password s Host name s Port number s SID for database instance of the Workflow server The OWB client uses this information to establish a session with the Oracle Workflow server. 4. Click Next. Managing Dependencies Using Oracle Workflow Periodic Loads and Updates 9-13 The wizard returns its Maps page which solicits a list of mappings to be deployed. This page has two panes: s Available maps s Selected Maps Move the names of those maps that belong to the process to the Selected pane. 5. Click Next. Managing Dependencies Using Oracle Workflow 9-14 Oracle Warehouse Builder 3i Users Guide The wizard returns its Functions page which identifies the names that Oracle Warehouse Builder assigned to the individual mappings. Within Workflow, these mappings are called "functions." To display the internal function name for a specific mapping, select the mappings name in the navigation tree. The wizard returns the functions internal name, display name, and description. You cannot modify names on this page. 6. Click Next. s The Item Type page displays. Specify the following: s An internal name of no more than eight characters. s A display name of no more than eighty characters. The name can include spaces. Managing Dependencies Using Oracle Workflow Periodic Loads and Updates 9-15 s A Description of no more than 240 characters. 7. Click Next. The Process page displays. Specify the following information: s An internal name of no more than thirty characters. s A display name of no more than eighty characters. This name can include spaces. s A Description of no more than 240 characters. 8. Click Next. Note: If the Item Type you are defining already exists with the Workflow server, it is overwritten. This includes any modifications such attribute values and process diagram dependencies. Managing Dependencies Using Oracle Workflow 9-16 Oracle Warehouse Builder 3i Users Guide The Finish page displays. Verify the contents of this page. Use the Back button to make changes. 9. Check the Deploy the workflow process to OEM? box if you intend to schedule the workflow process using Oracle Enterprise Manager. The wizard deploys the name of the process to Enterprise Managers Job Library only if the warehouse module is configured for Oracle Enterprise Manager. No other method is available to deploy the process name to the Job Library. 10. Click Finish. The wizard deploys the mappings as a set of Workflow functions defined with the specified Item Type to the Workflow server. You can now start the Oracle Workflow Builder and define the process to initially load the Warehouse. Access Level: The mappings deployed to the Workflow server are assigned an access level of 20. Oracle Workflow Builder can edit these mappings only if it has an access level less than or equal to 20. Managing Dependencies Using Oracle Workflow Periodic Loads and Updates 9-17 Defining the Workflow Process To define a process, open the Oracle Workflow Builder client and set its access level, open the Item Type that contains the deployed mappings, and then define the workflow process. 1. Start Oracle Workflow Builder Oracle Workflow Builder returns its Navigator window. 2. To set the clients access level, Select Help > About Oracle Workflow Builder. The About Oracle Workflow window displays. This window defines the access level. You can change its access level by entering a new value. Each function stored in a Workflow Server has a specific access level between zero and one thousand. The Workflow Builder client can edit a function only if its access level is less than or equal to that of the function. The Workflow Deployment Wizard assigns each function an access level of (20) during the deployment operation. Thus, to edit these functions, you must set the access level of the OWB client to a value less than or equal to twenty. 3. Set the Workflow Builder clients access level to a value less or equal to twenty and Click OK. 4. Select File > Open or click the Open icon. Managing Dependencies Using Oracle Workflow 9-18 Oracle Warehouse Builder 3i Users Guide OWB returns its Open dialog. 5. Select the Database button and then enter connection information for the Oracle Workflow server: s User name and password pair for the server s SQL*Net connect string 6. Click OK. Oracle Workflow Builder returns a list of Item Types in a selection panel. 7. Move the Item Type to the Visible pane and click OK. Managing Dependencies Using Oracle Workflow Periodic Loads and Updates 9-19 The Workflow client returns it Navigator window. 8. Open your load process. Oracle Workflow Builder inserts into its navigation tree entries for the mappings deployed from Oracle Warehouse Builder. 9. Fully expand the navigation tree and open the deployed process. Oracle Workflow returns the process functions in a separate window. Oracle Workflow stacks the functions which represent the individual mappings on top of each other. To define the workflow process, drag each function to a logical place within the window, and then graph the workflow process. 10. Define the dependencies among the functions as a Workflow process. Note: Do not modify OWD_STD_ERROR_PROCESS. This is generated by OWB and there are no user-modifiable parts. Managing Dependencies Using Oracle Workflow 9-20 Oracle Warehouse Builder 3i Users Guide The graph below illustrates the Workflow process that properly sequences the Workflow functions to load the data warehouse. Running the Process You can now run the process immediately using the Oracle Workflow Monitor or schedule the process using Enterprise Manager. If you run the job using the Oracle Workflow Monitor, you must manually start the OWB Workflow Queue Listener. The example in the next section shows you how to run a process immediately using Oracle Workflow Monitor. If you run the job using Oracle Workflow Monitor or Oracle Enterprise Manager, then you can monitor its progress using the Oracle Workflow Monitors Find Process function. Note: When the Oracle Workflow server executes this job: s The functions are processed so that the execution of a function depends on the successful completion of all its predecessors. s The dimension tables are loaded in parallel but the remaining two jobs run sequentially. Managing Dependencies Using Oracle Workflow Periodic Loads and Updates 9-21 After the job completes, normally or abnormally, you can view its audit and log entries using the OWB Runtime Audit Viewer. For more information, see "Viewing the Runtime Audit" on page 9-23. Running the Process Immediately When you run a process immediately using Oracle Workflow Monitor, you must manually start the OWB Workflow Queue Listener. You can start the Queue Listener before or after you start the workflow process. To run a job immediately using Oracle Workflow Monitor: 1. Select Start > Programs > Oracle Warehouse Builder > Oracle Warehouse Builder Workflow Queue Listener The OWB Queue Listener returns its Workflow Login panel which solicits information required to log on to an Oracle Workflow server: Where: 2. Enter the information solicited and then click Logon. The Queue Listener is now waiting for a message which is identified by the Item Type. username/password Log on pair for an Oracle Workflow server Hostname Name of the host for Oracle Workflow server Port Listener port on the Workflow host SID Oracle System Identifier Item Type Internal name of the Oracle Workflow Item Type Managing Dependencies Using Oracle Workflow 9-22 Oracle Warehouse Builder 3i Users Guide 3. Open Oracle Workflow Monitor from your web browser. 4. Select Launch Processes. Workflow Monitor returns the Launch Processes page. 5. Select a specific Item Type to run (GCC Warehouse Load Processes). Workflow Monitor returns the Initiate Workflow page. 6. Select a Process Name from the down-arrow list, enter an Item Key, a User Key, and, if necessary, enter new runtime parameters. 7. Click OK. Oracle Workflow Monitor starts the process and returns its Notifications List page. Choose View Diagram on this page to monitor the process. Viewing the Runtime Audit Periodic Loads and Updates 9-23 Scheduling the Workflow Process You can schedule the process using Oracle Enterprise Manager provided you deployed it to Enterprise Manager s Job Library. To deploy the process, select the Deploy checkbox on the Oracle Workflow Wizards Finish page. There is no other way to deploy a workflow process to OEM. To schedule the process, find its name in the Enterprise Manager s Job Library, modify runtime parameters if necessary, and set up the schedule. Viewing the Runtime Audit Use the OWB Runtime Audit Viewer to view the runtime audit information. This is how you can see specific details of a job after it is run. This information can be useful when you are scheduling jobs. The Audit Viewer displays the contents of the OWB Runtime Library for a specific load or refresh job. For example, you can display the number of records read, number of records inserted or updated, and detailed information about individual records when errors occur. This information might be important to you if you are troubleshooting. You can look for the rows that did not load correctly and you can Viewing the Runtime Audit 9-24 Oracle Warehouse Builder 3i Users Guide Setting the Audit Level The Audit Viewer can only report on audit information that has been stored during the relevant mapping runs. For PL/ SQL mappings, there are four different audit levels: s None: No Task, Detailed Mapping, Error or Error Detail information is available. s Statistics: Task and Detailed Mapping information is available, but Error and Error Details are not. s Error Details (Default): All the information relevant to the Audit Viewer is available. s Complete: This level stores additional information for all rows mapped, but this extra information is not used by the Audit Viewer. The Default Audit Level for a mapping is defined in the Mappings Configuration Parameters, but this value can be overridden in the runtime TCL parameters. Using the OWB Runtime Audit Viewer To use the Runtime Audit Viewer: 1. Select Start > Programs > Oracle Warehouse Builder > Oracle Warehouse Builder Runtime Audit Viewer. The Runtime Audit Logon dialog displays. 2. Specify the runtime connection information and click OK. Viewing the Runtime Audit Periodic Loads and Updates 9-25 The Runtime Audit Viewer opens and displays. The Runtime Audit Viewer window has two panes. The left pane contains a navigation tree. The right pane contains a list of details for the node currently selected in the navigation tree. 3. Use the Navigation tree to select objects in the left pane to display detailed information in the right pane. When the Audit Viewer first opens, all of the objects are rolled up under the Jobs node. As you expand the nodes, you will see the following layers of objects available to view: s Jobs s Job Instances s Tasks s Detailed Mappings "Understanding the Audit Viewer" on page 9-29 describes each of these areas in detail. Viewing the Runtime Audit 9-26 Oracle Warehouse Builder 3i Users Guide Viewing By Date Range By default, only the most recent job instance is displayed in the tree, but this may be controlled using the Date Range facility which can be accessed from either the View menu or the toolbar. There are three options: Option Description Recent (Default) Only the most recent instance of each job is displayed All All instances of each job are displayed Range Only those instances run within a specific date range are displayed Note: The Default Job node only has one job instance. The date associated with this single job instance is the date the warehouse tables were established, so it may disappear from the tree for some values of the Range option. Viewing the Runtime Audit Periodic Loads and Updates 9-27 Searching the Navigation Tree The Find option from the Edit menu or the toolbar can be used to search the navigation tree for objects with a specific name. You can select these options: Clicking the Find button displays a list of objects matching the search criteria. You can then click over one of these entries and then click the Select in Tree button, to select the relevant object in the navigation tree. Option Description Name This is where you type in the name of the object you are searching for. Names of targets are enclosed in double quotes. When entering the name, you can use * to abbreviate the name. For example: "D* will match all names beginning with D, and "D*2 will match all names beginning with D and ending with 2. Object Type Allows you to search all object types, or one specific type. Search In This establishes the scope of the search. Current Folder - searches currently selected object folder in the navigation tree. All Folders searches whole navigation tree. Match Case Allows you to determine whether or not the alphabetic case used in the Name you are searching for is significant. Viewing the Runtime Audit 9-28 Oracle Warehouse Builder 3i Users Guide The Search In option has two settings. If you select Current Folder, the current folder you have selected in the navigation tree will be searched for a object with the name you specified. If the object is found, you may click the Select in Tree button. This will drill you down in the folder to the object you were searching on. If the object is in another folder, and you have Current Folder selected, the object will not be found. Refreshing the Audit Viewer You can refresh the data displayed in the Audit Viewer by selecting View > Refresh. This is useful if you are doing error checking, and you are trouble shooting a job. Purging Runtime Entries You can purge jobs from the runtime tables using the Run Time Audit Viewer. To purge runtime entries select the Purge option from the Job menu or the toolbar. The Audit Viewer displays a pop-up dialog box. You can purge jobs according to Job Date or Job Name. Viewing the Runtime Audit Periodic Loads and Updates 9-29 You should note that as there is only a single job instance entry for Default Job, it is not possible to selectively purge the Default Job entries. Since the date associated with this single entry for Default Job corresponds to the date the runtime tables were first established, you should be careful not to purge all the Default Job audit information accidentally. Understanding the Audit Viewer The Runtime Audit Viewer window has two panes. The left pane contains a navigation tree with objects grouped by object type and rolled up into nodes, or levels of detail. The right pane displays information about the node that is currently selected in the navigation tree. In most cases this lists the objects contained within the object selected with a certain level of information about each object displayed. So, if the selected node in the navigation tree is expanded, the list of objects under that node should be the same as the list of objects displayed in the right pane. The following sections go through and explain each node; what they represent and what kind of information is supported and displayed by Runtime Audit Viewer for each. Node Details Nodes Viewing the Runtime Audit 9-30 Oracle Warehouse Builder 3i Users Guide Viewing the Job Audit The nodes in the navigation tree below the top Jobs node each represent a job. They are listed in alphabetical order. The normal way to set up a mapping job is to register it as a Workflow Process. The corresponding node will appear in the navigation tree after the first run of the Workflow process. When the "Jobs" node at the top of the navigation tree is selected, the right-hand pane displays the list of jobs. This includes DEFAULT_JOB and any other named jobs that have been run. Default Jobs and Named Jobs A mapping can be executed directly from either Warehouse Builder or Oracle Enterprise Manager or it can be used as a component execution unit within an Oracle Workflow process. When a mapping is executed directly, the audit and error information is stored under the category "Default Job". But when a Workflow process is run, the audit and error details for the mappings it invokes are stored under the name of the Workflow process. Viewing the Job Instance Audit Expanding one of these job nodes will cause one or more job instance nodes to be displayed in the navigation tree, each representing a specific instance of its parent Viewing the Runtime Audit Periodic Loads and Updates 9-31 job. Each time a run of the parent job is started, a new job instance is added. The text alongside shows the name of the job and the time at which the run started. Selecting a job node in the tree or selecting a job entry in the right hand pane and clicking the View Instances button displays details about instances of the job. A job instance represents a specific run of the job. The following information is displayed for each job instance: The Date Range option affects the job instance entries displayed here in exactly the same way as the job instances shown in the navigation tree. By default, only the latest job instance of a job is displayed. Column Name Description Job The name of the job. Date The date and time the run started. Status The status of the run. BEGIN indicates the job has started. COMPLETED indicates the job completed. FAILED indicates that the job failed to complete. Errors For a COMPLETED run, this indicates whether any errors occurred in any of the mappings. It is set to either YES or NO. Viewing the Runtime Audit 9-32 Oracle Warehouse Builder 3i Users Guide If the View errors only box is checked, a job instance will not be displayed unless it has the Errors column = YES. Viewing the Task Audit A node representing a job instance can be expanded to display a set of Tasks carried out as part of the run. For example, the execution of a PL/ SQL mapping or an SQL*Loader run will be represented as a Task. If a Workflow Process consists of several tasks, each carrying out a PL/ SQL mapping, a node representing a run of the Workflow Process will have a subordinate Task node for each of those mappings. Selecting a job instance node in the tree or selecting a job instance entry in the right-hand pane and clicking the View Tasks button will display information about the Tasks performed by the job instance. A Task entry represents a run of a mapping. Task entries are set up both for mappings that are run as part of a Workflow process and for those that are run directly. When the job instance represents a run of a Workflow process, there will be a Task entry for each mapping that is carried out by the process. However, when a Note: The Default Job will always have only one job instance. The date corresponds to the date the runtime tables were established, and it will always have Status=BEGIN and Errors= NO, regardless of the status and error details of the Tasks associated with it. Viewing the Runtime Audit Periodic Loads and Updates 9-33 mapping is executed directly, the Task entry will be associated with the single job instance entry for Default Job. If the View errors only box is checked, only those Task entries with errors will be displayed. The following information is displayed for each Task: The statistics values are normally the sum of the corresponding values in the relevant Detailed Mapping entries described later. However, the value for the number of records selected may sometimes be less than the sum of the records selected in the individual Detailed Mapping entries. (For example, this may occur in the row-based modes, if mappings to multiple target tables are combined within a single loop in the generated PL/ SQL.) Viewing the Task Details Audit A node representing a Task can be expanded to display a set of Detailed Mappings. These are also referred to as target entries for that Task. In many cases there will Column Name Description Task The name of the task. Date The date and time the task was started. Type The type of the mapping (e.g. PL/ SQL). Selected The number of rows selected. Inserted The number of rows inserted into the target tables. Updated The number of rows updated in the target tables. Deleted The number of rows deleted in the target tables. Errors The number of errors detected. Status The status of the task. BEGIN indicates the task has started. COMPLETE indicates the task completed. FAILED indicates that the task failed to complete. Viewing the Runtime Audit 9-34 Oracle Warehouse Builder 3i Users Guide only be one Detailed Mapping for a Task, but various scenarios may cause a Task to have more than one Detailed Mapping. Selecting a task node in the tree, or the Targets node below it, or selecting a task entry in the right-hand pane and clicking the View Task Details button will display information about its Detailed Mappings. A Detailed Mapping entry represents a mapping to a specific target table. A Task that affects multiple target tables will have a Detailed Mapping entry for each target table. Also, if a PL/ SQL mapping is run in set based fail over mode, and the set based run detects errors for a specific target table, there will be two Detailed Mapping entries for the table. One will be for the set based run and one for the row based run. Also, if the relevant detailed mapping cannot be started when processing a Task, there will be no Detailed Mapping entry for it. For example, a set-based run cannot be started with certain Loading Types (e.g. DELETE). In these circumstances, a set-based fail over run will immediately switch to the relevant row-based mode, but a pure set-based run will cause the mapping to be abandoned, the Task itself will not be marked as COMPLETE, and it will end up with a reduced or empty list of Detailed Mappings. If the View errors only box is checked, only those Detailed Mapping entries with errors will be displayed. Viewing the Runtime Audit Periodic Loads and Updates 9-35 The following details are displayed for each Detailed Mapping: Note that for PL/ SQL mappings, the values of the statistics reported may depend to some extent on the mode in which the mapping is run; for example in: s Set-based mode, if any errors are found, normally only the first error detected will be logged, and the number of errors will be set to 1. s Row-based modes, multiple errors may be logged. Column Name Description Detail Mapping Identifies the detailed mapping. For PL/ SQL mappings the form depends on the mode in which the detailed mapping was run: s For set-based mode, it is the name of the mapping followed by _Bat. s For row-based mode, it is a character-string ending with _ p. s For row-based (target) mode, it is a character-string ending with _t. In row-based and row-based (target) modes, the beginning of the character-string corresponds to the name of a significant object in the mapping.) Status The status of the detailed mapping: BEGIN indicates the detailed mapping has started. END indicates the detailed mapping ended. Selected The number of rows selected. Inserted The number of rows inserted into the target table. Updated The number of rows updated in the target table. Deleted The number of rows deleted in the target table. Errors The number of errors detected. Discarded The number of records discarded (during a SQL*Loader run). Elapsed The elapsed time of the detailed mapping in seconds. Source Identifies the source of the mapping. (For a PL/ SQL mapping, this may be "table-name" or "table-name"@"schema-name", or a list of these. For a SQL*Loader run, it identifies the input file.) Target Identifies the target of the mapping (e.g. a target table). Viewing the Runtime Audit 9-36 Oracle Warehouse Builder 3i Users Guide s Set-based mode, the value for number of records selected is always the same as the number of records inserted in the target table. If any errors are detected, the transaction is rolled back, and the number of records selected and the number of records inserted will both be zero. s Row-based (target) mode, the value for the number of records selected is derived from a cursor that implements all the stages of a mapping. Thus the value normally corresponds to the number of rows that will be applied to the target table. s Row-based mode, the cursor used to derive the number of records selected may omit the final stage of the mapping in some circumstances. For example, if the final stage involves a splitter, the number of records selected will reflect the records selected as input to the split operation, and so its value may be larger in row-based mode than in set-based or row-based (target) modes. Viewing the Errors When the Detailed Mappings are displayed, clicking on the View Errors button will display a list of the errors that were detected while running the Detailed Mapping that was selected. The list of Detailed Mappings will be replaced by this list of errors. The error entries identify the target table, the column, the error number and the error message. However, in many cases the column is set to * as the identification of the problem column is not readily available. Viewing the Runtime Audit Periodic Loads and Updates 9-37 If an error occurs in a set-based mapping, there will normally be a single error message of the form: error signaled in parallel query server P000 Further details of the cause of the error can be obtained by selecting this entry and clicking the View Error Details button. This error message indicates that the set-based mapping has been abandoned and the transaction has been rolled back. If the mapping is being run in set based fail over mode, the relevant row-based mapping will be run. (This should record the error that caused the set-based run to fail, and possibly others as well.) Understanding Errors When you look at the errors, it is important to understand that an error does not always mean that the data did not load correctly. And conversely a message stating "Rows are inserted" does not mean all is fine. Rows can be inserted but they may violate foreign key constraints. For example, if a row violates a foreign key constraint, an error is logged by OWB in the runtime audit and you can see the error through the audit viewer. The violating row is still inserted, but the foreign key constraint cannot be fully validated due to this violation. Therefore, OWB places the foreign key constraint in "enabled novalidate" state. The foreign key constraint is enabled, but it does not reinforce the constraint on rows already in the table, but it will be as normal for future incoming rows. In this way, OWB allows you to load data without forcing the process to fail every time there is a violation. Taking the Next Step 9-38 Oracle Warehouse Builder 3i Users Guide Viewing the Error Details Error Details are obtained by selecting an Error entry and clicking the View Error Details button.This identifies the type of SQL statement in which the error occurred (e.g. BATCH INSERT, SELECT, INSERT, DELETE, etc.) and the error message. For PL/ SQL mappings run in the row-based modes, the error details will also include a list of the values of each of the columns of the row on which the error occurred.
Taking the Next Step This chapter showed you how to set up and schedule jobs that initially load and periodically refresh a data warehouse. Two methods were illustrated that manage job dependencies: the first was manual and the second automated. The manual scheduling used Oracle Enterprise Manager. The automated method used Oracle Workflow to define a process that manages job dependencies. The next chapter covers several administrative topics. Note: The column name displayed is actually the variable name used by the cursor fetch in the generated code. The information displayed in the Audit Viewer uses these variables, and not the actual columns due to the complexity of the generated code. Administration 10-1 10 Administration This chapter shows you how to complete the several administrative tasks using Oracle Warehouse Builder utilities. There are two types of administration discussed; Repository Administration and Reports Administration. The chapter covers the following topics: s Repository Administration s The OWB Metadata Loader s The OWB Transfer Wizard s The Archive/ Restore Utility s Reports Administration s Using the Administration Pages s Registering an OWB Repository s User Access s Registering a Custom Report Repository Administration 10-2 Oracle Warehouse Builder 3i Users Guide Repository Administration Repository Administration covers information relating to the repository and the metadata that is stored within it. This section discusses the OWB Metadata Loader (MDL) as well as the OWB Transfer Wizard and the Archive/ Restore Utility. When you first install or upgrade OWB, you have the option of using the following wizards to help you set up your OWB warehouse: s OWB Repository Assistant sOWB Runtime Assistant s OWB MDL File Upgrade Utility You can either run these at installation or upgrade, or defer to a later time. Please refer to the OWB Installation Guide for specific instructions on how to use each of these features. The OWB Metadata Loader The OWB Metadata Loader (MDL) allows you to export objects from an OWB repository to a file and also import objects from a file to an OWB repository. This is a very useful way to move the metadata into a different repository. The MDL supports the exporting and importing of different type of objects such as tables, facts, dimensions, materialized views mapping, transformation categories, transformations, flat files, sequences and any other pertinent information belonging to that object including columns, constraints, parameters, and named attribute sets. You can export an entire project or just a subset of objects from a project. You can also execute the export and import utilities using a command line in an MS-DOS window. Using the MDL With Multiple Users Starting in version 3i, OWB gives you the ability to have multiple users accessing the same repository at the same time. It does this by applying locks to objects that are being modified. These locks help to maintain work done on the repository. This feature primarily affects metadata import. When you import metadata, OWB applies locks to all individual objects you are importing. When the import is complete, the changes are committed. Using the MDL with Large Files Large MDL exports and imports may require a large amount of memory. For large exports and imports, if the default -Xmx of 256M in exp.bat, imp.bat, or owbclient.bat is not enough, MDL will fail due to lack of memory. You need to increase the -Xmx setting in the appropriate bat file to fix this problem. The OWB Metadata Loader Administration 10-3 When importing a large MDL file that will result in a large number of updates and/ or adds, OWB may not have the capacity to lock all of the objects affected depending on the OWB repository configuration. In this case, OWB will attempt to switch to single user mode. Also, if more than 300 locks or 1/ 6 of the total enqueue resources for the OWB repository, whichever is less, are required to import a MDL data file, OWB will attempt to automatically switch to single user mode. This allows the MDL to avoid performance degradation when using a large number of locks. This also helps limit the potential of running out of enqueue resources. If the MDL switches to single user mode, no other users will be able to log on to the repository until after the MDL import completes. Whenever possible, large MDL exports and imports should be run on the database server where the OWB repository resides in order to get the best performance. This following sections covers these topics: s Metadata Export s Metadata Import s Metadata Loader Command Line Utility Metadata Export The Metadata Export Utility exports objects from an OWB repository to an operating system file. The Utility can export an entire project, a set of modules, or a set of objects defined by a module. It can also export the global-shared transformation library. The Export File The export file is a delimited character file which can be read with a text editor. Each record within the file begins with a keyword followed by one or more variable-length fields separated by a pipe ( | ) or caret ( ^ ). The separator character is specified when the file is exported. The pipe ( | ) is the default separator character. Notes: sTo avoid potential loss of work, large imports are best done when usage is at a minimum. The OWB Metadata Loader 10-4 Oracle Warehouse Builder 3i Users Guide Example 101 Sample Records from an Export File #Project data <PhysicalName> <LogicalName> <UniversalID> <Version Label> PROJECT|WarehouseName|Warehouse Name|A86184D5336911D58E9000B0D02A59E4|null #Dimension <PhysicalName> <LogicalName> <UniversalID> <Prefix> <UsageType> <Imported> <Generated> DIMENSION|Channels|Channels Dimension Data Mart|7E727655029911D58DC900C04F48E9ED|ch|null|N|N Subsets of Objects You can export all of the objects in a project, or just a subset of those objects. For example, you can export the entire Warehouse project to a file, a few selected modules, or selected objects within a source or warehouse module. When you export a subset of objects, the Utility exports definitions for each object selected and all of that objects ancestors. For example, if you export only the dimension, the export file will contain definitions for: s The dimension s The dimensions parent (The warehouse module) s The warehouse modules parent (The warehouse project) Table 101 lists the repository objects that can be exported and imported by the Metadata Loader. Note: If you are exporting a subset of objects, make sure that you export all referenced objects as well. The Metadata Import Utility allows you to import repository objects even if that objects references to other repository objects cannot be satisfied. Table 101 Repository Objects Projects Modules Module Objects OWB Project Source Files Sequences Tables Views The OWB Metadata Loader Administration 10-5 Exported Values for Configuration Parameters By default, the Metadata Export Utility always exports values for configuration parameters. You can override this behavior when you run the Metadata Export Utility directly from the command line by including the statement CONFIGPARAM=N in the Metadata Export Utilitys parameter file. For more information see "Metadata Loader Command Line Utility" on page 10-17. Exporting Metadata 1. Select the name of the object(s) you want to export. To export a project: 1. Select the Projects name from the Project Tree view. 2. Select Project > Metadata Export > File To export a module or set of modules from a project: 1. Select the names of the module(s) to be exported from the Project Tree view. To select multiple modules, press CTRL and select the additional modules. Warehouse Business Areas Dimensions Facts Mappings Materialized Views Sequences Tables Transformation Categories Transformations Views Table 101 Repository Objects Projects Modules Module Objects The OWB Metadata Loader 10-6 Oracle Warehouse Builder 3i Users Guide 2. Select Project > Metadata Export > File To export a subset of objects in a module: 1. Select a module from the Project Tree view and double-click to open the Module Editor. 2. Select the names of the objects or object types to be exported. To select multiple objects, press CTRL and select another object. 3. Select Module > Metadata Export 2. If you have made any changes, the Metadata Export Confirmation dialog displays. Click Commit to save any changes or Rollback to ignore changes and revert to previously saved version. The OWB Metadata Loader Administration 10-7 The Metadata Export Utility window displays. This window displays the name of the objects you are exporting. 3. Specify the following: s File Name - Enter the name of the export file to be created. Use the Browse button to assist you in locating a directory or file. s Log File - Enter the name of the log file to be created. The MDL writes diagnostic and statistical information to the log file. Use the Browse button to assist you in locating a directory or file. s Field Separator - Select a field separator. The MDL uses the field separator to separate the object and its attributes as shown in Example 101. The default is the pipe symbol (| ) but you can change it to a caret symbol (^ ) using the down-arrow list. s Character Set - Select the type of character set you want used to create the export file. The default character set is defined by the machine that hosts the OWB client. Use the down-arrow list to change the output character set. 4. Click Export. The OWB Metadata Loader 10-8 Oracle Warehouse Builder 3i Users Guide The Metadata Export Progress panel displays the progress. When the export completes, the Metadata Export Results dialog displays. This dialog displays the object types and the number of each type that were exported. For a more detailed look at the export process, click View Log File. This displays the entire log file. Metadata Import The Metadata Loader imports objects from an export file into an OWB repository. The Utility operates in one of four modes: add, replace, add and replace, and add and merge. Import Searching When you use the MDL Import Utility, it searches the repository for any repository objects that exist in the repository and in the file you are importing. The MDL allows three different way for searching for objects: Universal Identifiers, Logical Names, and Physical Names. s Universal Identifiers - The Utility exports a universal identifier, also known as the Universal Object IDentifier or UniversalID, for each object it exports. These identifiers can be used to determine whether an object should be created, replaced, or merged during an import operation. The OWB Metadata Loader Administration 10-9 s Logical and Physical Names - The Utility exports each objects physical and logical name to the export file. These names determine whether an object should be created, replaced, or merged during an import operation. An example best illustrates the search algorithm. Suppose the search is by the logical name of a repository object in the export file. The Import Utility searches the repository for the objects logical name. If the Import Utility does not find the logical name, the resulting actions are based on the import mode. If the Import Utility does find the logical name, the following actions result based on the import mode: If an object cannot be created, replaced, or merged due to a name conflict with another object, an error is written to the import log file. The Import Utility uses the same algorithm for the other case: just exchange the roles of logical and physical in the above steps. Import Mode Result Add Mode A new object will be created if physical name does not conflict with an already existing object. Replace Mode A warning message is written to the log file that the object could not be found to replace and the object is skipped. Add and Replace Mode A new object will be created if physical name does not conflict with an already existing object. Add and Merge Mode A new object will be created if physical name does not conflict with an already existing object. Import Mode Result Add Mode A message is written to the log file that the object already exists and the object is skipped. Replace Mode The object will be replaced if physical name does not conflict with an already existing object. Add and Replace Mode The object will be replaced if physical name does not conflict with an already existing object. Add and Merge Mode The object will be merged if physical name does not conflict with an already existing object. The OWB Metadata Loader 10-10 Oracle Warehouse Builder 3i Users Guide When importing in Add and Replace, and Replace mode, the import completely replaces the existing objects children so that the final object is exactly the same as the source object. Specifically, any children of a repository object that are not replaced or added are deleted. This occurs regardless of whether a child occurs in a mapping or is a foreign, primary, or unique key column. For example, in the MDL export file, Table CUST contains 3 columns with physical name Last_Name, First_Name and Middle_Init. In the OWB repository, the same table CUST contains 4 columns with physical name Last_Name, First_Name, Status and SSN. During a replace operation, the columns Last_Name and First_Name will be replaced, column Middle_Init will be added, and column Status and SSN will be deleted. The final result is that Table CUST in the OWB repository contains the same metadata from Table CUST in the export file. The ultimate consequence is that after the import operation, mappings and foreign, primary, and unique key constraints can be malformed. For example, after the import a mapping may reference non-existent columns. Note: A Replace operation can lead to constraints and mapping definitions that are malformed. Table 102 Examples of Repository Objects and their Children Repository Object Children Tables, Views, Materialized Views Columns, Foreign Keys, Unique Keys, Primary Keys, Check Constraints, Named Attributes, Configurations Dimensions Levels, Level Attributes, Hierarchies, Level Relationships, Columns, Foreign Keys, Unique Keys, Primary Keys, Check Constraints, Named Attributes, Configurations Facts Measures, Columns, Fact Foreign Keys, Segmented Unique Keys, Foreign Keys, Unique Keys, Primary Keys, Check Constraints, Named Attributes, Configurations Transformations Parameters, Implementations, Configurations Files Records, Fields, Configurations Business Areas Business Area Relationships The OWB Metadata Loader Administration 10-11 Importing Metadata To import objects from an export file: 1. Select Administration > MetaData Import from the Administration Tree view. If you have made any changes, the Metadata Import Confirmation dialog displays. Click Commit to save any changes or Rollback to ignore changes and revert to previously saved version. The MetaData Import Utility window displays. 2. Specify the following: File Name - Type in or browse to the export file you want to import. Log File - Create a log file for the import. OWB reads and processes the exported metadata and writes status and diagnostic information in the log file. The OWB Metadata Loader 10-12 Oracle Warehouse Builder 3i Users Guide Import Option s Add new metadata only - This option allows you to add new objects to a repository. If you import a file that contains objects that already exist in your repository, they are ignored. The old objects remain unchanged. s Add new metadata and replace existing objects - This option allows you to add new objects to a repository and update existing objects. s Add new metadata and merge existing objects - This option allows you to add new objects and merge existing objects in your repository. This allows you to reconcile any changes you have made to your repository with any changes or additions that have been elsewhere and then exported to an MDL data file. s Replace existing objects only - This option allows you to replace existing objects in your repository. Search by s Ignore Universal Identifier - This option allows you to ignore searching for the objects you are importing using Universal Identifiers. By default, the Universal Identifiers are used to determine if an object already exists or not. s Logical Names - Select this option to search your repository using the logical names of the objects you are importing to make sure the objects do not already exist. s Physical Names - Select this option to search your repository using the physical names of the objects you are importing to make sure the objects do not already exist Note: Beginning in version 3i, OWB creates universal object identifiers (UniversalIDs) for each object in the repository. MDL imports run in merge mode must use UniversalID for the search criteria in order to merge into existing mappings. If Ignore Universal Identifier is checked, any mappings from the MDL file that already exist in the target repository will be skipped. Also, if the mapping in the MDL file does not have a Universal Identifier, the mapping cannot be merged into a mapping that matches by name. The OWB Metadata Loader Administration 10-13 Character Set - Select the type of character set used to create the export file. The default character set is defined by the machine that hosts the OWB client. Use the down-arrow list to change the output character set. 3. Click Scan to display the exported metadata header information and a summary of the total number of object types contained in the exported metadata file. 4. Click Import. The Metadata Import Confirmation dialog displays only if the exported metadata data information has not been reviewed. 5. Click Import to continue. The OWB Metadata Loader 10-14 Oracle Warehouse Builder 3i Users Guide The Metadata Import Progress panel displays. When the import is complete, the Metadata Import Results dialog displays. This dialog displays the object types and the number of each type that were imported or skipped. For a more detailed look at the import process, click View Log File. This displays the entire log file. The Log File There are three types of status messages that can show up in the log file: s Informational - This is intended to provide information about the import or export. s Warning - This is intended to caution you about the import or export of an object. s Error - This is intended to indicate that the MDL export or import was aborted and did not complete successfully. The OWB Metadata Loader Administration 10-15 Example 102 displays the contents of an import log file. Notice that the import statistics display the total number of objects that have been added, replaced, and skipped. Example 102 Sample Import Log File Import started at 04/25/2001 4:59:46 PM ******************************************************************************** * Import for OWB Release: 3.0.0.0.0 Version: 3.0.0.3.0 * User: user30_3i Connect String: epaglina-pc:1521:ora8i * Data File: d:\owb3000\sco_dim_time_phy_m_tgt.mdl * Log File: d:\owb3000\imp_dim_time_phy_m_tgt.log * Trace: B * Trace File: d:\owb3000\imp_dim_time_phy_m_tgt.trc * Physical Names: Y Mode: CREATE * Ignore Universal Identifier: Y Commit At End: Y ******************************************************************************** Informational at line 15: MDL-1207 PROJECT with physical name <PRJ_Dimension> not imported because it already exists. Informational at line 21: MDL-1207 DATAWAREHOUSE with physical name <WH> not imported because it already exists. Informational: MDL-1134 COMMIT issued at end of import data file. Counts for OWB Import Utility ----------------------------- Total Projects Processed by Import = 1 ------------------------------- Project = PRJ_Dimension Entity in Project Added Replaced Skipped ------------------------------- ----- -------- ------- DATAWAREHOUSE: 0 0 1 DIMENSION: 1 0 0 LEVEL: 3 0 0 HIERARCHY: 2 0 0 LEVELRELATIONSHIP: 4 0 0 COLUMN: 18 0 0 UNIQUEKEY: 8 0 0 PRIMARYKEY: 2 0 0 CONFIGPARAM: 60 0 60 CHILDCONFIG: 14 0 0 Import ended at 04/25/2001 4:59:52 PM The OWB Metadata Loader 10-16 Oracle Warehouse Builder 3i Users Guide Detailed Error Messages If you are running an MDL Import or Export and get an error, a brief error message displays. You can click Detail to display a detailed error log that will pinpoint the repository object and the object line that the error occurred in. This is extremely useful for ad hoc troubleshooting. In the example below, the repository object is the dimension CUST and the object is the level TOTAL. Standard Error Message Detailed Error Message The OWB Metadata Loader Administration 10-17 Metadata Loader Command Line Utility For a greater degree of control, flexibility and maintainability, you can execute the Metadata Loader Utilities from a MS-DOS window. These two utilities allow you to disable the export of configuration values, vary the separator character within an export file, and maintain directive files for selected export operations. These operations simplify and automate the consolidation or synchronization of metadata in multiple repositories that have a similar project structure. The scripts for executing both the export and import utilities reside in the $OWBHOME\owb\bin\win32 directory for Windows platform and $OWBHOME/owb/bin/solaris for UNIX platforms. Both the export and import utilities are driven by a set of directives. You can specify the directives in three ways: 1. Provide directives as a response to command-line prompts 2. Create a directives file 3. Combination of one and two The following examples show you how to export and import OWB definitions using these utilities with a directive file. Directive Keywords for the Export Utility You can direct the export utility to export a file by answering prompts or by creating a file with a set of directives. Table 103 summaries the keywords used to form export directives. The format for a directive is: Keyword=Value except for the comment line indicator (#). You can use the comment indicator to document your directive scripts: put the indication in the first column of a record and follow it with text. You can also form a directive by following the Keword-Equal sign with the wildcard character (*), which matches any string, or with a list of named objects: Keyword=* Keyword=(value-1, value-2, , k) For example, you can specify a set of tables to be exported as: TABLES=(Customers, Products, Days) The OWB Metadata Loader 10-18 Oracle Warehouse Builder 3i Users Guide The directives can be written in a simple text file. Table 103 Keywords for Export Utility Directives Utility Prompt Keyword Description Username/ passw@ host:port:sid USERID Username, password and connection as a string. USERNAME The user name for accessing OWB repository. PASSWORD The password that matches USERNAME. HOST Host for OWB repository. PORT Port for OWB repository. SID SID for OWB repository. Project Name PROJECT Project name. Wildcard format supported for Project, but if used, no other object type keywords may follow. In order to export shared transformations, use PROJECT=Global Shared. Export File FILE File name for the exported data. Field Separator FIELDSEPARATOR Field separators: | , ^ or ~ use without quotes. Log File LOG File name for the status and statistics of the export. Parameter File PARFILE Parameter file contains keyword directives. CONFIGPARAM Export configuration values (Y/ N). Default is Y. TRACE Debug messages. Options: S - write messages to screen Y - write messages to a file B - write messages to screen and a file TRACEFILE Trace file name. PHYSICALNAMES Use physical names (Y/ N) for lookup of objects to be exported. Default is N. CHARACTERSET Specifies the character set to use for the export data file. MODULES If wildcard or multi-value format used for MODULE, no other object type keywords may follow. If simple format is used, this keyword may appear multiple times, directly followed by keywords for any of its owned object types which may be selected using any format (simple, wildcard, multiple). TABLES VIEWS The OWB Metadata Loader Administration 10-19 Export a Project This example shows you how to export the entire GCCWarehouse project. The operation requires two steps: create the directives file and then execute the Metadata Export Utility (exp). Create the Directives File The directives file is a simple text file that contains a set of directives for the export utility. USERID=GCCWH/GCCWH@dwdoc11-pc:1521:ora816 PROJECT=GCCWarehouse FILE=e:\MDL\GCCWarehouse-exp-JUL01 FIELDSEPARATOR=| LOG=e:\MDL\GCCWarehouse-exp-JUL01-LOG CONFIGPARAM=N Execute the Export Utility The following command invokes the Export Utility and specifies the above directive file: w:\owb\bin\win32>exp parfile=e:\MDL\EXP_Directives Processing ... Export successful. FILES SEQUENCES MATERIALIZEDVIEWS DIMENSIONS FACTS TRANSFORM CATEGORIES For wildcard or multi-value format, no FUNCTIONS keyword may follow. If simple format then this keyword may appear multiple times, directly followed by a FUNCTIONS keyword, which may use any format (simple, wildcard, multiple). FUNCTIONS MAPPINGS HELP Use HELP=Y for a complete list # Comment line used in a parameter file Table 103 Keywords for Export Utility Directives Utility Prompt Keyword Description The OWB Metadata Loader 10-20 Oracle Warehouse Builder 3i Users Guide The objects have now been exported to the file and can be imported into the same or another repository using the Import utility. Directive Keywords for the Import Utility You can direct the import utility to import objects from a file by answering prompts or by creating a file with a set of directives. Table 104 summaries the keywords used to form a directive. The format for each directive is Keyword=value. Table 104 Keywords for Import Utility Directives Utility Prompt Keyword Description Username/ passw@ host:port:sid USERID Username, password and connection as a string. USERNAME The user name for accessing OWB repository. PASSWORD The user password that matches USERNAME. HOST Host for OWB repository. PORT Port for OWB repository. SID SID for OWB repository. Import File FILE File name for the data to be imported. Import Mode MODE CREATE, REPLACE, UPDATE, or INCREMENTALUPDATE Log File LOG File name for the status and statistics of the export. Parameter File PARFILE Parameter file contains keyword directives. CONFIGPARAM Import configuration values (Y/ N). Default is Y. TRACE Debug messages. Options: S - write messages to screen Y - write messages to a file B - write messages to screen and a file TRACEFILE Trace file name. PHYSICALNAMES Use physical names (Y/ N) to lookup objects to be imported. Default is Y. CHARACTERSET Specifies the character set to use for the export data file. HELP Use HELP=Y for a complete list # Comment line used in a parameter file IGNOREUniversalID Ignore (Y/ N) the universal id as the search criteria. Default is N. The OWB Metadata Loader Administration 10-21 If a MODE directive is not included, then the default is CREATE. Import Selected Modules This example shows you how to import a two modules using the Import Utility. The operation requires two steps: create a directive file and then execute the Metadata Import Utility (imp). Create the Directives File The directives file is a simple text file that contains a set of directives for the export utility. USERID=GCCWH/GCCWH@dwdoc11-pc:1521:ora816 FILE=e:\MDL\gccstar-exp LOG=e:\MDL\gccstar-imp-LOG MODE=CREATE CONFIGPARAM=N Execute the Import Utility The following command invokes the Import Utility and specifies the above directive file: w:\owb\bin\win32>imp parfile=e:\MDL\IMP_Directives.txt Processing ... Import successful. Validation Rules Governing Import When you import a set of definitions from exported metadata, the Import utility can update existing definitions in an OWB Project. To increase flexibility, the Import utility was designed to relax the rules governing updates to definitions. These relaxed rules provide greater flexibility but can also have a few side-effects. Mapping Definitions A mapping definition can be updated in a repository that is not identical to the mapping in the exported metadata file if any objects (tables, facts, transformations, etc.) that the mapping references cannot be found in the target repository. A warning message is written to the log file any time this occurs. PRESERVEDESCRIPT ION Preserve the description (Y/ N) of already existing objects if the MDL data file does not have a description for the object. Default is N. SINGLEUSER Request a single user lock (Y/ N) for running the import. Default is N. Table 104 Keywords for Import Utility Directives The OWB Transfer Wizard 10-22 Oracle Warehouse Builder 3i Users Guide Code Generation Before you generate scripts from imported definitions, you should first configure the definitions and then validate them. The validation will trap malformed definitions. For additional information on the configuration, validation and generation of scripts, refer to the discussion and examples in Chapter 8, "Configure, Generate and Deploy". Foreign Key Definitions A foreign key definition can be updated in a repository that is not identical to the foreign key in the exported metadata file if its referenced unique or primary key does not exist in the target repository. A warning message is written to the log file that the foreign key does not contain a referenced key. The OWB Transfer Wizard The Oracle data warehousing technology consists of a comprehensive platform that provides new tools for integrating, documenting, and browsing metadata within the data warehouse. This technology includes services aimed at technical and business users for data lineage, impact analysis, annotation, classification, and reporting. However, sharing of metadata between tools is often difficult because each data warehousing tool has its own representation of metadata. The OWB Transfer Wizard enables you to effectively synchronize, integrate, and use metadata stored in multiple sources and formats. The OWB Transfer Wizard Overview The OWB Transfer Wizard allows you to import metadata from and export metadata to a number of data warehousing tools, such as OWB, Oracle Discoverer, and Oracle Express, CA ERwin, and Powersoft PowerDesigner,. The Transfer Wizard performs two major tasks, as shown in Figure 101: 1. Exporting selected metadata from the OWB repository (the source) via a bridge (for example, the OMG bridge) to a variety of targets. The target can be: s Object Management Group (OMG) file s Oracle Discoverer versions 3.1 and 4i s Oracle Express 2. Importing selected metadata from source tools into the OWB repository (the target) via a bridge. The source can be: s Object Management Group (OMG) file The OWB Transfer Wizard Administration 10-23 s Computer Associates ERwin (3.5.1) s Powersoft PowerDesigner (version 6) The Transfer Wizard creates an intermediate XML file conforming to the XML Metadata Interchange (XMI) standard. This process is transparent when you use the Transfer Wizard. You only need to provide the source and target parameters; the Transfer Wizard performs the exporting, conversion, and downloading tasks. Figure 101 shows an example of the transfer process for exporting metadata from Builder into Oracle Discoverer. Figure 101 OWB Transfer Wizard Model Transfer Considerations Before you transfer metadata between two data warehousing tools, you need to perform tasks within the source tool to ensure that the metadata transfers successfully and displays appropriately in the target tool. For detailed information on transfer considerations for metadata import and export, refer to Appendix A, "OWB Bridges: Transfer Parameters and Considerations". Importing Metadata into Oracle Warehouse Builder After you have prepared your source tools to ensure a successful transfer, you can use the OWB Transfer Wizard to import the metadata. For more information on transfer considerations, refer to Appendix? The OWB Transfer Wizard enables you to import metadata from the following types of sources: s A flat file that conforms with the OMG standard The OWB Transfer Wizard 10-24 Oracle Warehouse Builder 3i Users Guide s Computer Associates ERwin s Powersoft PowerDesigner Launching the OWB Transfer Wizard for Import To launch the Transfer Wizard for an import bridge, follow these steps: 1. Open Oracle Warehouse Builder. 2. Select the Administration mode from the vertical tool bar at the left of the Builder client window. 3. From the Administration menu, select MetaData Import > Bridge. The Oracle Transfer Wizard Welcome window displays, identifying the steps you perform while using the Transfer Wizard. The OWB Transfer Wizard Administration 10-25 4. If you want to display version information about the Transfer Wizard, press the About Oracle WB Transfer Tool... button. For version information about the individual bridges, press the Bridge Versions ... button. 5. Click Ok to close the version information window. 6. Click Next to use the Transfer Wizard to import metadata using the OWB Transfer Wizard. Importing Metadata using the OWB Transfer Wizard This section contains instructions for using the OWB Transfer Wizard to import metadata into OWB: For more information on transfer considerations, refer to Appendix A, "OWB Bridges: Transfer Parameters and Considerations". Note: Each of the Transfer Wizard pages allows you to click the Help button or F1 to display help for the current window. The OWB Transfer Wizard 10-26 Oracle Warehouse Builder 3i Users Guide 1. In the Administration mode, select Administration > MetaData Import > Bridge as described in "Launching the OWB Transfer Wizard for Import" on page 10-29. The OWB Transfer Wizard Welcome window displays. 2. Click Next. The Data Source and Target Identification window, Figure 102, displays. 3. In the From field, identify your metadata source (OMG, CA ERwin, PowerDesigner). In the To field, accept the default (OWB import). 4. Optionally, enter a Description of the metadata to be transferred, as shown in Figure 102. (This description displays in the progress bar during the transfer process). Figure 102 Data Source and Target Identification Window Note: This description is useful when you are performing several transfers (each one starting after the preceding transfer completes); the title on the transfer window identifies the current transfer (see Step 8 on page 10-33). The OWB Transfer Wizard Administration 10-27 5. Click Next. The Data Object/ Filenames and Detailed Transfer Parameters window displays. Figure 103 Transfer Parameter Identification Window The Transfer Parameters window displays a different list of parameters based upon the metadata source you selected in the previous step. For detailed information on the Transfer parameters, refer to Appendix A, "OWB Bridges: Transfer Parameters and Considerations". 6. Click Next. The Confirmation of OWB Transfer window displays. The OWB Transfer Wizard 10-28 Oracle Warehouse Builder 3i Users Guide Figure 104 Summary Window 7. Review your entries. If any are incorrect, click Back to return to the previous screen and make the necessary changes. 8. Click Finish on the Confirmation window to begin the transfer process. The transfer window displays with a status bar. Figure 105 My Data Transfer Progress Panel The title of the transfer window is the description you assigned to the transfer on the Choose Data Source and Target Types window. If you did not provide a description, a title does not display. Note: The transfer can require several minutes or an hour or more to complete, depending on the amount of metadata you transfer. The OWB Transfer Wizard Administration 10-29 9. Do one of the following: s If the transfer completes successfully (100% displays), click Ok. s If a transfer failure message displays, click View Log File and review the log. (You can also view the log of a successful transfer.) s If you determine the cause of the failure from the Information Log, note the data requiring update. Close the log by clicking Ok. On the transfer window, click Return to Wizard and update the erroneous data on the Transfer Parameters window. Then transfer the data again. s If you cannot determine the cause of the failure from the Information log, you can create a Trace log. Close the current log by clicking Ok. On the transfer window, click the Return to Wizard button and change the Log Level to Trace on the Transfer Parameters window. Then transfer the data again. During a successful transfer, the Transfer Wizard creates the output file and stores it in the location you specified. Exporting Metadata from Oracle Warehouse Builder After you have prepared your target tools to ensure a successful transfer, you can use the OWB Transfer Wizard to export the metadata. For more information on transfer considerations, refer to Appendix A, "OWB Bridges: Transfer Parameters and Considerations". The OWB Transfer Wizard enables you to export metadata to the following types of targets: s A file that conforms with the OMG standard s Oracle Discoverer 3.1 s Oracle Express s Oracle Discoverer 4i Launching the OWB Transfer Wizard for Export To launch the Transfer Wizard for an export bridge, follow these steps: Note: To save the log for reference, click Save As to open the Save dialog box. Select the folder where you want to store the log and click Save. The OWB Transfer Wizard 10-30 Oracle Warehouse Builder 3i Users Guide 1. Open Oracle Warehouse Builder. 2. Select the Project mode from the vertical tool bar at the left of the Builder client window. 3. From the Project menu, select MetaData Export > Bridge. The Oracle Transfer Wizard Welcome window displays, identifying the steps you perform while using the Transfer Wizard. The OWB Transfer Wizard Administration 10-31 4. Click Next to use the Transfer Wizard to export metadata from OWB to a target. Exporting Metadata from OWB to a Target This section contains instructions for using the OWB Transfer Wizard to export metadata from an OWB Project to a target. For more information on transfer considerations, refer to Appendix A, "OWB Bridges: Transfer Parameters and Considerations". To transfer metadata from OWB: 1. In Project mode, select Project > MetaData Export > Bridge as described in "Launching the OWB Transfer Wizard for Export" on page 10-29. The Oracle Transfer Wizard Welcome window displays. 2. Click Next. The Data Source and Target Identification window, Figure 106, displays. 3. In the From field, accept the default (OWB Export) and in the To field, select your target (OMG, Discoverer 3.1, Express, or Discoverer 4i) to identify the target for your export. 4. Optionally, enter a Description of the metadata to be transferred, as shown in Figure 106. (This description displays in the progress bar during the transfer process). Note: Each of the Transfer Wizard pages allows you to click the Help button or F1 to display help for the current window. Note: This description is useful when you are performing several transfers (each one starting after the preceding transfer completes), because the title on the transfer window identifies the current transfer. The OWB Transfer Wizard 10-32 Oracle Warehouse Builder 3i Users Guide Figure 106 Source and Target Information Window 5. Click Next. The Data Object/ Filenames and Detailed Transfer Parameters window displays. Figure 107 Transfer Parameter Identification Window The OWB Transfer Wizard Administration 10-33 The Transfer Parameters window table lists parameters that you must enter or select. This window displays a different set of parameters depending on the target you selected in the previous step. For detailed information on transfer parameters for each target type, refer to Appendix A, "OWB Bridges: Transfer Parameters and Considerations". 6. Click Next. The Confirmation of OWB Transfer window displays. Figure 108 Summary Window 7. Review your entries. If any are incorrect, click Back to return to the previous screen and make the necessary changes. 8. Click Finish on the Confirmation window to begin the transfer process. The transfer window displays with a status bar. The OWB Transfer Wizard 10-34 Oracle Warehouse Builder 3i Users Guide Figure 109 Data Transfer Progress Panel The title of the transfer window is the description you assigned to the transfer on the Choose Data Source and Target Types window. If you did not provide a description, a title does not display. 9. Do one of the following: s If the transfer completes successfully, click OK. Figure 1010 Data Transfer Complete Panel s If a transfer failure message displays, click View Log File and review the log. You can also view the log of a successful transfer. s If you determine the cause of the failure from the Information Log, note the data requiring update. Close the log by clicking OK. On the transfer Note: The transfer can require several minutes or an hour or more to complete, depending on the amount of metadata you transfer. Note: To save the log for reference, click Save As to open the Save dialog box. Select the folder where you want to store the log and click Save. The Archive/Restore Utility Administration 10-35 window, click Return to Wizard and update the erroneous data on the Transfer Parameters window. Then transfer the data again. s If you cannot determine the cause of the failure from the Information log, you can create a Trace log. Close the current log by clicking OK. On the transfer window, click Return to Wizard and change the Log Level to Trace on the Transfer Parameters window. Then transfer the data again. During a successful transfer, the Transfer Wizard creates the output file and stores it in the location you specified. The Archive/Restore Utility The Archive/ Restore Utility allows you to save and load versions of your OWB repository metadata. The Archive and Restore utilities initially write to a file system. You can then move files from this file system into a third-party version control tool such as Oracle Repository, ClearCase, or SourceSafe. Use the following sections to assist you with your Archive/ Restore: s Setting up Preferences s Archiving a Project s Restoring a Project OWB Software 3rd Party Oracle Warehouse Builder Microsoft SourceSafe Clearcase Oracle Repository File System Versioned Repository OWB Repository Archive Check In Add Check Out Get Undo Check Out Restore The Archive/Restore Utility 10-36 Oracle Warehouse Builder 3i Users Guide There are two places in OWB where you can set up the version label used in the archive/ restore. The first is in the New Project Wizard. The New Project Wizard contains a step that allows you to define Version Properties. The version label that you set here is the version label that is used when that project is archived. After you have created a project, you can edit the version label by opening the Properties dialog for the Project. Click on the Version Properties tab to modify the projects version label. It is important to note that Archive and Restore are different from Import and Export. The following two tables describe the main differences between these features. Note: You must set up your Archive/ Restore settings in the projects Preferences page before you can archive or restore your project. If you attempt to archive or restore without setting these preferences, you will get an error. Table 105 Differences Between Archive and Export Feature Archive Export Character Set UTF8 User Configured Field Separator | (pipe character) User Configured Read-only Detection Detects and prompts you to re-try Detects and then fails Dump Format MDL MDL Log File Name Generated Generated and User configured Table 106 Differences Between Restore and Import Feature Restore Import Character Set UTF8 User Configured Complete Project Replacement Yes User must first delete project Dump Format MDL MDL The Archive/Restore Utility Administration 10-37 Setting up Preferences You can set up most of the archive and restore specifications using the Preferences dialog from the Administration or Project Windows. Use this dialog to specify labeling options, archive/ restore directory locations and log folders. To set up Archive/ Restore preferences: 1. Select Preferences from the Administration or Project menus depending on what view you are in. The Preferences dialog displays. This is a generic preferences dialog that is not specific to the Archive/ Restore. You must select the Archive/Restore tab if it is not displayed on top. 2. Browse to or create a default root folder for archiving. The archive service creates additional folders. UniversalID Preservation Always User Configured Name Preservation Always User Configured Log File Name Generated Generated and User configured Table 106 Differences Between Restore and Import Feature Restore Import The Archive/Restore Utility 10-38 Oracle Warehouse Builder 3i Users Guide 3. Specify how you want to handle the label. There are three options: 4. Browse to or create a location for your Archive Log Folder. This folder contains the archive log files as they are created. The log files contain detailed information about the archive including how the file was created and the contents of what was archived. The name of each log file is automatically derived from the logical project name which is being archived. All archive log files start with "A" and incorporate the label, date, and time to make them unique. 5. Browse to or create a location for your Restore Log Folder. This folder contains the restore log files as they are created. The log files contain detailed information about the restore including how the file was created and the contents of what was restored. The name of each log file is automatically derived from the logical project name contained in the archive. All restore log files start with "R" and incorporate the label, date, and time to make them unique. 6. Click OK. The preferences have been set. You can now proceed to archiving and restoring your project. Label Option When to Use Generated Path Name Do not include For Source Control Management when the path name never changes. The path is consistent and not dependent on the version label value. Parent Folder For Source Control Management when the label value changes slowly. For example, labels like Development, Alpha, Beta, and Production. Also good for dumps to a file system. The generated path changes with each new version label. File name When dumping files to a file system. The generated path changes with each new version label. This appends the label name as the base part of the terminal file name. For example, if the label is development, the file name will be development.mdl. If there is no label and this mode is selected, the file name will be nolabel.mdl. The Archive/Restore Utility Administration 10-39 Archiving a Project Archiving a project allows you to copy metadata stored within an OWB repository to an external location for the purpose of securing that data at a fixed point in time. OWB provides an Archive Wizard to assist you in this process. To archive a project: 1. Select Archive from the Project menu or from the Administration menu depending on what view you are in. You can also select Archive from the right-click menu when a Project is selected. The Archive Wizard Welcome page displays. This page does not require any settings, however, you can uncheck the Show this page the next time box to have this page skipped during future archives 2. Click Next. The Summary page displays. This page displays a summary of the archive settings prior to running the archive process. If you want to see the details of Note: Before you archive your project, you can update the projects Version Label with its Project Properties dialog. The Archive/Restore Utility 10-40 Oracle Warehouse Builder 3i Users Guide your archive after the archive process is complete, check the Show details dialog following a successful archive box. 3. Click Finish. This begins the archive process. A progress window appears. When the progress bar reaches 100%, the archive process is complete. Note: No changes can be made from the wizard. If you notice an error in the Archive Wizard Summary page, click CANCEL and make the appropriate changes to your Archive/ Restore Preferences before continuing with the archive. The Archive/Restore Utility Administration 10-41 If you checked the Show details dialog following a successful archive box, the Archive Results dialog display. This dialog displays the name of each object type and how many of each were archived. For a more detailed look at the archive process, click View Log File. This displays the entire log file. Restoring a Project Restoring a project allows you to recreate metadata within an OWB repository from an external location. To restore a project: 1. Select Restore from the Administration menu in the Administration View. The Archive/Restore Utility 10-42 Oracle Warehouse Builder 3i Users Guide The Restore Wizard Welcome page displays. This page does not require any settings, however, you can uncheck the Show this page the next time box to have this page skipped during future restore processes. 2. Click Next. The Select Archive page displays. Browse to or type in the Archive File you want to restore. 3. Click Next. The Archive/Restore Utility Administration 10-43 The Summary page displays. This page displays a summary of the restore settings prior to running the restore process. If you want to see the details of your restore after the restore process is complete, check the Show details dialog following a successful restore box. 4. Click Finish. This begins the restore process. A progress window appears. When the progress bar reaches 100%, the restore process is complete. If you checked the Show details dialog following a successful restore box, the Restore Results dialog display. This dialog displays the name of each object type, how many of each were restored, and how many of each were skipped. The Archive/Restore Utility 10-44 Oracle Warehouse Builder 3i Users Guide For a more detailed look at the archive process, click View Log File. This displays the entire log file. Using the Administration Pages Administration 10-45 Reports Administration Reports Administration covers information relating to setting up reporting on your metadata. Since OWB relies on the OWB Browser as the main supported method of reporting on your metadata, this section is devoted to administration tasks that you do directly from the OWB Browser. When you first install or upgrade OWB, you have the option of using the OWB Browser Assistant to set up your OWB Browser. You can either run this wizard during your installation or upgrade process, or defer it to a later time. Please refer to the OWB Installation Guide for specific instructions on how to use this wizard. Before you can set up the Administration features for the OWB Browser you must have Oracle Portal running on your web browser. Using the Administration Pages From the Oracle Portal Home Page, select the Administer Warehouse Builder Browser link from the Warehouse Builder Browser portlet to access the Warehouse Builder Administration pages. Using the Administration Pages 10-46 Oracle Warehouse Builder 3i Users Guide The Warehouse Builder Administration pages can only be accessed if the Oracle Portal username you are logged in as has Full Administrator privileges. Administration Actions Across the top of the Administration page there are administration actions to choose from: s Register an OWB Repository enables you to register a new Warehouse Builder Repository for use with the OWB Browser. For more details, see Registering an OWB Repository on page 10-48. s Register a Custom Report enables you to register a new Custom Report with the browser system. Custom Reports are created as Application Components using the Oracle Portal system. For more details, see Registering a Custom Report on page 10-57. s Purge Stale User Information removes information relating to users which have been deleted from the single sign-on repository. This information will remain unused within the browser system repository until it is removed by using this operation. For more details, see Purging Stale User Information on page 10-57. OWB Browser Resources At the bottom of the Administration Page is a table listing all of the existing configurable resources within the browser system. Various Administration Actions OWB Browser Resources Using the Administration Pages Administration 10-47 operations can be performed on these resources. The different types of resource are listed in Table 107. Table 107 Browser Resources Resource Type Description Available Actions Portlet A portlet provided by the browser system. In this release of the OWB Browser, the portlet is named Launcher Portlet. The portlet provides access to the repository navigation, reporting, favorites, and administration functionality. Access - allows you to grant or revoke repository access privileges to the users. Repository An OWB Repository that is browsable through the browser system. Access - allows you to grant or revoke repository access privileges to the users. Edit - allows you to edit repository properties. Unregister - allows you to unregister the repository. After unregistration, the repository can no longer be browsed using the browser system. You must re-register the repository if you want to browse it again. Role A viewpoint of the data in a Warehouse Builder Repository. You are able to define what users belong to each role. In this release of the OWB Browser, the roles provided are Warehouse Engineer, QA User, and Warehouse User. For more information, see "Understanding Roles" on page 10-54. Access - allows you to assign users and groups to roles. A user can have access to multiple roles. Registering an OWB Repository 10-48 Oracle Warehouse Builder 3i Users Guide Registering an OWB Repository Before you can report on metadata in a repository using the OWB Browser, that repository must be registered. This is important because this is how the link is created to the repository. You can register an OWB Repository by clicking Register an OWB Repository on the Warehouse Builder Administration home page. This takes you to the Register Custom Report This represents a Custom Report on a Warehouse Builder repository object. Role - allows you to assign a report to one or more roles. When the report is assigned to a role, it appears in the appropriate report list for that role. Edit - allows you to edit custom report properties. Unregister - allows you to unregister the repository. After unregistration, the report can no longer be browsed using the browser system. Note: When you go to register an OWB repository, the database link to that repository must already exist. If it does not, you must create one. Skip to Administer Database Links on page 10-51 to create a database link before continuing. Table 107 Browser Resources(Cont.) Resource Type Description Available Actions Registering an OWB Repository Administration 10-49 Repository page where you can register OWB repositories, and administer database links. To Register an OWB Repository: 1. Specify the OWB repository properties using Table 108. Table 108 OWB Repository Properties Field Description Name This field is the user-defined name by which the repository will be known within the browser system. For example, this name is displayed in the navigation pages. Host Name This field is the name of the computer on which the repository is stored. This name must be identical to the name used within the Warehouse Builder client or else it will not be possible to invoke reports for this repository from that client. Note: If you left the host name as the default "LocalHost" when you ran the repository Assistant, you must enter LocalHost here even though IAS may be on a different machine. Registering an OWB Repository 10-50 Oracle Warehouse Builder 3i Users Guide 2. Click Apply to register the repository. You may continue to edit the repository properties by applying your changes. Click OK when you are finished. The repository now displays in the Warehouse Builder Administration home page. Database SID This field is the SID of the database in which the repository is stored. This name must be identical to the name used within the Warehouse Builder client or else it will not be possible to invoke reports from that client. OWB Repository Name This field is the name of the schema in which the repository is stored. This name must be identical to the name used within the Warehouse Builder client or else it will not be possible to invoke reports from that client. Database Link This field is the name of the database link which is used to access the repository. The link must already have been created, using the Administer Database Links page for example. This field must be specified and must refer to a valid link even if the repository is on the same database as the browser system. Description This field is the user-defined descriptive text which appears within the navigation pages for this repository. Table 108 OWB Repository Properties(Cont.) Field Description Registering an OWB Repository Administration 10-51 Administer Database Links You can create, view, edit, or drop database links to OWB repositories from the Administer Database Links page. Each of these actions are described below. To create a database link: 1. Select Create Database Link from the Administer Database Links page. 2. Specify the database link name. 3. Specify the OWB repository user name and password. Registering an OWB Repository 10-52 Oracle Warehouse Builder 3i Users Guide 4. Specify the remote database information. You have two options: s Enter the TNS name for the database, or s Enter the host address, service name, protocol and host port number. 5. Click Apply to connect the link. You may continue to edit the link by applying your changes. Click OK when you are finished. The new link displays on the Administer Database Links page. To view a database link: 1. Select the name of the database link from the Administer Database Links page. The View Database Link page displays with a detailed report on the database link you selected. 2. Click OK when you are finished. The browser takes you back to the Administer Database Links page. To edit a database link: 1. From the Administer Database Links page, select edit for the database link you want to alter. The edit link is under the Actions column. Note: The View Database Link page does not allow you to edit the link. Use the Edit Database Link page to make changes and updates to the link. Registering an OWB Repository Administration 10-53 The Edit Database Link page displays. 2. Make any edits to the database link and click Apply. You may continue to edit the link by applying your changes. Click OK when you are finished. To drop a database link: s From the Administer Database Links page, select drop for the database link you want to drop. The drop link is under the Actions column. The database link is dropped and you are returned to the Administer Database Links page. You cannot undo this action. If the database link has been used to register OWB repositories, unregister the OWB repositories first. Note: Once you drop a database link it is permanently destroyed. You must recreate the link if you want to use it again. User Access 10-54 Oracle Warehouse Builder 3i Users Guide Unregistering a Repository To unregister a Repository: 1. Select the Administer Warehouse Builder Browser link from the Browser home page. The Warehouse Builder Administration page displays with the registered repositories listed in the table at the bottom of the page. 2. Select the repository you want to unregister and click on the unregister link. The repository is unregistered and no longer appears in the list of registered repositories. You can no longer browse it using the Browser. User Access The Warehouse Builder Administration pages allow you define the access level available for each user. You can use these features to determine which users have access to what repositories. You can also maintain the amount of user information that gets saved. The following sections define these features in detail: s Understanding Roles s Granting or Revoking Access s Assigning a Custom Report to a Role s Purging Stale User Information Understanding Roles In order to browse the OWB Repository, you must first choose one of the three pre-defined roles to use. These roles are: s Warehouse Developer - someone that uses OWB to create the warehouse User Access Administration 10-55 s QA User - someone that will test the quality of the warehouse before it is deployed s Warehouse User - someone that will be using the deployed Warehouse to understand the underlying metadata The Administrator can restrict access to these roles to certain users and groups. All of the pre-defined Reports and Navigation pages are available to all roles. When you add custom reports or register repositories, you can choose which Roles they will be available for. For example, you can have one Repository be available to the Warehouse Developer Role, and another Repository available to all Roles. There is one extra feature that is available when browsing as a QA user. Any objects that have failed validation will have an error icon next to them in the Contents tab of the Navigation page.
Granting or Revoking Access Select the access action link for one of the OWB Browser resources to display the Browser Access page. This page allows you to grant or revoke access to browser resources such as repositories and custom reports from browser users or groups. The resource to be accessed is indicated at the top left corner of the page. In the example below, the resource is the Launcher Portlet. A list of the current access QA User Error Icon User Access 10-56 Oracle Warehouse Builder 3i Users Guide status is provided in the table at the bottom of the page. This contains the users or groups that currently have access to this resource. s To revoke an access right, click the revoke action link in the appropriate table row from the Change Access section of the page. s To grant an access right to a browser user, enter the name of the user in the text box, and then click the grant user button. s To grant an access right to a browser group, enter the name of the group in the text box, and then click the grant group button. s To close this page and return to the main administration page, click Close. Assigning a Custom Report to a Role Assigning a Custom Report to a role enables Custom Reports to appear in the appropriate report list page for that role. The name and subject type of the report is indicated at the top left corner of the page. A list of the available roles is provided in the table at the bottom of the page. s To add access to a given role, click on the add action in the appropriate table row. Registering a Custom Report Administration 10-57 s To remove access from a given role, click on the remove action in the appropriate table row. s To close this page and return to the main administration page, click the close button. Purging Stale User Information When you use the OWB Browser Administration Access facilities to grant access to users and groups, these access rights are stored in the OWB Browser. If you subsequently remove a user or group using the Oracle Portal facilities, you will still see these users in the Access pages of the OWB Browser. Use the Purge Stale Use Information link to remove these users and groups. Registering a Custom Report A Custom Report is an Application Component created using the Oracle Portal facilities. Registration of the report provides the browser system with the information required to invoke the report. In order for the report to be available to the end user, it is necessary after registration to assign it to one or more roles. You can also edit a Custom Report using the same form as registration. For more information about creating Custom Reports, see "Creating Custom Reports" on page 11-26. To register a Custom Report: 1. Select Register a Custom Report from the Warehouse Builder Administration page. The Register a Custom Report page displays. Note: There is no way to undo this action. Once you click Purge Stale User Information, the information is purged. There is no confirmation message displayed. Registering a Custom Report 10-58 Oracle Warehouse Builder 3i Users Guide 2. Use Table 109 to register a Custom Report. 3. Click Apply or OK to complete the registration. Click Cancel to cancel the registration. Table 109 Custom Report Properties Field Description Display Name This is the name of the report which is seen in the reports list page. Type Name This is the name of the data type reported on by this report. Package This is the full name of the PL/SQL package which implements this report. Repository This is the name of the repository containing target objects for this report. 11-1 11 OWB Metadata Reports Metadata reporting allows you to examine the data that is stored in your repository. Builder offers comprehensive metadata reporting using the OWB Public Views and the OWB Browser. The OWB Browser integrates with Oracle Portal and makes it possible for you to create your own metadata portal. The Browser works together with the Public Views to create reports that look at all repository objects and relationships between objects. You can choose to run these reports from the client or from a web browser. You can also use the Public Views to create your own custom reports. In this chapter: s Viewing OWB Reports s Using the OWB Browser s Available OWB Reports s Creating Custom Reports Viewing OWB Reports OWB provides a set of standard metadata reports that come with the product. You can view these reports from the OWB Browser. You can start the Browser from either the OWB Client or Oracle Portal. Note: In order to use the OWB Browser you must have access to an Oracle Portal site with the OWB Browser portlet installed. For more information about installing the OWB Browser portlet, see the Installation Guide. Viewing OWB Reports 11-2 Oracle Warehouse Builder 3i Users Guide Viewing Reports from OWB You can view a report on an item while you are using OWB. To do this you must first set up your OWB Preferences. Once this is complete you can view the reports directly from the client. Setting up Browser Preferences Before you can use the Reports feature from OWB, you must set the Browser Options in the OWB Preferences. To set Browser Options: 1. Select Project > Preferences. The Preferences window for the project displays. 2. Click the Browser tab of the Preferences window. The Browser tab displays. 3. Specify the following information: s Oracle Portal Host Name s Oracle Portal Port Number Viewing OWB Reports 11-3 s Oracle Portal DAD s OWB Browser Schema Name 4. Click OK. The information is stored. Viewing the Reports To view reports: 1. Select the name of an item. You can select a module, or you can open the Module Editor and select any of the items within the module. 2. Select View > Reports. The Oracle Portal site you defined in the Preferences opens and displays the logon. 3. Enter your Internet Application Server (IAS) Single Sign-On username and password the first time you run a report during this session. Viewing OWB Reports 11-4 Oracle Warehouse Builder 3i Users Guide After you are logged on, the OWB Browser displays a list of reports available for the selected item. 4. Click on the view action link for the report you want to view. The report you selected displays in a separate window. In the example below, the Table Summary Report is displayed. 5. To print the report, select the print operation from your web browser s menu or toolbar. Viewing OWB Reports 11-5 Viewing Reports from the OWB Browser You can also view reports from the OWB Browser. This does not require any OWB installations. The OWB Browser is a portlet that you add to any page in Oracle Portal. If you want to use the OWB Browser, you must start by launching your Oracle Portal. From there you can select the OWB Browser functions. To view OWB Reports: 1. Open your standard internet browser and set the URL to your Oracle Portal address. 2. Click Login. The OWB Browser uses a standard Oracle Portal login screen. This is not specific to the OWB Browser. 3. Logon to the Single Sign-On using a user-name and password that has access privileges to the OWB Browser. The Internet Application Server (IAS) Administrator is responsible for providing user accounts. Each user has their own user-id and password. After logging in, your default homepage displays with an OWB Browser Portlet on the page. If you have not already added the OWB Browser portlet to your Viewing OWB Reports 11-6 Oracle Warehouse Builder 3i Users Guide Oracle Portal Home Page, it will not display. You can use the Customize Page option to add it. 4. Select an OWB Repository to work on from the Repository drop-down list in the Warehouse Builder Browser section of the page. The Repository list contains repositories that have been previously setup using the OWB Browser Administration pages by your Warehouse Administrator. 5. Select the role you want to use: s Warehouse Engineer s QA User s Warehouse User OWB Browser Portlet Other Portlets Using the OWB Browser 11-7 Depending on the role, different reports are available. Roles are setup in the OWB Browser Administration pages by your Warehouse Administrator. For more information, see "User Access" on page 10-54. 6. Click Browse. This takes you to the Warehouse Builder Navigation Pages. You can then select and navigate around your repository and find the item that you want to report on. Then select the Reports tab to display the available reports for that item. Select the view link to view the report. Using the OWB Browser The OWB Browser is used for navigating and examining the metadata in your OWB repository and for launching reports on the metadata.When you click the Browse button from the Warehouse Builder Browser Portlet, the Warehouse Builder Navigation Pages for the selected repository display. There are also Warehouse Builder Favorites pages that allow you to browse certain pages and reports that you bookmark while going through the Navigation pages. Browsing Your Repository The Navigation pages consist of the following parts: s Header s Item Definition s Item Details Note: For additional information about Oracle Portal, please refer to the Oracle Portal documentation. Using the OWB Browser 11-8 Oracle Warehouse Builder 3i Users Guide . Header The header defines the type of page you are looking at and has links to other common pages. The header contains the following: The Title The title lets you know the type of page you are looking at. For example, in this case you are looking at a Warehouse Builder Navigation page. There are also Warehouse Builder Administration and Favorites pages. The Home Link This link takes you to your default home page. The Date The date shows you when the page is displayed. This is typically the current date. If you print a report, this can be helpful to keep track of when the report was run. Header Item Definition Item Details Using the OWB Browser 11-9 Add To My Favorites Select this link to add the current Navigation page to your Favorites pages. Use this to bookmark important reports. Browse My Favorites Select this link to view the page containing your favorite Navigations and Reports that you previously book marked. Customize Select this link to go to the Customize Warehouse Builder Navigations Pages page and customize your page display. Logout Select this link to log out of the OWB Browser and Oracle Portal. Item Definition The Item Definition gives you the information you need to identify the item and its type. The item definition contains the following: The Item Name The name of the item that you have navigated to. In the case above, it is the Warehouse Builder Repository. Help/Question Mark The Help link will take you to online help for the Navigation Page The Path The Path shows you the context of the item. It lists the items in the hierarchy and also shows the type of each item. In the example above, Path: Warehouse Builder: Repository, the Repository named Warehouse Builder is the current item. You can click on any of the items in the path to make that the current item. The Role The role that you are logged in as is shown. In this case, it is Warehouse Engineer. Using the OWB Browser 11-10 Oracle Warehouse Builder 3i Users Guide Item Details The Item Details section consists of a Tabbed Table. The Table will show different information depending on which tab is currently selected. The item details contains the following tabs: The Properties Tab The Properties Tab displays property name-value pairs of the current Item. The property names will be different depending on the type of the current item. Table 111 Property Column Values Column Name Value Property Name The name of the property as defined in the OWB Public Views. For more details on the Public views, see Creating Other Custom Reports on page 11-28. Using the OWB Browser 11-11 The Contents Tab The Contents Tab lists the Items that are contained within the current item. For example, if the current item was a table, then the Contents Tab would list its columns, keys and foreign keys. This is the tab that you use when you are drilling down into the OWB Repository (by using the Contents Action on an item in the list). In the diagram below, the current item is the Warehouse Builder repository and the list of items contained in this repository are Admin Project, Global Shared, and My Project projects. Property Value The value of the property taken from the OWB Repository for the current item. Table 112 Contents Column Values Column Name Value Name The name of the item. Type An icon showing a graphical representation of the item type along with the item type name. Table 111 Property Column Values Column Name Value Using the OWB Browser 11-12 Oracle Warehouse Builder 3i Users Guide The Related Tab The Related Tab allows you to view relationships between items. For example, if the current item is a table, then you can use it to identify which tables it is related to with foreign keys. Like the Contents Tab, you can use this Tab to drill down into items in the repository. When you make one of these items the current item, the path will switch to represent the path of this new item. The example below shows the foreign key associations with a table. The cardinality of the association is shown before the name of the related objects, (0-*) symbolizes a zero to many association. Actions A set of actions you can perform on the item. Selecting one of these actions makes that item the current item and moves you to the tab associated with that item. s Properties - selecting this action makes the item on that row the current item and takes you straight to the Properties tab for that item. s Contents - selecting this action makes the item on that row the current item and takes you straight to the Contents tab for that item. s Related - selecting this action makes the item on that row the current item and takes you straight to the Related tab for that item. s Reports - selecting this action makes the item on that row the current item and take you straight to the Reports tab for that item. Note: You can also use the Contents action to select a new item and then switch tabs when the screen has refreshed. The Properties, Related and Reports actions can be thought of as shortcuts. Table 112 Contents Column Values(Cont.) Column Name Value Using the OWB Browser 11-13 Table 113 Related Column Values Column Name Value Name The name of the item. Type An icon showing a graphical representation of the item type along with the item type name. Actions A set of actions you can perform on the item. Selecting one of these actions makes that item the current item and moves you to the tab associated with that item. s Properties - selecting this action makes the item on that row the current item and takes you straight to the Properties tab for that item. s Contents - selecting this action makes the item on that row the current item and takes you straight to the Contents tab for that item. s Related - selecting this action makes the item on that row the current item and takes you straight to the Related tab for that item. s Reports - selecting this action makes the item on that row the current item and take you straight to the Reports tab for that item. You can also use the Contents action to select a new item and then switch tabs when the screen has refreshed. The Properties, Related and Reports actions can be thought of as shortcuts. Using the OWB Browser 11-14 Oracle Warehouse Builder 3i Users Guide The Reports Tab The Reports Tab shows you the available reports for the current item. Each role has its own set of reports available for each item type, so if you are a Warehouse User, you will see a different set of reports from a Warehouse Engineer. Browsing Your Favorites Selecting the Browse My Favorites link allows you to jump directly to your Warehouse Builder Favorites page. Every navigation page and every report has an Add to My Favorites link on the top right corner. When you select this link, a link to the report or Navigation page you are looking at is added to your Favorites list. You can also use the features on these pages to delete items from the list and to change the formatting of the list. The Favorites Page consists of the following: s Header s Filter s Navigation Favorites s Reports Favorites Table 114 Reports Column Values Column Name Value Name The name of the report. Actions Click on View to display the report. Using the OWB Browser 11-15 Header The header defines the type of page you are looking at, in this case Warehouse Builder Favorites, and has links to your default home page, your Favorites customization page, and log out. If you want to customize you Favorites page, select the Customize link. For details about customizing your page, see Customizing Your Favorites Page on page 11-17. Filter You can narrow down the content of the two Favorite Tables by selecting a particular Repository, Role or Item Type. Each of these Items has an All option. Header Filter Navigation Favorites Report Favorites Using the OWB Browser 11-16 Oracle Warehouse Builder 3i Users Guide Navigation Favorites This list contains items that you chose to Add to My Favorites while it was the current item in the Navigation Page. The table contains these columns: Type The Type of the item together with an icon representing the type Name The name of the Item. Selecting the name will take you straight to the Navigation content page of that item. Path The Path shows the fully qualified name of the item (as it would appear in the Path field on the Navigator page). If you find the Page is to cluttered, then this column can be removed using the Favorites Customize Options Page. Actions These Actions allow you to jump to the specific tab in the Navigator for that item. Description You can give a description to a favorite in the Favorites Customize Page. It will be displayed here. Reports Favorites This list contains Items that you chose to Add to My Favorites while it was the current item in a Report Page. The table contains these columns: Type The Type of the item together with an icon representing the type Name The name of the Item. Selecting the name will take you to the Navigator with this item as the current item. Using the OWB Browser 11-17 Report The name of the Report. Selecting the report name takes you straight to the report. Path The Path shows the fully qualified name of the item (as it would appear in the Path field on the Navigator page). If you find the Page is to cluttered, then this column can be removed using the Favorites Options Page. Description You can give a description to a favorite in the Favorites Customize Page. It will be displayed here. Customizing Your Favorites Page To customize your favorites page, click on the Customize link in the Warehouse Builder Favorites header. Customization Actions Options Using the OWB Browser 11-18 Oracle Warehouse Builder 3i Users Guide The Favorites Customize Display Options Page Selecting the Options link on the Favorites Customize Page takes you to the Customize Display Options page. This page allows you to change the way in which your Favorites are laid out on the Favorites Page. This Page contains the following options: Revert to Defaults The display settings will be reverted back to the install defaults. Apply Applies the changes without taking you back to the previous page OK Applies the changes and takes you back to the previous page Table 115 Customization Actions Action Description Edit The Edit action allows you to enter a description for this favorite. The description will appear as a column in the Favorites list. Choose OK or Cancel to return to the Customize Page. Delete From Favorites This action removes this entry from the Favorites table. (It does not remove the item from the Warehouse Builder Repository). Available OWB Reports 11-19 Cancel Ignores any changes you may have made and takes you back to the previous page Show All The Navigation List and the Reports List will contain all entries that you have added Limit List to n Favorites You can choose the maximum number of rows you wish to see in the tables. If there are more items than can be shown, you will see Next and Previous buttons that you can use to see the remaining items Show / Hide Descriptions You can stop the description column from being displayed. Show / Hide Path You can stop the path column from being displayed. The path can be quite long and can clutter the screen. Available OWB Reports The OWB Browser allows you to run a number of pre-built reports that you can use to examine your metadata. There are five types of reports available: s Summary Reports s Detailed Reports s Implementation Reports s Lineage and Impact Analysis Reports s Lineage and Impact Analysis Diagrams Summary Reports Summary Reports are run against a Module. Each Summary Report presents a list of items contained within the Module. Only items of a particular type are displayed. The type of items are determined by the choice of Summary Report. For example, A Table Summary Report will list all Tables in the Module. A Materialized View Summary Report will list all Materialized Views in the Module. There is also some header information that identifies the Module. Selecting the name of an item will take you to the Detailed Report for that Item. Available OWB Reports 11-20 Oracle Warehouse Builder 3i Users Guide These are the available Summary Reports for the Module. Here is a section from a Table Summary Report. Detailed Reports Detailed reports provide comprehensive information about a particular item. For example, a Detailed Table Report will list information about the tabless columns, keys, foreign keys and its physical configuration parameters. The Detailed Reports are: Business Area Summary Report Dimension Summary Report Fact Summary Report File Summary Report Function Library Summary Report Materialized View Summary Report Sequence Summary Report Table Summary Report Transform Map Summary Report View Summary Report Report Purpose Detailed Installation Report contains information on Projects Available OWB Reports 11-21 Detailed Project Report contains information on Modules and Business Areas Detailed Module Report contains information on Facts, Dimensions, Function Libraries, Materialized Views, Sequences, Tables, Views, Transform Maps, Files and physical configuration parameters Detailed Dimension Report contains information on Hierarchies, Levels, Level Attributes and physical configuration parameters Detailed Fact Report contains information on Measures, Dimensions and physical configuration parameters Detailed Table Report contains information on Columns, Keys, Foreign Keys and physical configuration parameters Detailed View Report contains information on Columns, Keys, Foreign Keys and physical configuration parameters Detailed Materialized View Report contains information on Columns, Keys, Foreign Keys, and physical configuration parameters Detailed Sequence Report contains information on Columns and physical configuration parameters Detailed File Report contains information on Records and physical configuration parameters Detailed Record Report contains information on Fields Detailed Business Area Report contains information on items in the Business Area Detailed Function Library Report contains information on Functions and physical configuration parameters Detailed Function Report contains information on Parameters and Function Implementations Detailed Transform Map Report contains information on Object Uses, Map Components, Item Maps and physical configuration parameters Report Purpose Available OWB Reports 11-22 Oracle Warehouse Builder 3i Users Guide Here is a section from a Detailed Transformation Map report. Implementation Reports Implementation Reports can be run on Dimensions and Facts. They provide information on how real items are used to implement abstract items. The Implementation Reports are: Report Purpose Fact Implementation Report provides information on Tables that implement the Fact, Columns that implement the Measures, Foreign Keys that implement the Fact to Dimension Use Dimension Implementation Report provides information on Levels and the tables that implement them, Level Attributes and the Columns that implement them Available OWB Reports 11-23 Lineage and Impact Analysis Reports Lineage and Impact Analysis Reports are available for Facts, Dimensions, Materialized Views, Tables, Views and Records. These reports should not be confused with the Lineage and Impact Analysis Diagrams that are discussed later in this chapter. Impact Analysis Reports Impact Analysis Reports list all items belonging to the subject of the report, that are used as a source of a mapping. The name of the mapping and the name of the item that it is mapped to will also be displayed. The report basically gives a one step impact analysis for all items of an object. For example, if you wanted to list all the columns in a table that are used as a source in any maps, this is the report you would use. Lineage Reports Lineage Reports are similar to Impact Analysis Reports. Rather than reporting the items that are the source of a map, it reports items that are the targets of a map. Here is a section from an Impact Analysis report Available OWB Reports 11-24 Oracle Warehouse Builder 3i Users Guide Lineage and Impact Analysis Diagrams A Lineage Diagram graphically displays all the objects and transformations that are used to make up the subject of the Diagram. Lineage can be performed at either the object level or the item level. At the Object Level, the diagram can contain: Tables, Views, Materialized Views, Dimensions, Facts, Records and Operators. At the item level the diagram can contain: Columns, Measures, Fields, Operator Parameters and Level Attributes. The Lineage Diagram is displayed with the subject on the right side of the screen. An Impact Analysis Diagram is identical except it shows all objects and transformations that might be affected by a change to the subject. The subject is displayed on the left side of the screen. Here is an example of an Impact Analysis Diagram This diagram shows how the Table, F006_SRC, is used as an input to a splitter. The outputs of this splitter feeds into a materialized view, MatView1, and two other views, View3 and View4. Each of these Views are used as inputs to a Join that has an output to the Table F006_TGT. Available OWB Reports 11-25 Here are some things to note on this diagram: s Each object on the diagram has an icon. s Underneath the Icon is the object name, above the icon is the "parent" name. For operators and transformations, the parent is the name of the map that it is on. s If an item appears more than once on a Diagram, a * will be shown in front of the name and the impact analysis from this item will only be shown on one of the duplicate items (the one without the *). See the example above where the Join appears three times, but only the first one is shown linking to the target table. s If you move the mouse over an Icon, a tool tip will appear giving you the fully qualified name of the item. For example, hovering the mouse over F00_SRC in the diagram above shows MyProject.MyWarehouse.F006_SRC. s If you click on an Icon, a new Impact Analysis diagram will be displayed with the selected item as the subject. s If you click on the name underneath the icon (the object name), the Navigator will be displayed with this item as the current item. This has a number of uses. You could use it to inspect the properties of any item on the diagram. You could drill down into the columns of a Table on the diagram and then display an Impact Analysis Diagram of a particular column. You could run an Impact Analysis Report to see which columns of a table are involved in the mapping shown s If you click on the name above the icon (the parent name) the Navigator will be displayed with the parent as the current item. This is very useful if you have an item level impact analysis diagram and now want to switch to an object level impact analysis diagram. just click on the parent name of the subject, choose the Reports tab and select Impact Analysis. s You can change the size of the items on the diagram using the Zoom drop-down. This setting will be maintained for all other Lineage and Impact Analysis diagrams you look at in that session. s You can limit the number of levels shown on a diagram. The diagram above has 4 levels (4 steps between F006_SRC to F006_TGT). If the diagram has been shortened because you are limiting the levels such that some items are not shown, then a warning is displayed on the diagram telling you the diagram is incomplete. You can select All Levels to see the whole diagram. Creating Custom Reports 11-26 Oracle Warehouse Builder 3i Users Guide Creating Custom Reports You can create custom reports on your metadata using the OWB Public Views. These views are designed to assist you in accessing your metadata repository tables. These views support flexible reporting with the goal of allowing you to report on your data definitions, transformations and deployment areas. You can use the OWB Browser or any other reporting tool to report on that data. Creating Custom Reports for the OWB Browser To create a custom report that can be viewed in the OWB Browser, follow these steps in these sections: s Creating a Custom Report in Oracle Portal s Registering the Custom Report with the OWB Browser s Adding a Custom Report to an OWB Browser Role s Viewing the Custom Report from the OWB Browser Creating a Custom Report in Oracle Portal To create a custom report: 1. Logon to Oracle Portal and select the Database Objects tab from the Navigator page. 2. Find the Schema that the OWB Browser is loaded in, and edit the Schema details to ensure that the Application Schema checkbox is selected. 3. Select the Applications tab, and click on the Create New... Application link. 4. Create a new application and click OK. 5. Select the application you just created and select the Create New... Report link. 6. Select the Report from SQL Query link from the page that displays next. 7. Enter the report name and display name and click Next. 8. Enter the SQL query to define the report and click Finish. Creating Custom Reports 11-27
Your queries need to reference a database link to the OWB repository. You can use the link created during the OWB Browser installation and named "default_owb_ link". The SQL query for a report can only call PUBLIC db link or links within the application schema in which a report resides. Reports can reside in a schema other than OWB Browser schema; however it has to be executable by the OWB Browser schema. To grant execution privilege of a portal report, go to Oracle Portal Home Page > Database Objects > Database Schemas > Report Schema > Report Package > Grant Access. For a simple project report which lists the information systems it contains, a suitable query is: select * from all_iv_information_systems@default_owb_link where project_id = :id When run from the OWB Browser Navigation pages, the marker :id will be automatically substituted at runtime by the appropriate value. You should verify that this statement executes successfully from SQL*Plus. Make sure that you are logged on as the user that owns the report. You can get this from the Develop page for the report. When doing this you need to replace the marker :id by a valid project_id. You should also verifying that you can run the report from the Oracle Portal. From the Develop page, select the Customize link, and enter a valid project_id in the edit box labelled "Id" and then click Run Report. Registering the Custom Report with the OWB Browser To register a custom report with the OWB Browser: 1. From the launcher portlet, click the Administer Warehouse Builder Browser link, and then select the Register a Custom Report link. 2. Select the required type name and repository from the drop down lists. 3. Enter a display name for the report. 4. Enter the qualified package name for the report. This has the form <schema>.<package> and can be found in the Develop page for the report. 5. Click OK. The report should appear in the resource list of the administration page. Note: You can customize the appearance of the custom report you are creating by continuing to click Next. You can also select Edit from the list of actions next to the report after the report is created. Creating Custom Reports 11-28 Oracle Warehouse Builder 3i Users Guide Adding a Custom Report to an OWB Browser Role To add a custom report to an OWB Browser role: 1. Click the roles action link from the resource list entry for the custom report. 2. Click the add action link for each role which is required to be able to view the report. Viewing the Custom Report from the OWB Browser To view the custom report from the OWB Browser: 1. Logon to the OWB Browser, select the role that you added the report to and click Browse. 2. Navigate to an instance of the type for which the report was registered and select the Reports tab. 3. The custom report should appear in the reports list. Select the view action link. The report should be displayed for the current instance of the type. Creating Other Custom Reports The OWB Public Views allow you to access the metadata in your repository and your runtime library to create custom reports with any reporting tool you choose. Use the Public Views to create your own custom reports. Use the table below to determine which views you want to report from based on the objects of interest to you. Table 116 lists repository object types and their associated public views, while Table 117 list runtime object types and their associated views. For detailed information on each view, including what objects they each contain, please see Appendix C, "OWB Public View Tables". Table 116 Repository Object Types and Associated Views Object Type Views Folder ALL_IV_INFORMATION_SYSTEMS ALL_IV_INSTALLATIONS ALL_IV_PROJECTS, ALL_IV_SCHEMAS ALL_IV_FILES Project ALL_IV_PROJECTS Module ALL_IV_INFORMATION_SYSTEMS Creating Custom Reports 11-29 Dimension ALL_IV_DIMENSIONS Level ALL_IV_DIM_LEVELS Level Attribute ALL_IV_DIM_LEVEL_ATTRIBUTES Hierarchies ALL_IV_DIM_HIERARCHIES ALL_IV_DIM_HIERARCHY_LEVELS Foreign Keys ALL_IV_DIM_IMPLS ALL_IV_KEYS ALL_IV_FOREIGN_KEYS Unique Keys ALL_IV_DIM_IMPLS ALL_IV_KEYS ALL_IV_FOREIGN_KEYS Key Column ALL_IV_KEY_COLUMN_USES ALL_IV_COLUMNS Configurations ALL_IV_OBJECT_CONFIGURATIONS Fact (Cube) ALL_IV_CUBES Fact Attribute ALL_IV_CUBE_MEASURES Foreign Keys ALL_IV_CUBE_IMPLS ALL_IV_KEYS ALL_IV_FOREIGN_KEYS ALL_IV_CUBE_MEASURE_DIM_USES Unique Keys ALL_IV_DIM_IMPLS ALL_IV_KEYS ALL_IV_FOREIGN_KEYS Key Column ALL_IV_KEY_COLUMN_USES ALL_IV_COLUMNS Configurations ALL_IV_OBJECT_CONFIGURATIONS Table ALL_IV_TABLES Columns ALL_IV_COLUMNS Foreign Keys ALL_IV_FOREIGN_KEYS Unique Keys ALL_IV_KEYS Table 116 Repository Object Types and Associated Views(Cont.) Object Type Views Creating Custom Reports 11-30 Oracle Warehouse Builder 3i Users Guide Key Column ALL_IV_KEY_COLUMN_USES ALL_IV_COLUMNS Configurations Indexes Partitions ALL_IV_OBJECT_CONFIGURATIONS View ALL_IV_VIEWS Columns ALL_IV_COLUMNS Foreign Keys ALL_IV_FOREIGN_KEYS Unique Keys ALL_IV_KEYS Key Column ALL_IV_KEY_COLUMN_USES ALL_IV_COLUMNS Configurations ALL_IV_OBJECT_CONFIGURATIONS Materialized View ALL_IV_VIEWS Columns ALL_IV_COLUMNS Foreign Keys ALL_IV_FOREIGN_KEYS Unique Keys ALL_IV_KEYS Key Column ALL_IV_KEY_COLUMN_USES ALL_IV_COLUMNS Configurations Indexes Partitions ALL_IV_OBJECT_CONFIGURATIONS Sequence ALL_IV_SEQUENCES Column ALL_IV_COLUMNS Configurations ALL_IV_OBJECT_CONFIGURATIONS File ALL_IV_FILES Record ALL_IV_RECORDS Field ALL_IV_FIELDS Unique Key ALL_IV_KEYS Configurations ALL_IV_OBJECT_CONFIGURATIONS Table 116 Repository Object Types and Associated Views(Cont.) Object Type Views Creating Custom Reports 11-31 Business Tree ALL_IV_CLASSIFICATIONS ALL_IV_CLASSIFICATION_ITEMS Business Area Shortcut ALL_IV_CLASSIFICATIONS ALL_IV_CLASSIFICATION_ITEMS Transform Category ALL_IV_FUNCTION_LIBRARIES Transforms ALL_IV_FUNCTIONS Transform ALL_IV_FUNCTIONS Parameters ALL_IV_FUNCTION_PARAMETERS Implementation ALL_IV_FUNCTION_IMPLS Map ALL_IV_XFORM_MAPS Stage ALL_IV_XFORM_MAP_COMPONENTS ALL_IV_XFORM_MAP_PROPERTIES ALL_IV_XFORM_MAP_PARAMETERS Data Source (DS) ALL_IV_TABLES ALL_IV_VIEWS ALL_IV_MATERIALIZED_VIEWS ALL_IV_RECORDS, ALL_IV_SEQUENCES DS Parameter Groups ALL_IV_XFORM_MAP_COMPONENTS DS Parameters ALL_IV_XFORM_MAP_PARAMETERS Data Target (DT) ALL_IV_TABLES ALL_IV_VIEWS ALL_IV_MATERIALIZED_VIEWS ALL_IV_RECORDS, ALL_IV_SEQUENCES DT Parameter Groups ALL_IV_XFORM_MAP_COMPONENTS Operator ALL_IV_XFORM_MAP_COMPONENTS Operator Parameter Groups ALL_IV_XFORM_MAP_PARAMETERS Variable ALL_IV_XFORM_MAP_COMPONENTS Parameter ALL_IV_XFORM_MAP_PARAMETERS Configurations ALL_IV_OBJECT_CONFIGURATIONS Table 116 Repository Object Types and Associated Views(Cont.) Object Type Views Creating Custom Reports 11-32 Oracle Warehouse Builder 3i Users Guide These are the Public Runtime Views: Configuration Usage ALL_IV_OBJECT_CONFIGURATIONS Configuration Parameters Usage ALL_IV_OBJECT_CONFIGURATIONS Configuration Parameter Values ALL_IV_OBJECT_CONFIGURATIONS Line ALL_IV_OBJECT_CONFIGURATIONS Generated Object ALL_IV_OBJECT_CONFIGURATIONS Impact Analysis ALL_IV_ALL_OBJECTS ALL_IV_IMPACT_DEPENDENTS ALL_IV_LINEAGE_DEPENDENTS Table 117 Runtime Objects and Associated Views Object Type Views Process Runs ALL_IV_PROCESS ALL_IV_PROCESS_RUN Audit Detail ALL_IV_MAP_RUN ALL_IV_RUN_TARGET Audit Error Log ALL_IV_RUN_ERROR Table 116 Repository Object Types and Associated Views(Cont.) Object Type Views Reserved Words A-1 A Reserved Words This chapter describes the reserved words that should not be used to name objects in Oracle Warehouse Builder. s Reserved Words Reserved Words A-2 Oracle Warehouse Builder 3i Users Guide Reserved Words The following table contains a listing of reserved words. Do not use these words as physical object names within an Oracle Warehouse Builder Project. Table A1 Oracle Warehouse Builder Reserved Words ABORT ACCEPT ACCESS ADD ALL ALTER AND ANY ARRAY ARRAYLEN AS ASC ASSERT ASSIGN AT AUDIT AUTHORIZ ATION AVG BASE_TABLE BEGIN BETWEEN BINARY_INTEGER BODY BOOLEAN BY CASE CHAR CHAR_BASE CHECK CLOSE CLUSTER CLUSTERS COLAUTH COLUMN COMMENT COMMIT COMPRESS CONNECT CONSTANT CRASH CREATE CURRENT CURRVAL CURSOR DATA_BASE DATABASE DATE DBA DEBUGOFF DEBUGON DECIMAL DECLARE DEFAULT DEFINITION DELAY DELETE DELTA DESC DIGITS DISPOSE DISTINCT DO DROP DUAL ELSE ELSIF END ENTRY EXCEPTION EXCEPTION_INIT EXCLUSIVE EXISTS EXIT FALSE FETCH FILE FLOAT FOR FORM FROM FUNCTION GENERIC GOTO GRANT GROUP HAVING IDENTIFIED IF IMMEDIATE IN INCREMENT INDEX Reserved Words Reserved Words A-3 INDEXES INDICATOR INITIAL INSERT INTEGER INTERFACE INTERSECT INTO IS LEVEL LIKE LIMITED LOCK LONG LOOP MAX MAXEXTEN TS MIN MINUS MLSLABEL MOD MODE MODIFY NATURAL NATURALN NEW NEXTVAL NOAUDIT NOCOMPRE SS NOT NOWAIT NULL NUMBER NUMBER_BASE OF OFFLINE ON ONLINE OPEN OPTION OR ORDER OTHERS OUT PACKAGE PARTITION PCTFREE PLS_INTEGER POSITIVE POSITIVEN PRAGMA PRIOR PRIVATE PRIVILEGES PROCEDURE PUBLIC RAISE RANGE RAW REAL RECORD REF RELEASE REMR RENAME RESOURCE RETURN REVERSE REVOKE ROLLBACK ROW ROWID ROWLABEL ROWNUM ROWS ROWTYPE RUN SAVEPOINT SCHEMA SELECT SEPARATE SESSION SET SIZE SMALLINT SPACE SQL SQLCODE SQLERRM START STATEMENT STDDEV SUBTYPE SUCCESSFUL SUM SYNONYM SYSDATE TABAUTH TABLE TABLES TASK TERMINATE TIME THEN TO TRIGGER TRUE TYPE UID UNION UNIQUE Reserved Words A-4 Oracle Warehouse Builder 3i Users Guide UPDATE USE USER VALIDATE VALUES VARCHAR VARCHAR2 VARIANCE VIEW VIEWS WHEN WHENEVER WHERE WHILE WITH WORK WRITE XOR The Mapper User Interface B-1 B The Mapper User Interface The Mapping Editor The following sections provide detailed descriptions of the user interfaces for the Mapping Editor The Mapping Menu Bar The menu provides access to all features of the mapping editor, including those commonly done by using the mouse (for accessibility purposes). The following menus and menu-items appear on the Menu Bar of the Editor: Table B1 The Mapping Editor Menu Bar Menu Menu Item Description Mapping Open This item launches the general OWB selection dialog and allows the user to open another Mapping Editor. The Mapping Editor B-2 Oracle Warehouse Builder 3i Users Guide Add> Mapping Table, Mapping View, Mapping Materialized View, Mapping Flat File, Mapping Fact, Mapping Dimension, Mapping Sequence Add> Mapping Transformation, Expression, Aggregator, Filter, Sorter, Joiner, Splitter, Deduplicator Add > Constant, Data Generator, External Process, Mapping Input Parameter, Mapping Output Parameter, Pre-Mapping Process, Post-Mapping Process Create an new operator. Validate Validates the Mapping and all operator expressions. Launches Validation Results Dialog Generate > Mapping Intermediate Result Generates the code for the mapping and launches a read-only code-editor to view the results.The generated code is simplified for demonstrating the transformation process performed and is not guaranteed to be deployable. Print Prints the contents of the Mapping Canvas Mapping Properties Launches the Mapping Property Sheet Close Window Closes the Mapping Editor Edit Properties Launches the Property Inspector for the currently selected operator, Attribute Group or Attribute. If no operator is selected, the menu item is disabled. Display Set Launches the Display Set dialog for the currently selected Attribute Group. If no Attribute Group is selected, the menu is disabled. Table B1 The Mapping Editor Menu Bar (Cont.) Menu Menu Item Description The Mapping Editor The Mapper User Interface B-3 Reconcile Inbound Launches the inbound reconciliation dialog for the current selected operator. Reconcile Outbound Launches the Reconcile Outbound dialog for the currently selected operator (if applicable). This can be used on operators containing attribute groups to copy operators. Add/ Remove Launches the Add/ Remove Attribute Group dialog if an operator was selected. Launches the Add Attribute dialog if an attribute group was selected. If no operator or attribute group is selected, the menu item is disabled. Rename Launches the Rename Dialog. If no operator, Attribute Group or Attribute is selected, the menu item is disabled. Delete Deletes the selected operator and mapping lines. If no Operators or mapping lines are selected, the menu item is disabled. Synchronize Synchronizes mapping operator attributes with the corresponding repository object attributes. View Expand Expands the current selected operator. If no operator is selected, the menu is disabled. Expand All Expands all operators on the canvas. Collapse Collapses the currently selected operator. If no operator is selected the menu is disabled. Collapse All Collapses all operators on the canvas. Select Display Set > All Opens a list of available display sets from which to choose. Only enabled (or active) when an Attribute Group of an operator is currently selected . Validation Messages Launches the Validation Messages Screen. Reports Launches the OWB Browser and provides a report on the current Mapping. Zoom > 400% 200% 100% 75% 50% 25% Zooms the Canvas to the selected level. Table B1 The Mapping Editor Menu Bar (Cont.) Menu Menu Item Description The Mapping Editor B-4 Oracle Warehouse Builder 3i Users Guide The Mapping Toolbar You can execute certain commands using the Mapping Toolbar rather than the main menu. The toolbar on the Editor contains the following buttons: Lineage Impact Analysis Window Arrange All Arranges all OWB editors on the users desktop. Dynamic Window List List of currently opened OWB Windows. Help Contents Launches the online help viewer with the "Contents" tab selected. Index Launches the online help viewer with the "Index" tab selected. Search Launches the online help viewer with the "Search" tab selected. Topics Launches the online help viewer with the with the Mapping Editor tab selected. About Launches the OWB About dialog. Table B2 The Mapping Editor Toolbar Button Help Text Type Description Palette Object Palette Button Toggles between hiding and showing the Object Palette. Print Print Mapping Diagram Button Prints a diagram of the Mapping Canvas. Validate Validate Mapping Button Validates the Mapping and launches the Validation Results Dialog. Generate Generates Code Button Generate code for the whole Mapping and launches the Code Editor. Table B1 The Mapping Editor Menu Bar (Cont.) Menu Menu Item Description The Mapping Editor The Mapper User Interface B-5 The Object Palette You can add operators to a mapping by dragging the corresponding icon from the Object Palette. Table B3 describes the mapping objects available on the Object Palette. The table groups the objects according to their object type. Reconcile Inbound Reconcile an Inbound Operator Button Invokes the reconciliation inbound dialog for the current selected operator. Only enabled for operators that can be derived from Repository Objects. Reconcile Outbound Reconcile outbound or Create a new repository object Button Launches the Reconcile Outbound dialog for the currently selected operator (if applicable). This can be used on operators containing attribute groups to copy operators. Properties Mapping Properties Button Launches the Mapping Properties Sheet with the general tab panel in focus. Synchronize Button Synchronizes mapping operator attributes with the corresponding repository object attributes. Help Help Button Launches the online Help Viewer, with the Contents Tab in focus. Table B3 The Object Palette Icons Group Icon/Tool Tip Description Entities Mapping Table, Mapping View, Mapping Materialized View, Mapping Sequence, Mapping Flat File The operator type objects that can be associated to entities in the Repository. Dimensional Dimension, Fact The operator type objects that can be associated to Dimensional Objects in the Repository. Transformations Transformations, Expression The operator type objects that transform data. Table B2 The Mapping Editor Toolbar (Cont.) Button Help Text Type Description The Mapping Editor B-6 Oracle Warehouse Builder 3i Users Guide The Operator Property Inspector You edit the properties of a mapping operator, attribute, or attribute group using the Operator Property Inspector. The content and organization of properties in the Operator Property inspector is different for each operator, attribute, or attribute group; each operator, attribute, or attribute group may have its unique set of properties and organization, but attributes will have a set of common properties (such as datatype, name, etc.). Table B4 describes the Operator Property Inspector. Other Filter, Joiner, Splitter Order, Pseudo Column, Aggregate All other operator Types. Table B4 The Operator Property Inspector Category Name Description Button Find Click the flashlight icon to start a search; enter a search string into the Find field. Use an asterisk (*)as a wild card. Field Find This field only appears after clicking the Find button. Enter a search string for attributes or properties in the name/ value grid. Button Find Next Click this icon to locate the next property matching the search string, going down Button Find Previous Click this icon to locate the previous property matching the search string, going up. Grid Name/ Value Each property tuple consist of a name and value pair. The value cell has a value-editor which include text-fields, numeric spin box and drop-down lists. A property may require Expression Builder to define it. A "[]" button located to the right of the value cellclick the value cell to reveal the button. Clicking this button will launch a custom editor or Expression Builder, depending upon the property. An attribute or property may have a read-only value and the value cell will not accept input. An attribute or property may have child objects. In those cases the name/ value tuple can be expanded and collapses by clicking on the +/ - icon next to its name. Text-area Description The description area shows relevant information for the currently selected name/ value tuple. Table B3 The Object Palette Icons (Cont.) Group Icon/Tool Tip Description The Mapping Editor The Mapper User Interface B-7 Using the Mouse in the Mapping Editor You use the mouse to conduct several mapping activities on the Mapping Canvas. The following sections outline those activities for which you can use a mouse. Left Mouse Click Operations The following actions can be performed using the left mouse button: s Dragging objects from the palette Clicking the left mouse button on an icon on the object palette, followed by a mouse-drag operation will start the creation of an operator or Transformation. s Mapping & Moving Clicking the left mouse button while the pointer is positioned over a node, followed by a mouse-drag operation could start a "Mapping" or "Move" operation. Depending on the position at the node where you start dragging, the mode of the source and target node (expanded, collapsed, minimized), the position you release the mouse-button will determine the type of mapping that takes place. See "Right Mouse-Click Operations" for detailed descriptions of the various mapping types and when they are initiated. s Single-Select Clicking the left mouse button while the pointer is located over an operator, an attribute (group), or an mapping line will cause the object to be selected. s Multi-Select Clicking the Shift button and the left mouse button while the pointer is located over an operator, an attribute, or an mapping line will add the object to the list of currently selected objects. s Undo selection Clicking the left or right mouse button followed by a mouse button release, anywhere on the canvas will cause the previous list of selected objects to be un-selected again. s Double-click Double-clicking for every object on the canvas causes the object to be selected. In addition, double-clicking on operators and transformations on the canvas will do the following: The Mapping Editor B-8 Oracle Warehouse Builder 3i Users Guide Double-clicking on a minimized node, will expand the node. Double-clicking on the header of an expanded Node, will launch the Operator Property Inspector either with the root node selected or the most common property selected, Filter: Filter condition or Join: Join condition for example. Double-clicking on an attribute group, will launch the Operator Property Inspector with the attribute group node opened and selected. Double-clicking on an attribute, will launch the Operator Property Inspector with the attribute node opened and selected. Tool Tips Rolling the mouse pointer over Node an Attribute, an Attribute Group the canvas will show a Tool Tip with information (e.g. non-truncated name, data type, direction, etc.). Note: roll-over for a Node, Attribute Group, Attribute while performing an Mapping will cause the object under the pointer to be high-lighted if the object is a legal target. Right Mouse-Click Operations Clicking your right mouse anywhere on the canvas and immediately releasing the mouse-button without moving the pointer, will cause a popup menu to show. The following Popup menus are available from the Canvas. Table B5 The Mouse Right-Click Menus From Where Menu Item Description Canvas Generate Generates the code for the mapping and launches the Generation Results Dialog. Note, where applicable the system may generate code for multiple strategies. For example, if a step can be implemented in set-based and row-based operating mode, the code viewer will include both modes. Mapping Properties Launches the Mapping Property Sheet Operator Header Operator Properties Launches the Property Inspector for the currently selected operator. The Mapping Editor The Mapper User Interface B-9 Reconcile Inbound.. Launches the reconciliation inbound dialog for the current selected operator. Reconcile Outbound Launches the Reconcile Outbound dialog for the currently selected operator (if applicable). This can be used on operators containing attribute groups to copy operators. Add/ Remove Groups Launches the Add or Remove Attribute Group Dialog . Rename Launches the Rename Dialog. Delete Deletes the currently selected operator. Expand Expands the current selected Operator. Collapse Collapses the currently selected operator. Reports Lineage Impact Analysis Attribute Group Attribute Group Properties Launches the Property Inspector for the currently selected attribute group. Display Set Launches the Display Set dialog for the currently selected Attribute Group. Expand Expands the current selected operator. Collapse Collapses the currently selected operator. Select Display Set > All Opens a list of available display sets from which to choose to appear in the active operator. Table B5 The Mouse Right-Click Menus(Cont.) (Cont.) From Where Menu Item Description The Mapping Editor B-10 Oracle Warehouse Builder 3i Users Guide Keyboard Operations You can navigate the Mapping Canvas using the "Tab" key and the keyboard "Arrow" keys. The "Tab" key will navigate the user to the "Next" object on the canvas and select it. When no objects are selected, the system will select the node which is proximity is closest to the left/ upper corner of the canvas.The order of navigation is determined by the position in which the objects appear on the canvas; the system will follow a "Z" navigation path: Within a selected attribute set, the user can navigate between the attribute (groups), using the up and down keys. When no attribute (group) is selected, the system navigates to the first attribute (group) whichever of the up/ down key is entered. When positioned on an input attribute, the left arrow key will allow the user to navigate to incoming mapping line (there can only be one incoming mapping line per Attribute). The right arrow key is not active Generate Intermediate Results Generates the code for the mapping up to the current operator and opens a read-only code viewer displaying the results. Does not persist the generation results in the Repository.If an input or in/ out group is selected the generator generates code for all upstream operators up to the current selected operator s Attribute Group only. If an output group is selected the generator generates code for all upstream operators and the current selected operator. Add/ Remove Attribute Launches the Add or Remove Attribute Dialog. Rename Launches the Rename Dialog (see Rename Attribute Group above). Attribute Attribute Properties Launches the Property Inspector for the currently selected Attribute. Rename Launches the Rename Dialog (see Rename Attribute above). Table B5 The Mouse Right-Click Menus(Cont.) (Cont.) From Where Menu Item Description Other Dialogs The Mapper User Interface B-11 When you select an object and then press the delete key, the selected object will be deleted from the canvas (and the Mapping). You can delete the following objects from the canvas using the "Del" key: s Operators (but not attribute groups or attributes). You will be prompted to confirm the delete before the delete takes place. s Mapping lines coming from or arriving at an individual Attribute. You cannot delete mapping lines that originate from, or arrive at, an Attribute Group or Header (without Or Header ). Other Dialogs When adding or editing operators, OWB will return dialogs in which you enter information that will define the operator. This section describes those operators. The Add Operator Dialog Table B6 describes the Add Operator Dialog Table B6 The Add Operator Dialog Category Name Description Radio Button Create unbound <operator type> with no attributes This option is only enabled for the following operator types: Tables, Views, Materialized Views Text Field Name Enter a valid name for the operator. Radio Button Create new <operator type> in repository and bind Creates an operator for the mappingusing a Wizard appropriate for the operator being added to the mappingthat inherits its structure (i.e. attribute groups and attributes) from a newly created repository object. Choice Box Parent Object Module Provides a list of available modules in the same project in which a object can be created. For mapping transforms, this gives a list of available transformation categories qualified by its owning modules. For other mapping objects such as mapping tables or mapping transforms, this gives a list of available modules. Radio Button Import <operator type> into repository and bind Creates an operator for the mappingusing the Import Object Wizardthat inherits its structure (i.e. attribute groups and attributes) from an imported repository object. Other Dialogs B-12 Oracle Warehouse Builder 3i Users Guide The Add/Remove Attributes Dialog Table B7 Describes the Add/ Remove Attributes dialog Choice Box Module Provides a list of available modules in the same project in which an object can be imported. By default the module that the Mapping belongs to is selected. Radio Button Select existing <object type> in repository and bind Adds an operator to the mapping that matches the type and structure of the object that is currently selected in the Tree List. Field Find Enter a search string for objects in the Tree List. The search is not case-sensitive, and you may use an asterisk (*) as a wild card character. Button Find The search string will be used to locate the first object in the Tree List that matches the string. Tree List <object type> Lists all the object that matches the type of the operator for all Modules in the same project. Table B7 The Add/Remove Attributes Dialog window Category Name Mandatory Description Text Field Name Yes Enter a name for the new Attribute.By default the old name of the object is displayed Button Add n/ a Adds an attribute to the current list of attributes using the name as specified in the Name text field. Names will be verified for correctness and uniqueness Button Remove n/ a Removes the selected attribute(s) from the list box of current attributes to be added. Button will be disabled if no attribute s are selected Listbox Current list of Attributes n/ a Lists the current list of attribute groups to be added. Table B6 The Add Operator Dialog (Cont.) (Cont.) Category Name Description Other Dialogs The Mapper User Interface B-13 The (Inbound and Outbound) Reconcile Dialog Table B8 describes the (Inbound and Outbound) Reconcile dialog..
Table B8 The (Inbound/Outbound) Reconcile Dialog Category Name Description Dropdown box Object Type Provides a list of repository object types to reconcile; Tables, Views, Materialized Views, Files, Sequences, Transforms Field Find Enter a search string for objects in the Tree List. By default the field contains the bound name of the object if the object is bound. Button Find Locates the first object in the Tree that matches the string when the Find button is clicked; use asterisks (*) as wildcards. Tree List Objects Lists all the objects that matches the object type of the operator for all Modules in the same project. By default the original module and object that the operator was derived from is selected (if available). Checkbox Match by Object Identifier Matches mapping operators to repository objects by their Object Identifier Checkbox Match by physical (bound) Name Matches mapping operators to repository objects by attribute physical (bound) names. Checkbox Match by Position Matches mapping operators to repository objects by the attribute position; line by line matching. The Physical Properties Inspector Table B9 describes the Physical Properties inspector. Table B9 The Physical Properties Inspector Configuration group Description General This group includes properties that apply to all steps, for example the auditing level property. Operators This group has sub-nodes for each operator that was defined in the mapping. Each operator contains a set of physical properties unique to the type of the operator. For example, a table operator has a database link node, a schema node, and a File operator has a file-name node. Other Dialogs B-14 Oracle Warehouse Builder 3i Users Guide Steps This group has sub-nodes for each Mapping Step that the "Analyze" operation (see later) has created. Each step has a set of physical properties that are determined by the type of the step. For example, a PL/ SQL step has different properties than a SQL*Loader step. Note, Steps are automatically created and maintained; you cannot add, delete or rename them. The Physical Properties Inspector Table B9 describes the Physical Properties inspector. Table B9 The Physical Properties Inspector(Cont.) Configuration group Description OWB Public View Tables C-1 C OWB Public View Tables Management Table C1 ALL_IV_INSTALLATIONS Column Name Data Type Description installation_id Number(9) object Id installation_name Varchar2(255) name of the project that business_name Varchar2(4000) display name for the project description Varchar2(4000) installed_version Varchar2(2000) release Varchar2(40) repository_model_version Number(9) public_view_version Char(6) Version Id of the public SQL views updated_on Date The time the object was last modified created_on Date The time this object was created Table C2 ALL_IV_PROJECTS Column Name Data Type Description project_id Number(9) object Id project_name Varchar2(255) name of the project that business_name Varchar2(4000) display name for the project description Varchar2(4000) description version_label Varchar2(255) label representing the projects version Management C-2 Oracle Warehouse Builder 3i Users Guide is_valid Varchar2(7) updated_on Date The time the object was last modified created_on Date The time this object was created Table C3 ALL_IV_INFORMATION_SYSTEMS Column Name Data Type Description project_id Number(9) object id project_name Varchar2(255) name of the project that information_system_id Number(9) Object Id of the information system information_system_name Varchar2(255) Name of the information system business_name Varchar2(4000) display name for the information system description Varchar2(4000) description for the information system product_type Varchar2(255) This is the application product type system_type Varchar2(255) System type version_label Number(9) vendor Varchar2(2000) This is optional. This would return PeopleSoft, Oralcle, SAP etc. database_link Varchar2(40) this helps connect to the physical external data source. For data warehouse information systems this is optional. Typically, if a warehouse has been created based on the design of an external database schema, this would be populated for warehouses as well. integrator_name Varchar2(255) Name of the integrator. For each Information System there is one and only one Integrator specified is_valid Varchar2(7) status Varchar2(40) The status of the information system. OWB supports Production, Development and Quality Assurance updated_on Date The time the object was last modified created_on Date The time this object was created Table C2 ALL_IV_PROJECTS Column Name Data Type Description Management OWB Public View Tables C-3 Data Model Table C4 ALL_IV_SCHEMAS Column Name Data Type Description project_id Number(9) Project Id project_name Varchar2(255) name of the project that the schema is in schema_id Number(9) Identifier for the schema. schema_name Varchar2(255) name of the schema business_name Varchar2(4000) display name for the schema description Varchar2(4000) description is_valid Varchar2(7) information_system_id Number(9) The information system for which this schema belongs to information_system_name Varchar2(255) The name of the information system. updated_on Date The time the object was last modified created_on Date The time this object was created Table C5 ALL_IV_DIMENSIONS Column Name Data Type Description schema_id Number(9) Owner (schema) of the dimension schema_name Varchar2(255) Name of the schema dimension_id Number(9) Object id of the dimension dimension_name Varchar2(255) Name of the dimension business_name Varchar2(4000) display name plural_name Varchar2(40) plural name for the dimension used for reporting purposes description Varchar2(4000) Description for dimension is_valid Varchar2(7) updated_on Date The time the object was last modified created_on Date The time this object was created Management C-4 Oracle Warehouse Builder 3i Users Guide Table C6 ALL_IV_DIM_LEVELS Column Name Data Type Description dimension_id Number(9) Id of the dimension dimension_name Varchar2(255) Name of the dimension level_id Number(9) object id level_name Varchar2(255) Level name. Unique within a dimension business_name Varchar2(4000) display name description Varchar2(4000) Description for level updated_on Date The time the object was last modified created_on Date The time this object was created Table C7 ALL_IV_DIM_LEVEL_ATTRIBUTES Column Name Data Type Description level_id Number(9) Id of the dimension level level_name Varchar2(255) Name of the dimension level attribute_id Number(9) Object id attribute_name Varchar2(255) Unique within a dimension level. business_name Varchar2(4000) display name description Varchar2(4000) Description for dimension attribute position Number(9) position of the level attribute within a level data_type Varchar2(255) Data type of the level attribute length Number(9) length of the attribute precision Number(9) precision of the attribute scale Number(9) scale of the attribute updated_on Date The time the object was last modified created_on Date The time this object was created Table C8 ALL_IV_DIM_HIERARCHIES Column Name Data Type Description dimension_id Number(9) Id of the dimension dimension_name Varchar2(255) Name of the dimension hierarchy_id Number(9) Object Id hierarchy_name Varchar2(255) Hierarchy name Management OWB Public View Tables C-5 business_name Varchar2(4000) display name description Varchar2(4000) Description for the hierarchy updated_on Date The time the object was last modified created_on Date The time this object was created Table C9 ALL_IV_DIM_HIERARCHY_LEVELS Column Name Data Type Description level_use_id Number(9) Object Id that represents a particular use of a level within a hierarchy. hierarchy_id Number(9) Id of the hierarchy hierarchy_name Varchar2(255) Hierarchy name level_id Number(9) Id of the level level_name Varchar2(255) Name of the level position Number Hierarchical position within this hierarchy, position 1 being the most detailed. Table C10 ALL_IV_CUBES Column Name Data Type Description schema_id Number(9) schema of the cube schema_name Varchar2(255) name of the schema cube_id Number(9) object Id of the cube cube_name Varchar2(255) Name of the cube business_name Varchar2(4000) display name description Varchar2(4000) Description for cube is_valid Varchar2(7) updated_on Date The time the object was last modified created_on Date The time this object was created Table C8 ALL_IV_DIM_HIERARCHIES Column Name Data Type Description Management C-6 Oracle Warehouse Builder 3i Users Guide Table C11 ALL_IV_CUBE_MEASURES Column Name Data Type Description cube_id Number(9) Id of the cube cube_name Varchar2(255) Name of the cube measure_id Number(9) object Id of the measure measure_name Varchar2(255) Name of the measure business_name Varchar2(4000) display name description Varchar2(4000) Description for measure position Number(9) position of the measure within the cube data_type Varchar2(255) data type of the measure length Number(9) length of the attribute precision Number(9) precision of the attribute scale Number(9) scale of the attribute updated_on Date The time the object was last modified created_on Date The time this object was created Table C12 ALL_IV_CUBE_MEASURE_DIM_USES Column Name Data Type Description cube_id Number(9) Object Id cube_name Varchar2(255) Name of the cube measure_id Number Id of the measure measure_name Varchar2(255) Name of the measure dimension_id Number Id of the dimension dimension_name Varchar2(255) name of dimension dimension_alias Varchar2(255) alias of dimension Table C13 ALL_IV_TABLES Column Name Data Type Description schema_id Number(9) Owner of the table schema_name Varchar2(255) Name of the schema table_id Number(9) object Id of the table table_name Varchar2(255) Name of the table business_name Varchar2(4000) Logical - business name for this object Management OWB Public View Tables C-7 description Varchar2(4000) description is_valid Varchar2(7) updated_on Date The time the object was last modified created_on Date The time this object was created Table C14 ALL_IV_VIEWS Column Name Data Type Description schema_id Number(9) Owner of the view schema_name Varchar2(255) Name of the schema view_id Number(9) object Id of the view view_name Varchar2(255) name of the view business_name Varchar2(4000) Logical - business name for this object description Varchar2(4000) description is_valid Varchar2(7) updated_on Date The time the object was last modified created_on Date The time this object was created Table C15 ALL_IV_MATERIALIZED_VIEWS Column Name Data Type Description schema_id Number(9) Owner of the view schema_name Varchar2(255) Name of the schema view_id Number(9) object Id of the view view_name Varchar2(255) name of the view business_name Varchar2(4000) Logical - business name for this object description Varchar2(4000) description is_valid Varchar2(7) updated_on Date The time the object was last modified created_on Date The time this object was created Table C13 ALL_IV_TABLES Column Name Data Type Description Management C-8 Oracle Warehouse Builder 3i Users Guide Table C16 ALL_IV_SEQUENCES Column Name Data Type Description schema_id Number(9) Owner of the sequence table schema_name Varchar2(255) Name of the schema sequence_id Number(9) Object Id of the key sequence_name Varchar2(255) Name of the sequence business_name Varchar2(4000) Display name. Note: This will more then likely be equal to the name of the sequence description Varchar2(4000) description is_valid Varchar2(7) updated_on Date The time the object was last modified created_on Date The time this object was created Table C17 ALL_IV_KEYS Column Name Data Type Description entity_id Number(9) Id of the entity that this key is attached to. entity_type Varchar2(4000) entity_name Varchar2(255) Name of the entity that the key is attached to. key_id Number(9) key_name Varchar2(255) Name of the foreign key business_name Varchar2(4000) Display name of the key description Varchar2(4000) description schema_id Number(9) Owner of the constraint schema_name Varchar2(255) Name of the schema updated_on Date The time the object was last modified created_on Date The time this object was created Table C18 ALL_IV_FOREIGN_KEYS Column Name Data Type Description entity_id Number(9) Id of the entity that this key is attached to. entity_type Varchar2(4000) entity_name Varchar2(255) Name of the entity that the key is attached to. Management OWB Public View Tables C-9 foreign_key_name Varchar2(255) Name of the foreign key foreign_key_id Number(9) Object Id business_name Varchar2(4000) Display name of the key description Varchar2(4000) description key_id Number(9) Id of the key that this foreign key is pointing to. key_name Varchar2(255) Name of the key that this foreign key is pointing to schema_id Number(9) Owner of the constraint schema_name Varchar2(255) Name of the schema updated_on Date The time the object was last modified created_on Date The time this object was created Table C19 ALL_IV_KEY_COLUMN_USES Column Name Data Type Description key_id Number(9) Object Id of the key key_type Varchar2(11) key_name Varchar2(255) Name of the foreign key column_id Number(9) Id of the column column_name Varchar2(255) Name of the key column position Number(9) Ordinal position of the key column within the key. Table C20 ALL_IV_COLUMNS Column Name Data Type Description entity_id Number(9) Id of the entity that this key is attached to. entity_type Varchar2(4000) entity_name Varchar2(255) Name of the entity that the key is attached to. column_id Number(9) Object id of the column column_name Varchar2(255) Name of the column business_name Varchar2(4000) Logical - business name for this object description Varchar2(4000) description position Number(9) position of the column within the table Table C18 ALL_IV_FOREIGN_KEYS Column Name Data Type Description Management C-10 Oracle Warehouse Builder 3i Users Guide Implementation Model data_type Varchar2(255) Data type of the column length Number(9) length of the column precision Number(9) precision of the column scale Number(9) scale of the column updated_on Date The time the object was last modified created_on Date The time this object was created Table C21 ALL_IV_DIM_IMPLS Column Name Data Type Description implementation_id Number(9) Object id of the dimension implementation item_id Number(9) Object id of the dimensional item item_type Char(15) Type of item item_name Varchar2(255) Name of the dimensional item dimension_id Number(9) Id of the dimension dimension_name Varchar2(255) Name of the dimension column_id Number(9) Id of the column column_name Varchar2(255) object Id position Number(9) e.g. Position of the level attribute column. Note that Oracle 8 attribute is single column. table_id Number(9) Object Id of the table table_name Varchar2(255) Name of the table that owns the column Table C22 ALL_IV_CUBE_IMPLS Column Name Type Description implementation_id Number(9) Object Id of the cube map NOTE: Exception to the naming convention rules: No map_name column exists. item_id Number(9) Object Id of the cube item item_type Varchar2(18) Type of item item_name Varchar2(255) Name of the cube item Table C20 ALL_IV_COLUMNS Column Name Data Type Description Management OWB Public View Tables C-11 Flat File/Record Model cube_id Number(9) Id of the cube cube_name Varchar2(255) Name of the cube dimension_id Number Id of the dimension that CUBE_DIM_USE is pointing to. dimension_name Varchar2(255) Name of the dimension dimension_alias Varchar2(255) Alias of the dimension column_id Number Id of the column column_name Varchar2(255) object Id position Number e.g. Position of the level attribute column. Note that Oracle 8 attribute is single column. table_id Number(9) Object Id of the table table_name Varchar2(255) Name of the table that owns the column or foreign key foreign_key_id Number Id of the foreign key foreign_key _name Varchar2(255) object Id dim_implementation_id Number Object Id of the dimension map that this foreign key relies on (map dependency from the CWM model. Table C23 ALL_IV_FILES Column Name Type Description information_system_id Number(9) Object Id of the information system information_system_name Varchar2(255) Name of the information system file_id Number(9) Object Id of the file file_name Varchar2(255) Name of the file business_name Varchar2(4000) display name description Varchar2(4000) description is_valid Varchar2(7) Table C22 ALL_IV_CUBE_IMPLS Column Name Type Description Management C-12 Oracle Warehouse Builder 3i Users Guide record_classifier_position Number(9) When a file consists of multiple record types, each row of information read from the file contains a field that encodes the record type of the data row. This points to the position of that field. record_classifier_length Number(9) This returns the length of the field that acts as record classifier. record_size Number(9) This indicates the maximum size of a record for non delimited files n_physical_records_in_logical Number(9) The number of physical records that make up a logical record. This is only used where a fixed number of physical records make up a logical record. Continuation_at_end Char(1) If the continuation string is given, this indicates when the continuation string is found at the end of the physical record. If false the continuation string will be at the start. Continuation_delimiter Varchar2(40) When the number of physical records per logical record is variable, this indicates how a continuation record is identified. Note that this is more like a physical record delimiter when compared to recordDelimiter which is logical. is_self_describing Char(1)) True if fields in the first record contain column names applicable to subsequent records Source from hasFieldNamesInFirstRow (will be added to OWB) is_fixed_width Char(1) Indicates that all records are of the same fixed length. record_delimiter Varchar2(40) The string that delimits a record. If this is not set then the RecordSize should be used to determine the end of a record text_start_delimiter Varchar2(40) The delimiter of a text string in the record such as a quote text_end_delimiter Varchar2(40) updated_on Date The time the object was last modified created_on Date The time this object was created Table C23 ALL_IV_FILES Column Name Type Description Management OWB Public View Tables C-13 Table C24 ALL_IV_RECORDS Column Name Type Description file_id Number(9) Object Id of the file file_name Varchar2(255) Name of the file record_id Number(9) Object id of the record record_name Varchar2(255) Name of the record business_name Varchar2(4000) display name description Varchar2(4000) description updated_on Date The time the object was last modified created_on Date The time this object was created Table C25 ALL_IV_FIELDS Column Name Type Description record_id Number(9) Object id of the record record_name Varchar2(255) Name of the record field_id Number(9) Id for the field field_name Varchar2(255) Name of the field business_name Varchar2(4000) display name description Varchar2(4000) description position Number(9) position of the field within the record data_type Varchar2(255) Data type of the field length Number(9) length of the field precision Number(9) precision of the field scale Number(9) scale of the field picture Varchar2(40) Picture clause sign_type Number(9) The type of sign usage Varchar2(40) How data is stored internally occurs Number(9) structure _id Number(9) This is a foreign key to the parent field (structure) structure_name Varchar2(255) Name of the parent field (structure) updated_on Date The time the object was last modified created_on Date The time this object was created Management C-14 Oracle Warehouse Builder 3i Users Guide Impact Analysis/Data Lineage Model Table C26 ALL_IV_ALL_OBJECTS Column Name Type Description object_id Number(9) Id of the object object_type Varchar2(4000) Type of the object object_name Varchar2(255) Name of the object. business_name Varcahr2(4000) Display name for the object context_name Varcahr2(4000) The fully qualified name for the object. For example a table ORDERS in schema OP in project MyProject would be MyProject.OP.ORDERS description Varchar2(4000) description parent_object_id Number Object id of the parent parent_object_type Varchar2(4000) Type of the parent object parent_object_name Varchar2(4000) Name of the parent object. For example, Table is considered to be parent of Column, Dimension is considered to be parent of Level, and Level Attribute, Cube is considered to be parent of Measures, EntityMap is considered to be parent of ItemMaps, etc. is_valid Varchar2(7) updated_on Date The time the object was last modified created_on Date The time this object was created Table C27 ALL_IV_IMPACT_DEPENDENTS Column Name Type Description object_id Number Object id of the object whose dependents are listed object_type Varchar2(4000) Type of the object object_name Varchar2(255) Name of the object. business_name Varchar2(4000) display name of the object context_name Varcahr2(4000) The fully qualified name for the object. For example a table ORDERS in schema OP in project MyProject would be MyProject.OP.ORDERS description Varchar2(4000) description parent_object_id Number Object id of the parent parent_object_type Varchar2(4000) Type of the parent object Management OWB Public View Tables C-15 parent_object_name Varchar2(4000) Name of the parent object. For example, Table is considered to be parent of Column, Dimension is considered to be parent of Level, and Level Attribute, Cube is considered to be parent of Measures, EntityMap is considered to be parent of ItemMaps, etc. dep_object_id Number Id of the dependent object dep_object_type Varchar2(4000) type of the dependent object dep_object_name Varchar2(255) Name of the dependent object dep_object_business_ name Varchar2(4000) Display name for the dependent object dep_object_context_ name Varchar2(4000) Display name for the dependent object dep_expression Varchar2(4000) If a dependent object has an expression, it will be returned here. For example, if a column x is the result of a function h=g+1 and x is wired to h, when you get the dependents of x, you will get h together with the h=g+1 expression. In addition, if the dependent object is an item (e.g.column), where an entity level expression (such as a filter or join condition on the stage component) applies, this expression will also be returned with each item (e.g. stage component parameter) that is aggregated by that entity (e.g. stage component) -Suggestion: that the expressions coming from the entities are prefixed by the type of expression. for example: FILTER:d>1. This way, we can differentiate what immediately applies to the item and what comes from the parent entity. -Strong Recommendation: Replace %1 %2 with the actual argument names. dep_object_parent_id Number Id of the dependent object dep_object_parent_type Varchar2(4000) type of the dependent object dep_object_parent_name Varchar2(4000) Name of the dependent object dep_object_operator_ type Varchar2(4000) Name of the dependent object Table C28 ALL_IV_LINEAGE_DEPENDENTS Column Name Type Description object_id Number Object id of the object whose dependents are listed object_type Varchar2(4000) Type of the object Table C27 ALL_IV_IMPACT_DEPENDENTS Column Name Type Description Management C-16 Oracle Warehouse Builder 3i Users Guide object_name Varchar2(255) Name of the object. business_name Varchar2(4000) display name of the object context_name Varcahr2(4000) The fully qualified name for the object. For example a table ORDERS in schema OP in project MyProject would be MyProject.OP.ORDERS description Varchar2(4000) description parent_object_id Number Object id of the parent parent_object_type Varchar2(4000) Type of the parent object parent_object_name Varchar2(4000) Name of the parent object. For example, Table is considered to be parent of Column, Dimension is considered to be parent of Level, and Level Attribute, Cube is considered to be parent of Measures, EntityMap is considered to be parent of ItemMaps, etc. dep_object_id Number Id of the dependent object dep_object_type Varchar2(4000) type of the dependent object dep_object_name Varchar2(255) Name of the dependent object dep_object_business_ name Varchar2(4000) Display name for the dependent object dep_object_context_ name Varchar2(4000) dep_expression Varchar2(4000) If a dependent object has an expression, it will be returned here. For example, if a column x is the result of a function h=g+1 and x is wired to h, when you get the dependents of x, you will get h together with the h=g+1 expression. In addition, if the dependent object is an item (e.g.column), where an entity level expression (such as a filter or join condition on the stage component) applies, this expression will also be returned with each item (e.g. stage component parameter) that is aggregated by that entity (e.g. stage component) -Suggestion: that the expressions coming from the entities are prefixed by the type of expression. for example: FILTER:d>1. This way, we can differentiate what immediately applies to the item and what comes from the parent entity. -Strong Recommendation: Replace %1 %2 with the actual argument names. dep_object_parent_id Number Id of the dependent object Table C28 ALL_IV_LINEAGE_DEPENDENTS Management OWB Public View Tables C-17 Classification Model dep_object_parent_type Varchar2(4000) type of the dependent object dep_object_parent_name Varchar2(4000) Name of the dependent object dep_object_operator_ type Varchar2(4000) Table C29 ALL_IV_CLASSIFICATIONS Column Name Type Description information_system_id Number(9) Object id of the information system information_system_name Varchar2(255) Name of the information system classification_id Number(9) Object id of the catalog classification_name Varchar2(255) Name of the catalog. May be null if the root of the catalog tree. business_name Varchar2(4000) display name description Varchar2(4000) description classification_type Varchar2(14) Type of the classification updated_on Date The time the object was last modified created_on Date The time this object was created Table C30 ALL_IV_CLASSIFICATION_ITEMS Column Name Type Description classification_id Number Object Id of the classification. classification_name Varchar2(255) name of the classification classified_object_id Number(9) Id of the object that is classified classified_object_type Varchar2(4000) Type of the object classified_object_name Varchar2(255) Name of the object classified Table C28 ALL_IV_LINEAGE_DEPENDENTS Management C-18 Oracle Warehouse Builder 3i Users Guide Expression & Function Model Table C31 ALL_IV_FUNCTION_LIBRARIES Column Name Type Description information_system_id Number(9) Id of the information system information_system_name Varchar2(255) Name of the information system function_library_id Number(9) Unique Id of the function library function_library_name Varchar2(255) Name of the function library. business_name Varchar2(4000) Display name. description Varchar2(4000) description is_valid Varchar2(7) updated_on Date The time the object was last modified created_on Date The time this object was created Table C32 ALL_IV_FUNCTIONS Column Name Type Description function_library_id Number(9) Id of the function library function_library_name Varchar2(255) Name of the function library. function_id Number(9) Unique Id of the function function_name Varchar2(255) Name of the function. Note that this name by itself may not be unique across the project. Use signature to display a unique function signature. business_name Varchar2(4000) Display name. Note that this name by itself may not be unique across the project. Use signature to display a unique function signature. description Varchar2(4000) description signature Varchar2(4000) signature string. function_type Varchar2(40) Type of the function. is_valid Varchar2(7) updated_on Date The time the object was last modified created_on Date The time this object was created Management OWB Public View Tables C-19 Table C33 ALL_IV_FUNCTION_PARAMETERS Column Name Type Description function_id Number(9) Unique id of the function function_name Varchar2(255) name of the function parameter_id Number(9) Id of the parameter parameter_name Varchar2(255) Name of the parameter business_name Varchar2(4000) display name description Varchar2(4000) Description position Number(9) Position of the parameter within function data_type Varchar2(255) Data type of the parameter length Number(9) length of the parameter precision Number(9) Precision scale Number(9) scale of the parameter parameter_type varchar2(2000) Whether the parameter is input, output or input_ output. updated_on Date The time the object was last modified created_on Date The time this object was created Table C34 ALL_IV_FUNCTION_IMPLS Column Name Type Description function_id Number(9) Unique id of the function function_name Varchar2(255) name of the function function_implementation_id Number(9) Id of the parameter unction_implementation_name Varchar2(255) Name of the parameter business_name Varchar2(4000) display name description Varchar2(4000) Description language Varchar2(255) Implementation language script Varchar2(4000) The text of the script updated_on Date The time the object was last modified created_on Date The time this object was created Management C-20 Oracle Warehouse Builder 3i Users Guide Configuration & Deployment Model Transformation/Mapping Model Table C35 ALL_IV_OBJECT_CONFIGURATIONS Column Name Type Description configured_object_id Number(9) Id of the configured object. configured_object_type Varchar2(4000) Type of the object configured_object_name Varchar2(255) Name of the configured object physical_name Varchar2(255) Name of the physical object configuration_parameter_id Number(9) Unique identifier for the configuration structure configuration_parameter_name Varchar2(4000) Name of the configuration structure configuration_parameter_type Varchar2(23) Name of the configuration structure business_name Varchar2(4000) Display name for the configuration parameter description Varchar2(4000) description for the configuration parameter position Number(9) The position of the parameter within the parameter group. argument Varchar2(2000) The actual value assigned for a configuration item (parameter). This is of type string. complex_argument Varchar2(4000) group_name Varchar2(64) Parent configuration structure identifier. This may be null. updated_on Date The time the object was last modified created_on Date The time this object was created Table C36 ALL_IV_XFORM_MAPS Column Name Type Description information_system_id Number(9) id of the parent information system information_system _name Varchar2(255) name of the parent information system map_id Number(9) id of the map map_name Varchar2(255) name of the map business_name Varchar2(4000) display name description Varchar2(4000) description Management OWB Public View Tables C-21 composite_map_component_id Number(9) id of the root map component composite_map_component_ name Varchar2(255) name of the root map component is_valid Varchar2(7) updated_on Date The time the object was last modified created_on Date The time this object was created Table C37 ALL_IV_XFORM_MAP_COMPONENTS Column Name Type Description map_id Number(9) id of the map map_name Varchar2(255) name of the map map_component_id Number(9) id of the map component map_component_name Varchar2(255) name of the map component business_name Varchar2(4000) display name description Varchar2(4000) description operator_type Varchar2(4000) type of the operator. composite_component_id Number(9) Parent transformation map component id. This is a foreign key to self. composite_component_name Varchar2(255) Name of the parent transformation map component data_entity_id Number(9) unique id of the object that is passed as argument. data_entity_type Varchar2(4000) type of the object data_entity_name Varchar2(255) Name of the object updated_on Date The time the object was last modified created_on Date The time this object was created Table C38 ALL_IV_XFORM_MAP_PROPERTIES Column Name Type Description map_component_id Number(9) id of the map component map_component_name Varchar2(255) name of the map component property_id Number(9) Id of the property Table C36 ALL_IV_XFORM_MAPS Column Name Type Description Management C-22 Oracle Warehouse Builder 3i Users Guide property_name Varchar2(255) name of the property business_name Varchar2(4000) display name description Varchar2(4000) description property_group_name Varchar2(255) group name property_value Varchar2(255) value of the property Table C39 ALL_IV_XFORM_MAP_PARAMETERS Column Name Type Description map_component_id Number(9) id of the map component map_component_name Varchar2(255) name of the map component parameter_id Number(9) id of the map parameter parameter_name Varchar2(255) name of the parameter business_name Varchar2(4000) display name description Varchar2(4000) description map_id Number(9) id of the transformation map map_name Varchar2(255) Name of the map. parameter_group_name Varchar2(255) name of the parameter group parameter_group_id Number(9) parameter_type varchar2(4000) Whether the parameter is input, output or input_ output. Position Number(9) position of the parameter within the mapping data_type Varchar2(2000) Data type of the parameter transformation_expression Varchar2(4000) Expression that defines the transformation. NOTE: The expression string should be parsed, and the tokens has to be replaced with real object names before presenting here. data_item_id Number(9) unique id of the object that is passed as argument. data_item_type Varchar2(2000) type of the data object data_item_name Varchar2(255) Name of the data object source_parameter_id Number(9) unique id of the object that is passed as argument. Table C38 ALL_IV_XFORM_MAP_PROPERTIES Column Name Type Description Management OWB Public View Tables C-23 Runtime Public Views Source_parameter_name Varchar2(255) Name of the source parameter object updated_on Date The time the object was last modified created_on Date The time this object was created Table C40 ALL_IV_PROCESS Column Name Type Description process_id Number process_name Varchar2(4000) updated_on Date The time the object was last modified created_on Date The time this object was created Table C41 ALL_IV_PROCESS_RUN Column Name Type Description process_id Number FK to ALL_IV_PROCESS.process_id process_run_id Number id of the process run start_time Date The time the process was started end_time Date The time the process was completed status Varchar2(4000) The status of the process. Could be RUNNING, COMPLETE, FAILURE number_records_selected Number Number of records selected number_records_inserted Number Number of records inserted number_records_updated Number Number of records updated number_records_deleted Number Number of records deleted number_records_discarded Number Number of records discarded number_errors Number Number of errors in the process run updated_on Date The time the object was last modified created_on Date The time this object was created Table C39 ALL_IV_XFORM_MAP_PARAMETERS Column Name Type Description Management C-24 Oracle Warehouse Builder 3i Users Guide Table C42 ALL_IV_MAP_RUN Column Name Type Description Process_run_id Number FK to ALL_IV_PROCESS_RUN.process_run_id map_id Number not currently used map_name Varchar2(4000) name of the map. map_run_id Number id of the map run. Start_time Date The time the map run was started. end_time Date The time the map run was completed. Status Varchar2(4000) The status of the map run. Could be RUNNING, COMPLETE, or FAILURE. number_records_selected Number Number of records selected. Sum of the number of records selected for all component steps. number_records_inserted Number Number of records inserted. Sum of the number of records inserted for all component steps. number_records_updated Number Number of records updated. Sum of the number of records updated for all component steps. number_records_deleted Number Number of records deleted. Sum of the number of records deleted for all component steps. number_records_discarded Number Number of records discarded. Sum of the number of records discarded for all component steps. number_errors Number Number of errors in the map run. Sum of the errors for all component steps. updated_on Date The time the object was last modified. The latest update time of the component steps. Created_on Date The time this object was created. The earliest creation time of the component steps. Table C43 ALL_IV_RUN_TARGET Column Name Type Description map_run_id Number FK to ALL_IV_MAP_RUN.map_run_id run_target_name Varchar2(4000) name of the target run_target_id Number id of the run target entry map_name Varchar2(4000) name of the mapping step number_records_inserted Number Number of records inserted number_records_updated Number Number of records updated number_records_deleted Number Number of records deleted Management OWB Public View Tables C-25 number_errors Number number of errors in this mapping step updated_on Date The time the object was last modified created_on Date The time this object was created Table C44 ALL_IV_RUN_ERROR Column Name Type Description map_run_id Number FK to ALL_IV_MAP_RUN.map_run_id run_target_name Varchar(4000) name of the target run_target_id Number Id of the run target entry. FK to ALL_IV_RUN_ TARGET.run_target_id run_error_id Number id of the map run error_number Number Number of the error error_message Varchar2(4000) Text of the error message updated_on Date The time the object was last modified created_on Date The time this object was created Table C43 ALL_IV_RUN_TARGET Column Name Type Description Management C-26 Oracle Warehouse Builder 3i Users Guide Using The XML Toolkit D-1 D Using The XML Toolkit This section describes how to retrieve and store data from XML documents in a target schema using the XML Toolkit. This toolkit can extract data from XML documents, which can reside in a variety of sources and formats, and store that data in Oracle8i tables, CLOB database columns, Advanced Queues, and other Oracle8i database objects. An XML document is one that conforms to the Extensible Markup Language (XML) specification. XML is a subset of SGML, and this simplified dialect allows developers to design their own customized markup languages which can be optimized for delivery of documents on the World Wide Web and to support implementation of e-commerce and numerous other applications. The XML Toolkit is set of PL/SQL procedures, functions, and packages which you can use to retrieve and load data from XML documents into Oracle8i database objects. For example, you can retrieve data from an XML document that resides in a file and load it into several tables that reside in a data warehouse. Retrieving Data From Sources The XML Toolkit can retrieve data from a variety of XML documents, and these documents can have various formats. You can retrieve data from XML documents that reside in the following: s File s Multiple files s CLOB database column s Raw Based Advanced Queue s Object/CLOB based Advanced Queue D-2 Oracle Warehouse Builder 3i Users Guide s URL You identify the source of the data to the XML Toolkit using an element name defined by the Toolkit. For example, you identify a file with the element name file. Subsequent sections and examples describe individual element names, their respective attributes (if any), and the required syntax. Storing Data In Targets The XML Toolkit can store data retrieved from XML documents in a variety of Oracle8i database objects. The Toolkit can store data in the following objects: s Table or multiple tables in a target Oracle8i database s Updatable views s Object tables s Updatable object views s Object/CLOB based Advanced Queues You identify the target for the data using an element name defined by the Toolkit. For example, you identify a target table with the element name target. The element name target has several attributes which provide control over runtime operations. For example, if the data extracted from the XML document does not match the target table, you can specify an XSL style sheet as an attribute that reformats the data accordingly. Subsequent sections and examples describe the individual element names, their respective attributes, and the required syntax. Using Runtime Controls During the extraction and load process, you can control when to commit the load as well as control the size of individual batches. These controls are specified using the element name runtimeConfig. How to Call the XML Toolkit The XML Toolkit is implemented as a set of PL/SQL procedures and functions. Thus, to use the Toolkit, you must first create a OWB transformation that invokes one of its procedures or functions. After the transformation has been created, a mapping can call the transformation using a pre-map or post-map trigger. A typical scenario would be to create a transformation that extracts and loads data from an XML document into a staging table. The transformation could be Using The XML Toolkit D-3 generalized by referencing the XML document as a runtime parameter. After you create the transformation, you could then create a mapping that uses a pre-map trigger to call the transformation to load the staging table. The mapping could then transform the data in the staging table and load it into a target table. A post-map trigger could in turn truncate the staging table before the mapping terminates. Two Entrances A transformation can invoke the XML Toolkit using one of the following entry points into the API: s The PL/SQL procedure wb_xml_load(control_file) s The PL/SQL function wb_xml_load_f(control_file) Both of these calls extract and load data from XML documents into database targets. The function, however, returns the number of documents read during the operation. The control file, itself an XML document, specifies the source of the XML documents, the targets, and any runtime controls. After the transformation has been defined, a mapping typically calls the transformation as a pre-map or post-map trigger. The following example illustrates a script that can be used to implement an OWB transformation which extracts data from an XML document stored in the file products.xml and loads it into the target table books. begin wb_xml_load( <OWBXMLRuntime> || <XMLSource> <file>\ora817\GCCAPPS\products.xml</file> </XMLSource> <targets> <target XSLFile="\ora817\XMLstyle\GCC.xsl">books</target> </targets> || || || || || || </OWBXMLRuntime>); end; D-4 Oracle Warehouse Builder 3i Users Guide The control file itself is an XML document, and the element name OWBXMLRuntime defines the top-level, or root element, of this document. The remaining element names which define the document sources and targets are self-explanatory. After you create and name this transformation, you can reference it by name in a mapping. The most common reference for the transformation would be in a pre-map or post-map trigger. Typical Control Files The nine example control files in this section are well-formed and valid control files that extract and load data from the following sources into a database target: 1. A single file 2. A single file whose element names must be renamed 3. A single file to multiple targets 4. Multiple files 5. A large file split into several parts 6. CLOB column 7. URL 8. Raw Advanced Queue 9. Object Type Based Advanced Queue The first five examples extract data from XML documents that reside in files; the remaining cases extract data from database objects or a document addressed by a URL. The examples cover all the sources that are managed by the XML Toolkit. Notes: s The file names for the document source and the XSL style sheet are relative to the node that supports the Oracle8i instance. s The white space in the example control file that separates the text from the concatenate characters is unnecessary and is included only to improve readability. s The control file is a well-formed XML document. See the control files Document Type Definition (DTD) on page D-19. Using The XML Toolkit D-5 XML Documents Stored in Files The XML Toolkit can extract data from documents stored in a single file or in multiple files when they reside in the same directory. The Toolkit can also extract data from these sources and store it into multiple tables. Often, the data in an XML document fails to match the target object in which case you can reformat the data to match the target by including an XSL style sheet. The second example shows you how to reference an XSL style sheet by specifying a value for the XSLFile attribute for the target. The use of style sheets to reformat the source data is probably the most common case as the data from XML documents rarely match the column names of target tables in a data warehouse. Finally, you can improve load efficiency for large XML documents by splitting the documents into parts and performing a separate load operation for each part. This is accomplished by specifying a value for the splitElement attribute for the source file. The Most Basic Case When the element names of an XML document exactly match the column names in a target table and the document resides in a file, you can easily create a control file to extract and load data from the document into the target table. The XML Document The XML document below (ORDERS) resides in a file named \ora817\examples\ex1.xml. This file name is relative to the Oracle8i database instance. <ORDERS> <?xml version="1.0"?> <ROW> <ID>100</ID> <ORDER_DATE>2000.12.20</ORDER_DATE> <SHIPTO_NAME>Jeff Q. Vintner</SHIPTO_NAME> <SHIPTO_STREET>500 Marine World Parkway</SHIPTO_STREET> <SHIPTO_CITY>Redwood City</SHIPTO_CITY> <SHIPTO_STATE>CA</SHIPTO_STATE> <SHIPTO_ZIP>94065</SHIPTO_ZIP> </ROW> </ORDERS> D-6 Oracle Warehouse Builder 3i Users Guide The Target Table The column names in the target table (PURCHASE_ORDERS) described by the DDL below match the element names of the XML document described above. create table Purchase_Orders ( id varchar2(10) not null, order_date date not null, shipto_name varchar2(60) not null, shipto_street varchar2(80) not null, shipto_city varchar2(30) not null, shipto_state varchar2(2) not null, shipto_zip varchar2(9)) The Control File The control file below directs the XML Toolkit to extract and load the data from the ORDERS document into the Purchase_Orders table. <OWBXMLRuntime>|| <XMLSource>|| <file>\ora817\examples\ex1.xml</file>|| </XMLSource>|| <targets>|| <target dateFormat="yyyy.MM.dd">Purchase_Orders</target>|| </targets>|| </OWBXMLRuntime> About the dateFormat Attribute The dateFormat attribute defined for the target element is necessary so that the XML SQL Utility (XSU) can correctly store the text data in the table in the order_date column which has a data type of DATE. Refer to the Oracle8i XML Reference Guide Release 3 (8.1.7) for additional information. from the XSL processor can store the stored in various modes. Each example is a control file. When a transformation uses the wb_xml_load procedure of the wb_xml_load_f function to call the XML Toolkit, it must include a complete control file. wb_xml_load (control_info VARCHAR2) Using The XML Toolkit D-7 wb_xml_load_f (control_info VARCHAR2) RETURN NUMBER The control file is an XML document that describes the sources, the targets, and any runtime controls. This section uses several examples that show you how to define control files for most situations. A Builder Transformation If you create a transformation using this control file and call the transform from a pre-map trigger, the following actions occur: 1. The XML Toolkit reads the document ORDERS into a buffer and parses its data. 2. The Toolkit calls Oracle8i XML SQL utility (XSU) to load the parsed data into the Purchase_Orders table. 3. The Transformation then returns control to the mapping. This is a simple case where the document and the table match up exactly. The next example shows you handle the more common case inexact matches. Inexact Matches When the element names in the XML document fail to match the column names in the target column, you must include an XSL style sheet that reformats the data before it is loaded into the target table. This example shows you how to reformat the data by specifying a style sheet using the XSLFile attribute. The control file extracts and reformats the data before loading it into the Purchase_Orders table. The XML Document The XML document below (purchaseORDER) resides in the file \ora817\examples\ex2.xml. This file name is relative to the Oracle8i database instance. <purchaseOrder> <id>103123-4</id> <orderDate>2000-10-20</orderDate> <shipTo country="US"> <name>Alice Smith</name> <street>123 Maple Street</street> <city>Mill Valley</city> <state>CA</state> <zip>90952</zip> </shipTo> <comment>Hurry, my lawn is going wild!</comment> <items> <item> D-8 Oracle Warehouse Builder 3i Users Guide <partNum>872-AA</partNum> <productName>Lawnmower</productName> <quantity>1</quantity> <USPrice>148.95</USPrice> <comment>Confirm this is electric</comment> </item> <item> <partNum>845-ED</partNum> <productName>Baby Monitor</productName> <quantity>1</quantity> <USPrice>39.98</USPrice> <shipDate>1999-05-21</shipDate> </item> </items> </purchaseOrder> The Target Table The column names in the target table (Purchase_Orders) are described in the previous example on page D-6. The Control File The control file below directs the XML Toolkit to extract and reformat the data from the purchaseORDER document before loading it into the target table Purchase_Orders. <OWBXMLRuntime>|| <XMLSource>|| <file>\ora817\examples\ex2.xml</file>|| </XMLSource>|| <targets>|| <target XSLFile="\ora817\examples\ex2_1.xsl" dateFormat="yyyy-MM-dd">|| Purchase_Orders|| </target>'|| ' </targets>'|| '</OWBXMLRuntime>' The only addition to this control file is the XSLFile attribute for the element target. The Style Sheet The style sheet itself is a straightforward XML document that associates a parsed data item from the XML document with a column name in the Using The XML Toolkit D-9 target table. The following snippet from the style sheet shows how it associates a parsed data item with a column in the target: <SHIPTO_NAME> <xsl:value-of select="shipTo/name"/> </SHIPTO_NAME> <SHIPTO_STREET> <xsl:value-of select="shipTo/street"/> </SHIPTO_STREET> <SHIPTO_CITY> <xsl:value-of select="shipTo/city"/> </SHIPTO_CITY> <SHIPTO_STATE> <xsl:value-of select="shipTo/state"/> </SHIPTO_STATE> This control statement could now be used in a transform to extract data from the XML document and load it into the target table. A Builder Transformation If you create a transformation using this control file and call the transform from a pre-map trigger, the following actions occur: 1. The XML Toolkit reads the document ORDERS into a buffer and parses its data. 2. The Toolkit uses the style sheet to associate parsed data with table columns. 3. The Oracle8i XML SQL utility (XSU) loads the parsed data into Purchase_Orders. The Transformation then returns control to the mapping. Multiple Targets This control statement extracts and reformats data from the purchaseORDER document and stores it into two target tables: Purchase_Orders and Items. The tables have different column names, and so a style sheet must be specified for each target. The XML Document The XML script for the previous example on page D-7 describes the source document (purchaseORDER). The Target Tables The column names for Purchase_Orders are described on page D-6; the column names for items are not required to understand the control file script. D-10 Oracle Warehouse Builder 3i Users Guide The Control File The control file below directs the XML Toolkit to extract and reformat the data from the purchaseORDER document before loading it into two target tables. <OWBXMLRuntime>|| <XMLSource>|| <file>\ora817\examples\ex2.xml</file>|| </XMLSource>|| <targets>|| <target XSLFile="\ora817\examples\ex2_1.xsl" dateFormat="yyyy-MM-dd">|| Purchase_Orders|| </target>'|| ' <target XSLFile="\ora817\examples\ex2_2.xsl" dateFormat="yyyy-MM-dd">|| Items|| </target>'|| ' </targets>'|| '</OWBXMLRuntime>' This control file is like the previous example except that it specifies an additional target (Items), and a style sheet (ex2_2.xsl) which associates parsed data items from the document with columns in the Items table. Very Large XML Documents When you extract data from a very large XML document, the memory requirements for the processing can impact load performance. In this case, you can often reduce the memory requirements and improve the efficiency of the operation by dividing the XML documents into multiple parts. To divide an XML document into parts, specify a value for the splitElement attribute of the XMLSource element. The value is the name of an element within the source XML document, preferably one that delineates blocks of text. A good example is an XML document that organizes books into categories using the element name Category. There are many categories, and each category defines numerous books. Thus, if you specify the splitElement attribute as in the example below, then the XML Toolkit divides the source document into as many parts as there are categories. The Control File The control file below directs the XML Toolkit to extract the data from the BookAreUs document by dividing the document according to its catalogs, and then load each part into the target table books. '<OWBXMLRuntime>'|| Using The XML Toolkit D-11 <XMLSource splitElement="Category">|| <file>\ora817\examples\ex4.xml</file>|| </XMLSource>|| <targets>|| <target XSLFile="ora817\examples\ex4.xsl">books</target>|| </targets>|| </OWBXMLRuntime> How well this operation improves efficiency depends on the selection of the split element. If the split element defines only a few parts, then memory resources may not be reduced enough to improve performance. XML Documents Stored as Other Objects The XML Toolkit can extract data from documents stored as a database object or at a location defined by a URL. The examples in this section illustrate control files for these cases. Document Stored as a CLOB This example control file loads the Purchase_Orders table with data extracted from an XML document that is stored in a table (clientOrders) as a CLOB column. The clientOrders table contains detailed information about each order, including multimedia information. The order itself is stored in the xmlOrder column, a voice recording of the order is stored in the voiceOrder column, and a picture of the client who made the order is stored in the clientOrder column. The XML Document The XML document which is stored in the first row of the clientOrders table is exactly the same as the order described in the first example. The complete XML document that defines the order is described on page D-5. clientOrders ID xmlOrder voiceOrder clientOrder Number CLOB BLOB BLOB D-12 Oracle Warehouse Builder 3i Users Guide The Control File The control file below directs the XML Toolkit to extract data from the document that resides in the first row of the clientOrders table and load it into the Purchase_Orders table. <OWBXMLRuntime>|| <XMLSource>|| <CLOB whereClause="where id=1">|| <table>clientOrders</table>|| <CLOBColumn>xmlOrder</CLOBColumn>|| </CLOB>|| </XMLSource>|| <targets>|| <target dateFormat="yyyy.MM.dd">Purchase_Orders</target>|| </targets>|| </OWBXMLRuntime> Comments on the Control File A few comments are in order regarding this control file, especially regarding the values assigned to attributes. 1. The first row of the table is specified as an attribute of the CLOB element: CLOB whereClause="where id=1" The ID column of the clientOrders table identifies each row. 2. The dateFormat attribute describes the format of the character value parsed from the ORDER_DATE element. This information is necessary to load the parsed data into the Date datatype column of Purchase_Orders. 3. A style sheet is not required because the element names in the XML document exactly match the column names of the Purchase_Orders table. If they did not, then a style sheet would be required. 4. You could store the data parsed from the XML document in multiple tables by including additional target elements. Of course, a style sheet would be required whenever the column names of a target table fail to match the XML elements. Using The XML Toolkit D-13 Document Stored As An Object-Based Advanced Queue This example control file loads the Purchase_Orders table with data extracted from XML documents that are placed on an Oracle8i Advanced Queue (AQ). An e-commerce application can use an Advanced Queue to automate the acceptance, processing, routing, and completion of orders. You can configure an AQ to remain active and wait for entries to be placed on the queue and also specify a maximum wait time before the queue deactivates itself. The following control file loads the Purchase_Orders table with data extracted XML documents that are periodically placed on newOrders, an Object-Based AQ. The control file specifies that the queue wait for new entries and specifies a time-out value in seconds or forever. The CLOBColumn attribute indicated the column in the object type containing the XML to be loaded. For additional information on Advanced Queues, refer to Oracle8i Application Developers Guide - Advanced Queuing Release 2 (8.1.6). The XML Document The XML documents which are periodically placed on the newOrders queue is exactly the same as the order described in the first example. The complete XML document that defines the order is described on page D-5. The Control File The control file below directs the XML Toolkit to extract data from documents that are periodically place on the newOrdersAQ and store them into the Purchase_Orders table. <OWBXMLRuntime>|| <XMLSource>|| <AQ wait="WAIT" waitTime="WAIT_FOREVER">|| <AQName>newOrders</AQName>|| </AQ>|| </XMLSource>|| <targets>|| <target dateFormat="yyyy.MM.dd">Purchase_Orders</target>|| </targets>|| </OWBXMLRuntime>); The waitTime attribute is configured so that a dequeue call wait until an entry is available on the queue before it is released. D-14 Oracle Warehouse Builder 3i Users Guide Document Stored as a Raw Advanced Queue In this example, a control file loads the Purchase_Orders table with data extracted from XML documents that are placed on an Oracle8i Advanced Queue (AQ). An e-commerce application can use an Advanced Queue to automate the acceptance, processing, routing, and completion of orders. You can configure an AQ to remain active and wait for entries to be placed on the queue and also specify a maximum wait time before the queue deactivates itself. The following control file loads the Purchase_Orders table with data extracted XML documents that are periodically placed on newOrders, a RAW AQ. The control file specifies that the queue wait for new entries and specifies a time-out value in seconds or forever. The next example shows how to rewrite the control file for a queue that resides in an object based Advanced Queue. For additional information on Advanced Queues, refer to Oracle8i Application Developers Guide - Advanced Queuing Release 2 (8.1.6). The XML Document The XML documents which are periodically placed on the newOrdersAQ queue is exactly the same as the order described in the first example. The complete XML document that defines the order is described on page D-5. The Control File The control file below directs the XML Toolkit to extract data from documents that are periodically place on the newOrdersAQ Advanced Queue and store them into the Purchase_Orders table. <OWBXMLRuntime>|| <XMLSource>|| <AQ wait="WAIT" waitTime="20">|| <AQName>newOrdersAQ</AQName>|| <ObjectType>xmlMessages</ObjectType>|| <CLOBColumn>xmlEntry</CLOBColumn>|| </AQ>|| </XMLSource>|| <targets>|| <target dateFormat="yyyy.MM.dd">Purchase_Orders</target>|| </targets>|| </OWBXMLRuntime>); The waitTime attribute specifies that a dequeue call made on an empty queue will wait a maximum of twenty seconds for a document to be placed on the queue. Using The XML Toolkit D-15 Document Stored at URL This example control file loads the Books table with data extracted from a document produced by a URL. <OWBXMLRuntime>|| <XMLSource>|| <URL parameterTable="ptable">|| <![CDATA[http://oracle.com//xmlserv/library]]>'|| </URL>'|| '</XMLSource>'|| '<targets>'|| '<target XSLFile="\ora817\examples\lib.xsl" ignoreCase="TRUE">Books'|| </target>'|| '</targets>'|| '</OWBXMLRuntime>' The targets element includes two attributes: one for a style sheet to match the source element names with the target column names, and another to ignore case. The parameter table is used when dynamically generating XML, the column names of the parameter table are used as parameter names, and the data as parameter values. Only VARCHAR2, VARCHAR, CHAR, and NUMBER columns are used to form dynamic parameters. Control File Element Names and Attributes The control file for the XML Toolkit is an XML document, and the next two sections describe all of its elements. The first section informally describes each element that can be used to create a control file; the second section is the Document Type Definition (DTD) of the control file, which formally defines the control file structure. The following description of the control file elements is divided into three parts: elements that describe the source for XML documents, elements that describe the target of the load operation, and elements that determine the runtime environment for the XML Toolkit. Source Elements This table describes all the elements used to define sources of XML documents and their respective attributes. D-16 Oracle Warehouse Builder 3i Users Guide Name Description OWBXMLRuntime Root element for the control file document. Attributes: None. XMLSource Defines all the sources using additional elements. Attributes: splitElement Divides the source XML documents into multiple documents. Used to reduce the memory requirements when loading large XML documents. If used, all information in the DOMParserConfig is ignored. file Specifies a single XML document that resides in a single file as the source. Attributes: None. directory Specifies a single directory as the source of XML documents. The Toolkit extracts all the XML documents that match a mask which is specified in the mask attribute. Attributes: mask Filter for files that reside in a directory. The Toolkit supports only Windows * masks. For example, "*.xml". AQ Specifies that the source of XML document resides in an Advanced Queue. Attributes: consumerName dequeueMode navigation visibility wait waitTime messageID correlation These attributes represent the various dequeue options supported by Advanced Queues. For complete information, refer to Oracle8i Application Developers Guide - Advanced Queuing Release 2 (8.1.7). Using The XML Toolkit D-17 AQFullName Specifies the complete name of an Advanced Queue. You can specify as SCHEMA.AQ_NAME or AQ_NAME. Attributes: None. CLOBColumn Specifies that an object based on an Advanced Queue contains a CLOB column (object/ CLOB based AQ). If not specified, indicates that the AQ is a RAW based AQ. Attributes: None. CLOB Specifies the source of XML document resides in a CLOB column of a database table. Attributes: whereClause Specifies where clause for the query that retrieves the XML document from the CLOB column. CLOB.table Specifies the table or view that contains a CLOB column. Attributes: None. CLOBColumn Specifies the name of a CLOB column. Attributes: None. URL Specifies that the XML document resides at an address given as a URL. Attributes: proxy Port number of the proxy server parameterTable Directs the Toolkit to retrieve documents from the URL multiple times according to the parameters. The parameter names are the parameter tables column names. The Toolkit reads only columns that have VARCHAR, VARCHAR2, CHAR and NUMBER data types. lowerParameterNames If TRUE, parameter names are in lowercase; if FALSE, parameter names are in uppercase. Name Description D-18 Oracle Warehouse Builder 3i Users Guide Target Elements This table describes all the elements used to define the load targets and their respective attributes. You can define multiple targets for the data parsed from a set of XML documents. Runtime Control This table describes elements that determine the runtime environment. Name Description targets Specifies all the database targets. You specify individual targets using target elements. Attributes: None. target Specifies the target of a database load. Attributes: truncateFirst If TRUE, then truncate the target object before the load. XSLfile Address of the XSL style sheet which the Toolkit uses to associate parsed data with individual target columns. XSLRefURL URL the Toolkit uses to resolve external references made by the XSL style sheet LoadType Specifies the load operation as INSERT, UPDATE, or DELETE. When UPDATE or DELETE is specified, truncateFirst and truncateBeforeEachLoad are set to FALSE. The following attributes are all XU attributes which are defined in the Oracle8i XML Reference Guide Release 3 (8.1.7). ignoreCase commitBatch rowTag (to set, use VARCHAR NULL) dateFormat batchSize keyColumnList updateColumnList Using The XML Toolkit D-19 Document Type Definition for the Control File The Document Type Definition below describes all the possible elements which may occur in a control file for the XML Toolkit, their respective attributes, their ordering, and the conditions of their use. <!ELEMENT OWBXMLRuntime (XMLSource, targets, runtimeConfig?)> <!ELEMENT XMLSource ((file | directory | URL | CLOB | AQ ),DOMParserConfig?)> <!ATTLIST XMLSource splitElement CDATA #IMPLIED> <!ELEMENT file (#PCDATA)> <!ELEMENT directory (#PCDATA)> <!ATTLIST directory mask CDATA "*.xml"> <!ELEMENT CLOB (table, CLOBColumn)> <!ELEMENT table (#PCDATA)> <!ATTLIST CLOB whereClause CDATA #IMPLIED> <!ELEMENT URL (#PCDATA)> <!ATTLIST URL proxy CDATA #IMPLIED proxyPort CDATA "80" parameterTable CDATA #IMPLIED lowerParameterNames (TRUE | FALSE) "TRUE"> <!ELEMENT AQ (AQName, (ObjectType, CLOBColumn)?)> <!ELEMENT AQName (#PCDATA)> <!ELEMENT ObjectType (#PCDATA)> <!ELEMENT CLOBColumn (#PCDATA)> <!ATTLIST AQ consumerName CDATA #IMPLIED Name Description runtimeConfig Specifies the runtime configuration for the XML Toolkit. Attributes: commitAfterLoad If TRUE, then issue commit at the end of the load. D-20 Oracle Warehouse Builder 3i Users Guide dequeueMode (REMOVE | BROWSE | LOCKED) "REMOVE" navigation (NEXT_MESSAGE | NEXT_TRANSACTION | FIST_MESSAGE "NEXT_MESSAGE" visibility (ON_COMMIT | IMMEDIATE) "IMMEDIATE" wait (WAIT_FOREVER | WAIT_NONE | WAIT) "WAIT_FOREVER" waitTime CDATA #IMPLIED messageID CDATA #IMPLIED correlation CDATA #IMPLIED> <!ELEMENT targets (target+)> <!ELEMENT target (#PCDATA)> <!ATTLIST target truncateFirst (TRUE | FALSE) "TRUE" truncateBeforeEachLoad (TRUE | FALSE) "FALSE" XSLFile CDATA #IMPLIED XSLRefURL CDATA #IMPLIED loadType (INSERT| UPDATE | DELETE) "INSERT" ignoreCase (TRUE | FALSE) "TRUE" commitBatch CDATA "0" rowTag CDATA "ROW" dateFormat CDATA "MM/dd/yyyy HH:mm:ss" batchSize CDATA "17" keyColumnList NMTOKENS #IMPLIED updateColumnList NMTOKENS #IMPLIED> <!ELEMENT runtimeConfig EMPTY> <!ATTLIST runtimeConfig commitAfterLoad (TRUE | FALSE) "TRUE" batchSize CDATA "10"> <!ELEMENT DOMParserConfig EMPTY> <!ATTLIST DOMParserConfig showWarnings (TRUE | FALSE) "FALSE" retainCDATASection (TRUE | FALSE) "FALSE" debugMode (TRUE | FALSE) "FALSE" preserveWhitespace (TRUE | FALSE) "FALSE" validationMode (TRUE | FALSE) "FALSE" baseURL CDATA #IMPLIED> Using The XML Toolkit D-21 Reference Materials The XML Toolkit employs several Oracle8i XML SDK software components. For more in-depth discussion of the XML support provided by Oracle, refer to the following: s Application Developers Guide - Advanced Queuing Release 2 (8.1.7) s Oracle8i Application Developers Guide - XMLRelease 3 (8.1.7) s Oracle8i XML Reference Guide - Release 3 (8.1.7) Two web sites that contain specific and useful information on the XML specification are the following: http://www.w3.org/XML/ http://www.XML.com There are also numerous other web sites as well as a plethora of text books on building XML applications. D-22 Oracle Warehouse Builder 3i Users Guide Batch Services E-1 E Batch Services This appendix contains the following topics describe Batch Services (BS) in OWB 3i: s Overview s Batch Service Interface s Batch Service Command Line Feature Overview The Batch Services (BS) feature in Oracle Warehouse Builder allows users to run batch oriented operations like validation, import or generation off-line. These operations are available within the OWB Client Application, but there are a couple of reasons why these operations are more suitable to be run as batch services: Bulk metadata operations on the OWB repository can be time consuming (multi-hour) processes which preferably should be run off-line and/ or scheduled by a system administrator BS in OWB, provides a framework with the necessary characteristics to run batch oriented operations separate from the OWB Client Application. Batch Service Interface The OWB Batch Services interface is called WBBatchInvokeAPI.java. All third parties should use this Batch Service API to integrate and invoke essential services that are provided by OWB. The services supported through BS are metadata import, validation, generation, and deployment. WBBatchInvokeAPI can be accessed by importing in a Java class by using the following: Batch Service Interface E-2 Oracle Warehouse Builder 3i Users Guide 1. oracle/wh/service/sdk/batchservice/WBBatchInvokeAPI.java 2. oracle/wh/service/impl/batchservice/WBBatchInvoke.java The Batch Service interface provides the following methods to enable third party developer to access OWB 3i through batch interaction: s Open and Close Repository Connection s Projects s Applications s Metadata Import s Validation s Generation s Deployment Open and Close Repository Connection To establish a connection with the Metadata Repository, use the openConnection method. It needs five parameters: the username, the password, the host name, the port number, and the SID. This method will throw WBConnectionFailureException, and WBException if the method fails to establish a connection with the repository. public void openConnection( String username, String password, String host, String port, String sid) throws WBConnectionFailureException, WBException; Close the connection without committing the repository data public void closeConnection() throws WBException, SQLException; Close the connection after committing the repository data. This allows the user to have control over committing the data before closing the connection. public void commitCloseConnection() throws WBException, SQLException; Projects Get an array of projects that are currently available. Batch Service Interface Batch Services E-3 public String[] getProjects () throws WBException; Make a project a current project. This can be done by providing project name to the current method. public void setCurrentProject (String projectName) throws WBException; Applications To get all the applications that are present within a project we use the getModules() method. It returns an array of application names that are present in given Project. public String[] getModules() throws WBException; We can assign an application to a current application by using setCurrentModule method by providing the application name as the parameter. The return type is void. WBException is thrown when OWB is not able to set the current module. public void setCurrentModule(String appName) throws WBException; Metadata Import The following method allows importing of MDL exported data as an inputstream. The log file parameter must include the full path name. The Mode denotes the mode of importing (Create, Update). usePhyiscal Name parameter is to ensure that the user uses physical name while importing. public void importMetaData( InputStream importFileStream, String logFile, String mode, boolean usePhysicalNames) throws WBException, IOException; The following method allows the import of Metadata using an MDL exported file without using the character set. The default character set is used. public void importMetaData( String m_import_file, String m_log_file, String m_mode, boolean m_usePhysicalNames) throws WBException; Batch Service Interface E-4 Oracle Warehouse Builder 3i Users Guide The following method allows the import of Metadata using an MDL exported file. A valid character set string would look like WE8MSWIN1252 public void importMetaData( String m_import_file, String m_log_file, String m_mode, boolean m_usePhysicalNames, String characterSet) throws WBException; Validation An element like a dimension or a table needs to be validated. That can be done by using the validate method. It needs an application name, the element name, and type of the element. The typename could be any of the following: TABLE, VIEW, MATERIALIZEDVIEW, DIMENSION, FACT, SEQUENCE, MAPPING. public boolean validate( String appName, String elementname, String typename) throws WBException; Once the validation has been performed. We can get the validation results in an array of strings. public String[] getValidationResults( String appName, String element, String typeName) throws WBException; We can display the validation results in console by using the following method: public void displayValidationResults ( String appName, String element, String type) throws WBException; Generation To generate an element use the following generate method. The typename could be any of the following: TABLE, VIEW, MATERIALIZEDVIEW, DIMENSION, FACT, SEQUENCE, MAPPING. public void generate( String appName, String elementname, String typename) throws WBException; Batch Service Command Line Feature Batch Services E-5 Deployment The following methods allow you to deploy your previously generated scripts to either a file system or database. The typename could be any of the following: TABLE, VIEW, MATERIALIZEDVIEW, DIMENSION, FACT, SEQUENCE, MAPPING. Deployment into a File System public void deployInFileSystem( String appName, String elementname, String typeName, String deployDir) throws WBException; Deployment in a Database public void deployInDatabase( String appName, String elementname, String typename, String user, String passwd, String host, String port, String sid) throws WBException, Exception; Deployment Results can extracted by using getDeploymentResult() public String[] getDeploymentResults () throws WBException; Batch Service Command Line Feature The Batch Service Command Line feature allows you to execute main OWB 3i commands without invoking the standard user interface. You can validate, generate, and deploy to file systems or databases any first class object in batch mode. It also allows you to control multiple repositories and execute the different first class objects like tables, views, and dimensions. Note: Batch Services does not generate objects when you run the deployment methods. You must generate the scripts first by using the generation method. Batch Service Command Line Feature E-6 Oracle Warehouse Builder 3i Users Guide Validate batchservice_class.ksh <user> <passwd> <host> <SID> <port> -validate <project name> <application name> <element name> <element type> [<logDir>] For Example: batchservice_class.ksh owb34 owb34 sroychow-pc3 owbqa817 1521 -validate "My_ Table E1 Batch Service Command Line Notations Notation Description <user> user connection to repository <passwd> password required by the user <host> repository host name <SID> SID of the repository <port> port number <project name> project name <application name> application name <element name> element name <element type> This is the element type of any first class object. It can take any of the following strings: TABLE,VIEW, MATERIALIZEDVIEW, DIMENSION, FACT, MAPPING, SEQUENCES. Note: The project name may have spaces between them. Since the Kornshell does not recognize a string enclosed in quotes as one argument, use the standard typesetter ("_^") notation of inserting space to denote spaces. For Example: s "My Project" (with one space) should be written as "My_^Project" in the command line. s "My Project" (with two spaces) should be written as "My_^_^Project" in the command line. Batch Service Command Line Feature Batch Services E-7 ^Project" im1 tab1 TABLE batchservice_class.ksh owb34 owb34 sroychow-pc3 owbqa817 1521 -validate "My_ ^Project" im1 tab1 TABLE /tmp/.owbObjects Generate batchservice_class.ksh <user> <passwd> <host> <SID> <port> -generate <project name> <application name> <element name> <element type> [<logDir>] For Example: batchservice_class.ksh owb34 owb34 sroychow-pc3 owbqa817 1521 -generate "My_ ^Project" im1 tab1 TABLE batchservice_class.ksh owb34 owb34 sroychow-pc3 owbqa817 1521 -generate "My_ ^Project" im1 tab1 TABLE /tmp/.owbObjects Deploy into a File System batchservice_class.ksh <user> <passwd> <host> <SID> <port> -deployfile <project name> <application name> <element name> <element type> <deploy directory path> [<logDir>] For Example: batchservice_class.ksh owb34 owb34 sroychow-pc3 owbqa817 1521 -deployfile "My_ ^Project" im1 tab1 TABLE /tmp/scott batchservice_class.ksh owb34 owb34 sroychow-pc3 owbqa817 1521 -deployfile "My_ ^Project" im1 tab1 TABLE /tmp/scott /tmp/.owbObjects Deploy into a Database batchservice_class.ksh <user> <passwd> <host> <SID> <port> -deploydb <project Note: the single and attached dash in front of an action is mandatory. Thus -validate is valid, but validate or validate are not valid. Batch Service Command Line Feature E-8 Oracle Warehouse Builder 3i Users Guide name> <application name> <element name> <element type> <duser> <dpasswd> <dhost> <dSID> <dport> [<logDir>] where <duser> = runtime database <dpasswd> = runtime password <dhost> = runtime database host name <dSID> = SID of the runtime database <dport> = port number of runtime database For Example: batchservice_class.ksh owb34 owb34 sroychow-pc3 owbqa817 1521 -deploydb "My_ ^Project" im1 tab1 TABLE owb34run owb34run sroychow-pc3 ora817 1521 batchservice_class.ksh owb34 owb34 sroychow-pc3 owbqa817 1521 -deploydb "My_ ^Project" im1 tab1 TABLE owb34run owb34run sroychow-pc3 ora817 1521 /tmp/.owbObjects Metadata Loader Import Type 1 batchservice_class.ksh <user> <passwd> <host> <SID> <port> -import <import file name> <log file name> < mode> <use_physical_names > where <mode> = CREATE, REPLACE, UPDATE, or INCREMANTALUPDATE <use_physical_names> = true or false For Example: batchservice_class.ksh owb34 owb34 sroychow-pc3 owbqa817 1521 -import class_ project.mdl log1.log CREATE true Type 2 batchservice_class.ksh <user> <passwd> <host> <SID> <port> -import <import file Batch Service Command Line Feature Batch Services E-9 name> <log file name> < mode> <use_physical_names > <character set> where <mode> = CREATE, REPLACE, UPDATE, or INCREMANTALUPDATE <use_physical_names> = true or false For Example: batchservice_class.ksh owb34 owb34 sroychow-pc3 owbqa817 1521 -import class_ project.mdl log1.log CREATE true WE8MSWIN1252 Type 3 batchservice_class.ksh <user> <passwd> <host> <SID> <port> -import <import file name> <log file name> < mode> <use_physical_names > <ignoreUniversalID> <character set> where <mode> = CREATE, REPLACE, UPDATE, or INCREMANTALUPDATE <use_physical_names> = true or false <ignoreUniversalID> = true or false For Example: batchservice_class.ksh owb34 owb34 sroychow-pc3 owbqa817 1521 -import class_ project.mdl log1.log CREATE true true WE8MSWIN1252 Using Batch Services Through a Parameter File The parameter file aggregates all of the command line operations into one batch activity. Parameter files enhance performance because they provide a single connection for all of the operations listed in the file. Parameter file ======================================================= # pound sign is for comments # validate "My_^Project" <application Name> <element Name> <element Type> <logdir> validate "My_^Project" IM1 TAB1 TABLE /tmp/.owbObjects # generate Batch Service Command Line Feature E-10 Oracle Warehouse Builder 3i Users Guide # generate "My_^Project" <application Name> <element Name> <element Type> <logdir> generate "My_^Project" IM1 TAB1 TABLE /tmp/.owbObjects #deploy # similarly we do for deploy in file system and the database. deploydb "My_^Project" IM1 TAB1 TABLE <logdir> import import.mdl importmdl.log CREATE true true WEMSWIN1251 ======================================================== Parameter file command line batchservice_class.ksh <user> <passwd> <host> <SID> <port> -paramfile <paramfile> [<duser> <dpasswd> <dhost> <dSID> <dport>] For Example: batchservice_class.ksh owb34 owb34 sroychow-pc3 owbqa817 1521 -paramfile batchservce.param owb34run owb34run sroychow-pc3 ora817 1521 If the parameter file has not deployment in database action, then it is enough for the user to provide. Therefore, this option will work for Validation, Generation, and Deployment into File Systems. batchservice_class.ksh owb34 owb34 sroychow-pc3 owbqa817 -paramfile batchservce.param Batch Service Command Line Scripts UNIX s shiphome/owb/bin/solaris/batchservice.ksh DOS s shiphome/owb/bin/win32/batchservice.ksh Batch Service Command Line Output The output for Validation, Generation, Deployment are stored in their respective log files. Note: The keywords are in lowercase, and there is no dash in front of the keyword like in the command line option. Batch Service Command Line Feature Batch Services E-11 The log file has the following typical structure. The messages are shown in italics. The action and the date of activity are highlighted in bold. Action = validate Date = Mon Apr 02 14:30:26 GMT-08:00 2001 project = im1, element = tab1, type = TABLE, "ERROR", "No Columns", "Add new Column" Validation Examples Example 1 BEGIN ----------------------------------------------- Input: owb34 owb34 sroychow-pc3 owbqa817 1521 -validate My_^Project im1 tab1 TABLE Output: directory = /owb/shiphome/owb/bin/im1_tab1_validate.log Action = validate Date = Mon Apr 02 14:30:26 GMT-08:00 2001 project = im1, element = tab1, type = TABLE, "ERROR", "No Columns", "Add new Column" END ----------------------------------------------- Example 2 BEGIN ----------------------------------------------- Input: owb34 owb34 sroychow-pc3 owbqa817 1521 -validate My_^Project im1 dim1 DIMENSION Output: directory = /owb/shiphome/owb/bin/im1_dim1_validate.log Action = validate Date = Mon Apr 02 14:30:39 GMT-08:00 2001 project = im1, element = dim1, type = DIMENSION, "SUCCESS", "validation success", "validation success" END ----------------------------------------------- Note: The generic structure of the log files is defined as: ApplicationName_FCOName_ACTION.log Batch Service Command Line Feature E-12 Oracle Warehouse Builder 3i Users Guide Metadata Import Examples Example 1 BEGIN ----------------------------------------------- Input: owb34 owb34 sroychow-pc3 owbqa817 1521 -import /m/whdev/2.0/owb/bin/MyPPP.mdl /m/whdev/2.0/owb/bin/log.log CREATE true Output: Import file =/m/whdev/2.0/owb/bin/MyPPP.mdl Log file =/m/whdev/2.0/owb/bin/log.log Mode =CREATE Use Physical Names =true 5 Percent Complete 10 Percent Complete 15 Percent Complete 20 Percent Complete 25 Percent Complete 30 Percent Complete 35 Percent Complete 40 Percent Complete 44 Percent Complete 49 Percent Complete 54 Percent Complete 60 Percent Complete 65 Percent Complete 70 Percent Complete 75 Percent Complete 80 Percent Complete 85 Percent Complete 90 Percent Complete 95 Percent Complete Import successful. END ----------------------------------------------- Example 2 BEGIN ----------------------------------------------- Input: owb34 owb34 sroychow-pc3 owbqa817 1521 -import /m/whdev/2.0/owb/bin/my_ project-20010330_1938.mdl /m/whdev/2.0/owb/bin/log.log CREATE true Output: Import file = /m/whdev/2.0/owb/bin/my_project-20010330_1938.mdl Batch Service Command Line Feature Batch Services E-13 Log file =/m/whdev/2.0/owb/bin/log.log Mode =CREATE Use Physical Names =true 5 Percent Complete 10 Percent Complete 15 Percent Complete 20 Percent Complete 25 Percent Complete 30 Percent Complete 35 Percent Complete 40 Percent Complete 44 Percent Complete 49 Percent Complete 54 Percent Complete 60 Percent Complete 65 Percent Complete 70 Percent Complete 75 Percent Complete 80 Percent Complete 85 Percent Complete 90 Percent Complete 95 Percent Complete Import successful. END ----------------------------------------------- Example 3 BEGIN ----------------------------------------------- Input: owb34 owb34 sroychow-pc3 owbqa817 1521 -import /m/whdev/2.0/owb/bin/doesNotExist.mdl /m/whdev/2.0/owb/bin/log.log CREATE true Output: Import file =/m/whdev/2.0/owb/bin/doesNotExist.mdl Log file =/m/whdev/2.0/owb/bin/log.log Mode =CREATE Use Physical Names =true ************************************************************* Error: MDL-1122 Import data file /m/whdev/2.0/owb/bin/doesNotExist.mdl not found ************************************************************* ************************************************************* Error: java.lang.NullPointerException ************************************************************* Batch Service Command Line Feature E-14 Oracle Warehouse Builder 3i Users Guide Error at line 0: MDL-2007 Error during import abort processing: java.io.IOException: Stream closed END ----------------------------------------------- Generation Example BEGIN ----------------------------------------------- Input: owb34 owb34 sroychow-pc3 owbqa817 1521 -generate My_^Project im1 dim1 DIMENSION Output: directory = /owb/shiphome/owb/bin/admin/im1_dim1_generate.log Action = generate Date = Mon Apr 02 14:31:35 GMT-08:00 2001 project = im1, element = dim1, type = DIMENSION, ERROR Generation oracle.wh.util.Assert: Cannot get integrator map END ----------------------------------------------- Deployment into File System Examples BEGIN ----------------------------------------------- Input: owb34 owb34 sroychow-pc3 owbqa817 1521 -deployfile My_^Project im1 dim1 DIMENSION /tmp/.sroychow Output directory = /owb/shiphome/owb/bin/admin/im1_dim1_deployFS.log Action = deployFS Date = Mon Apr 02 14:31:47 GMT-08:00 2001 project = im1, element = dim1, type = DIMENSION, ERROR Deployment oracle.wh.util.Assert: Cannot get integrator map END ----------------------------------------------- Deployment in to a Database Example Example 1 BEGIN ----------------------------------------------- Batch Service Command Line Feature Batch Services E-15 Input: owb34 owb34 sroychow-pc3 owbqa817 1521 -deploydb My_^Project im1 dim1 DIMENSION sroychow sroychow sroychow-pc3 ora817 1521 Output: directory = /owb/shiphome/owb/bin/admin/im1_dim1_deployDB.log Action = deployDB Date = Mon Apr 02 14:32:12 GMT-08:00 2001 project = im1, element = dim1, type = DIMENSION, ERROR Deployment oracle.wh.util.Assert: Cannot get integrator map END ----------------------------------------------- Example 2 BEGIN ----------------------------------------------- Input: owb34 owb34 sroychow-pc3 owbqa817 1521 -deploydb My_^Project im1 dim1 DIMENSION customerDB customerDB sroychow-pc3 ora817 1521 Output: directory = /owb/shiphome/owb/bin/admin/im1_dim1_deployDB.log END ----------------------------------------------- Batch Service Command Line Feature E-16 Oracle Warehouse Builder 3i Users Guide Performance Enhancements F-1 F Performance Enhancements This appendix contains information about partition exchange loading, which is available during the physical configuration of the warehouse. These are the sections of contained in this appendix: s Overview s When to Use Partition Exchange Loading s Using Partition Exchange Loading s Restrictions s Performance Comparisons s The Internal Workings s Function Example 1: GET_PN s Function Example 2: GET_VC Overview There are many reasons for creating and maintaining data warehouses. One of them is the expected high performance that a data warehouse could bring to business applications, such as DSS, OLAP, and Data Mining. These applications share a salient characteristic: they often need to query a large amount of historical data organized along a time dimension. The horizontal data partitioning technology, available since Oracle 8 TM , can effectively help the queries cut down the amount of data that have to be processed by many levels of magnitude. This is done through an automatic mechanism in the Oracle SQL optimizer known as partition pruning. Data partitioning is also an enables the backup and/ or removal of dormant data that has aged out over time. Overview F-2 Oracle Warehouse Builder 3i Users Guide The way data is partitioned is a key data warehouse design issue, which must be considered after the logical design of a data warehouse. Oracle Warehouse Builder 3i provides assistance in partitioning various physical objects, including tables, dimensions, facts, materialized views, and indexes. Starting with OWB3i, design for data partitioning has one added importance: it can also be leveraged within a certain context to achieve significant performance gains in the process of loading a data warehouse. This new OWB runtime performance feature is known as Partition Exchange Loading (PEL). PEL technique performs loading of new data by exchanging them into a target table as a partition. What actually get exchanged are merely identities, in the following manner. The table that holds the new data takes over the identity of one empty partition from the target table; at the same time, this empty partition assumes the identity of the source table. The whole exchange process is purely a DDL operation. No data movement is involved. The general idea of PEL is depicted in Figure F1. Figure F1 Fundamental Concept of Partition Exchange In Figure F1, data from the Source table needs to be inserted into a target table consisting of four partitions: Target_P1, Target_P2, Target_P3, and Target_P4. Suppose that the new data needs to be loaded into Target_P3. The partition exchange operation will swap the names on the data objects. The actual data never moves. After the exchange, the Source table is renamed Target_P3, and the Target_P3 is renamed Source. Most importantly, the target table is still composed of four partitions named Target_P1, Target_P2, Target_P3, and Target_P4. The partition exchange operation provided by Oracle8 TM completes the loading process without generating the need for data movement. Figure F1 shows a highly simplified case where only one table is presented as a source, and there is an implicit assumption that this source table is readily available for changing its identity. Being a general data warehouse design tool, OWB operations are not this simple. OWB must be prepared for the more general Overview Performance Enhancements F-3 possibility, that the source may be the results from a join of multiple source tables. Some of these source tables may also be located in other databases. In general, there may not be a single local table for partition exchange. The design decision in OWB3i is to automatically create a temporary table which materializes results from source processing before the partition exchange actually happens. Will the materialization of results from source processing hurt performance? Will it render the PEL useless since data movement (i.e., DML) has not been avoided? In the case that there is no single local table for partition exchange, it is quite obvious that avoiding DML entirely is not possible. As such, the PELs performance potential can only be gauged by an examination of how it does DMLs, and specifically, the INSERTs. The remainder of this section presents a brief explanation of how PEL works. This should answer the above questions. Getting familiar with the technical background will be very important to you if you plan to use the Partition Exchange Loading feature. Overhead-Free INSERT When you use PEL, a new temporary or staging table is automatically created to hold results from source processing. Because this staging table is purely internal to the PEL process and invisible to general users, the PEL process does not create any indexes and constraints before the INSERT operation is completed. This measure can drastically reduce the amount of data that need to be processed due to index maintenance and foreign key lookups. In contrast, using the normal non-PEL style processing requires data to be directly inserted into the target that often has to carry several huge indexes and many constraints. Of course, you could choose to drop all the indexes and disable all the constraints before loading, but when the target has accumulated enough historical data, recreating indexes and re-enabling constraints after loading may be a very costly decision. Fast Index Creation After new data has been loaded into the temporary table, the PEL method creates all the indexes on the temporary table only. The index creations benefit from a parallel index creation mechanism provided by the Oracle database server. It also uses the NOLOGGING option to further speed up the index building process. Using the PARALLEL NOLOGGING options, index creations generate writes for only the index structures, and I/ O conflicts are minimized. As the result, the index creation time is minimized. Overview F-4 Oracle Warehouse Builder 3i Users Guide Fast Constraint Maintenance After indexes have been created on the temporary table, primary and unique keys are instantly enabled and validated a by using the USING INDEX option of the ALTER TABLE ADD CONSTRAINT command. Foreign key constraints are maintained by making use of a new parallel validation feature in Oracle8i. This is done by first adding a foreign key constraint in ENABLED NOVALIDATE state. Then, a subsequent command does the real foreign key validation in parallel. Adding a foreign key with ENABLED NOVALIDATE status does not hinder performance. The subsequent parallel constraint validation can fully utilize system resources. The overall design is an optimal constraint maintenance mechanism. In contrast, if the PEL is not used, it can be an almost impossible task to disable and re-enable existing constraints. Suppose, for example, 500,000 rows of new data generated from one-day activities have been inserted into a target table already containing 500 million rows of data; and one primary constraint and one foreign key constraint were disabled before loading and they must now be re-enabled. This whole process amounts to scanning more than 500 million rows of data twice, plus building a unique index. It could be hundreds of times slower than if only the new data was manipulated as in PEL. Error Handling Whenever an error is encountered during inserting into the temporary table, building indexes, or enabling constraints, the temporary table is dropped and the error logged. The target is not altered. Data in the target is still in its previous state. Minimum Target Locking During the entire PEL loading process, except for the final partition exchange step (which normally takes almost no time because it is a pure DDL operation), the target table is always available for any database operations, including any DML operations. Parallel Direct-Path INSERT In PEL, insertion into the temporary table is always done using parallel direct-path INSERT. On a multiprocessor machine with a correctly striped tablespace, more than 90% utilization on each CPU can be consistently achieved. The insertion completes in the shortest possible time. When to Use Partition Exchange Loading Performance Enhancements F-5 In contrast, if the PEL method is not used, several factors can hinder the loading process from fully utilizing the CPU resource. First, index maintenance can generate more I/ O requests and reduce CPU utilization. (The PEL technique would have no index maintenance during insert.) Second, when loading into a partitioned table, a more serious issue results from the fact that Oracle 8i assigns only one server process to each partition. If all data must go into same partition as is true for incremental loading of daily or monthly data, the loading process runs serially, as shown in Figure F2. Figure F2 One Server Process Assigned to Each Partition A concrete measurement can help understand the performance impact of incremental loading into only one partition of a partitioned table. On a Sun Ultra 450 server with four 400Mhz CPUs, 6 million rows from one daily collection are inserted into a target table that is partitioned by day. The completion time is 3 minutes 9 seconds. But when the same amount of data is inserted into a non-partitioned table using parallel direct-path INSERT, it completes in 1 minute 48 seconds. So the serial insert in the partitioned case lost roughly two times of performance on the 4-processor machine. The PEL method solves this problem by always performing parallel direct-path INSERT into a non-partitioned table. When to Use Partition Exchange Loading Like all performance features, the PEL technique is advantageous only in a certain context, and can be deemed not useful in some other situations. The foremost condition required by the PEL is that the target table must be partitioned. Another condition under which the PEL could provide greatest performance benefit is if target table has already accumulated a huge amount of historical data. An example is a clickstream-like application where the warehouse must collect data from an OLTP database or Web When to Use Partition Exchange Loading F-6 Oracle Warehouse Builder 3i Users Guide log files daily. Each row of source data is about one mouse click from a Web user. Depending on the popularity of the site, as much as several million rows per day could be generated. These data then need to be transformed and loaded into a data warehouse holding a lot more historical data, e.g., from the past five years. For this type of situations, the PEL will be the key to scalable loading performance that makes the loading time correlated only to the amount of new data. A third condition under which the PEL can help greatly is when all the new data must be loaded into the same partition in a target table, as shown in Figure F2. If the data warehouse collection is scheduled on a regular basis, this condition can normally be satisfied. Again, take the clickstream warehouse as an example. If the target table is partitioned by day, then all the daily data will have to be loaded into one partition. The non-PEL loading will go serially, but the PEL will be able to run parallel direct-path INSERT. In case the target table is partitioned by month and the collection of new clickstream data is still scheduled daily, new data may have to be inserted into a nonempty partition. In case the target partition is not empty, the PEL process will not actually perform partition exchange. It will directly insert new data into one explicitly specified partition, still using parallel direct-path INSERT. The drawback on performance is that the insertion is accompanied by simultaneous index and constraint maintenance. Therefore, using the PEL method to load data into a nonempty partition will go slower than if the target partition is empty. Still a third case in our clickstream example: assume the target table is partitioned by hour and new data is still collected daily. New data collected from one day will be able to go into separate partitions in parallel. In this case, the PEL is not useful anymore. In fact, in OWB 3i, the PEL does not work for such a case. Table 1 below summarizes these three possibilities. Table F1 Loading Size vs. Partition Granularity Considerations SITUATION PERFORMANCE Loading size = Partition Granule (For example, load daily and partition by day.) Highest performance. Loading size < Partition Granule (For example, load daily and partition by month.) High performance but slower than previous case. Using Partition Exchange Loading Performance Enhancements F-7 As Table F1 shows, the PEL method can be used in the case that the loading size is less than or equal to the partition granularity of the target. In OWB 3i, you will be able to specify the target partition granularity. But there is currently no validation to check if the loading size is not larger than the partition granularity. Using Partition Exchange Loading This section presents a step-by-step example showing how the Partition Exchange Loading feature can be used in OWB 3i. Two areas need configuration: the mapping and the target table, dimension, or fact. This section first describes an example mapping, then explains what needs to be configured in the mapping and the target. Figure F3 Example: The GET_SALES_SUMMARY Mapping Suppose it has been decided that all new data to ORDER_SUMMARY table will always go into same partition, and most times they go into an empty partition. Partition Exchange Loading into the ORDER_SUMMARY table appears to be a good strategy under these conditions. Loading size > Partition Granule (For example, load daily and partition by hour.) Does not work in OWB3i using PEL. Table F1 Loading Size vs. Partition Granularity Considerations SITUATION PERFORMANCE Using Partition Exchange Loading F-8 Oracle Warehouse Builder 3i Users Guide Configuring the Mapping Figure F4 Mapping Property Inspector for Mapping GET_ORDER_SUMMARY In order to configure the mapping to use PEL, open the Properties inspector. The Properties inspector for the GET_SALES_SUMMARY is shown above in Figure F4. In the property inspector window and within the Operators category, each of the sources and the target starts a property group. Expand the property group for ORDER_SUMMARY as in Figure F4. You will see a subgroup labeled Partition Exchange Loading. Within this group, two properties must be configured. The first is a Boolean property named Use PEL. Set to true for this property. The other property is named Partition Granularity. This property tells OWBs PL/ SQL code generator how the target table (ORDER_SUMMARY) is partitioned. There are seven levels of partition granularity you can choose from, as shown in Figure F4. Suppose the ORDER_SUMMARY table is partitioned by day. We then select Day from the drop-down list as the value for the Partition Granularity property. And that is all we need to do on the mapping in order to enable the Partition Exchange Loading feature. Once the PEL is enabled, the OWB code generator will generate different kind of batch processing code. Using Partition Exchange Loading Performance Enhancements F-9 Configuring the Target After the mapping is configured, there is still one more area of configuration to be done. This second area of configuration is for the target. It turns out that the process for configuring the target is a lot more complicated than the process for configuring the mapping. To configuring the target, follow these steps: 1. Create All Partitions 2. Create All Indexes Using the LOCAL option 3. Specify the USING INDEX Option for Primary/ Unique Keys Create All Partitions All the partitions must be created before the PEL method can be used. OWB users can use the table/ dimension/ fact inspector to add partitions and to modify their properties. After a partitioned object has been deployed into database, the DBA can also add more partitions or to merge several partitions together. In the following explanation about the Step 1, only the instructions on how to use OWB to create partitioned table is given. On the possible DBA operations, please refer to Oracle8i Server Administration Guide. In our example, since we selected Day as partition granularity for ORDER_ SUMMARY table when configuring the mapping, we need to create all the needed daily partitions, one for each day of new data. To create partitions for a table, dimension, or fact, use its property inspector. Figure F5 below shows the property inspector window for table ORDER_SUMMARY. This figure shows that six partitions have been added for this table. (The instructions on how these partitions are added are skipped.) Notice the naming of all the partitions in Figure F5. Although OWB users must manually key in those names when they use OWB to add partitions, all partition names must follow a strict naming convention. For example, for the partition that will hold data for May 3, 2001, its name must be Y2001_Q2_M05_W1_D03. Using Partition Exchange Loading F-10 Oracle Warehouse Builder 3i Users Guide Figure F5 Property Inspector for Table ORDER_SUMMARY The partition naming convention is in fact very straightforward. For example, the partition name Y2001_Q2_M05_W1_D03 corresponds to Year 2001, Quarter 2, Month 5, Week 1, and Day 3, which is exactly May 3, 2001. But can the same partition be simply named Y2001_M05_D03 without the quarter and week information? The answer is no. The partition name must contain information of all the time levels leading to the intended granularity. The detailed naming scheme is designed for ease of partition administration later. If the DBA decides to merge all the daily partitions into monthly partitions, he/ she can easily find which partitions to merge together based on their names. For example, all the daily partitions for May 2001 are named like Y2001_Q2_M05*. (Here the asterisk means a wildcard matching a string of characters.) They can be merged together to form a renamed partition Y2001_Q2_M05. Furthermore, this naming scheme helps create a consistency between the partitions name ordering and value ordering. That is, if the partitions are sorted based on the partition names, the partitions holding smaller DATE values will appear earlier. However, to many people, figuring out to which quarter and week a date belongs, as is required by the naming convention, is not as immediate as figuring out its day, Using Partition Exchange Loading Performance Enhancements F-11 month, and year components. This may lead to errors during partition creations. If partitions are created with errors, it will be extremely difficult to correct them later. To help eliminate errors during partition creations, you may find the two PL/ SQL functions toward the end of this appendix very useful. "Function Example 1: GET_PN" on page F-18 contains source code for a stand-alone PL/ SQL function named get_pn, which stands for get partition name. It can help OWB users calculate partition name from a given date value. For example, if we want to find the correct partition name for the partition containing value May 3, 2001, and we want to partition target table by day, the correct partition name can be obtained by issuing a SQL statement that calls the get_pn function in the following manner. SQL> SELECT get_pn(TO_DATE(03-MAY-2001,DD-MON-YYYY), 2 DAY) partition_name 3 FROM DUAL; PARTITION_NAME --------------------- Y2001_Q2_M05_W1_D03 The first argument is a date value, and the second argument is the partition granularity. The function returns the correct partition name under the intended partition granularity for the partition containing the date value. The above example shows that the correct partition name for the partition containing May 3, 2001 is Y2001_Q2_M05_W1_D03. If, however, the second argument to the get_pn function was Month instead of Day, the partition name result would be Y2001_Q2_M05. After all the partitions are added with correct names, the partitions must be further configured for their VALUE LESS THAN properties as shown in Figure F6 below. Figure F6 shows that the VALUE LESS THAN property for partition Y2001_Q2_M05_W1_D08 is being configured. The property value is TO_ DATE(09-05-2001,DD-MM-YYYY). Using the name and the VALUE LESS THAN property, OWB will generate a DDL script for creating the partitioned table. Part of the DDL script related to the partition shown in Figure F6 will read as follows. . . . PARTITION Y2001_Q2_M05_W1_D08 VALUES LESS THAN (TO_DATE(09-05-2001,DD-MM-YYYY)), . . . Using Partition Exchange Loading F-12 Oracle Warehouse Builder 3i Users Guide Figure F6 Specifying Value Less Than Property for a Partition The get_vc function is used in a similar manner as get_pn. First, find any value that would fall within the partition you are trying to calculate the VALUE LESS THAN property value. Then call the get_vc function as follows. SQL> SELECT get_vc(TO_DATE(08-MAY-2001,DD-MON-YYYY), 2 DAY) value_less_than 3 FROM DUAL; VALUE_LESS_THAN ----------------------------------- TO_DATE(09-05-2001,DD-MM-YYYY) Using Partition Exchange Loading Performance Enhancements F-13 Create All Indexes Using the LOCAL option Figure F7 Configure an Index as a Local Index Figure F7 above shows an index (ORDER_SUMMARY_PK) is added to ORDER_ SUMMARY table. This index has two columns, ORDER_DATE and ITEM_ID. The Index Type parameter is set to UNIQUE and the Local Index parameter is set to true. If other indexes existed on the ORDER_SUMMARY table, they should all be configured in this way, i.e., with the Local Index parameter set to true. Finally, be reminded that if an index is created as a local index, the partition key column must be the leading column of the index, or the Oracle server does not create the index. In Figure F7 above, the partition key is ORDER_DATE. It is the leading column in index ORDER_SUMMARY_PK. Specify the USING INDEX Option for Primary/Unique Keys The very last step in configuring the target is for specifying that all primary and unique key constraints are created with the USING INDEX option. Using the USING INDEX option, a constraint will not trigger automatic index creation when it is being added to the table. Instead, the server will look among existing Restrictions F-14 Oracle Warehouse Builder 3i Users Guide indexes for one with same column list as that of the constraint. This naturally implies that each primary or unique key constraint must be backed by a user-defined unique local index. Figure F8 below shows an example in which the primary key constraint on ORDER_SUMMARY table is specified with the USING INDEX option. The index that supports this option was created in Step 2 earlier. Figure F8 Specify a Constraint with USING INDEX option Indexes and constraints can share the same name without problem, and it is customary to create a constraint and its underlying index with the same name as shown in Figure F7 and Figure F8. Restrictions There are several restrictions that OWB users need to be aware of. These restrictions are necessary to minimize the complexity of implementation in the debut release of the PEL feature. Some of the restrictions might be lifted in future releases. Performance Comparisons Performance Enhancements F-15 Restriction 1: Allow One Date Partition Key Currently, only one partition key column is allowed, and this column must be of DATE data type. Numeric partition key is not supported in OWB 3i. This will most likely be enhanced in near future releases. Restriction 2: Allow Natural Calendar System Only The current PEL method supports only the natural calendar system adopted worldwide. Specific business calendar systems with user-defined fiscal and quarter endings are currently not supported. But this support will most likely be added in near future releases. Restriction 3: All Data Partitions Must be in the Same Tablespace The current release requires that all partitions of a target (table, dimension, or fact) must be created in the same tablespace. This requirement is needed for eliminating user intervention during the PEL process. Otherwise, users must pass a runtime parameter to the PEL mapping to specify the target tablespace. Restriction 4: All Index Partitions Must be in the Same Tablespace This requirement is established for the same purpose as the restriction 3. Users are thus not required to pass a runtime parameter specifying the index tablespace. However, the index tablespace can be different from the data tablespace. Performance Comparisons There can be three possible methods for doing incremental loading into a target, which cumulates historical data. This type of target is usually a fact table. The three loading methods are s Method 1: Use the PEL method and load into a partitioned target table. s Method 2: Do not use the PEL method and load into a partitioned table. s Method 3: Do not use the PEL method and load into a non-partitioned table. The qualitative performance advantages of PEL can clearly be seen in the section on General Feature Description. This section demonstrates the quantitative performance advantage of the PEL over the other two incremental loading methods. The Internal Workings F-16 Oracle Warehouse Builder 3i Users Guide An experiment is performed on a four-processor Sun Ultra 450 server. Each CPU has a speed of 400MHz. The memory size is 2GB. There are twenty 9.1GB internal disk drives. The database is created using Oracle8i Release 3. The experiment uses each of the three loading methods to do exactly the same loading operations. The loading operations consist of a series of inserts into a target table. Each insert is for loading one-day worth of new data, with a volume of 500,000 rows. In this case the target is partitioned (by day), and each batch goes into one partition. The most important aspect about the target is that there is an index created on it. The index is built on four numeric columns and one DATE column. If the target is partitioned (as is true in Methods 1 and 2), the index is also local. In this setting, Method 1 (the PEL) will have advantage over other two methods in index maintenance. Method 1 will also have advantage over Method 2 in doing parallel direct-path insert. Method 2, which employs partitions for index maintenance, will be able to avoid having to traverse in the global index structure that can grow larger as more data are loaded. The results confirmed the qualitative analysis discussed so far. Method 3 (no PEL on non-partitioned table) is clearly not a good choice. (Imagine what the loading time would be after one year!) Method 2 does maintain independence of load time from already loaded data, but it is about three times slower than the PEL. The lack of parallelism is believed to be the main culprit for the loss of performance in Method 2. linally, thc comparison bctwccn Mcthods 2 and clcarly dcmonstratcs how much wc can savc ii wc can rcducc thc ovcrhcad in indc maintcnancc. The Internal Workings This section is presented only for the purpose of helping OWB users better understand how the Partition Exchange Loading works so that they can better plan and use this feature. The explanation below should not be taken as a recipe for custom warehouse construction because the method reported herein might be subject to patent protection, and Oracle only supports the PEL-style PL/ SQL code that is generated from OWB designer without modifications. The PEL technique is a set-based processing method (or generally called batch processing method in the data warehouse field). Therefore, to actually run a mapping with the PEL technique, the processing mode must be set to set-based or set-based with fail-over. The Internal Workings Performance Enhancements F-17 When a mapping using the PEL technique is run, the following things happen, in this order. Step 1: Find targets tablespace name. If the target has more than one tablespace, the loading process is aborted with an error message like the following. Fatal error: all partitions of target table RT_OWNER.ORDER_SUMMARY must be in one tablespace Step 2 : Find the tablespace name for all indexes. If there are more than one tablespace name found, the loading process is aborted with an error message like the following. Fatal error: all partitions of local indexes on table RT_OWNER.ORDER_SUMMARY must be in one tablespace Step 3. Create a temporary table that has same schema definition as the target. Should this step fail, the loading process is aborted with the following error message. Fatal error: failed the attempt to create a temporary table like table RT_OWNER.ORDER_ SUMMARY Step 4 : Parallel direct-path load into the temporary table. If there is anything wrong in this step, the batch processing terminates with an Oracle server error, but the entire loading process is not aborted. The loading process may drop down to row-by-row auditing mode to find error details if the set-based with fail-over mode is selected. Step 5 : Find partition name from the first row of freshly loaded data. If this step fails, the following error message is logged in OWB audit trail. Fatal error: calculating partition name failed Step 6 : If the target partition is not empty, then parallel direct-path insert into the target partition. Oracle server errors may occur and be logged in OWB audit trail. But the loading process is not aborted. It may drop down to row-by-row processing if user so configured the mapping. No matter if errors occur, the mapping terminates. Step 7 : If the target partition is empty, all indexes are created on the temporary table. If errors occur in this step, the loading process is not aborted. The row-based process may be invoked if user selected set-based with fail-over mode. Step 8 : Add all constraints, including primary/ unique key constraints and foreign key constraints. If errors occur in this step, the loading process is not aborted. The row-based process may be invoked if user selected set-based with fail-over mode. Function Example 1: GET_PN F-18 Oracle Warehouse Builder 3i Users Guide Step 9 : Exchange partition. If error occurs, the loading process is aborted with the following error message. Fatal error: failed exchanging partition Y2001_Q2_M05_W1_D03 in target RT_OWNER.ORDER_ SUMMARY Step 10. Drop the temporary table. If any error occurs, a warning is logged in OWB audit trail, but the mapping is still considered being successful. The warning looks like the following. Warning: cannot drop table RT_OWNER.T53 Function Example 1: GET_PN Deploy this function in any Oracle database (version 7 or later). CREATE OR REPLACE FUNCTION get_pn (p_date IN DATE, p_granularity IN VARCHAR2) RETURN VARCHAR2 IS x_pn VARCHAR2(30); x_granularity VARCHAR2(30) := UPPER(p_granularity); BEGIN x_pn := Y||TO_CHAR(p_date, YYYY); IF x_granularity != YEAR THEN x_pn := x_pn||_Q||TO_CHAR(p_date,Q); IF x_granularity != QUARTER THEN x_pn := x_pn||_M||TO_CHAR(p_date,MM); IF p_granularity != MONTH THEN x_pn := x_pn||_W||TO_CHAR(p_date,W); IF p_granularity != WEEK THEN x_pn := x_pn||_D||TO_CHAR(p_date,DD); IF p_granularity != DAY THEN x_pn := x_pn||_H||TO_CHAR(p_date,HH24); IF p_granularity != HOUR THEN x_pn := x_pn||_M||TO_CHAR(p_date,MI); END IF; END IF; END IF; END IF; END IF; END IF; RETURN x_pn; END get_pn; / Function Example 2: GET_VC Performance Enhancements F-19 Function Example 2: GET_VC Deploy this function in any Oracle database (version 7 or later). CREATE OR REPLACE FUNCTION get_vc (p_date IN DATE, p_granularity IN VARCHAR2 DEFAULT DAY) RETURN VARCHAR2 IS x_vc VARCHAR2(80); x_granularity VARCHAR2(30) := UPPER(p_granularity); x_year NUMBER; x_quarter NUMBER; x_month NUMBER; x_day NUMBER; x_hour NUMBER; x_minute NUMBER; BEGIN IF x_granularity NOT IN (YEAR, QUARTER, MONTH, WEEK, DAY, HOUR, MINUTE) THEN x_granularity := DAY; END IF; x_year := TO_NUMBER(TO_CHAR(p_date, YYYY)); IF x_granularity = YEAR THEN x_year := x_year + 1; x_vc := TO_DATE(01-01-||x_year||,DD-MM-YYYY); ELSE x_quarter := TO_NUMBER(TO_CHAR(p_date, Q)); IF x_granularity = QUARTER THEN x_quarter := x_quarter + 1; IF x_quarter = 5 THEN x_year := x_year + 1; x_quarter := 1; END IF; IF x_quarter = 1 THEN x_vc := TO_DATE(01-01-||x_year||,DD-MM-YYYY); ELSIF x_quarter = 2 THEN x_vc := TO_DATE(01-04-||x_year||,DD-MM-YYYY); ELSIF x_quarter = 3 THEN x_vc := TO_DATE(01-07-||x_year||,DD-MM-YYYY); ELSE Function Example 2: GET_VC F-20 Oracle Warehouse Builder 3i Users Guide x_vc := TO_DATE(01-10-||x_year||,DD-MM-YYYY); END IF; ELSE x_month := TO_NUMBER(TO_CHAR(p_date, MM)); IF x_granularity = MONTH THEN x_month := x_month + 1; IF x_month = 13 THEN x_year := x_year + 1; x_month := 1; END IF; x_vc := TO_DATE(01-||TO_CHAR(x_month,09)||-||x_year|| ,DD- MM-YYYY); ELSE IF x_granularity = WEEK THEN IF p_date+7 > LAST_DAY(p_date) THEN x_day := 1; x_month := x_month + 1; IF x_month = 13 THEN x_year := x_year + 1; x_month := 1; END IF; ELSE x_day := (TRUNC(TO_NUMBER(TO_CHAR(p_date,DD))/7)+1)*7+1; END IF; x_vc := TO_DATE(||TO_CHAR(x_day,09)||-|| TO_CHAR(x_month,09)||-||x_year|| ,DD- MM-YYYY); ELSE IF x_granularity = DAY THEN x_vc := TO_DATE(||TO_CHAR(p_date+1,DD-MM-YYYY)|| ,DD-MM-YYYY); ELSE x_day := TO_NUMBER(TO_CHAR(p_date,DD)); IF x_granularity = HOUR THEN x_hour := TO_NUMBER(TO_CHAR(p_date,HH24)); IF x_hour = 23 THEN x_vc := TO_DATE(||TO_CHAR(p_date+1,DD-MM-YYYY)|| ,DD-MM-YYYY); ELSE x_hour := x_hour + 1; x_vc := TO_DATE(||TO_CHAR(p_date,DD-MM-YYYY)|| TO_CHAR(x_hour,09)|| ,DD-MM-YYYY HH24); END IF; ELSE Function Example 2: GET_VC Performance Enhancements F-21 x_minute := TO_NUMBER(TO_CHAR(p_date,MI)); IF x_minute = 59 THEN IF x_hour = 23 THEN x_vc := TO_DATE(||TO_CHAR(p_date+1,DD-MM-YYYY)|| ,DD-MM-YYYY); ELSE x_hour := x_hour + 1; x_vc := TO_DATE(||TO_CHAR(p_date,DD-MM-YYYY)|| TO_CHAR(x_hour,09)|| ,DD-MM-YYYY HH24); END IF; ELSE x_minute := x_minute + 1; x_vc := TO_DATE(||TO_CHAR(p_date,DD-MM-YYYY HH24:)|| TO_CHAR(x_minute,09)|| ,DD-MM-YYYY HH24: MI); END IF; END IF; END IF; END IF; END IF; END IF; END IF; RETURN x_vc; END get_vc; / Function Example 2: GET_VC F-22 Oracle Warehouse Builder 3i Users Guide OWB Bridges: Transfer Parameters and Considerations G-1 G OWB Bridges: Transfer Parameters and Considerations Transfer Parameters The OWB Transfer Wizard allows you to export metadata from data warehousing tools and file systems, such as OMG CWM, CA ERwin and Powersoft PowerDesigner, and import the metadata into Oracle warehousing tools, such as Oracle Discoverer, and Oracle Express. When you use the OWB Transfer Wizard, you are required to enter transfer parameter information based upon the metadata source or target you have selected. The following tables list details about the transfer parameters you need to provide for each data source or target type. Table G1 Transfer Parameters for OMG CWM FIle Import Transfer Parameter Name Description OMG CWM Input File Name of the file where you want to store the imported metadata in the OWB repository. Click ... to browse for the file location. OWB Project OWB name for the imported projectthis project will be located under the Projects node within OWB. Name Matching Mode Name matching mode you want to utilize during the import; physical name mode is the default. Import Mode Import modecreate, replace, update, incremental update. Transfer Parameters G-2 Oracle Warehouse Builder 3i Users Guide Table G2 Transfer Parameters for Importing from CA ERwin Log Level Log level to be assigned to the transfer: Errors, Information, and Trace. Errorlist the errors, if any, generated by the transfer. Informationlists the transfer details about the metadata objects, including any errors. Traceproduces a trace for debugging purposes, which also includes the errors and information log details. If you need assistance with the trace, contact Oracle World Wide Support. Trace type provides the most detailed log information. Transfer Parameter Name Description Input Model File Location of the ERwin Input file. Click ... to browse for the file location. OWB Project OWB name for the imported projectthis project will be located under the Projects node within OWB. OWB Model Name Name of the .erx file you are importing from ERwin. Name Matching Mode Name matching mode you want to utilize during the import. Physical name mode is the default. Import Mode Import modecreate, replace, update, incremental update. Log Level Log level to be assigned to the transfer: Errors, Information, and Trace. Errorlist the errors, if any, generated by the transfer. Informationlists the transfer details about the metadata objects, including any errors. Traceproduces a trace for debugging purposes, which also includes the errors and information log details. If you need assistance with the trace, contact Oracle World Wide Support. Trace type provides the most detailed log information. Transfer Parameter Name Description Transfer Parameters OWB Bridges: Transfer Parameters and Considerations G-3 Table G3 Transfer Parameters for importing from Powersoft PowerDesigner Table G4 Transfer Parameters when OMG CWM is selected as the target Transfer Parameter Name Description Input Model File Location of the PowerDesigner Input file. Click ... to browse for the file location. OWB Project OWB name for the newly imported Projectthis Project will be located under the Projects node within OWB. OWB Model Name Name of the model you are importing from ERwin into OWB. CDM Name Mapping Choose whether you want to use the logical (NAME) name or physical (CODE) name for the Conceptual Data Model (CDM). Default is NAME. PDM Name Mapping Choose whether you want to use the logical (NAME) name or physical (CODE) name for the Physical Data Model (CDM). Default is CODE. Submodel Mapping Name of the submodel you want to import into OWB. The default choice (*) imports all the sub models within the model Name Matching Mode Name matching mode you want to utilize during the import. Physical name mode is the default. Import Mode Import modecreate, replace, update, incremental update. Log Level Log level to be assigned to the transfer: Errors, Information, and Trace. Errorlist the errors, if any, generated by the transfer. Informationlists the transfer details about the metadata objects, including any errors. Traceproduces a trace for debugging purposes, which also includes the errors and information log details. If you need assistance with the trace, contact Oracle World Wide Support. Trace type provides the most detailed log information. Transfer Parameter Name Description OWB Exported Business Areas Select the Business Areas you want to export from OWB. The default is All Business Areas; you can select individual Business Areas from the drop-down list. Transfer Parameters G-4 Oracle Warehouse Builder 3i Users Guide Table G5 Transfer Parameters when Discoverer is selected as target OMG CWM Output File The path and filename for the OMG CWM output file. Log Level Enter the log level to be assigned to the transfer: Errors, Information, Trace. Errors produces just a list of errors, if any, generated by the transfer. Information lists the transfer details about the metadata objects, including any errors. Trace produces a trace for debugging purposes, which also includes the errors and information log details. If you need assistance with the trace, contact Oracle World Wide Support. . Transfer Parameter Name Parameter Description OWB Exported Business Areas Select the Business Areas you want to export out of Builder from the drop-down menu; the default is All Business Areas. Discoverer EUL Owner Name of the owner of the Discoverer EUL Discoverer Schema Owner Name of the owner of the Discoverer schema Dimensional Reuse True or False option for dimensional reuse (defined as two or more foreign key constraints in a fact that point to the same dimension). Select True if you changed the logical names of foreign key columns and you want them to appear as separate folders in Discoverer. Discoverer Output File Path and name of the Discoverer .EEX file (which will contain the metadata) that is generated by the Transfer Wizard to be imported into Discoverer Enter the path or select it by clicking Browse in the Transfer Parameter Value column of the Discoverer Output File parameter. Create a name of your choosing for the .EEX file and enter it at the end of the path. Transfer Parameter Name Description Transfer Parameters OWB Bridges: Transfer Parameters and Considerations G-5 Table G6 Transfer Parameters when Express is selected as target Log Level The log displays data about successful and unsuccessful transfers. The Transfer Wizard provides two additional log levels, Errors and Trace, to assist you in debugging. Errors produces just a list of errors, if any, generated by the transfer. Information lists the transfer details about the metadata objects, including any errors. Trace produces a trace for debugging purposes, which also includes the errors and information log details. If you need assistance with the trace, contact Oracle World Wide Support. Transfer Parameter Name Parameter Description OWB Exported Business Areas From the drop-down list, select the business areas you want to export out of OWB. The default is All Business Areas. Express User User id for connecting to the Express Repository Express User Password Password for the Express User Express Connect String Connecting odbc string for Express, using the following syntax: host_machine_name:port_number:database_sid Express Table Owner Name of the Express Table Owner RAA Version Number Relational Access Administrator version. Log Level The log displays data about successful and unsuccessful transfers. The Transfer Wizard provides two additional log levels, Errors and Trace, to assist you in debugging. Errors produces just a list of errors, if any, generated by the transfer. Information lists the transfer details about the metadata objects, including any errors. Trace produces a trace for debugging purposes, which also includes the errors and information log details. If you need assistance with the trace, contact Oracle World Wide Support. Transfer Parameter Name Parameter Description Transfer Considerations G-6 Oracle Warehouse Builder 3i Users Guide Transfer Considerations Before you transfer metadata into the OWB repository or out of the OWB repository, you need to perform tasks within the source and target tools to ensure that the metadata can be transferred successfully. This Appendix provides instructions for preparing to transfer metadata from a source tool to the OWB repository or from the OWB repository to a target tool. It also lists the objects that are extracted from the source tool during the transfer and their corresponding OWB objects. The OWB Transfer Wizard exports metadata from the OWB repository as Business Areas. Before you export metadata, you must create business areas within the OWB repository. This section includes the following topics: s Creating Business Areas in OWB s Importing Metadata from an Object Management Group (OMG) CWM Standard System s Importing Metadata from Computer Associates ERwin 3.5.1 s Importing Metadata from Powersoft PowerDesigner 6.0 s Exporting Metadata to Oracle Discoverer 3.1 and 4i s Exporting Metadata to an Object Management Group (OMG) CWM Standard System s Exporting Metadata to Oracle Express Creating Business Areas in OWB You must organize your OWB repository objects into one or more business areas before you can export them from OWB to other target tools using the OWB Transfer Wizard. To create business areas within OWB: 1. Open OWB. 2. From the main OWB navigator tree, expand the project that includes the warehouse target module from which you plan to export its metadata. 3. Double-click the appropriate warehouse target module (for example, WH). The Warehouse Module Editor window displays. Creating Business Areas in OWB OWB Bridges: Transfer Parameters and Considerations G-7 4. Select the Business Tree tab. 5. Right-click in the tree area to open the pop-up menu and select Create Business Area. The Business Area Dialog, Figure G1, displays. 6. In the Name field, enter a name for the business area you want to create (for example, Orders). 7. In the Description field, appropriately describe the new business area. Figure G1 Business Area Dialog with All Dimensions and Facts Selected Creating Business Areas in OWB G-8 Oracle Warehouse Builder 3i Users Guide Figure G2 Business Area Dialog for a Single Fact 8. In the lower section of the dialog, Please Select Objects, do one of the following: s To include all the facts, dimensions, tables and/ or views in the module in the business area, click (check mark) Fact, Dimension, Table, and/ or View in the Selected column, Figure G1 on page G-7. s To include specific facts, dimensions, tables, and views from the module in the business area, expand the fact, dimension, table, or view in the Type column and then select the desired facts (Figure G2), dimensions, tables, or views to be added to the business area. (For example, you may want to create individual business areas for each fact and its dimensions.) An 9. Click OK to return to the Warehouse Module Editor. The new business area now displays in the warehouse target module, Figure G3 on page G-9. Importing Metadata from an Object Management Group (OMG) CWM Standard System OWB Bridges: Transfer Parameters and Considerations G-9 Figure G3 Warehouse Module Editor with New Business Area 10. To create additional business areas for the module, repeat Steps 19. 11. Commit your changes. After you complete this task and any other applicable tasks in this chapter, you are ready to transfer the metadata to Discoverer. Importing Metadata from an Object Management Group (OMG) CWM Standard System While importing a file that conforms to the OMG CWM standard into the OWB repository, you can refer to the following terminology matrix to check how objects in OMG CWM file system are mapped in the OWB repository. Table G7 Object Conversion for Import from OMG CWM into OWB Object in OMG CWM Imported into OWB Package Project Schema Module Importing Metadata from Computer Associates ERwin 3.5.1 G-10 Oracle Warehouse Builder 3i Users Guide Importing Metadata from Computer Associates ERwin 3.5.1 While importing a file from ERwin into the OWB repository, save the file containing the metadata for transfer as an .ERX file. Additionally, you may refer to the following terminology matrix to check how objects in ERwin are mapped in the OWB repository. Importing Metadata from Powersoft PowerDesigner 6.0 While importing a file from PowerDesigner into the OWB repository, save the file containing the metadata for transfer as a .PDM file. Additionally, you may refer to the following terminology matrix to check how objects in PowerDesigner are mapped in the OWB repository. Table Table Column Columns Foreign Key Foreign Keys Unique Constraint/ Primary Key Unique Keys View View Column Column Table G8 Object Conversion for Import from ERwin into OWB Object in ERwin Imported into OWB Table Table Column Columns Foreign Key Foreign Keys Primary Key Unique Keys View View Column Columns Table G7 Object Conversion for Import from OMG CWM into OWB (Cont.) Object in OMG CWM Imported into OWB Exporting Metadata to Oracle Discoverer 3.1 and 4i OWB Bridges: Transfer Parameters and Considerations G-11 Exporting Metadata to Oracle Discoverer 3.1 and 4i When you export a file from the OWB repository to a Discoverer target using the OWB Transfer Wizard, you can refer to the following terminology matrix to check how objects in OWB repository are mapped in Discoverer. Table G9 Object Conversion for Import from PowerDesigner into OWB Object in PowerDesigner Imported into OWB Table Table Column Columns Index Foreign Keys Index Unique Keys View View Column Columns Object in OWB Object Exported to Discoverer Dimension Folder Level Hierarchy Node Level Attributes Item Hierarchies Hierarchy Fact Folder Fact Attributes Item Table Folder Columns Item Unique Keys Key View Folder (View) Columns Item Attributes flagged as Item Classes Item Classes Business Tree Business Area Exporting Metadata to Oracle Discoverer 3.1 and 4i G-12 Oracle Warehouse Builder 3i Users Guide Before you transfer metadata to Discoverer, you need to perform some additional tasks within OWB to ensure that the metadata transfers successfully and displays appropriately in Discoverer. These tasks include: s Creating business areas in a OWB warehouse module so that the transferred metadata displays as business areas within Discoverer. s Hiding some OWB table columns so that they are grayed in Discover Administration Edition and unavailable in Discoverer User Edition s Specifying that some attributes are to be used as Item Classes in Discoverer s Using names to indicate which attributes are to be placed in the hierarchy nodes within Discoverer The following sections provide instructions for performing these tasks. Configuring OWB for Dimensional Reuse Within OWB, a fact may contain two or more foreign key constraints pointing to a single dimension, such as ordered date and shipped date for a calendar dimension. This is referred to as dimensional reuse. Within Discoverer, each of these constraints needs its own folder so the dimension roles display properly. Dimension roles are required in Discoverer to facilitate query building. (In the calendar example, one folder should display for ordered date and another one for shipped date). To enable dimensional roles to display correctly in Discoverer, you need to create dummy tables in OWB. This dummy table then becomes the dimension role and the columns in the table become the cube dimension use. The recommended naming convention for the dummy table is <dimension reference>##<role reference>, where the dimension and role references help you remember what role on which dimension this table represents. The logical name of the table is the role name. To further identify the dummy table, you need to mark it as Deployable false. Note: In future releases, these tasks will be streamlined for ease of use. Important: Follow these steps only if you are going to set Dimensional Reuse parameter value to TRUE in the OWB Transfer Wizard. Exporting Metadata to Oracle Discoverer 3.1 and 4i OWB Bridges: Transfer Parameters and Considerations G-13 The actual dummy table to real dimension association relies on any one of the contained columns. The dummy column to fact association is determined by making the columns physical name the same as the physical name of the foreign key constraint in the fact. The following example illustrates how dimensional roles can be set up for exporting to Discoverer: Defining dimensions and facts in OWB 1. Create dimensionsDAY, PRODUCT and OPERATORin the normal way. 2. Create factsSALES and FLIGHTSin the normal way. 3. Create the following constraints: s SALES to DAY (for ORDER_DATE) Constraint Physical Name is SALES_ DAY_FK s SALES to DAY (for SHIP_DATE) Constraint Physical Name is SALES_ DAY2_FK s SALES to PRODUCT (for PURCHASE) Constraint Physical Name is SALES_PROD_FK s FLIGHTS to DAY (for ORDER_DATE) Constraint Physical Name is FLIGHTS_DAY_FK s FLIGHTS to PRODUCT (for TICKET) Constraint Physical Name is FLIGHTS_PROD_FK s FLIGHTS to OPERATOR Constraint Physical Name is FLIGHTS_ OPERATOR_FK Defining the Dummy Tables Create the dummy tables as follows using the standard Table editor: 1. For the DAY (ORDER_DATE) role, define the table as follows: s Physical Name DAY##ORD (logical name ORDER_DATE) s Column Physical Name SALES_DAY_FK s Column Physical Name FLIGHTS_DAY_FK 2. For the DAY (SHIP_DATE) role, define the table as follows; s Physical Name DAY##SHIP (logical name SHIP_DATE) Exporting Metadata to Oracle Discoverer 3.1 and 4i G-14 Oracle Warehouse Builder 3i Users Guide s Column Physical Name SALES_DAY2_FK 3. For the PRODUCT (PURCHASE) role, define the table as follows: s Physical Name PROD##PUR (logical name PURCHASE) s Column Physical Name SALES_PROD_FK 4. For the PRODUCT (TICKET) role, define the table as follows: s Physical Name PROD##TIC (logical name TICKET) s Column Physical Name FLIGHTS_PROD_FK 5. To set the Deployable flag to false; right-click the dummy table, select Configure from the pop-up menu, and set the Deployable value to False. Hiding Data Prior to Transfer You can hide specific OWB table columns (such as obsolete or unused columns) so they are grayed in the Discoverer Administration Edition and unavailable in the Discoverer User Edition. To hide these columns in this release, you create a custom entity attribute set in OWB. To hide table columns in OWB: 1. From the main OWB navigator tree, double-click the warehouse target module in which you want to hide table columns prior to transfer to Discoverer. The Warehouse Module Editor displays. 2. Select the Logical Tree tab and expand the Warehouse Module Editor navigation tree. 3. Right-click the fact or dimension table in which you want to hide columns and select Properties from the pop-up menu. OWB returns the Properties window for that table. 4. Select the Attribute Sets tab on the Properties window. 5. Create a new Attribute Set for the export by clicking Add and setting the Attribute Type to be BRIDGE_TYPE. 6. Select the attributes to include within this set and click Advanced... OWB returns the Advanced Attribute Set Properties dialog. 7. Use this dialog to choose the columns to hide in Discoverer by checking the corresponding box under the Hidden field. Exporting Metadata to Oracle Discoverer 3.1 and 4i OWB Bridges: Transfer Parameters and Considerations G-15 8. You can also use this dialog to define Item Class and set default aggregation and default position of table columns within Discoverer. Importing Transferred Data into Discoverer G-16 Oracle Warehouse Builder 3i Users Guide Importing Transferred Data into Discoverer After transferring the selected metadata using the Transfer Wizard, you import the .EEX file into Discoverer. For detailed information on accessing the projects imported from OWB in Discoverer 3.1 and 4i, refer to the Oracle Discoverer Admin Guide. Dimensional Reuse Naming Conventions in Discoverer When you transfer metadata from OWB to Discoverer using the Transfer Wizard and you select TRUE for the Dimensional Reuse parameter, a flag is set so dimensional roles display correctly in Discoverer. (Refer to the section on "Configuring OWB for Dimensional Reuse"). Dimension roles are required to make query building easier in Discoverer. When two tables have multiple joins between them, Discoverer asks the join to use the first time the two folders representing the tables are referred to in a query, and then always uses that join in further parts of the query. If the query requires different joins at different times, the only way to support this is to provide another folder based on the same table. This multi-join scenario is common in dimensional models. The support for dimension roles provides Discoverer with a separate folder for each role of the dimension. For example if a DAY dimension is joined to the SALES fact, once for ORDER_DATE and once for SHIP_DATE, two dimension role folders are created in Discoverer. If the same role is required for a different fact, the original role is reused. For example if a FLIGHTS fact was also using the ORDER_DATE role, it would reference the same folder as the one referenced by the SALES fact. Dimensional reuse (two or more foreign key constraints pointing to the same dimension in OWB) is denoted in the Discoverer navigation tree in two ways: s OWB dimension folder names may appear in the following format: DIMENSION LOGICAL NAME : RoleName s When no role is defined, a default role is generated in Discoverer: DIMENSION LOGICAL NAME : Default Examples (following from the example in "Configuring OWB for Dimensional Reuse") of the appearance of the folders in Discoverer: DAY (hidden) DAY : ORDER_DATE Importing Transferred Data into Discoverer OWB Bridges: Transfer Parameters and Considerations G-17 DAY : SHIP_DATE FLIGHTS OPERATOR (hidden) OPERATOR : Default (see default role above) PRODUCT (hidden) PRODUCT : PURCHASE PRODUCT : TICKET SALES Exporting Metadata to an Object Management Group (OMG) CWM Standard System G-18 Oracle Warehouse Builder 3i Users Guide Exporting Metadata to an Object Management Group (OMG) CWM Standard System When you export metadata from the OWB repository to an OMG CWM standard, the OWB Transfer Wizard converts it to a file that conforms to the OMG CWM standard. You can refer to the following terminology matrix to check how objects in OWB repository are mapped in the OMG CWM standard. Exporting Metadata to Oracle Express After you export metadata from OWB to Oracle Express, use the following steps to access the metadata within Express: 1. Open RAA. 2. Enter the username and password for your RAA repository. 3. Open the project that you just transferred (after you run the bridge, the project will appear in the drop-down list) Object in OWB Object Exported to OMG CWM Project Package Module Schema Dimension Dimension Level Level Level Attributes Attribute Hierarchy Hierarchy Fact Cube Fact Attributes Measure Table Base Table Column Column Foreign Key Foreign Key Unique Key Unique Constraint/ Primary Key View View Column Column Exporting Metadata to Oracle Express OWB Bridges: Transfer Parameters and Considerations G-19 4. Once the project is open, select Express Database Maintenance and enter the user name, password, and service name for your RAA repository. 5. Choose Create Maintenance Procedure. Provide the procedure and select general maintenance. The RDBMS login displays automatically. 6. Select Generate Qualified Selects at Runtime. 7. Select defaults for dimension processing. 8. Name your Express database and log file (these will be saved on the OES server machine under d:\ orant\ database\ express_info). 9. Choose OES > OES Batch Manager to monitor this job (Jobs -> Monitor). 10. You have now created your raw Express database. OSA Configuration Before you look at the data, you need to follow these configuration steps: 1. Open the OSA Application Manager. 2. Select Database Setup > Create. You can now create a .dsc file (give it a name which reflects the project that you transferred). After you enter a name, choose edit and select remote thin-client. For the configuration file, go to the directory where you saved the database and log file and choose the .rdc file that relates to the project that you just transferred. 3. Select Communication Setup > Define > Create. 4. Name your setup and choose thin-client, remote system (this is the server machine dwqa1-pc). For RPC, choose TCP/ IP. The server login is OESDBA/ OESDBA; the relational login is your RAA repository user name/ password. 5. Save. 6. You can now open OSA and choose New > Reports or Graphs to view your data. Youll need to first select the correct dimensions, hierarchies, and measures based on your OWB repository. Exporting Metadata to Oracle Express G-20 Oracle Warehouse Builder 3i Users Guide Index-1 Index A access level Oracle Workflow Builder client, 9-17 add objects with Metadata Import Utility, 10-8 administration environment, 2-15 after the load, 9-38 agents generic connectivity, 4-51 transparent gateways, 4-4, 4-51 aggregation mixed levels, 3-13 application generic file based, 4-15 Archive/ Restore archiving a project, 10-39 restoring a project, 10-41 setting up preferences, 10-37 using the utility, 10-35 attribute sets, 3-29 attribute sets, default, 5-5 automated scheduling, 9-2 B bind, 5-5 bitmap index, 8-20 bridges business areas, 3-70 versions, 10-25 bridge-type attribute set, 3-29 Builder exporting metadata, 10-29 importing metadata, 10-23 Metadata Loader, 10-2 Runtime Library, 9-38 Builder console major component, 2-11 operational environments, 2-12 toolbar, 2-11, 2-18 Utilities button, 2-11 utilities button, 2-15 Wizard button, 2-11 Builder Project a road map, 2-2 create, 2-7 new project wizard, 2-10 business area, 3-69, G-6 Business Tree tab, 3-71 creating a definition, 3-71 Oracle CWM bridges, 3-70 business tree business area, 3-69 creating a business area, 3-71 C Change Management Pack, 8-41, 8-42 character set flat file, 4-29 check constraint, 3-28 column attribute sets, 3-29 default order, 3-25 generated, 3-16 reorder, 3-25, 3-26 commit toolbar, 2-10 Index-2 configuration Oracle Reports, 11-2 parallel, 8-13, 8-16 partition parameter, 8-13, 8-16 performance parameter, 8-13, 8-16 configuration parameters Dimension, 8-11 Facts, 8-14 generation options, 8-13 generation preferences, 8-8 identification, 8-11, 8-14, 8-16 materialized Views, 8-16 materialized views, 8-18 OEM registration, 8-8 Oracle Enterprise Manager, 8-9 overview, 8-2 partition, 8-13, 8-16 performance,parallel, 8-12, 8-15 rules for export, 10-5 runtime directories, 8-9 Sequences, 8-18 storage space, 8-13, 8-16 target directories, 8-10 time dimension generation, 8-13 warehouse module, 8-7 Workflow Queue Listener, 8-9 configuration phase general discussion, 1-5 configuring data warehouse, 8-2 database links, 8-7 dimension, 8-11 facts, 8-14 hash partitions, 8-23 indexes, 8-20 logical objects, 8-3 materialized views, 8-16 new physical objects, 8-4 OEM Preferences, 8-43 partitions, 8-21 physical instance, 8-2 range partitions, 8-23 range with hash subpartitions partitions, 8-23 sequences, 8-18 tables, 8-19 views, 8-23 warehouse module, 8-7 connection information data source update connection, 4-27 non-Oracle database system, 4-4 Oracle Designer Repository, 3-6 source modules, 4-2 warehouse module, 3-6 console see Builder Console, 2-11 constraint change, 3-26 change UK to PK, 3-44 check, 3-28 DEFAULTIF and NULLIF, 4-30 fact table foreign key reference, 3-39 fact table foreign key references, 3-3 generated unique key constraints, 3-26 unique key constraint on levels, 3-12 view, 3-55 container database sources, 4-7 flat file sources, 4-13 flat files, 4-13 Oracle Designer Repository, 4-12 source module, 4-2 warehouse modules, 3-5 create definition Builder Project, 2-7 delimited flat file, 4-41 dimension, 3-11 fact table, 3-37 flat file, 4-28 flat file sources, 4-28 hierarchy, 3-20 materialized view, 3-46 Oracle Workflow process, 9-17 transformation, 3-64 view, 3-52 create Windows NT user, 9-4 creating database links, 4-5, 10-51 diagram of source definitions, 4-27 Index-3 dimensions, 3-14 indexes,partitions,database links, 8-4 physical objects, 8-4 source modules, 4-7 time dimension, 3-34 warehouse module, 3-5 CWM bridges business areas, 3-70 CWM Transfer Wizard online help, 10-25, 10-31 D data source, 4-1 data transformation, 6-1 overview, 3-58 data type flat file fields, 4-30 portable, 4-30 data warehouse configuration parameters, 8-2 loading, 9-2 database links configuring, 8-7 creating, 4-5, 8-4 IBM DB2 system, 4-6 New Database Link Panel, 4-4 Oracle database system, 4-5 source module, 4-2 warehouse module, 3-8 definition attribute, 3-18 business area, 3-69 delimited file, 4-41 delimited flat file, 4-41 fixed-length file, 4-31 flat file, 4-28 materialized view, 3-46 transformation, 3-64 valid and invalid, 8-25 validate a set of definitions, 8-28 warehouse key column, 3-20 definitional phase, 1-2 delimited file definition, 4-41 deploy job scripts, 9-4 PL/ SQL mapping, 8-32 table, 8-33 deploying jobs, 9-4 scripts, 8-37 Designer Repository import definitions into Builder repository, 3-45 desktop view, 3-52 developmental phases, 1-2 diagram generate with Source Module Editor, 4-17 source definitions, 4-27 dimension attribute definition, 3-18 basic definition, 3-2 create definition, 3-11 define a level of aggregation, 3-16 denormalized dimension table, 3-13 dimension object, 3-2 generated scripts, 8-30 hierarchies, 3-19 hierarchy, 3-12, 3-20 levels, 3-12 mapping, 5-8 New Dimension Wizard, 3-11 object Builder rules, 3-12 property sheet, 3-22 products, 3-14 create, 3-14 property sheet, 3-22 table property sheet, 3-23 time creating, 3-34 time dimension, 3-23 update, 3-21 validate example, 8-26 dimension table default column order, 3-25 physical aspects, 3-2 reordering the columns, 3-25 dimensions Index-4 creating, 3-14 directives Metadata Export Utility, 10-18 Metadata Import Utility, 10-20 Discoverer bridge business area, G-6 business areas, 3-70 Display Sets default attribute sets, 5-5 definition of, 5-4 display welcome page, 2-24 drawer utilities, 2-15 E edit DML Type, 5-27 editing dimensions, 3-21 invalid objects, 8-27 editor dimension, 3-21 materialized view editor, 3-51 source module editor generate diagram, 4-17, 4-27 view editor, 3-56 warehouse module editor, 3-10 environment, 2-12 administration, 2-15 Builder console, 2-12 project, 2-12 export business area, G-6 exporting from Builder repository, 10-29 list of first-class objects, 10-4 log file, 10-29, 10-34 metadata export file, 10-3 metadata with Metadata Export Utility, 10-3 subset of objects, 10-4 exporting MDL, 10-3 Express bridge business areas, 3-70 extract transformation, 5-36, 7-2 F fact table create definition, 3-37 create definitions, 3-37 default column order, 3-25 define foreign key reference, 3-39 physical aspects, 3-3 property sheet, 3-42 reordering the columns, 3-26 sales fact table description, 3-38 update definition, 3-41 filter data with a transform, 3-64 first-class objects, 10-4 fixed-length file definition, 4-31 flat file character set, 4-29 create definition, 4-28 data sample, 4-32 definition, 4-28 delimited create a format, 4-41 update format, 4-47 delimited format, 4-41 field constraint, 4-30 field mask, 4-30 field type, 4-30 fixed-length, 4-31 create format, 4-32 format, 4-35 generic file based application, 4-15 infrastructure requirements, 4-52 multiple physical records per logical record, 4-29 multiple physical records per logical records, 4-41 physical record length, 4-29 source module, 4-13 update format definition, 4-39 Flat File Sample Wizard, 4-28, 4-31 Folder-in-Use Lock, definition of, 2-4 format flat file, 4-35 Index-5 G gateways transparent, 4-4 generated column name, 3-16 unique key constraints, 3-26 generated scripts dimension, 8-30 general description, 8-28 mapping, 8-32 materialized view, 8-31 sequences, 8-33 tables, 8-33 types, 8-43 generating upgrade scripts, 8-43 generation phase, 1-2 general discussion, 1-5 generic file based application, 4-15 group by operation as a transformation operation, 6-21 H hash partitions, 8-22, 8-23 help for CWM Transfer Wizard, 10-25, 10-31 heterogeneous source, 4-2 hierarchies dimension, 3-19 hierarchy, 3-12 create definition, 3-20 how to create a source module, 4-7 switch projects, 2-9 update definition of a dimension, 3-23 update definition of a fact table, 3-41 update definition of a materialized view, 3-51 update definition of a view, 3-56 I icon commit, 2-10 ID column rename as warehouse key column, 3-20 import definition database source, 4-17 database systems, 4-17 Import Metadata Wizard, 4-21 re-import, 4-21 definitions into a warehouse module, 3-45 importing into Builder repository, 10-23 Metadata Import Utility, 10-8 selected modules, 10-21 validation rules, 10-21 Import Metadata Wizard, 4-31 importing metadata, 10-8, 10-11 indexes bitmap,unique, 8-20 configuring, 8-20 creating, 8-4 infrastructure requirements flat file sources, 4-52 source modules, 4-51 warehouse module, 3-73 init.ora, 4-6 integrators, 4-2 J job dependencies Oracle Workflow, 9-19 job scheduling, 9-2 job scripts deploy, 9-4 schedule, 9-7 schedule job with parameters, 9-8 jobs deploying, 9-4 registering, 9-4 scheduling, 9-2, 9-7 updating,loading, 9-2 L levels dimension, 3-12 library transformation, 3-59 Index-6 load and manage phase, 1-2 loading jobs, 9-2 log file export to OMG CWM, 10-29, 10-34 logical constraints view, 3-55 logical name, 2-21 creating, 2-3, 2-20 descriptions, 2-3, 2-20 maximum length, 2-22 propagate physical to logical name, 2-23 uniqueness requirements, 2-22 logical tree set naming mode, 2-21 logon to a repository, 2-8 M major component Builder console, 2-11 manage and load phase general discussion, 1-6 managing dependencies Workflow, 9-10 manual scheduling, 9-2 mapping, 5-1 definition of, 1-4 definitions, 5-2 dimension, 5-8 edit DML Type, 5-27 extract transformation, 5-36, 7-2 generated scripts, 8-32 transformations, 6-1 triggers, 5-36, 7-2 Mapping Editor about, 5-8 keyboard operations, B-10 menu bar, B-1 mouse operations, B-7 object palette, B-5 toolbar, B-4 materialized view, 3-4 configuring, 8-16 create definition, 3-46 default column order, 3-25 generated scripts, 8-31 performance, 3-4 refreshing, 8-18 reordering the columns, 3-26 update definition, 3-51 Materialized View Editor, 3-51 metadata detailed reports, 11-20 implementation reports, 11-22 lineage and impact analysis diagrams, 11-24 lineage and impact analysis reports, 11-23 summary reports, 11-19 Metadata Export Utility directives, 10-18 Metadata Import Utility directives, 10-20 Metadata Loader Export Utility, 10-3 Import Utility, 10-8 Loader Utilities in MS-DOS window, 10-17 Pure*Extract, 4-49 Pure*Integrate, 4-49 metadata loader (MDL), 10-2 command line utility, 10-17 exporting, 10-3 exporting metadata, 10-5 import options, 10-12 import searching, 10-8 importing, 10-8, 10-11 large imports,large exports, 10-2 mulitple user considerations, 10-2 metadata reports, 11-1, 11-19 migrating a repository, 10-2 mixed levels of aggregation, 3-13 mode of operation administration environment, 2-15 logical or physical naming mode, 2-21 project environment, 2-12 transformation libraries, 2-14 module definition of, 1-3 mapping, 1-4 source, 1-3 source module, 4-2 Index-7 database definitions, 4-7 flat files, 4-13 unique names in project, 2-20 warehouse module, 3-5 creating a business area, 3-71 module editor source module generate diagram, 4-17, 4-27 warehouse basic function, 1-xxii warehouse module, 3-10 multiple user access, 2-4 N name Builder object names, 2-3, 2-20 creating logical and physical names, 2-3, 2-20, 2-21 default naming mode, 2-21 naming policies, 2-20 renaming an object, 2-21 named attribute sets, 3-29 naming mode, 2-3 default naming mode, 2-21 default preferences, 2-21 preferences persistence of naming preferences, 2-20 propagate physical to logical name, 2-23 remembering preferences, 2-21 setting preferences, 2-21 naming policies, 2-20 navigation search navigation tree, 2-25 warehouse module navigation tree, 3-10 wizards, 2-23 navigation tree, 2-11 New Database Link Panel, 4-4 New Fact Table Wizard, 3-37 New Materialized View Wizard, 3-46 New Module Wizard, 4-13 New Time Dimension Wizard, 3-33 New Transform Wizard, 3-64 New View Wizard, 3-52 non-Oracle database systems, 4-4 O object children of first-class objects, 10-10 import selected modules, 10-21 import with Metadata Import Utility, 10-8 OEM configuring preferences, 8-43 scheduling, 9-2, 9-3, 9-7 open_links, 4-6 optimizer hints as a transformation, 7-13 Oracle Designer, 8-8 import definitions into Builder repository, 3-45 source module, 4-12 Oracle Discoverer business areas, 3-70 Oracle Enterprise Manager configuration parameters, 8-9 create Windows NT user, 9-4 Job History Pane, 9-38 summary, 9-2 Oracle Express business areas, 3-70 Oracle Portal, 11-1 Oracle Reports configuration, 11-2 Oracle Transparent Gateway, 4-4 infrastructure requirements, 4-51 Oracle Warehouse Builder Metadata Loader, 10-2 Oracle Warehouse Builder Runtime Library, 9-38 Oracle Workflow access level client, 9-17 Builder client access level, 9-17 define process, 9-17 Deployment Wizard, 9-11 server, 9-11 OWB Browser administration, 10-45 assigning a custom report to a role, 10-56 browsing the favorites, 11-14 browsing the repository, 11-7 contents tab, 11-11 creating custom reports, 11-26 creating database links, 10-51 Index-8 customizing the favorites pages, 11-17 detailed reports, 11-20 dropping database links, 10-53 editing database links, 10-52 granting and revoking access, 10-55 implementation reports, 11-22 lineage and impact analysis diagrams, 11-24 lineage and impact analysis reports, 11-23 properties tab, 11-10 purging stale user information, 10-57 QA user role, 10-55 registering a custom report, 10-46, 10-57 registering a repository, 10-46, 10-48 related tab, 11-12 reports, 11-1, 11-19 reports tab, 11-14 roles, 10-47, 10-54 setting up preferences, 11-2 summary reports, 11-19 unregistering a repository, 10-54 user access, 10-54 viewing database links, 10-52 warehouse developer role, 10-54 warehouse user role, 10-55 OWB MDL File Upgrade Utility, 10-2 OWB Reports, 11-1 OWB Repository Assistant, 10-2 OWB Runtime Assistant, 10-2 OWB Transfer Wizard, 10-22 P parallel, 8-12, 8-15 parameter basic copy transformation, 3-64 physical partition, 8-13, 8-16 schedule job with parameters, 9-8 transformation, 3-59 Partition Exchange Loading, 8-23 partitions configuring, 8-21 creating, 8-4 hash, 8-23 hash,range, 8-22 range, 8-23 range with hash subpartitions, 8-23 performance materialized views, 3-4 warehouse keys, 3-3 physical aspects dimension table, 3-2 fact table, 3-3 physical name, 2-23 creating, 2-3, 2-20 propagate physical to logical name, 2-23 syntax, 2-20, 2-23 uniqueness requirements, 2-20 physical partition configuration parameters, 8-13, 8-16 post-load tasks, 9-38 predefined attribute set, 3-29 predefined time dimension elements, 3-33 preface conventions table sample, 1-xxiv Preference.properties file, 2-20 Preferences display welcome page on all wizards, 2-24 naming mode, 2-20, 2-21 default preferences, 2-21 OWB Browser, 11-2 utility, 2-16 add a utility, 2-16 remove a utility, 2-17 update a utility, 2-17 print report, 11-4 project definition of, 1-3 how to switch projects, 2-9 new project wizard, 2-10 unique module names, 2-20 project environment, 2-12 propagate physical to logical name, 2-23 property icon, 2-18 property sheet, 2-18 default column order, 3-25 dimension, 3-22 object, 3-22 table, 3-23 fact table, 3-42 materialized view, 3-51 Index-9 view, 3-56 Q query rewrite conditions, 3-4 R range partitions, 8-22, 8-23 with hash subpartitions, 8-23 reference materials distributed database systems, 4-4 non-Oracle database systems, 4-4 Oracle8i Data Warehousing Guide, 3-2, 3-4 Ralph Kimball books, 3-2, 3-4 refreshing materialized view, 8-18 registering jobs, 9-4 rename in logical or physical mode, 2-21 rename warehouse object view, 3-57 reorder columns, 3-25 warehouse objects, 3-26 reporting, 11-1 Reports, 11-19 creating custom, 11-26 detailed, 11-20 implementation, 11-22 lineage and impact analysis, 11-23 lineage and impact analysis diagrams, 11-24 printing, 11-4 summary, 11-19 repository logon to a repository, 2-8 repository administration, 10-2 reserved words, A-2 road map of a Builder project, 2-2 Row-Sets definition of, 5-2 rules dimension object, 3-12 running TCL scripts, 9-9 Workflow process, 9-20 running scripts, 8-36 Runtime Audit Viewer, 9-23, 9-24, 9-29 purging runtime entries, 9-28 refreshing the view, 9-28 searching the navigation tree, 9-27 setting the audit level, 9-24 viewing by date range, 9-26 viewing the error details, 9-38 viewing the errors, 9-36 viewing the job audit, 9-30 viewing the job instance audit, 9-30 viewing the task audit, 9-32 viewing the task details audit, 9-33 runtime auditing, 9-23 runtime library infrastructure requirements, 3-73 schema objects, 9-38 S sales fact table, 3-37 description, 3-38 sample data sample for flat file, 4-32 scalar function transformation, 3-64 schedule job scripts, 9-4 scheduling automated, 9-2 jobs, 9-2, 9-7 manual, 9-2 with OEM, 9-2, 9-3 with Workflow, 9-2 Workflow process, 9-23 search a navigation tree, 2-25 secondary source view, 3-55 semi-colon, 3-49 sequences configuring, 8-18 generated scripts, 8-33 services heterogeneous services, 4-4 short name defining, 1-9 Index-10 reserved words, A-2 software integrators, 4-2 source data source, 4-1 source definition update, 4-25 source module, 4-2 connection information, 4-2 create definition, 4-13 creating, 4-7 database definitions, 4-7 database links, 4-2 flat files, 4-13 general description, 2-3 import definitions, 4-17 Oracle Designer Repository, 4-12 Source Module Editor generate diagram, 4-17, 4-27 SQL optimizer hints as a transformation, 7-13 star schema basic example, 3-2 summarize data with a transformation, 6-21 switch projects, 2-9 Synchronization, 2-6 syntax errors extra semi-colon, 3-49 T table attribute sets, 3-29 configuring, 8-19 default column order, 3-25 generate and deploy, 8-33 generated scripts, 8-33 reordering the columns, 3-26 runtime library tables, 9-38 table properties dimension table, 3-22 fact table, 3-42, 3-43 tablespace, 8-13, 8-16 testing deployed scripts, 8-36 time dimension create, 3-23 time dimension generation physical configuration, 8-13 toolbar Builder console, 2-11, 2-18 commit, 2-10 property icon, 2-18 Transfer Wizard log file export, 10-29, 10-34 online help, 10-25, 10-31 version, 10-25 transform create definition, 3-64 New Transform Wizard, 3-64 SQL optimizer hint, 7-13 transformation, 6-1 basic copy transformation, 3-64 create definition, 3-64 custom transformation example, 3-64 extract, 5-36, 7-2 filter data, 3-64 group by operation, 6-21 library, 3-59 overview, 3-58 parameters, 3-59 scalar function, 3-64 transformation category unique transformation names, 2-20 transformation library environment, 2-14 transparent agents, 4-4 gateways (infrastructure requirements), 4-51 transportable tablespace, 8-21 trigger general discussion, 5-36, 7-2 U unbound, 5-5 unique index, 8-20 unique key constraint on levels, 3-12 Universal Identifiers (UniversalIDs), 10-8 update connection data source, 4-27 Index-11 flat file format, 4-47 operations on specific columns, 7-14 source definitions, 4-25 fixed format file, 4-39 update definition dimension, 3-23 materialized view, 3-51 view, 3-56 updating jobs, 9-2 user-defined attribute set, 3-29 using Properties inspector, 8-4, 8-7 Workflow, 9-11 Utilities button, 2-11 utilities button, 2-15 utilities drawer, 2-15 add a utility, 2-16 configuration, 2-16 remove a utility, 2-17 V validating, 8-24, 8-26 editing invalid objects, 8-27 invalid definitions, 8-25 valid definitions, 8-25 validation codes, 8-27 validation messages, 8-27 validation general discussion, 1-5 rationale and general discussion, 8-24 rules for imported definitions, 10-21 verifying OEM registration, 9-5 versions bridges, 10-25 Transfer Wizard, 10-25 view conventional, 3-5 create definition, 3-52 default column order, 3-25 logical constraints, 3-55 property sheet, 3-56 reordering the columns, 3-26 View Editor, 3-56 viewing OWB Reports, 11-1 runtime audit, 9-23 validation details, 8-27 Views configuring, 8-23 W warehouse keys column, 3-20 performance, 3-3 warehouse module, 3-5 connection information, 3-6 creating, 3-5 creating a business area, 3-71 database links, 3-8 displaying, 3-10 general description, 2-3 import definitions, 3-45 navigation tree, 3-10 rename objects materialized view, 3-51, 3-57 unique object names, 2-20 Warehouse Module Editor, 3-10 function, 1-xxii warehouse object, 3-5 rename objects, 3-51 materialized view, 3-51 view, 3-57 warehouse schema, 1-3 welcome page display welcome page, 2-24 wizard display welcome page, 2-24 Flat File Sample Wizard, 4-28 New Dimension Wizard, 3-11 New Fact Table Wizard, 3-37 New Materialized View, 3-46 New Module Wizard, 3-5, 4-13 new project wizard, 2-10 New Time Dimension Wizard, 3-33 New Transform Wizard, 3-64 Index-12 New View Wizard, 3-52 online help, 10-25, 10-31 Oracle Workflow Deployment Wizard, 9-11 wizard navigation, 2-23 Wizard button, 2-11 Workflow defining the process, 9-17 deploying mappings, 9-11 launching processes, 9-22 managing dependencies, 9-10 process example, 9-20 running the process, 9-20 scheduling, 9-2, 9-10 scheduling the process, 9-23 Workflow Deployment Wizard, 9-11 Workflow Queue Listener, 9-21 configuration parameters, 8-9 Write Lock, definition of, 2-4 Index-13 Index-14 Index-15 Index-16 Index-17 Index-18 Index-19 Index-20 Index-21 Index-22 Index-23 Index-24 Index-25 Index-26 Index-27 Index-28 Index-29 Index-30 Glossary-1 Glossary additive Describes a fact (or measure) that can be summarized through addition. An additive fact is the most common type of fact. Examples include Sales, Cost, and Profit. (Contrast with nonadditive, semi-additive.) attribute A descriptive characteristic of one or more levels. Attributes represent logical groupings that enable end users to select data based on like characteristics. Note that in relational modeling, an attribute is defined as a characteristic of an entity. In Oracle8i, an attribute is a column in a dimension that characterizes elements of a single level. aggregation The process of consolidating data values into a single value. For example, sales data could be collected on a daily basis and then be aggregated to the week level, the week data could be aggregated to the month level, and so on. The data can then be referred to as aggregate data. Aggregation is synonymous with summarization, and aggregate data is synonymous with summary data. aggregate Summarized data. For example, unit sales of a particular product could be aggregated by day, month, quarter and yearly sales. ancestor A value at any level above a given value in a hierarchy. For example, in a Time dimension, the value 1999 might be the ancestor of the values Q1-99 and Jan-99. (See also descendant, hierarchy, level.) Glossary-2 attribute A descriptive characteristic of one or more levels. For example, the Product dimension for a clothing manufacturer might contain a level called Item, one of whose attributes is Color. Attributes represent logical groupings that enable end users to select data based on like characteristics. Note that in relational modeling, an attribute is defined as a characteristic of an entity. In Oracle8i, an attribute is a column in a dimension that characterizes elements of a single level. child A value at the level below a given value in a hierarchy. For example, in a Time dimension, the value Jan-99 might be the child of the value Q1-99. A value can be a child for more than one parent if the child value belongs to multiple hierarchies. (See also hierarchy, level, parent.) cleansing The process of resolving inconsistencies and fixing the anomalies in source data, typically as part of the ETT process. (See also ETT.) Common Warehouse Meta Data A repository standard used by Oracle data warehousing, decision support, and OLAP tools including Oracle Warehouse Builder. The CWM repository schema is a stand-alone product that other products can shareeach product owns only the objects within the CWM repository that it creates. console The main window of the Oracle Warehouse Builder application. It contains a menu bar, launcher, and navigator. data source A database, application, repository, or file that contributes data. data mart A data warehouse that is designed for a particular line of business, such as sales, marketing, or finance. In a dependent data mart, the data can be derived from an enterprise-wide data warehouse (as a dependent data mart). In an independent data mart, data can be collected directly from sources. (See also data warehouse.) Glossary-3 data warehouse A relational database that is designed for query and analysis rather than transaction processing. A data warehouse usually contains historical data that is derived from transaction data, but it can include data from other sources. It separates analysis workload from transaction workload and enables a business to consolidate data from several sources. In addition to a relational database, a data warehouse environment often consists of an ETT solution, an OLAP engine, client analysis tools, and other applications that manage the process of gathering data and delivering it to business users. (See also ETT, OLAP.) ddl Data Definition Language. denormalize The process of allowing redundancy in a table so that it can remain flat. (Contrast with normalize.) derived fact (or measure) A fact (or measure) that is generated from existing data using a mathematical operation or a data transformation. Examples include averages, totals, percentages, and differences. dimension A structure, often composed of one or more hierarchies, that categorizes data. Several distinct dimensions, combined with measures, enable end users to answer business questions. Commonly used dimensions are Customer, Product, and Time. In Oracle8i, a dimension is a database object that defines hierarchical (parent/child) relationships between pairs of column sets. In Express a dimension is a database object that consists of a list of values. dimension value One element in the list that makes up a dimension. For example, a computer company might have dimension values in the Product dimension called LAPPC and DESKPC. Values in the Geography dimension might include Boston and Paris. Values in the Time dimension might include MAY96 and JAN97. Glossary-4 dml statement There are four types of data manipulation, or dml, statements: select, insert, update, or delete. drill To navigate from one item to a set of related items. Drilling typically involves navigating up and down through the levels in a hierarchy. When selecting data, you can expand or collapse a hierarchy by drilling down or up in it, respectively. (See also drill down, drill up.) drill down To expand the view to include child values that are associated with parent values in the hierarchy. (See also drill, drill up.) drill up To collapse the list of descendant values that are associated with a parent value in the hierarchy. element An object or process. For example, a dimension is an object, a mapping is a process, and both are elements. ETT Extraction, transformation, and transportation. ETT refers to the methods involved in accessing and manipulating source data and loading it into a data warehouse. The order in which these processes are performed varies. Note that ETL (extraction, transformation, load) and ETM (extraction, transformation, move) are sometimes used instead of ETT. (See also data warehouse, extraction, transformation, transportation.) editor A window in Oracle Warehouse Builder used to define or edit objects and their relationships to each other. extraction The process of taking data out of a source as part of an initial phase of ETT. (See also ETT.) Glossary-5 fact table A table in a star schema that contains facts. A fact table typically has two types of columns: those that contain facts and those that are foreign keys to dimension tables. The primary key of a fact table is usually a composite key that is made up of all of its foreign keys. A fact table might contain either detail level facts or facts that have been aggregated (fact tables that contain aggregated facts are often instead called summary tables). A fact table usually contains facts with the same level of aggregation. fact/measure Data, usually numeric and additive, that can be examined and analyzed. Values for facts or measures are usually not known in advance; they are observed and stored. Examples include Sales, Cost, and Profit. Fact and measure are synonymous; fact is more commonly used with relational environments, measure is more commonly used with multidimensional environments. file-to-table mapping Maps data from flat files to tables in the warehouse. hierarchy A logical structure that uses ordered levels as a means of organizing data. A hierarchy can be used to define data aggregation; for example, in a Time dimension, a hierarchy might be used to aggregate data from the Month level to the Quarter level to the Year level. A hierarchy can also be used to define a navigational drill path, regardless of whether the levels in the hierarchy represent aggregated totals. (See also dimension, level.) hub module The metadata container for process data. integrator Software that works with Oracle Warehouse Builder to facilitate definition, design, and extraction of source data. Examples of integrators include the Oracle Applications Integrator, the PeopleSoft Integrator, and the SAP Integrator. launcher The panel in the Oracle Warehouse Builder console window that contains buttons to control the active environment. These environments include the Project environment, Administration environment, and Transformation Library Glossary-6 environment. Environments act as separators between the separate functions of the Oracle Warehouse Builder product. For example, you use the Transformation Library to define the reusable transformation functions, and you use the Project environment to define the metadata used to create a data warehouse. level A position in a hierarchy. For example, a Time dimension might have a hierarchy that represents data at the Month, Quarter, and Year levels. (See also family, hierarchy.) level value table A database table that stores the values or data for the levels you created as part of your dimensions and hierarchies. mapping The definition of the relationship and data flow between source and target objects. measure Data, usually numeric and additive, that can be examined and analyzed. Values for facts or measures are usually not known in advance; they are observed and stored. Examples include Sales, Cost, and Profit. Fact and measure are synonymous; fact is more commonly used with relational environments, measure is more commonly used with multidimensional environments. metadata Data that describes data and other structures, such as objects, business rules, and processes. For example, the schema design of a data warehouse is typically stored in a repository as metadata, which is used to generate scripts used to build and populate the data warehouse. A repository contains metadata. Examples include: for data, the definition of a source to target transformation that is used to generate and populate the data warehouse; for information, definitions of tables, columns and associations that are stored inside a relational modeling tool; for business rule, discount by 10 percent after selling 1,000 items. model An object that represents something to be made. A representative style, plan, or design. metadata that defines the structure of the data warehouse. module (source, target and hub) The metadata containers for source data, process data, and warehouse data. Glossary-7 navigator A panel in the Oracle Warehouse Builder console that displays the objects for the active environment in a tree structure. nonadditive Describes a fact (or measure) that cannot be summarized through addition. An example includes Average. (Contrast with additive, semi-additive.) normalize In a relational database, the process of removing redundancy in data by separating the data into multiple tables. (Contrast with denormalize.) The process of removing redundancy in data by separating the data into multiple tables. (Kimball) operational data store (ODS) The cleaned, transformed data from a particular source database. OLAP Online analytical processing. OLAP functionality is characterized by dynamic, multidimensional analysis of historical data, which supports activities such as the following: n Calculating across dimensions and through hierarchies n Analyzing trends n Drilling up and down through hierarchies n Rotating to change the dimensional orientation OLAP tools can run against a multidimensional database or interact directly with a relational database. Oracle Warehouse A data warehouse built using Oracle Warehouse Builder. Oracle Warehouse Builder The Oracle product used to build and manage an Oracle Warehouse. Glossary-8 Oracle Warehouse Builder Repository Stores the Oracle Warehouse Builder metadata definitions used to build the Warehouse. A storage facility for metadata. For example, the Designer 2000 repository stores ER diagrams and business processes. (See also metadata.) Oracle Warehouse Builder Repository administrator The repository administrator is the information specialist that manages the Repository and other administration needs. parent A value at the level above a given value in a hierarchy. For example, in a Time dimension, the value Q1-99 might be the parent of the value Jan-99. (See also child, hierarchy, level.) physical instance A collection of related database objects that, when deployed, become a schema. Objects include tables, views, and other objects. project A project contains all design definitions and information needed by the Oracle Warehouse Builder to build a warehouse. Oracle Warehouse Builder provides a project structure to help users organize their work, and support versioning. report-to-table mapping Maps PeopleSoft trees to tables in the warehouse. schema A collection of related database objects. Relational schemas are grouped by database user ID and include tables, views, and other objects. (See also snowflake schema, star schema.) semi-additive Describes a fact (or measure) that can be summarized through addition along some, but not all, dimensions. Examples include Headcount and On Hand Stock. (Contrast with additive, nonadditive.) sequence A database schema object supported by Oracle8i that is a series of sequential numbers generated by Oracle8is sequence generator. Oracle stores sequences as rows in a single data dictionary table in the SYSTEM tablespace. A sequence Glossary-9 definition indicates general information: the name of the sequence, whether it ascends or descends, the interval between numbers, and other information. short name The short name is a unique identifier that is used as the root name for all related objects created in Oracle Warehouse Builder. snowflake schema A type of star schema in which the dimension tables are partly or fully normalized. (See also schema, star schema.) Software Library Contains the integrators currently installed for Oracle Warehouse Builder. Accessed via the Administration environment. source A database, application, file, or other storage facility from which the data in a data warehouse is derived. star schema A relational schema whose design represents a multidimensional data model. The star schema consists of one or more fact tables and one or more dimension tables that are related through foreign keys. (See also schema, snowflake schema.) subject area A classification system that represents or distinguishes parts of an organization or areas of knowledge. A data mart is often developed to support a subject area such as sales, marketing, or geography. (See also data mart.) table A layout of data in columns. target Holds the intermediate or final results of any part of the ETT process. The target of the entire ETT process is the data warehouse. (See also data warehouse, ETT.) transformation The process of manipulating data. Any manipulation beyond copying is a transformation. Examples include cleansing, aggregating, and integrating data from multiple sources. Glossary-10 Transformation Library Stores the reusable formulas for the transformation of data as it moves between source and target objects. Accessed via a button in the Oracle Warehouse Builder launcher. transportation The process of moving copied or transformed data from a source to a data warehouse. (See also transformation.) tree-to-dimension mapping A map between a PeopleSoft tree and an Oracle Warehouse Builder dimension. validation The process of verifying metadata definitions and configuration parameters. versioning The ability to create new versions of a data warehouse project for new requirements and changes. Warehouse administrator The warehouse administrator is the information specialist that manages the warehouse database and warehouse management applications. For example, the warehouse administrator would be responsible for managing and monitoring periodic updates of the warehouse database. Warehouse designer The warehouse designer is the information specialist that uses Oracle Warehouse Builder to define the metadata that is used to create the data warehouse(s) for your enterprise.