Welcome to Scribd, the world's digital library. Read, publish, and share books and documents. See more
Buy Now $50.00
Standard view
Full view
of .
Save to My Library
Look up keyword or section
Like this
14Activity

Table Of Contents

Data Warehouse versus Business Intelligence
Define the Project
Assess Your Readiness for DW/BI
Strong Senior Business Management Sponsor(s)
Compelling Business Motivation
Feasibility
Factors Not Considered Readiness Deal Breakers
Address Shortfalls and Determine Next Steps
Poor Quality Data
Weak Business Sponsor or IT-Only Sponsor
Too Much Demand from Multiple Business Sponsors
Well Meaning, But Overly Aggressive Business Sponsor
Legacy of Underperforming, Isolated Data Silos
Develop the Preliminary Scope and Charter
Focus on a Single Business Process
The Role of Rapid Application Development
Document the Scope/Charter
Build the Business Case and Justification
Determine the Financial Investments and Costs
Determine the Financial Returns and Benefits
Combine the Investments and Returns to Calculate ROI
Plan the Project
Establish the Project Identity
Staff the Project
Front Office: Sponsors and Drivers
Coaches: Project Managers and Leads
Regular Lineup: Core Project Team
Special Teams
Free Agents
Convert Individual Talent into a Team
Develop the Project Plan
Develop the Communication Plan
Project Team
Sponsor and Driver Briefings
Business User Community
Communication with Other Interested Parties
Manage the Project
Conduct the Project Team Kickoff Meeting
Monitor Project Status
Project Status Meetings
Project Status Reports
Maintain the Project Plan
Consolidate the Project Documentation
Manage the Scope
Track Issues
Control Changes
Manage Expectations
Recognize Project Trouble Signs
Manage the Program
Establish Governance Responsibility and Processes
Elevate Data Stewardship to Enterprise Level
Scribe
Observers
Research the Organization
Select the Interviewees
Business Interviewees
IT and Compliance/Security Interviewees
Develop the Interview Questionnaires
Schedule the Interviews
Sequence the Interviews
Establish the Interview Time and Place
Prepare the Interviewees
Review Interviewing Ground Rules
Remember Your Interview Role
Assume You Will Learn
Verify Communications
Be Conversational
Maintain Interview Schedule Flexibility
Manage Expectations Continuously
Conduct the Interview
Program Business Interviews
Program IT Interviews
Program Compliance/Security Interviews
Wrap Up the Interview
Determine the Success Criteria
Say Thanks and See You Later
Review the Interview Results
Synthesize Around Business Processes
Prepare and Publish Requirements Deliverables
Interview Write-Ups
Program Requirements Findings Document
Prioritize and Agree on Next Steps
Culminate with a Review and Prioritization Meeting
Close the Loop
Adjustments for Project Level Requirements
Project Level Approach
Prepare for the Project Requirements Interview
Conduct the Interviews
Review Existing Reports and Analyses
Dig into the Data
Prepare and Publish Project Deliverables
Agree on Next Steps and Close the Loop
Deal with Challenging Interviewees
Abused User
Overbooked/Substitute User
Comatose User
Overzealous User
Know-It-All User
Clueless User
Nonexistent User
The Value of Architecture
Technical Architecture Overview
Flow from Source System to User Screen
Common Architecture Features
Metadata Driven
Flexible Services Layers
Evolution of Your DW/BI Architecture
Back Room Architecture
General ETL Requirements
Build versus Buy
Back Room ETL Flow
Source Systems
Enterprise Resource Planning Systems
Operational Data Stores
Reporting Operational Data Stores
Master Data Management
XML Sources
Message Queues, Log Files, and Redo Files
Proprietary Formats
Extract
Clean and Conform
Deliver
ETL Management Services
Additional Back Room Services and Trends
Data Service Providers
Functional Service Providers
Data Delivery Services
ETL Data Stores
ETL System Data Stores
Lookup and Decode Tables
Data Quality Data Stores
ETL Metadata
Back Room Summary
Presentation Server Architecture
Business Requirements for Information
Detail Atomic Data
Aggregates
Aggregate Navigation
Design Disciplines within the Presentation Server
Adjusting the Presentation Server Architecture
Organizational Considerations
Presentation Server Metadata
Presentation Server Summary
Front Room Architecture
BI Application Types
BI Management Services
Shared Services
Vendor Specific Architectural Choices
BI Data Stores
Stored Reports
Application Server Caches
Local User Databases
Disposable Analytic Data Stores
Results from Analytic Applications
Downstream Systems
Data Store Security
Desktop Tool Architecture Approaches
BI Metadata
Front Room Summary
Infrastructure
Infrastructure Drivers
Back Room and Presentation Server Infrastructure Factors
Parallel Processing Hardware Architectures
Symmetric Multiprocessing (SMP)
Massively Parallel Processing (MPP)
Non-Uniform Memory Architecture (NUMA)
Clusters
Warehouse Appliances
Partitioning Hardware
Considerations Common to All Parallel Architectures
Hardware Performance Boosters
Disk Issues
CPUs
Secondary Storage
Database Platform Factors
Characteristics of Relational Engines
Characteristics of OLAP Engines
Front Room Infrastructure Factors
Application Server Considerations
Desktop Considerations
Connectivity and Networking Factors
Infrastructure Summary
Metadata
Value of Metadata Integration
Impact Analysis
Audit and Documentation
Metadata Quality and Management
Options for Metadata Integration
Single Source DW/BI System Vendors
Core Vendor Product
Do It Yourself
Metadata Summary
Security
Security Vulnerabilities
Threats to Physical Assets
Threats to Information and Software Assets
Threats to Business Continuance
Network Threats
Security Summary
Create the Architecture
Architecture Development Process
Develop the Application Architecture Plan
Step 1 — Form an Architecture Task Force
Step 2 — Gather Architecture-Related Requirements
Step 4 — Create the Architecture Model
Step 6 — Design and Specify the Subsystems
Step 8 — Review the Draft
Example Application Architecture Plan Outline and Model
Select Products
Keep a Business Focus
Major DW/BI Evaluation Areas
Evaluate Options and Choose a Product
Step 1 — Understand the Purchasing Process
Step 2 — Develop the Product Evaluation Matrix
Step 3 — Conduct Market Research
Step 4 — Narrow Your Options to a Short List
Step 5 — Evaluate the Candidates
Step 6 — Recommend a Product
Step 7 — Trial
Step 8 — Contract Negotiations
Considerations for the Back Room and Presentation Server
Hardware Platform
DBMS Platform
ETL Tool
Considerations for the Front Room
Manage the Metadata
Appoint the Metadata Manager
Create the Metadata Strategy
Secure the System
Secure the Hardware and Operating System
Secure the Development Environment
Secure the Network
Network Components
Encryption
Authenticate the Users
Secure the Data
Provide Open Access for Internal Users
Itemize Sensitive Data
Minimize or Mask Sensitive Data
Secure the Data Access
Task List
Making the Case for Dimensional Modeling
What Is Dimensional Modeling?
What about Normalized Modeling?
Benefits of Dimensional Modeling
Dimensional Modeling Primer
Fact Tables
Fact Table Keys
Fact Table Granularity
Dimension Tables
Dimension Table Keys
Conformed Dimensions
Four-Step Dimensional Design Process
Step 1 — Choose the Business Process
Step 2 — Declare the Grain
Step 3 — Identify the Dimensions
Step 4 — Identify the Facts
Enterprise Data Warehouse Bus Architecture
Planning Crisis
Bus Architecture
Value Chain Implications
Common Matrix Mishaps
Taking the Pledge
More on Dimensions
Date and Time
Surrogate Date Keys
Time of Day
Date/Timestamps
Multiple Time Zones
Degenerate Dimensions
Slowly Changing Dimensions
Type 1: Overwrite the Dimension Attribute
Type 2: Add a New Dimension Row
Type 3: Add a New Dimension Attribute
Mini-Dimensions: Add a New Dimension
Hybrid Slowly Changing Dimension Techniques
Role-Playing Dimensions
Junk Dimensions
Snowflaking and Outriggers
Handling Hierarchies
Fixed Hierarchies
Variable Depth Hierarchies via Bridge Tables
Many-Valued Dimensions with Bridge Tables
More on Facts
Three Fundamental Grains
Transaction Fact Tables
Periodic Snapshot Fact Tables
Accumulating Snapshot Fact Tables
Facts of Differing Granularity and Allocation
Multiple Currencies and Units of Measure
Factless Fact Tables
Consolidated Fact Tables
Fables and Falsehoods About Dimensional Modeling
Fables Caused by Focusing on Departmental Reports
Fables Caused by Premature Summarization
Fables Caused by Overvaluing Normalization
Modeling Process Overview
Get Organized
Identify Design Participants
Revisit the Requirements
Use Modeling Tools
Establish Naming Conventions
Provide for Source Data Research and Profiling
Obtain Facilities and Supplies
Recall the Four-Step Modeling Process
Step 1 – Choose the Business Process
Step 2 – Declare the Grain
Step 3 – Identify the Dimensions
Step 4 – Identify the Facts
Design the Dimensional Model
Build the High Level Dimensional Model
Conduct the Initial Design Session
Document the High Level Model Diagram
Identify the Attributes and Metrics
Develop the Detailed Dimensional Model
Identify the Data Sources
Establish Conformed Dimensions
Identify Base Facts and Derived Facts
Document the Detailed Table Designs
Update the Bus Matrix
Identify and Resolve Modeling Issues
Review and Validate the Model
Perform IT Data Model Review
Review with Core Users
Present to the Business Users
Finalize the Design Documentation
Embrace Data Stewardship
Develop Standards
Follow Naming Conventions
To Null or Not to Null?
Place Staging Tables
Develop File Location Standards
Use Synonyms or Views for User Accessible Tables
Primary Keys
Foreign Keys
Develop the Physical Data Model
Design the Physical Data Structure
Finalize the Source-to-Target Map
Star versus Snowflake
Use a Data Modeling Tool
Develop Initial Sizing Estimates
Build the Development Database
Design Processing Data Stores
Develop the Initial Index Plan
Indexing and Query Strategy Overview
B-Tree Index
Clustered Index
Bitmapped Index
Other Index Types
Star Schema Optimization
Indexing Dimension Tables
Indexing Fact Tables
Indexing for Loads
Indexing for OLAP
Analyze Tables and Indexes after the Load
Design the OLAP Database
OLAP Data Granularity and Drillthrough
Perfecting the OLAP Dimensions
Defining OLAP Calculations
Build the Test Database
Design Aggregations
Deciding How to Aggregate
Deciding What to Aggregate
Maintaining Aggregations
Finalizing Indexes
Design and Build the Database Instance
Memory
Block Size
Save the Database Build Scripts and Parameter Files
Develop the Physical Storage Structure
Compute Table and Index Sizes
Develop the Partitioning Plan
Set up Storage
Fault Tolerance
Storage Area Networks
Configuration of Volumes and Drives
Round Up the Requirements
Business Needs
Compliance
Data Quality
Data Integration
Data Latency
Archiving and Lineage
User Delivery Interfaces
Available Skills
Legacy Licenses
The 34 Subsystems of ETL
Extracting Data
Subsystem 1 – Data Profiling
Subsystem 2 – Change Data Capture System
Subsystem 3 – Extract System
Cleaning and Conforming Data
Improving Your Data Quality Culture and Processes
Subsystem 4 – Data Cleansing System
Quality Screens
Responding to Quality Events
Subsystem 5 – Error Event Schema
Subsystem 6 – Audit Dimension Assembler
Subsystem 7 – Deduplication System
Subsystem 8 – Conforming System
Delivering Data for Presentation
Subsystem 9 – Slowly Changing Dimension Manager
Type 1: Overwrite
Type 2: Create a New Row
Type 3: Add a New Column
Hybrid: Combination of Types
Subsystem 10 – Surrogate Key Generator
Subsystem 11 – Hierarchy Manager
Subsystem 12 – Special Dimensions Manager
Subsystem 13 – Fact Table Builders
Transaction Grain Fact Table Loader
Periodic Snapshot Fact Table Loader
Accumulating Snapshot Fact Table Loader
Subsystem 14 – Surrogate Key Pipeline
Subsystem 16 – Late Arriving Data Handler
Subsystem 17 – Dimension Manager System
Subsystem 18 – Fact Provider System
Subsystem 19 – Aggregate Builder
Subsystem 20 – OLAP Cube Builder
Subsystem 21 – Data Propagation Manager
Managing the ETL Environment
Subsystem 22 – Job Scheduler
Subsystem 23 – Backup System
Backup
Archive and Retrieval
Subsystem 24 – Recovery and Restart System
Subsystem 25 – Version Control System
Subsystem 26 – Version Migration System
Subsystem 27 – Workflow Monitor
Subsystem 28 – Sorting System
Subsystem 29 – Lineage and Dependency Analyzer
Subsystem 30 – Problem Escalation System
Subsystem 31 – Parallelizing/Pipelining System
Subsystem 32 – Security System
Subsystem 33 – Compliance Manager
Subsystem 34 – Metadata Repository Manager
Real Time Implications
Real Time Triage
Real Time Tradeoffs
Real Time Partitions in the Presentation Server
Transaction Grain Real Time Partition
Periodic Snapshot Real Time Partition
ETL Process Overview
Getting Started
Develop the ETL Plan
Step 1 – Draw the High Level Plan
Step 2 – Choose an ETL Tool
Step 3 – Develop Default Strategies
Step 4 – Drill Down by Target Table
Ensure Clean Hierarchies
Develop Detailed Table Schematics
Develop the ETL Specification Document
Develop a Sandbox Source System
Develop One-Time Historic Load Processing
Step 5 – Populate Dimension Tables with Historic Data
Populate Type 1 Dimension Tables
Dimension Transformations
Dimension Table Loading
Load Type 2 Dimension Table History
Populate Date and Other Static Dimensions
Step 6 – Perform the Fact Table Historic Load
Historic Fact Table Extracts
Fact Table Transformations
Fact Table Loading
Test, Test, and Test Again
Develop Incremental ETL Processing
Step 7 – Dimension Table Incremental Processing
Dimension Table Extracts
Identify New and Changed Dimension Rows
Process Changes to Dimension Attributes
Step 8 – Fact Table Incremental Processing
Fact Table Extract and Data Quality Checkpoint
Fact Table Transformations and Surrogate Key Pipeline
Late Arriving Facts and the Surrogate Key Pipeline
Incremental Fact Table Load
Speed Up the Load Cycle
Step 9 – Aggregate Table and OLAP Loads
Step 10 – ETL System Operation and Automation
Schedule Jobs
Handle Predictable Exceptions and Errors Automatically
Handle Unpredictable Errors Gracefully
Maintain Database Objects
Develop and Test ETL Automation
Importance of Business Intelligence Applications
Analytic Cycle for Business Intelligence
Stage 1: Monitor Activity
Stage 2: Identify Exceptions
Stage 3: Determine Causal Factors
Stage 4: Model Alternatives
Stage 5: Take Action and Track Results
More Implications of the Analytic Cycle
Types of Business Intelligence Applications
Direct Access Query and Reporting Tools
Query Formulation
Analysis and Presentation Capabilities
User Experience
Technical Features
Standard Reports
Analytic Applications
Pre-Built Analytic Applications
Read/Write Analytic Applications
Dashboards and Scorecards
Operational Business Intelligence
Data Mining
Data Mining Overview
Data Mining in the Applications Architecture
Navigating Applications via the BI Portal
Density Considerations
Navigation Structure Based on Business Processes
Additional Portal Functions
Application Interface Alternatives
Business Intelligence Application Resource Planning
Role of the BI Application Developer
Who Does the BI Applications Job?
Lifecycle Timing
Business Intelligence Application Specification
Create Application Standards and Templates
Determine Naming Standards
Create the Application Templates
Create Dashboard and Analytic Application Templates
Determine the Initial Application Set
Identify Report Candidates
Consolidate the Candidate List
Prioritize the Report List
Develop Detailed Application Specifications
Specify Application Content
Create the Navigational BI Portal
Set Up Report Scheduling
Test and Verify the Applications and Data
Complete the Documentation
Plan for Deployment
Business Intelligence Application Maintenance
BI Application Specification
Development
System Deployment
Pre-Deployment Testing
System Testing Procedures
Data Quality Assurance Testing
Operations Process Testing
Live Testing
Performance Testing
Usability Testing
Desktop Readiness and Configuration
Deployment
Relational Database Deployment
ETL Deployment
OLAP Database Deployment
Report Deployment
Documentation and Training
Core Documentation
Business Process Dimensional Model Descriptions
Table and Column Descriptions
Report Descriptions
Additional Documentation
User Training
Design and Approach
Develop Training Materials
Create a Training Database
Plan for the Level of Effort
Maintenance and Support
Manage the Front Room
Provide User Support
Maintain the BI Portal
Manage Security
Monitor Usage
Report on Usage
Manage the Back Room
Support Data Reconciliation
Execute and Monitor the ETL System
Monitor Resources
Manage Disk Space
Tune for Performance
Backup and Recovery
Long Term Archiving
Manage the Existing Environment
Reach the Business Users
Manage Up
Measure and Market Your Success
Evaluate ROI
Monitor Success and Service Metrics
Proactively Market the Data Warehouse
Communicate Constantly
Prepare for Growth and Evolution
Assess Your Current Environment
Prioritize Opportunities for Growth
Prioritize Minor Enhancements
Prioritize Major Initiatives
Manage Iterative Growth
Glossary
P. 1
The Data Warehouse Lifecycle Toolkit

The Data Warehouse Lifecycle Toolkit

Ratings:

3.78

(9)
|Views: 11,708|Likes:
Published by Wiley
A thorough update to the industry standard for designing, developing, and deploying data warehouse and business intelligence systems

The world of data warehousing has changed remarkably since the first edition of The Data Warehouse Lifecycle Toolkit was published in 1998. In that time, the data warehouse industry has reached full maturity and acceptance, hardware and software have made staggering advances, and the techniques promoted in the premiere edition of this book have been adopted by nearly all data warehouse vendors and practitioners. In addition, the term "business intelligence" emerged to reflect the mission of the data warehouse: wrangling the data out of source systems, cleaning it, and delivering it to add value to the business.

Ralph Kimball and his colleagues have refined the original set of Lifecycle methods and techniques based on their consulting and training experience. The authors understand first-hand that a data warehousing/business intelligence (DW/BI) system needs to change as fast as its surrounding organization evolves. To that end, they walk you through the detailed steps of designing, developing, and deploying a DW/BI system. You'll learn to create adaptable systems that deliver data and analyses to business users so they can make better business decisions.

A thorough update to the industry standard for designing, developing, and deploying data warehouse and business intelligence systems

The world of data warehousing has changed remarkably since the first edition of The Data Warehouse Lifecycle Toolkit was published in 1998. In that time, the data warehouse industry has reached full maturity and acceptance, hardware and software have made staggering advances, and the techniques promoted in the premiere edition of this book have been adopted by nearly all data warehouse vendors and practitioners. In addition, the term "business intelligence" emerged to reflect the mission of the data warehouse: wrangling the data out of source systems, cleaning it, and delivering it to add value to the business.

Ralph Kimball and his colleagues have refined the original set of Lifecycle methods and techniques based on their consulting and training experience. The authors understand first-hand that a data warehousing/business intelligence (DW/BI) system needs to change as fast as its surrounding organization evolves. To that end, they walk you through the detailed steps of designing, developing, and deploying a DW/BI system. You'll learn to create adaptable systems that deliver data and analyses to business users so they can make better business decisions.

More info:

Publish date: Nov 9, 2011
Added to Scribd: Dec 06, 2011
Copyright:Traditional Copyright: All rights reservedISBN:9781118075043
List Price: $50.00 Buy Now

Availability:

Read on Scribd mobile: iPhone, iPad and Android.
See more
See less

07/06/2014

674

9781118075043

$50.00

USD

You're Reading a Free Preview
Pages 14 to 92 are not shown in this preview.
You're Reading a Free Preview
Pages 106 to 254 are not shown in this preview.
You're Reading a Free Preview
Pages 268 to 554 are not shown in this preview.
You're Reading a Free Preview
Pages 568 to 651 are not shown in this preview.
You're Reading a Free Preview
Pages 665 to 674 are not shown in this preview.

Activity (14)

You've already reviewed this. Edit your review.
1 hundred reads
1 thousand reads
jkl316 liked this
Thembani Kubayi liked this
jkl316 liked this
Vignesh Vicky liked this
Teresa Barros liked this
apgolu liked this

You're Reading a Free Preview

Download