You are on page 1of 27

Data Warehouse

Concepts
&
Architecture

© Principle Partners, Inc.


Info@PrinciplePartners.Com
Page 1
PP I
Data Warehouse Concepts

Topics To Be Discussed:

• Why Do We Need A Data Warehouse ?

• The Goal Of A Data Warehouse ?

• What Exactly Is A Data Warehouse ?

• Comparison Of A Data Warehouse And


An Operational Data Store.

• Data Warehouse Trends.

© Principle Partners, Inc.


Info@PrinciplePartners.Com
Page 2
PP I
Data Warehouse Concepts

Why Do We Need A Data Warehouse ?

We Can Only
See - What We
Can See !

© Principle Partners, Inc.


Info@PrinciplePartners.Com
Page 3
PP I
Data Warehouse Concepts

Why Do We Need A Data Warehouse ?

BETTER !
FASTER ! FUNCTIONALLY COMPLETE !
CHEAPER !

© Principle Partners, Inc.


Info@PrinciplePartners.Com
Page 4
PP I
Data Warehouse Concepts
Data Warehouse Development Perspective

Data Driven Vs. Function Driven

A/P
O/P
Order
Processing
Data
EIS
Data
DSS

© Principle Partners, Inc.


Info@PrinciplePartners.Com
Page 5
PP I
Data Warehouse Concepts

What Do We Need To Do ?

Use Operational Legacy Systems’ Data:


To Build Operational Data Store,
That Integrate Into Corporate Data Warehouse,
That Spin-off Data Marts.

Some May Tell You To Develop These In Reverse!

© Principle Partners, Inc.


Info@PrinciplePartners.Com
Page 6
PP I
Data Warehouse Concepts

Our Goal for A Data Warehouse ?

• Collect Data-Scrub, Integrate & Make It Accessible

• Provide Information - For Our Businesses

• Start Managing Knowledge

• So Our Business Partners Will Gain Wisdom !

© Principle Partners, Inc.


Info@PrinciplePartners.Com
Page 7
PP I
Data Warehouse Concepts

Data Warehouse Definition


A Data Warehouse Is A Structured Repository
of Historic Data.

It Is Developed in an Evolutionary Process


By Integrating Data From Non-integrated
Legacy Systems.

It Is Usually:
• Subject Oriented
• Integrated
• Time Variant
• Non-volatile

© Principle Partners, Inc.


Info@PrinciplePartners.Com
Page 8
PP I
Data Warehouse Concepts

Subject Oriented
Data is Integrated and Loaded by Subject

Cust 2001

2003
Prod D/W
Data
2002

O/P
2002

A/R

© Principle Partners, Inc.


Info@PrinciplePartners.Com
Page 9
PP I
Data Warehouse Concepts
Time Variant

Operational System Data Warehouse

• View of The Business • Designated Time Frame


Today (3 - 10 Years)

• Operational Time Frame • One Snapshot Per Cycle

• Key Need Not Have Date • Key Includes Date

© Principle Partners, Inc.


Info@PrinciplePartners.Com
Page 10
PP I
Data Warehouse Concepts

Integrated

Operational Systems

Order Processing Order ID = 10


D/W
Accounts Receivable Order ID = 12
Order ID = 16
Product Management Order ID = 8

HR System Sex = M/F


D/W
Payroll Sex = 1/2
Sex = M/F
Product Management Sex = 0/1

© Principle Partners, Inc.


Info@PrinciplePartners.Com
Page 11
PP I
Data Warehouse Concepts

Non-Volatile

Operational System Data Warehouse

• “CRUD” Actions • No Data Update

Insert
Read
Load
Create
Read Read

Update Replace Read

Delete Read

© Principle Partners, Inc.


Info@PrinciplePartners.Com
Page 12
PP I
Data Warehouse Concepts
Data Warehouse Environment Architecture
Contains Integrated Data From Multiple Legacy Applications

Update
A/P
Insert Data
Load Mart
O/P
Integration Read
ODS
Replace
Pay Data
Criteria Mart
All Or Part Delete
Of System of
Mktg ria
Record Data
C rite
d
L oa
HR e
o us Data
a reh Mart
A/R a W D/W Load Loads
a t
D
Best System of Read D/W
Record Data

© Principle Partners, Inc.


Info@PrinciplePartners.Com
Page 13
PP I
Data Warehouse Concepts
Meta Data - Map of Integration
The Data That Provides the “Card Catalogue” Of
References For All Data Within The Data Warehouse

System of Record
Data Source

D/W Structure
Source Data Structure

Definition

Allowable
Domains Aliases

Data Relationships

© Principle Partners, Inc.


Info@PrinciplePartners.Com
Page 14
PP I
Data Warehouse Concepts

ODS Vs. Data Warehouse

Operational Data Store Data Warehouse

Characteristics: Data Focused Integration Subject Oriented


From Transaction Processing Integrated
Focused Systems Non-Volatile
Time Variant
Age Of The Data: Current, Near Term Historic
(Today, Last Week’s) (Last Month, Qtrly, Five
Years)
Primary Use: Day-To-Day Decisions Long-Term Decisions
Tactical Reporting Strategic Reporting
Current Operational Results Trend Detection
Frequency Of Load: Twice Daily , Daily, Weekly Weekly, Monthly, Quarterly

© Principle Partners, Inc.


Info@PrinciplePartners.Com
Page 15
PP I
Data Warehouse Concepts
Building The Data Warehouse

Tasks Deliverables

• Define Project Scope • Scope Definition


• Define Business Reqmts • Logical Data Model
• Define System of Record • Physical Database Data
Data Model
• Define Operational Data • Operational Data Store
Store Reqmts Model
• Map SOR to ODS • ODS Map
• Acquire / Develop • Extract Tools and
Extract Tools Software
• Extract Data & Load ODS • Populated ODS

© Principle Partners, Inc.


Info@PrinciplePartners.Com
Page 16
PP I
Data Warehouse Concepts
Building The Data Warehouse
(Continued)

Tasks Deliverables

• Define D/W Data Reqmts • Transition Data Model


• Map ODS to D/W • D/W Data Integration Map
• Document Missing Data • To Do Project List
• Develop D/W DB Design • D/W Database Design
• Extract and Integrate D/W • Integrated D/W Data
Data Extracts
• Load Data Warehouse • Initial Data Load
• Maintain Data Warehouse • On-going Data Access
and Subsequent Loads

© Principle Partners, Inc.


Info@PrinciplePartners.Com
Page 17
PP I
Data Warehouse Concepts

Relationship Among Data Warehouse Data Models

Business Business
Partner Business Requirements Requirements

Knowledge Logical
& Wisdom Model
Data
Warehouse Strategic
Validation Business
Structured
of Current Physical Requirements
Model Requirements
Data

Operational
Data Store
Physical Tactical Business
Data Model Reqmts & Structures
Load

Current Data Whse


Database Requirements
Current Structure
Physical Transition
Model Model

© Principle Partners, Inc.


Info@PrinciplePartners.Com
Page 18
PP I
Data Warehouse Concepts
Sources of Data Warehouse Data

Archives
(Historic Data)

Current Systems
of Record
(Recent History)

Enterprise
Data Warehouse
Operational
Transactions
(Future Data Source)

© Principle Partners, Inc.


Info@PrinciplePartners.Com
Page 19
PP I
Data Warehouse Concepts

Appropriate Uses of Data Warehouse Data

• Produce Reports For Long Term Trend Analysis

• Produce Reports Aggregating Enterprise Data

• Produce Reports of Multiple Dimensions


(Earned revenue by month by product by branch)

© Principle Partners, Inc.


Info@PrinciplePartners.Com
Page 20
PP I
Data Warehouse Concepts

Inappropriate Uses of Data Warehouse Data

• Replace Operational Systems

• Replace Operational Systems’ Reports

• Analyze Current Operational Results

© Principle Partners, Inc.


Info@PrinciplePartners.Com
Page 21
PP I
Data Warehouse Concepts

Levels of Granularity of Data Warehouse Data

•Atomic (Transaction)

•Lightly Summarized

•Highly Summarized

© Principle Partners, Inc.


Info@PrinciplePartners.Com
Page 22
PP I
Data Warehouse Concepts

Options for Viewing Data

• Text

9 0
8 0
7 0
6 0
5 0

• 4
3
2
1
0
0
0
0
0
1 s t 2 n d 3 rd 4 th
Q tr Q tr Q tr Q tr

© Principle Partners, Inc.


Info@PrinciplePartners.Com
Page 23
PP I
Data Warehouse Concepts

Next Steps In Data Warehouse Evolution

• Use It - Analyze Data Warehouse Data

• Determine Additional Data Requirements

• Define Sources For Additional Data

• Add New Data (Subject Areas) to


Data Warehouse

© Principle Partners, Inc.


Info@PrinciplePartners.Com
Page 24
PP I
Data Warehouse Concepts

Future Trends In Data Warehouse

• Increased Data Mining


Exploration
Prove Hypothesis

• Increase Competitive Advantage


(i.e., Identify Cross-selling Opportunities)

• Integration into Supply Chain & e-Business

© Principle Partners, Inc.


Info@PrinciplePartners.Com
Page 25
PP I
Data Warehouse Concepts
Summary

A Data Warehouse Is A Structured Repository


of Historic Data.

It Is: • Subject Oriented


• Integrated
• Time Variant
• Non-volatile
It Contains:
• Business Specified Data,
To Answer Business Questions

© Principle Partners, Inc.


Info@PrinciplePartners.Com
Page 26
PP I
Data Warehouse Concepts

Questions and Answers

© Principle Partners, Inc.


Info@PrinciplePartners.Com
Page 27
PP I