INF 485 | 785

Data Warehousing

2024 Meeting 2
Data Warehouse Project Planning
and Project Management
Meeting Agenda

Introductory Notes

Approaches

Kimball Methodology

Planning and Project Management

The Data Warehouse Methodology

Agenda - Next meeting

CLICK TO EDIT MASTE
Introductory
Subheading
So you want a data warehouse

• Where do you start?

• How do you approach it?
• Who do you involve?
• How do you manage it?
Planning concerns

• Value and expectations

• Risk Assessment
• Which approach to be used ? top down or bottom
• Build or Buy - Should you build your data warehou
• Single Vendor or Best-of- Breed?
• The Overall Plan
• Business Requirements, not Technology
• Top Management Support
• Justify for the data warehouse project
Planning your Data Warehous

• Inadequate project management and imprope

• Determine if your organization really need a d
• Develop criteria for accessing “values” expec
• It is the sole responsibility of the organization
warehouse needs to be built and where it mu
• Who will use the data warehouse?
• How the data warehouse must be used need
Project management consider

• Effective project management plays an very i

of a data warehouse project.
• In project management considerations, variou
considered, basic project management princi
possible success factors are listed.

…. Typical project manag

Approaches
CLICK TO EDIT MASTE
Which is relevant and which is not?
Subheading
General system development

1. Prototyping?
2. Extreme programming and development?
3. Agile development cycles?
4. Traditional System Analysis and Design?
5. Waterfall technique?

Consider data volume, urgency, cycles, iterations, target a

business focus, business drivers, final objectives and if a d
….. considering …..

• Dimensional analysis? Building a data warehouse an

Methods used for gathering requirements for operation
warehouse. The usage of information is unpredictable.
• Operational systems are formulated for discrete tasks
The consideration is the here and the now.

• Dimensional nature of business data? The dimensi

the following queries:
• Measurement units which are important for a sp
• Every user department can convey the attribute
any particular department.
• Insights can be gained from the user about how
combined for the purpose of strategic decision m
• The consideration with regards to a DW? The past an
How Data Warehousing projec
• Operational systems are modeled to automate and/or rec
• Data warehouse projects are increasingly being driven by
strong business justifications.
• Data warehouse front-end applications tend to support ad
opposed to static data entry forms and canned reports
• DW systems are built up with data from existing transacti
are critical steps
• Security and Compliance are two areas that require spec
organization will be combined together
• Operational systems capture revenues fairly accurately, h
costs are allocated against revenues often raise complex
• Technology resources are non-trivial: Data storage, Archi
etc.
Kimball Methodology
CLICK TO EDIT MASTE
• Focus on adding business value across the ente
• Subheading
Dimensionally structure the data that’s delivered
• Iteratively develop the DW/BI environment in ma
than attempting a galactic Big Bang approach
So today….

• We will be demystifying some of the aspects

1. First we will cover the DW Project Manag
2. We will cover the DW methodology.

• We will cover the designing and building in la

• Please note that the following notes are base

highly recommended and recommended text

• Warning: There is a lot of text on the way……

diagrams….
Planning
CLICKand Project
TO EDIT Ma
MASTE
• This is mostly traditional project manag
Subheading
• There are critical variations that is poin
Planning & organizing the dat

• Defining Scope and Objectives **

• Avoiding Major Data Warehouse Mistakes
• Choosing Enterprise Data Warehouse vs.
• Getting the Right Sponsor
• Forming the Team **
• Producing the Project Roadmap and Plans
• Determining the Budget
• Training the Team
Defining Scope and Objective

• Defining the correct scope and setting realistic • D

objectives are key to data warehouse project s
success. Scope defines project boundaries c
including: p
• Business requirements addressed
• Users • T
• Subject Areas

• Objectives define project success criteria

including quantified planned benefits.
Avoiding Mistakes… Common

• Focus on technology instead of people and proce

• Lack of sponsorship and management support
• Overly ambitious or undefined scope
• Undefined requirements
• Unrealistic expectations
• Failure to architect a long term solution
• Failure to obtain high quality data
• Failure to consider future requirements
• Trying to turn the prototype into the final solution
• Designed around one tool/vendor
• Failure to scale up
• Failure to store at the right level of detail / grain
Enterprise Data Warehouse vs

• The Enterprise Data Warehouse is: • T

• Enterprise Wide
• All purpose
• Takes 2 to 5 Years to Build
• Requires Executive Sponsor
Typical PM: Getting The Right

• A good project sponsor typically is a:

• Person with large stake in the project outcome
• Person with authority over resources appropriat
more authority and resources than Data Mart)

• The project sponsor fills a number of roles inc

• Developer of the business case
• Harvester of benefits
• Overseer of the project
• Link to upper management
• Project champion - promoting the project across
Forming the Data Warehousin

• Sponsor and Data Warehousing Champions

• Project Leader / Manager
• Business Subject Matter Experts (SMEs)
• Coaches
• Business Analyst
• Enterprise Architect
• Data Warehouse Trainer
• Data Modeler
• Database Administrator
• Technical Architect
• Extract/Transform/Load Designer/Developers
Typical PM: Project Roadmap

• Develop Scope Definition

• Develop Work Package Plan
• Develop Project Schedule
• Assemble Project Budgets
• Approve Plan
Typical PM: Project Questions

• What are the expected outcomes of this project?

• What are the inputs and outputs?
• Why are we doing this project?
• When will the project begin and end?
• How will the outcomes be accomplished?
• Where will the project take place?
• Who is involved with this project?
• Who is the customer?
• Who is the sponsor?
• Who will contribute to the project?
Typical PM: The Proposed Pro

• The purpose of the proposed project plan is:

• Communication tool (decision makers and t
• Decision making tool (should this project be
• Guides the project phases and activities

• The proposed project plan contains:

• Overview of project (Mission, Scope, Goals,
• Project activity description (Activity List and
• Project timeline and critical path
• Resource requirements
• Cost estimate / budget
Typical PM: Developing Scope

• Scope plan
• Scope definition
• Alternative development
Typical PM: Develop Work Pac

• Activity reconciliation - Prioritize, combine

• Activity definition
• Activity selection
• Work Breakdown Structure (WBS)
Typical PM: Determining the B

• People (Employees, Consultants)

• Hardware
• Software (Development, Infrastructure)
• External Data
Training the Team **

• Project Management
• Data Warehouse Architecture
• Data Warehouse Modelling
• Requirements Analysis
• Specific Tools
The Data
CLICKWarehouse Me
TO EDIT MASTE
The typical approach followed. This is ve
Subheading
adapted to different situations, however
carefully as a data warehouse project is
Data Warehousing Methodolog

• Initiation: Evaluating Readiness and Opportunities

• Analysis: Analysis and Requirements Determination
• Design: Data Warehouse and Data Mart Models (Star
• Design: Technical Architecture
• Design: Obtain Data Warehouse Inputs
• Construct: Data Load
• Construct: Presentation/Analysis Tools
• Quality Assurance: Test
• Rollout : Deploy in Production
• Iterate: Make Incremental Changes
DW Methodology - vs - IT Meth

• Data Warehousing Methodology:

• Use of data is exploratory and less predictab
• Multidimensional Modeling
• Focus is on loading and presenting data

• Traditional IT Methodology:
• Automated processes are repeated and pre
• ERD Data Modeling
• Focus is on rapid on-line updating of data
Initiation: Evaluating Readines

• If there are business drivers, an appropriate • Y

project sponsor and an organizational culture
that includes use of data for decisions and
cooperation between the business and IT, then
the organization is ready.
• Examples of business drivers include
• Competitive advantage or survival
• High potential revenue gain (need new
markets, products, etc.)
• High potential cost savings
Analysis: Requirements Defin

• Requirements describe the needed solution in business term

Requirements for Data Warehousing are defined.

• Who do you involve?

• Stakeholders
• Users
• Requirements
• Data elements: fact classes, dimensions
• Recording of data in terms of time
• Data extracts from source systems
• Business rules: attributes, ranges, domains, operatio
Requirements Definition Docu
1. Introduction. State the purpose and scope of the project. Incl
executive summary of each subsequent section.
2. General Requirements Descriptions. Describe the source sy
Broadly state what types of information requirements are need
3. Specific Requirements. Include details of source data neede
requirements. Describe the types of information delivery metho
4. Information Packages. Provide as much detail as possible fo
form of package diagrams.
5. Other Requirements. Cover miscellaneous requirements suc
methods, and locations to which information must be delivered
6. User Expectations. State the expectations in terms of problem
expect to use the data warehouse.
7. User Participation and Sign-Off. List the tasks and activities
throughout the development life cycle.
8. General Implementation Plan. At this stage, give a high-leve
Design: Technical Architecture

• Data warehousing Technical Architecture is the high le

system. It includes technical specifications for:
• Metadata Management
• Input Sources
• Extracting
• Middleware
• Physical Storage and Operation
• Database Management System
• Mapping, Transforming, Enriching, and Loading
• Communicating
• Analyzing and Presenting
• Managing, Operating, and Securing
Design: Data Warehouse Mode

• Data Mart
• Dimensional / Star Schema Data Model

• Data Warehouse
• Entity Relationship Diagram
• Dimensional / Star Schema Data Model

• Staging
• Entity Relationship Diagram
Construct: Obtain Data Wareh

In this phase we select that data that will be inclu

system. To learn how good the data is we use d
assessment.
Construct: Extract, Transform

• Extract data from the data source

• Transform of modify data so that it is ready to be l
• Load so that data can be placed in the appropriate

• The ETL Process

• Capture
• Scrub or data cleansing
• Transform
• Load and Index
Construct : Presentation/Anal

• The groundwork for presentation and analysis

• Will be used of statistical evaluation engine
Business Intelligence Dashboards.

• Metadata must defined so that data warehousing

such as:
• Query from Multiple Dimensions
• Drill Down - explore greater detail
• Roll Up - summarize
• Pivot - change query directions
Quality Assurance: Testing

• Test the data warehouse against set param

limited to:
• Requirements validation
• Test cases
• Test data
• Scenarios and simulations
• Typical system tests (load test, smoke
• User acceptance
Rollout : Deploy in Production

• Data and Application Readiness

• Installation in the Production Environment
• Training

…....
Iterate : Make Incremental Cha

• Cycle though design and change management pr

• As needs change you might need to:
• Create an additional data warehouse
• Create new data marts
• Create new dashboards
• Create new analytical models
So w
co
CLICK TO EDIT MASTE
Agenda - Next
Subheading
2024 - Meeting 3

• Prof. O Daramola
• Class Test 1  First 15 minutes.
• Meeting 3 - Data Warehouse Architecture an
• Class attendance at end of session.