P. 1
BW EDW Workshop

BW EDW Workshop

|Views: 493|Likes:
Published by jagunsojr

More info:

Published by: jagunsojr on Jan 13, 2012
Copyright:Attribution Non-commercial

Availability:

Read on Scribd mobile: iPhone, iPad and Android.
download as PDF, TXT or read online from Scribd
See more
See less

08/12/2013

pdf

text

original

PDEBW1 PDEBW1

The Layered Scalable Architecture (LSA) for BW on Corporate The Layered Scalable Architecture (LSA) for BW on Corporate
Scale (EDW) Scale (EDW)
Juergen Haupt, Architect
SAP Intelligence Platform and NetWeaver RIG EMEA
Daniel Rutschmann, Specialist
SAP Palo Alto - Service
BW Layered, Scalable Architecture (LSA) for EDW
BI Maturity & Data Logistic 11
Model-driven SAP BW as DWH Best Practice
LSA Implementation
33
44
22
Agenda PDEBW1
Recap of LSA - LSA Related Topics 55
BW Data Model and Data Integration 66
Appendix - BI Organization & Governance 77
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 2
BI Maturity & Data Logistic
BI is the Top Priority of CIOs 1.I 1.I
11
Agenda PDEBW1
Data Logistic Types and BI Maturity - Overview 1.II 1.II
Simple Data Logistics – Data Mart Based 1.III 1.III
Advanced Data Logistics – Data Warehouse Based 1.IV 1.IV
BI Data Logistic Excellence – Hinderers & Enablers 1.V 1.V
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 3
The CI Os Vi ew and Pr i or i t i es
Information
Excellence
Process
Excellence
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 4
BI & Ent erpr i se Appl i c at i on Ex c el l enc e dr i ve
BI Dat a Logi st i c Ex c el l enc e
Information Excellence
(Prio 1: BI Applications)
Process Excellence
(Prio 2/4: Enterprise Applications)
Dat a
Logi st i c
BI
No BI-, CPM- and
Planning Excellence
without trustable,
accurate, consistent Data
= standardization &
consolidation of BI data
logistic (EDW)
Standardization and
consolidation of Enterprise
Applications (processes) =
standardization &
consolidation of BI data
logistic (EDW)
BI
Dat a
Logi st i c
Simple Truth: Simple Truth:
No BI without Data No BI without Data
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 5
A Val i d BI Dat a Logi st i c as
Pr erequi si t e f or BI Val ue
BI Busi ness
Val ue Dr i ver s
SAP ERP SAP CRM Other
Legacy
Data-Infrastructure, Data-Logistic
BI Applications and Functionality
Standard Reporting, Agile BI
Corporate Performance Management
BI Fr amewor k
Þ Consistency
Þ Flexibility
Þ Efficiency
Þ Suitability
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 6
BI Maturity & Data Logistic
BI is the Top Priority of CIOs 1.I 1.I
11
Agenda PDEBW1
Data Logistic Types and BI Maturity - Overview 1.II 1.II
Simple Data Logistics – Data Mart Based 1.III 1.III
Advanced Data Logistics – Data Warehouse Based 1.IV 1.IV
BI Data Logistic Excellence – Hinderers & Enablers 1.V 1.V
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 7
TDWI – Ac c el er at i ng BI Mat ur i t y
The Chasm on The Way t o The EDW Shor e
Figure 1. The TDWI Maturity Model shows how many organizations evolve their BI solutions.
The Gulf and Chasm represent places where many companies get stuck; the bell shaped curve
represents the number of companies in each stage; and the labels above the bell curve
represent the data structure dimension in the model.
Business Value
Integration
Consolidation
3. Child 4. Teenager 5. Adult 6. Sage
GULF
“ Management
Reporting ”

“ Data
Marts”
“ Data
Warehouses ”

DW”
“ BI Services ”
1. Prenatal 2. Infant
GULF
CHASM
“Management
Reporting”
“Spreadmarts”
“Data
Marts”
“Data
Warehouses”
“Enterprise
DW”
“BI Services”
Moving towards a centralized EDW based architecture means we have
to overcome profound challenges (chasm):
Non-IT ones like politics, missing strategy and sponsorship and
IT ones .....
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 8
Cor r el at i on of BI Val ue & Dat a Logi st i c s Ex c el l enc e
The Need of Cent r al i zat i on
Operational
Reporting
Spreadmarts Data Marts
Data
Warehouses
Enterprise
DW
Architecture
Financial
System
Executive
System
Analytical
System
Monitoring
System
Strategic
System
Business
Service
Type of
System
“ Drive the
Business ”
“ Drive the
Market ”
“ Monitor
Performance ”
“ Empower
Workers ”
“ Inform
Executives ”
“ Cost
Center ”
Executive
Perception
Value
Cost
ROI
Infant Child Adult Teenager Sage
Static Reports Spreadsheets
OLAP/
Ad hoc Reports
Dashboards/
Scorecards
Analytical
Tools
Predictive
Analytics
Customer BI
Embedded BI
Prenatal
ROI
Analytic
Services
Operational
Reporting
Spreadmarts Data Marts
Data
Warehouses
Enterprise
DW
Architecture
Financial
System
Executive
System
Analytical
System
Monitoring
System
Strategic
System
Business
Service
Type of
System
“ Drive the
Business ”
“ Drive the
Market ”
“ Monitor
Performance ”
“ Empower
Workers ”
“ Inform
Executives ”
“ Cost
Center ”
Executive
Perception
“ Drive the
Business ”
“ Drive the
Market ”
“ Monitor
Performance ”
“ Empower
Workers ”
“ Inform
Executives ”
“ Cost
Center ”
Executive
Perception
Infant Child Adult Teenager Sage
Static Reports Spreadsheets
OLAP/
Ad hoc Reports
Dashboards/
Scorecards
Analytical
Tools
Predictive
ROI
Beyond the Basics: Accelerating BI Maturity
Wayne W. Eckerson
Director, TDWI Research
The Data Warehousing Institute 2007
Best practice: centralize before you federate
‘...interestingly, organizations that try to federate development and
administration before they have centralized these tasks often have
limited success.’
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 9
BI Maturity & Data Logistic
BI is the Top Priority of CIOs 1.I 1.I
11
Agenda PDEBW1
Data Logistic Types and BI Maturity - Overview 1.II 1.II
Simple Data Logistics – Data Mart Based 1.III 1.III
Advanced Data Logistics – Data Warehouse Based 1.IV 1.IV
BI Data Logistic Excellence – Hinderers & Enablers 1.V 1.V
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 10
BI Dat a-Logi st i c Types
‘St andal one’ Dat a Mar t Ar c hi t ec t ure
Continental Europe:
Sources
Data
Marts
Business Value
Semantic Integration
Data Consolidation
1. Prenatal 2. Infant 3. Child 4. Teenager 5. Adult 6. Sage
GULF
CHASM
“Management
Reporting”
“Spreadmarts”
“Data
Marts”
“Data
Warehouses”
“Enterprise
DW”
“BI Services”
Business Value
Semantic Integration
Data Consolidation
1. Prenatal 2. Infant 3. Child 4. Teenager 5. Adult 6. Sage
GULF
CHASM
“Management
Reporting”
“Spreadmarts”
“Data
Marts”
“Data
Warehouses”
“Enterprise
DW”
“BI Services”
“Management
Reporting”
“Spreadmarts”
“Data
Marts”
“Data
Warehouses”
“Enterprise
DW”
“BI Services”
A definition:
An implementation of an analytics
application serving a single
department, subject area, or limited
part of the organization. Usually
refers to a physical platformon
which summarized data is stored for
decision support.
Intra-Departmental:
Complexity &
Misalignment
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 11
‘St andal one’ Dat a Mar t s
Sales
Standalone Data Marts are independently built BI solutions often
based on different BI tools.
Þ Different data models i.e. inconsistent semantics & values
Þ Different data management for Data Mart and different BI tools
Þ Inhomogeneous, multiple extractions & staging
High departmental acceptance but islands (silos, stovepipes)
HR
Material
Management
Sources
Standalone
Data Marts
#
Employee
Material
#
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 12
BI Dat a-Logi st i c Types – Fl i er :
St andal one Dat a Mar t s
Standalone Data Marts = Departmental BI
1. Organization
Þ Departmental, High daily involvement by local Business Analyst
Þ Limited IT support
2. Architecture, Reach of Data Logistic
Þ Inhomogeneous, multiple extractions & staging
Þ Data Mart tools have often own data management, multi-dimensional, sql-generator, mdx
Þ Departmental reach
3. Data Model
Þ No integration between Data Marts,
Þ Different data models i.e. inconsistent semantics & values
4. Issues
Þ Cross-departmental view: misalignment, islands, communication confusion, high costs,
not pursuable, ...
5. Advantages
Þ Departmental view: flexible, autonomy
6. Adequateness for Customer types
Þ Adequate only from departmental perspective or for small customer (< 100) with IT
knowledge
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 13
BI Maturity & Data Logistic
BI is the Top Priority of CIOs 1.I 1.I
11
Agenda PDEBW1
Data Logistic Types and BI Maturity - Overview 1.II 1.II
Simple Data Logistics – Data Mart Based 1.III 1.III
Advanced Data Logistics – Data Warehouse Based 1.IV 1.IV
BI Data Logistic Excellence – Hinderers & Enablers 1.V 1.V
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 14
BI Dat a-Logi st i c Types
Dat a War ehouses
Þ BI tools work on Data Marts or more general on special prepared sets of data
Þ Because of the missing integration (islands) between Data Marts, Data
Warehouses are established as a common integrated data foundation for Data
Marts.
ÞTo show the new quality we call these kind of Data Marts ‘Architected’
Business Value
Semantic Integration
Data Consolidation
3. Child 4. Teenager 5. Adult 6. Sage
GULF
“ Management
Reporting ”

“ Data
Marts”
“ Data
Warehouses ”

DW”
“ BI Services ”
1. Prenatal 2. Infant
GULF
CHASM
“Management
Reporting”
“Spreadmarts”
“Data
Marts”
“Data
Warehouses”
“Enterprise
DW”
“BI Services”
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 15
BI Dat a-Logi st i c Types
Dat a War ehouse Arc hi t ec t ur e
Sources Data
Marts
Data
Warehouse
O
r
g

U
n
i
t

A
Staging
Area
Bill Inmon - The Data Warehouse
‘The Data Warehouse is
a subject-oriented, integrated, time-variant, non-volatile
collection of data used to support the strategic decision-making process for
the enterprise. It is the central point of data integration for business
intelligence and is the source of data for the data marts, delivering a common
view of enterprise data.’
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 16
A Dat a War ehouse as I nt egr at ed Foundat i on
f or Dat a Mar t s – Arc hi t ec t ed Dat a Mar t s
Material
Management
Sales HR
Material
=
Employee
=
Staging Area
Sources
employee
material
customer
vendor
Data Warehouse
Architected
Data Marts
Before Data Marts are loaded the data are integrated to common semantics
given by the dwh data model and the values are harmonized.
The integrated results are stored persistently and build the data warehouse.
Data Marts are filled from the Data Warehouse.
A Data Warehouse is a necessary condition for consistent BI.
Data Warehouse:
Þ Standardized extraction & load
Þ Integrate data with respect to
integrated semantics & values
Þ Store Integrated results
Þ Load integrated data into Data
Marts (Architected Data Marts)
Þ From this perspective the
data warehouse represents a
layer in the information
system architecture
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 17
Bi l l I nmon’s Corpor at e I nf ormat i on Fac t or y &
The Dat a War ehouse
Copyright ©1999 by William H. Inmon
Conceptual Details
Þ Subject-oriented
Þ Integrated
Þ historical complete
Þ comprehensive
Þ application neutral
Þ granular
Þ Higher organizational-unit owned
Þ non-volatile…
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 18
Dat a War ehouse Pr i nc i pl es
I nf or mat i on Compl et eness, Reusabi l i t y, Gr anul ar i t y
Customer Dimension Material Dimension
A 4712
4713
Customer Material Time
Amount
Company
Currency
A 4712 200211 100
A 4713 200211 150
Time Dimension
200211
Customer Time DocNo Pos Material
Amount
Local
Currency
Amount
Company
Currency
A 20021107-10am 1 10 4711 100 50
A 20021107-3pm 1 10 4712 200 100
A 20021107-4pm 2 10 4713 300 150
Customer Time DocNo Pos Material
Amount
Local
Currency
A 20021107-10am 1 10 4711 100 New booking
A 20021107-3pm 1 10 4712 200 Correction booking
A 20021107-4pm 2 10 4713 300 New booking
Sourcesystem
BW Data Warehouse
Layer
daily
weekly
daily monthly
other InfoCube other InfoCube
BW Architected Data Mart Layer
InfoCube
granularity : week granularity : day granularity : month
Ex ampl e:
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 19
DWH & Arc hi t ec t ed Dat a Mar t Layer
Introducing a Data Warehouse Layer the architecture has two main Layer:
DWH Layer
Þ no business driven transformations just
cleansing, unification and integration
transformations
Þ Based on common definitions
Architected Data Mart Layer
Þ business driven transformations
Þ business defined
Þ BWA area
BW
D
W
H
A
r
c
h
i
t
e
c
t
e
d
D
a
t
a

M
a
r
t
s
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 20
Bi l l I nmon ‘What i s a Dat a War ehouse?’
Data warehouses are significantly different from data marts.
From ‘Data Mart Does Not Equal Data Warehouse’, Bill Inmon, DM Direct, November 1999
Þ Integrated, subject-oriented
Þ The data warehouse data is integrated from the many legacy sources.
Þ Data warehouses are arranged around the corporate subject areas found in the
corporate data model.
ÞCorporate owned or larger part of the organization
Þ The data warehouse represents a truly corporate/ higher organizational-unit effort.
Þ Usually the data warehouse is built and owned by centrally coordinated organizations
Þ time-variant = complete history, non-volatile = permanent
Þ Granular
Þ The data warehouse contains the most granular data the corporation has. Data mart
data is usually much less granular than data warehouse data. The volume of data found
in the data warehouse is significantly different from the data found in the data mart.
Þ Application neutral, non-flavored
Þ The structure and the content of the data in the data warehouse do not reflect the bias of
any particular department, but represent the corporation's needs for data.
Þ Comprehensive
Þ The data warehouse is broad in scope and keeps more data than actual requested by
the business/ data marts
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 21
BI Dat a-Logi st i c Types – Fl i er :
Dat a War ehouse
Data Warehouse & Architected Data Marts: Integrated, Cross-organizational BI
(on DWH definition level)
1. Organization
Þ IT managed sometimes BI Competency Center (BI CC)
2. Architecture, Reach of Data Logistic
Þ Common extraction & staging techniques (ETL) for all cross departmental Data Marts
Þ Data Integration results have own persistence
Þ Data Marts served by the data warehouse (architected, reliable data marts)
Þ Cross organizational on upper Org-unit
3. Data Model
Þ Cross Subject-area integrated
4. Issues, Challenges
Þ Business-user view: Often IT-driven, missing business acceptance, needs business buy-
in, time to market
5. Advantages
Þ Data warehouse owner view: Integrated reporting, standards, consistency, less TCO than
pure Data Mart architectures
6. Adequateness for Customer types
Þ For independent operating higher organizational units
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 22
BI Dat a-Logi st i c Types
Ent erpr i se Dat a War ehouse (EDW)
Business Value
Semantic Integration
Data Consolidation
3. Child 4. Teenager 5. Adult 6. Sage
GULF
“ Management
Reporting ”

“ Data
Marts”
“ Data
Warehouses ”

DW”
“ BI Services ”
1. Prenatal 2. Infant
GULF
CHASM
“Management
Reporting”
“Spreadmarts”
“Data
Marts”
“Data
Warehouses”
“Enterprise
DW”
“BI Services”
Þ Data Warehouses cover a certain part of an organization (country, business
unit, process e.g. FI), which means for larger companies there exist multiple
Data Warehouses.
ÞThus from a higher level point of view they suffer from similar issues like
standalone Data Marts. The Data Warehouses are often isolated.
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 23
The Mul t i pl e Dat a War ehouse Landsc ape I ssue:
J ust i f i c at i on of an Ent er pr i se Dat a War ehouse
Source
Source
Source
Source
Source
Source
Source
Source
Local
SAP BW
Local
SAP BW
Source
Source
Source
Local
SAP BW
Unit 2
SAP BW
Local
SAP BW
Local
SAP BW
Stream B
DWH
Stream C
SAP BW
Group
SAP BW Stream A
SAP BW
Unit 1
DWH
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 24
BI Dat a-Logi st i c Types
I mpac t s of Mul t i pl e Dat a Warehouses
O
r
g

U
n
i
t

A
O
r
g

U
n
i
t

B
O
r
g

U
n
i
t

C
Often we find multiple DWHs
in an Organization =
Complexity &
Misalignment
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 25
Bi l l I nmon’s Corpor at e I nf ormat i on Fac t or y &
Ent erpr i se Dat a War ehouse (EDW)
Copyright ©1999 by William H. Inmon
Enterprise Data Warehouse (EDW):
A single instantiation of a data
warehouse layer for the entire
corporation or big parts of the
organization is often called the
Enterprise Data Warehouse
EDW-Keywords
Þ offer a ‘single version of truth’
Þ extract once & deploy many
Þ support the ‘unknown’
Þ re-build
Þ new-build
Þ controlled redundancy
Þ provide a corporate memory
Conceptual Details
Þ Integrated
Þ historical complete
Þ comprehensive
Þ application neutral
Þ granular
Þ corporate owned
Þ non-volatile…
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 26
Tr eat i ng I nf or mat i on as Cor por at e Asset -
Fl ex i bi l i t y i s Key of Grow t h & Sur vi val
The Known
Don’t limit yourself by just focusing on
current requirements
The Unknown
Be prepared for
unpredictable future needs
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 27
The Whol e Pi c t ur e – The EDW
SAP BW @Henkel
Peter Hinzmann, CIO of Henkel, summarizes the motivation:
“We are increasingly focusing on the whole picture, which in turn raises the
importance of centralized management of the company. This process is actively
supported by IT.”
Werner Böttiger, IT project manager at Henkel’s business intelligence competence
center:
“The enterprise data warehouse (EDW) layer does lead to a degree of redundant
data, but this can be kept under control.
All transactional data from the source systems need to pass the EDW layer before it
is written to the final data destinations or data marts.
This enables us to safeguard data quality and create a single version of truth that
contains all historical data,”
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 28
BI Dat a-Logi st i c Types – Fl i er :
EDW
EDW & Architected Data Marts: Integrated, Corporate BI
1. Organization
Þ BI Competency Center – BI CC (IT, analysts, business)
2. Architecture, Reach of Data Logistic
Þ Like DWH but for entire Corporation (may be division)
Þ Single Point of Truth, historical complete, comprehensive, granular, integrated,
Þ Architected Data Marts built on EDW
3. Data Model
Þ Cross Company
4. Issues, Challenges
Þ IT as driver, missing C-level sponsorship
Þ Missing business acceptance, needs business buy-in, time to market
5. Advantages
Þ BI on entire value chain,
Þ Integration, standardization, new information culture, competitive advantages
Þ Enabler for consistent ‘BI as service’
6. Adequateness for Customer types
Þ For all customers within highly competitive markets, High volume, Global organization
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 29
Ral ph Ki mbal l ’s vi ew on Dat a War ehouse
Layer
Figure 1.1 The basic elements of the data warehouse.
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 30
Met a Gr oup I
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 31
Bi l l I nmon' s Cor por at e I nf or mat i on Fac t or y and
The Ent er pr i se Dat a War ehouse
DSS Applications
Departmental Data Marts
EDW
Marketing
Acctg Finance
Sales
ERP
ERP
ERP
CRM
eComm.
Bus. Int.
ETL
Global
ODS
Oper.
Mart
Exploration
warehouse/
data mining
* Source: Bill Inmon
S
t
a
g
i
n
g

A
r
e
a
local
ODS
Dialogue
Manager
Cookie
Cognition
Preformatted
dialogues
Cross media
Storage Management
Near line
Storage
Web Logs
Session
Analysis
Internet
ERP
Corporate
Applications
Changed
Data
Granularity
Manager
Archives
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 32
BI Maturity & Data Logistic
BI is the Top Priority of CIOs 1.I 1.I
11
Agenda PDEBW1
Data Logistic Types and BI Maturity - Overview 1.II 1.II
Simple Data Logistics – Data Mart Based 1.III 1.III
Advanced Data Logistics – Data Warehouse Based 1.IV 1.IV
BI Data Logistic Excellence – Hinderers & Enablers 1.V 1.V
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 33
Why ar e t her e Chasms & Gul f s ?
Business Value
Semantic Integration
Data Consolidation
3. Child 4. Teenager 5. Adult 6. Sage
GULF
“ Management
Reporting ”

“ Data
Marts”
“ Data
Warehouses ”

DW”
“ BI Services ”
1. Prenatal 2. Infant
GULF
CHASM
“Management
Reporting”
“Spreadmarts”
“Data
Marts”
“Data
Warehouses”
“Enterprise
DW”
“BI Services”
zero high
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 34
Di f f er ent Per spec t i ves on BI
Busi ness & I T
Business perspective on Information
solution solution, business driven BI business driven BI
Þ Flexible, time to market
Þ Autonomy, agility
Þ Performance
Þ Consistency
Þ High availability
IT perspective on Information
data, technology driven BI data, technology driven BI
Þ Manage BI (Development, Maintenance,
Operation, Administration, Housekeeping)
Þ Lower TCO, TCD (Total Cost of
Ownership, Development)
Þ Reduce number of BI tools (BI
Consolidation)
BI Busi ness
Val ue Dr i ver s
SAP ERP SAP CRM Other
Legacy
Data-Infrastructure, Data-Logistic
BI Applications and Functionality
Standard Reporting, Agile BI
Corporate Performance Management
BI Fr amewor k
Þ Consistency
Þ Flexibility
Þ Efficiency
Þ Suitability
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 35
Þ The preferred BI flavor/ tool differs from
employee to employee, from department
to department, from organizational-unit to
organizational-unit
=Offering different, adequate BI
functionalities to different business-user
types is fine but often results in an
uncontrolled bunch of BI-tools, which
means per se in high costs
But this is not the main issue:
Þ A dispersed BI landscape implies as
experiences show an island like,
dispersed data infrastructure.
=This results in even higher costs but
more important doubts the value of BI
Busi ness Per spec t i ve on BI
Or gani zat i onal Egoi sm
Group
Division A Division B Division C Division D
......
BU
Unit
....... .......
Sales FI HR
Spain
UK
France
Nordic
Different BI tools and
Data infrastructures
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 36
SAP AG 2007, SAP Skills 2007 Conference / B2 / 37
Loc al Busi ness and I ssues of Di sper sed BI Dat a
Logi st i c s
Source
Source
Source
Source
Source
Source
Source
Source
Local
SAP BW
Local
SAP BW
Source
Source
Source
Local
SAP BW
Unit 2
SAP BW
Or gani zat i onal / busi ness and al so I T egoi sms c ause di sper sed
BI dat a-l ogi st i c s. They of t en hi nder any evol ut i onar y st ep
t owar ds an ar chi t ec t ur ed BI dat a-l ogi st i c .
Local
SAP BW
Local
SAP BW
Stream B
DWH
Stream C
SAP BW
Group
SAP BW Stream A
SAP BW
Unit 1
DWH
Di sper sed BI Dat a Logi st i c :
ÞMissing Service level definition
– Uncontrolled data flows
– Uncontrolled extractions
ÞRedundant development
ÞInconsistent data model, KPIs
ÞNo overall BI / DWH Strategy
Thi s r esul t s i n:
Þ Silos, Islands
Þ Unreliable Information-basis
Þ Uncontrolled redundancy
Þ High Costs
Di sper sed BI Dat a Logi st i c :
ÞMissing Service level definition
– Uncontrolled data flows
– Uncontrolled extractions
ÞRedundant development
ÞInconsistent data model, KPIs
ÞNo overall BI / DWH Strategy
Thi s r esul t s i n:
Þ Silos, Islands
Þ Unreliable Information-basis
Þ Uncontrolled redundancy
Þ High Costs
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 37
Why ar e t her e Chasms & Gul f s ?
Non-I T Reasons
Mature BI Data logistics are hindered by local interests:
If there is no organizational momentum toward a common goal, then the best
architecture, the best framework in the world is bound to fail.
W.H. Inmon

But any advanced BI data logistic like an Enterprise Data Warehouse depends on
establishing a high degree of central governance. This again depends on a
Clear commitment of c- (orporate) management (sponsorship) – not CIO
Particular, Local interests overrule Group, Corporate interests
Delivering consistent information across the business depends on shared
definitions and business rules for collecting, processing and delivering it.
Any significant improvement in information – in terms of
quality, consistency, timeliness and accuracy
– depends on gaining control of information processes and definitions.
a customer architecture council

On the other hand side we find the awareness:
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 38
I s I T Ready f or EDW?
I T and I ssues of BI Dat a Logi st i c s
D
W
H
D
W
H
D
W
H
d
a
t
a

m
a
r
t
s
Staging
g
e
n
e
r
a
t
e
d
e
m
a
n
d
Source
Extraction
d
e
m
a
n
d
s
u
p
p
l
y
Staging
Source
Extraction
volume
...
C
o
p
a
Staging
Source
Extraction
no of applications
We mi ss:
ÞService level definitions
ÞGuidelines & Best Practices
ÞQuality-control in Development
ÞComprehensiveness
ÞCompleteness
Þ....
Thi s r esul t s i n:
ÞIncreasing Complexity
– Increasing operational cost
– Increasing maintenance costs
ÞLow flexibility to answer ‘the unknown’
– Increasing time-to-market
ÞConsistency issues
We mi ss:
ÞService level definitions
ÞGuidelines & Best Practices
ÞQuality-control in Development
ÞComprehensiveness
ÞCompleteness
Þ....
Thi s r esul t s i n:
ÞIncreasing Complexity
– Increasing operational cost
– Increasing maintenance costs
ÞLow flexibility to answer ‘the unknown’
– Increasing time-to-market
ÞConsistency issues
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 39
Why ar e t her e Chasms & Gul f s ?
I s I T Ready For EDW ?
BI is totally underestimated and misunderstood by IT! (and others)
BI is highly different than dealing with operative requirements.
Þ BI is daily business-related (at all org-levels) and thus highly volatile/ dynamic
= BI data-logistics must be highly flexible
Þ BI deals with historical data what creates high data volume growth
= BI data-logistics have to manage often 10 times or more of data
Þ BI never ends
BI is implemented incrementally, BI-application after BI-application,
= Data volume growth but just so important,
= Growth of meta-data
BI means steadily increasing maintenance-, administration- and operation-workload
BI data-logistics have to be highly transparent, standardized robust and automated
Þ BI is steadily under time-to-market pressure
= BI data-logistics cannot rely on today. They must ‘support the unknown’ the unforeseeable
.
BI deals with data, operative-business with processes!
As BI is underestimated, the EDW is underestimated!
This is the best prerequisite to fail!
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 40
Crossi ng BI Dat a Logi st i c Gul f s & Chasms –
I s I T Ready f or a BI Ex c el l enc e Fr amew ork ? *
BI Busi ness
Val ue Dr i ver s
SAP ERP SAP CRM Other
Legacy
Infrastructure, Data-Logistic
Enterprise Data Warehouse Layered, Scalable
Architecture
Applications and Functionality
Standard Reporting, Agile BI
Corporate Performance Management
Organization Process
Strategy
Information as Corporate Asset
Methodology
BI Competency Centre
BI Fr amewor k *
Þ Consistency
Þ Flexibility
Þ Efficiency
Þ Suitability
Þ Realization
Þ Alignment
* BI Framework introduced by Gartner
BI Excellence is more than Infrastructure/ Data-Logistic Excellence:
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 41
Tr eat I nf or mat i on as Cor por at e Asset -
Mat ur e BI Dat a-Logi st i c s (EDW)
Þ Increasing awareness about importance of a proper SAP BW data-logistic for Data and
Meta Data
Customer investments for an adequate SAP BW infrastructure to manage data and meta
data consistently and in a comprehensive manner
Þ Activities run under the title ‘Enterprise Data Warehouse’
Þ Most common to all activities is the concept of a layered architecture of information systems
introduced by Bill Inmon = The Corporate Information Factory (CIF)
SAP BW
Layered, Scalable Architecture
(LSA)
C
o
m
p
l
e
x
i
t
y

&

C
o
s
t
:
A
d
m
i
n
i
s
t
r
a
t
i
o
n
,
M
a
i
n
t
e
n
a
n
c
e
No of Application Scenarios (Data Marts)/ Volume
SAP BW
with no corporate standards
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 42
BW Layered, Scalable Architecture (LSA) for EDW
BI Maturity & Data Logistic 11
Model-driven SAP BW as DWH Best Practice
LSA Implementation
33
44
22
Agenda PDEBW1
Recap of LSA - LSA Related Topics 55
BW Data Model and Data Integration 66
Appendix - BI Organization & Governance 77
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 43
BI Dat a-Logi st i c & Appl i c at i ons Model i ng w i t h
SAP BW
M
o
d
e
l
i
n
g

A
c
t
i
o
n
s
O
p
e
r
a
t
e
/

A
d
m
i
n
i
s
t
r
a
t
i
o
n
Business requirements
DWH data model
Multi dimensional model
Physical Star Schema
Logical DWH & Staging
Physical DWH & Staging
Data Transfer,
Transformation, Error-handling
Locate source &
map data & model, extractor
Erwin, Visio, Eclipse (planned)
BCT BW Reference Data Model
BW InfoCube/ MultiCube
Star Schema/ BWA index
BW DSOs
Generated PSA, DSOs tables
InfoPackage, DTPs, DAPs
Rule-editor, Error-DTP
BCT DWH Data Model,
Extractors: BCT, Generic, UDC
Process Design
Deploy BI applications
Monitor BI applications
Performance, Tuning
Run, Recover
Housekeeping
BW Process Chains
BW Transport
BW Monitoring & Statistics
BW Aggregates, BIA, NLS
BW Requests/ Process Chains
BW Housekeeping Proc Chains
BW models and implement all relevant BI
data-logistic areas. Data Warehousing with
BW is end to end model-driven.
This is a prerequisite for designing data
warehouses in a transparent, standardized,
robust and best practice way.
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 44
Non-SAP-Source
data model
BW Model -Dr i ven Dat a War ehousi ng based on
BCT Dat a War ehouse Ref er enc e Dat a Model
Building
blocks
DWH Stores:
DTPs, DSOs
Architected
Data Marts:
InfoCubes/ BWA
SAP-Source
data model
Non-SAP Source B SAP Source A
BW Model-driven DWH:
1. DWH data modeling
Þ Reference data model
InfoObjects + relations
2. DWH structure modeling
Þ InfoProviders
3. DWH operations modeling
Þ Extract, load, transform
Þ Transfer, Error-handling
Þ Admin, Monitor
ETL/ Staging
PSA, InfoPackages map models:
provided by BW
extractor
map models:
user-defined
extractor
Subject-areas
BW is from all perspectives fully model-driven
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 45
SAP BW Model -Dr i ven Dat a War ehousi ng
Corner st ones
Designing a Data-Logistic with SAP BW is from all perspectives fully model-driven:
The Reference Data Model (BCT)
Þ Provides mapping of SAP-source data model to BW reference data model (InfoObjects)
Þ For all Non-SAP sources
Þ Customer does not start on a ‘blank’ sheet of paper (-> reference model)
Þ Mapping of source data model to reference data model by customer
Þ Flexible, expandable
The Meta-objects to design persistent & virtual elements of a data-logistic
Þ Persistent Data-containers (InfoProviders: InfoCube, InfoObject, DSO, PSA)
Þ Access Virtualization (InfoProviders: MultiProvider, Virtual InfoProvider, InfoSet,..)
Þ Staging Virtualization (DataSource, InfoSource)
The Meta-objects to design the operations between persistent elements
Þ Extract, Transport, Load (InfoPackage)
Þ Transformation (Transformation Rules)
Þ Error-Handling
Þ Transfers (DTPs)
Þ Complex data movements (Process chains)
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 46
Conformed dimensions enable virtual SAP BW MultiProvider scenarios
FACT Table
Material_Dimension_ID
..........._Dimension_ID
.............Dimension_ID
.............Dimension_ID
Key figure 1
Key figure 2
InfoCube
B
Local Part of
Material Dimension
Material_Dimension_ID
0MATERIAL
0PROD_HIER
Material Dimension
Table
FACT Table
Material_Dimension_ID
SalesOrg_Dimension_ID
Time_Dimension_ID
Customer_Dimension_ID
Sales Amount
Quantity
InfoCube
A
Material_Dimension_ID
0MATERIAL
Material Dimension
Table
Gebiet 1 Gebiet 2 Gebi et 3
Bezirk 1
Gebi et 3a
Be zi rk 2
Region 1
Gebi et 4 Gebiet 5
Bezirk 3
Regi on 2
Gebiet 6
Bezirk 4
Gebiet 7 Gebiet 8
Bezirk 5
Region 3
Vertriebsorganisation
Material Hierarchy
Table
0MATERIAL
Language Code
Material Name
Material Text Table
Material Master Table
0MATERIAL
Material Type
Local Part of
Material Dimension
Conformed, Shared
Part of
Material Dimension
Material Dimension:
local & conformed part
BW Dat a War ehouse & Ar c hi t ec t ed DMs:
Conf ormed Di mensi ons - Pr event i ng Si l os
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 47
The SAP BW Mul t i Pr ovi der Conc ept
End-User
Reporting & Analysis need
logical (dimensional) model
SAP BW InfoCube(s),DS-Object
Þ physical implementation
Þ basic facts
Þ not report specific
SAP BW MultiProvider
Þ virtual Structures
Þ basic facts
Þ not report specific
Þ optional
SAP BW BEx Query
Þ virtual Structures
Þ complex KPIs
Þ navigation behavior
Þ report (type) specific
SAP BW BEx Report
Þ end-user requirement
specific
SAP BW
Concept of SAP BW MultiProvider
MultiProvider separate physical persistent SAP BW Structures
from Queries and Query dependent BEx-objects
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 48
BW Dat a War ehouse & Ar c hi t ec t ed DMs:
Mul t i Pr ovi der and Fl ex i bi l i t y
+ C1 C1
Þ Expand the scenario
(C1) seamless
introducing a new
scenario
¬Flexibility
¬Save investments
C1 C1
Managing
Þ Workload (logical
partitions)
¬Performance
Þ New content
¬Flexibility
Þ Rebuild
¬Flexibility
SAP BW Queries
virtual
SAP BW MultiProvider
Þ Separate optimal
physical DB Schema
from logical Schema
¬Performance
C1 C1
persistent
SAP BW Structures
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 49
BW Dat a War ehouse & Ar c hi t ec t ed DMs:
Dat a Mar t s Bui l t on BCT Ref erenc e Model
Material
Management
Sales
HR
OLTP
Material
=
=
Employee
Staging Area:
BW PSA
Load:
BW InfoPackage
Architected
Data Marts:
InfoCubes, MPs
Transfer:
DTPs, Open Hub
BW
Þ Achieving Data Mart harmonization applying the BW reference data model
Þ Not necessarily explicit data warehouse persistency
Transformation
Rules
Non-SAP
Extraction:
BW Extractors,
BW DBConnect,
BW Generic,
FlatFile,
Univ.DataConnect
B
W

D
a
t
a
-
L
o
g
i
s
t
i
c

M
o
d
e
l
i
n
g
SAP
B
W

R
e
f
e
r
e
n
c
e

D
a
t
a


M
o
d
e
l
0EMPLOYEE
0MATERIAL
0CUSTOMER
0VENDOR
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 50
BW Dat a War ehouse & Ar c hi t ec t ed DMs:
Li mi t at i ons
The ‘just use’ SAP BW DWH
Þ focused on InfoCubes,
Þ InfoObject master data
Þ more or less summarized
Þ support the known
SAP BWs are built on today's end-user requirements
Þ Designed to answer the known
Þ Answering the unknown is limited by
Þ Granularity of InfoCubes and DS-Objects (data content)
– (historical completeness)
Þ Business-scope driven extraction
– (extracted fields – process comprehensiveness)
Þ Business rules applied to original extracted data
Þ Availability of master data history
Limited Completeness & comprehensiveness
thus overall flexibility is limited to react on time
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 51
BW Dat a War ehouse & Ar c hi t ec t ed DMs:
Summar y
+ Usage of BCT reference data model
=Common model-based consistent Data Marts
+ Relying on the power of the BW model-driven approach
=Enables implementation of even most complex BI scenarios
Þ Customers who have no explicit or corporate BI/ DWH strategy except ‘use SAP
BW’ - Little awareness about the meaning of incremental BI
=limited scalability, increasing TCO when BW grows
Þ Solution focus on InfoCubes/ reporting - usage of intermediate storage (DSOs)
only if required to cover concrete business scope
= Limited flexibility
Þ Little standards & guidelines, which exceeds the generic BW offering
We find the ‘just use SAP BW approach’ with a lot of customers. The
approach works well until the BW-system reaches a certain size in terms of
data-volume and number of BI applications. Then the TCO will increase and
scalability, maintenance, performance and operational issues will likely occur.
For corporate BI data warehouse architectures just relying on BWs generic
data warehouse approach is not sufficient!
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 52
I mpl ement i ng an EDW-Based Arc hi t ec t ur e i n
SAP BW
I definitely need to go
for an EDW-based
layered architecture
for SAP BW!
But how to do it ?
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 53
BW Layered Scalable Architecture (LSA) for EDW
BI Maturity & Data Logistic 11
Model-driven SAP BW as DWH Best Practice
LSA Implementation
33
44
22
Agenda PDEBW1
Recap of LSA - LSA Related Topics 55
BW Data Model and Data Integration 66
Appendix - BI Organization & Governance 77
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 54
The Layered, Scalable Architecture (LSA) for EDW
Reference LSA Overview
3.I 3.I
33
Agenda PDEBW1
LSA Building Blocks – Data Layer
3.III 3.III
LSA Building Blocks – Data Domains
3.V 3.V
LSA Building Blocks – Data Model
3.VI 3.VI
LSA Building Blocks – Layer & Master Data
3.IV 3.IV
Reference LSA & Customer LSA
3.II 3.II
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 55
Ever yt hi ng Seems t o be Cl ear , Doesn‘ t I t ?
• EDW &
Layering
• Chasms
?
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /20090210/ Page 56
The Broad Vi ew of BW on EDW
Traditional EDW SAP BW EDW
Companies
situation
Þoperational world fragmentation
Þno view across the business
ÞIncreasing operational world
harmonization
ÞYet limited view across the business
Scope-Areas
Cross-/
Corporate BI
Þ In scope
Þ little information freshness (monthly)
ÞIn scope
Þ high information freshness (daily)
Þ high flexibility
Local- /
depart. BI
Þ Not in scope ÞIn scope
Þ standard reporting with local adoptions
Þ flexible roll out
Operational BI Þ Not in scope ÞPartly in scope
Evaluation ÞLong time for ROI
ÞHigh Latency (e.g. monthly)
ÞHigh costs
ÞLow synergies for local/
departmental BI
ÞOverall acceptance problems
ÞIncremental approach to EDW
ÞImmediate ROI (local BI)
ÞStd. DWH Latency (day) to Low (RDA)
ÞStandardization lowers costs
ÞHigh synergies for all BI flavors
ÞIncreasing acceptance over time
Note: of course we find also the ‘traditional’ EDW approach using BW!
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 57
Crossi ng t he EDW Chasm:
The BW Layer ed Sc al abl e Ar c hi t ec t ure
SAP Customer Experiences & Feedback :
Nestlé
Kraft Foods
Arla Foods
Adidas
Edeka
Beiersdorf
Henkel
Japan
Tobacco
Philips
Samsung
Novartis
Syngenta
BASF
Land
Hessen
Shell
Downstream
Mc Kesson
Statoil
Best Practices & Best Practices &
EDW Principles EDW Principles
BW Layered Scalable Architecture (LSA) –
The Reference Architecture
for BW on a Corporate Scale
LSA
Nike
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 58
The Layered, Scalable Architecture (LSA) for EDW
Reference LSA Overview
3.I 3.I
33
Agenda PDEBW1
LSA Building Blocks – Data Layer
3.III 3.III
LSA Building Blocks – Data Domains
3.V 3.V
LSA Building Blocks – Data Model
3.VI 3.VI
LSA Building Blocks – Layer & Master Data
3.IV 3.IV
Reference LSA & Customer LSA
3.II 3.II
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 59
The BW Layered Scalable Architecture (LSA)
describes the design of
service-level oriented, scalable, best practice BW
architectures founded on accepted EDW principles*.
The BW LSA serves as a reference architecture
to design transparent, complete, comprehensive
customer DWH architectures (Customer LSA).
The Customer LSA describes corporate standards
to build BI applications in a
performant, maintainable, flexible manner.
BW Layer ed Sc al abl e Ar c hi t ec t ure (LSA)
* As introduced in Bill Inmon‘s Corporate Information Factory (CIF)
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /20090210/ Page 60
BW LSA: The Ref erenc e Arc hi t ec t ur e
LSA
Building Blocks
Reference
Describes
core structures &
definitions
LSA
Implementation
Reference
Describes
design standards to
build BI applications
founded on building
blocks
LSA
Operations
Reference
Describes
Support
Scenarios
Meta Data Management
ONaming Conventions
OOrganization (InfoAreas)
Life Cycles
OInformation
OMeta Object
OLSA
Administration
OData Base
OHousekeeping
OMonitoring
Transport
Security
Data Staging/ Management for
transactional & master data
OPersistent Objects
OFlows - scheduled/ recovery
OTransformation
OProgramming (Abap)
OOrganization (Process Ch.)
BW LSA
Landmark Building Blocks
Layer
Domains
Data Model &
Data Integration
Assistant Building Blocks
Data Quality
Landscape
ETL
Storage
Ownership/ Organize
Development concept
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /20090210/ Page 61
LSA Bui l di ng Bl oc k s –
The Ar c hi t ec t ur e Cor ner st ones
The LSA Building Blocks
Þ are the cornerstones of your future
architecture thus having a decisive influence
on the overall success of your future BW EDW
Þ describe the general BW EDW layout
independent from concrete BI applications and
BI projects.
Landmark Building Blocks deal with architecture
areas that need fundamental decisions before
doing implementations – they play the same role
like the supporting structures (pillars & ceilings) of
buildings.
Assistant Building Blocks deal with architecture
areas that are normally less prior with respect to
the point in time you should decide and roll out
corporate standards.
LSA
Building Blocks
Reference
Describes
core structures &
definitions
Landmark Building Blocks
Layer
Domains
Data Model &
Data Integration
Assistant Building Blocks
Data Quality
Landscape
ETL
Storage
Ownership/ Organize
Development concept
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 62
LSA Landmar k Bui l di ng Bl oc k s
Layer & Domai ns
There are two areas to standardize the architecture of persistent data stores:
1. Structure the data stores in a flow from PSA to InfoCubes with respect to
their role and the offered services – define Data Layer
2. Structure (split/ collect) the data within the Layer to guarantee Layer and
advanced services – define Data Domains
LSA Ar chi t ec t ur ed Non-Ar chi t ec t ur ed
Domain
Layer
d
a
t
a

f
l
o
w
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 63
LSA Assi st ant Bui l di ng Bl oc k s I
ETL (Extraction, Transformation, Load)
Do you expect extensively data from non-SAP sources?
ÞIf the answer is ‘yes’, it is meaningful investigating an ETL-tool like SAP Data Integrator
ÞIf SAP systems are the initial and primary BW EDW sources, you just don’t care. May be later
Data Quality
Do you have considerable data quality issues?
ÞIf the answer is ‘yes’, it makes sense thinking about a Data Quality tool.
ÞIf integrated SAP systems are the initial and primary BW EDW sources, you don’t care.
Landscape
often reduced to ‘Do I need a single or a multiple BW instance’ landscape. This topic has
become more and more an assistant one, because of the arrival of new technologies and the
transparent support by BW (BW Accelerator, Near Line Storage s. ‘Storage’).
The landscape architecture comes into focus
Þ if we have to support mission critical BI or to observe legal restrictions or with other customer
specific requirements
Þ if it comes to agile BI and local autonomy (federated landscapes, SAP Newton appliances
and BW EDW)
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 64
LSA Assi st ant Bui l di ng Bl oc k s I I
Storage
Traditionally all data of a BW DWH are hosted by an RDBMS.
This is for large scale BW EDWs not adequate (this applies also to smaller BWs)
Þ Use RDBMS compression if available and usable (e.g. DB2 ‘deep compression’)
ÞRarely used data should be hosted by Near Line Storage (NLS). NLS tools compress your
data up to 95% (e.g. SAND) without destroying your Service Level Agreements (SLAs).
ÞThe BW Accelerator (BWA) must be part of the overall architecture
Ownership/ Organization
Designing, implementing and operating a BW EDW need strong governance.
A BI Competency Center (BICC) should be established if not existing yet.
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 65
The Layer ed Sc al abl e Ar c hi t ec t ur e Goal s -
St andar di zat i on, Tr anspar enc y, Ef f i c i enc y
architectured architectured
non non- -architectured architectured
Non-
Arc hi t ec t ur ed
D

a

t

a

f

l

o

w
D

a

t

a

f

l

o

w
LSA
Arc hi t ec t ur ed
Large scale BW
Data Warehouses
(EDWs) should
follow architecture
principles like we
can observe
constructing large
buildings:
standardized,
scalable, no
squiggles, efficient
usage of materials,
earth quake save.
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /20090210/ Page 66
DEPLOYI NG WEBI ON SAP
With large scale BWs there is only one way to be successful:
standardize, standardize, standardize ...
in a way making your BW transparent, robust & in the broadest sense
scalable
The BW Layered Scalable Architecture is all about:
What, Where, When & How to Standardize
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 67
The Layered, Scalable Architecture (LSA) for EDW
Reference LSA Overview
3.I 3.I
33
Agenda PDEBW1
LSA Building Blocks – Data Layer
3.III 3.III
LSA Building Blocks – Data Domains
3.V 3.V
LSA Building Blocks – Data Model
3.VI 3.VI
LSA Building Blocks – Layer & Master Data
3.IV 3.IV
Reference LSA & Customer LSA
3.II 3.II
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 68
Fr om LSA Ref er enc e Arc hi t ec t ur e t o
Cust omer LSA
LSA
Building Blocks
Reference
LSA
Implementation
Reference
LSA
Operations
Reference
SAP‘s LSA
Reference

Customer-LSA
Building
Blocks
Customer-LSA
Implementation
Standards
Customer-LSA
Operations
Standards
Customer-LSA
Standards
¡
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /20090210/ Page 69
BI -Pr oj ec t s bui l t on Cust omer LSA St andards
LSA
Building Blocks
Reference
LSA
Implementation
Reference
LSA
Operations
Reference
SAP‘s LSA
Reference

Customer-LSA
Building
Blocks
Customer-LSA
Implementation
Standards
Customer-LSA
Operations
Standards
Customer-LSA
Standards
¡
BI-Project-Design
Based on
Customer LSA
Plan-Build-Run
Apply: Individuell BI project design
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /20090210/ Page 70
Cust omer LSA ‘Handbook ’
Shel l
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 71
Cust omer LSA ‘Handbook ’
Shel l
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 72
Cust omer LSA ‘Handbook ’
Land Hessen
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 73
BI-Project-Design
Li f e Cyc l e of The Cust omer LSA
BW LSA: The Reference Architecture
Customer LSA : Standards - Handbook
BI-Project-Design
BI Project Design
Step 3:
Perfect Perfect
Customer LSA
Step 4:
Update Update
Customer LSA
Step 1:
Design Design
Customer LSA
Step 2:
Apply Apply
Customer LSA
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /20090210/ Page 74
Di f f er ent St art i ng Poi nt s –
Di f f er ent Cust omer LSAs
BWon Corporate/ Divisional Level - BW EDW
• Consolidation of multiple BWs into a single often global BW – (consolidation)
• Reengineering of a BW getting on in years – (reengineering)
• Replacement of multiple legacy DWHs by a single BW (replacement)
• Allied set up of a BW with an (global )ERP roll-out (alliance)
BW1
BW2
BW3
BWn
C
o
n
s
o
l
i
d
a
t
i
o
n
P
r
o
l
i
f
e
r
a
t
e
d

B
W
R
e
e
n
g
i
n
e
e
r
i
n
g
DWH1
DWH2
DWH3
DWHn
R
e
p
l
a
c
e
m
e
n
t
N
e
w

C
o
r
p
o
r
a
t
e
E
R
P
(
s
)
A
l
l
i
a
n
c
e
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 75
Ref er enc e LSA & Cust omer LSA
As background and targets of each customer differ, the preferred services
differ and thus the Customer LSAs will differ
Customers differ with respect to:
OState of integration (master data, source systems (processes)
OPainful experience -> Control & Influence (outsourcing)
OSkills
OOverall governance, sponsorship, commitment.
O In general expections with respect to:
OAvailability, error-tolerance, robustness, audits :
OProtection against human errors, Data flows and persistency's (DSO-
Objects) should be resistible against all kind of errors
OReport availability at 8 am (local time), 24x7 operations
OFast (SLA) recovery without touching source system
OScalability (application scalability)
OFast new-build of BI-applications without accessing Sources
OScalability (performance)
OFlexibility, Protection of investment
Omanage seamless changes in OLTP world
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /20090210/ Page 76
The Layered, Scalable Architecture (LSA) for EDW
Reference LSA & Customer LSA
3.I 3.I
33
Agenda PDEBW1
LSA Building Blocks – Data Layer
3.III 3.III
LSA Building Blocks – Data Domains
3.V 3.V
LSA Building Blocks – Data Model
3.VI 3.VI
LSA Building Blocks – Layer & Master Data
3.IV 3.IV
Reference LSA Overview
3.II 3.II
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 77
‘Layering’ means horizontal structuring/ modeling of BW
The data flows are organized in a unified, service-oriented way. Parameters are:
Þ Coverage, comprehensiveness (Process, User demands)
Þ Granularity
Þ History (completeness)
Þ Reusability (BI application scalability)
Þ Recovery (robustness, availability)
Þ Quality
Þ Integration status
Þ Access-rights (End-user)
Þ Life-cycle
Þ .....
LSA Landmar k Bui l di ng Bl oc k s
Dat a Layer / Layer i ng of Dat a
Non-
Arc hi t ec t ur ed
D

a

t

a

f

l

o

w
D

a

t

a

f

l

o

w
Value of Data Layer:
+ Highly Transparent & Flexible
+ Development, Maintenance
+ Administration, Operations
+ Database-Integration
LSA
Arc hi t ec t ur ed
Layer
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 78
Basi c Desi gn Aspec t s of BW LSA
EDW & Ar c hi t ec t ed Dat a Mar t s Layer
There are two main Layer:
EDW Layer
Þ Single Point of Truth
Þ reusable
Þ no business driven transformations
just cleansing, unification and
integration
transformations
Þ based on common definitions
Þ naming
Þ Corporate Memory or
Þ Corporate/ Group Data Repository

Architected Data Mart Layer
Þ business driven transformations
Þ business defined
Þ BWA area
BW
E
D
W
A
r
c
h
i
t
e
c
t
e
d
D
a
t
a

M
a
r
t
s
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 79
LSA Ref er enc e Layer Qui c k I nt r o
L
S
A
Reporting Layer (Architected Data Marts)
Business Transformation Layer
O
p
e
r
a
t
i
o
n
a
l

D
a
t
a

S
t
o
r
e
Data Propagation Layer
Quality & Harmonisation Layer
Corporate
Memory
Data Acquisition Layer
Virtualization Layer
EDW layer
ÞSingle Point of truth
ÞReusable
ÞCorporate owned
ÞGranular
Architected
Data Mart Layer
User
Sources
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 80
LSA Ref er enc e Layer Qui c k I nt r o
L
S
A
Reporting Layer (Architected Data Marts)
Business Transformation Layer
O
p
e
r
a
t
i
o
n
a
l

D
a
t
a

S
t
o
r
e
Data Propagation Layer
Quality & Harmonisation Layer
Corporate
Memory
Data Acquisition Layer
Virtualization Layer
extractor inbox, 1:1 from
extraction, temporary
source system like service level,
comprehensive, complete, master the
unknown, long term
create harmonized
view, guarantee
quality
EDW Layer
ÞSingle Point of truth
ÞReusable
ÞCorporate owned
ÞGranular
BI Applications/
Architected
Data Mart Layer
digestible, ready to
consume, integrated,
unified data
apply business logic
reporting, analysis ready abstraction
near real time,
operational like
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 81
LSA Ref er enc e Layer Basi c s
Fr om Ac qui si t i on t o Repor t i ng Layer
Source
Systems
EDW Layer
Data flow
Corporate Memory
Business
Transformation
Layer
Reporting
Layer
Propagation
Layer
Acquisition
Layer
Data Mart Layer
Harmonize/
Quality
V-
Layer
DM1
DM3
DM2
Layering applies to transactional & master data!
To master data however in a simpler form
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 82
LSA Ref er enc e Layer
Ac qui si t i on Layer
Source
Systems
EDW Layer
Data flow
Corporate Memory
Business
Transformation
Layer
Reporting
Layer
Propagation
Layer
Acquisition
Layer
Data Mart Layer
Harmonize/
Quality
V-
Layer
DM1
DM3
DM2
Acquisition Layer
O Per DataSource per source system
O DataSource should be comprehensive (offer all
relevant process information)
O Fast inbound & outbound to targets
O Accept data from extraction with as less as
possible overhead – no early checks, merges,
transformations i.e. (1:1)
O Stamps the extracted records uniquely for
consistent internal DWH processing and tracking
(request handling)
O Provides abstraction of DWH from sources
O Provides short term history of extracted data for
immediate / short term data inspection
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 83
LSA Ref er enc e Layer
Dat a Ac qui si t i on Layer Fl i er
Criteria Characteristics
potential sources Operational systems, other data warehouses, any source
potential targets Data Propagation, Harmonisation & Quality, Corporate Memory, ODS
reusability Yes
transformations None, 1:1; possible enrichment of reusable informationen
granularity single records, granularity of DataSource/ most granular
content DataSource comprehensive: more fields than actually requested by Data-Marts
e.g. all fields of a Business Content DataSource
history 2-4 weeks, short term storage
main services · Decouple OLTP extraction cycle from internal BW data processing
· Robust & fast inbound and distribution
store & deploy · Unmerged data (DataSource & source system specific) – copy of delta queue
· Fast store: PSA Element, (Write Optimized DSO)
· Fast distribution, for large DWH: Scalability thru parallelism – single thread from
source system, scalable parallelism (logical/ semantical partitioning) to
consumers (targets)
update Request by request; no overwrite/ update of loaded records.
housekeeping Regular content deletion
I
m
p
l
e
m
e
n
t
.
O
p
.
D
W
H

S
e
r
v
i
c
e
s
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /20090210/ Page 84
LSA Ref er enc e Layer
Corpor at e Memor y Layer I
Source
Systems
EDW Layer
Data flow
Corporate Memory
Business
Transformation
Layer
Reporting
Layer
Propagation
Layer
Acquisition
Layer
Data Mart Layer
Harmonize/
Quality
V-
Layer
DM1
DM3
DM2
Corporate Memory (CM)
‘Think about the Corporate Memory Layer as your DWH and BI life
insurance‘:
Mastering tasks, which are unforeseeable (‘the unknown’) and
manageable only violating SLAs:
O Avoiding re-init from source(s)
O(Disaster-) recovery
OEnhancing and new build of Data Marts
OChange of source master data transformations
OReorganizations
=the CM layer offers a source system like Service Level:
OGranular data
OComprehensive data (high coverage of underlying process)
OComplete history of loaded data
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 85
LSA Ref er enc e Layer
Corpor at e Memor y Layer I I
Source
Systems
EDW Layer
Data flow
Corporate Memory
Business
Transformation
Layer
Reporting
Layer
Propagation
Layer
Acquisition
Layer
Data Mart Layer
Harmonize/
Quality
V-
Layer
DM1
DM3
DM2
Corporate Memory (CM)
‘Think about the Corporate Memory Layer as your DWH and BI life
insurance‘:
O Mastering tasks, which are unforeseeable (‘the unknown’) and
manageable only violating SLAs
O The CM layer offers a source system like Service Level
O 1:1 from Acquisition as rule of thumb
O in addition harmonized data may be stored in a dedicated Corporate
Memory Object after complex harmonization/ transformation
O If we have the same DataSource offered by multiple source-systems we
should go for a single CM DSO. (Data lifecycle management must exist!)
O NLS is the right place for CM data
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 86
LSA Ref er enc e Layer
Val ue of a Cor por at e Memor y
You can think about the Corporate Memory as the area, which protects your investments, which
protects you against all kind of surprises. Surprises in an architecture are unwelcome but they
will happen.
Þ you should never rely on what business ask you to deliver
Þ the corporate memory stores more information (fields) about a process than you are actually asked for.
In fact we recommend to store all available describing fields of a process. This is easy for SAP sources
as we extract data using all fields of a delivered DataSource.
– Thus the Corporate Memory is comprehensive.
Þ we do not aggregate information/ bookings. We store all extracted records.
– Thus the Corporate Memory is historical complete (e.g. for bookings we have the complete history
of a document)
Þ you may believe that you can repeat the loads or do again an initial load if you experience
problems or new requirements. This is often not realistic or just not possible.
Þ Not realistic:
– To repeat historic loads with delta loads
– initial loads cause a downtime in the operative system what is often not acceptable
Þ Not possible
– In the operative system the data will be archived
Þ you should always be aware that when you transform/ map original data (= Quality &
Harmonization Layer) the mapping rules may change.
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 87
LSA Ref er enc e Layer
Val ue of a Cor por at e Memor y - Summar y
The CM is a layer of choice
Þ to guarantee utmost possible flexibility to react on the unknown, the unforeseeable
Þ if downtime of operative systems for recovery purposes (new initialization) is
unacceptable (24x7) or if archiving takes place in operative system. In most
recovery situations the Corporate Memory will help.
Þ in general for robust and standardized recovery proceeding
Þ to keep more information than actually needed or which can actually be integrated
without impacting daily operations
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 88
© SAP 2008 / Page 89
LSA Ref er enc e Layer
Corpor at e Memor y Fl i er
Criteria Characteristics
potential sources Data Acquisition Layer, (Harmonization Layer)
potential targets Data Propagation Layer (Business Transformation Layer, Reporting Layer)
reusability Yes
transformations Often 1:1, technical harmonization (compounding)
granularity All extracted records for delta loads (copy of delta queue)
content · comprehensive (all fields of a DataSource)
· additional fields to simplify administration & access
history complete
main services · source system like SLA
·‘Single Point of Truth’ of extracted data for delta/ changed data for full
· ultimate area for application recovery, new-build and DWH reorganization
· manage the unknown
store & deploy · DataSource specific
· Write-optimized (wo) DSO for delta loads, normal DSO + wo DSO for full
· not directly in flow to applications (branched out)
update Request by request; no overwrite/ update of loaded records
Data life cycle Should mainly reside on NLS (Near Line Storage)
D
W
H

S
e
r
v
i
c
e
s
I
m
p
l
e
m
e
n
t
O
p
.
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /20090210/ Page 89
LSA Ref er enc e Layer
Har moni zat i on & Qual i t y Layer
Source
Systems
EDW Layer
Data flow
Corporate Memory
Business
Transformation
Layer
Reporting
Layer
Propagation
Layer
Acquisition
Layer
Data Mart Layer
Harmonize/
Quality
V-
Layer
DM1
DM3
DM2
Harmonization & Quality Layer
The services of this layer are data and data model
integration and quality checked data e.g.
O Technical harmonization
O format, length
O simple content harmonization (e.g. date)
O integrating local master data to a single model (key
compounding, concatenation)
O Integration of local master data to group master data
(group data model)
O Checking Integrity of loaded data
O Unification of data: adding e.g. load time, origin..
O Advanced content cleansing
O e.g. detect duplicates, address cleansing etc. -> Data
Services
O ...
Harmonizing data and guaranteeing quality of data can
mean high efforts or no efforts at all. Customers who
invested in process integration and common coded/
integrated master data will harvest now.
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 90
LSA Ref er enc e Layer
Har moni sat i on & Qual i t y Layer Fl i er
Criteria Characteristics
potential sources Acquisition Layer, CM
potential targets Data Propagation Layer, (occasionally Corporate Memory)
reusability Yes
transformations Yes, to achieve harmonized, quality-assured data
granularity single records, granular
content Those fields/ InfoObjects, which can be harmonized and/ or quality assured
history Often no persistency – Lookup-Mappings: long term
main services · Provide consistent, harmonized data across multiple sources
· Provide quality assured data
store & deploy · often only virtual as InfoSource or just as Transformation Rule
· If explicit persitency needed ‚normal‘ DSO with overwrite
· Lookup tables as DSOs/ InfoObjects/ Z-tables (history!)
update Request by request
housekeeping
I
m
p
l
e
m
e
n
t
.
O
p
.
D
W
H

S
e
r
v
i
c
e
s
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /20090210/ Page 91
Det ai l i ng LSA Ref erenc e Layer s
Pr opagat or Layer I
Source
Systems
EDW Layers
Data flow
Corporate Memory
Business
Transformation
Layer
Reporting
Layer
Acquisition
Layer
Data Mart Layers
Harmonize/
Quality
V-
Layer
DM1
DM3
DM2
Propagator Layer
OThe Propagator Layer is the interface to all business related Layers. It stands for ‘extract once deploy
many’ i.e. scalability of Data Mart development
O Propagator Layer Objects offer data that are easy to digest, ready to consume for all business
purposes (Data Marts internal & external)
Propagation
Layer
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 92
Propagation
Layer
Det ai l i ng LSA Ref erenc e Layer s
Pr opagat or Layer I I
Source
Systems
EDW Layers
Data flow
Corporate Memory
Business
Transformation
Layer
Reporting
Layer
Acquisition
Layer
Data Mart Layers
Harmonize/
Quality
V-
Layer
DM1
DM3
DM2
Easy to digest means standardized, unified data :
OFrom Harmonization & Quality Layer transformations we get:
O Compliance of data to corporate quality and consistency standards
O Integration of local master data to group master data for group reporting
O Integration of local master data into a single BW data model (compounding)
Easy to digest means standardized, unified data :
OPropagator Layer adds:
O Option to enhance or merge Data to simplify Data Mart building - but no business specific
transformations (no flavor for all EDW Layers)
O Unified Data transfer behavior out of Propagator Objects (e.g. additive delta)
O Suitable portions of data (-> Domains) to fulfill SLAs related to local autonomy, report
availability, robustness, recovery, administration and operations
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 93
Det ai l i ng LSA Ref erenc e Layer s
Pr opagat i on Layer & Tr i mmi ng Dat a
DataSource A
Source 2
DataSource A
Source 1
‘DataSource A’
Propagator
3. Collect
DataSource B DataSource A
‘DataSource A & B’
Propagator
2.Merge
DataSource A
‘DataSource A +’
Propagator
1.Extend
Add
data
DataSource A
‘DataSource A’
Propagator 1
4.Split
‘DataSource A’
Propagator 2
Propagator DSOs: Unified BI application experience – additive delta

via
queue

via
generic

via
full
moving

via full
complete
full
incomplete
full
?
5.Unify
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 94
Det ai l i ng LSA Ref erenc e Layer
Pr opagat i on Layer & Di gest i bl e Dat a
The core service of a Propagation Layer is to offer digestible data to applications.
Digestible may mean
Þ harmonized data in the broadest sense (for more details s. Chapter on Data Model)
Þ integrating data: common semantics, common values
Þ smoothing data: common semantics, technically unified values (e.g. compounding)
Þ harmonize behavior of extractors (additive delta to Data Marts)
Þ trimmed to fit DataSources and Data persistency‘s to
Þ Reduce data complexity for applications
1. Extending DataSources by looking up information, which applications frequently ask
for. Note: introduced parent-child relationship must be stable otherwise realignment
issues!
2. Merging different but highly related DataSources and store data in a single propagator,
If application always or frequently request them together (e.g. HR InfoTypes, avoiding
extractor enhancements)
Þ Provide sound data portions for better support of application services (availability etc)
3. Collecting data from the same (or similar) DataSource but from different source
systems to less or a single source system independent propagator (s) (= LSA
domains)
4. Splitting data from a DataSources into multiple persistency’s with identical structure
(= LSA domains)
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 95
Data flow
2LIS_11_VASTI
2LIS_11_VASCL
YGTCS_SUMMARY
2LIS_12_VCSCL
Acquisition
Corp Memory
EDW
propagation
R/3
Corp Memory
Corp Memory
Corp Memory
EDW Layer
Cust omer Ex ampl e
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 96
LSA Ref er enc e Layer
Dat a Propagat i on Layer Fl i er
Criteria Characteristics
potential sources Data Acquisition Layer, Harmonization Layer, Corporate Memory
potential targets Business Transformation Layer, Reporting Layer
reusability Yes
transformations Additional, stable fields to increase (re-)useability & accessibility.
No application-specific rules!
granularity single records, granularity defined by DataSource business-key
content · DataSource specific
· as comprehensive as possible, if propagator is expecting volatile requierments
· Merge of different DataSources to reduce complexity changes
history Minimum history defined by requirements of target-applications/ dependent from
Corporate Memory existence
main services · ‘Single Point of Truth’ for BI applications (Business Transf. & Reporting Layer)
· Provide digestible (additive delta, content, performance) data for BI applications
· application recovery, rebuilt
store & deploy · ‚normal‘ DSO in overwrite
· semantical/ logical partitioned for large scale DWH/ time-zone support
update driven by BI application requirements (report availability)
housekeeping Regular delete of DSO change-log content
D
W
H

S
e
r
v
i
c
e
s
I
m
p
l
.
O
p
.
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /20090210/ Page 97
LSA Ref er enc e Layer s
Dat a Mar t Layer s
Source
Systems
EDW Layers
Data flow
Corporate Memory
Business
Transformation
Layer
Reporting
Layer
Propagation
Layer
Acquisition
Layer
Data Mart Layers
Harmonize/
Quality
V-
Layer
DM1
DM3
DM2
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 98
Data flow
2LIS_11_VASTI
2LIS_11_VASCL
YGTCS_SUMMARY
2LIS_12_VCSCL
Acquisition
Corp Memory
I
n
f
o
S
o
u
r
c
e
I
n
f
o
S
o
u
r
c
e
Reporting
EDW
propagation
R/3
Appl specific
Transformation
Corp Memory
Corp Memory
Corp Memory
Acquisition/
Pass-Thru
Corporate Mem.
Propagation
Transformation
Reporting
Multi-provider
Corporate Mem.
Busi ness Tr ansf or mat i on Layer
Cust omer Ex ampl e
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 99
LSA Ref er enc e Layer
Busi ness Tr ansf or mat i on Layer Fl i er
Criteria Characteristics
potential sources Data Propagation Layer
potential targets Reporting Layer
reusability No
transformations Applications specific rules, aggregation, disaggregation, lookups
granularity Defined by application needs
content Defined by application needs
history Defined by application needs
main services · area where complex merging of propagators objects will take place
· area where complex calculations/ enrichment (lookups) will take place
· guarantees reporting layer consistency (e.g. Synchronization of DataSources)
store & deploy · ‚normal‘ DSO in overwrite, if any: often only virtual as InfoSource/Transf. Rule
· semantical/ logical partitioned for large scale DWH/ time-zone support
update driven by BI application requirements (report availability)
housekeeping Regular delete of DSO change-log content
D
W
H

S
e
r
v
i
c
e
s
I
m
p
l
.
O
p
.
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /20090210/ Page 100
Det ai l i ng LSA Ref erenc e Layer s
Sub-Layer s of Report i ng/ Dat a Mar t Layer
Access Abstraction-/
Virtual Layer
Reporting/ Data Mart Layer
Analytics-/
Dimensional Layer
Flexible Reporting/
Granular Reporting
Layer
O highly granular
O highly comprehensive
O short life cycle
O flat, multidimensional
O less granular
O less comprehensive
O long life cycle
O multidimensional
O optimized performance
O abstract from physics
O ‘virtual’
O flexible ‘view’ generation
O protect front-end investments
Planning Layer
O dedicated for planning
O direct input structures
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 101
Det ai l i ng t he Report i ng Layer
Cust omer Ex ampl e
Flexible
Layer
Dimensional
Layer
Virtual
Layer
Reporting
Granular
Reporting
Performance
Long term
Abstraction
Flexible
Data flow
Reporting Layer
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 102
© SAP 2008 / Page 103
LSA Ref er enc e Layer
Report i ng Layer Fl i er
Criteria Characteristics
potential sources Data Propagation Layer, Business Transformation Layer
potential targets -
reusability No
transformations Applications specific rules, aggregation, disaggregation, lookups
granularity Defined by application needs
content Defined by application needs
history Defined by application needs
main services · reporting & analysis
· save investments in front-end area: separation of physical Objects (InfoCube,
DSO) and virtual Objects( Virtual InfoProvider, InfoSet) by using logical views
(MultiProvider)
· different services of Reporting Layer often modeled by sub-Layer
· Access Abstraction Layer (MultiProvider logical view)
· Analytic Layer (InfoCubes)
· Flexible Reporting (granular InfoCubes, DSOs)
store & deploy · InfoCubes, DSO-Objects, InfoSets, Virtual InfoProvider, MultiProvider
· Queries work always on MutliProvider
· semantical/ logical partitioned for large scale DWH/ time-zone support
update driven by BI application requirements (report availability)
housekeeping Archiving, NLS, Compression
I
m
p
l
.
O
p
.
D
W
H

S
e
r
v
i
c
e
s
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /20090210/ Page 103
LSA Ref er enc e Layer –
Oper at i onal Dat a St ore (ODS)
L
S
A
Reporting Layer (Architected Data Marts)
Business Transformation Layer
O
p
e
r
a
t
i
o
n
a
l

D
a
t
a

S
t
o
r
e
Data Propagation Layer
Quality & Harmonisation Layer
Corporate
Memory
Data Acquisition Layer
Virtualization Layer
EDW layer
ÞSingle Point of truth
ÞReusable
ÞCorporate owned
ÞGranular
Architected
Data Mart Layer
User
Sources
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 104
3rd Party
Landscape
S
A
P

B
W
PIPE
POS Inbound Processing Engine
Upload of POS Data to SAP BW
via POS Inbound Processing Engine
mySAP
Landscape
Sales Analyses of POS Data
Store
Contr.
Promo
Analysis
Loss
Prevent
B
I
-
R
e
p
o
r
t
i
n
g
T
e
m
p
l
a
t
e
s
Outbound Interfaces
Ret ai l Hi gh Vol ume SAP BW ODS – POS Dat a
Management
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 105
3rd Party
Landscape
SAP BW
mySAP
Landscape
Reporting-Templates
SAP BW
Masterdata
Exchange
Infrastructure
R/3 POS Inbound
Card Payment
Billing
Forecast &
Replenisment
Inventory
Management
SAP BW
Custom-Built
Interface
IDocs
Standards
Sales
Transactions
VMI
XML
Transaction
database
Sales Audit
• Monitor
• Analyses
• Editor
BAPI
PIPE
Access Module
Overvi ew POS I nbound Pr oc essi ng Engi ne
(PI PE)
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 106
Ex ampl e f or Dedi c at ed SAP BW Oper at i onal
Dat a St ores
SAP BW
PIPE-ODS
APA
SAP BW
PIPE-ODS
Americas
SAP BW
PIPE-ODS
UK
SAP BW
PIPE-ODS
EMEA
Central
SAP BW
Central
SAP for Retail
Master Data
POS
Transaction
Data
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 107
Hybr i d Pr ovi der
Real-Time Business Intelligence
HybridProvider
· Combines mass data with latest delta
information at query runtime
· Consists of a DataStore object and a
InfoCube with automatically generated
data flow in between
· DSO object can be connected to a realtime
data acquisition DataSource/DTP
· If the DataSource can provide appropriate
delta information in direct access mode a
VirtualProvider can be used instead of the
DSO.
· Facilitates replication of DSO-
/VirtualProvider data to SAP NetWeaver
BW Accelerator by switching off database
persistency of the InfoCube
(RDA) DTP
Replicate
DSO
latest data mass data
InfoCube
HybridProvider
OLTP
Implement operational reporting scenarios leveraging new
modeling objects
BWA
optionally
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 108
LSA Ref er enc e Layer
Oper at i onal Dat a St ore Layer Fl i er
Criteria Characteristics
potential sources Data Acquisition Layer
potential targets application specific (Reporting Layer), external (e.g. Pipe)
reusability Limited
transformations simple
granularity extracted records granularity
content application specific
history application specific
main services · near real time reporting
· mass data management (e.g. POS)
store & deploy · normal DSO (Hybrid Provider)
· mass data
· Write-optimized (wo) DSO
· No PSA
· Pipe (point of sale inbound processing engine)
update RDA, request by request
D
W
H

S
e
r
v
i
c
e
s
I
m
p
l
e
m
e
n
t
O
p
.
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /20090210/ Page 109
LSA Ref er enc e Layer
Ac qui si t i on Layer and Subsequent Layer
Why are there different paths thru LSA? The general answer is simple:
In a BW we have to fulfill different competing SLAs (Service Level Agreements)
Experience shows that a single staging approach with single persistent stores for all
purposes cannot achieve this (RDBMS limits) ! This applies the more challenging the
conditions are for BW.
Note: This means on the other hand side that if we found less complex conditions the LSA
Building Blocks set up can be simpler. This applies as well for an overall customer LSA set up
as for specific data sources within a full blown customer LSA !
What are complex conditions and
challenging requirements?
Þ 24x7, time zones
Þ high volume, not split able loads
Þ small recovery window (e.g. 6h)
Þ out sourcing (skills?)
Þ off shoring (skills?)
Þ high operational robustness
Þ high report availability (e.g. >96%)
Þ high application flexibility
Þ ....
L
S
A
Reporting Layer (Architected Data Marts) –
Shared Master Data (InfoObjects)
Business Transformation Layer
O
p
e
r
a
t
i
o
n
a
l

D
a
t
a

S
t
o
r
e
Data Propagation Layer
Harmonization Layer
CM
Data Acquisition Layer
C
C
C
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 110
LSA Layer s @Cust omer
Summar y
OThe LSA describes services that are relevant at large scale BW
implementations. The services are grouped by Layers.
O The LSA suggested Layers are a basic offering. Based on customer
situation and preferences additional ones may be introduced even
later on (Customer LSA).
OAs with all LSA areas customer adaptation of Layers follows the
80:20 rule i.e. concentrate on services first, which offer the most
benefit for the majority of data
O Not all Layers have necessarily own persistent InfoProviders for all
data flows. Whether and when persistent InfoProviders exist are given
in the LSA Implementation standards
O The Customer LSA Layer definitions apply throughout the BW EDW
– No exceptions without approval!

© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 111
BI-Projekt-Design
The Layer ed Sc al abl e Ar c hi t ec t ure
Cust omer Adapt at i on & Ut i l i zat i on
SAP LSA: The Reference Architecture
Customer LSA : Standards - Handbook
BI-Projekt-Design
Customer BI Projects (Plan-Build-Run)
Step 3:
Perfect Perfect
Customer LSA
Step 4:
Update Update
Customer LSA
Step 1:
Design Design
Customer LSA
Step 2:
Apply Apply
Customer LSA
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /20090210/ Page 112
LSA Bui l di ng Bl oc k s
Ref er enc e LSA & Cust omer LSA – Ex ampl e I
L
S
A
Reporting Layer (Architected
Data Marts)
Business Transformation Layer
O
p
e
r
a
t
i
o
n
a
l

D
a
t
a

S
t
o
r
e
Data Propagation
Layer
Quality &
Harmonisation Layer
Corpo
rate
Memo
ry
Data Acquisition Layer
From Reference LSA to Customer LSA:
Þ Not all Layer
Þ Additional Layer
Þ Different visualization and branding of Layer
Reference LSA
Customer LSA
S
A
P
S
o
u
r
c
e
s
O
t
h
e
r
S
o
u
r
c
e
s
Operational Data Store
D
a
t
a

A
c
q
u
i
s
i
t
i
o
n
D
a
t
a

P
r
o
p
a
g
a
t
i
o
n
Q
u
a
l
i
t
y
T
r
a
n
s
f
o
r
m
a
t
i
o
n
s
B
u
s
i
n
e
s
s

T
r
a
n
s
f
o
r
m
a
t
i
o
n
F
l
e
x
i
b
l
e
R
e
p
o
r
t
i
n
g
Subsidiary
Reporting &
Analytics
Business-Unit
Spanning
Reporting &
Analytics
· Scorecards
· Group
Near real time
Reporting
A
c
c
e
s
s

A
b
s
t
r
a
c
t
i
o
n

L
a
y
e
r
P
r
o
j
e
c
t
-
d
e
f
i
n
e
d

R
e
p
o
r
t
i
n
g

&

A
n
a
l
y
t
i
c
s
E
n
d
-
u
s
e
r

R
e
p
o
r
t
i
n
g

&

A
n
a
l
y
t
i
c
s
Corporate
Memory
Enterprise
Data Warehouse
BI Application Area
Data flow
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 113
LSA Bui l di ng Bl oc k s
Ref er enc e LSA & Cust omer LSA – Ex ampl e I
L
S
A
Reporting Layer (Architected
Data Marts)
Business Transformation Layer
O
p
e
r
a
t
i
o
n
a
l

D
a
t
a

S
t
o
r
e
Data Propagation
Layer
Quality &
Harmonisation Layer
Corpo
rate
Memo
ry
Data Acquisition Layer
From Reference LSA to Customer LSA:
Þ Detailing
Reference LSA
Customer LSA
Acquisition
Layer
Corporate
Memory
Layer
Propagation
Layer
Business
Transformation
Layer
Flexible
Layer
Dimensional
Layer
Virtual
Layer
(YADSSnnn)
YCDSSnnn
YPDSSnnn
YBAPPnnn
YFAPPnnn
YVAPP1nn
YVAPP2nn

YDAPPnnn
D a t a f l o w
lookup
1:1
Unlink
Unflavored
Integrated
Granular
Ready to consume
Apply
business-logic
Reportin
g
Granular
Reporting
Performance
Long term
Abstraction
Flexible
Complete
Comprehensive
Most granular
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 114
LSA Bui l di ng Bl oc k s
Ref er enc e LSA & Cust omer LSA – Ex ampl e I I
L
S
A
Reporting Layer (Architected Data
Marts)
Business Transformation Layer
O
p
e
r
a
t
i
o
n
a
l

D
a
t
a

S
t
o
r
e
Data Propagation Layer
Quality & Harmonisation
Layer
Corpor
ate
Memor
y
Data Acquisition Layer
From Reference LSA to Customer LSA:
Þ Detailing
Þ Not all Layer
Þ Additional Layer
Þ Different visualization and branding of Layer
Þ Individual domains
Reference LSA
Customer LSA
Data flow
2LIS_11_VASTI
2LIS_11_VASCL
YGTCS_SUMMARY
2LIS_12_VCSCL
Acquisition
Corp Memory
Reporting
EDW
propagation
R/3
Appl specific
Transformation
Corp Memory
Corp Memory
Corp Memory
Corporate Mem.
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 115
The Layered, Scalable Architecture (LSA) for EDW
Reference LSA & Customer LSA
3.I 3.I
33
Agenda PDEBW1
LSA Building Blocks – Data Layer
3.III 3.III
LSA Building Blocks – Data Domains
3.V 3.V
LSA Building Blocks – Data Model
3.VI 3.VI
LSA Building Blocks – Layer & Master Data
3.IV 3.IV
Reference LSA Overview
3.II 3.II
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 116
Si t uat i on of Mast er Dat a
With master data we have similar challenges as with transaction data and additional ones
Þ LSA topics
1. High volume master data tables have a load impact
1. With full loads for InfoObjects (even if delta-load possible), which are used with extractor
enhancements to catch the changes of an enhancement (=merging DataSources in
Propagator Layer)
2. With change run (aggregate maintenance)
2. Scenario reporting readiness depends on master data load time
1. Transaction data can only be processed into target subsequent to master data loads
2. Different scenario or market reporting readiness requires different points in time of master
data readiness
3. Recovery of InfoCubes or new ones often need master data history
Þ BIA topics
1. Change Run
2. High volume master data tables (shared dimensions) have a negative impact on query
performance as with large fact tables
3. Limitations of external hierarchies on high volume master data
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 117
LSA & Mast er Dat a
Si t uat i on
Master data have two main tasks to fulfill:
– Being target of lookups during transactional data load (referential integrity, adding
information
– Being the shared dimensions of InfoCubes for reporting (MultiProvider
Today InfoObject hosted master data (P,Q,S,X,Y tables) serve for both purposes.
InfoObject
Master data
Shared Dimension
(Navigational Attributes)
Integrity,
Lookups
Transactional loads
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 118
I ssues wi t h Dat a Mar t Level Managed Mast er Dat a
P
r
o
c
e
s
s

C
h
a
i
n
D
a
t
a

M
a
r
t
A
Transactional
loads
With non-architected BWs the master data loads
are often managed on BI application/ Data Mart
level what leads to redundant, uncontrolled
master data processing.
This is unacceptable for large scale BWs
Data Mart level managed master data:
Master data
Load e.g.
0CUST_SALES
+ Change Run
3:00
Process Chain
Master Data
Master data
Load e.g.
0CUST_SALES
+ Change Run
Transactional
loads
First Step: BW level managed master data:
Master data should be collected/ prioritized and
loaded across the Data Marts thus become BW
managed
Transactional
loads
Master data
Load e.g.
0CUST_SALES
+ Change Run
P
r
o
c
e
s
s

C
h
a
i
n
D
a
t
a

M
a
r
t
B
1:00 1:15
Transactional
loads
Process Chain
Data Mart B
Process Chain
Data Mart A
2:30
1:00 1:15
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 119
LSA I mpl ement at i on
Layer i ng Mast er Dat a
L
S
A
Reporting Layer
(Architected Data Marts)
Propagation/
CM Layer
Quality & Harmonisation Layer
Data Acquisition Layer
InfoObject tables
Master Data
DSO
Shared Dimension
x,y,p,q,s tables
(Navigational Attributes)
Integrity,
Lookups
Picture shows master data with delta load, master data with
full loads need additional ‘assistant DSO’ to determine delta
Large scale BWs should layer
the master data:
1.Separation of staging and
reporting tasks
O InfoObject tables for
reporting
O Propagator Master Data
DSOs for staging
2.Storing master data history
(introducing ‘active from’
‘active-to’ time-slice in
Propagator Master Data
DSO)
`
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 120
LSA I mpl ement at i on
Dec oupl i ng EDW & Dat a Mar t Layer Loads
Process Chains
Master Data
InfoObject
Change Run
Propagator/
Corporate Mem
Process Chain
Data Mart A
LSA managed master data loads
O EDW transactional & master data loads are decoupled from Data Mart &
InfoObject loads
O It still remains challenging at what point in time to schedule master data
loads in a global system
Process Chains
EDW Master Data
Propagator for
0CUST_SALES
Process Chains
EDW Transactional
Data
Data Mart Layer
B Trans Layer
InfoObject
Update
EDW Layers
Data Mart Layers
Process Chain
Data Mart B
Data Mart Layer
B Trans Layer
multiple loads
per day
loads by
business
requirements
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 121
LSA I mpl ement at i on
Real Ti me Dat a Ac qui si t i on (RDA) f or Mast er Dat a
Process Chains
Master Data
InfoObject
Change Run
Propagator/
Corporate Mem
Process Chain
Data Mart A
LSA managed master data loads (RDA)
O With NW BW 7.30 there will be Real Time Data Acquisition (RDA) for
Master Data .
OThis will increase the robustness of global BW EDWs considerably
RDA
EDW Master Data
Propagator for
0CUST_SALES
Process Chains
EDW Transactional Data
Data Mart Layer
B Trans Layer
InfoObject
Update
EDW Layers
Data Mart Layers
Process Chain
Data Mart B
Data Mart Layer
B Trans Layer
multiple loads
per day
loads by
business
requirements
continuous loads
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 122
LSA and Mast er Dat a
Keep mast er dat a hi st or y
Customer CustomerGroup
A Y after 2nd Load
Customer CustomerGroup
A X 1st Load: 20020101
A Y 2nd Load: 20020401
InfoObject Master Data Table
Master Data from Source or Master Data Server
InfoObject master data keep only the actual version of master data:
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 123
LSA and Compl et e Hi st or y of Mast er Dat a
Corporate Memory DS-Objects store all versions of master data
customer mother-comp customer mother-comp
AAA Y
BBB Y
CCC X
InfoObject Customer Master
customer ActiveFrom mother-comp customer ActiveFrom mother-comp
AAA 19980101 X
AAA 19981101 Y
BBB 19980101 Y
CCC 19980101 X
I.
Simple Customer Master CM
DS-Object
customer mother-comp
AAA X
BBB Y
CCC X
customer mother-comp
AAA Y
load from 19980101
load from 19981101
customer ActiveTo ActiveFrom mother-comp customer ActiveTo ActiveFrom mother-comp
AAA 19981031 19980101 X
AAA 99991231 19981101 Y
BBB 99991231 19980101 Y
CCC 99991231 19980101 X
II.
Time slice Customer Master CM
DS-Object
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 124
Al t er nat i ves Managi ng Mast er Dat a i n LSA
1. Slim Staging: direct load into InfoObject (like in normal BW) for all Master Data not
addressed in 2 or 3.
2. Complete history of master data changes in a Propagator/ Corporate Memory* via
time-stamping records – (restricted usability)
Þ Flexible staging
Þ Delta loads direct into CM DSO
Þ Full loads via intermediate normal DSO (special type of Pass Thru) for delta determination
Þ Same handling for transactional full loads
3. Complete history of master data changes in a Propagator/ Corporate Memory* via
time-sliced history (easy accessable in loockup)
Þ Flexible staging
Þ Propagator/ CM normal DSO with active-from, active-to (via function module set) slice, active-
to is part of key
Þ Delta loads direct into Propagator/ CM DSO
Þ Full loads via intermediate normal DSO (special type of Pass Thru) for delta determination
* whether you make it a Propagator or a CM member is a matter of taste
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 125
SAP BW EDW Func t i ons and Model i ng
Det er mi ne Ti me sl i c e of Mast er Dat a I
InfoObject
Characteristic Active-to Active-from Origin Attribute1 Attr2
4711 99991231 20030501 S1 Y A
4711 20030430 20030401 S1 X A
4712 99991231 20030401 S1 X B
Characteristic Origin Attribute1 Attr2
4711 S1 X->Y A
4712 S1 X B
Characteristic Attribute1 Attr2
4711 Y A
4712 X B
Transformation Rule:
• Read DSO-Object with 4711 and 99991231
• Update 99991231 record with new values and Active-fr
• Use Return table to create new record
Characteristic Attribute1 Attr2
4711 X->Y A
4712 X B
Corporate Memory
‘normal DSO’
‘Pass Thru’
‘normal DSO’
Load (here full)
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 126
LSA and Mast er Dat a
Completeness of Master Data History
At least for the ‘big entities’ all historical master data versions
should be stored in the SAP BW Data Warehouse Layer using
flexible master data staging.
For less important entities it is sufficient to keep the actual
versions of master data in the InfoObject master data table thus
using direct staging.
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 127
The Layered, Scalable Architecture (LSA) for EDW
Reference LSA & Customer LSA
3.I 3.I
33
Agenda PDEBW1
LSA Building Blocks – Data Layer
3.III 3.III
LSA Building Blocks – Data Domains
3.V 3.V
LSA Building Blocks – Data Model
3.VI 3.VI
LSA Building Blocks – Layer & Master Data
3.IV 3.IV
Reference LSA Overview
3.II 3.II
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 128
I nc r ement al EDW
Reasons & Chal l enges
Incremental set up is targeting to:
Þ Align actual business requirements & BI/ EDW excellence
Þ Reduce complexity
Þ Step by step process & organizational excellence
Þ ...
Nevertheless an incremental approach is challenging as we have to guarantee
consistency & scalability during the stony path of roll-out:
Þ Consistency
– Between the various Data Marts (FI, CO, LO, CRM...)
– consistent development of Data Marts
– Sometimes between different SAP BW systems
– consistent deployment of Data Marts
Þ Scalability
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 129
EDW Li f e Cyc l e Di mensi ons
I nc r ement al I mpl ement at i on & Sc al abi l i t y
2. Incremental in terms of organizational coverage – organizational roll-out
D
e
m
a
n
d

P
l
a
n
n
i
n
g
G
e
n
e
r
a
t
i
n
g

D
e
m
a
n
d
P
r
o
c
u
r
e

2

P
a
y
C
O
P
A
..... .....
EDW
C C C n
BI Application Coverage & EDW
O
r
g

U
n
i
t

A
O
r
g

U
n
i
t

B
O
r
g

U
n
i
t

C
O
r
g

U
n
i
t

N
..... .....
EDW
C C C n
Organizational Coverage & EDW
Needs Scalability
that is addressed by
(EDW)- layers
Needs Scalability
that is addressed by
domains (& data
model)
1. Incremental in terms of functional coverage – BI application/ Data Mart roll-out
An EDW implementation is always an incremental one, never a big-bang
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 130
LSA Ref erenc e Layer s Hi ghl y Si mpl i f i ed -
Layer i ng Does Not Resol ve Al l Chal l enges
Source
Systems
EDW Layers
Data flow
Corporate Memory
Business
Transformation
Layer
Reporting
Layer
Propagation
Layer
Acquisition
Layer
Data Mart Layers
Harmonize/
Quality
V-
Layer
How to support:
O a world wide continuous roll-out
O BW-wide load scalability
O 24x7 operations & administration
O different reliability of sources
O local autonomy
O overall local & group SLAs
O...
?
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 131
LSA Landmar k Bui l di ng Bl oc k s
Dat a Domai ns
Non-
Arc hi t ec t ur ed
D

a

t

a

f

l

o

w
D

a

t

a

f

l

o

w
LSA
Arc hi t ec t ur ed
Layer
Domain
Domains means structuring/ modeling of data within the layers:
Transparent, disjunct structuring of transactional data using stable criterion.
Target is the support of:
Þ Independency/ autonomy of organizations
Þ 24x7, time-zones
Þ Scalability / performance/ low latency
(parallel vers sequential)
Þ Challenging recovery-window
Þ Embedding into RDBMS
Þ Implementation & Operational robustness
Value of Data Domains:
+ Transparency & Flexibility
+ Development, Maintenance
+ Administration, Operations
+ Scalability & Robustness
+ Application
+ Load/ Query Performance
+ Database-Integration
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 132
LSA Domai n Conc ept - St r at egi c BW Par t i t i oni ng
Ac c r oss t he Layer s – For Al l Fl ows I
Source
Systems
EDW Layers
Data flow
Corporate Memory
Business
Transformation
Layer
Reporting
Layer
Propagation
Layer
Acquisition
Layer
Data Mart Layers
Harmonize/
Quality
V-
Layer
DM1
Domains apply to transactional data
Normally not to master data!
2
L
I
S
_
1
1
_
V
A
I
T
M
Belongs to Domain A
Belongs to Domain B
Belongs to Domain C
S
i
n
g
l
e


E
R
P
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 133
LSA Domai n Conc ept - St r at egi c BW Par t i t i oni ng
Ac c r oss t he Layer s – For Al l Fl ow s I I
Source
Systems
EDW Layers
Data flow
Corporate Memory
Business
Transformation
Layer
Reporting
Layer
Propagation
Layer
Acquisition
Layer
Data Mart Layers
Harmonize/
Quality
V-
Layer
DM1
Domains apply to transactional data
Normally not to master data!
Belongs to Domain A
Belongs to Domain B
Belongs to Domain C
2
L
I
S
_
1
1
_
V
A
I
T
M
M
u
l
t
i
p
l
e

E
R
P
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 134
Domain US
LSA Bui l di ng Bl oc k s &
Ex t rac t Onc e - Depl oy Many
EDW
Propagators
Data Marts
DataSources
O
r
d
e
r
D
e
l
i
v
e
r
y
S
u
p
p
l
y

C
h
a
i
n
O
r
d
e
r

I
n
f
o
1
:
n
D
e
l
i
v
e
r
y

I
n
f
o
Domain AMS
O
r
d
e
r
D
e
l
i
v
e
r
y
S
u
p
p
l
y

C
h
a
i
n
O
r
d
e
r

I
n
f
o
D
e
l
i
v
e
r
y

I
n
f
o
Domain APJ
O
r
d
e
r
D
e
l
i
v
e
r
y
S
u
p
p
l
y

C
h
a
i
n
O
r
d
e
r

I
n
f
o
D
e
l
i
v
e
r
y

I
n
f
o
Filling Domains with respect to domain criteria
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 135
LSA Dat a Domai ns
BW Landsc ape Consol i dat i on (Consol i dat i on BW)
Europe
Japan
Asia Pacific
ERP ERP
ERP ERP
ERP ERP
ERP ERP
ERP ERP
BW
BW
BW
BW
BW ERP ERP
North America
South America
A BW EDW replaces a bunch of existing BWs and/ or legacy DWHs
(BI Consolidation) spread across the organization
To enable comparable services like we had in a distributed, multiple DWH instance world
(yes, there are some nice things) we introduce Data Domains in a BW EDW that divide
the transactional data but use identical meta data & master data.
BW
OUsing Domains in a BW EDW stands for manageability & flexibility
ODomains allow SLAs in a BW EDW like in a distributed BW world
X
X
Domain Americas Domain Americas
X X
Domain Europe Domain Europe
X
X
Domain Asia Pacific Domain Asia Pacific
BW EDW BW EDW
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 136
LSA Dat a Domai ns
BW i n Par al l el t o ERP Rol l out (Pr i mary BW)
Europe
Japan
Asia Pacific
North America
South America
A single BW EDW shall offer standard reporting & analytics for all organizational units
in a global ERP rollout.
To enable comparable services like we had in a distributed, multiple BW instance
world we introduce Data Domains in a BW EDW that
divide the transactional data but use identical meta data & master data.
OUsing Domains in a BW EDW stands for manageability & flexibility
ODomains allow SLAs in a BW EDW like in a distributed BW world
AMS AMS
Germany Germany APA APA US US EMEA EMEA
Global ERP Global ERP
BW EDW BW EDW
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 137
LSA Dat a Domai ns
Di vi de Dat a by Sour c es-‘Qual i t y’ (I nt egr at i on BW)
BW EDW BW EDW
main
ERP
remote
ERP 1
remote
ERP 2
Main Domain Main Domain
Remote Domain Remote Domain
less stable, no control stable, controlled
OUsing Domains in a BW EDW stands for manageability & flexibility
ODomains allow SLAs in a BW EDW like in a distributed BW world
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 138
1
st
Summar y: Domai ns Tar get s t o Keep Lar ge BW
EDWs Manageabl e & Sc al abl e
D
o
m
a
i
n

W
e
s
t
D
o
m
a
i
n

W
e
s
t
D
o
m
a
i
n

C
e
n
t
r
a
l
D
o
m
a
i
n

C
e
n
t
r
a
l
D
o
m
a
i
n

E
a
s
t
D
o
m
a
i
n

E
a
s
t
BW EDW BW EDW
Company
is operating
In North America
ÞUsing Domains in a BW EDW stands for manageability & flexibility
ÞDomains allow SLAs in a BW EDW like in a distributed BW world
North America
South America
Domain South Domain South
AMS AMS
Company is
expanding to
South America
D
o
m
a
i
n
D
o
m
a
i
n
W
e
s
t
W
e
s
t
D
o
m
a
i
n
D
o
m
a
i
n
C
e
n
t
r
a
l
C
e
n
t
r
a
l
D
o
m
a
i
n
D
o
m
a
i
n
E
a
s
t
E
a
s
t
BW EDW BW EDW
Europe
North America
South America
Domain South Domain South
AMS AMS
D
o
m
a
i
n
D
o
m
a
i
n
W
e
s
t
W
e
s
t
D
o
m
a
i
n
D
o
m
a
i
n
C
e
n
t
r
a
l
C
e
n
t
r
a
l
D
o
m
a
i
n
D
o
m
a
i
n
E
a
s
t
E
a
s
t
Domain Domain
EU EU
Company is
expanding to
Europe
BW EDW BW EDW
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 139
LSA Dat a Domai ns Bac k ground
Evol ut i onar y BW EDW
An BW EDW that has to provide ‘local’ BI services faces
Þ by far more data
Þ by far more and more complete BI applications – finally the entire value-chain
Þ more source systems
Þ higher query workload
than any other DWH in the organization before!
But with service expectations necessary to support daily business, like:
Þ High performance, robustness, high availability, low latency, organizational autonomy, fast
recovery ......
In addition BW EDWs on a global scale have to observe time-zones and
24x7 operations
These challenges cannot be answered by just layering data: The LSA answer
is the Data Domain concept:
LSA Data Domains divide (transactional) data in the BW Layer in a disjunctive
as stable as possible manner enabling scalable EDW services for most BI
needs across the business
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 140
LSA Bui l di ng Bl oc k s
Ref er enc e LSA & Cust omer LSA – Ex ampl e
Acquisition/
PSA Layer
Corporate
Memory
Layer
Propagation
Layer
Business
Transformation
Layer
Flexible
Layer
Dimensional
Layer
Virtual
Layer
(YADSS100)
YCDSS100
YPDSS1DX
YPDSS1GX
YPDSS1WX
YPDSS1UX
YBAPP1AX
YBAPP1DX
YBAPP1GX
YBAPP1WX
YBAPP1UX
YFAPP1AX
YFAPP1DX
YFAPP1GX
YFAPP1WX
YFAPP1UX
YDAPP1AX
YDAPP1UX
YVAPP1XX
YVAPP1XX

YDAPP1WX
YDAPP1DX
YDAPP1GX
Lookup-tables
Asia
Europe
Americas
Germany
US
YPDSS1AX
Control
tables
L
S
A
Reporting Layer (Architected Data
Marts)
Business Transformation Layer
O
p
e
r
a
t
i
o
n
a
l

D
a
t
a

S
t
o
r
e
Data Propagation Layer
Quality & Harmonisation
Layer
Corpor
ate
Memor
y
Data Acquisition Layer
From Reference LSA to Customer LSA:
Þ Individual Domains
Reference LSA
Customer LSA
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 141
Tr anspar ent , Sc al abl e St ruc t ur i ng of BW:
LSA Domai ns & Layer s
L
S
A
Reporting Layer (Architected Data Marts)
Business Transformation Layer
Data Propagation Layer
Quality & Harmonisation Layer
Corporat
e
Memory
Data Acquisition Layer
Access Abstraction Layer
(MultiProvider)
SingleSource System (Layer)
L
S
A
Reporting Layer (Architected Data Marts)
Business Transformation Layer
Data Propagation Layer
Quality & Harmonisation Layer
Corporat
e
Memory
Data Acquisition Layer
Access Abstraction Layer
(MultiProvider)
Multiple Source Systems (Layer)
Distribution
actively designed:
Domains
Distribution
inherited
LSA Domains distribute the transactional data for the entire BW EDW in a
disjunctive manner. The meta data of domains are identical.
The LSA addresses an evolutionary EDW approach introducing Data
Domains enabling ‘local’ BI services, 24x7 operations and
administration without neglecting the broader EDW picture.
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /20090210/ Page 142
LSA Dat a Domai ns and Layer
Pr opert i es of LSA Domai ns
As a rule of thumb:
Þ Domains organize data by a general criteria driven by Reporting, BI and
Manageability requirements i.e. domains often differ from operational data
organization
Þ Domains are from transactional data point of view disjunct – harmonized master
data are shared
Þ Domains use identical meta data definitions
Þ Cross views are achieved via MultiProviders or dedicated persistent InfoProviders
Þ Domains should be stable
Þ Domains should be consistent across all Layer (across all flows)
Þ Domains should be general for all Layer, exceptions:
Þ The Acquisition Layer inherits the structuring from source systems/ extractors
Þ Domains do not apply on the Corporate Memory
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 143
Cri t er i a Def i ni ng LSA Domai ns
Golden rule defining domains:
as many domains as necessary – as less domains as possible
Defining Domains – rules of thumb
ÞOften a geography driven domain concept works well
Þ Often one basic domain per continent makes sense (=rough time zone handling)
e.g. APA, EMEA, Americas
Þ time zone aspects may lead to e.g. 3 Asian domains East-Asia, Mid-Asia, West-Asia
Þ E.g. for continental BW EDW a starting point could be East, Middle, West...
ÞExpected Volume (realistic volume estimates are a key input defining domains )
Þ Large APA volume contribution & large (for your business important) countries may get an
own domain e.g. China and US
ÞIndependency (special service level agreement) for certain countries
Þ Important countries/ markets get an own domain
ÞRobustness: take Quality/ stability of different sources into consideration
Þ (potential) instable data offerings from sources may lead to an own domain
ÞEach customer may have additional criteria, normally it is a mixture of multiple items
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 144
LSA Domai n Cri t er i a & Busi ness Vi ew
Domains should be as stable as possible (TCO) on the one hand side but on
the other hand side from the overall service perspective it makes sense
defining domains with respect the business view on data, but this is not
necessarily stable as it refers to organization.
We may find a business view of data that is not reflected by organizational
units of the sources (e.g. sources know Company-Code, planning area etc but
business view is ‘market’ what is not reflected by the sources)
Domains offer the opportunity resolving this using domains in an EDW for
better BI services (reporting) and more transparency (domains)
Note: As domains are a mixture of application & technical dimensions we
should not overload domains with a purely business view -> observe Golden
rule
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 145
Domai ns and Sour c e Syst ems
Def i ni ng Adequat e Domai ns
...... ......
......
...... Acquisition
Propagation
Acquisition
Propagation
DataSource from single corporate source-system
DataSource from multiplesource-systems
?
?
?
No
domains
No
domains
adequate
domains
adequate
domains
Too many
domains
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 146
LSA Domai ns Def i ni t i ons & EDW Rol l -Out
Managi ng Vol ume Unc er t ai nt y
Especially at the beginning of an EDW roll-out there is often a large degree of
uncertainty with respects to volume expected in the future.
But there is normally at least a classification of organizational units that drive the
domains possible in terms of t-shirt sizes (S,M,L)
Þ S & M organizational units reside in standard domains (ASIA; AMERICAS; EMEA)
Þ L organizations/ markets get an own domain
Possible challenges during roll-out:
Þ Domains will become too big e.g. EMEA -> Create an additional Domain EMEA2
Þ Large Domains -> think about additional splitting criteria (note: time-partitioning helps
normally little with load issues)
This leads to initial domain concept for your EDW
Þ e.g. Asia, EMEA, AMERICAS, CHINA, US
Note: Existing Domains must continuously be observed with respect to SLA compliance
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 147
LSA Domai ns
Domai n DWH Char ac t er i st i c
BW EDW
All data
Domain ‘B’
EMEA
Domain ‘A’
APA
Domain ‘C’
AMS
Country/ Market
E.g. ‘F’
E.g. ‘UK’
,,,,,,,,
Organizational-unit
0COMPCODE = FR01
0COMPCODE = FR02
In BW EDW all loaded
records will be qualified/
‘stamped’ assigning a stable
‘domain- driving’
characteristic like market/
country to organizational
criteria like in this example
0COMPCODE coming from
source system.
This allows easy redistribution of a
domain data if service levels
cannot be kept
assign
assign
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 148
LSA Domai ns
Domai n DWH Char ac t er i st i c – New Domai n
BW EDW
All data
Domain ‘B’
EMEA – ‘F’
Domain ‘A’
APA
Domain ‘C’
(AMS)
E.g. ‘UK’
,,,,,,,,
Organizational-unit
0COMPCODE = FR01
0COMPCODE = FR02
Domain ‘F’
Country ‘F’
France ‘F’ gets an own
domain
All records in Domain ‘B’
with country ‘F’ are moved
to the new domain
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 149
LSA & I mpor t anc e of Vol ume Est i mat es
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 150

Agenda PDEBW1
1 2 3 4 5 6 7

BI Maturity & Data Logistic Model-driven SAP BW as DWH Best Practice BW Layered, Scalable Architecture (LSA) for EDW LSA Implementation Recap of LSA - LSA Related Topics

BW Data Model and Data Integration Appendix - BI Organization & Governance

© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 2

Agenda PDEBW1

1 1.I 1.II 1.III

BI Maturity & Data Logistic BI is the Top Priority of CIOs Data Logistic Types and BI Maturity - Overview Simple Data Logistics – Data Mart Based Advanced Data Logistics – Data Warehouse Based BI Data Logistic Excellence – Hinderers & Enablers

1.IV 1.V

© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 3

EDW & BW Layered.The CIOs View and Priorities Information Excellence Process Excellence © SAP 2009 / PDEBW1. Scalable Architecture (LSA). Juergen Haupt /200910/ Page 4 .

consistent Data standardization & consolidation of BI data logistic (EDW) BI BI Data Data Logistic Logistic Standardization and consolidation of Enterprise Applications (processes) standardization & consolidation of BI data logistic (EDW) Simple Truth: No BI without Data Process Excellence (Prio 2/4: Enterprise Applications) © SAP 2009 / PDEBW1. CPM.BI & Enterprise Application Excellence drive BI Data Logistic Excellence Information Excellence (Prio 1: BI Applications) No BI-.EDW & BW Layered.and Planning Excellence without trustable. Scalable Architecture (LSA). Juergen Haupt /200910/ Page 5 . accurate.

Scalable Architecture (LSA).EDW & BW Layered. Data-Logistic SAP ERP SAP CRM Other Legacy BI Framework BI Business Value Drivers © SAP 2009 / PDEBW1.A Valid BI Data Logistic as Prerequisite for BI Value BI Applications and Functionality Standard Reporting. Agile BI Corporate Performance Management Efficiency Suitability Consistency Flexibility Data-Infrastructure. Juergen Haupt /200910/ Page 6 .

Juergen Haupt /200910/ Page 7 .EDW & BW Layered.II 1.Overview Simple Data Logistics – Data Mart Based Advanced Data Logistics – Data Warehouse Based BI Data Logistic Excellence – Hinderers & Enablers 1.V © SAP 2009 / PDEBW1. Scalable Architecture (LSA).I 1.III BI Maturity & Data Logistic BI is the Top Priority of CIOs Data Logistic Types and BI Maturity .IV 1.Agenda PDEBW1 1 1.

.TDWI – Accelerating BI Maturity The Chasm on The Way to The EDW Shore “ Data “Data Marts ” “ Data “Data Warehouses ” Marts” ” “Spreadmarts” “ Management “Management Reporting Reporting” ” GULF GULF Warehouses” “ “Enterprise DW ” DW” CHASM 2. Prenatal Figure 1. Infant 3.EDW & BW Layered. Teenager Business Value Integration Consolidation 5. Juergen Haupt /200910/ Page 8 .. © SAP 2009 / PDEBW1. The TDWI Maturity Model shows how many organizations evolve their BI solutions. Sage 1. Adult “ BI Services ” “BI Services” 6. and the labels above the bell curve represent the data structure dimension in the model.. Moving towards a centralized EDW based architecture means we have to overcome profound challenges (chasm): Non-IT ones like politics. Scalable Architecture (LSA). missing strategy and sponsorship and IT ones .. Child 4. The Gulf and Chasm represent places where many companies get stuck. the bell shaped curve represents the number of companies in each stage.

EDW & BW Layered. TDWI Research The Data Warehousing Institute 2007 © SAP 2009 / PDEBW1.interestingly.Correlation of BI Value & Data Logistics Excellence The Need of Centralization Prenatal Type of System Analytical Tools Infant Executive System Spreadsheets “ Inform Executives ” Child Analytical System OLAP/ Ad hoc Reports “ Empower Workers ” Teenager Monitoring System Dashboards/ Scorecards “ Monitor Performance Adult Strategic System Predictive Analytics “ Drive the Business ” Sage Business Service Customer BI Embedded BI “ Drive the Market ” Financial System Static Reports “ Cost Center ” Executive Perception ” ROI ROI Cost ROI Value Architecture Operational Reporting Spreadmarts Data Marts Data Warehouses Enterprise DW Analytic Services Best practice: centralize before you federate ‘. organizations that try to federate development and administration before they have centralized these tasks often have limited success. Eckerson Director. Scalable Architecture (LSA)..’ Beyond the Basics: Accelerating BI Maturity Wayne W.. Juergen Haupt /200910/ Page 9 .

Scalable Architecture (LSA).II 1.IV 1.III BI Maturity & Data Logistic BI is the Top Priority of CIOs Data Logistic Types and BI Maturity .I 1.V © SAP 2009 / PDEBW1.EDW & BW Layered.Overview Simple Data Logistics – Data Mart Based Advanced Data Logistics – Data Warehouse Based BI Data Logistic Excellence – Hinderers & Enablers 1. Juergen Haupt /200910/ Page 10 .Agenda PDEBW1 1 1.

Usually refers to a physical platform on which summarized data is stored for decision support. Prenatal 2. © SAP 2009 / PDEBW1. Infant Intra-Departmental: Complexity & ContinentalData Europe: Misalignment Sources 3.EDW & BW Layered. Teenager Business Value Semantic Integration Data Consolidation 5. subject area.BI Data-Logistic Types ‘Standalone’ Data Mart Architecture “Data Marts” “Data Warehouses” “Spreadmarts” “Management Reporting” GULF CHASM “Enterprise DW” “BI Services” 1. or limited part of the organization. Sage Marts A definition: An implementation of an analytics application serving a single department. Scalable Architecture (LSA). Adult 6. Juergen Haupt /200910/ Page 11 . Child 4.

multiple extractions & staging High departmental acceptance but islands (silos.EDW & BW Layered. inconsistent semantics & values Different data management for Data Mart and different BI tools Inhomogeneous. Different data models i.e.‘Standalone’ Data Marts Standalone Data Marts are independently built BI solutions often based on different BI tools. Scalable Architecture (LSA). stovepipes) Material # # Employee Standalone Data Marts Sources Material Management Sales HR © SAP 2009 / PDEBW1. Juergen Haupt /200910/ Page 12 .

mdx Departmental reach 3.. multiple extractions & staging Data Mart tools have often own data management. Data Model No integration between Data Marts. Advantages Departmental view: flexible. Architecture. High daily involvement by local Business Analyst Limited IT support 2. Issues Cross-departmental view: misalignment. sql-generator. Different data models i. . high costs. inconsistent semantics & values 4. islands. Organization Departmental. Scalable Architecture (LSA). not pursuable. autonomy Adequateness for Customer types Adequate only from departmental perspective or for small customer (< 100) with IT knowledge © SAP 2009 / PDEBW1.BI Data-Logistic Types – Flier : Standalone Data Marts Standalone Data Marts = Departmental BI 1. communication confusion.EDW & BW Layered.e. multi-dimensional.. 6. 5. Juergen Haupt /200910/ Page 13 . Reach of Data Logistic Inhomogeneous.

IV 1. Scalable Architecture (LSA).Agenda PDEBW1 1 1.EDW & BW Layered.I 1. Juergen Haupt /200910/ Page 14 .III BI Maturity & Data Logistic BI is the Top Priority of CIOs Data Logistic Types and BI Maturity .V © SAP 2009 / PDEBW1.II 1.Overview Simple Data Logistics – Data Mart Based Advanced Data Logistics – Data Warehouse Based BI Data Logistic Excellence – Hinderers & Enablers 1.

Juergen Haupt /200910/ Page 15 . Prenatal 2.BI Data-Logistic Types Data Warehouses “ Data “Data Marts ” Marts” ” “Spreadmarts” “ Management GULF “Management GULF Reporting ” Reporting” “ Data “Data Warehouses ” Warehouses” “ “Enterprise DW ” CHASM DW” “ BI Services ” “BI Services” 1.EDW & BW Layered. To show the new quality we call these kind of Data Marts ‘Architected’ © SAP 2009 / PDEBW1. Scalable Architecture (LSA). Teenager 5. Infant 3. Child 4. Sage Business Value Semantic Integration Data Consolidation BI tools work on Data Marts or more general on special prepared sets of data Because of the missing integration (islands) between Data Marts. Data Warehouses are established as a common integrated data foundation for Data Marts. Adult 6.

The Data Warehouse ‘The Data Warehouse is a subject-oriented. Juergen Haupt /200910/ Page 16 Org Unit A .BI Data-Logistic Types Data Warehouse Architecture Sources Staging Data Area Warehouse Data Marts Bill Inmon . non-volatile collection of data used to support the strategic decision-making process for the enterprise.’ © SAP 2009 / PDEBW1. Scalable Architecture (LSA). It is the central point of data integration for business intelligence and is the source of data for the data marts. integrated.EDW & BW Layered. delivering a common view of enterprise data. time-variant.

A Data Warehouse as Integrated Foundation for Data Marts – Architected Data Marts Before Data Marts are loaded the data are integrated to common semantics given by the dwh data model and the values are harmonized. Data Warehouse: Standardized extraction & load Integrate data with respect to integrated semantics & values Store Integrated results Load integrated data into Data Marts (Architected Data Marts) From this perspective the data warehouse represents a layer in the information system architecture Material Employee = = Architected Data Marts vendor customer material employee Data Warehouse Staging Area Sources Material Management Sales HR © SAP 2009 / PDEBW1. The integrated results are stored persistently and build the data warehouse. Data Marts are filled from the Data Warehouse. Juergen Haupt /200910/ Page 17 . Scalable Architecture (LSA).EDW & BW Layered. A Data Warehouse is a necessary condition for consistent BI.

Juergen Haupt /200910/ Page 18 Copyright ©1999 by William H.EDW & BW Layered.Bill Inmon’s Corporate Information Factory & The Data Warehouse Conceptual Details Subject-oriented Integrated historical complete comprehensive application neutral granular Higher organizational-unit owned non-volatile… © SAP 2009 / PDEBW1. Scalable Architecture (LSA). Inmon .

Data Warehouse Principles InfoCube Customer Dimension A Information Completeness. Granularity Material Dimension 4712 4713 Customer A A Material 4712 4713 Time 200211 200211 Amount Company Currency 100 150 Time Dimension 200211 other InfoCube Example: BW Architected Data Mart Layer other InfoCube granularity : week monthly BW Data Warehouse Layer Customer A A A Time granularity : month weekly DocNo 1 1 2 Pos 10 10 10 Material 4711 4712 4713 granularity : day daily Amount Local Currency 100 200 300 Amount Company Currency 50 100 150 20021107-10am 20021107-3pm 20021107-4pm daily Sourcesystem Customer A A A Time 20021107-10am 20021107-3pm 20021107-4pm DocNo 1 1 2 Pos 10 10 10 Material 4711 4712 4713 Amount Local Currency New booking 100 Correction booking 200 New booking 300 © SAP 2009 / PDEBW1. Juergen Haupt /200910/ Page 19 . Scalable Architecture (LSA). Reusability.EDW & BW Layered.

unification and integration transformations Based on common definitions © SAP 2009 / PDEBW1. Scalable Architecture (LSA).DWH & Architected Data Mart Layer Introducing a Data Warehouse Layer the architecture has two main Layer: Architected Data Marts Architected Data Mart Layer business driven transformations business defined BWA area BW DWH Layer no business driven transformations just cleansing.EDW & BW Layered. Juergen Haupt /200910/ Page 20 DWH .

Bill Inmon ‘What is a Data Warehouse?’ Data warehouses are significantly different from data marts. Scalable Architecture (LSA). Bill Inmon. The volume of data found in the data warehouse is significantly different from the data found in the data mart. Corporate owned or larger part of the organization The data warehouse represents a truly corporate/ higher organizational-unit effort. DM Direct. Application neutral. November 1999 Integrated. Juergen Haupt /200910/ Page 21 . non-volatile = permanent Granular The data warehouse contains the most granular data the corporation has. From ‘Data Mart Does Not Equal Data Warehouse’. non-flavored The structure and the content of the data in the data warehouse do not reflect the bias of any particular department. subject-oriented The data warehouse data is integrated from the many legacy sources. Usually the data warehouse is built and owned by centrally coordinated organizations time-variant = complete history. Data warehouses are arranged around the corporate subject areas found in the corporate data model. Comprehensive The data warehouse is broad in scope and keeps more data than actual requested by the business/ data marts © SAP 2009 / PDEBW1.EDW & BW Layered. Data mart data is usually much less granular than data warehouse data. but represent the corporation's needs for data.

missing business acceptance.BI Data-Logistic Types – Flier : Data Warehouse Data Warehouse & Architected Data Marts: Integrated. Cross-organizational BI (on DWH definition level) 1. Advantages Data warehouse owner view: Integrated reporting. time to market 5. needs business buyin. standards. Scalable Architecture (LSA). Challenges Business-user view: Often IT-driven. Data Model Cross Subject-area integrated Issues. 4. Adequateness for Customer types For independent operating higher organizational units © SAP 2009 / PDEBW1. Juergen Haupt /200910/ Page 22 . Organization IT managed sometimes BI Competency Center (BI CC) Architecture.EDW & BW Layered. consistency. reliable data marts) Cross organizational on upper Org-unit 3. 2. Reach of Data Logistic Common extraction & staging techniques (ETL) for all cross departmental Data Marts Data Integration results have own persistence Data Marts served by the data warehouse (architected. less TCO than pure Data Mart architectures 6.

Adult 6. process e. Teenager 5. business unit.g. which means for larger companies there exist multiple Data Warehouses. Infant 3. Thus from a higher level point of view they suffer from similar issues like standalone Data Marts. Sage Business Value Semantic Integration Data Consolidation Data Warehouses cover a certain part of an organization (country. © SAP 2009 / PDEBW1. Juergen Haupt /200910/ Page 23 . Child 4. Scalable Architecture (LSA).BI Data-Logistic Types Enterprise Data Warehouse (EDW) Marts” ” “Spreadmarts” “ Management GULF “Management GULF Reporting ” Reporting” “ Data “Data Marts ” “ Data “Data ” Warehouses Warehouses” “ “Enterprise DW ” CHASM DW” “ BI Services ” “BI Services” 1. The Data Warehouses are often isolated. Prenatal 2. FI).EDW & BW Layered.

EDW & BW Layered. Scalable Architecture (LSA).The Multiple Data Warehouse Landscape Issue: Justification of an Enterprise Data Warehouse Source Source Source Unit Source 1 Local SAP BW Source DWH Stream Source Local SAP BW Unit C Group SAP BW Local SAP BW Stream A SAP BW SAP BW 2 SAP BW Local SAP BW Local SAP BW Source Source Stream B DWH Source Source Source © SAP 2009 / PDEBW1. Juergen Haupt /200910/ Page 24 .

EDW & BW Layered.BI Data-Logistic Types Impacts of Multiple Data Warehouses Often we find multiple DWHs in an Organization Complexity & Misalignment Org Unit A Org Unit B © SAP 2009 / PDEBW1. Scalable Architecture (LSA). Juergen Haupt /200910/ Page 25 Org Unit C .

Bill Inmon’s Corporate Information Factory & Enterprise Data Warehouse (EDW) Enterprise Data Warehouse (EDW): A single instantiation of a data warehouse layer for the entire corporation or big parts of the organization is often called the Enterprise Data Warehouse EDW-Keywords offer a ‘single version of truth’ extract once & deploy many support the ‘unknown’ re-build new-build controlled redundancy provide a corporate memory Conceptual Details Integrated historical complete comprehensive application neutral granular corporate owned non-volatile… Copyright ©1999 by William H. Juergen Haupt /200910/ Page 26 .EDW & BW Layered. Inmon © SAP 2009 / PDEBW1. Scalable Architecture (LSA).

Juergen Haupt /200910/ Page 27 .EDW & BW Layered.Treating Information as Corporate Asset Flexibility is Key of Growth & Survival The Known Don’t limit yourself by just focusing on current requirements The Unknown Be prepared for unpredictable future needs © SAP 2009 / PDEBW1. Scalable Architecture (LSA).

Scalable Architecture (LSA).” © SAP 2009 / PDEBW1. CIO of Henkel. All transactional data from the source systems need to pass the EDW layer before it is written to the final data destinations or data marts. IT project manager at Henkel’s business intelligence competence center: “The enterprise data warehouse (EDW) layer does lead to a degree of redundant data. which in turn raises the importance of centralized management of the company.EDW & BW Layered. summarizes the motivation: “We are increasingly focusing on the whole picture. This enables us to safeguard data quality and create a single version of truth that contains all historical data. Juergen Haupt /200910/ Page 28 . but this can be kept under control. This process is actively supported by IT.The Whole Picture – The EDW SAP BW @Henkel Peter Hinzmann.” Werner Böttiger.

BI Data-Logistic Types – Flier : EDW
EDW & Architected Data Marts: Integrated, Corporate BI
1. Organization
BI Competency Center – BI CC (IT, analysts, business)

2.

Architecture, Reach of Data Logistic
Like DWH but for entire Corporation (may be division) Single Point of Truth, historical complete, comprehensive, granular, integrated, Architected Data Marts built on EDW

3.

Data Model
Cross Company

4.

Issues, Challenges
IT as driver, missing C-level sponsorship Missing business acceptance, needs business buy-in, time to market

5.

Advantages
BI on entire value chain, Integration, standardization, new information culture, competitive advantages Enabler for consistent ‘BI as service’

6.

Adequateness for Customer types
For all customers within highly competitive markets, High volume, Global organization

© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 29

Ralph Kimball’s view on Data Warehouse Layer
Figure 1.1 The basic elements of the data warehouse.

© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 30

Meta Group I

© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 31

Int. Juergen Haupt /200910/ Page 32 .EDW & BW Layered. Scalable Architecture (LSA).Bill Inmon's Corporate Information Factory and The Enterprise Data Warehouse DSS Applications Departmental Data Marts Acctg Finance Marketing Sales ERP ERP ERP CRM ETL Staging Area Changed Data eComm. Mart Bus. ERP Corporate Applications Exploration warehouse/ data mining local ODS Granularity Manager Cross media Storage Management Near line Storage Session Analysis Dialogue Manager Cookie Cognition Preformatted dialogues Archives Web Logs Internet * Source: Bill Inmon © SAP 2009 / PDEBW1. EDW Global ODS Oper.

V © SAP 2009 / PDEBW1.Overview Simple Data Logistics – Data Mart Based Advanced Data Logistics – Data Warehouse Based BI Data Logistic Excellence – Hinderers & Enablers 1.Agenda PDEBW1 1 1.III BI Maturity & Data Logistic BI is the Top Priority of CIOs Data Logistic Types and BI Maturity .IV 1.II 1. Juergen Haupt /200910/ Page 33 . Scalable Architecture (LSA).EDW & BW Layered.I 1.

Infant 3. Juergen Haupt /200910/ Page 34 . Child 4. Scalable Architecture (LSA). Adult 6.Why are there Chasms & Gulfs ? “ Data “Data Marts ” Marts” ” “Spreadmarts” “ Management GULF “Management GULF Reporting ” Reporting” Warehouses” “ “Enterprise DW ” CHASM DW” “ Data “Data Warehouses ” “ BI Services ” “BI Services” 1. Prenatal 2. Sage zero Business Value Semantic Integration Data Consolidation high © SAP 2009 / PDEBW1. Teenager 5.EDW & BW Layered.

Development) Reduce number of BI tools (BI Consolidation) Business perspective on Information solution. Scalable Architecture (LSA).Different Perspectives on BI Business & IT IT perspective on Information data.EDW & BW Layered. TCD (Total Cost of Ownership. agility Performance Consistency High availability BI Applications and Functionality Standard Reporting. Juergen Haupt /200910/ Page 35 BI Business Value Drivers . technology driven BI Manage BI (Development. time to market Autonomy. Operation. Housekeeping) Lower TCO. solution business driven BI Flexible. Administration. Agile BI Corporate Performance Management Efficiency Suitability Consistency Flexibility Data-Infrastructure. Data-Logistic SAP ERP SAP CRM Other Legacy BI Framework © SAP 2009 / PDEBW1. Maintenance.

. from organizational-unit to organizational-unit Offering different...Business Perspective on BI Organizational Egoism The preferred BI flavor/ tool differs from employee to employee. from department to department. which means per se in high costs But this is not the main issue: A dispersed BI landscape implies as experiences show an island like. Sales FI HR Spain UK France Nordic Different BI tools and Data infrastructures ........ dispersed data infrastructure.. . Scalable Architecture (LSA). This results in even higher costs but more important doubts the value of BI © SAP 2009 / PDEBW1.... BU Unit . adequate BI functionalities to different business-user types is fine but often results in an uncontrolled bunch of BI-tools. Juergen Haupt /200910/ Page 36 Group Division A Division B Division C Division D ..EDW & BW Layered...

Local Business and Issues of Dispersed BI Data Logistics Source Dispersed BI Data Logistic: Source 1 Missing Service level definition – Uncontrolled data flows Local – UncontrolledSAP BW extractions Source Redundant development Inconsistent data model. Islands Unreliable Information-basis Stream B Uncontrolled redundancy DWH High Costs Source Source SAP BW Source Source Unit DWH Stream Source Local SAP BW Unit C SAP BW 2 SAP BW Local SAP BW Local SAP BW Source Source Organizational/ business and also IT egoisms cause dispersed Source BI data-logistics. SAP AG 2007. Scalable Architecture (LSA).EDW & BW Layered. SAP Skills 2007 Conference / B2 / 37 © SAP 2009 / PDEBW1. KPIs Local NoGroup overall BI / DWH Strategy BW SAP SAP BW This results in:Stream A Silos. Juergen Haupt /200910/ Page 37 . They often hinder any evolutionary step towards an architectured BI data-logistic.

processing and delivering it. consistency. Any significant improvement in information – in terms of quality. W. This again depends on a Clear commitment of c.(orporate) management (sponsorship) – not CIO “ If there is no organizational momentum toward a common goal.H. timeliness and accuracy – depends on gaining control of information processes and definitions. Corporate interests On the other hand side we find the awareness: “ Delivering consistent information across the business depends on shared definitions and business rules for collecting. then the best architecture. Inmon © SAP 2009 / PDEBW1.EDW & BW Layered. a customer architecture council But any advanced BI data logistic like an Enterprise Data Warehouse depends on establishing a high degree of central governance. Local interests overrule Group. Juergen Haupt /200910/ Page 38 .Why are there Chasms & Gulfs ? Non-IT Reasons Mature BI Data logistics are hindered by local interests: Particular. Scalable Architecture (LSA). the best framework in the world is bound to fail.

...EDW & BW Layered. Scalable Architecture (LSA).Is IT Ready for EDW? IT and Issues of BI Data Logistics We miss: no of applications data marts generate demand demand supply Copa Service level definitions Guidelines & Best Practices Quality-control in Development Comprehensiveness Completeness ... Juergen Haupt /200910/ Page 39 DWH Extraction DWH DWH Staging Staging Extraction Source Staging Extraction Source Source volume . © SAP 2009 / PDEBW1. This results in: Increasing Complexity – Increasing operational cost – Increasing maintenance costs Low flexibility to answer ‘the unknown’ – Increasing time-to-market Consistency issues .

Scalable Architecture (LSA). BI-application after BI-application. the EDW is underestimated! This is the best prerequisite to fail! © SAP 2009 / PDEBW1.Why are there Chasms & Gulfs ? Is IT Ready For EDW ? BI is totally underestimated and misunderstood by IT! (and others) BI is highly different than dealing with operative requirements. BI is daily business-related (at all org-levels) and thus highly volatile/ dynamic BI data-logistics must be highly flexible BI deals with historical data what creates high data volume growth BI data-logistics have to manage often 10 times or more of data BI never ends BI is implemented incrementally. BI deals with data. Juergen Haupt /200910/ Page 40 . Data volume growth but just so important. Growth of meta-data BI means steadily increasing maintenance-. operative-business with processes! As BI is underestimated.EDW & BW Layered. administration. standardized robust and automated BI is steadily under time-to-market pressure BI data-logistics cannot rely on today. They must ‘support the unknown’ the unforeseeable .and operation-workload BI data-logistics have to be highly transparent.

Crossing BI Data Logistic Gulfs & Chasms – Is IT Ready for a BI Excellence Framework? * BI Excellence is more than Infrastructure/ Data-Logistic Excellence: Strategy Information as Corporate Asset Alignment Organization BI Competency Centre Process Methodology Realization Efficiency Suitability Consistency Flexibility Applications and Functionality Standard Reporting.EDW & BW Layered. Agile BI Corporate Performance Management Enterprise Data Warehouse Layered. Scalable Architecture SAP ERP SAP CRM Other Legacy Infrastructure. Juergen Haupt /200910/ Page 41 BI Business Value Drivers * BI Framework introduced by Gartner . Scalable Architecture (LSA). Data-Logistic BI Framework* © SAP 2009 / PDEBW1.

Juergen Haupt /200910/ Page 42 .Treat Information as Corporate Asset Mature BI Data-Logistics (EDW) Increasing awareness about importance of a proper SAP BW data-logistic for Data and Meta Data Customer investments for an adequate SAP BW infrastructure to manage data and meta data consistently and in a comprehensive manner Activities run under the title ‘Enterprise Data Warehouse’ Most common to all activities is the concept of a layered architecture of information systems introduced by Bill Inmon The Corporate Information Factory (CIF) SAP BW with no corporate standards Complexity & Cost: Administration. Maintenance SAP BW Layered. Scalable Architecture (LSA) No of Application Scenarios (Data Marts)/ Volume © SAP 2009 / PDEBW1.EDW & BW Layered. Scalable Architecture (LSA).

Scalable Architecture (LSA).EDW & BW Layered.Agenda PDEBW1 1 2 3 4 5 6 7 BI Maturity & Data Logistic Model-driven SAP BW as DWH Best Practice BW Layered. Juergen Haupt /200910/ Page 43 . Scalable Architecture (LSA) for EDW LSA Implementation Recap of LSA .LSA Related Topics BW Data Model and Data Integration Appendix .BI Organization & Governance © SAP 2009 / PDEBW1.

Tuning BW Aggregates. Transformation. UDC Process Design BW Process Chains Deploy BI applications BW Transport Monitor BI applications BW Monitoring & Statistics Performance. NLS Run. Error-handling InfoPackage. DAPs Rule-editor.BI Data-Logistic & Applications Modeling with SAP BW Business requirements Erwin. Extractors: BCT. Scalable Architecture (LSA). Juergen Haupt /200910/ Page 44 . © SAP 2009 / PDEBW1. Error-DTP Locate source & map data & model. DSOs tables Data Transfer. Visio.EDW & BW Layered. Generic. robust and best practice way. Data Warehousing with BW is end to end model-driven. standardized. DTPs. This is a prerequisite for designing data warehouses in a transparent. Eclipse (planned) DWH data model BCT BW Reference Data Model Multi dimensional model BW InfoCube/ MultiCube Physical Star Schema Star Schema/ BWA index Logical DWH & Staging BW DSOs Physical DWH & Staging Generated PSA. Recover BW Requests/ Process Chains Housekeeping BW Housekeeping Proc Chains Operate/ Administration Modeling Actions BW models and implement all relevant BI data-logistic areas. BIA. extractor BCT DWH Data Model.

DWH operations modeling Extract. load. DWH data modeling Reference data model InfoObjects + relations Subject-areas Building blocks BW is from all perspectives fully model-driven Architected Data Marts: InfoCubes/ BWA 2. Scalable Architecture (LSA). Monitor SAP-Source data model extractor extractor map models: provided by BW map models: user-defined ETL/ Staging PSA. DSOs 3.BW Model-Driven Data Warehousing based on BCT Data Warehouse Reference Data Model BW Model-driven DWH: 1. Error-handling Admin.EDW & BW Layered. InfoPackages Non-SAP-Source data model SAP Source A © SAP 2009 / PDEBW1. DWH structure modeling InfoProviders DWH Stores: DTPs. Juergen Haupt /200910/ Page 45 Non-SAP Source B . transform Transfer.

Scalable Architecture (LSA). InfoSource) The Meta-objects to design the operations between persistent elements Extract.SAP BW Model-Driven Data Warehousing Cornerstones Designing a Data-Logistic with SAP BW is from all perspectives fully model-driven: The Reference Data Model (BCT) Provides mapping of SAP-source data model to BW reference data model (InfoObjects) For all Non-SAP sources Customer does not start on a ‘blank’ sheet of paper (-> reference model) Mapping of source data model to reference data model by customer Flexible. InfoSet..) Staging Virtualization (DataSource. Juergen Haupt /200910/ Page 46 .. InfoObject.EDW & BW Layered. PSA) Access Virtualization (InfoProviders: MultiProvider. expandable The Meta-objects to design persistent & virtual elements of a data-logistic Persistent Data-containers (InfoProviders: InfoCube. DSO. Transport. Virtual InfoProvider. Load (InfoPackage) Transformation (Transformation Rules) Error-Handling Transfers (DTPs) Complex data movements (Process chains) © SAP 2009 / PDEBW1.

Scalable Architecture (LSA)..........Dimension_ID ._Dimension_ID ... Juergen Haupt /200910/ Page 47 .............. Shared Part of Material Dimension Local Part of Material Dimension Local Part of Material Dimension Material Master Table 0MATERIAL Material_Dimension_ID 0MATERIAL Material Dimension Table Material_Dimension_ID SalesOrg_Dimension_ID Time_Dimension_ID Customer_Dimension_ID Sales Amount Quantity Material Type Material_Dimension_ID 0MATERIAL 0PROD_HIER Material Dimension Table Material_Dimension_ID .Dimension_ID Material Text Table 0MATERIAL Language Code Material Name Material Hierarchy Table Vertriebsorg anis ation Regio n 1 Regi on 2 R egion 3 InfoCube A Bezirk 1 Be zi r k 2 Bezirk 3 Bezirk 4 Bezirk 5 G ebiet 1 Geb iet 2 G ebi et 3 G ebi et 3a Gebi et 4 G ebiet 5 Geb iet 6 G ebie t 7 Geb iet 8 InfoCube 2 Key figure B FACT Table Key figure 1 FACT Table Material Dimension: local & conformed part Conformed dimensions enable virtual SAP BW MultiProvider scenarios © SAP 2009 / PDEBW1...Preventing Silos Conformed.BW Data Warehouse & Architected DMs: Conformed Dimensions ......EDW & BW Layered....

Juergen Haupt /200910/ Page 48 . Scalable Architecture (LSA).EDW & BW Layered.The SAP BW MultiProvider Concept Concept of SAP BW MultiProvider End-User Reporting & Analysis need SAP BW BEx Report end-user requirement specific logical (dimensional) model SAP BW InfoCube(s).DS-Object physical implementation basic facts not report specific SAP BW MultiProvider virtual Structures basic facts not report specific optional SAP BW BEx Query virtual Structures complex KPIs navigation behavior report (type) specific SAP BW MultiProvider separate physical persistent SAP BW Structures from Queries and Query dependent BEx-objects © SAP 2009 / PDEBW1.

BW Data Warehouse & Architected DMs: MultiProvider and Flexibility SAP BW Queries virtual SAP BW MultiProvider persistent SAP BW Structures C1 C1 + C1 Managing Workload (logical partitions) Performance New content Flexibility Rebuild Flexibility Separate optimal physical DB Schema from logical Schema Performance Expand the scenario (C1) seamless introducing a new scenario Flexibility Save investments © SAP 2009 / PDEBW1.EDW & BW Layered. Juergen Haupt /200910/ Page 49 . Scalable Architecture (LSA).

Scalable Architecture (LSA). FlatFile. BW Generic.EDW & BW Layered.DataConnect © SAP 2009 / PDEBW1. BW DBConnect. MPs BW Data-Logistic Modeling = Employee 0VENDOR 0CUSTOMER 0MATERIAL 0EMPLOYEE Transfer: DTPs.BW Data Warehouse & Architected DMs: Data Marts Built on BCT Reference Model Achieving Data Mart harmonization applying the BW reference data model Not necessarily explicit data warehouse persistency BW BW Reference Data Model Material = Architected Data Marts: InfoCubes. Juergen Haupt /200910/ Page 50 . Open Hub Transformation Rules Staging Area: BW PSA Load: BW InfoPackage SAP Non-SAP Material Management Sales HR OLTP Extraction: BW Extractors. Univ.

Scalable Architecture (LSA). InfoObject master data more or less summarized support the known SAP BWs are built on today's end-user requirements Designed to answer the known Answering the unknown is limited by Granularity of InfoCubes and DS-Objects (data content) – (historical completeness) Business-scope driven extraction – (extracted fields – process comprehensiveness) Business rules applied to original extracted data Availability of master data history Limited Completeness & comprehensiveness thus overall flexibility is limited to react on time © SAP 2009 / PDEBW1.BW Data Warehouse & Architected DMs: Limitations The ‘just use’ SAP BW DWH focused on InfoCubes. Juergen Haupt /200910/ Page 51 .EDW & BW Layered.

which exceeds the generic BW offering We find the ‘just use SAP BW approach’ with a lot of customers. increasing TCO when BW grows Solution focus on InfoCubes/ reporting . performance and operational issues will likely occur.usage of intermediate storage (DSOs) only if required to cover concrete business scope Limited flexibility Little standards & guidelines.Little awareness about the meaning of incremental BI limited scalability. Juergen Haupt /200910/ Page 52 . For corporate BI data warehouse architectures just relying on BWs generic data warehouse approach is not sufficient! © SAP 2009 / PDEBW1. Then the TCO will increase and scalability. Scalable Architecture (LSA).EDW & BW Layered.BW Data Warehouse & Architected DMs: Summary + Usage of BCT reference data model Common model-based consistent Data Marts + Relying on the power of the BW model-driven approach Enables implementation of even most complex BI scenarios Customers who have no explicit or corporate BI/ DWH strategy except ‘use SAP BW’ . The approach works well until the BW-system reaches a certain size in terms of data-volume and number of BI applications. maintenance.

EDW & BW Layered. Scalable Architecture (LSA).Implementing an EDW-Based Architecture in SAP BW I definitely need to go for an EDW-based layered architecture for SAP BW! But how to do it ? © SAP 2009 / PDEBW1. Juergen Haupt /200910/ Page 53 .

Agenda PDEBW1
1 2 3 4 5 6 7

BI Maturity & Data Logistic Model-driven SAP BW as DWH Best Practice BW Layered Scalable Architecture (LSA) for EDW LSA Implementation Recap of LSA - LSA Related Topics

BW Data Model and Data Integration Appendix - BI Organization & Governance

© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 54

Agenda PDEBW1

3
3.I 3.II 3.III 3.IV 3.V 3.VI

The Layered, Scalable Architecture (LSA) for EDW Reference LSA Overview Reference LSA & Customer LSA LSA Building Blocks – Data Layer LSA Building Blocks – Layer & Master Data LSA Building Blocks – Data Domains LSA Building Blocks – Data Model

© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 55

Everything Seems to be Clear, Doesn‘t It?
• EDW & Layering • Chasms

?

© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /20090210/ Page 56

EDW & BW Layered. DWH Latency (day) to Low (RDA) Standardization lowers costs High synergies for all BI flavors Increasing acceptance over time Note: of course we find also the ‘traditional’ EDW approach using BW! © SAP 2009 / PDEBW1. Scalable Architecture (LSA). monthly) High costs Low synergies for local/ departmental BI Overall acceptance problems Partly in scope Incremental approach to EDW Immediate ROI (local BI) Std.g.The Broad View of BW on EDW Traditional EDW Companies situation Scope-Areas Cross-/ Corporate BI Local. Juergen Haupt /200910/ Page 57 ./ depart. BI Operational BI Evaluation operational world fragmentation no view across the business In scope little information freshness (monthly) SAP BW EDW Increasing operational world harmonization Yet limited view across the business In scope high information freshness (daily) high flexibility Not in scope In scope standard reporting with local adoptions flexible roll out Not in scope Long time for ROI High Latency (e.

Scalable Architecture (LSA). Juergen Haupt /200910/ Page 58 BW Layered Scalable Architecture (LSA) – The Reference Architecture for BW on a Corporate Scale .EDW & BW Layered.Crossing the EDW Chasm: The BW Layered Scalable Architecture SAP Customer Experiences & Feedback : Statoil Samsung Philips Beiersdorf Henkel Novartis Kraft Foods Arla Foods Edeka Mc Kesson Adidas Land Hessen Japan Nike Tobacco Shell Downstream Syngenta Nestlé BASF Best Practices & EDW Principles LSA © SAP 2009 / PDEBW1.

Juergen Haupt /200910/ Page 59 .IV 3. Scalable Architecture (LSA) for EDW Reference LSA Overview Reference LSA & Customer LSA LSA Building Blocks – Data Layer LSA Building Blocks – Layer & Master Data LSA Building Blocks – Data Domains LSA Building Blocks – Data Model © SAP 2009 / PDEBW1.VI The Layered.Agenda PDEBW1 3 3.II 3.EDW & BW Layered.I 3.V 3. Scalable Architecture (LSA).III 3.

EDW & BW Layered. The Customer LSA describes corporate standards to build BI applications in a performant. best practice BW architectures founded on accepted EDW principles*. comprehensive customer DWH architectures (Customer LSA). Scalable Architecture (LSA). The BW LSA serves as a reference architecture to design transparent.BW Layered Scalable Architecture (LSA) The BW Layered Scalable Architecture (LSA) describes the design of service-level oriented. scalable. complete. flexible manner. * As introduced in Bill Inmon‘s Corporate Information Factory (CIF) © SAP 2009 / PDEBW1. maintainable. Juergen Haupt /20090210/ Page 60 .

) Meta Data Management Naming Conventions Organization (InfoAreas) Describes Support Scenarios Landmark Building Blocks Layer Domains Data Model & Data Integration Assistant Building Blocks Data Quality Landscape ETL Storage Ownership/ Organize Development concept Life Cycles Information Meta Object LSA Administration Data Base Housekeeping Monitoring Transport Security © SAP 2009 / PDEBW1.BW LSA: The Reference Architecture LSA Building Blocks Reference LSA Implementation Reference LSA Operations Reference BW LSA Describes core structures & definitions Describes design standards to build BI applications founded on building blocks Data Staging/ Management for transactional & master data Persistent Objects Flows . Juergen Haupt /20090210/ Page 61 .EDW & BW Layered. Scalable Architecture (LSA).scheduled/ recovery Transformation Programming (Abap) Organization (Process Ch.

Scalable Architecture (LSA). Landmark Building Blocks deal with architecture areas that need fundamental decisions before doing implementations – they play the same role like the supporting structures (pillars & ceilings) of buildings.LSA Building Blocks – The Architecture Cornerstones The LSA Building Blocks are the cornerstones of your future architecture thus having a decisive influence on the overall success of your future BW EDW describe the general BW EDW layout independent from concrete BI applications and BI projects. Assistant Building Blocks deal with architecture areas that are normally less prior with respect to the point in time you should decide and roll out corporate standards. Juergen Haupt /200910/ Page 62 LSA Building Blocks Reference Describes core structures & definitions Landmark Building Blocks Layer Domains Data Model & Data Integration Assistant Building Blocks Data Quality Landscape ETL Storage Ownership/ Organize Development concept . © SAP 2009 / PDEBW1.EDW & BW Layered.

Juergen Haupt /200910/ Page 63 data flow . Scalable Architecture (LSA). Structure (split/ collect) the data within the Layer to guarantee Layer and advanced services – define Data Domains Non-Architectured LSA Architectured Domain Layer © SAP 2009 / PDEBW1.LSA Landmark Building Blocks Layer & Domains There are two areas to standardize the architecture of persistent data stores: 1.EDW & BW Layered. Structure the data stores in a flow from PSA to InfoCubes with respect to their role and the offered services – define Data Layer 2.

EDW & BW Layered. Juergen Haupt /200910/ Page 64 . Transformation. If integrated SAP systems are the initial and primary BW EDW sources. May be later Data Quality Do you have considerable data quality issues? If the answer is ‘yes’. it makes sense thinking about a Data Quality tool. Near Line Storage s. Load) Do you expect extensively data from non-SAP sources? If the answer is ‘yes’. ‘Storage’). you don’t care.LSA Assistant Building Blocks I ETL (Extraction. you just don’t care. SAP Newton appliances and BW EDW) © SAP 2009 / PDEBW1. The landscape architecture comes into focus if we have to support mission critical BI or to observe legal restrictions or with other customer specific requirements if it comes to agile BI and local autonomy (federated landscapes. Scalable Architecture (LSA). Landscape often reduced to ‘Do I need a single or a multiple BW instance’ landscape. This topic has become more and more an assistant one. because of the arrival of new technologies and the transparent support by BW (BW Accelerator. it is meaningful investigating an ETL-tool like SAP Data Integrator If SAP systems are the initial and primary BW EDW sources.

g. A BI Competency Center (BICC) should be established if not existing yet. Scalable Architecture (LSA). NLS tools compress your data up to 95% (e. © SAP 2009 / PDEBW1. SAND) without destroying your Service Level Agreements (SLAs). implementing and operating a BW EDW need strong governance. This is for large scale BW EDWs not adequate (this applies also to smaller BWs) Use RDBMS compression if available and usable (e.g. Juergen Haupt /200910/ Page 65 . The BW Accelerator (BWA) must be part of the overall architecture Ownership/ Organization Designing.EDW & BW Layered. DB2 ‘deep compression’) Rarely used data should be hosted by Near Line Storage (NLS).LSA Assistant Building Blocks II Storage Traditionally all data of a BW DWH are hosted by an RDBMS.

Efficiency Dataflow © SAP 2009 / PDEBW1. scalable. Juergen Haupt /20090210/ Page 66 Dataflow Large scale BW Data Warehouses (EDWs) should follow architecture principles like we can observe constructing large buildings: standardized. earth quake save. efficient usage of materials.The Layered Scalable Architecture Goals Standardization.EDW & BW Layered. no squiggles. Transparency. nonnon-architectured architectured NonArchitectured LSA Architectured . Scalable Architecture (LSA).

EDW & BW Layered. Where.. Scalable Architecture (LSA).. in a way making your BW transparent.SAP WEBI ON standardize. standardize . robust & in the broadest sense scalable The BW Layered Scalable Architecture is all about: What. Juergen Haupt /200910/ Page 67 . When & How to Standardize © SAP 2009 / PDEBW1.With large scale BWs there is only one way to be successful: DEPLOYING standardize.

Juergen Haupt /200910/ Page 68 .V 3.EDW & BW Layered.III 3.II 3.VI The Layered.Agenda PDEBW1 3 3. Scalable Architecture (LSA) for EDW Reference LSA Overview Reference LSA & Customer LSA LSA Building Blocks – Data Layer LSA Building Blocks – Layer & Master Data LSA Building Blocks – Data Domains LSA Building Blocks – Data Model © SAP 2009 / PDEBW1.I 3. Scalable Architecture (LSA).IV 3.

EDW & BW Layered.From LSA Reference Architecture to Customer LSA LSA Building Blocks Reference LSA Implementation Reference LSA Operations Reference SAP‘s LSA Reference Customer-LSA Building Blocks Customer-LSA Implementation Standards Customer-LSA Operations Standards Customer-LSA Standards © SAP 2009 / PDEBW1. Juergen Haupt /20090210/ Page 69 . Scalable Architecture (LSA).

Juergen Haupt /20090210/ Page 70 . Scalable Architecture (LSA).BI-Projects built on Customer LSA Standards LSA Building Blocks Reference LSA Implementation Reference LSA Operations Reference SAP‘s LSA Reference Customer-LSA Building Blocks Customer-LSA Implementation Standards Customer-LSA Operations Standards Customer-LSA Standards Plan-Build-Run Apply: Individuell BI project design BI-Project-Design Based on Customer LSA © SAP 2009 / PDEBW1.EDW & BW Layered.

EDW & BW Layered.Customer LSA ‘Handbook’ Shell © SAP 2009 / PDEBW1. Juergen Haupt /200910/ Page 71 . Scalable Architecture (LSA).

EDW & BW Layered.Customer LSA ‘Handbook’ Shell © SAP 2009 / PDEBW1. Scalable Architecture (LSA). Juergen Haupt /200910/ Page 72 .

Customer LSA ‘Handbook’ Land Hessen © SAP 2009 / PDEBW1. Scalable Architecture (LSA).EDW & BW Layered. Juergen Haupt /200910/ Page 73 .

Life Cycle of The Customer LSA BW LSA: The Reference Architecture Step 1: Step 4: Design Customer LSA Update Customer LSA Customer LSA : Standards . Juergen Haupt /20090210/ Page 74 . Scalable Architecture (LSA).EDW & BW Layered.Handbook Step 2: Apply Customer LSA Step 3: Perfect Customer LSA BI-Project-Design BI-Project-Design BI Project Design © SAP 2009 / PDEBW1.

Scalable Architecture (LSA).BW EDW • • • • Consolidation of multiple BWs into a single often global BW – (consolidation) Reengineering of a BW getting on in years – (reengineering) Replacement of multiple legacy DWHs by a single BW (replacement) Allied set up of a BW with an (global )ERP roll-out (alliance) BW1 Proliferated BW Reengineering BW2 BW3 BWn Consolidation DWH1 DWH2 DWH3 DWHn New Corporate ERP(s) Replacement © SAP 2009 / PDEBW1. Juergen Haupt /200910/ Page 75 Alliance .Different Starting Points – Different Customer LSAs BW on Corporate/ Divisional Level .EDW & BW Layered.

robustness. Juergen Haupt /20090210/ Page 76 . Protection of investment manage seamless changes in OLTP world © SAP 2009 / PDEBW1. sponsorship. source systems (processes) Painful experience -> Control & Influence (outsourcing) Skills Overall governance. the preferred services differ and thus the Customer LSAs will differ Customers differ with respect to: State of integration (master data. 24x7 operations Fast (SLA) recovery without touching source system Scalability (application scalability) Fast new-build of BI-applications without accessing Sources Scalability (performance) Flexibility. error-tolerance. commitment. audits : Protection against human errors. Data flows and persistency's (DSOObjects) should be resistible against all kind of errors Report availability at 8 am (local time). Scalable Architecture (LSA).EDW & BW Layered.Reference LSA & Customer LSA As background and targets of each customer differ. In general expections with respect to: Availability.

Scalable Architecture (LSA) for EDW Reference LSA & Customer LSA Reference LSA Overview LSA Building Blocks – Data Layer LSA Building Blocks – Layer & Master Data LSA Building Blocks – Data Domains LSA Building Blocks – Data Model © SAP 2009 / PDEBW1.III 3. Juergen Haupt /200910/ Page 77 . Scalable Architecture (LSA).V 3.EDW & BW Layered.Agenda PDEBW1 3 3.IV 3.II 3.VI The Layered.I 3.

service-oriented way. Maintenance Access-rights (End-user) Administration.. User demands) Granularity History (completeness) Reusability (BI application scalability) Value of Data Layer: Recovery (robustness. Scalable Architecture (LSA). + + + + © SAP 2009 / PDEBW1. Juergen Haupt /200910/ Page 78 . availability) Highly Transparent & Flexible Quality Integration status Development... comprehensiveness (Process.LSA Landmark Building Blocks Data Layer/ Layering of Data NonArchitectured LSA Architectured Dataflow Dataflow Layer ‘Layering’ means horizontal structuring/ modeling of BW The data flows are organized in a unified. Operations Life-cycle Database-Integration . Parameters are: Coverage..EDW & BW Layered.

unification and integration transformations based on common definitions naming Corporate Memory or Corporate/ Group Data Repository … © SAP 2009 / PDEBW1.Basic Design Aspects of BW LSA EDW & Architected Data Marts Layer There are two main Layer: Architected Data Marts Architected Data Mart Layer business driven transformations business defined BWA area BW EDW Layer Single Point of Truth reusable no business driven transformations just cleansing. Juergen Haupt /200910/ Page 79 EDW .EDW & BW Layered. Scalable Architecture (LSA).

Scalable Architecture (LSA). Juergen Haupt /200910/ Page 80 .LSA Reference Layer Quick Intro User Architected Data Mart Layer Virtualization Layer Reporting Layer (Architected Data Marts) Operational Data Store Business Transformation Layer LSA Data Propagation Layer EDW layer Single Point of truth Reusable Corporate owned Granular Corporate Memory Quality & Harmonisation Layer Data Acquisition Layer Sources © SAP 2009 / PDEBW1.EDW & BW Layered.

1:1 from extraction. master the unknown. guarantee quality extractor inbox. complete.LSA Reference Layer Quick Intro reporting. long term . integrated. Scalable Architecture (LSA). operational like Virtualization Layer Reporting Layer (Architected Data Marts) Operational Data Store apply business logic digestible. unified data EDW Layer Single Point of truth Reusable Corporate owned Granular create harmonized view. analysis ready BI Applications/ Architected Data Mart Layer abstraction near real time. comprehensive.EDW & BW Layered. temporary © SAP 2009 / PDEBW1. ready to consume. Juergen Haupt /200910/ Page 81 Business Transformation Layer LSA Data Propagation Layer Corporate Memory Quality & Harmonisation Layer Data Acquisition Layer source system like service level.

Scalable Architecture (LSA). Juergen Haupt /200910/ Page 82 .EDW & BW Layered.LSA Reference Layer Basics From Acquisition to Reporting Layer Source Systems Acquisition Layer EDW Layer Harmonize/ Quality Data Mart Layer Propagation Layer Business Transformation Layer Reporting Layer DM2 VLayer Layering applies to transactional & master data! To master data however in a simpler form DM1 DM3 Corporate Memory Data flow © SAP 2009 / PDEBW1.

merges.LSA Reference Layer Acquisition Layer Source Systems Acquisition Layer EDW Layer Harmonize/ Quality Data Mart Layer Business VPropagation Reporting Transformation Layer Acquisition Layer Layer Layer Layer Per DataSource per source system DataSource should be comprehensive (offer all relevant process information) DM2 Fast inbound & outbound to targets Accept data from extraction with as less as possible overhead – no early checks.EDW & BW Layered. (1:1) Stamps the extracted records uniquely for DM1 consistent internal DWH processing and tracking (request handling) Provides abstraction of DWH from sources Provides short term history of extracted data for immediate / short term data inspection DM3 Corporate Memory Data flow © SAP 2009 / PDEBW1. transformations i. Juergen Haupt /200910/ Page 83 .e. Scalable Architecture (LSA).

granularity of DataSource/ most granular DataSource comprehensive: more fields than actually requested by Data-Marts e. ODS Yes None. Op. possible enrichment of reusable informationen single records. Scalable Architecture (LSA). Corporate Memory. transformations granularity content history main services store & deploy update housekeeping © SAP 2009 / PDEBW1. all fields of a Business Content DataSource 2-4 weeks. short term storage Decouple OLTP extraction cycle from internal BW data processing Robust & fast inbound and distribution Unmerged data (DataSource & source system specific) – copy of delta queue Fast store: PSA Element. scalable parallelism (logical/ semantical partitioning) to consumers (targets) Request by request. other data warehouses. any source Data Propagation. Regular content deletion DWH Services Implement.EDW & BW Layered. Harmonisation & Quality. no overwrite/ update of loaded records. for large DWH: Scalability thru parallelism – single thread from source system.LSA Reference Layer Data Acquisition Layer Flier Criteria potential sources potential targets reusability Characteristics Operational systems. Juergen Haupt /20090210/ Page 84 . 1:1.g. (Write Optimized DSO) Fast distribution.

LSA Reference Layer Corporate Memory Layer I Source Systems Acquisition Layer Data Mart Layer EDW Layer Business Harmonize/ Propagation Reporting Corporate Memory (CM) Transformation Quality Layer Layer ‘Think about the Corporate Memory Layer as your DWH and BI life Layer insurance‘: Mastering tasks.EDW & BW Layered. which are unforeseeable (‘the unknown’) and DM2 manageable only violating SLAs: Avoiding re-init from source(s) (Disaster-) recovery Enhancing and new build of Data Marts Change of source master data transformations DM1 Reorganizations the CM layer offers a source system like Service Level: Granular data Comprehensive data (high coverage of underlying process) Complete history of loaded data VLayer DM3 Corporate Memory Data flow © SAP 2009 / PDEBW1. Scalable Architecture (LSA). Juergen Haupt /200910/ Page 85 .

EDW & BW Layered. Scalable Architecture (LSA). which are unforeseeable (‘the unknown’) and DM2 manageable only violating SLAs The CM layer offers a source system like Service Level 1:1 from Acquisition as rule of thumb in addition harmonized data may be stored in a dedicated Corporate Memory Object after complex harmonization/ transformation DM1 If we have the same DataSource offered by multiple source-systems we should go for a single CM DSO. (Data lifecycle management must exist!) NLS is the right place for CM data DM3 Corporate Memory Data flow © SAP 2009 / PDEBW1. Juergen Haupt /200910/ Page 86 .LSA Reference Layer Corporate Memory Layer II Source Systems Acquisition Layer Data Mart Layer EDW Layer Business VHarmonize/ Propagation Reporting Corporate Memory (CM) Transformation Layer Quality Layer Layer ‘Think about the Corporate Memory Layer as your DWH and BI life Layer insurance‘: Mastering tasks.

EDW & BW Layered. Scalable Architecture (LSA). In fact we recommend to store all available describing fields of a process. We store all extracted records. This is often not realistic or just not possible.LSA Reference Layer Value of a Corporate Memory You can think about the Corporate Memory as the area. Juergen Haupt /200910/ Page 87 Quality & . which protects your investments. for bookings we have the complete history of a document) you may believe that you can repeat the loads or do again an initial load if you experience problems or new requirements. you should never rely on what business ask you to deliver the corporate memory stores more information (fields) about a process than you are actually asked for. we do not aggregate information/ bookings.g. Not realistic: – To repeat historic loads with delta loads – initial loads cause a downtime in the operative system what is often not acceptable Not possible – In the operative system the data will be archived you should always be aware that when you transform/ map original data ( Harmonization Layer) the mapping rules may change. – Thus the Corporate Memory is historical complete (e. which protects you against all kind of surprises. This is easy for SAP sources as we extract data using all fields of a delivered DataSource. Surprises in an architecture are unwelcome but they will happen. – Thus the Corporate Memory is comprehensive. © SAP 2009 / PDEBW1.

In most recovery situations the Corporate Memory will help.EDW & BW Layered. Scalable Architecture (LSA).LSA Reference Layer Value of a Corporate Memory . in general for robust and standardized recovery proceeding to keep more information than actually needed or which can actually be integrated without impacting daily operations © SAP 2009 / PDEBW1.Summary The CM is a layer of choice to guarantee utmost possible flexibility to react on the unknown. the unforeseeable if downtime of operative systems for recovery purposes (new initialization) is unacceptable (24x7) or if archiving takes place in operative system. Juergen Haupt /200910/ Page 88 .

Juergen Haupt /20090210/ Page 89 2009 Page 89 . (Harmonization Layer) Data Propagation Layer (Business Transformation Layer. normal DSO + wo DSO for full not directly in flow to applications (branched out) Request by request. Scalable Architecture (LSA). new-build and DWH reorganization manage the unknown DataSource specific Write-optimized (wo) DSO for delta loads. no overwrite/ update of loaded records Should mainly reside on NLS (Near Line Storage) DWH Services Implement Op.EDW & BW Layered.LSA Reference Layer Corporate Memory Flier Criteria potential sources potential targets reusability transformations Characteristics Data Acquisition Layer. Reporting Layer) Yes Often 1:1. granularity content history main services store & deploy update Data life cycle © SAP 2008 / PDEBW1. technical harmonization (compounding) All extracted records for delta loads (copy of delta queue) comprehensive (all fields of a DataSource) additional fields to simplify administration & access complete source system like SLA ‘Single Point of Truth’ of extracted data for delta/ changed data for full ultimate area for application recovery.

. Corporate Memory Data flow © SAP 2009 / PDEBW1. concatenation) DM1 Integration of local master data to group master data (group data model) Checking Integrity of loaded data Unification of data: adding e.g. Customers who invested in process integration and common coded/ integrated master data will harvest now.g.g. origin. address cleansing etc. detect duplicates. load time. Juergen Haupt /200910/ Page 90 .g. Scalable Architecture (LSA). -> Data Services . DM2 Technical harmonization format.LSA Reference Layer Harmonization & Quality Layer Source Systems Acquisition Layer EDW Layer Harmonize/ Quality Data Mart Layer Propagation Layer Business Transformation Layer Reporting Layer VLayer Harmonization & Quality Layer The services of this layer are data and data model integration and quality checked data e. date) integrating local master data to a single model (key compounding. DM3 Harmonizing data and guaranteeing quality of data can mean high efforts or no efforts at all... Advanced content cleansing e. length simple content harmonization (e.EDW & BW Layered.

which can be harmonized and/ or quality assured Often no persistency – Lookup-Mappings: long term Provide consistent. Op. harmonized data across multiple sources Provide quality assured data often only virtual as InfoSource or just as Transformation Rule If explicit persitency needed ‚normal‘ DSO with overwrite Lookup tables as DSOs/ InfoObjects/ Z-tables (history!) Request by request DWH Services Implement.LSA Reference Layer Harmonisation & Quality Layer Flier Criteria potential sources potential targets reusability Characteristics Acquisition Layer. (occasionally Corporate Memory) Yes Yes. CM Data Propagation Layer. granular Those fields/ InfoObjects. transformations granularity content history main services store & deploy update housekeeping © SAP 2009 / PDEBW1. Scalable Architecture (LSA). Juergen Haupt /20090210/ Page 91 . to achieve harmonized. quality-assured data single records.EDW & BW Layered.

Detailing LSA Reference Layers Propagator Layer I Source Systems Acquisition Layer EDW Layers Harmonize/ Quality Data Mart Layers Propagation Layer Business Transformation Layer Reporting Layer DM2 VLayer DM1 DM3 Propagator Layer The Propagator Layer is the interface to all business related Layers. Scalable Architecture (LSA). It stands for ‘extract once deploy many’ i. ready to consume for all business purposes (Data Marts internal & external)Data flow © SAP 2009 / PDEBW1. Juergen Haupt /200910/ Page 92 . scalability of Data Mart development Corporate Memory Propagator Layer Objects offer data that are easy to digest.e.EDW & BW Layered.

EDW & BW Layered. recovery. unified data : DM3 Propagator Layer adds: Option to enhance or merge Data to simplify Data Mart building .but no business specific transformations (no flavor for all EDW Layers) Unified Data transfer behavior out of Propagator Objects (e. unified data : Layer Easy Layer From Harmonization & Quality Layer transformations we get: Compliance of data to corporate quality and consistency standards DM2 Integration of local master data to group master data for group reporting Integration of local master data into a single BW data model (compounding) VLayer DM1 Easy to digest means standardized. report (-> Domains) to fulfill availability. robustness. administration and operations Data flow © SAP 2009 / PDEBW1. Scalable Architecture (LSA). additive delta) Suitable portions of dataCorporate Memory SLAs related to local autonomy.g.Detailing LSA Reference Layers Propagator Layer II Data Mart Layers EDW Layers Business Harmonize/ Propagation Reporting Source Acquisition Transformation Quality Layer Layer Systems to digest means standardized. Juergen Haupt /200910/ Page 93 .

Juergen Haupt /200910/ Page 94 .Split DataSource A DataSource B DataSource A Propagator DSOs: Unified BI application experience – additive delta moving via queue via generic via full via full complete full incomplete full 5. Collect DataSource A Source 2 ‘DataSource A & B’ Propagator 2.Detailing LSA Reference Layers Propagation Layer & Trimming Data ‘DataSource A +’ Propagator Add data DataSource A 1.Merge ‘DataSource A’ Propagator 1 ‘DataSource A’ Propagator 2 4.Extend DataSource A Source 1 ‘DataSource A’ Propagator 3.EDW & BW Layered. Scalable Architecture (LSA).Unify ? © SAP 2009 / PDEBW1.

Extending DataSources by looking up information. Merging different but highly related DataSources and store data in a single propagator. Note: introduced parent-child relationship must be stable otherwise realignment issues! 2. avoiding extractor enhancements) Provide sound data portions for better support of application services (availability etc) 3. Chapter on Data Model) integrating data: common semantics. common values smoothing data: common semantics. If application always or frequently request them together (e. Collecting data from the same (or similar) DataSource but from different source systems to less or a single source system independent propagator (s) ( LSA domains) 4. Juergen Haupt /200910/ Page 95 . Splitting data from a DataSources into multiple persistency’s with identical structure ( LSA domains) © SAP 2009 / PDEBW1.g. Scalable Architecture (LSA).g. HR InfoTypes. compounding) harmonize behavior of extractors (additive delta to Data Marts) trimmed to fit DataSources and Data persistency‘s to Reduce data complexity for applications 1.EDW & BW Layered. Digestible may mean harmonized data in the broadest sense (for more details s. which applications frequently ask for.Detailing LSA Reference Layer Propagation Layer & Digestible Data The core service of a Propagation Layer is to offer digestible data to applications. technically unified values (e.

EDW & BW Layered. Juergen Haupt /200910/ Page 96 Data flow . Scalable Architecture (LSA).EDW Layer Customer Example R/3 2LIS_11_VASTI EDW propagation Acquisition Corp Memory 2LIS_11_VASCL Corp Memory YGTCS_SUMMARY Corp Memory 2LIS_12_VCSCL Corp Memory © SAP 2009 / PDEBW1.

rebuilt ‚normal‘ DSO in overwrite semantical/ logical partitioned for large scale DWH/ time-zone support driven by BI application requirements (report availability) Regular delete of DSO change-log content DWH Services Impl. content. Juergen Haupt /20090210/ Page 97 . if propagator is expecting volatile requierments Merge of different DataSources to reduce complexity changes Minimum history defined by requirements of target-applications/ dependent from Corporate Memory existence ‘Single Point of Truth’ for BI applications (Business Transf. stable fields to increase (re-)useability & accessibility. Harmonization Layer. No application-specific rules! single records. Scalable Architecture (LSA). granularity content history main services store & deploy update housekeeping © SAP 2009 / PDEBW1. & Reporting Layer) Provide digestible (additive delta. Op. Reporting Layer Yes Additional. performance) data for BI applications application recovery. granularity defined by DataSource business-key DataSource specific as comprehensive as possible.EDW & BW Layered.LSA Reference Layer Data Propagation Layer Flier Criteria potential sources potential targets reusability transformations Characteristics Data Acquisition Layer. Corporate Memory Business Transformation Layer.

Scalable Architecture (LSA).EDW & BW Layered.LSA Reference Layers Data Mart Layers Source Systems Acquisition Layer EDW Layers Harmonize/ Quality Data Mart Layers Propagation Layer Business Transformation Layer Reporting Layer DM2 VLayer DM1 DM3 Corporate Memory Data flow © SAP 2009 / PDEBW1. Juergen Haupt /200910/ Page 98 .

Business Transformation Layer Customer Example R/3 2LIS_11_VASTI EDW propagation Acquisition Appl specific Transformation Reporting Corp Memory InfoSource InfoSource 2LIS_11_VASCL Corp Memory YGTCS_SUMMARY Corp Memory Acquisition/ Pass-Thru Corporate Mem. © SAP 2009 / PDEBW1.EDW & BW Layered. Juergen Haupt /200910/ Page 99 Data flow . Scalable Architecture (LSA). Transformation Reporting Multi-provider 2LIS_12_VCSCL Propagation Corp Memory Corporate Mem.

disaggregation.EDW & BW Layered. Rule semantical/ logical partitioned for large scale DWH/ time-zone support driven by BI application requirements (report availability) Regular delete of DSO change-log content update housekeeping Op. aggregation.LSA Reference Layer Business Transformation Layer Flier Criteria potential sources potential targets reusability transformations granularity Characteristics Data Propagation Layer Reporting Layer No Applications specific rules. © SAP 2009 / PDEBW1. lookups Defined by application needs Defined by application needs Defined by application needs area where complex merging of propagators objects will take place area where complex calculations/ enrichment (lookups) will take place guarantees reporting layer consistency (e. ‚normal‘ DSO in overwrite. Synchronization of DataSources) DWH Services content history main services store & deploy Impl. if any: often only virtual as InfoSource/Transf. Scalable Architecture (LSA).g. Juergen Haupt /20090210/ Page 100 .

EDW & BW Layered. multidimensional Analytics-/ Dimensional Layer less granular less comprehensive long life cycle multidimensional optimized performance Access Abstraction-/ Virtual Layer abstract from physics ‘virtual’ flexible ‘view’ generation protect front-end investments Planning Layer dedicated for planning direct input structures © SAP 2009 / PDEBW1. Scalable Architecture (LSA). Juergen Haupt /200910/ Page 101 .Detailing LSA Reference Layers Sub-Layers of Reporting/ Data Mart Layer Reporting/ Data Mart Layer Flexible Reporting/ Granular Reporting Layer highly granular highly comprehensive short life cycle flat.

Scalable Architecture (LSA).Detailing the Reporting Layer Customer Example Flexible Layer Reporting Granular Dimensional Layer Virtual Layer Reporting Performance Long term Abstraction Flexible Reporting Layer Data flow © SAP 2009 / PDEBW1.EDW & BW Layered. Juergen Haupt /200910/ Page 102 .

Virtual InfoProvider. DSOs) DWH Services content history main services Impl.EDW & BW Layered. disaggregation. NLS. Juergen Haupt /20090210/ Page 103 2009 Page 103 . InfoSets. Scalable Architecture (LSA). MultiProvider Queries work always on MutliProvider semantical/ logical partitioned for large scale DWH/ time-zone support driven by BI application requirements (report availability) Archiving. aggregation. InfoSet) by using logical views (MultiProvider) different services of Reporting Layer often modeled by sub-Layer Access Abstraction Layer (MultiProvider logical view) Analytic Layer (InfoCubes) Flexible Reporting (granular InfoCubes.LSA Reference Layer Reporting Layer Flier Criteria potential sources potential targets reusability transformations granularity Characteristics Data Propagation Layer. store & deploy InfoCubes. DSO-Objects. DSO) and virtual Objects( Virtual InfoProvider. Compression update Op. lookups Defined by application needs Defined by application needs Defined by application needs reporting & analysis save investments in front-end area: separation of physical Objects (InfoCube. Business Transformation Layer No Applications specific rules. housekeeping © SAP 2008 / PDEBW1.

LSA Reference Layer – Operational Data Store (ODS)
User
Architected Data Mart Layer Virtualization Layer Reporting Layer (Architected Data Marts) Operational Data Store

Business Transformation Layer

LSA

Data Propagation Layer EDW layer Single Point of truth Reusable Corporate owned Granular Corporate Memory Quality & Harmonisation Layer Data Acquisition Layer

Sources
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 104

Retail High Volume SAP BW ODS – POS Data Management
Sales Analyses of POS Data
BI-Reporting Templates
Store Contr.
Loss Prevent

Promo Analysis

SAP BW

mySAP Landscape

PIPE
POS Inbound Processing Engine

Outbound Interfaces

3rd Party Landscape

Upload of POS Data to SAP BW via POS Inbound Processing Engine
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 105

Overview POS Inbound Processing Engine (PIPE)
Reporting-Templates

SAP BW IDocs

Sales Audit
• Monitor • Analyses • Editor

Exchange Infrastructure
R/3 POS Inbound

mySAP Landscape

Card Payment

BAPI Billing

Transaction database
Standards
Sales Transactions VMI XML

Forecast & Replenisment Inventory Management

PIPE

Access Module

Custom-Built Interface

3rd Party Landscape

SAP BW

SAP BW Masterdata

© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 106

Scalable Architecture (LSA).EDW & BW Layered. Juergen Haupt /200910/ Page 107 .Example for Dedicated SAP BW Operational Data Stores Central SAP for Retail Central SAP BW Master Data POS Transaction Data Americas SAP BW PIPE-ODS APA SAP BW PIPE-ODS EMEA SAP BW PIPE-ODS UK SAP BW PIPE-ODS © SAP 2009 / PDEBW1.

Facilitates replication of DSO/VirtualProvider data to SAP NetWeaver BW Accelerator by switching off database persistency of the InfoCube HybridProvider latest data mass data BWA optionally DSO (RDA) DTP Replicate InfoCube OLTP © SAP 2009 / PDEBW1. Scalable Architecture (LSA). Juergen Haupt /200910/ Page 108 .Hybrid Provider Real-Time Business Intelligence Implement operational reporting scenarios leveraging new modeling objects HybridProvider Combines mass data with latest delta information at query runtime Consists of a DataStore object and a InfoCube with automatically generated data flow in between DSO object can be connected to a realtime data acquisition DataSource/DTP If the DataSource can provide appropriate delta information in direct access mode a VirtualProvider can be used instead of the DSO.EDW & BW Layered.

g. Juergen Haupt /20090210/ Page 109 .g.LSA Reference Layer Operational Data Store Layer Flier Criteria potential sources potential targets reusability transformations Characteristics Data Acquisition Layer application specific (Reporting Layer). external (e. Scalable Architecture (LSA). Pipe) Limited simple extracted records granularity application specific application specific near real time reporting mass data management (e. POS) DWH Services granularity content history main services store & deploy Implement normal DSO (Hybrid Provider) mass data Write-optimized (wo) DSO No PSA Pipe (point of sale inbound processing engine) update RDA. request by request Op.EDW & BW Layered. © SAP 2009 / PDEBW1.

Juergen Haupt /200910/ Page 110 . >96%) high application flexibility .g.g.. This applies as well for an overall customer LSA set up as for specific data sources within a full blown customer LSA ! What are complex conditions and challenging requirements? 24x7. time zones high volume.LSA Reference Layer Acquisition Layer and Subsequent Layer Why are there different paths thru LSA? The general answer is simple: In a BW we have to fulfill different competing SLAs (Service Level Agreements) Experience shows that a single staging approach with single persistent stores for all purposes cannot achieve this (RDBMS limits) ! This applies the more challenging the conditions are for BW. Scalable Architecture (LSA)... not split able loads small recovery window (e. 6h) out sourcing (skills?) off shoring (skills?) high operational robustness high report availability (e.EDW & BW Layered. Reporting Layer (Architected Data Marts) – Operational Data Store Shared Master Data (InfoObjects) Business Transformation Layer Data Propagation Layer CM Harmonization Layer Data Acquisition Layer LSA © SAP 2009 / PDEBW1. Note: This means on the other hand side that if we found less complex conditions the LSA Building Blocks set up can be simpler.

Based on customer situation and preferences additional ones may be introduced even later on (Customer LSA). As with all LSA areas customer adaptation of Layers follows the 80:20 rule i.e. Whether and when persistent InfoProviders exist are given in the LSA Implementation standards The Customer LSA Layer definitions apply throughout the BW EDW – No exceptions without approval! © SAP 2009 / PDEBW1.EDW & BW Layered.LSA Layers @Customer Summary The LSA describes services that are relevant at large scale BW implementations. Scalable Architecture (LSA). The LSA suggested Layers are a basic offering. The services are grouped by Layers. concentrate on services first. Juergen Haupt /200910/ Page 111 . which offer the most benefit for the majority of data Not all Layers have necessarily own persistent InfoProviders for all data flows.

EDW & BW Layered. Juergen Haupt /20090210/ Page 112 .Handbook Step 2: Apply Customer LSA Step 3: Perfect Customer LSA BI-Projekt-Design BI-Projekt-Design Customer BI Projects (Plan-Build-Run) © SAP 2009 / PDEBW1. Scalable Architecture (LSA).The Layered Scalable Architecture Customer Adaptation & Utilization SAP LSA: The Reference Architecture Step 1: Step 4: Design Customer LSA Update Customer LSA Customer LSA : Standards .

EDW & BW Layered. Juergen Haupt /200910/ Page 113 End-user Reporting & Analytics Scorecards Group Access Abstraction Layer Reporting Layer (Architected Data Marts) Project-defined Reporting & Analytics Data Propagation Flexible Reporting Operational Data Store LSA . Scalable Architecture (LSA).LSA Building Blocks Reference LSA & Customer LSA – Example I Reference LSA Other Sources Customer LSA Operational Data Store Quality Transformations Near real time Reporting Business Transformation Data Propagation Layer Quality & Harmonisation Layer SAP Sources Corpo rate Memo ry Data Acquisition Business Transformation Layer Business-Unit Spanning Reporting & Analytics Data Acquisition Layer Corporate Memory Subsidiary Reporting & Analytics Enterprise Data Warehouse BI Application Area Data flow From Reference LSA to Customer LSA: Not all Layer Additional Layer Different visualization and branding of Layer © SAP 2009 / PDEBW1.

Juergen Haupt /200910/ Page 114 .EDW & BW Layered.LSA Building Blocks Reference LSA & Customer LSA – Example I Reference LSA Acquisition Layer Customer LSA Corporate Memory Layer Propagation Layer Business Transformation Layer Apply business-logic Flexible Layer Reportin g Granular Dimensional Layer Reporting Performance Long term Virtual Layer Reporting Layer (Architected Data Marts) Business Transformation Layer Data Propagation Layer Quality & Harmonisation Layer Corpo rate Memo ry Operational Data Store 1:1 Unlink Unflavored Integrated Granular Ready to consume LSA YVAPP1nn YFAPPnnn (YADSSnnn) YPDSSnnn YDAPPnnn YBAPPnnn Data Acquisition Layer Complete Comprehensive Most granular YVAPP2nn Abstraction Flexible YCDSSnnn lookup Dat aflow From Reference LSA to Customer LSA: Detailing © SAP 2009 / PDEBW1. Scalable Architecture (LSA).

From Reference LSA to Customer LSA: Detailing Not all Layer Additional Layer Different visualization and branding of Layer Individual domains © SAP 2009 / PDEBW1.EDW & BW Layered. Juergen Haupt /200910/ Page 115 .LSA Building Blocks Reference LSA & Customer LSA – Example II Reference LSA Reporting Layer (Architected Data Marts) Business Transformation Layer Data Propagation Layer Quality & Harmonisation Layer Corpor ate Memor y Customer LSA EDW Acquisition propagation Appl specific Transformation R/3 2LIS_11_VASTI Reporting Operational Data Store Corp M emory LSA 2LIS_11_VASCL Corp M emory YGTCS_SUMMARY Data Acquisition Layer Corp M emory 2LIS_12_VCSCL Data flow Corp M emory Corporate Mem. Scalable Architecture (LSA).

Juergen Haupt /200910/ Page 116 . Scalable Architecture (LSA).V 3.I 3. Scalable Architecture (LSA) for EDW Reference LSA & Customer LSA Reference LSA Overview LSA Building Blocks – Data Layer LSA Building Blocks – Layer & Master Data LSA Building Blocks – Data Domains LSA Building Blocks – Data Model © SAP 2009 / PDEBW1.Agenda PDEBW1 3 3.VI The Layered.IV 3.EDW & BW Layered.III 3.II 3.

Different scenario or market reporting readiness requires different points in time of master data readiness Recovery of InfoCubes or new ones often need master data history 1. 3. which are used with extractor enhancements to catch the changes of an enhancement ( merging DataSources in Propagator Layer) 2. 2. With change run (aggregate maintenance) Scenario reporting readiness depends on master data load time Transaction data can only be processed into target subsequent to master data loads 2.Situation of Master Data With master data we have similar challenges as with transaction data and additional ones LSA topics 1. BIA topics 1. High volume master data tables have a load impact 1. Juergen Haupt /200910/ Page 117 . 2.EDW & BW Layered. 3. With full loads for InfoObjects (even if delta-load possible). Scalable Architecture (LSA). Change Run High volume master data tables (shared dimensions) have a negative impact on query performance as with large fact tables Limitations of external hierarchies on high volume master data © SAP 2009 / PDEBW1.

Lookups Transactional loads © SAP 2009 / PDEBW1. Juergen Haupt /200910/ Page 118 .Y tables) serve for both purposes.S.X. adding information Being the shared dimensions of InfoCubes for reporting (MultiProvider Today InfoObject hosted master data (P.EDW & BW Layered.LSA & Master Data Situation Master data have two main tasks to fulfill: – – Being target of lookups during transactional data load (referential integrity.Q. Shared Dimension (Navigational Attributes) InfoObject Master data Integrity. Scalable Architecture (LSA).

This is unacceptable for large scale BWs Master data should be collected/ prioritized and loaded across the Data Marts thus become BW managed © SAP 2009 / PDEBW1. 0CUST_SALES + Change Run 1:00 1:15 Transactional loads Master data Load e.Issues with Data Mart Level Managed Master Data Data Mart level managed master data: Process Chain Data Mart B First Step: BW level managed master data: Master data Load e. uncontrolled master data processing.EDW & BW Layered.g.g. Juergen Haupt /200910/ Page 119 .g. Scalable Architecture (LSA). 0CUST_SALES + Change Run Transactional loads Process Chain Data Mart B Process Chain Master Data Transactional loads Process Chain Data Mart A Transactional loads 2:30 Process Chain Data Mart A Master data Load e. 0CUST_SALES + Change Run 1:00 1:15 3:00 With non-architected BWs the master data loads are often managed on BI application/ Data Mart level what leads to redundant.

master data with full loads need additional ‘assistant DSO’ to determine delta © SAP 2009 / PDEBW1.LSA Implementation Layering Master Data Shared Dimension x.p.EDW & BW Layered.Storing master data history (introducing ‘active from’ ‘active-to’ time-slice in Propagator Master Data DSO) Reporting Layer (Architected Data Marts) Integrity.q. Juergen Haupt /200910/ Page 120 . Scalable Architecture (LSA).y.Separation of staging and reporting tasks InfoObject tables for reporting Propagator Master Data DSOs for staging 2. Lookups LSA Propagation/ CM Layer Master Data DSO Quality & Harmonisation Layer Data Acquisition Layer Picture shows master data with delta load.s tables (Navigational Attributes) InfoObject tables Large scale BWs should layer the master data: 1.

EDW & BW Layered. Juergen Haupt /200910/ Page 121 .LSA Implementation Decoupling EDW & Data Mart Layer Loads LSA managed master data loads EDW Layers Data Mart Layers Process Chains EDW Master Data Process Chains EDW Transactional Data Propagator/ Corporate Mem multiple loads per day Process Chains Master Data InfoObject Change Run Process Chain Data Mart B Data Mart Layer B Trans Layer Process Chain Data Mart A Data Mart Layer B Trans Layer loads by business requirements Propagator for 0CUST_SALES InfoObject Update EDW transactional & master data loads are decoupled from Data Mart & InfoObject loads It still remains challenging at what point in time to schedule master data loads in a global system © SAP 2009 / PDEBW1. Scalable Architecture (LSA).

EDW & BW Layered. This will increase the robustness of global BW EDWs considerably © SAP 2009 / PDEBW1.LSA Implementation Real Time Data Acquisition (RDA) for Master Data LSA managed master data loads (RDA) EDW Layers Process Chains EDW Transactional Data multiple loads Propagator/ Corporate Mem per day Process Chains Master Data InfoObject Change Run Data Mart Layers Process Chain Data Mart B Data Mart Layer B Trans Layer Process Chain Data Mart A Data Mart Layer B Trans Layer loads by business requirements RDA EDW Master Data Propagator for 0CUST_SALES continuous loads InfoObject Update With NW BW 7. Juergen Haupt /200910/ Page 122 .30 there will be Real Time Data Acquisition (RDA) for Master Data . Scalable Architecture (LSA).

Scalable Architecture (LSA).LSA and Master Data Keep master data history InfoObject master data keep only the actual version of master data: InfoObject Master Data Table Customer A CustomerGroup Y after 2nd Load Customer A A CustomerGroup X Y 1st Load: 20020101 2nd Load: 20020401 Master Data from Source or Master Data Server © SAP 2009 / PDEBW1.EDW & BW Layered. Juergen Haupt /200910/ Page 123 .

Simple Customer Master CM DS-Object customer ActiveFrom mother-comp AAA AAA BBB CCC 19980101 19981101 19980101 19980101 X Y Y X BBB CCC Y Y X II.LSA and Complete History of Master Data Corporate Memory DS-Objects store all versions of master data InfoObject Customer Master customer mother-comp AAA I.EDW & BW Layered. Juergen Haupt /200910/ Page 124 Y load from 19981101 . Scalable Architecture (LSA). Time slice Customer Master CM DS-Object 19981031 19980101 99991231 19981101 99991231 19980101 99991231 19980101 X Y Y X customer ActiveTo ActiveFrom mother-comp AAA AAA BBB CCC customer mother-comp AAA BBB CCC X Y X load from 19980101 customer mother-comp AAA © SAP 2009 / PDEBW1.

EDW & BW Layered. Juergen Haupt /200910/ Page 125 . 2. Scalable Architecture (LSA). Complete history of master data changes in a Propagator/ Corporate Memory* via time-sliced history (easy accessable in loockup) Flexible staging Propagator/ CM normal DSO with active-from. activeto is part of key Delta loads direct into Propagator/ CM DSO Full loads via intermediate normal DSO (special type of Pass Thru) for delta determination * whether you make it a Propagator or a CM member is a matter of taste © SAP 2009 / PDEBW1. Slim Staging: direct load into InfoObject (like in normal BW) for all Master Data not addressed in 2 or 3. active-to (via function module set) slice.Alternatives Managing Master Data in LSA 1. Complete history of master data changes in a Propagator/ Corporate Memory* via time-stamping records – (restricted usability) Flexible staging Delta loads direct into CM DSO Full loads via intermediate normal DSO (special type of Pass Thru) for delta determination Same handling for transactional full loads 3.

SAP BW EDW Functions and Modeling Determine Time slice of Master Data I InfoObject Characteristic Attribute1 Attr2 4711 X->Y A 4712 X B Corporate Memory ‘normal DSO’ Characteristic 4711 4711 4712 Active-to Active-from Origin Attribute1 Attr2 S1 Y A 99991231 20030501 20030430 20030401 S1 X A 99991231 20030401 S1 X B • Read DSO-Object with 4711 and 99991231 • Update 99991231 record with new values and Active-fr • Use Return table to create new record Transformation Rule: ‘Pass Thru’ ‘normal DSO’ Characteristic Origin Attribute1 Attr2 S1 X->Y A 4711 4712 S1 X B Characteristic Attribute1 Attr2 4711 Y A 4712 X B Load (here full) © SAP 2009 / PDEBW1. Juergen Haupt /200910/ Page 126 .EDW & BW Layered. Scalable Architecture (LSA).

EDW & BW Layered. © SAP 2009 / PDEBW1. Scalable Architecture (LSA). Juergen Haupt /200910/ Page 127 .LSA and Master Data Completeness of Master Data History At least for the ‘big entities’ all historical master data versions should be stored in the SAP BW Data Warehouse Layer using flexible master data staging. For less important entities it is sufficient to keep the actual versions of master data in the InfoObject master data table thus using direct staging.

Scalable Architecture (LSA).EDW & BW Layered.Agenda PDEBW1 3 3.I 3.II 3. Scalable Architecture (LSA) for EDW Reference LSA & Customer LSA Reference LSA Overview LSA Building Blocks – Data Layer LSA Building Blocks – Layer & Master Data LSA Building Blocks – Data Domains LSA Building Blocks – Data Model © SAP 2009 / PDEBW1.III 3.VI The Layered.IV 3. Juergen Haupt /200910/ Page 128 .V 3.

.. Nevertheless an incremental approach is challenging as we have to guarantee consistency & scalability during the stony path of roll-out: Consistency – Between the various Data Marts (FI.) – consistent development of Data Marts – Sometimes between different SAP BW systems – consistent deployment of Data Marts Scalability © SAP 2009 / PDEBW1. CO.. LO.. Scalable Architecture (LSA). CRM. Juergen Haupt /200910/ Page 129 .Incremental EDW Reasons & Challenges Incremental set up is targeting to: Align actual business requirements & BI/ EDW excellence Reduce complexity Step by step process & organizational excellence .EDW & BW Layered.

Incremental in terms of organizational coverage – organizational roll-out Organizational Coverage & EDW © SAP 2009 / PDEBW1...layers ..EDW Life Cycle Dimensions Incremental Implementation & Scalability An EDW implementation is always an incremental one. never a big-bang 1. EDW .... Incremental in terms of functional coverage – BI application/ Data Mart roll-out BI Application Coverage & EDW n Procure 2 Pay COPA Generating Demand Needs Scalability that is addressed by (EDW)..... 2.....EDW & BW Layered... Juergen Haupt /200910/ Page 130 Demand Planning . Org Unit C Org Unit B Org Unit N Org Unit A . Scalable Architecture (LSA). EDW n Needs Scalability that is addressed by domains (& data model) .

Juergen Haupt /200910/ Page 131 How to support: a world wide continuous roll-out BW-wide load scalability 24x7 operations & administration different reliability of sources local autonomy overall local & group SLAs .. ? ..EDW & BW Layered. Scalable Architecture (LSA).LSA Reference Layers Highly Simplified Layering Does Not Resolve All Challenges Source Systems Acquisition Layer EDW Layers Harmonize/ Quality Data Mart Layers Propagation Layer Business Transformation Layer Reporting Layer VLayer Corporate Memory Data flow © SAP 2009 / PDEBW1.

disjunct structuring of transactional data using stable criterion. Scalable Architecture (LSA).EDW & BW Layered.LSA Landmark Building Blocks Data Domains NonArchitectured LSA Architectured Domain Dataflow Dataflow Layer Domains means structuring/ modeling of data within the layers: Transparent. Juergen Haupt /200910/ Page 132 . time-zones +Development. Maintenance Scalability / performance/ low latency +Administration. Target is the support of: Independency/ autonomy of organizations Value of Data Domains: + Transparency & Flexibility 24x7. Operations (parallel vers sequential) + Scalability & Robustness Challenging recovery-window +Application +Load/ Query Performance Embedding into RDBMS +Database-Integration Implementation & Operational robustness © SAP 2009 / PDEBW1.

Strategic BW Partitioning Accross the Layers – For All Flows I Source Systems Acquisition Layer EDW Layers Harmonize/ Quality Data Mart Layers Propagation Layer Business Transformation Layer Belongs to Domain C 2LIS_11_VAITM Reporting Layer VLayer © SAP 2009 / PDEBW1. Juergen Haupt /200910/ Page 133 Single ERP Belongs to Domain B DM1 Belongs to Domain A Domains apply to transactional data Normally not to master data! Corporate Memory Data flow .LSA Domain Concept .EDW & BW Layered. Scalable Architecture (LSA).

Juergen Haupt /200910/ Page 134 Multiple ERP Belongs to Domain B DM1 2LIS_11_VAITM Corporate Memory Data flow Belongs to Domain A Domains apply to transactional data Normally not to master data! .Strategic BW Partitioning Accross the Layers – For All Flows II Source Systems Acquisition Layer EDW Layers Harmonize/ Quality Data Mart Layers Propagation Layer Business Transformation Layer Belongs to Domain C Reporting Layer VLayer © SAP 2009 / PDEBW1. Scalable Architecture (LSA).LSA Domain Concept .EDW & BW Layered.

Deploy Many Domain US Domain AMS Domain APJ Supply Chain Supply Chain Supply Chain Delivery Info Delivery Info Data Marts EDW Delivery Delivery Propagators Delivery Order Order Order DataSources 1:n Filling Domains with respect to domain criteria © SAP 2009 / PDEBW1. Juergen Haupt /200910/ Page 135 Delivery Info Order Info Order Info Order Info .LSA Building Blocks & Extract Once .EDW & BW Layered. Scalable Architecture (LSA).

there are some nice things) we introduce Data Domains in a BW EDW that divide the transactional data but use identical meta data & master data. multiple DWH instance world (yes. X North America BW Europe BW XX BW BW X Japan ERP ERP ERP ERP Domain Americas Domain Europe Domain Asia Pacific Asia Pacific BW EDW South America ERP X X BW BW ERP Using Domains in a BW EDW stands for manageability & flexibility Domains allow SLAs in a BW EDW like in a distributed BW world © SAP 2009 / PDEBW1.EDW & BW Layered.LSA Data Domains BW Landscape Consolidation (Consolidation BW) A BW EDW replaces a bunch of existing BWs and/ or legacy DWHs (BI Consolidation) spread across the organization To enable comparable services like we had in a distributed. Scalable Architecture (LSA). Juergen Haupt /200910/ Page 136 .

LSA Data Domains BW in Parallel to ERP Rollout (Primary BW) A single BW EDW shall offer standard reporting & analytics for all organizational units in a global ERP rollout. multiple BW instance world we introduce Data Domains in a BW EDW that divide the transactional data but use identical meta data & master data. Scalable Architecture (LSA). Europe North America Japan AMS US EMEA Germany APA Asia Pacific BW EDW South America Global ERP Using Domains in a BW EDW stands for manageability & flexibility Domains allow SLAs in a BW EDW like in a distributed BW world © SAP 2009 / PDEBW1. Juergen Haupt /200910/ Page 137 .EDW & BW Layered. To enable comparable services like we had in a distributed.

controlled remote ERP 1 remote ERP 2 less stable. Juergen Haupt /200910/ Page 138 .LSA Data Domains Divide Data by Sources-‘Quality’ (Integration BW) Main Domain Remote Domain BW EDW main ERP stable.EDW & BW Layered. Scalable Architecture (LSA). no control Using Domains in a BW EDW stands for manageability & flexibility Domains allow SLAs in a BW EDW like in a distributed BW world © SAP 2009 / PDEBW1.

EDW & BW Layered. Juergen Haupt /200910/ Page 139 . Scalable Architecture (LSA).1st Summary: Domains Targets to Keep Large BW EDWs Manageable & Scalable Domain West Domain Central Domain East Domain West Domain Central Domain Central Domain West Domain East North America North America Domain East Europe BW EDW South America BW EDW South America Domain EU BW EDW Company is operating In North America Domain South AMS Company is expanding to South America Domain South AMS Company is expanding to Europe Using Domains in a BW EDW stands for manageability & flexibility Domains allow SLAs in a BW EDW like in a distributed BW world © SAP 2009 / PDEBW1.

organizational autonomy.LSA Data Domains Background Evolutionary BW EDW An BW EDW that has to provide ‘local’ BI services faces by far more data by far more and more complete BI applications – finally the entire value-chain more source systems higher query workload than any other DWH in the organization before! But with service expectations necessary to support daily business.. like: High performance. high availability.. Scalable Architecture (LSA). low latency. Juergen Haupt /200910/ Page 140 . In addition BW EDWs on a global scale have to observe time-zones and 24x7 operations These challenges cannot be answered by just layering data: The LSA answer is the Data Domain concept: LSA Data Domains divide (transactional) data in the BW Layer in a disjunctive as stable as possible manner enabling scalable EDW services for most BI needs across the business © SAP 2009 / PDEBW1. robustness... fast recovery ..EDW & BW Layered.

Juergen Haupt /200910/ Page 141 . Scalable Architecture (LSA).LSA Building Blocks Reference LSA & Customer LSA – Example Reference LSA Reporting Layer (Architected Data Marts) Business Transformation Layer Data Propagation Layer Quality & Harmonisation Layer Corpor ate Memor y Control tables Customer LSA Acquisition/ Corporate Propagation PSA Layer Memory Layer Layer Business Transformation Layer Flexible Layer Dimensional Layer Virtual Layer Operational Data Store Asia YPDSS1AX YBAPP1AX YFAPP1AX YDAPP1AX Europe YPDSS1DX YBAPP1DX YFAPP1DX YDAPP1DX YVAPP1XX LSA Americas (YADSS100) YFAPP1GX YPDSS1GX YDAPP1GX YBAPP1GX YFAPP1WX Germany US YPDSS1WX YDAPP1WX YBAPP1WX YFAPP1UX YVAPP1XX Data Acquisition Layer YPDSS1UX YCDSS100 YBAPP1UX YDAPP1UX Lookup-tables From Reference LSA to Customer LSA: Individual Domains © SAP 2009 / PDEBW1.EDW & BW Layered.

Juergen Haupt /20090210/ Page 142 . Scalable Architecture (LSA). Scalable Structuring of BW: LSA Domains & Layers LSA Domains distribute the transactional data for the entire BW EDW in a disjunctive manner. Access Abstraction Layer (MultiProvider) Reporting Layer (Architected Data Marts) Distribution actively designed: Domains Access Abstraction Layer (MultiProvider) Reporting Layer (Architected Data Marts) LSA LSA Business Transformation Layer Business Transformation Layer Data Propagation Layer Quality & Harmonisation Layer Corporat e Memory Data Propagation Layer Quality & Harmonisation Layer Corporat e Memory Data Acquisition Layer Data Acquisition Layer Single Source System (Layer) Distribution inherited Multiple Source Systems (Layer) The LSA addresses an evolutionary EDW approach introducing Data Domains enabling ‘local’ BI services.EDW & BW Layered. 24x7 operations and administration without neglecting the broader EDW picture. © SAP 2009 / PDEBW1. The meta data of domains are identical.Transparent.

LSA Data Domains and Layer Properties of LSA Domains As a rule of thumb: Domains organize data by a general criteria driven by Reporting. BI and Manageability requirements i. Juergen Haupt /200910/ Page 143 . domains often differ from operational data organization Domains are from transactional data point of view disjunct – harmonized master data are shared Domains use identical meta data definitions Cross views are achieved via MultiProviders or dedicated persistent InfoProviders Domains should be stable Domains should be consistent across all Layer (across all flows) Domains should be general for all Layer.e. exceptions: The Acquisition Layer inherits the structuring from source systems/ extractors Domains do not apply on the Corporate Memory © SAP 2009 / PDEBW1. Scalable Architecture (LSA).EDW & BW Layered.

EDW & BW Layered. China and US Independency (special service level agreement) for certain countries Important countries/ markets get an own domain Robustness: take Quality/ stability of different sources into consideration (potential) instable data offerings from sources may lead to an own domain Each customer may have additional criteria.g. West. EMEA..g. Juergen Haupt /200910/ Page 144 . Scalable Architecture (LSA). APA. normally it is a mixture of multiple items © SAP 2009 / PDEBW1.. Americas time zone aspects may lead to e. Middle. Mid-Asia.Criteria Defining LSA Domains Golden rule defining domains: as many domains as necessary – as less domains as possible Defining Domains – rules of thumb Often a geography driven domain concept works well Often one basic domain per continent makes sense ( rough time zone handling) e. for continental BW EDW a starting point could be East.g. 3 Asian domains East-Asia.g. Expected Volume (realistic volume estimates are a key input defining domains ) Large APA volume contribution & large (for your business important) countries may get an own domain e. West-Asia E.

LSA Domain Criteria & Business View Domains should be as stable as possible (TCO) on the one hand side but on the other hand side from the overall service perspective it makes sense defining domains with respect the business view on data. but this is not necessarily stable as it refers to organization. We may find a business view of data that is not reflected by organizational units of the sources (e. sources know Company-Code. planning area etc but business view is ‘market’ what is not reflected by the sources) Domains offer the opportunity resolving this using domains in an EDW for better BI services (reporting) and more transparency (domains) Note: As domains are a mixture of application & technical dimensions we should not overload domains with a purely business view -> observe Golden rule © SAP 2009 / PDEBW1.EDW & BW Layered.g. Scalable Architecture (LSA). Juergen Haupt /200910/ Page 145 .

Domains and Source Systems Defining Adequate Domains
No domains Propagation adequate domains

?
Acquisition DataSource from No domains Propagation adequate domains

single corporate source-system

Too many domains

?
...... DataSource from

......

?
Acquisition ...... ......

multiple source-systems

© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 146

LSA Domains Definitions & EDW Roll-Out Managing Volume Uncertainty
Especially at the beginning of an EDW roll-out there is often a large degree of uncertainty with respects to volume expected in the future. But there is normally at least a classification of organizational units that drive the domains possible in terms of t-shirt sizes (S,M,L) S & M organizational units reside in standard domains (ASIA; AMERICAS; EMEA) L organizations/ markets get an own domain Possible challenges during roll-out: Domains will become too big e.g. EMEA -> Create an additional Domain EMEA2 Large Domains -> think about additional splitting criteria (note: time-partitioning helps normally little with load issues) This leads to initial domain concept for your EDW e.g. Asia, EMEA, AMERICAS, CHINA, US Note: Existing Domains must continuously be observed with respect to SLA compliance
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 147

LSA Domains Domain DWH Characteristic
BW EDW All data

Domain ‘B’
EMEA assign

Domain ‘A’
APA

Domain ‘C’
AMS

In BW EDW all loaded records will be qualified/ ‘stamped’ assigning a stable ‘domain- driving’ characteristic like market/ country to organizational criteria like in this example 0COMPCODE coming from source system.
This allows easy redistribution of a domain data if service levels cannot be kept

Country/ Market
E.g. ‘F’ assign Organizational-unit 0COMPCODE = FR01

0COMPCODE = FR02

E.g. ‘UK’

,,,,,,,,
© SAP 2009 / PDEBW1- EDW & BW Layered, Scalable Architecture (LSA), Juergen Haupt /200910/ Page 148

EDW & BW Layered. ‘UK’ Organizational-unit 0COMPCODE = FR01 .LSA Domains Domain DWH Characteristic – New Domain France ‘F’ gets an own domain All records in Domain ‘B’ with country ‘F’ are moved to the new domain Domain ‘B’ EMEA – ‘F’ BW EDW All data Domain ‘A’ APA Domain ‘C’ (AMS) Domain ‘F’ Country ‘F’ E..g. Scalable Architecture (LSA). Juergen Haupt /200910/ Page 149 ..... 0COMPCODE = FR02 © SAP 2009 / PDEBW1...

Juergen Haupt /200910/ Page 150 .LSA & Importance of Volume Estimates © SAP 2009 / PDEBW1. Scalable Architecture (LSA).EDW & BW Layered.

You're Reading a Free Preview

Download
scribd
/*********** DO NOT ALTER ANYTHING BELOW THIS LINE ! ************/ var s_code=s.t();if(s_code)document.write(s_code)//-->