SAP NetWeaver 7.

0:
Enterprise Data Warehousing
Overview

Product Management SAP NetWeaver BI

November 2007
 Agenda

1. Overview
2. Data Modeling
2.1. Data Warehousing Workbench
2.2. DataSource
2.3. Transformation
2.4. DataStore Objects
2.5. Modeling Data Marts
3. Data Flow Design
3.1. Data Transfer Process
3.2. Process Chains
4. Administration & Monitoring
4.1. Administration Cockpit
4.2. Information Lifecycle Management
5. Maintaining Data Security
6. Performance Management
5.1. SAP NetWeaver BI Accelerator
5.2. Other Performance Techniques

© SAP 2007 / Page 2

 Enterprise Data Warehousing

Provide each organizational unit or better each role with the needed reliable,
consolidated, integrated, up-to-date, and historical information

e.g.,
at business unit level provide a:
local/subsidary view
regional view
global view

at headquarter level
across business units
provide
regional view
global view

© SAP 2007 / Page 3

 The Challenge of Enterprise Data Warehousing

EDW

With an centralised Enterprise Data Warehouse:

People will find the right information
Related information is connected
Collaboration and information exchange between people does work

© SAP 2007 / Page 4

 Architecture SAP NetWeaver BI

SAP NetWeaver Portal Enterprise Search Knowledge Management Worklists Collaboration

Information Broadcasting

UIs can be embedded

Planning Enterprise
Composite Ad Hoc BI App Layout Report MS Excel

Services & Visual Composer Business Explorer Suite (BEx)

BAPIS

Embedded BI Web Web Application Report

BI Kit Analyzer
Analyzer Designer Designer

BI
BI Consumer
Consumer Services
Services Web Services

BI Layer ODBO/XMLA
Query Designer

Master Data

Repository
Meta Data
SAP NetWeaver
Virtual- Planning Modeler BI Accelerator
Provider

Analytic Engine Appliance

Data Marts Downstream System

Operational
Data Store Open Hub
Data Warehouse
Service Near-Line
Storage
PSA

Data Sources SAP Non-SAP SAP NetWeaver

3rd-Party BI Data
Operational Data Operational Data BI Data

© SAP 2007 / Page 5

 BI Architecture:
Enterprise BI Data Management
Enterprise Query, Reporting & Analysis

Analytic Engine
BI Accelerator

Data Flow Control / Process Chains

Calculation Caching

Monitoring / Administration
Aggregation Security
Meta Data Repository / Documents

InfoObjects / Master Data

Analysis Process Design Planning Services

Near-Line Storage

Enterprise Data Warehouse

(Architected)
Operational Data Marts Open
Data Store Hub
(volatile) Data Warehouse Layer Service
(historical)

DataSource / PSA

Source Systems
© SAP 2007 / Page 6
 Enterprise Data Warehousing - Processes

Data Modeling Data Flow Maintaining Data Administration Performance

for EDW Design Security and Monitoring Management

Data Modeling for EDW

Business (Process) Experts can define the basis for the enterprise reporting. They define data containers
(InfoProviders) and data consolidation rules. Multiple-layer EDW-architectures are supported.

Data Flow Design

The data flow (in particular DTP and InfoPackage) is defined at this level.

Maintaining Data Security

This process enables organizations to model the company‘s security rules into the software in a highly
flexible matter.

Administration and Monitoring

The BI administrator is offered a central admin and monitoring tool (NetWeaver Administrator), which
facilitates monitoring of complex landscapes. In addition, lifecycle management tasks can be initiated from
here.

Performance Management
Provides all means to accelerate query performance, in particular the SAP NetWeaver BI Accelerator.

© SAP 2007 / Page 7

 Agenda

1. Overview
2. Data Modeling
2.1. Data Warehousing Workbench
2.2. DataSource
2.3. Transformation
2.4. DataStore Objects
2.5. Modeling Data Marts
3. Data Flow Design
3.1. Data Transfer Process
3.2. Process Chains
4. Administration & Monitoring
4.1. Administration Cockpit
4.2. Information Lifecycle Management
5. Maintaining Data Security
6. Performance Management
5.1. SAP NetWeaver BI Accelerator
5.2. Other Performance Techniques

© SAP 2007 / Page 8

 Overview

Data Warehousing Workbench with SAP NetWeaver 7.0

Modeling and Administration view

© SAP 2007 / Page 9

 Data Warehousing Workbench

Usability Features
Favorites
Personalization
Advanced Search
Complete data flow at a glance

© SAP 2007 / Page 10

 Agenda

1. Overview
2. Data Modeling
2.1. Data Warehousing Workbench
2.2. DataSource
2.3. Transformation
2.4. DataStore Objects
2.5. Modeling Data Marts
3. Data Flow Design
3.1. Data Transfer Process
3.2. Process Chains
4. Administration & Monitoring
4.1. Administration Cockpit
4.2. Information Lifecycle Management
5. Maintaining Data Security
6. Performance Management
5.1. SAP NetWeaver BI Accelerator
5.2. Other Performance Techniques

© SAP 2007 / Page 11

 Conceptual Layers of Data Warehousing

Information Access

(Architected)
Data Marts
Operational
Data Store
Data
Warehouse

(Persistent) Staging Area DataSources

Any Source

© SAP 2007 / Page 12

 Data Acquisition Layer – Data Sources

Sources
Support of virtually all sources

DataSource

DB UD BI Service File Web

BAPI
Connect Connect API Interface Service

ETL Tool

Multi-
Relational SAP Legacy
Dimensional File XML
Source Source Applications
Source

e.g. IBM DB2, e.g. Hyperion e.g. SAP CRM e.g. SAP e.g. ORACLE
Teradata NetWeaver PI Financials
(via proxy
framework)

© SAP 2007 / Page 13

 New BI DataSource concept with
SAP NetWeaver 7.0

Highlights
unique look and feel for all of the DataSource Types
PSA is attached to DataSource
InfoPackage writes to PSA
Data Transfer Process writes from PSA to data targets
direct/remote access is optional
preview feature is standard
automated conversions (e.g. date format detection)

© SAP 2007 / Page 14

 Source System Tree

Source sytems categories:

SAP vs. non SAP

File vs. database
Relational vs.
Multidimensional DB
ABAP vs. Java
XML vs. Text/Binary
Pull vs. Push
Realtime vs. Batch

© SAP 2007 / Page 15

 DataSource Example – One fits all approach

General Information
Descriptions
Reconciliation flag (not
functional)
Opening Balance
(inventory)
Error handling (duprecs)

© SAP 2007 / Page 16

 Data Flow Concept in SAP NetWeaver 7.0

SAP NetWeaver BI

Process Chain
InfoProvider
Data Transfer
Process
Transformation

DataSource / PSA
InfoPackage

Source System 1

Source

© SAP 2007 / Page 17

 Data Flow Concept in SAP NetWeaver 7.0
Simplified

SAP NetWeaver BI

Process Chain
(optional) InfoProvider

Data Transfer Transformation

Process

Restrictions: X
DataSource / PSA

Not optimized for

mass data transfer
Source System 1
No packaging of data

Full Mode Only Source

© SAP 2007 / Page 18

 Agenda

1. Overview
2. Data Modeling
2.1. Data Warehousing Workbench
2.2. DataSource
2.3. Transformation
2.4. DataStore Objects
2.5. Modeling Data Marts
3. Data Flow Design
3.1. Data Transfer Process
3.2. Process Chains
4. Administration & Monitoring
4.1. Administration Cockpit
4.2. Information Lifecycle Management
5. Maintaining Data Security
6. Performance Management
5.1. SAP NetWeaver BI Accelerator
5.2. Other Performance Techniques

© SAP 2007 / Page 19

 Data Flow in SAP NetWeaver 7.0 BI

SAP NetWeaver Business Intelligence Downstream Systems

InfoProvider Open Hub Destination

Transformation Transformation
DTP DTP

InfoProvider

Transformation Data
Transfer
Process
DataSource / PSA

InfoPackage InfoPackage SAP NetWeaver PI

Non-SAP SAP NetWeaver BI SAP

Any Source
© SAP 2007 / Page 20
 Transformation

Target
Universal transformation from source
to target objects

Transformation Transformation types:

Move, aggregate, constant, master data look
up, …
End Routine *
Business rules, e.g. unit + currency translation
Formula builder with rich predefined functions
library
Expert Transformation Rule 1 ABAP routines incl. regular expressions
Routine
Transformation Rule n
* SAP NetWeaver 7.0 Enhancements
Intuitive UI
Start Routine * Unit conversion
Unified transfer + update rules into all-in-one
capability
Integration of Open Hub Service
Package 1
Semantic
Package 2
Groups *
Package m
Source

* optional
© SAP 2007 / Page 21
 Transformation – Definition

Access from the Data Warehousing Workbench

New transformation
Unification of transfer and update rules
InfoSource not mandatory anymore
Former concept of update rules
Small square next to the transformation icon
Access from context menu via ‘additional functions’
Links sources and target
New source: InfoSet
Other sources: DataSource, InfoCube, DataStore object, InfoObject, InfoSource
Targets: InfoCube, DataStore object, InfoObject, InfoSource, Open Hub Destination

Transformation

Update rule

© SAP 2007 / Page 22

 Transformation – Graphical UI

Source Target
fields fields

Rules per
Note: Key figures, characteristics and date group
fields are shown on the same level
(transformation group)

© SAP 2007 / Page 23

 Transformation Rules

Transformation rule details

Information on
Rule type
Currency/
Unit Conversion
Source fields
Target Fields

© SAP 2007 / Page 24

 Enhanced Data Flow in SAP NetWeaver 7.0 BI

SAP NetWeaver Business Intelligence

InfoProvider

Transformation (optional) Transformation (optional)

InfoSource (optional) InfoSource (optional)

Data
Transfer
Transformation Process Transformation

DataSource / PSA InfoProvider

© SAP 2007 / Page 25

 Transformations – InfoSource – 1 –

InfoSource
Transformation directly links from a source InfoProvider (or DataSource) to a target
InfoProvider
An InfoSource is usually not needed
New InfoSource architecture is used (flat InfoObject-based structure)
Scenarios for (flexible) InfoSource
A flexible InfoSource is necessary in order to use currency or unit conversion
from the source DataSource Define InfoSource as an intermediate structure
You can use a flexible InfoSource as a uniform source for several targets; the
InfoSource can the be target from different sources (see next slide)
Note: for ‘direct’ InfoSources (for master data updates), there is no difference
between ‘old’ and ‘new’ InfoSource, i.e. you can define a transformation as well as
transfer rules
Pre-requisite: InfoObject is defined as InfoProvider

© SAP 2007 / Page 26

 Transformations – InfoSource – 2 –

InfoSource
Scenario: InfoSource as a uniform source for several targets and as target from
different sources
SAP NetWeaver Business Intelligence

InfoProvider
InfoProvider 1 InfoProvider 2 … InfoProvider m

Transformation
…

InfoSource

Transformation
…

DataSource 1 DataSource 2 … DataSource n

DataSource

© SAP 2007 / Page 27

 Transformation Groups – 1 –

Transformation Groups
Summarize key figures with the same characteristics assignments
All key figures of one transformation are updated based on the same characteristic
values
If other characteristic updates are necessary for particular key figures, a new
transformation is created

© SAP 2007 / Page 28

 Transformation Groups – 2 –

Transformation Groups
Use / Example
Scenario: overview on bonus-relevant sales of all employees
An employee generates a certain sales volume, which is the basis for his/her
bonus
The manager of the employee will be assigned 10% of the employee’s bonus as
manager’s bonus relevant
two transformation groups are generated (e.g. ‘employee’ and ‘manager’)

Source
Employee Sales Volume Manager
Johnson 1000 Giles Transformation Group 1
Employee Employee
Sales Volume Bonus-relevant Sales

Target Transformation Group 1

Employee Bonus-relevant Sales Manager Employee
Johnson 1000 Sales Volume*0,1 Bonus-relevant Sales
Giles 100

© SAP 2007 / Page 29

 Agenda

1. Overview
2. Data Modeling
2.1. Data Warehousing Workbench
2.2. DataSource
2.3. Transformation
2.4. DataStore Objects
2.5. Modeling Data Marts
3. Data Flow Design
3.1. Data Transfer Process
3.2. Process Chains
4. Administration & Monitoring
4.1. Administration Cockpit
4.2. Information Lifecycle Management
5. Maintaining Data Security
6. Performance Management
5.1. SAP NetWeaver BI Accelerator
5.2. Other Performance Techniques

© SAP 2007 / Page 30

 DataStore Object Types

DataStore Object types – overview

Primary Usage Structure

Activ-
EDW ODS Delta / Change Fast Access Active Change ation Integration
DataStore object type layer Layer Data Capture (no activation) Others Data Log Queue into data flow

Delta
determination
from after
Standard DataStore images via staging
Object X X on record level X X X (DTP)

Staging layer
esp. for large
sets of data
with
Write-optimized On request (generally) via staging
DataStore Object X level X unique key X (DTP)

for external via APIs,

applications Staging into
and analysis subsequent
DataStore Object for processes targets
direct update No X (APD) X possible

© SAP 2007 / Page 31

 Standard DataStore Object – 1 –

Details
Option ‘Generation of SID Values’
Improves query performance
Queries are also possible if SID
values are not generated
Option ‘Unique data records’
Only available if ‘Generation of
SID Values’ is set
Activation process is optimized
(only inserts, no sorting, no
before image)
Note: error if key already exists
For (non-reporting) scenarios, write-optimized DataStores are recommended instead of
standard DataStore objects with unique flag
Performance Improvement
Rollback
Instead of rolling back in serial and in one transaction
rollback now is in parallel and for each data package there is a single task

© SAP 2007 / Page 32

 Standard DataStore Object – 2 –

Structure
Activation queue
Used to store data to be updated in DataStore Object which has not been activated
After activation the data can be deleted
Technical key: Request SID, Package ID, Record number
Active Data Table
Same structure as
the DataStore Object
definition

Change Log
Change history for delta mechanism from the DataStore Object into other InfoProvider
Key fields:
– Request GUID, Package ID, Record number

© SAP 2007 / Page 33

 Write-optimized DataStore Object – 1 –

Scenarios
Fast EDW inbound layer (no activation needed)
For large sets of data records on detailed level (e.g. document level)
“wide” structure is possible (16 key fields, 749 data fields)
“Load of new records”:
Every record has a new key No update, only inserts
E.g. for POS data
“Load & Drop”:
Full Upload into DataStore Object
Update subsequent InfoProvider
Drop DataStore Object data
Continue with full uploads

© SAP 2007 / Page 34

 Write-optimized DataStore Object – 2 –

Details
Definition
Only active data table (key: request ID, Packet No., Record No.)
– No change log and no activation queue
– Technical key is unique
Partitioned on request ID
No SID generation
–Nevertheless: Reporting is possible (but not optimized for performance)
Fully integrated in data flow: usable as data source and data target
–Export into InfoProviders via request delta
Can be included into MultiProvider or InfoSet
Uniqueness of Data
Checkbox “Do not check Uniqueness of data”
Performance improvement during data load
– Does not create/maintain unique index on
semantic key

© SAP 2007 / Page 35

 DataStore Object For Direct Update – 1 –

Details
Definition
Only active data table
Can be used as data target within APD
cannot be used for transformation (upload) scenarios
no loading process within BI
but export into next InfoProvider is possible
Reporting is possible
Scenarios
Used for direct input of (external) transactional data
– E.g. BI table for user interaction
An API is available with a set of function modules
(some are RFC enabled)
Fed by APD processes

© SAP 2007 / Page 36

 Agenda

1. Overview
2. Data Modeling
2.1. Data Warehousing Workbench
2.2. DataSource
2.3. Transformation
2.4. DataStore Objects
2.5. Modeling Data Marts
3. Data Flow Design
3.1. Data Transfer Process
3.2. Process Chains
4. Administration & Monitoring
4.1. Administration Cockpit
4.2. Information Lifecycle Management
5. Maintaining Data Security
6. Performance Management
5.1. SAP NetWeaver BI Accelerator
5.2. Other Performance Techniques

© SAP 2007 / Page 37

 InfoCube

InfoCube
Star Schema optimized for multi-dimensional reporting

Master
Master Master
Master
Data
Data Data
Data
Surrogate Key
Dimension
Dimension Dimension
Dimension

Dimension ID

Fact Support of
FactTable
Table degenerated
dimensions

Dimension
Dimension Dimension
Dimension

Master
Master Master
Data Master
Data Data
Data
© SAP 2007 / Page 38
 InfoCube

Example:

InfoCube in Meta Data

Repository

© SAP 2007 / Page 39

 MultiProvider

MultiProvider
Logical definition without data persistency
Abstraction level for queries
Can integrate the following InfoProviders via union operator
InfoCube
InfoObject
DataStore Object
VirtualProvider
InfoSet
Aggregation Level

© SAP 2007 / Page 40

 InfoSet

InfoSet
Logical definition without data persistency
Can integrate InfoCubes, DataStore Objects and InfoObjects
join and outer join operator

InfoSet
w/o data
persistency

InfoCube DataStore Object InfoObject /

Master Data

© SAP 2007 / Page 41

 Agenda

1. Overview
2. Data Modeling
2.1. Data Warehousing Workbench
2.2. DataSource
2.3. Transformation
2.4. DataStore Objects
2.5. Modeling Data Marts
3. Data Flow Design
3.1. Data Transfer Process
3.2. Process Chains
4. Administration & Monitoring
4.1. Administration Cockpit
4.2. Information Lifecycle Management
5. Maintaining Data Security
6. Performance Management
5.1. SAP NetWeaver BI Accelerator
5.2. Other Performance Techniques

© SAP 2007 / Page 42

 Data Transfer Process: Complex Example

SAP Netweaver BI
TR
Process Chain
InfoSource Process Chain
DTP
TR DTP
TR
DataStore Object 3

DTP TR TR DTP

DataStore Object 1 DataStore Object 2

DTP DTP
TR TR

DataSource
DataSource (PSA)
(PSA) DataSource
DataSource (PSA)
(PSA)

IP IP

Source System 1 Source System 2

© SAP 2007 / Page 43
 Benefits of New Data Transfer Process

Data Transfer Process (DTP) - Data Distribution within SAP NetWeaver BI

Loading data from one layer to others except InfoSources
Separation of delta mechanism for different data targets
Enhanced filtering in dataflow
Improved transparency of staging processes across data
warehouse layers
(PSA, DWH layer, ODS layer, Architected Data Marts)
Improved performance: optimized parallelization
Enhanced error handling for DataStore object (error stack)
Enables real-time data acquisition

© SAP 2007 / Page 44

 Filter in Data Transfer Process

With filter it is possible to

load a set of data to the
data target instead of the
complete volume of data.
Different data selections
can be made via different
data transfer processes for
the same or for different
data targets.

Extraction
mode: Delta or
Full

© SAP 2007 / Page 45

 Data Transfer Process

Loading directly
into Data Target
without PSA

Prerequisite
DataSource is enabled for direct access (table ROOSOURCE)
Or SAP Basis Plug-In Release 2005.1 SP8, or SAP Plug-In Release 2004.1 4.6C Support
Package 13
Or if you want to have the correction before the above support packages are released, please
apply note 923783.

© SAP 2007 / Page 46

 Error Handling Overview

Process chain
can automate
the loading
process
DTP
DTP Scheduler
Scheduler

IP DTP
DataSource
DataSource (PSA)
(PSA)

Source System

There is no error handling

available for an InfoPackage. Error DTP
Error Stack
In case of invalid records,
data needs to be reloaded
from the source system.

Invalid records can be corrected in the error

stack and updated into the data target
© SAP 2007 / Page 47
 Error Handling Features

Error Handling
Possibility to choose in the scheduler to...
abort process when errors occur
process the correct records but do not allow reporting on them
process the correct records and allow reporting on them
Number of wrong records which lead to a wrong request
Invalid records can be written into an error stack
Keys should be defined for error stack to enable the error handling of DataStore object
Temporary data storage can be switched on/off for each substep of the loading process
Invalid records can be updated into data targets after their correction.

© SAP 2007 / Page 48

 Error Handling

Error Handling
Once errors occur, the whole
Data Package is terminated.
The request is not released
for reporting.

Valid records are updated.

After manual release of the
request, data is valid for
reporting.

Valid records are updated and

available for reporting

© SAP 2007 / Page 49

 Error Stack – 1 –

Error Stack
Stores erroneous records
Automatic checks: Existence of master data, conversion exit (restricted, e.g.
Alpha)
Customer-defined checks in transformation routines (see appendix for more
information)
Keeps the right sequence of records for consistent DataStore handling
Key of error stack defines which data should be detained from the update after the
erroneous data record
After correction, Error-DTP updates data from error stack to data target
Note: Once the request in the source object is deleted, the related data records in
error stack are automatically deleted

© SAP 2007 / Page 50

 Error Stack – 2 –

Error Stack
Key of Error Stack = Semantic Groups
Subset of the key of the target object
Max. 16 fields
Defining which data should be detained from the update after the erroneous
data record (for DataStore Object)
Semantic groups bundle records with the same semantic group key into the
same request see transformation chapter for more details (for DataStore
Object and InfoProvider)

© SAP 2007 / Page 51

 Temporary Data Storage

Temporary Data Storage

Help for tracing the erroneous records and transformations
Data records from different steps within the data transfer process can be stored
temporarily
Stores complete set of data (erroneous as well as valid records)
Scenario:
If the debugging mode is switched on
Trace the erroneous records
Trace Transformation

© SAP 2007 / Page 52

 Temporary Data Storage

Settings for Temporary Data Storage

Level of detail
Tracing the erroneous records
Tracing transformation by package
Tracing transformation by record
Deletion of temporary storage
With request status ‚green‘
If request is deleted
After X days

Switch on/off the

temporary data
storage for data
loading steps

© SAP 2007 / Page 53

 Data Transfer Process Monitor – 1 –

DTP Monitor
Integrated in InfoProvider management screen
Integrated in DTP maintenance
Additional information: duration of each step
Temporary storage access – if activated
Error Stack is displayed in DTP Monitor

Error Stack

Data display in
temporary storage

© SAP 2007 / Page 54

 DTP and Open Hub

Open Hub Destination

as
DTP DataTarget

© SAP 2007 / Page 55

 Open Hub Destinations

© SAP 2007 / Page 56

 DTP Initialization without Data Transfer

Data Transfer
Process Initialization
without data transfer

© SAP 2007 / Page 57

 DTP Monitor – Header

Monitor Data transfer

process ‚header‘

© SAP 2007 / Page 58

 DTP Monitor – Detail

New with SPS08:

Monitor Data
transfer process
‚Detail‘

© SAP 2007 / Page 59

 Agenda

1. Overview
2. Data Modeling
2.1. Data Warehousing Workbench
2.2. DataSource
2.3. Transformation
2.4. DataStore Objects
2.5. Modeling Data Marts
3. Data Flow Design
3.1. Data Transfer Process
3.2. Process Chains
4. Administration & Monitoring
4.1. Administration Cockpit
4.2. Information Lifecycle Management
5. Maintaining Data Security
6. Performance Management
5.1. SAP NetWeaver BI Accelerator
5.2. Other Performance Techniques

© SAP 2007 / Page 60

 Introduction: Typical Data Load Cycle

Load into PSA Data Load

Monitor
Drop Indices
Load into DataStore

Start

Activate
Data in
DataStore
Object

… Load into InfoCube

Data Target Roll up to BIA

Maintenance Index
© SAP 2007 / Page 61
 Process Chain Example

© SAP 2007 / Page 62

 Three Different Views in the Transaction

Planning view: Build and change process chains

Grey: unplanned processes
Green: planned prozesses
Yellow: planned but unknown processes
Red: multiple planned processes

Check view: Check for errors in design

Green: Error-free processes
Yellow: Process with warnings
Red: Process with errors

Log view: Monitoring of process chains

Grey: Not yet run
Green: Finished without error
Yellow: running
Red: broken or failed

© SAP 2007 / Page 63

 Process Chains:
Failed processes can send email

Write a message
and fill in recipient
and type. Info
saved within
process variant.

Planning view
context menu

© SAP 2007 / Page 64

 Agenda

1. Overview
2. Data Modeling
2.1. Data Warehousing Workbench
2.2. DataSource
2.3. Transformation
2.4. DataStore Objects
2.5. Modeling Data Marts
3. Data Flow Design
3.1. Data Transfer Process
3.2. Process Chains
4. Administration & Monitoring
4.1. Administration Cockpit
4.2. Information Lifecycle Management
5. Maintaining Data Security
6. Performance Management
5.1. SAP NetWeaver BI Accelerator
5.2. Other Performance Techniques

© SAP 2007 / Page 65

 BI Administration Cockpit - Motivation

Easy administration
for complex
Enterprise Data Warehouses
using the BI
Administration Cockpit

© SAP 2007 / Page 66

 BI Administration Cockpit - Scope

Support the BI administrator in …including context-specific

Status tracking Drill-down to details
Performance optimization Processing options
Strategic administration Exceptions (optional)
…in the areas of …using proven technology
Enterprise Data Warehousing BI Queries
Enterprise Query, Reporting and Analysis BI Web Applications
Business Planning and Analytical Services SAP NetWeaver Portal
…by providing a central point of entry with …to make administration
cockpits easier and faster
Real-time monitors …and thus to lower the TCO
Runtime Statistics
Cross system monitoring

© SAP 2007 / Page 67

 BI Administration Cockpit - Overview

Central access to most important BI

monitoring information

Context menu for

access to more
detailed information
or BI Transaction

Exception definition for

intuitive display of critical
monitoring data (optional)

Monitoring of
multiple BI systems
in one view

Flexible filtering of
relevant information Graphical display

© SAP 2007 / Page 68

 BI Administration Cockpit –
Architecture
SAP NetWeaver Portal Business Package „BI
Administration 1.0“ from
Portal the Portal Content
Pages iViews iViews iViews
Portfolio. BI Administration
Cockpit can run in a
central or in a local portal.
BI Web Applications Technical Content for SAP
Queries NetWeaver BI (software
component BI_CONT,
InfoProviders / MultiProviders release 7.0.2), The
DataSources Technical Content is
entirely based on SAP BW
3.x functionality not
Data Load Data Load Query Runtime requiring BI_JAVA.
Statistics Status Statistics
SAP NetWeaver 7.0 BI
technology
Data BI BI
(software component
Warehousing Platform Suite
SAP_BW)
SAP NetWeaver BI
© SAP 2007 / Page 69
 BI Administration Cockpit –
Main building blocks
SAP NetWeaver Portal
BI Administration Cockpit (Business Package)
Portal ed
SingleiViews
Pages iViewsandiViews
point of entry integration with other (non BI related)
n d
portal content (example: Universal Work List) e
m
om
Technical Content (Web Application and Queries) ec
BI Web Applications R
Flexible analysis of statistics data and sophisticated
presentation of information (graphs, charts, tables)
Queries
InfoProviders
Technical / MultiProviders
Content (InfoProviders and DataSources) r y
t o
DataSources
Central Data Basis for BI Administration Cockpit and BI system d a
a n
load transaction ST03 M
Persistent Data Storage and Remote Access to BI Statistics
Information
Data Load Data Load Query Runtime
Statistics Status Statistics I n
BI Statistics li t-
Detailed Runtime Statistics Data collection for various BI Bu
BI
Data in Data Warehousing,
Objects BI
Enterprise Reporting and Planning
Warehousing Platform Suite
SAP NetWeaver BI
© SAP 2007 / Page 70
 New BI Statistics and Technical Content

Main enhancements
New Technical Content for new and enhanced BI Statistics
New Query Runtime Statistics
Process Chain and DTP Statistics
BI Object Request and Process Status
Technical Content for direct access and analysis on persistent data
Per default, queries from the Technical Content filter on reading from
persistent InfoProviders only
Reading from Virtual Providers can be enabled on query level by customers
Technical Content on detailed and aggregated level
For Query Runtime Statistics
New maintenance for statistics data collection
Enabling statistics and selection of detail level for statistics

© SAP 2007 / Page 71

 Analysis of BI Statistics data in SAP
NetWeaver 7.0

New: System Load

analysis for BI based
on Technical Content

BI Administration Cockpit

New and enhanced

Technical Content
ST03 – BW System Load
New: Persistent
data storage and Statistics tables (RSSDSTAT)
direct access
New: Ad hoc
analysis of
statistics data

Direct analysis Expert mode “profiling”

Query Monitor (RSRT) of tables RSDDSTAT* in the (new) BEx Web
© SAP 2007 / Page 72
 Agenda

1. Overview
2. Data Modeling
2.1. Data Warehousing Workbench
2.2. DataSource
2.3. Transformation
2.4. DataStore Objects
2.5. Modeling Data Marts
3. Data Flow Design
3.1. Data Transfer Process
3.2. Process Chains
4. Administration & Monitoring
4.1. Administration Cockpit
4.2. Information Lifecycle Management
5. Maintaining Data Security
6. Performance Management
5.1. SAP NetWeaver BI Accelerator
5.2. Other Performance Techniques

© SAP 2007 / Page 73

 BI Architecture:
Platform & Data Warehouse
Enterprise Query, Reporting & Analysis

Analytic Engine
BI Accelerator

Data Flow Control / Process Chains

Calculation Caching

Monitoring / Administration
Aggregation Security
Meta Data Repository / Documents

InfoObjects / Master Data

Analysis Process Design Planning Services

Near-Line Storage

Enterprise Data Warehouse

(Architected)
Operational Data Marts Open
Data Store Hub
(volatile) Data Warehouse Layer Service
(historical)

DataSource / PSA

Source Systems
© SAP 2007 / Page 74
 Data-Aging Strategies – Initial Steps

Categorizing Information According to Importance:

Online Database Near-Line Storage Classic Archive

Frequently read /
changed data

Rarely read data

Very rarely read data

© SAP 2007 / Page 75

 Persistent Data Warehouse Layers –
Strategic Aspects

BI

Data Warehouse

Architected
Data Marts
Operational
Data Store
(volatile)

NLS Engine
Data Warehouse Layer
BIA Engine

(historical) NLS
InfoProvider
InfoCubes
DataStore-Objects
• Multidimensional Model
• High Performance Capabilities
• High Volume Capabilities
• Optimized TCO

© SAP 2007 / Page 76

 Modeling Aspects –
Perfect InfoCube Design Example

BI

InfoCube
Offline Archive

BIA NLS

Indexing Archiving

Staging

RDBMS

© SAP 2007 / Page 77

 Information Lifecycle Management Aspects

BI

InfoCube
Offline Archive

BIA Engine
NLS

high frequently frequently non frequently rarely

Accelerated Online Nearline Offline

RDBMS

© SAP 2007 / Page 78

 Reporting Aspects

Business Explorer Suite (BEx)

Transparent Access No Access

BI
Adjoint InfoProvider

InfoProvider

BIA Engine NLS Engine

NearlineProvider Offline Archive

RDBMS

© SAP 2007 / Page 79

 Dataflow Aspects

• timeslices + dimensions
• ADK, ADK/NLS, NLS
BI • new process type in
ProcessChains
Data Mart
• flexible for structural changes
BIA Engine • Archive and deleteNLSinEngine
one LUW
Indexing • write protection for removed
areas in Data Store objects

EDW DAP

DTP DTP

PSA

• Reload via DTP available

Sources
© SAP 2007 / Page 80
 The Near-Line Storage Solution for
SAP NetWeaver BI

Near-Line Storage
Separation of frequently used data and rarely used data via Admin Cockpit
capabilities
NLS support for InfoCubes and DataStore objects
Transparent access to „non-archived“ and „archived“ data for queries
Open interface for certified partners
Development partners
PBS Software – CBW®
FileTek – StorHouse®
OuterBay - LiveArchive®
SAND-Technologies - Searchable Archive®

© SAP 2007 / Page 81

 SAP NetWeaver 7.0 BI: NLS-Based Archiving

Query Properties Data Archiving Process

Near-line storage to be read as well
Defining a flat view of the InfoProvider
without navigational attributes and SIDs

InfoProvider Scheduling via Process Chain

InfoCubes
Data Store Objects Archive Type
Offline, ADK only (like BW 3.x)
Near-Line only
Offline and Near-Line (NLS
indexing Offline Archive)

Selection Schema
Time-Slice Archiving
relative archiving periods, delta oriented,
DSO and compressed InfoCube,
range protection for incoming data

Pure Request-based
for uncompressed InfoCubes

Flexible Selections
Online DB Archive
© SAP 2007 / Page 82
no support for periodic processing
 Agenda

1. Overview
2. Data Modeling
2.1. Data Warehousing Workbench
2.2. DataSource
2.3. Transformation
2.4. DataStore Objects
2.5. Modeling Data Marts
3. Data Flow Design
3.1. Data Transfer Process
3.2. Process Chains
4. Administration & Monitoring
4.1. Administration Cockpit
4.2. Information Lifecycle Management
5. Maintaining Data Security
6. Performance Management
5.1. SAP NetWeaver BI Accelerator
5.2. Other Performance Techniques

© SAP 2007 / Page 83

 Authorizations Levels

Authorizations can be defined

On characteristic level
On InfoCube level
On characteristic level
On characteristic value level
On key figure level
Autho-
On hierarchy node level rization

On characteristic value level On key figure level

Authorization
Authorization

© SAP 2007 / Page 84

 Introduction to Analysis Authorizations

Authorization Check ok
Query results will be shown if
query selection is a proper
subset of the authorization Query
Selection

Authorizations
Authorization Check not ok
Query results will not be shown at all (‘not authorized’)
– even if parts of the authorizations are met

Query
Selection

Authorizations

© SAP 2007 / Page 85

 Authorization Relevant Characteristics

Before restricting
authorizations on
characteristics, you have
to mark them as
authorization-relevant.

InfoObject maintenance / transaction RSD1

© SAP 2007 / Page 86

 Authorizing Characteristic Values – 1 –

Central maintenance for

(analysis) authorizations /
transaction RSECADMIN

Scenario: A group of users

is authorized only to
specific sales
organizations (e.g. Berlin
and Birmingham)

© SAP 2007 / Page 87

 Authorizing Characteristic Values – 2 –

A group of users is
authorized only to specific
sales organizations (e.g.
Berlin and Birmingham)

(Berlin)
(Birmingham)

Possible Values
EQ: single value
BT: range of values
CP: contains (simple) patterns ending with ‘*’ or ‘+’
(e.g. XY*)
© SAP 2007 / Page 88
 Authorizing Navigational Attributes – 1 –

If you want to grant

authorizations on
navigational attributes,
mark them in the attribute
tab strip as authorization
relevant.

© SAP 2007 / Page 89

 Authorizing Hierarchies – 1 –

On the same level like the

value authorization, you
can also grant
authorizations on
hierarchy levels.

Assume you’ll have a

sales organization as
depicted.

© SAP 2007 / Page 90

 Authorizing Hierarchies – 2 –

Now you grant access for

the complete Americas
and France.

You can also use

variables for flexibly
and dynamically
determining
hierarchy nodes.

© SAP 2007 / Page 91

 Special Authorizations

Special authorizations
* (asterisk): denotes a set of arbitrary characters
+ (plus): denotes exactly one character (e.g. 01.++.2005 until 10.++.2005 : allows
access only the first 10 days of each month in 2005 - only available for time
validity (0TCAVALID))
: (colon): allows only aggregated access to data (e.g. allows information on all
sales areas only on aggregated level – not on particular countries)

Key figure authorizations

For key figure authorizations, you can include 0TCAKYFNM as

characteristic into the authorization. Note: hierarchy authorizations are not
allowed on this characteristic.

Note: Once you define 0TCAKYFNM authorization-relevant, key figures

are checked for every InfoProvider.

© SAP 2007 / Page 92

 Selection and Authorization

Check of Authorizations
Selection of query will be checked against the union of the authorizations
Example:
One authorization grants access to cost center 1000 for year 2004, a second
one grants access to the same cost center for year 2005
Access to a query selection with cost center 1000 and years 2004 and 2005 will
be granted

Cost

Year 2005
Year 2004
Center

CC 1000

Year

Note: In the former concept of authorization objects, the query selection had to
be in the intersection of the two authorization object if the authorization should
be checked (i.e. the mentioned query was not authorized)

© SAP 2007 / Page 93

 Comparing Authorization Concept

Comparison Analysis Authorizations

<= SAP NetWeaver 2004 vs. SAP NetWeaver 7.0
Most important differences

<=SAP NetWeaver 2004 SAP NetWeaver 7.0

Technical Foundation Authorization Objects Analysis Authorization

Not Changeable
Maintenance Changeable
Afterwards
Number of InfoObjects
Number of objects 10 objects
not limited
Navigational Attributes Only on global basis Individually
Via GUID and Equivalent to value
Hierarchy Authorizations
0TCTAUTHH authorizations
Composition of Intersection of business
Union (‚as expected‘)
authorizations objects
Per InfoObject AND
Authorization Relevance Only InfoObject setting
InfoCube

© SAP 2007 / Page 94

 Migration

Migration Support
ABAP program RSEC_MIGRATION (use transaction SA38)
No complete, automatic migration, but support
About 80% automatic migration expected
The more complex the existing authorization concept, the more manual
migration work might be necessary
Customer-exit variables for 0TCTAUTHH cannot be migrated; the respective
hierarchy nodes must be assigned manually
Intensive tests are highly recommended
Singular event, not for scheduling
During migration to the new authorization concept, the existing concept won‘t be
changed

© SAP 2007 / Page 95

 Agenda

1. Overview
2. Data Modeling
2.1. Data Warehousing Workbench
2.2. DataSource
2.3. Transformation
2.4. DataStore Objects
2.5. Modeling Data Marts
3. Data Flow Design
3.1. Data Transfer Process
3.2. Process Chains
4. Administration & Monitoring
4.1. Administration Cockpit
4.2. Information Lifecycle Management
5. Maintaining Data Security
6. Performance Management
5.1. SAP NetWeaver BI Accelerator
5.2. Other Performance Techniques

© SAP 2007 / Page 96

 Customer Pain Points

Increasing data Information at the

volume speed of thought

Increasing number
Quick and easy
of information
scalability
workers

Additional Reduce
Administration cost of operation
effort significantly

© SAP 2007 / Page 97

 SAP NetWeaver BI Accelerator
Value Proposition

Very fast query Increased Stable query

response time quality of response time
information/
Performance Extended Independent of
improvements BI reach DB optimizer,
by factor 10 – 100 aggregates, ...

SAP NetWeaver BI Accelerator

Implemented for latest No aggregate

blade server hardware Significant maintenance, minimized
platforms reduction of roll-up/change run
operation
High scalability costs Low maintenance

© SAP 2007 / Page 98

 SAP NetWeaver BI Accelerator

SAP NetWeaver BI Accelerator for high performance BI

A new transparent approach to boost BI query performance
Performance speedup factor between 10 and 100
Without changing the BI user experience (transparent to users)
Pre-requisite: BI in SAP NetWeaver 7.0

Queries Queries

SAP NetWeaver
Business BI Accelerator
Intelligence
DBMS

X
Database

© SAP 2007 / Page 99

 SAP NetWeaver BI Accelerator Scenarios

Ready for high data volumes

Queries that routinely involve access to many millions of records and may involve up to
billions of records
Examples: retail, utilities, telephone companies
Challenging response time SLAs
Example: service level agreements for call center operators demand short response times
for good closure rates
Unpredictable types of queries
Far more different data sets and aggregations than traditional optimization and caching
strategies can handle
Excellent response times for any drill-down, slice & dice, …
Examples: on-demand reporting for different user groups,
ad hoc analyses
Minimizing costs of operation
Maintenance of aggregates can be significantly reduced
Reduced roll-up and change run times

© SAP 2007 / Page 100

 Agenda

1. Overview
2. Data Modeling
2.1. Data Warehousing Workbench
2.2. DataSource
2.3. Transformation
2.4. DataStore Objects
2.5. Modeling Data Marts
3. Data Flow Design
3.1. Data Transfer Process
3.2. Process Chains
4. Administration & Monitoring
4.1. Administration Cockpit
4.2. Information Lifecycle Management
5. Maintaining Data Security
6. Performance Management
5.1. SAP NetWeaver BI Accelerator
5.2. Other Performance Techniques

© SAP 2007 / Page 101

 Aggregates

Aggregates
Pre-aggregated (sub-)InfoCubes
Alternative to SAP NetWeaver BI Accelerator

Month Material Revenue

July Hammer 10
July Nail 20
August Hammer 10
August Nail 20
Analytic Engine
Month Revenue
Database /
July 30
Selection August 30

Month Revenue
July 30
August 30
Aggregate

© SAP 2007 / Page 102

 Query Cache

Query Cache
Stores query results in cross-transactional application buffer
Re-use of similar query results – also for other users
Can be actively used for performance improvement pre-load the cache via
information broadcasting

Query Cache

Aggregates or
SAP NetWeaver BIA

InfoCube
(if BIA is used, InfoCube data on database is not read)

© SAP 2007 / Page 103

 Other Performance Options

Modeling options
MultiProvider (semantic) partitioning
Line-item dimensions

Database features
Indexing
Database Statistics

© SAP 2007 / Page 104

 Compression

Compression
Move data from F to E fact table
Compression usually reduces the number of records by combining records with
the same key that has been loaded in separate requests
When dealing with non-cumulative key figures, it is highly recommended to
regularly compress (also when using SAP NetWeaver BI Accelerator)
Double fact table
As InfoPackages are added, F
“F” Table
Up fact table partitions are created
– Request Information loa
d
– Typically small F - Table
– Optimised for Loading
REQUEST No. Time Material Sales
and Deleting
“E” Table
Compression Fact Table
– Optimised for Queries
– Typically large E - Table
– User-defined DB Partitioning REQUEST No. Time Material Sales
(depending on the DBMS)
– But: no information on requests
© SAP 2007 / Page 105
 Copyright 2007 SAP AG
All rights reserved

No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP AG. The information contained herein may be changed
without prior notice.
Some software products marketed by SAP AG and its distributors contain proprietary software components of other software vendors.
SAP, R/3, mySAP, mySAP.com, xApps, xApp, SAP NetWeaver, Duet, Business ByDesign, ByDesign, PartnerEdge and other SAP products and services mentioned herein as well as their
respective logos are trademarks or registered trademarks of SAP AG in Germany and in several other countries all over the world. All other product and service names mentioned and
associated logos displayed are the trademarks of their respective companies. Data contained in this document serves informational purposes only. National product specifications may vary.

The information in this document is proprietary to SAP. This document is a preliminary version and not subject to your license agreement or any other agreement with SAP. This document
contains only intended strategies, developments, and functionalities of the SAP® product and is not intended to be binding upon SAP to any particular course of business, product strategy,
and/or development. SAP assumes no responsibility for errors or omissions in this document. SAP does not warrant the accuracy or completeness of the information, text, graphics, links, or
other items contained within this material. This document is provided without a warranty of any kind, either express or implied, including but not limited to the implied warranties of
merchantability, fitness for a particular purpose, or non-infringement.
SAP shall have no liability for damages of any kind including without limitation direct, special, indirect, or consequential damages that may result from the use of these materials. This limitation
shall not apply in cases of intent or gross negligence.
The statutory liability for personal injury and defective products is not affected. SAP has no control over the information that you may access through the use of hot links contained in these
materials and does not endorse your use of third-party Web pages nor provide any warranty whatsoever relating to third-party Web pages

Weitergabe und Vervielfältigung dieser Publikation oder von Teilen daraus sind, zu welchem Zweck und in welcher Form auch immer, ohne die ausdrückliche schriftliche Genehmigung durch
SAP AG nicht gestattet. In dieser Publikation enthaltene Informationen können ohne vorherige Ankündigung geändert werden.
Einige von der SAP AG und deren Vertriebspartnern vertriebene Softwareprodukte können Softwarekomponenten umfassen, die Eigentum anderer Softwarehersteller sind.
SAP, R/3, mySAP, mySAP.com, xApps, xApp, SAP NetWeaver, Duet, Business ByDesign, ByDesign, PartnerEdge und andere in diesem Dokument erwähnte SAP-Produkte und Services
sowie die dazugehörigen Logos sind Marken oder eingetragene Marken der SAP AG in Deutschland und in mehreren anderen Ländern weltweit. Alle anderen in diesem Dokument erwähnten
Namen von Produkten und Services sowie die damit verbundenen Firmenlogos sind Marken der jeweiligen Unternehmen. Die Angaben im Text sind unverbindlich und dienen lediglich zu
Informationszwecken. Produkte können länderspezifische Unterschiede aufweisen.

Die in diesem Dokument enthaltenen Informationen sind Eigentum von SAP. Dieses Dokument ist eine Vorabversion und unterliegt nicht Ihrer Lizenzvereinbarung oder einer anderen
Vereinbarung mit SAP. Dieses Dokument enthält nur vorgesehene Strategien, Entwicklungen und Funktionen des SAP®-Produkts und ist für SAP nicht bindend, einen bestimmten
Geschäftsweg, eine Produktstrategie bzw. -entwicklung einzuschlagen. SAP übernimmt keine Verantwortung für Fehler oder Auslassungen in diesen Materialien. SAP garantiert nicht die
Richtigkeit oder Vollständigkeit der Informationen, Texte, Grafiken, Links oder anderer in diesen Materialien enthaltenen Elemente. Diese Publikation wird ohne jegliche Gewähr, weder
ausdrücklich noch stillschweigend, bereitgestellt. Dies gilt u. a., aber nicht ausschließlich, hinsichtlich der Gewährleistung der Marktgängigkeit und der Eignung für einen bestimmten Zweck
sowie für die Gewährleistung der Nichtverletzung geltenden Rechts.
SAP übernimmt keine Haftung für Schäden jeglicher Art, einschließlich und ohne Einschränkung für direkte, spezielle, indirekte oder Folgeschäden im Zusammenhang mit der Verwendung
dieser Unterlagen. Diese Einschränkung gilt nicht bei Vorsatz oder grober Fahrlässigkeit.
Die gesetzliche Haftung bei Personenschäden oder die Produkthaftung bleibt unberührt. Die Informationen, auf die Sie möglicherweise über die in diesem Material enthaltenen Hotlinks
zugreifen, unterliegen nicht dem Einfluss von SAP, und SAP unterstützt nicht die Nutzung von Internetseiten Dritter durch Sie und gibt keinerlei Gewährleistungen oder Zusagen über
Internetseiten Dritter ab.
Alle Rechte vorbehalten.

© SAP 2007 / Page 106