SAP NetWeaver 7.

0: Enterprise Data Warehousing

Overview

Product Management SAP NetWeaver BI November 2007

Agenda

1. Overview 2. Data Modeling

2.1. 2.2. 2.3. 2.4. 2.5. 3.1. 3.2. 4.1. 4.2. Data Warehousing Workbench DataSource Transformation DataStore Objects Modeling Data Marts Data Transfer Process Process Chains Administration Cockpit Information Lifecycle Management

3. Data Flow Design 4. Administration & Monitoring 5. Maintaining Data Security 6. Performance Management
5.1. 5.2. SAP NetWeaver BI Accelerator Other Performance Techniques

SAP 2007 / Page 2

Enterprise Data Warehousing

Provide each organizational unit or better each role with the needed reliable, consolidated, integrated, up-to-date, and historical information
e.g., at business unit level provide a: local/subsidary view regional view global view at headquarter level across business units provide regional view global view

SAP 2007 / Page 3

The Challenge of Enterprise Data Warehousing

EDW

With an centralised Enterprise Data Warehouse:

People will find the right information Related information is connected Collaboration and information exchange between people does work

SAP 2007 / Page 4

Architecture SAP NetWeaver BI

SAP NetWeaver Portal Enterprise Search Knowledge Management Worklists Collaboration

Information Broadcasting UIs

can be embedded

Composite Services & BAPIS Visual Composer Embedded BI BI Kit

Ad Hoc

BI App

Planning Layout

Enterprise Report

MS Excel

Business Explorer Suite (BEx)

Web Analyzer

Web Application Designer

Report Designer

Analyzer

BI Consumer Services BI Consumer Services

Web Services

BI Layer Query Designer VirtualProvider Planning Modeler Analytic Engine Master Data Meta Data Repository

ODBO/XMLA SAP NetWeaver BI Accelerator Appliance

Operational Data Store

Data Marts Data Warehouse PSA Open Hub Service

Downstream System Near-Line Storage

Data Sources

SAP Operational Data

Non-SAP Operational Data

3rd-Party BI Data

SAP NetWeaver BI Data

SAP 2007 / Page 5

BI Architecture: Enterprise BI Data Management

Enterprise Query, Reporting & Analysis

BI Accelerator

Analytic Engine
Calculation Caching Security Planning Services

Meta Data Repository / Documents

InfoObjects / Master Data

Aggregation Analysis Process Design

Near-Line Storage

Enterprise Data Warehouse

Operational Data Store (volatile) (Architected) Data Marts Data Warehouse Layer (historical) DataSource / PSA Open Hub Service

Source Systems
SAP 2007 / Page 6

Data Flow Control / Process Chains

Monitoring / Administration

Enterprise Data Warehousing - Processes

Data Modeling for EDW

Data Flow Design

Maintaining Data Security

Administration and Monitoring

Performance Management

Data Modeling for EDW

Business (Process) Experts can define the basis for the enterprise reporting. They define data containers (InfoProviders) and data consolidation rules. Multiple-layer EDW-architectures are supported.

Data Flow Design

The data flow (in particular DTP and InfoPackage) is defined at this level.

Maintaining Data Security

This process enables organizations to model the companys security rules into the software in a highly flexible matter.

Administration and Monitoring

The BI administrator is offered a central admin and monitoring tool (NetWeaver Administrator), which facilitates monitoring of complex landscapes. In addition, lifecycle management tasks can be initiated from here.

Performance Management
Provides all means to accelerate query performance, in particular the SAP NetWeaver BI Accelerator.

SAP 2007 / Page 7

Agenda

1. Overview 2. Data Modeling

2.1. 2.2. 2.3. 2.4. 2.5. 3.1. 3.2. 4.1. 4.2. Data Warehousing Workbench DataSource Transformation DataStore Objects Modeling Data Marts Data Transfer Process Process Chains Administration Cockpit Information Lifecycle Management

3. Data Flow Design 4. Administration & Monitoring 5. Maintaining Data Security 6. Performance Management
5.1. 5.2. SAP NetWeaver BI Accelerator Other Performance Techniques

SAP 2007 / Page 8

Overview

Data Warehousing Workbench with SAP NetWeaver 7.0

Modeling and Administration view

SAP 2007 / Page 9

Data Warehousing Workbench

Usability Features
Favorites Personalization Advanced Search Complete data flow at a glance

SAP 2007 / Page 10

Agenda

1. Overview 2. Data Modeling

2.1. 2.2. 2.3. 2.4. 2.5. 3.1. 3.2. 4.1. 4.2. Data Warehousing Workbench DataSource Transformation DataStore Objects Modeling Data Marts Data Transfer Process Process Chains Administration Cockpit Information Lifecycle Management

3. Data Flow Design 4. Administration & Monitoring 5. Maintaining Data Security 6. Performance Management
5.1. 5.2. SAP NetWeaver BI Accelerator Other Performance Techniques

SAP 2007 / Page 11

Conceptual Layers of Data Warehousing

Information Access

Operational Data Store

(Architected) Data Marts Data Warehouse DataSources

(Persistent) Staging Area

Any Source

SAP 2007 / Page 12

Data Acquisition Layer Data Sources

Sources
Support of virtually all sources

DataSource
DB Connect UD Connect BI Service API File Interface Web Service

BAPI

ETL Tool Relational Source MultiDimensional Source

e.g. Hyperion

SAP Source

File

XML

Legacy Applications

e.g. IBM DB2, Teradata

e.g. SAP CRM

e.g. SAP NetWeaver PI (via proxy framework)

e.g. ORACLE Financials

SAP 2007 / Page 13

New BI DataSource concept with SAP NetWeaver 7.0

Highlights unique look and feel for all of the DataSource Types PSA is attached to DataSource InfoPackage writes to PSA Data Transfer Process writes from PSA to data targets direct/remote access is optional preview feature is standard automated conversions (e.g. date format detection)

SAP 2007 / Page 14

Source System Tree

Source sytems categories: SAP vs. non SAP File vs. database Relational vs. Multidimensional DB ABAP vs. Java XML vs. Text/Binary Pull vs. Push Realtime vs. Batch

SAP 2007 / Page 15

DataSource Example One fits all approach

General Information Descriptions Reconciliation flag (not functional) Opening Balance (inventory) Error handling (duprecs)

SAP 2007 / Page 16

Data Flow Concept in SAP NetWeaver 7.0

SAP NetWeaver BI Process Chain Data Transfer Process

InfoProvider

Transformation

InfoPackage

DataSource / PSA

Source System 1 Source

SAP 2007 / Page 17

Data Flow Concept in SAP NetWeaver 7.0 Simplified

SAP NetWeaver BI Process Chain (optional) Data Transfer Process

InfoProvider

Transformation

Restrictions: Not optimized for mass data transfer No packaging of data Full Mode Only
SAP 2007 / Page 18

DataSource / PSA

Source System 1 Source

Agenda

1. Overview 2. Data Modeling

2.1. 2.2. 2.3. 2.4. 2.5. 3.1. 3.2. 4.1. 4.2. Data Warehousing Workbench DataSource Transformation DataStore Objects Modeling Data Marts Data Transfer Process Process Chains Administration Cockpit Information Lifecycle Management

3. Data Flow Design 4. Administration & Monitoring 5. Maintaining Data Security 6. Performance Management
5.1. 5.2. SAP NetWeaver BI Accelerator Other Performance Techniques

SAP 2007 / Page 19

Data Flow in SAP NetWeaver 7.0 BI

SAP NetWeaver Business Intelligence InfoProvider
Transformation

Downstream Systems Open Hub Destination

Transformation

DTP

DTP

InfoProvider

Transformation

Data Transfer Process

DataSource / PSA InfoPackage Non-SAP InfoPackage SAP SAP NetWeaver PI

SAP NetWeaver BI Any Source

SAP 2007 / Page 20

Transformation
Target

Universal transformation from source to target objects

Transformation types:
Move, aggregate, constant, master data look up, Business rules, e.g. unit + currency translation Formula builder with rich predefined functions library ABAP routines incl. regular expressions

Transformation End Routine * Expert Routine

Transformation Rule 1 Transformation Rule n

SAP NetWeaver 7.0 Enhancements Start Routine

Intuitive UI Unit conversion Unified transfer + update rules into all-in-one capability Integration of Open Hub Service

Package 1 Package 2 Package m Source

Semantic Groups *

* optional
SAP 2007 / Page 21

Transformation Definition

Access from the Data Warehousing Workbench

New transformation Unification of transfer and update rules InfoSource not mandatory anymore Former concept of update rules Small square next to the transformation icon Access from context menu via additional functions Links sources and target New source: InfoSet Other sources: DataSource, InfoCube, DataStore object, InfoObject, InfoSource Targets: InfoCube, DataStore object, InfoObject, InfoSource, Open Hub Destination

Transformation Update rule

SAP 2007 / Page 22

Transformation Graphical UI

Source fields

Target fields

Note: Key figures, characteristics and date fields are shown on the same level (transformation group)
SAP 2007 / Page 23

Rules per group

Transformation Rules

Transformation rule details

Information on Rule type Currency/ Unit Conversion Source fields Target Fields

SAP 2007 / Page 24

Enhanced Data Flow in SAP NetWeaver 7.0 BI

SAP NetWeaver Business Intelligence

InfoProvider

Transformation (optional) InfoSource (optional) Data Transfer Process

Transformation (optional) InfoSource (optional)

Transformation

Transformation

DataSource / PSA

InfoProvider

SAP 2007 / Page 25

Transformations InfoSource 1

InfoSource
Transformation directly links from a source InfoProvider (or DataSource) to a target InfoProvider An InfoSource is usually not needed New InfoSource architecture is used (flat InfoObject-based structure) Scenarios for (flexible) InfoSource A flexible InfoSource is necessary in order to use currency or unit conversion from the source DataSource Define InfoSource as an intermediate structure You can use a flexible InfoSource as a uniform source for several targets; the InfoSource can the be target from different sources (see next slide) Note: for direct InfoSources (for master data updates), there is no difference between old and new InfoSource, i.e. you can define a transformation as well as transfer rules Pre-requisite: InfoObject is defined as InfoProvider

SAP 2007 / Page 26

Transformations InfoSource 2

InfoSource
Scenario: InfoSource as a uniform source for several targets and as target from different sources SAP NetWeaver Business Intelligence InfoProvider InfoProvider 2 Transformation InfoSource Transformation DataSource 1 DataSource 2 DataSource DataSource n

InfoProvider 1

InfoProvider m

SAP 2007 / Page 27

Transformation Groups 1

Transformation Groups
Summarize key figures with the same characteristics assignments All key figures of one transformation are updated based on the same characteristic values If other characteristic updates are necessary for particular key figures, a new transformation is created

SAP 2007 / Page 28

Transformation Groups 2

Transformation Groups
Use / Example Scenario: overview on bonus-relevant sales of all employees An employee generates a certain sales volume, which is the basis for his/her bonus The manager of the employee will be assigned 10% of the employees bonus as managers bonus relevant two transformation groups are generated (e.g. employee and manager)
Source Employee Johnson Sales Volume 1000 Manager Giles

Transformation Group 1
Employee Employee Sales Volume Bonus-relevant Sales

Target Employee Johnson Giles Bonus-relevant Sales 1000 100

Transformation Group 1
Manager Employee Sales Volume*0,1 Bonus-relevant Sales

SAP 2007 / Page 29

Agenda

1. Overview 2. Data Modeling

2.1. 2.2. 2.3. 2.4. 2.5. 3.1. 3.2. 4.1. 4.2. Data Warehousing Workbench DataSource Transformation DataStore Objects Modeling Data Marts Data Transfer Process Process Chains Administration Cockpit Information Lifecycle Management

3. Data Flow Design 4. Administration & Monitoring 5. Maintaining Data Security 6. Performance Management
5.1. 5.2. SAP NetWeaver BI Accelerator Other Performance Techniques

SAP 2007 / Page 30

DataStore Object Types

DataStore Object types overview

Primary Usage

Structure
Activation Queue

DataStore object type

EDW layer

ODS Layer

Delta / Change Data Capture

Fast Access (no activation)

Others

Active Data

Change Log

Integration into data flow

Standard DataStore Object

Delta determination from after images on record level

via staging (DTP)

Write-optimized DataStore Object

On request level

Staging layer esp. for large sets of data with (generally) unique key for external applications and analysis processes (APD)

via staging (DTP) via APIs, Staging into subsequent targets possible

DataStore Object for direct update

No

SAP 2007 / Page 31

Standard DataStore Object 1

Details
Option Generation of SID Values Improves query performance Queries are also possible if SID values are not generated Option Unique data records Only available if Generation of SID Values is set Activation process is optimized (only inserts, no sorting, no before image) Note: error if key already exists For (non-reporting) scenarios, write-optimized DataStores are recommended instead of standard DataStore objects with unique flag Performance Improvement Rollback Instead of rolling back in serial and in one transaction rollback now is in parallel and for each data package there is a single task
SAP 2007 / Page 32

Standard DataStore Object 2

Structure
Activation queue Used to store data to be updated in DataStore Object which has not been activated After activation the data can be deleted Technical key: Request SID, Package ID, Record number Active Data Table Same structure as the DataStore Object definition

Change Log Change history for delta mechanism from the DataStore Object into other InfoProvider Key fields: Request GUID, Package ID, Record number

SAP 2007 / Page 33

Write-optimized DataStore Object 1

Scenarios
Fast EDW inbound layer (no activation needed) For large sets of data records on detailed level (e.g. document level) wide structure is possible (16 key fields, 749 data fields) Load of new records: Every record has a new key E.g. for POS data Load & Drop: Full Upload into DataStore Object Update subsequent InfoProvider Drop DataStore Object data Continue with full uploads No update, only inserts

SAP 2007 / Page 34

Write-optimized DataStore Object 2

Details
Definition Only active data table (key: request ID, Packet No., Record No.) No change log and no activation queue Technical key is unique

Partitioned on request ID No SID generation Nevertheless: Reporting is possible (but not optimized for performance) Fully integrated in data flow: usable as data source and data target

Export into InfoProviders via request delta Can be included into MultiProvider or InfoSet

Uniqueness of Data Checkbox Do not check Uniqueness of data Performance improvement during data load Does not create/maintain unique index on semantic key

SAP 2007 / Page 35

DataStore Object For Direct Update 1

Details
Definition Only active data table Can be used as data target within APD cannot be used for transformation (upload) scenarios no loading process within BI but export into next InfoProvider is possible Reporting is possible Scenarios Used for direct input of (external) transactional data E.g. BI table for user interaction An API is available with a set of function modules (some are RFC enabled)

Fed by APD processes

SAP 2007 / Page 36

Agenda

1. Overview 2. Data Modeling

2.1. 2.2. 2.3. 2.4. 2.5. 3.1. 3.2. 4.1. 4.2. Data Warehousing Workbench DataSource Transformation DataStore Objects Modeling Data Marts Data Transfer Process Process Chains Administration Cockpit Information Lifecycle Management

3. Data Flow Design 4. Administration & Monitoring 5. Maintaining Data Security 6. Performance Management
5.1. 5.2. SAP NetWeaver BI Accelerator Other Performance Techniques

SAP 2007 / Page 37

InfoCube

InfoCube
Star Schema optimized for multi-dimensional reporting Master Master Data Data Dimension Dimension Dimension Dimension
Dimension ID

Master Master Data Data

Surrogate Key

Fact Table Fact Table

Support of degenerated dimensions

Dimension Dimension Master Master Data Data

SAP 2007 / Page 38

Dimension Dimension Master Master Data Data

InfoCube

Example: InfoCube in Meta Data Repository

SAP 2007 / Page 39

MultiProvider

MultiProvider
Logical definition without data persistency Abstraction level for queries Can integrate the following InfoProviders via union operator InfoCube InfoObject DataStore Object VirtualProvider InfoSet Aggregation Level

SAP 2007 / Page 40

InfoSet

InfoSet
Logical definition without data persistency Can integrate InfoCubes, DataStore Objects and InfoObjects join and outer join operator

InfoSet
w/o data persistency

InfoCube

DataStore Object

InfoObject / Master Data

SAP 2007 / Page 41

Agenda

1. Overview 2. Data Modeling

2.1. 2.2. 2.3. 2.4. 2.5. 3.1. 3.2. 4.1. 4.2. Data Warehousing Workbench DataSource Transformation DataStore Objects Modeling Data Marts Data Transfer Process Process Chains Administration Cockpit Information Lifecycle Management

3. Data Flow Design 4. Administration & Monitoring 5. Maintaining Data Security 6. Performance Management
5.1. 5.2. SAP NetWeaver BI Accelerator Other Performance Techniques

SAP 2007 / Page 42

Data Transfer Process: Complex Example

SAP Netweaver BI
Process Chain InfoSource
DTP TR DataStore Object 3 TR

TR

Process Chain
DTP

DTP

TR DataStore Object 1

TR

DTP

DataStore Object 2

DTP

TR DataSource (PSA) DataSource (PSA)

TR DataSource (PSA) DataSource (PSA)

DTP

IP

IP

Source System 1
SAP 2007 / Page 43

Source System 2

Benefits of New Data Transfer Process

Data Transfer Process (DTP) - Data Distribution within SAP NetWeaver BI

Loading data from one layer to others except InfoSources Separation of delta mechanism for different data targets Enhanced filtering in dataflow Improved transparency of staging processes across data warehouse layers (PSA, DWH layer, ODS layer, Architected Data Marts) Improved performance: optimized parallelization Enhanced error handling for DataStore object (error stack) Enables real-time data acquisition

SAP 2007 / Page 44

Filter in Data Transfer Process

With filter it is possible to load a set of data to the data target instead of the complete volume of data. Different data selections can be made via different data transfer processes for the same or for different data targets.

Extraction mode: Delta or Full

SAP 2007 / Page 45

Data Transfer Process

Loading directly into Data Target without PSA

Prerequisite
DataSource is enabled for direct access (table ROOSOURCE) Or SAP Basis Plug-In Release 2005.1 SP8, or SAP Plug-In Release 2004.1 4.6C Support Package 13 Or if you want to have the correction before the above support packages are released, please apply note 923783.

SAP 2007 / Page 46

Error Handling Overview

Process chain can automate the loading process

DTP Scheduler DTP Scheduler

IP DataSource (PSA) DataSource (PSA) Source System There is no error handling available for an InfoPackage. In case of invalid records, data needs to be reloaded from the source system.

DTP

Error Stack

Error DTP

Invalid records can be corrected in the error stack and updated into the data target
SAP 2007 / Page 47

Error Handling Features

Error Handling
Possibility to choose in the scheduler to... abort process when errors occur process the correct records but do not allow reporting on them process the correct records and allow reporting on them Number of wrong records which lead to a wrong request Invalid records can be written into an error stack Keys should be defined for error stack to enable the error handling of DataStore object Temporary data storage can be switched on/off for each substep of the loading process Invalid records can be updated into data targets after their correction.

SAP 2007 / Page 48

Error Handling

Error Handling Once errors occur, the whole Data Package is terminated. The request is not released for reporting. Valid records are updated. After manual release of the request, data is valid for reporting. Valid records are updated and available for reporting

SAP 2007 / Page 49

Error Stack 1

Error Stack Stores erroneous records Automatic checks: Existence of master data, conversion exit (restricted, e.g. Alpha) Customer-defined checks in transformation routines (see appendix for more information) Keeps the right sequence of records for consistent DataStore handling Key of error stack defines which data should be detained from the update after the erroneous data record After correction, Error-DTP updates data from error stack to data target Note: Once the request in the source object is deleted, the related data records in error stack are automatically deleted

SAP 2007 / Page 50

Error Stack 2

Error Stack Key of Error Stack = Semantic Groups Subset of the key of the target object Max. 16 fields Defining which data should be detained from the update after the erroneous data record (for DataStore Object) Semantic groups bundle records with the same semantic group key into the same request see transformation chapter for more details (for DataStore Object and InfoProvider)

SAP 2007 / Page 51

Temporary Data Storage

Temporary Data Storage Help for tracing the erroneous records and transformations Data records from different steps within the data transfer process can be stored temporarily Stores complete set of data (erroneous as well as valid records) Scenario: If the debugging mode is switched on Trace the erroneous records Trace Transformation

SAP 2007 / Page 52

Temporary Data Storage

Settings for Temporary Data Storage Level of detail Tracing the erroneous records Tracing transformation by package Tracing transformation by record Deletion of temporary storage With request status green If request is deleted After X days

Switch on/off the temporary data storage for data loading steps

SAP 2007 / Page 53

Data Transfer Process Monitor 1

DTP Monitor Integrated in InfoProvider management screen Integrated in DTP maintenance Additional information: duration of each step Temporary storage access if activated Error Stack is displayed in DTP Monitor

Error Stack

Data display in temporary storage

SAP 2007 / Page 54

DTP and Open Hub

Open Hub Destination as DTP DataTarget

SAP 2007 / Page 55

Open Hub Destinations

SAP 2007 / Page 56

DTP Initialization without Data Transfer

Data Transfer Process Initialization without data transfer

SAP 2007 / Page 57

DTP Monitor Header

Monitor Data transfer process header

SAP 2007 / Page 58

DTP Monitor Detail

New with SPS08: Monitor Data transfer process Detail

SAP 2007 / Page 59

Agenda

1. Overview 2. Data Modeling

2.1. 2.2. 2.3. 2.4. 2.5. 3.1. 3.2. 4.1. 4.2. Data Warehousing Workbench DataSource Transformation DataStore Objects Modeling Data Marts Data Transfer Process Process Chains Administration Cockpit Information Lifecycle Management

3. Data Flow Design 4. Administration & Monitoring 5. Maintaining Data Security 6. Performance Management
5.1. 5.2. SAP NetWeaver BI Accelerator Other Performance Techniques

SAP 2007 / Page 60

Introduction: Typical Data Load Cycle

Load into PSA Drop Indices

Data Load Monitor

Load into DataStore

Start
Activate Data in DataStore Object Load into InfoCube

Data Target Maintenance

SAP 2007 / Page 61

Roll up to BIA Index

Process Chain Example

SAP 2007 / Page 62

Three Different Views in the Transaction

Planning view: Build and change process chains

Grey: unplanned processes Green: planned prozesses Yellow: planned but unknown processes Red: multiple planned processes

Check view: Check for errors in design

Green: Error-free processes Yellow: Process with warnings Red: Process with errors

Log view: Monitoring of process chains

Grey: Not yet run Green: Finished without error Yellow: running Red: broken or failed

SAP 2007 / Page 63

Process Chains: Failed processes can send email

Write a message and fill in recipient and type. Info saved within process variant.

Planning view context menu

SAP 2007 / Page 64

Agenda

1. Overview 2. Data Modeling

2.1. 2.2. 2.3. 2.4. 2.5. 3.1. 3.2. 4.1. 4.2. Data Warehousing Workbench DataSource Transformation DataStore Objects Modeling Data Marts Data Transfer Process Process Chains Administration Cockpit Information Lifecycle Management

3. Data Flow Design 4. Administration & Monitoring 5. Maintaining Data Security 6. Performance Management
5.1. 5.2. SAP NetWeaver BI Accelerator Other Performance Techniques

SAP 2007 / Page 65

BI Administration Cockpit - Motivation

Easy administration for complex Enterprise Data Warehouses using the BI Administration Cockpit

SAP 2007 / Page 66

BI Administration Cockpit - Scope

Support the BI administrator in

Status tracking Performance optimization Strategic administration

including context-specific
Drill-down to details Processing options Exceptions (optional)

in the areas of
Enterprise Data Warehousing Enterprise Query, Reporting and Analysis Business Planning and Analytical Services

using proven technology

BI Queries BI Web Applications SAP NetWeaver Portal

by providing a central point of entry with cockpits

Real-time monitors Runtime Statistics Cross system monitoring

to make administration easier and faster and thus to lower the TCO

SAP 2007 / Page 67

BI Administration Cockpit - Overview

Central access to most important BI monitoring information Context menu for access to more detailed information or BI Transaction Exception definition for intuitive display of critical monitoring data (optional)

Monitoring of multiple BI systems in one view

Flexible filtering of relevant information

SAP 2007 / Page 68

Graphical display

BI Administration Cockpit Architecture

SAP NetWeaver Portal Portal Pages iViews iViews iViews

Business Package BI Administration 1.0 from the Portal Content Portfolio. BI Administration Cockpit can run in a central or in a local portal. Technical Content for SAP NetWeaver BI (software component BI_CONT, release 7.0.2), The Technical Content is entirely based on SAP BW 3.x functionality not requiring BI_JAVA. SAP NetWeaver 7.0 BI technology (software component SAP_BW)

BI Web Applications Queries InfoProviders / MultiProviders DataSources

Data Load Statistics

Data Load Status

Query Runtime Statistics

Data Warehousing SAP NetWeaver BI

SAP 2007 / Page 69

BI Platform

BI Suite

BI Administration Cockpit Main building blocks

SAP NetWeaver Portal Portal iViews iViews Single point of entry andiViews integration with other (non BI related) Pages

BI Administration Cockpit (Business Package)

portal content (example: Universal Work List)

Technical Content (Web Application and Queries)

BI Flexible Web Applications analysis of statistics data and sophisticated presentation of information (graphs, charts, tables) Queries InfoProviders / MultiProviders Technical Content (InfoProviders and DataSources)

ed nd e m om ec R

a load transaction ST03 M Persistent Data Storage and Remote Access to BI Statistics Information Query Runtime Data Load Data Load
Statistics

DataSources Central Data Basis for BI Administration Cockpit and BI system

at d

ry o

BI Statistics

Status

Statistics

Warehousing
SAP 2007 / Page 70

Detailed Runtime Statistics Data collection for various BI BI Data Objects in Data Warehousing, BI Enterprise Reporting and Planning
Platform Suite

Bu

I lti

SAP NetWeaver BI

New BI Statistics and Technical Content

Main enhancements
New Technical Content for new and enhanced BI Statistics New Query Runtime Statistics Process Chain and DTP Statistics BI Object Request and Process Status Technical Content for direct access and analysis on persistent data Per default, queries from the Technical Content filter on reading from persistent InfoProviders only Reading from Virtual Providers can be enabled on query level by customers Technical Content on detailed and aggregated level For Query Runtime Statistics New maintenance for statistics data collection Enabling statistics and selection of detail level for statistics

SAP 2007 / Page 71

Analysis of BI Statistics data in SAP NetWeaver 7.0

New: System Load analysis for BI based on Technical Content

BI Administration Cockpit New and enhanced Technical Content

ST03 BW System Load

New: Persistent data storage and direct access

Statistics tables (RSSDSTAT)

New: Ad hoc analysis of statistics data
Direct analysis of tables RSDDSTAT* Expert mode profiling in the (new) BEx Web

Query Monitor (RSRT)

SAP 2007 / Page 72

Agenda

1. Overview 2. Data Modeling

2.1. 2.2. 2.3. 2.4. 2.5. 3.1. 3.2. 4.1. 4.2. Data Warehousing Workbench DataSource Transformation DataStore Objects Modeling Data Marts Data Transfer Process Process Chains Administration Cockpit Information Lifecycle Management

3. Data Flow Design 4. Administration & Monitoring 5. Maintaining Data Security 6. Performance Management
5.1. 5.2. SAP NetWeaver BI Accelerator Other Performance Techniques

SAP 2007 / Page 73

BI Architecture: Platform & Data Warehouse

Enterprise Query, Reporting & Analysis

BI Accelerator

Analytic Engine
Calculation Caching Security Planning Services

Meta Data Repository / Documents

InfoObjects / Master Data

Aggregation Analysis Process Design

Near-Line Storage

Enterprise Data Warehouse

Operational Data Store (volatile) (Architected) Data Marts Data Warehouse Layer (historical) DataSource / PSA Open Hub Service

Source Systems
SAP 2007 / Page 74

Data Flow Control / Process Chains

Monitoring / Administration

Data-Aging Strategies Initial Steps

Categorizing Information According to Importance:

Online Database

Near-Line Storage

Classic Archive

Frequently read / changed data

Rarely read data

Very rarely read data

SAP 2007 / Page 75

Persistent Data Warehouse Layers Strategic Aspects

BI

Data Warehouse

Architected Data Marts Operational Data Store (volatile)

BIA Engine

Data Warehouse Layer (historical)

NLS

InfoProvider InfoCubes DataStore-Objects Multidimensional Model High Performance Capabilities High Volume Capabilities Optimized TCO

SAP 2007 / Page 76

NLS Engine

Modeling Aspects Perfect InfoCube Design Example

BI
InfoCube

Offline Archive

BIA

NLS

Indexing Staging

Archiving

RDBMS

SAP 2007 / Page 77

Information Lifecycle Management Aspects

BI
InfoCube

Offline Archive BIA Engine high frequently Accelerated frequently Online

NLS non frequently Nearline rarely Offline

RDBMS

SAP 2007 / Page 78

Reporting Aspects
Business Explorer Suite (BEx) Transparent Access

No Access

BI
Adjoint InfoProvider

InfoProvider

BIA Engine
NearlineProvider

NLS Engine Offline Archive

RDBMS

SAP 2007 / Page 79

Dataflow Aspects
timeslices + dimensions ADK, ADK/NLS, NLS new process type in ProcessChains flexible for structural changes Archive and delete inEngine NLS one LUW write protection for removed areas in Data Store objects DAP

BI
Data Mart

BIA Engine Indexing

EDW

DTP

DTP

PSA

Reload via DTP available

Sources
SAP 2007 / Page 80

The Near-Line Storage Solution for SAP NetWeaver BI

Near-Line Storage
Separation of frequently used data and rarely used data via Admin Cockpit capabilities NLS support for InfoCubes and DataStore objects Transparent access to non-archived and archived data for queries Open interface for certified partners Development partners PBS Software CBW FileTek StorHouse OuterBay - LiveArchive SAND-Technologies - Searchable Archive

SAP 2007 / Page 81

SAP NetWeaver 7.0 BI: NLS-Based Archiving

Query Properties Near-line storage to be read as well

Data Archiving Process

Defining a flat view of the InfoProvider without navigational attributes and SIDs

InfoProvider InfoCubes Data Store Objects

Scheduling via Process Chain Archive Type Offline, ADK only (like BW 3.x) Near-Line only Offline and Near-Line (NLS indexing Offline Archive) Selection Schema Time-Slice Archiving
relative archiving periods, delta oriented, DSO and compressed InfoCube, range protection for incoming data

Pure Request-based
for uncompressed InfoCubes

Online DB
SAP 2007 / Page 82

Archive

Flexible Selections
no support for periodic processing

Agenda

1. Overview 2. Data Modeling

2.1. 2.2. 2.3. 2.4. 2.5. 3.1. 3.2. 4.1. 4.2. Data Warehousing Workbench DataSource Transformation DataStore Objects Modeling Data Marts Data Transfer Process Process Chains Administration Cockpit Information Lifecycle Management

3. Data Flow Design 4. Administration & Monitoring 5. Maintaining Data Security 6. Performance Management
5.1. 5.2. SAP NetWeaver BI Accelerator Other Performance Techniques

SAP 2007 / Page 83

Authorizations Levels

Authorizations can be defined

On InfoCube level On characteristic level On characteristic value level On key figure level On hierarchy node level

On characteristic level

Authorization

On characteristic value level

On key figure level

Authorization

Authorization

SAP 2007 / Page 84

Introduction to Analysis Authorizations

Authorization Check ok
Query results will be shown if query selection is a proper subset of the authorization

Query Selection

Authorizations

Authorization Check not ok

Query results will not be shown at all (not authorized) even if parts of the authorizations are met

Query Selection Authorizations

SAP 2007 / Page 85

Authorization Relevant Characteristics

Before restricting authorizations on characteristics, you have to mark them as authorization-relevant.

InfoObject maintenance / transaction RSD1

SAP 2007 / Page 86

Authorizing Characteristic Values 1

Central maintenance for (analysis) authorizations / transaction RSECADMIN

Scenario: A group of users is authorized only to specific sales organizations (e.g. Berlin and Birmingham)

SAP 2007 / Page 87

Authorizing Characteristic Values 2

A group of users is authorized only to specific sales organizations (e.g. Berlin and Birmingham)

(Berlin) (Birmingham)

Possible Values
EQ: single value BT: range of values CP: contains (simple) patterns ending with * or + (e.g. XY*)
SAP 2007 / Page 88

Authorizing Navigational Attributes 1

If you want to grant authorizations on navigational attributes, mark them in the attribute tab strip as authorization relevant.

SAP 2007 / Page 89

Authorizing Hierarchies 1

On the same level like the value authorization, you can also grant authorizations on hierarchy levels. Assume youll have a sales organization as depicted.

SAP 2007 / Page 90

Authorizing Hierarchies 2

Now you grant access for the complete Americas and France.

You can also use variables for flexibly and dynamically determining hierarchy nodes.

SAP 2007 / Page 91

Special Authorizations

Special authorizations
* (asterisk): denotes a set of arbitrary characters + (plus): denotes exactly one character (e.g. 01.++.2005 until 10.++.2005 : allows access only the first 10 days of each month in 2005 - only available for time validity (0TCAVALID)) : (colon): allows only aggregated access to data (e.g. allows information on all sales areas only on aggregated level not on particular countries)

Key figure authorizations For key figure authorizations, you can include 0TCAKYFNM as characteristic into the authorization. Note: hierarchy authorizations are not allowed on this characteristic. Note: Once you define 0TCAKYFNM authorization-relevant, key figures are checked for every InfoProvider.

SAP 2007 / Page 92

Selection and Authorization

Check of Authorizations
Selection of query will be checked against the union of the authorizations Example: One authorization grants access to cost center 1000 for year 2004, a second one grants access to the same cost center for year 2005 Access to a query selection with cost center 1000 and years 2004 and 2005 will be granted
Year 2004 Year 2005 Cost Center

CC 1000 Year

Note: In the former concept of authorization objects, the query selection had to be in the intersection of the two authorization object if the authorization should be checked (i.e. the mentioned query was not authorized)

SAP 2007 / Page 93

Comparing Authorization Concept

Comparison Analysis Authorizations <= SAP NetWeaver 2004 vs. SAP NetWeaver 7.0
Most important differences <=SAP NetWeaver 2004 Technical Foundation Maintenance Number of objects Navigational Attributes Hierarchy Authorizations Authorization Objects Not Changeable Afterwards 10 objects Only on global basis SAP NetWeaver 7.0 Analysis Authorization Changeable Number of InfoObjects not limited Individually

Via GUID and Equivalent to value 0TCTAUTHH authorizations Composition of Intersection of business Union (as expected) authorizations objects Per InfoObject AND Authorization Relevance Only InfoObject setting InfoCube

SAP 2007 / Page 94

Migration

Migration Support
ABAP program RSEC_MIGRATION (use transaction SA38) No complete, automatic migration, but support About 80% automatic migration expected The more complex the existing authorization concept, the more manual migration work might be necessary Customer-exit variables for 0TCTAUTHH cannot be migrated; the respective hierarchy nodes must be assigned manually Intensive tests are highly recommended Singular event, not for scheduling During migration to the new authorization concept, the existing concept wont be changed

SAP 2007 / Page 95

Agenda

1. Overview 2. Data Modeling

2.1. 2.2. 2.3. 2.4. 2.5. 3.1. 3.2. 4.1. 4.2. Data Warehousing Workbench DataSource Transformation DataStore Objects Modeling Data Marts Data Transfer Process Process Chains Administration Cockpit Information Lifecycle Management

3. Data Flow Design 4. Administration & Monitoring 5. Maintaining Data Security 6. Performance Management
5.1. 5.2. SAP NetWeaver BI Accelerator Other Performance Techniques

SAP 2007 / Page 96

Customer Pain Points

Increasing data volume Increasing number of information workers Additional Administration effort

Information at the speed of thought Quick and easy scalability Reduce cost of operation significantly

SAP 2007 / Page 97

SAP NetWeaver BI Accelerator Value Proposition

Very fast query response time Performance improvements by factor 10 100 Increased quality of information/ Extended BI reach Stable query response time Independent of DB optimizer, aggregates, ...

SAP NetWeaver BI Accelerator

Implemented for latest blade server hardware platforms High scalability No aggregate maintenance, minimized roll-up/change run Low maintenance

Significant reduction of operation costs

SAP 2007 / Page 98

SAP NetWeaver BI Accelerator

SAP NetWeaver BI Accelerator for high performance BI A new transparent approach to boost BI query performance
Performance speedup factor between 10 and 100 Without changing the BI user experience (transparent to users) Pre-requisite: BI in SAP NetWeaver 7.0

Queries

Queries

SAP NetWeaver Business Intelligence DBMS

BI Accelerator

X
Database

SAP 2007 / Page 99

SAP NetWeaver BI Accelerator Scenarios

Ready for high data volumes Queries that routinely involve access to many millions of records and may involve up to billions of records Examples: retail, utilities, telephone companies Challenging response time SLAs Example: service level agreements for call center operators demand short response times for good closure rates Unpredictable types of queries Far more different data sets and aggregations than traditional optimization and caching strategies can handle Excellent response times for any drill-down, slice & dice, Examples: on-demand reporting for different user groups, ad hoc analyses Minimizing costs of operation Maintenance of aggregates can be significantly reduced Reduced roll-up and change run times

SAP 2007 / Page 100

Agenda

1. Overview 2. Data Modeling

2.1. 2.2. 2.3. 2.4. 2.5. 3.1. 3.2. 4.1. 4.2. Data Warehousing Workbench DataSource Transformation DataStore Objects Modeling Data Marts Data Transfer Process Process Chains Administration Cockpit Information Lifecycle Management

3. Data Flow Design 4. Administration & Monitoring 5. Maintaining Data Security 6. Performance Management
5.1. 5.2. SAP NetWeaver BI Accelerator Other Performance Techniques

SAP 2007 / Page 101

Aggregates

Aggregates
Pre-aggregated (sub-)InfoCubes Alternative to SAP NetWeaver BI Accelerator

Month July July August August

Material Revenue Hammer 10 Nail 20 Hammer 10 Nail 20

Analytic Engine
Month Revenue July 30 August 30

Database / Selection
Month Revenue July 30 August 30

Aggregate

SAP 2007 / Page 102

Query Cache

Query Cache
Stores query results in cross-transactional application buffer Re-use of similar query results also for other users Can be actively used for performance improvement information broadcasting pre-load the cache via

Query Cache

Aggregates or SAP NetWeaver BIA InfoCube

(if BIA is used, InfoCube data on database is not read)

SAP 2007 / Page 103

Other Performance Options

Modeling options
MultiProvider (semantic) partitioning Line-item dimensions

Database features
Indexing Database Statistics

SAP 2007 / Page 104

Compression

Compression
Move data from F to E fact table Compression usually reduces the number of records by combining records with the same key that has been loaded in separate requests When dealing with non-cumulative key figures, it is highly recommended to regularly compress (also when using SAP NetWeaver BI Accelerator) Double fact table As InfoPackages are added, F F Table fact table partitions are created Up loa Request Information d Typically small F - Table Optimised for Loading REQUEST No. Time Material Sales and Deleting E Table Fact Table Compression Optimised for Queries E - Table Typically large

SAP 2007 / Page 105

User-defined DB Partitioning (depending on the DBMS) But: no information on requests

REQUEST No. Time Material Sales

Copyright 2007 SAP AG All rights reserved

No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP AG. The information contained herein may be changed without prior notice. Some software products marketed by SAP AG and its distributors contain proprietary software components of other software vendors. SAP, R/3, mySAP, mySAP.com, xApps, xApp, SAP NetWeaver, Duet, Business ByDesign, ByDesign, PartnerEdge and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP AG in Germany and in several other countries all over the world. All other product and service names mentioned and associated logos displayed are the trademarks of their respective companies. Data contained in this document serves informational purposes only. National product specifications may vary. The information in this document is proprietary to SAP. This document is a preliminary version and not subject to your license agreement or any other agreement with SAP. This document contains only intended strategies, developments, and functionalities of the SAP product and is not intended to be binding upon SAP to any particular course of business, product strategy, and/or development. SAP assumes no responsibility for errors or omissions in this document. SAP does not warrant the accuracy or completeness of the information, text, graphics, links, or other items contained within this material. This document is provided without a warranty of any kind, either express or implied, including but not limited to the implied warranties of merchantability, fitness for a particular purpose, or non-infringement. SAP shall have no liability for damages of any kind including without limitation direct, special, indirect, or consequential damages that may result from the use of these materials. This limitation shall not apply in cases of intent or gross negligence. The statutory liability for personal injury and defective products is not affected. SAP has no control over the information that you may access through the use of hot links contained in these materials and does not endorse your use of third-party Web pages nor provide any warranty whatsoever relating to third-party Web pages

Weitergabe und Vervielfltigung dieser Publikation oder von Teilen daraus sind, zu welchem Zweck und in welcher Form auch immer, ohne die ausdrckliche schriftliche Genehmigung durch SAP AG nicht gestattet. In dieser Publikation enthaltene Informationen knnen ohne vorherige Ankndigung gendert werden. Einige von der SAP AG und deren Vertriebspartnern vertriebene Softwareprodukte knnen Softwarekomponenten umfassen, die Eigentum anderer Softwarehersteller sind. SAP, R/3, mySAP, mySAP.com, xApps, xApp, SAP NetWeaver, Duet, Business ByDesign, ByDesign, PartnerEdge und andere in diesem Dokument erwhnte SAP-Produkte und Services sowie die dazugehrigen Logos sind Marken oder eingetragene Marken der SAP AG in Deutschland und in mehreren anderen Lndern weltweit. Alle anderen in diesem Dokument erwhnten Namen von Produkten und Services sowie die damit verbundenen Firmenlogos sind Marken der jeweiligen Unternehmen. Die Angaben im Text sind unverbindlich und dienen lediglich zu Informationszwecken. Produkte knnen lnderspezifische Unterschiede aufweisen. Die in diesem Dokument enthaltenen Informationen sind Eigentum von SAP. Dieses Dokument ist eine Vorabversion und unterliegt nicht Ihrer Lizenzvereinbarung oder einer anderen Vereinbarung mit SAP. Dieses Dokument enthlt nur vorgesehene Strategien, Entwicklungen und Funktionen des SAP-Produkts und ist fr SAP nicht bindend, einen bestimmten Geschftsweg, eine Produktstrategie bzw. -entwicklung einzuschlagen. SAP bernimmt keine Verantwortung fr Fehler oder Auslassungen in diesen Materialien. SAP garantiert nicht die Richtigkeit oder Vollstndigkeit der Informationen, Texte, Grafiken, Links oder anderer in diesen Materialien enthaltenen Elemente. Diese Publikation wird ohne jegliche Gewhr, weder ausdrcklich noch stillschweigend, bereitgestellt. Dies gilt u. a., aber nicht ausschlielich, hinsichtlich der Gewhrleistung der Marktgngigkeit und der Eignung fr einen bestimmten Zweck sowie fr die Gewhrleistung der Nichtverletzung geltenden Rechts. SAP bernimmt keine Haftung fr Schden jeglicher Art, einschlielich und ohne Einschrnkung fr direkte, spezielle, indirekte oder Folgeschden im Zusammenhang mit der Verwendung dieser Unterlagen. Diese Einschrnkung gilt nicht bei Vorsatz oder grober Fahrlssigkeit. Die gesetzliche Haftung bei Personenschden oder die Produkthaftung bleibt unberhrt. Die Informationen, auf die Sie mglicherweise ber die in diesem Material enthaltenen Hotlinks zugreifen, unterliegen nicht dem Einfluss von SAP, und SAP untersttzt nicht die Nutzung von Internetseiten Dritter durch Sie und gibt keinerlei Gewhrleistungen oder Zusagen ber Internetseiten Dritter ab. Alle Rechte vorbehalten.

SAP 2007 / Page 106