presented by

Marco Russo
marco@sqlbi.com

sqlbi.com sqlbi.com

Who am I Latest conferences

BI Experts and Consultants

Problem Solving

Complex Project Assistance

DataWarehouse Assesments and Development

Courses, Trainings and Workshops

Microsoft Business Intelligence Partners

Book Writers

PASS Europe 2009 – Neuss – Germany

PASS 2009 – Seattle – USA

SQL Conference 2010 – Milan – Italy

Teched 2010 – New Orleans – USA

24 Hours of PASS 2010 – Online

PASS 2010 – Seattle – USA

sqlbi.com

Agenda

What is BISM?

What is going to change?

The road to BISM: PowerPivot

BISM vs UDM
 Patterns design comparison
 Dimensional vs Relational Modeling

What to do today?

sqlbi.com sqlbi.com

sqlbi.com 1
 BI Semantic Model
BISM – BI Semantic Model
Third-Party BI Reporting Excel PowerPivot SharePoint
Services Dashboards &

One semantic model for Business Intelligence Applications Reports Workbooks Applications
Scorecards
BI Applications

 For reporting, analytics, scorecards, dashboards

 Unification of OLAP (UDM) and report (SMDL) models
Relational Multidimensional Data Model

Can be queried through:
 MDX DAX MDX Business Logic
 DAX
 SQL (?) VertiPaq Realtime Data Access

Different engine than SSAS UDM

 Vertipaq – in-memory columnar database
OData
 Realtime – query passthrough to SQL Server Files Feeds
Data Sources

PowerPivot + Analysis Services

Seamless Transition Across BI Spectrum
Architecture Analysis Services

Personal BI Team BI Organizational BI SharePoint UDM

Internet Explorer
Empowered Aligned
Excel Services

My Context Our Context The Org’s Context

BI solution created BI Solution created by BI Solution created by

Reporting Services
by user. Context is power user. Context IT, Established corporate PowerPivot for Excel
xlsx
only for user & exists is for a small team & context & is reusable, BI Development
as document. it’s managed on a scalable and backed up. Studio / Project
Juneau
server. PowerPivot for
SharePoint
(Analysis Services) Analysis Services
Excel

BISM
xlsx

Third Party Apps

PowerPivot For Excel PowerPivot For SharePoint Analysis Services

Personal BI Team BI Organizational BI

Instance level choice Road to BISM: PowerPivot

A SSAS instance can be:
PowerPivot for Excel
 UDM (the classical one)  Free AddIn for Microsoft Excel 2010
 PowerPivot for SharePoint (since SQL2008 R2)  Available since June 2010
 BISM (only Denali)  Different releases for 32/64 bits

You can install Denali with different roles in side-  Contains the Vertipaq Engine
by-side instances on the same server  SSAS running in process with Excel

PowerPivot for SharePoint
 Vertipaq running server side
 Integrated with SharePoint

sqlbi.com 2
 Road to BISM: Denali Vertipaq: change your mind!

Denali
Vertipaq in Denali
 New release of SQL Server  Not a simple evolutionary step
 SSAS / Vertipaq running server side  It is a complete new environment
 Works standalone (PowerPivot for SP cannot)  New calculation paradigms
 Vertipaq / SSAS selected at instance level  New data modeling techniques
 New release of PowerPivot  New formula authoring experience
 Parent-child hierarchies, perspectives, KPIs
Want to learn BISM?
 New DAX functions (like Ranking)
 Forget MDX

PowerPivot 1.0 today:  Forget SQL
 Is the road to learn new Denali features  Then, Learn DAX. ☺

The role of DAX in BISM What is changing?

DAX is DENALI SQL 2008 R2
 A formula language (like in PowerPivot 1.0) BISM UDM

 A query language (new to BISM) DAX MDX

RELATIONAL MULTIDIMENSIONAL
 Syntax based on CALCULATETABLE (existing in 1.0)
 But now you can define result shape (ADDCOLUMNS) IN MEMORY ON DISK

 And control the order of results (EVALUATE / ORDER BY) FAST VERTIPAQ ENGINE
LEAF LEVEL COMPUTATION
 The fastest way to query BISM
FILTER / ROW CONTEXT
 MDX available, but might add some overhead
SELF SERVICE + CORPORATE
EASY DATA INTEGRATION
AGGREGATIONS
CALCULATED MEMBERS
MDX FUNCTIONS, TUPLES ETC
CORPORATE ONLY
BI PROFESSIONALS NEEDED

Relational vs Multidimensional Complexity in BISM

UDM
BISM: simple data model
 Facts, Dimensions, Bridges, Snowflakes
Complexity is moved into:
 Relationships: Regular, Referenced, M2M  Calculated columns
 Hierarchies, Attribute Relationships  Measures

BISM  DAX, DAX, DAX, DAX, DAX and… DAX!
 Tables
Every column is a dimensional attribute
 Relationships (1:N, One Column only)
Every column is a measure

sqlbi.com 3
 Semi Additive Measures
A SMALL EXAMPLE OF HOW TO AGGREGATE DATA IN DAX

Additive Measure
 SUM over all dimensions

Non Additive Measure
 Different function over all dimensions
 Example: average of the sale price

Semi Additive Measure
 SUM over some dimensions
 Different function over other dimensions
 Time is the standard exception for aggregations
 Examples
 Warehouse stocking
sqlbi.com  Current account balance

Current Account Balance Semi Additive Measures in UDM

Aggregation function:
 LastChild
 LastNonEmpty

Performances not very impressive

Aggregations always needed

The aggregation is handled by the data model

In DAX, there is no default aggregation function.

Month level correct
Thus, DAX is needed.

Quarter level wrong

Year level wrong

Semi Additive Measures In Memory vs On Disk

Create a calendar table

Add a new measure:
 CALCULATE: to set the filter
 LASTDATE: to find the last child
LastBalance = CALCULATE (
SUM( Balances[Balance] ),
LASTDATE(Date[Date])
)

UDM
 Aggregations
 Usage Based Optimization
 Aimed to optimize Storage Engine

BISM
 No Storage Engine: everything in memory
 No aggregations, fast in memory computation
 Non standard aggregations = Measures

sqlbi.com 4
 Calculated Columns / Measures Calculated Columns / Measures

Measures
GrossMargin = SalesAmount – ProductCost

 Similar to MDX Calculated Members  Calculated column
 Work on aggregations
GrossMargin%=GrossMargin / SalesAmount
 Might go leaf-level, if needed  Cannot be computed row by row

Calculated columns
Measures needed
 New kind of calculation

 Computed once in the data model

∑ 

Can access the whole data model


 
 ∑  



 Different from calc columns in the UDM DSV 

In Memory vs On Disk Self Service vs Corporate BI

Vertipaq and DAX
Self Service BI
 Fast in memory computations  Creative Solutions in the hand of Power Users
 Leaf level calculations  No need to ask for help to IT
 Calculated columns  Less work for IT

The data model can be different
Example

Less ETL code  Banding of prices

More DAX code
 No locking, No I/O, very fast
 Data is not persisted on disk for other clients

Analysis of product sell price

BANDING IN THE SELF SERVICE BI WORLD

Price changes over time

 Discounts
 Price variations

Continuous dimension

High fragmentation

BANDING
 From 0 to 100 USD
 From 101 to 500
sqlbi.com

sqlbi.com 5
 The quick and dirty solution Banding: a Data Driven Model
= IF (
BandName FromPrice ToPrice
FactResellerSales[DiscountedPrice] <= 5,
"01 LOW", VERY LOW 0 5
IF ( LOW 5 30
FactResellerSales[DiscountedPrice] <=30, MEDIUM 30 100
"02 MEDIUM", HIGH 100 500
IF ( SELECT VERY HIGH 500 9999
FactResellerSales[DiscountedPrice] <=100, P.BandName,
"03 MEDIUM", SUM (S.ExtendedAmount)
IF ( FROM dbo.FactResellerSales S
FactResellerSales[DiscountedPrice] <= 500, JOIN PriceBands P
"04 HIGH", ON S.UnitPrice BETWEEN P.FromPrice AND
"05 VERY HIGH")))) P.ToPrice
GROUP BY
Even if this works… a better data P.BandName
model would be welcome!

Data Model with PowerPivot Band Expansion

BandName Price
BandName FromPrice ToPrice BandName Price
VERY LOW 1
VERY LOW 0 5 VERY LOW 1
SELECT VERY LOW 2
LOW 5 30 VERY LOW 2
P.BandName, VERY LOW 3
MEDIUM 30 100 VERY LOW 3
VERY LOW 4
SUM (S.ExtendedAmount) HIGH 100 500 VERY LOW 4
VERY LOW 5
FROM dbo.FactResellerSales S LOW 6
VERY HIGH 500 9999 VERY LOW 5
LOW 6
JOIN PriceBands P LOW 7
ON S.UnitPrice = P.Price
Improve the Data Model LOW 7
… …
… …
GROUP BY
Cannot do this in PowerPivot
P.BandName
Write VBA code to transform data

Needs some VBA knowledge

Banding With DAX Banding With CALCULATE

Link only the configuration table
Works with the filter context

No need to perform expansion
Leverages
 CALCULATE

The complexity moves to the expression  VALUES
 FILTER

Again, DAX programming needed

= CALCULATE(
VALUES (PriceBands[PriceBand]),
FILTER (
PriceBands,
FactSales[DiscountedPrice] >= PriceBands[MinPrice]
&& FactSales[DiscountedPrice] < PriceBands[MaxPrice]
)
)

sqlbi.com 6
 Aggregations Over Time
SOME INSIGHTS INTO BISM TIME INTELLIGENCE FUNCTIONS

Many interesting aggregations
 YTD: Year To Date
 QTD: Quarter To Date
 MTD: Month To Date

They all need a Calendar Table

And some understanding of CALCULATE ☺

Tool dimension is not affordable
 No DateTool dimension!

sqlbi.com

Year To Date PARALLELPERIOD

CALCULATE( CALCULATE (
SUM ( SalesOrderDetail[LineTotal] ), SUM( SalesOrderDetail[LineTotal] ),
DATESYTD ( OrderDate[Date] ) PARALLELPERIOD
) (OrderDate[Date], -1, YEAR)

DATESYTD: Returns a set of dates )
 From the start of the year
Returns a set of dates (a table)

 Up to the parameter date  Shifted in time

CALCULATE  The whole period is returned, regardless dates in the
first parameter
 Creates a filter contexts
 Perform the SUM operation

Moving Annual Total Counting Working Days

CALCULATE(
How many working days in a date range?
SUM (SalesOrderDetail[LineTotal]),  Easily solved with Calendar table
DATESBETWEEN (
 Define a new column «WorkingDays»
OrderDate[Date],
NEXTDAY(  Aggregate with SUM
SAMEPERIODLASTYEAR(
Handles any date range
LASTDATE( OrderDate[Date] )
)

Works on periods with «holes»
),
Difference between BISM and UDM
LASTDATE( OrderDate[Date] )  UDM solution requires ETL
)  BISM solution can be self-contained
)

sqlbi.com 7
 ABC and Pareto Analysis
CALCULATED COLUMNS AT WORK WITH ABC PARETO ANALYSIS

Pareto principle
 80% of effects come from 20% of the causes

sqlbi.com

ABC and Pareto Analysis ABC Analysis with Excel

The formula is straightforward:

ABC Analysis =IF(H6<=0,7;"A";IF(H6<=0,9;"B";"C"))
 Class A contains items for 70% of total value
 Class B contains items for 20% of total value
 Class C contains items for 10% of total value

100%

50%

0%
0 10 20 30 40 50 60 70 80 90100

ABC Analysis
A COMPLEX EXAMPLE INVOLVING MANY TO MANY RELATIONSHIPS

Calculate the class

SortedWeightProduct =
Product[CumulatedProduct]
/ SUM( Product[SalesAmountProduct] )

[ABC Class Product] =

IF( Product[SortedWeightProduct] <= 0.7, "A",
IF( Product[SortedWeightProduct] <= 0.9, "B", "C" ) )

sqlbi.com

sqlbi.com 8
 Audience Audience: background

Taken from a real world analysis
Info taken from the remote control

Audience data of TV broadcasts
Once a minute
 Simple data model  Individual watching
 Channel watched
 Pretty big database
 Analysis at a very detailed level
Once a day
 All the individuals watching

Current solution
 Detect the influence of each individual
 Ad-hoc software

Individual

Proof of Concept  Categories
 Analysis with PowerPivot and Vertipaq  Targets

Audience: The Data Model Many-to-many relationship

The Individuals table is
filtered to show only
10,000 individuals, then
76,000 the Audience table is read
10,000 only once.

CALCULATE (
…,
FILTER (
VALUES (Individuals),
CALCULATE (
COUNTROWS (
TargetsForIndividuals
1,300,000 ) > 0
10,000 …
66,000,000 )
9,000,000 )

140 1

IS DIMENSIONAL MODELING STILL NECESSARY FOR POWERPIVOT /

Dimensional vs Relational: UDM
BISM?

UDM
 Strong separation between facts and dimensions
 Advanced relationship types
 Great aggregation level computations
 Poor leaf level computation performances

Designed, optimized for dimensional models

No brain option: go dimensional

sqlbi.com

sqlbi.com 9
 Dimensional vs Relational: BISM Dimensional vs Relational: BISM

BISM
Relational Modeling

 No concept of fact and dimension: only tables
 Very simple concept of relationship
 No aggregations, in memory computation
 Great leaf level computation performances

Optimized for simple models

Two options
 Dimensional modeling
 Relational modeling
 Does not mean third normal form!
 Users like facts and dimensions

Denormalization
 Needed for the user experience
 And for performances

Each relationship has a cost, remember it!

BISM: Advanced Relationships BISM: New Power

Advanced Relationships in BISM
 Are not supported
 But can be implemented through DAX
 Between (Banding)
 Many to many (Audience)
 Parent / Child (Naturalization - some support in BISM)
 Reference Dimensions (RELATED)
 LastChild (CALCULATE)
 Fact Relationships (not needed)

Thus, nothing is really missing

Calculated columns
 Materialize complex calculations
 ABC Pareto is a good example (see book/blogs)
 Have access to the whole data model
 Not like calculated columns in UDM DSV!

Linked tables
 Two steps computation

Easy integration with many sources

Power to the end user

BISM: DAX What is changing?

Brand new language DENALI SQL 2008 R2

Mixes programming and querying BISM UDM
DAX MDX

Much cleaner and easier than MDX RELATIONAL MULTIDIMENSIONAL

Designed for efficiency on modern CPU IN MEMORY ON DISK
FAST VERTIPAQ ENGINE AGGREGATIONS

Hard to enter the «DAX Mind»
LEAF LEVEL COMPUTATION CALCULATED MEMBERS
 For MDX and SQL programmers FILTER / ROW CONTEXT MDX FUNCTIONS, TUPLES ETC
 Don’t search same concepts, they are not there! SELF SERVICE + CORPORATE CORPORATE ONLY

Once you master it… awesome! EASY DATA INTEGRATION BI PROFESSIONALS NEEDED

Simple, not easy

sqlbi.com 10
 A NEW BOOK FROM SQLBI

sqlbi.com sqlbi.com

Links

SQLBI Website
www.sqlbi.com

PowerPivot Workshop
www.powerpivotworkshop.com

Marco Russo blog

www.sqlblog.com/blogs/marco_russo

Alberto Ferrari blog

www.sqlblog.com/blogs/alberto_ferrari

For any question contact us at

info@sqlbi.com

sqlbi.com

sqlbi.com 11