Best Practices for Multi-Dimensional Design using Cognos 8 Framework Manager

Bruce Zornes The Boeing Company

Business Intelligence & Competency Center

Copyright Boeing 2007. All rights reserved

Best Practice: Knowledge and Experience

Use Best Practice to reduce cost and produce quality Best Practice means Knowledge about the solution Theory is OK if produces results Use Patterns re-apply Knowledge Experience with the tools How to, what to do, when to do it People have Experience and Knowledge

Copyright Boeing 2007. All rights reserved

Best Practice: Use Design Principals

Design quality in rather than Testing it in Framework Manager model is great place to implement quality design Understand the business case and drivers Understand the data schema - 3NF, star, recursive Understand the technology for optimization Teaming include DBAs and Server Admins Common model and keep it simple Understand your customer reports team too
Copyright Boeing 2007. All rights reserved

Best practices: Measures for success - Project

Success criteria checklist expectations, funding, resources
Complete solution phase the release(s) Our case: 11 business units, 20 cubes, one model, 4 rels Follow a process Dev Test Prod, CM, docs, SQA Design reviews and open communication Customer acceptance requirements met End-user acceptance navigation and UI, portals Cost and schedules hidden cost, unplanned work Reports performance team collaboration Support and operations team training, procedures
Copyright Boeing 2007. All rights reserved

Customer Requirements: Business case

How to produce quality criteria for design?
Design choices: Star schema Three layer design Nice theory Too slow for large data?
Copyright Boeing 2007. All rights reserved

Narrow fact tables Familiar navigation Fast query response rqmt Minimize load time ETL Reporting mart

Best Practice: Framework Model Design

Framework Model Three layer design [Physical Database Layer] SQL data source objects [Business Model Layer] Model objects and building blocks Business rules and data logic [Dimensional Presentation Layer] Multi-dimensional views and format

Copyright Boeing 2007. All rights reserved

Data Design: Parent-Child to Denormal table

PARENT- CHILD tree

Denormalized table for Reporting

Note: Non-proprietary scrubbed test data

Copyright Boeing 2007. All rights reserved

Framework model: Construct model objects

Building blocks: parent-child Query Subjects (QS) and Query Sets
Generation Specific QS Namespace scope Shortcut to physical Explicit relationship QS building block Placeholder Parents QS Naming convention User Naming for Reports
Copyright Boeing 2007. All rights reserved

Folders organize

Query Set Union Leaf members

Framework Model: Core Design Elements

Conformed star schema (multiple fact tables) Parent-child to denormal dimension tables Unbalanced hierarchies Building block Query Subjects (QS) Parent place-holders at each generation Unions via Querysets: QSParent to QSGen Create leaf member level coalesce(case when) Link dimensions to facts at leaf level Narrow fact table - one key per dim Create dimensional views from building blocks
Copyright Boeing 2007. All rights reserved

Framework model: with SQL Generation

Building blocks: Generate Top-of-Tree SQL
with E123_SCENARIO21 as (select E123_SCENARIO.ESB_FKID, E123_SCENARIO.ESB_PARENT, E123_SCENARIO.ESB_CHILD, E123_SCENARIO.ALIAS1, E123_SCENARIO.DATASTORE, E123_SCENARIO.GENERATION from "FabFBI Datasource"..AFAB.E123_SCENARIO), ScenarioGen2 as (select E123_SCENARIO21.ESB_FKID as ESB_FKID, E123_SCENARIO21.ESB_PARENT as PARENT, E123_SCENARIO21.ESB_CHILD as CHILD, E123_SCENARIO21.ALIAS1 as ALIAS1, E123_SCENARIO21.DATASTORE as DATASTORE, E123_SCENARIO21.GENERATION as GENERATION from E123_SCENARIO21 where (E123_SCENARIO21.GENERATION in ('2')) ),

Cognos Connection

Primary reference to PARENTCHILD physical table

Schema owner Top-of-Tree constraint

Copyright Boeing 2007. All rights reserved

Framework model: SQL Magic for Placeholders

Building blocks: QuerySet union with placeholders SQL
ScenarioGen23Union517(ESB_FKID, PARENT, CHILD, ALIAS1, DATASTORE, GENERATION) as ((select E123_SCENARIO15.ESB_FKID as ESB_FKID, E123_SCENARIO15.ESB_CHILD as PARENT, cast( null as varchar (80)) as CHILD, cast(null as varchar (80)) as ALIAS1, cast(null as varchar (80)) as DATASTORE, E123_SCENARIO15.GENERATION as GENERATION from E123_SCENARIO15 where (E123_SCENARIO15.GENERATION in ('2')) ) UNION (select E123_SCENARIO16.ESB_FKID as ESB_FKID, E123_SCENARIO16.ESB_PARENT as PARENT, E123_SCENARIO16.ESB_CHILD as CHILD, E123_SCENARIO16.ALIAS1 as ALIAS1, E123_SCENARIO16.DATASTORE as DATASTORE, E123_SCENARIO16.GENERATION as GENERATION from E123_SCENARIO16 where (E123_SCENARIO16.GENERATION in ('3')) ) ),

Promote CHILD as PARENT NULL CHILD Placeholder Parents at Gen 2 UNION to Members at Gen 3

NULL types coerced

Copyright Boeing 2007. All rights reserved

Framework model: SQL tricks for Leaf members

Building blocks: coalesce(case(tree)) as LEAF_MEMBER
ScenarioLeaf as (select coalesce( case when (ScenarioGen34Union18.CHILD is NULL) then null else ScenarioGen34Union18.ESB_FKID end ,case when (ScenarioGen23Union19.CHILD is NULL) then null else ScenarioGen23Union19.ESB_FKID end ,case when (ScenarioGen2.CHILD is NULL) then null else ScenarioGen2.ESB_FKID end ) as LEAF_FKID, coalesce( case when (ScenarioGen34Union18.CHILD is NULL) then null else ScenarioGen34Union18.PARENT end ,case when (ScenarioGen23Union19.CHILD is NULL) then null else ScenarioGen23Union19.PARENT end ,case when (ScenarioGen2.CHILD is NULL) then null else ScenarioGen2.PARENT end as LEAF_PARENT, coalesce(ScenarioGen34Union18.CHILD ,ScenarioGen23Union19.CHILD ,ScenarioGen2.CHILD as LEAF_CHILD,

Case switch to NOT promote placeholders

Walk trees in reverse 3 2 1

)

Build leaf members

Copyright Boeing 2007. All rights reserved

Framework model: Construct leaf and case logic

Building blocks: final assembly leaf model objects
Placeholders promote members No cross joins allowed! Model objects need relationships Leaf links to fact tables Coalesce Case switch

Coalesce is tree walker Scan right to first non-null

Conditional members

Copyright Boeing 2007. All rights reserved

Framework model: SQL loop joins

Building blocks: SQL loop joins w/placeholder filter
Join-Join causes repeating output
My tricky SQL? (really just ANSI sql
F041-03 F041-05)

ScenarioLeaf as . from ScenarioGen2 left outer join ScenarioGen23Union517 ScenarioGen23Union19 on (ScenarioGen2.CHILD = ScenarioGen23Union19.PARENT) left outer join ScenarioGen34Union214 ScenarioGen34Union18 on (ScenarioGen23Union19.CHILD = ScenarioGen34Union18.PARENT) where (not (coalesce( ScenarioGen34Union18.CHILD ,ScenarioGen23Union19.CHILD ,ScenarioGen2.CHILD) is NULL)) )

Put em in & take em out Seems like extra work for unbalanced

Copyright Boeing 2007. All rights reserved

Framework model: Determinants fix multi-grain

Building blocks: use determinants on leaf data groups
Grouped by generation (nice to have: determinant filters)

Leaf model objects are generation specific

Naming suffix 1 2 3 CHILD1 is top-of-tree

Uniqueness based on composite business keys

Copyright Boeing 2007. All rights reserved

Framework model: Dimensional construction Presentation Layer: connecting the pieces

Dimensional model construction from Model objects
Hierarchy name Member levels Unique leaf level Drag drop & test Set businesskey and caption

Build presentation layer from BP model building blocks

Copyright Boeing 2007. All rights reserved

Framework model: Test in Query Studio

Published package: validate unbalanced structure
First look at dimensional output Unbalanced member names promoted to leaf_level Leaf members connect to facts
Note: Non-proprietary scrubbed test data
Copyright Boeing 2007. All rights reserved

Suppress first two layers in final publish

All reports connect to presentation layer

Framework model: Design challenges met

Interface to legacy system OLAP data cubes Iterative design not all data structure understood ETL expensive and difficult Minimize business rules in ETL Narrow fact tables - faster than wide Link all tables via SID (business keys) High volume data (200 Million rows) partitioned Users expect 2 - 4 seconds click report response
Copyright Boeing 2007. All rights reserved

Framework model: Test in Analysis Studio

Verify to legacy system OLAP data cubes
User familiar names Ready to build reports

Is it fast In AS 10-30 secs In QS 3-5 secs

Do aggregate values match

Mixed grains

Copyright Boeing 2007. All rights reserved

Note: Non-proprietary scrubbed test data - all aggregate values are row counts

Example 1 of Unbalanced Hierarchy

Unbalanced Dimensions: promotion of members to leaf level Facts link to leaf member level
Measures dimension
Fringe Dollar Adjustments

Data Rollups
Regular Dollars

Mixedgrain data

Link by Root_child
Note: Non-proprietary scrubbed test data
Copyright Boeing 2007. All rights reserved

Example 2 of Unbalanced Hierarchy - SHARED

Unbalanced Dimensions: complex network members SHARED dimension members have root_child
Cloned parents and child data

Cloned parent
Cloned child data from root_child

Attached to different ancestor

Note: Non-proprietary scrubbed test data
Copyright Boeing 2007. All rights reserved

Design strategy: Model meets the needs

Who: model end-user vs. reports end-user Adapt to Teams, Roles and Org maturity
Three layer design only two Developer model not end-user 3NF structure Denormalize in rdbms Priorities may change Key success drivers Compromise and validate Rely on principals

Copyright Boeing 2007. All rights reserved

Best practices: Measures for success check!

So howd we do? Success criteria / performance indicators
First release due: July 2006, Delivered first reports: March 2007 Followed process: Dev Test Prod, docs Tech reviews: hold daily status with PMs Customer and End-user acceptance: improved their data quality Really appreciate common business model and Cognos rpts Cost and schedules: re-planned to meet realities Performance: designed-in, mat views added, DBA involved Support and operations: OJT, handover challenges, teaming improvements, working together, find-a-way Users: want more BI reports and faster deployments
Copyright Boeing 2007. All rights reserved

Evaluation Forms

Each completed evaluation qualifies you to win one of five $100 American Express gift certificates given away daily. Complete evaluations for every session you attend and qualify to win an additional $500! An overall conference survey will be available at Cognos Central on Thursday morning, and will also be emailed to you. Complete the survey by May 25, 2007, and you'll qualify to win $500!
Copyright Boeing 2007. All rights reserved