You are on page 1of 19

Data Management

& Warehousing
http://www.datamgmt.com

An introduction to
Process Neutral
Data Modelling
© 2006 Data Management & Warehousing UKOUG: Business Intelligence & Reporting Tools SIG Page 1 of 19
Speaker: David M. Walker Institute of Physics, 76 Portland Place, London 31 January 2006
Data Management & Warehousing

•  Founded 1995 by David Walker
–  Operates with up to 15 consultants
•  Specialists in Enterprise Data Warehousing
•  Clients have included:
–  Manufacturing: Diageo, Mars ISI
–  Retail: Albert Heijn, Nectar
–  Financial: Virgin Money
–  Transport: Network Rail, Swissair
–  Telco: Turkcell, Swisscom Mobile, Telkom SA

© 2006 Data Management & Warehousing UKOUG: Business Intelligence & Reporting Tools SIG Page 2 of 19
Speaker: David M. Walker Institute of Physics, 76 Portland Place, London 31 January 2006
What is Process Neutral Modelling ?

•  A method of designing a data model for a data
warehouse that is less affected by changes in
source system and/or business process
•  A technique that incorporates the metadata
within the data model (in a similar way to XML
which incorporates metadata in a data file)
•  A consistent, self similar modelling method that
allows easy model management in data
warehouses

© 2006 Data Management & Warehousing UKOUG: Business Intelligence & Reporting Tools SIG Page 3 of 19
Speaker: David M. Walker Institute of Physics, 76 Portland Place, London 31 January 2006
Where would you use it ?

•  Data Warehouses that:
–  Feed multiple data marts
–  Have many source systems that are poorly integrated
–  Are in organisations undergoing large business process change
–  Support a recognised need for integrated business intelligence

•  But not in organisations that:
–  are small and can’t afford Enterprise Data Warehousing
–  have a few or one source system with little external data
–  have very stable business processes
–  want to build an Online Transaction Processing (OLTP) Systems
for reporting
© 2006 Data Management & Warehousing UKOUG: Business Intelligence & Reporting Tools SIG Page 4 of 19
Speaker: David M. Walker Institute of Physics, 76 Portland Place, London 31 January 2006
Overcomes Some DWH Requirements Issues

•  Stops the need to closely define certain things
from the requirements in the data model e.g.
•  Define CUSTOMER
–  Marketing say it is everyone they communicate with
–  Sales say it is everyone in their prospect database.
–  Customer Support say it is people who have bought
the product
–  Service Team say it is people who have a support
contract

© 2006 Data Management & Warehousing UKOUG: Business Intelligence & Reporting Tools SIG Page 5 of 19
Speaker: David M. Walker Institute of Physics, 76 Portland Place, London 31 January 2006
Major Entities

•  Rules
–  Lifetime value attributes
only
–  Always has a start date
and an optional end date
•  Examples
–  Party
–  Geography
–  Calendar
–  Electronic Address
–  Product

© 2006 Data Management & Warehousing UKOUG: Business Intelligence & Reporting Tools SIG Page 6 of 19
Speaker: David M. Walker Institute of Physics, 76 Portland Place, London 31 January 2006
Major Entity Types

•  Rules
–  List of valid types and when they are valid (metadata)
•  Examples
–  Party
•  Individual, Sole Trader, Partnership, Ltd Co, Plc, Trust
–  Geography
•  PAF Address, Co-ordinate Point

© 2006 Data Management & Warehousing UKOUG: Business Intelligence & Reporting Tools SIG Page 7 of 19
Speaker: David M. Walker Institute of Physics, 76 Portland Place, London 31 January 2006
Major Entity Properties

•  Rules
–  Attributes of the Major Entity that
change over time listed in the ‘Type
table’ and their association with the
major entity
•  Examples
–  Party
•  Individual: Marital Status, Income
•  Plc: Turnover, Number of employees
© 2006 Data Management & Warehousing UKOUG: Business Intelligence & Reporting Tools SIG Page 8 of 19
Speaker: David M. Walker Institute of Physics, 76 Portland Place, London 31 January 2006
Major Entity Events

•  Rules
–  Things that happen to a
major entity
•  Examples
–  Party
•  Individual: Marriage
–  Address
•  Change of use approved

© 2006 Data Management & Warehousing UKOUG: Business Intelligence & Reporting Tools SIG Page 9 of 19
Speaker: David M. Walker Institute of Physics, 76 Portland Place, London 31 January 2006
Major Entity Links

•  Rules
–  Relates to entries in a major
entity, and relationship is
defined by the type table
•  Examples
–  Party
•  Individual 1 is married to
individual 2
•  Individual 1 is employed by
Organisation 3
© 2006 Data Management & Warehousing UKOUG: Business Intelligence & Reporting Tools SIG Page 10 of 19
Speaker: David M. Walker Institute of Physics, 76 Portland Place, London 31 January 2006
Major Entity Segments

•  Rules
–  Creates a collection of entries from a
major entity
•  Examples
–  Party
•  Marketing Group 1: Males >40 with 1 or
more children (data derived from the
other tables, e.g. properties and links)

© 2006 Data Management & Warehousing UKOUG: Business Intelligence & Reporting Tools SIG Page 11 of 19
Speaker: David M. Walker Institute of Physics, 76 Portland Place, London 31 January 2006
The Major Entity Collection

© 2006 Data Management & Warehousing UKOUG: Business Intelligence & Reporting Tools SIG Page 12 of 19
Speaker: David M. Walker Institute of Physics, 76 Portland Place, London 31 January 2006
Major Entity / Major Entity History

•  Rules •  Examples
–  Relates two –  Party / Address
different major •  Individual 1 lives at
entities via a Address 2
history type •  Individual 3 works at
Address 4

© 2006 Data Management & Warehousing UKOUG: Business Intelligence & Reporting Tools SIG Page 13 of 19
Speaker: David M. Walker Institute of Physics, 76 Portland Place, London 31 January 2006
Occurrences and Major Entities

•  Rules •  Examples
–  These are the –  Sales
tables with define •  Party 1 is supplier
interactions •  Party 2 is the
between all the customer
major entities •  Address 3 is the
store location
•  Product 4 is item
purchased
© 2006 Data Management & Warehousing UKOUG: Business Intelligence & Reporting Tools SIG Page 14 of 19
Speaker: David M. Walker Institute of Physics, 76 Portland Place, London 31 January 2006
Key Elements

•  Self Similar modelling
–  All _TYPE tables have the same structure, etc.
–  Naming conventions are consistent everywhere
•  Insert ‘heavy’ / Update ‘light’
–  Most ETL will result in an insert, there will be very few updates
•  Manages ‘Slowly Changing Dimensions’
–  Inherent in the Major Entity Collection
–  Significantly reduces overhead in the Data Mart build
•  Data Driven
–  Types provide metadata
•  Natural Star Schemas
–  Occurrences will map to FACTS, Major Entity Collections will
collapse into DIMENSIONS

© 2006 Data Management & Warehousing UKOUG: Business Intelligence & Reporting Tools SIG Page 15 of 19
Speaker: David M. Walker Institute of Physics, 76 Portland Place, London 31 January 2006
Pros & Cons

•  Development Cost front-loaded
–  Most of the costs are in the early part of the (ETL)
development, later stages are then quicker and faster.
This will put some organisations off
•  Pivoting Data vs. Slowly Changing Dimensions
–  Questions about the cost of loading ‘property tables’
and ‘pivoting’ data. In reality this is easily offset by the
extra code and effort of managing slowly changing
dimensions

© 2006 Data Management & Warehousing UKOUG: Business Intelligence & Reporting Tools SIG Page 16 of 19
Speaker: David M. Walker Institute of Physics, 76 Portland Place, London 31 January 2006
Pros & Cons (cont.)

•  Two stage process: Source -> TR - Mart
–  Design patterns exist to mitigate this
–  Allows loading whilst users continue to work
–  Allows for the development of flip-flop marts
•  Larger Initial Data Volumes
–  But smaller over the long term due to data sparsity

© 2006 Data Management & Warehousing UKOUG: Business Intelligence & Reporting Tools SIG Page 17 of 19
Speaker: David M. Walker Institute of Physics, 76 Portland Place, London 31 January 2006
Is this all there is to it ?

•  At a high level – YES
•  BUT:
–  There are methods for dealing with data quality
–  Special case methods for some lifetime attributes
•  e.g. Handling women changing their names at marriage
–  Insert/Update methods for performance
–  Design Patterns for implementation
–  Other detailed techniques
•  This talk could only ever be:
“An introduction to
Process Neutral Data Modelling”
© 2006 Data Management & Warehousing UKOUG: Business Intelligence & Reporting Tools SIG Page 18 of 19
Speaker: David M. Walker Institute of Physics, 76 Portland Place, London 31 January 2006
Data Management & Warehousing

Thank you !

•  For more information:
–  Visit our website at http://www.datamgmt.com
–  Call us on 07050 028 911
–  E-mail davidw@datamgmt.com

Winning Teams - Great Team Players

Data Management & Warehousing are proud player sponsors for the 2005/06 season of
Joe Worsley, utility back row with the English Rugby Premiership Champions London Wasps.

Joe has helped London Wasps win the Zurich Premiership in 2002-03, 2003-04 and 2004-05
©as
2006well as the Heineken
Data Management & Warehousing Cup in 2003-04. Joe was
UKOUG: Business also
Intelligence a member
& Reporting Tools SIG of the England World Cup squad
Page 19 of 19
Speaker: David M. Walker Institute of Physics, 76 Portland Place, London 31 January 2006
and was awarded an MBE by the Queen.