White Paper - Process Neutral Data Modelling© 2009 Data Management & WarehousingPage 2
Table of Contents
Table of Contents......................................................................................................................2
Synopsis....................................................................................................................................4
Intended Audience....................................................................................................................4
About Data Management & Warehousing.................................................................................4
Introduction................................................................................................................................5
The Problem..............................................................................................................................6
The Example Company.........................................................................................................6
The Real World.....................................................................................................................9
The Customer Paradigm.........................................................................................................10
Requirements of a Data Warehouse Data Model....................................................................12
Assumptions........................................................................................................................12
Requirements......................................................................................................................12
The Data Model.......................................................................................................................14
Major Entities......................................................................................................................14
Type Tables........................................................................................................................17
Band Tables........................................................................................................................19
Property Tables...................................................................................................................20
Event Tables.......................................................................................................................22
Link Tables..........................................................................................................................23
Segment Tables..................................................................................................................24
The Sub-Model........................................................................................................................25
History Tables.....................................................................................................................26
Occurrences and Transactions...........................................................................................27
Implementation Issues............................................................................................................33
The ‘Party’ Special Case.....................................................................................................33
Partitioning..........................................................................................................................35
Data Cleansing....................................................................................................................36
Null Values..........................................................................................................................36
Indexing Strategy................................................................................................................36
Enforcing Referential Integrity.............................................................................................36
Data Insert versus Data Update..........................................................................................37
Row versus Set Based Loading in ETL...............................................................................37
Disk Space Utilisation.........................................................................................................38
Implementation Effort..........................................................................................................38
Data Commutativity.................................................................................................................39
Data Model Explosion and Compression................................................................................40
How big does the data model get?......................................................................................40
Can the data model be compressed?.................................................................................40
Which Results to Store?..........................................................................................................41
The Holistic Approach.............................................................................................................42
Summary.................................................................................................................................43
Appendix 1 – Data Modelling Standards.................................................................................44
General Conventions..........................................................................................................44
Table Conventions..............................................................................................................44
Column Conventions...........................................................................................................46
Index Conventions..............................................................................................................50
Standard Table Constructs.................................................................................................50
Sequence Numbers For Primary Keys................................................................................52
Appendix 2 – Understanding Hierarchies................................................................................53
Sales Regions.....................................................................................................................53
Internal Organisation Structure...........................................................................................53
Appendix 3 – Industry Standard Data Models.........................................................................55
Appendix 4 – Information Sparsity..........................................................................................57
Appendix 5 – Set Processing Techniques...............................................................................59
Appendix 6 – Standing on the shoulders of giants..................................................................60
Leave a Comment