Professional Documents
Culture Documents
SQL Master NG PDF
SQL Master NG PDF
Strike Back
Bill Karwin
1
Monday, April 20, 2009 1
Antipattern Categories
Database Design Database Creation
Antipatterns Antipatterns
CREATE TABLE BugsProducts (
bug_id INTEGER REFERENCES Bugs,
product VARCHAR(100) REFERENCES Products,
PRIMARY KEY (bug_id, product)
);
Query Application
Antipatterns Antipatterns
SELECT b.product, COUNT(*) $dbHandle = new PDO(‘mysql:dbname=test’);
FROM BugsProducts AS b $stmt = $dbHandle->prepare($sql);
GROUP BY b.product; $result = $stmt->fetchAll();
2
Monday, April 20, 2009 2
Antipattern Categories
Database Design Database Creation
Antipatterns Antipatterns
CREATE TABLE BugsProducts (
bug_id INTEGER REFERENCES Bugs,
product VARCHAR(100) REFERENCES Products,
PRIMARY KEY (bug_id, product)
);
Query Application
Antipatterns Antipatterns
SELECT b.product, COUNT(*) $dbHandle = new PDO(‘mysql:dbname=test’);
FROM BugsProducts AS b $stmt = $dbHandle->prepare($sql);
GROUP BY b.product; $result = $stmt->fetchAll();
3
Monday, April 20, 2009 3
Database Design Antipatterns
1. Metadata Tribbles
2. Entity-Attribute-Value
3. Polymorphic Associations
4. Naive Trees
4
Monday, April 20, 2009 4
Metadata Tribbles
I want these things off the ship. I don’t care if it takes
every last man we’ve got, I want them off the ship.
— James T. Kirk
5
Monday, April 20, 2009 5
Metadata Tribbles
6
Monday, April 20, 2009 6
Metadata Tribbles
7
Monday, April 20, 2009 7
Metadata Tribbles
8
Monday, April 20, 2009 8
Metadata Tribbles
9
Monday, April 20, 2009 9
Metadata Tribbles
10
Monday, April 20, 2009 10
Metadata Tribbles
11
Monday, April 20, 2009 11
Metadata Tribbles
• Solution #1: use horizontal partitioning
• Physically split, while logically whole
BUGS
(2007)
BUGS
BUGS (2008)
BUGS
(2009)
12
Monday, April 20, 2009 12
Metadata Tribbles
• Solution #2: use vertical partitioning
• Move bulky and seldom-used columns to a
second table in one-to-one relationship
( 1 .. 1 )
PRODUCTS INSTALLERS
13
Monday, April 20, 2009 13
Metadata Tribbles
14
Monday, April 20, 2009 14
Metadata Tribbles
• Solution #3: add a dependent table
CREATE TABLE BugsProducts (
bug_id
BIGINT REFERENCES bugs,
product_id
BIGINT REFERENCES products,
PRIMARY KEY (bug_id, product_id)
);
(1 .. *) BUGS
BUGS PRODUCTS
15
Monday, April 20, 2009 15
Entity-Attribute-Value
If you try and take a cat apart to see how it works,
the first thing you have on your hands is a non-working cat.
— Richard Dawkins
16
Monday, April 20, 2009 16
Entity-Attribute-Value
• Objective: make a table with a variable
set of attributes
bug_id bug_type priority description severity sponsor
17
Monday, April 20, 2009 17
Entity-Attribute-Value
18
Monday, April 20, 2009 18
Entity-Attribute-Value
19
Monday, April 20, 2009 19
Entity-Attribute-Value
• Difficult to rely on attribute names
20
Monday, April 20, 2009 20
Entity-Attribute-Value
• Difficult to enforce data type integrity
21
Monday, April 20, 2009 21
Entity-Attribute-Value
22
Monday, April 20, 2009 22
Entity-Attribute-Value
• Difficult to enforce referential integrity for
attribute values
bug_id attr_name attr_value
1234 priority new
3456 priority fixed
5678 priority banana
23
Monday, April 20, 2009 23
Entity-Attribute-Value
• Difficult to reconstruct a row of attributes:
SELECT b.bug_id, need one JOIN
e1.attr_value AS created_date, per attribute
e2.attr_value AS priority,
e3.attr_value AS description,
e4.attr_value AS status,
e5.attr_value AS reported_by
FROM Bugs b
LEFT JOIN eav e1 ON (b.bug_id = e1.bug_id AND e1.attr_name = ‘created_date’)
LEFT JOIN eav e2 ON (b.bug_id = e2.bug_id AND e2.attr_name = ‘priority’)
LEFT JOIN eav e3 ON (b.bug_id = e3.bug_id AND e3.attr_name = ‘description’)
LEFT JOIN eav e4 ON (b.bug_id = e4.bug_id AND e4.attr_name = ‘status’)
LEFT JOIN eav e5 ON (b.bug_id = e5.bug_id AND e5.attr_name = ‘reported_by’);
24
Monday, April 20, 2009 24
Entity-Attribute-Value
25
Monday, April 20, 2009 25
Entity-Attribute-Value
• Solution #1: Single Table Inheritance
• One table with many columns
26
Monday, April 20, 2009 26
Entity-Attribute-Value
• Solution #2: Concrete Table Inheritance
• Define similar tables for similar types
27
Monday, April 20, 2009 27
Entity-Attribute-Value
• Solution #2: Concrete Table Inheritance
• Use UNION to search both tables:
SELECT * FROM (
SELECT issue_id, description FROM Bugs
UNION ALL
SELECT issue_id, description FROM Features
) unified_table
WHERE description LIKE ...
28
Monday, April 20, 2009 28
Entity-Attribute-Value
• Solution #3: Class Table Inheritance
• Common columns in base table
• Subtype-specific columns in subtype tables
CREATE TABLE Bugs ( CREATE TABLE Features (
issue_id
BIGINT PRIMARY KEY,
issue_id
BIGINT PRIMARY KEY,
severity
VARCHAR(20),
sponsor
VARCHAR(100),
FOREIGN KEY (issue_id)
FOREIGN KEY (issue_id)
REFERENCES Issues (issue_id)
REFERENCES Issues (issue_id)
); );
30
Monday, April 20, 2009 30
Entity-Attribute-Value
31
Monday, April 20, 2009 31
Polymorphic Associations
Of course, some people do go both ways.
— The Scarecrow
32
Monday, April 20, 2009 32
Polymorphic Assocations
• Objective: reference multiple parents
BUGS
COMMENTS
FEATURES
33
Monday, April 20, 2009 33
Polymorphic Assocations
• Can’t make a FOREIGN KEY constraint
reference two tables:
CREATE TABLE Comments (
comment_id
SERIAL PRIMARY KEY,
comment
TEXT NOT NULL,
issue_type
VARCHAR(15) CHECK
(issue_type IN (‘Bugs’, ‘Features’)),
issue_id
BIGINT NOT NULL,
FOREIGN KEY issue_id REFERENCES
);
you need this to be
Bugs or Features
34
Monday, April 20, 2009 34
Polymorphic Assocations
• Instead, you have to define table with no
FOREIGN KEY or referential integrity:
CREATE TABLE Comments (
comment_id
SERIAL PRIMARY KEY,
comment
TEXT NOT NULL,
issue_type
VARCHAR(15) CHECK
(issue_type IN (‘Bugs’, ‘Features’)),
issue_id
BIGINT NOT NULL
);
35
Monday, April 20, 2009 35
Polymorphic Assocations
comment
comment issue_type issue_id
issue_id id issue_id
Query result:
c. b. f.
comment_id comment issue_type
issue_id issue_id issue_id
6789 “It crashes” Bug 1234 1234 NULL
“Great
9876 Feature 2345 NULL 2345
idea!”
36
Monday, April 20, 2009 36
Polymorphic Assocations
• You can’t use a different table for each row.
You must name all tables explicitly.
SELECT * FROM Comments
JOIN
USING (issue_id);
37
Monday, April 20, 2009 37
Polymorphic Assocations
• Instead, join to each parent table:
SELECT *
FROM Comments c
LEFT JOIN Bugs b
ON (c.issue_type = ‘Bugs’
AND c.issue_id = b.issue_id)
LEFT JOIN Features f ON (c.issue_type = ‘Features’
AND c.issue_id = f.issue_id);
38
Monday, April 20, 2009 38
Polymorphic Assocations
• Solution #1: exclusive arcs
CREATE TABLE Comments (
comment_id
SERIAL PRIMARY KEY,
comment
TEXT NOT NULL,
bug_id
BIGINT, both columns are nullable;
feature_id
BIGINT, exactly one must be non-null
FOREIGN KEY bug_id
REFERENCES Bugs(bug_id)
FOREIGN KEY feature_id
REFERENCES Features(feature_id)
);
39
Monday, April 20, 2009 39
Polymorphic Assocations
• Solution #1: exclusive arcs
• Referential integrity is enforced
• But hard to make sure exactly one is non-null
• Queries are easier:
SELECT * FROM Comments c
LEFT JOIN Bugs b
USING (bug_id)
LEFT JOIN Features f USING (feature_id);
40
Monday, April 20, 2009 40
Polymorphic Assocations
• Solution #2: reverse the relationship
BUGS
COMMENTS BUGS
COMMENTS
FEATURES
COMMENTS FEATURES
41
Monday, April 20, 2009 41
Polymorphic Assocations
• Solution #2: reverse the relationship
CREATE TABLE BugsComments (
comment_id
BIGINT NOT NULL,
bug_id
BIGINT NOT NULL,
PRIMARY KEY (comment_id),
FOREIGN KEY (comment_id) REFERENCES Comments(comment_id),
FOREIGN KEY (bug_id) REFERENCES Bugs(bug_id)
);
42
Monday, April 20, 2009 42
Polymorphic Assocations
• Solution #2: reverse the relationship
• Referential integrity is enforced
• Query comments for a given bug:
SELECT * FROM BugsComments b
JOIN Comments c USING (comment_id)
WHERE b.bug_id = 1234;
43
Monday, April 20, 2009 43
Polymorphic Assocations
• Solution #3: use a base parent table
COMMENTS
44
Monday, April 20, 2009 44
Polymorphic Assocations
• Solution #3: use a base parent table
CREATE TABLE Issues (
issue_id
SERIAL PRIMARY KEY
);
CREATE TABLE Bugs (
issue_id
BIGINT PRIMARY KEY,
...
FOREIGN KEY (issue_id) REFERENCES Issues(issue_id)
);
CREATE TABLE Comments (
comment_id
SERIAL PRIMARY KEY,
comment
TEXT NOT NULL,
issue_id
BIGINT NOT NULL,
FOREIGN KEY (issue_id) REFRENCES Issues(issue_id)
);
45
Monday, April 20, 2009 45
Polymorphic Assocations
• Solution #3: use a base parent table
• Referential integrity is enforced
• Queries are easier:
SELECT * FROM Comments
JOIN Issues USING (issue_id)
LEFT JOIN Bugs USING (issue_id)
LEFT JOIN Features USING (issue_id);
46
Monday, April 20, 2009 46
Polymorphic Assocations
• Enforcing disjoint subtypes:
CREATE TABLE Issues (
issue_id
SERIAL PRIMARY KEY,
issue_type
CHAR(1) NOT NULL CHECK (issue_type IN (‘B’, ‘F’)),
UNIQUE KEY (issue_id, issue_type)
);
referential integrity
CREATE TABLE Bugs (
issue_id
BIGINT PRIMARY KEY,
issue_type
CHAR(1) NOT NULL CHECK (issue_type = ‘B’),
...
FOREIGN KEY (issue_id, issue_type)
REFERENCES Issues(issue_id, issue_type)
);
47
Monday, April 20, 2009 47
Naive Trees
48
Monday, April 20, 2009 48
Naive Trees
• Bill of materials
• Threaded discussions
49
Monday, April 20, 2009 49
Naive Trees
(1) Fran:
What’s the cause
of this bug?
(7) Kukla:
That fixed it.
50
Monday, April 20, 2009 50
Naive Trees
• Adjacency List
• Naive solution nearly everyone uses
• Each entry in the tree knows immediate parent
comment_id parent_id author comment
1 NULL Fran What’s the cause of this bug?
2 1 Ollie I think it’s a null pointer.
3 2 Fran No, I checked for that.
4 1 Kukla We need to check valid input.
5 4 Ollie Yes, that’s a bug.
6 4 Fran Yes, please add a check
7 6 Kukla That fixed it.
51
Monday, April 20, 2009 51
Naive Trees
• Adjacency List
• Easy to inserting a new comment:
INSERT INTO Comments (parent_id, author, comment)
VALUES (7, ‘Kukla’, ‘Thanks!’);
52
Monday, April 20, 2009 52
Naive Trees
• Adjacency List
• Querying a node’s immediate children is easy:
SELECT * FROM Comments c1
LEFT JOIN Comments c2
ON (c2.parent_id = c1.comment_id);
53
Monday, April 20, 2009 53
Naive Trees
• Adjacency List
• Hard to query all descendants in a deep tree:
SELECT * FROM Comments c1
LEFT JOIN Comments c2 ON (c2.parent_id = c1.comment_id)
LEFT JOIN Comments c3 ON (c3.parent_id = c2.comment_id)
LEFT JOIN Comments c4 ON (c4.parent_id = c3.comment_id)
LEFT JOIN Comments c5 ON (c5.parent_id = c4.comment_id)
LEFT JOIN Comments c6 ON (c6.parent_id = c5.comment_id)
LEFT JOIN Comments c7 ON (c7.parent_id = c6.comment_id)
LEFT JOIN Comments c8 ON (c8.parent_id = c7.comment_id)
LEFT JOIN Comments c9 ON (c9.parent_id = c8.comment_id)
LEFT JOIN Comments c10 ON (c10.parent_id = c9.comment_id)
...
it still doesn’t support
unlimited depth!
54
Monday, April 20, 2009 54
Naive Trees
• Solution #1: Path Enumeration
• Store chain of ancestors as a string in each node
• Good for breadcrumbs, or sorting by hierarchy
55
Monday, April 20, 2009 55
Naive Trees
• Solution #1: Path Enumeration
• Easy to query all ancestors of comment #7:
SELECT * FROM Comments
WHERE ‘1/4/6/7/’ LIKE path || ‘%’;
56
Monday, April 20, 2009 56
Naive Trees
• Solution #1: Path Enumeration
• Easy to add child of comment 7:
INSERT INTO Comments (author, comment)
VALUES (‘Ollie’, ‘Good job!’);
SELECT path FROM Comments
WHERE comment_id = 7;
UPDATE Comments
SET path = $parent_path || LAST_INSERT_ID() || ‘/’
WHERE comment_id = LAST_INSERT_ID();
57
Monday, April 20, 2009 57
Naive Trees
• Solution #2: Nested Sets
• Each comment encodes its descendants
using two numbers:
58
Monday, April 20, 2009 58
Naive Trees
• Solution #2: Nested Sets
(1) Fran:
What’s the cause
of this bug?
1 14
(2) Ollie: (4) Kukla:
I think it’s a null We need to check
pointer. valid input.
2 5 6 13
(7) Kukla:
That fixed it.
10 11
59
Monday, April 20, 2009 59
Naive Trees
• Solution #2: Nested Sets
60
Monday, April 20, 2009 60
Naive Trees
• Solution #2: Nested Sets
• Easy to query all ancestors of comment #7:
SELECT * FROM Comments child
JOIN Comments ancestor
ON (child.left BETWEEN
ancestor.nsleft
AND
ancestor.nsright)
WHERE child.comment_id = 7;
61
Monday, April 20, 2009 61
Naive Trees
• Solution #2: Nested Sets
ancestors
(1) Fran:
What’s the cause
of this bug?
1 14
(2) Ollie: (4) Kukla:
I think it’s a null We need to check
pointer. valid input.
2 5 6 13 child
(7) Kukla:
That fixed it.
10 11
62
Monday, April 20, 2009 62
Naive Trees
• Solution #2: Nested Sets
• Easy to query all descendants of comment #4:
SELECT * FROM Comments parent
JOIN Comments descendant
ON (descendant.left BETWEEN
parent.nsleft
AND
parent.nsright)
WHERE parent.comment_id = 4;
63
Monday, April 20, 2009 63
Naive Trees
• Solution #2: Nested Sets
(1) Fran:
parent
What’s the cause
of this bug?
1 14
descendants
(2) Ollie: (4) Kukla:
I think it’s a null We need to check
pointer. valid input.
2 5 6 13
(7) Kukla:
That fixed it.
10 11
64
Monday, April 20, 2009 64
Naive Trees
• Solution #2: Nested Sets
• Hard to insert a new child of comment #5:
UPDATE Comment
SET
nsleft = CASE WHEN nsleft >= 8 THEN nsleft+2 ELSE nsleft END,
nsright = nsright+2
WHERE nsright >= 7;
65
Monday, April 20, 2009 65
Naive Trees
• Solution #2: Nested Sets
(1) Fran:
What’s the cause
of this bug?
1 14
16
(2) Ollie: (4) Kukla:
I think it’s a null We need to check
pointer. valid input.
2 5 6 13
15
8 9 12
10 11
13
66
Monday, April 20, 2009 66
Naive Trees
• Solution #2: Nested Sets
• Hard to query the parent of comment #6:
SELECT parent.* FROM Comments AS c
JOIN Comments AS parent
ON (c.nsleft BETWEEN parent.nsleft AND parent.nsright)
LEFT OUTER JOIN Comments AS in_between
ON (c.nsleft BETWEEN in_between.nsleft AND in_between.nsright
AND in_between.nsleft BETWEEN parent.nsleft AND parent.nsright)
WHERE c.comment_id = 6 AND in_between.comment_id IS NULL;
67
Monday, April 20, 2009 67
Naive Trees
• Solution #3: Closure Table
• Store every path from ancestors to descendants
68
Monday, April 20, 2009 68
Naive Trees
• Solution #3: Closure Table
(1) Fran:
What’s the cause
of this bug?
(7) Kukla:
That fixed it.
69
Monday, April 20, 2009 69
Naive Trees
• Solution #3:
ancestor descendant
Closure Table 1 1
1 2
comment_id author comment
1 3
1 Fran What’s the cause of this bug?
1 4
2 Ollie I think it’s a null pointer.
1 5
3 Fran No, I checked for that.
1 6
4 Kukla We need to check valid input.
1 7
5 Ollie Yes, that’s a bug.
2 2
6 Fran Yes, please add a check
2 3
7 Kukla That fixed it.
3 3
4 4
requires O(n²) 4 5
rows at most 4 6
4 7
but far fewer
5 5
in practice
6 6
6 7
7 7
70
Monday, April 20, 2009 70
Naive Trees
• Solution #3: Closure Table
• Easy to query descendants of comment #4:
SELECT c.* FROM Comments c
JOIN TreePaths t
ON (c.comment_id = t.descendant)
WHERE t.ancestor = 4;
71
Monday, April 20, 2009 71
Naive Trees
• Solution #3: Closure Table
• Easy to query ancestors of comment #6:
SELECT c.* FROM Comments c
JOIN TreePaths t
ON (c.comment_id = t.ancestor)
WHERE t.descendant = 6;
72
Monday, April 20, 2009 72
Naive Trees
• Solution #3: Closure Table
• Easy to insert a new child of comment #5:
generates comment #8
INSERT INTO Comments ...
INSERT INTO TreePaths (ancestor, descendant)
VALUES (8, 8);
INSERT INTO TreePaths (ancestor, descendant)
SELECT ancestor, 8 FROM TreePaths
WHERE descendant = 5;
73
Monday, April 20, 2009 73
Naive Trees
• Solution #3: Closure Table
• Easy to delete a child comment #7:
DELETE FROM TreePaths
WHERE descendant = 7;
74
Monday, April 20, 2009 74
Naive Trees
• Solution #3: Closure Table
• Easy to delete the subtree under comment #4:
DELETE FROM TreePaths WHERE descendant IN
(SELECT descendant FROM TreePaths
WHERE ancestor = 4);
75
Monday, April 20, 2009 75
Naive Trees
• Solution #3:
ancestor descendant depth
Closure Table 1 1 0
1 2 1
JOIN TreePaths t 2 3 1
ON (c.comment_id = t.descendant) 3
4
3
4
0
0
WHERE t.ancestor = 4 4 5 1
AND t.depth = 1; 4 6 1
4 7 2
5 5 0
6 6 0
6 7 1
7 7 0
76
Monday, April 20, 2009 76
Naive Trees
• Summary of Designs:
Number of Query Query Modify Referential
Design
Tables Child Subtree Tree Integrity
Adjacency List 1 Easy Hard Easy Yes
Path
1 Easy Easy Hard No
Enumeration
77
Monday, April 20, 2009 77
Antipattern Categories
Database Design Database Creation
Antipatterns Antipatterns
CREATE TABLE BugsProducts (
bug_id INTEGER REFERENCES Bugs,
product VARCHAR(100) REFERENCES Products,
PRIMARY KEY (bug_id, product)
);
Query Application
Antipatterns Antipatterns
SELECT b.product, COUNT(*) $dbHandle = new PDO(‘mysql:dbname=test’);
FROM BugsProducts AS b $stmt = $dbHandle->prepare($sql);
GROUP BY b.product; $result = $stmt->fetchAll();
78
Monday, April 20, 2009 78
Database Creation Antipatterns
5. ENUM Antipattern
6. Rounding Errors
7. Indexes Are Magical
79
Monday, April 20, 2009 79
ENUM Antipattern
80
Monday, April 20, 2009 80
ENUM Antipattern
81
Monday, April 20, 2009 81
ENUM Antipattern
82
Monday, April 20, 2009 82
ENUM Antipattern
83
Monday, April 20, 2009 83
ENUM Antipattern
84
Monday, April 20, 2009 84
ENUM Antipattern
85
Monday, April 20, 2009 85
ENUM Antipattern
• Use a lookup table if values may change
CREATE TABLE BugStatus (
status
VARCHAR(10) PRIMARY KEY
);
INSERT INTO BugStatus (status)
VALUES (‘NEW’), (‘OPEN’), (‘FIXED’);
BUGS BUGSTATUS
86
Monday, April 20, 2009 86
ENUM Antipattern
87
Monday, April 20, 2009 87
ENUM Antipattern
88
Monday, April 20, 2009 88
Rounding Errors
10.0 times 0.1 is hardly ever 1.0.
— Brian Kernighan
89
Monday, April 20, 2009 89
Rounding Errors
90
Monday, April 20, 2009 90
Rounding Errors
91
Monday, April 20, 2009 91
Rounding Errors
• FLOAT is inexact
SELECT hours FROM Bugs
WHERE bug_id = 1234;
‣ 3.3
SELECT hours * 1000000000 FROM Bugs
WHERE bug_id = 1234;
‣ 3299999952.3163
92
Monday, April 20, 2009 92
Rounding Errors
93
Monday, April 20, 2009 93
Rounding Errors
94
Monday, April 20, 2009 94
Rounding Errors
95
Monday, April 20, 2009 95
Indexes are Magical
Whenever any result is sought, the question will then
arise — by what course of calculation can these results
be arrived at by the machine in the shortest time?
— Charles Babbage
96
Monday, April 20, 2009 96
Indexes are Magical
97
Monday, April 20, 2009 97
Indexes are Magical
• Antipatterns:
• Creating indexes blindly
98
Monday, April 20, 2009 98
Indexes are Magical
• Creating indexes blindly:
CREATE TABLE Bugs (
bug_id
SERIAL PRIMARY KEY,
date_reported DATE NOT NULL,
summary
VARCHAR(80) NOT NULL,
status
VARCHAR(10) NOT NULL, redundant index
hours
NUMERIC(9,2),
INDEX (bug_id), bulky index
INDEX (summary), unnecessary index
INDEX (hours), unnecessary
INDEX (bug_id, date_reported, status) covering index
);
99
Monday, April 20, 2009 99
Indexes are Magical
• Executing non-indexable queries:
• SELECT * FROM Bugs
WHERE description LIKE ‘%crash%’;
non-leftmost
•
no index spans
SELECT * FROM Accounts all rows
ORDER BY first_name, last_name;
non-leftmost
composite key match
100
Monday, April 20, 2009 100
Indexes are Magical
• Telephone book analogy
• Easy to search for Dean Thomas: uses index
to match
SELECT * FROM TelephoneBook
WHERE full_name LIKE ‘Thomas, %’;
101
Monday, April 20, 2009 101
Indexes are Magical
• Rejecting indexes because of their overhead:
80 Query w/ Index: O(log n)
Update Index: O(log n)
Query w/o Index: O(n)
60
20
0
0 100 200 300 400 500 600 700 800 9001000
102
Monday, April 20, 2009 102
Indexes are Magical
• Solution: “MENTOR” your indexes
Measure
Explain
Nominate
Test
Optimize
Repair
103
Monday, April 20, 2009 103
Indexes are Magical
• Solution: “MENTOR” your indexes
Measure • Profile your application.
Explain • Focus on the most costly
SQL queries:
Nominate • Longest-running.
Test • Most frequently run.
• Blockers, lockers, and
Optimize deadlocks.
Repair
104
Monday, April 20, 2009 104
Indexes are Magical
• Solution: “MENTOR” your indexes
Measure • Analyze the optimization
plan of costly queries,
Explain e.g. MySQL’s EXPLAIN
Nominate • Identify tables that aren’t
using indexes:
Test • Temporary table
Optimize • Filesort
Repair
105
Monday, April 20, 2009 105
Indexes are Magical
• Solution: “MENTOR” your indexes
Measure • Could an index improve
access to these tables?
Explain • ORDER BY criteria
Nominate • MIN() / MAX()
Test • WHERE conditions
• Which column(s) need
Optimize indexes?
Repair
106
Monday, April 20, 2009 106
Indexes are Magical
• Solution: “MENTOR” your indexes
Measure • After creating index,
measure again.
Explain • Confirm the new index
Nominate made a difference.
Repair
107
Monday, April 20, 2009 107
Indexes are Magical
• Solution: “MENTOR” your indexes
Measure • Indexes are compact,
frequently-used data.
Explain • Try to cache indexes in
Nominate memory:
• MyISAM: key_buffer_size,
Test LOAD INDEX INTO CACHE
• InnoDB: innodb_buffer_pool_size
Optimize
Repair
108
Monday, April 20, 2009 108
Indexes are Magical
• Solution: “MENTOR” your indexes
Measure • Indexes require periodic
maintenance.
Explain • Like a filesystem requires
Nominate periodic defragmentation.
109
Monday, April 20, 2009 109
Indexes are Magical
• Solution: “MENTOR” your indexes
• Sounds like the name BY THE AUTHOR OF THE BESTSELLER
WHY DOESN’T MY CACHE GET ANY HITS?
110
Monday, April 20, 2009 110
Antipattern Categories
Database Design Database Creation
Antipatterns Antipatterns
CREATE TABLE BugsProducts (
bug_id INTEGER REFERENCES Bugs,
product VARCHAR(100) REFERENCES Products,
PRIMARY KEY (bug_id, product)
);
Query Application
Antipatterns Antipatterns
SELECT b.product, COUNT(*) $dbHandle = new PDO(‘mysql:dbname=test’);
FROM BugsProducts AS b $stmt = $dbHandle->prepare($sql);
GROUP BY b.product; $result = $stmt->fetchAll();
111
Monday, April 20, 2009 111
Query Antipatterns
8. NULL antipatterns
9. Ambiguous Groups
10. Random Order
11. JOIN antipattern
12. Goldberg Machine
112
Monday, April 20, 2009 112
NULL Antipatterns
As we know, there are known knowns; there are things we know
we know. We also know there are known unknowns; that is to say
we know there are some things we do not know. But there are also
unknown unknowns — the ones we don't know we don't know.
— Donald Rumsfeld
113
Monday, April 20, 2009 113
NULL Antipatterns
114
Monday, April 20, 2009 114
NULL Antipatterns
• Antipatterns:
• Use NULL as an ordinary value
115
Monday, April 20, 2009 115
NULL Antipatterns
116
Monday, April 20, 2009 116
NULL Antipatterns
117
Monday, April 20, 2009 117
NULL Antipatterns
118
Monday, April 20, 2009 118
NULL Antipatterns
119
Monday, April 20, 2009 119
NULL Antipatterns
120
Monday, April 20, 2009 120
NULL Antipatterns
121
Monday, April 20, 2009 121
NULL Antipatterns
• Understanding NULL in expressions
Expression Expected Actual
NULL = 0 TRUE Unknown
NULL = 12345 FALSE Unknown
NULL <> 12345 TRUE Unknown
NULL + 12345 12345 Unknown
NULL || ‘string’ string’ Unknown
NULL = NULL TRUE Unknown
NULL <> NULL FALSE Unknown
122
Monday, April 20, 2009 122
NULL Antipatterns
• Understanding NULL in boolean expressions
Expression Expected Actual
NULL AND TRUE FALSE Unknown
NULL AND FALSE FALSE FALSE
NULL OR FALSE FALSE Unknown
NULL OR TRUE TRUE TRUE
NOT (NULL) TRUE Unknown
123
Monday, April 20, 2009 123
NULL Antipatterns
• SQL supports IS NULL predicate that
returns true or false, never unknown:
SELECT * FROM Bugs
WHERE assigned_to IS NULL;
SELECT * FROM Bugs
WHERE assigned_to IS NOT NULL;
124
Monday, April 20, 2009 124
NULL Antipatterns
• SQL-99 supports IS DISTINCT FROM
predicate that returns true or false:
SELECT * FROM Bugs
WHERE assigned_to IS DISTINCT FROM 123;
SELECT * FROM Bugs
WHERE assigned_to IS NOT DISTINCT FROM 123;
SELECT * FROM Bugs MySQL operator works like
WHERE assigned_to <=> 123; IS NOT DISTINCT FROM
125
Monday, April 20, 2009 125
NULL Antipatterns
• Change NULL to ordinary value on demand
with COALESCE():
SELECT COALESCE(
first_name || ‘ ’ || middle_initial || ‘ ’ || last_name,
first_name || ‘ ’ || last_name) AS full_name
FROM Accounts;
• Also called NVL() or ISNULL()
in some database brands.
126
Monday, April 20, 2009 126
Ambiguous Groups
Please accept my resignation. I don’t want to belong
to any club that will accept me as a member.
— Groucho Marx
127
Monday, April 20, 2009 127
Ambiguous Groups
128
Monday, April 20, 2009 128
Ambiguous Groups
• Antipattern: bug_id isn’t that of the
latest per product
product_name bug_id date_reported
Open RoundFile 1234 2007-12-19
Open RoundFile 2248 2008-04-01 product_name bug_id latest
Visual TurboBuilder 3456 2008-02-16 Open RoundFile 1234 2008-04-01
Visual TurboBuilder 4077 2008-02-10 Visual TurboBuilder 3456 2008-02-16
ReConsider 5678 2008-01-01 ReConsider 5678 2008-01-01
ReConsider 8063 2007-11-09
129
Monday, April 20, 2009 129
Ambiguous Groups
130
Monday, April 20, 2009 130
Ambiguous Groups
131
Monday, April 20, 2009 131
Ambiguous Groups
what bug_id
SELECT product_name, bug_id, has both the earliest
and the latest date?
MIN(date_reported) AS earliest,
MAX(date_reported) AS latest
FROM Bugs
GROUP BY product_name;
132
Monday, April 20, 2009 132
Ambiguous Groups
what if no bug_id
SELECT product_name, bug_id, matches this date?
AVG(date_reported) AS mean
FROM Bugs
GROUP BY product_name;
133
Monday, April 20, 2009 133
Ambiguous Groups
134
Monday, April 20, 2009 134
Ambiguous Groups
• For a given product_name, there is a single
value in each functionally dependent attribute.
135
Monday, April 20, 2009 135
Ambiguous Groups
• Solution #1: use only functionally
dependent attributes:
✘
SELECT product_name, bug_id,
MAX(date_reported) AS latest
FROM Bugs
GROUP BY product_name;
product_name latest
Open RoundFile 2008-04-01
Visual TurboBuilder 2008-02-16
ReConsider 2008-01-01
136
Monday, April 20, 2009 136
Ambiguous Groups
• Solution #2: use a derived table:
SELECT b.product_name, b.bug_id, m.latest
FROM Bugs b
JOIN (SELECT product_name, MAX(date_reported) AS latest
FROM Bugs GROUP BY product_name) m
ON (b.product_name = m.product_name
AND b.date_reported = m.latest);
137
Monday, April 20, 2009 137
Ambiguous Groups
• Solution #3: use an outer JOIN:
SELECT b1.product_name, b1.bug_id,
b1.date_reported AS latest
FROM Bugs b1
LEFT OUTER JOIN Bugs b2
ON (b1.product_name = b2.product_name
AND b1.date_reported < b2.date_reported)
WHERE b2.bug_id IS NULL;
138
Monday, April 20, 2009 138
Ambiguous Groups
• Solution #4: use another aggregate:
SELECT product_name, MAX(date_reported) AS latest,
MAX(bug_id) AS latest_bug_id
FROM Bugs
GROUP BY product_name;
if bug_id increases
in chronological order
139
Monday, April 20, 2009 139
Ambiguous Groups
• Solution #5: use GROUP_CONCAT():
SELECT product_name,
GROUP_CONCAT(bug_id) AS bug_id_list,
MAX(date_reported) AS latest
FROM Bugs
GROUP BY product_name;
140
Monday, April 20, 2009 140
Random Order
I must complain the cards are ill shuffled till I have a good hand.
— Jonathan Swift
141
Monday, April 20, 2009 141
Random Order
142
Monday, April 20, 2009 142
Random Order
143
Monday, April 20, 2009 143
Random Order
• Solution #1: pick random primary key
from list of all values:
$bug_id_list = $pdo->query(
‘SELECT bug_id FROM Bugs’ )->fetchAll();
$rand = random(count($bug_id_list));
$stmt = $pdo->prepare(
‘SELECT * FROM Bugs WHERE bug_id = ?’);
$stmt->execute( $bug_id_list[$rand][0] );
$rand_bug = $stmt->fetch();
144
Monday, April 20, 2009 144
Random Order
• Solution #1: pick random primary key
from list of all values:
$bug_id_list = $pdo->query(
‘SELECT bug_id FROM Bugs’ )->fetchAll();
145
Monday, April 20, 2009 145
Random Order
• Solution #2: pick random value between
1...MAX(bug_id); use that bug_id:
SELECT b1.* FROM Bugs b1
JOIN (SELECT CEIL(RAND() *
(SELECT MAX(bug_id) FROM Bugs)) rand_id) b2
ON (b1.bug_id = b2.rand_id);
146
Monday, April 20, 2009 146
Random Order
• Solution #2: pick random value between
1...MAX(bug_id); use that bug_id:
• Assumes bug_id starts at 1
and values are contiguous.
• If there are gaps, a random
bug_id may not match an
existing bug.
147
Monday, April 20, 2009 147
Random Order
• Solution #3: pick random value between
1...MAX(bug_id); use next higher bug_id:
SELECT b1.* FROM Bugs b1
JOIN (SELECT CEIL(RAND() *
(SELECT MAX(bug_id) FROM Bugs)) AS bug_id) b2
WHERE b1.bug_id >= b2.bug_id
ORDER BY b1.bug_id
LIMIT 1;
148
Monday, April 20, 2009 148
Random Order
• Solution #3: pick random value between
1...MAX(bug_id); use next higher bug_id:
• bug_id values after gaps
are chosen more often.
• Random values are evenly
distributed, but bug_id values aren’t.
149
Monday, April 20, 2009 149
Random Order
• Solution #4: pick random row from
0...COUNT, regardless of bug_id values:
$offset = $pdo->query(
‘SELECT ROUND(RAND() *
(SELECT COUNT(*) FROM Bugs))’ )->fetch();
$sql = ‘SELECT * FROM Bugs LIMIT 1 OFFSET ?’;
$stmt = $pdo->prepare( $sql );
$stmt->execute( $offset );
150
Monday, April 20, 2009 150
JOIN Antipattern
151
Monday, April 20, 2009 151
JOIN Antipattern
152
Monday, April 20, 2009 152
JOIN Antipattern
• Antipatterns:
• Senseless avoidance of JOIN.
153
Monday, April 20, 2009 153
JOIN Antipattern
154
Monday, April 20, 2009 154
JOIN Antipattern
• Example from the web (2009-4-18):
how to apply
conditions to stores?
SELECT *,
(SELECT name FROM stores WHERE id = p.store_id) AS store_name,
(SELECT username FROM stores WHERE id = p.store_id) AS store_username,
(SELECT region_id FROM stores WHERE id = p.store_id) AS region_id,
(SELECT city_id FROM stores WHERE id = p.store_id) AS city_id,
(SELECT name FROM categories_sub WHERE id=p.subcategory_id) subcat_name,
(SELECT name FROM categories WHERE id = p.category_id) AS category_name
FROM products p
WHERE p.date_start <= DATE(NOW()) AND p.date_end >= DATE(NOW());
optimizer can’t
six correlated
reorder JOINs
subqueries!
155
Monday, April 20, 2009 155
JOIN Antipattern
• Example revised with JOINs:
easier to
apply conditions
156
Monday, April 20, 2009 156
JOIN Antipattern
• Example: find an entry with three tags:
HAVING COUNT solution:
SELECT b.*
FROM Bugs b
JOIN BugsProducts p ON (b.bug_id = p.bug_id)
WHERE p.product_id IN (1, 2, 3)
GROUP BY b.bug_id
HAVING COUNT(*) = 3; must match all
three products
157
Monday, April 20, 2009 157
JOIN Antipattern
• Example: find an entry with three tags::
multiple-JOIN solution:
SELECT DISTINCT b.*
FROM Bugs b
JOIN BugsProducts p1 ON ((p1.bug_id, p1.product_id) = (b.bug_id, 1))
JOIN BugsProducts p2 ON ((p2.bug_id, p2.product_id) = (b.bug_id, 2))
JOIN BugsProducts p3 ON ((p3.bug_id, p3.product_id) = (b.bug_id, 3));
158
Monday, April 20, 2009 158
JOIN Antipattern
• Solution:
• JOIN is to SQL as while() is to other languages.
• One-size-fits-all rules (e.g. “joins are slow”)
don’t work.
• Measure twice, query once.
• Let the SQL optimizer work.
• Employ alternatives (e.g. JOIN decomposition)
as exception cases.
159
Monday, April 20, 2009 159
Goldberg Machine
Enita non sunt multiplicanda praeter necessitatem
(“Entities are not to be multiplied beyond necessity”).
— William of Okham
160
Monday, April 20, 2009 160
Goldberg Machine
161
Monday, April 20, 2009 161
Goldberg Machine
162
Monday, April 20, 2009 162
Goldberg Machine
163
Monday, April 20, 2009 163
Goldberg Machine
• Expected result versus actual result:
products
account name bugs reported comments
assigned
Bill 3 48 2 48 4 48
164
Monday, April 20, 2009 164
Goldberg Machine
• Run query without GROUP BY:
SELECT a.account_name,
br.bug_id AS bug_reported,
ba.bug_id AS bug_assigned,
bp.product_id AS product_assigned
c.comment_id
FROM Accounts a
LEFT JOIN Bugs br ON (a.account_id = br.reported_by)
LEFT JOIN (Bugs ba JOIN BugsProducts bp ON (ba.bug_id = bp.bug_id))
ON (a.account_id = ba.assigned_to)
LEFT JOIN Comments c ON (a.account_id = c.author);
165
Monday, April 20, 2009 165
Goldberg Machine
• Query result reveals a Cartesian Product:
account bug bug product
comment
name reported assigned assigned
Bill 1234 1234 1 6789
Bill 1234 1234 1 9876
Bill 1234 1234 1 4365
Bill 1234 1234 1 7698
Bill 1234 1234 3 6789
Bill 1234 1234 3 9876
Bill 1234 1234 3 4365
Bill 1234 1234 3 7698
166
Monday, April 20, 2009 166
Goldberg Machine
• Query result reveals a Cartesian Product:
account bug bug product
comment
name reported assigned assigned
Bill 1234 5678 1 6789
Bill 1234 5678 1 9876
Bill 1234 5678 1 4365
Bill 1234 5678 1 7698
Bill 1234 5678 3 6789
Bill 1234 5678 3 9876
Bill 1234 5678 3 4365
Bill 1234 5678 3 7698
167
Monday, April 20, 2009 167
Goldberg Machine
• Query result reveals a Cartesian Product:
account bug bug product
comment
name reported assigned assigned
Bill 2345 1234 1 6789
Bill 2345 1234 1 9876
Bill 2345 1234 1 4365
Bill 2345 1234 1 7698
Bill 2345 1234 3 6789
Bill 2345 1234 3 9876
Bill 2345 1234 3 4365
Bill 2345 1234 3 7698
168
Monday, April 20, 2009 168
Goldberg Machine
• Query result reveals a Cartesian Product:
account bug bug product
comment
name reported assigned assigned
Bill 2345 5678 1 6789
Bill 2345 5678 1 9876
Bill 2345 5678 1 4365
Bill 2345 5678 1 7698
Bill 2345 5678 3 6789
Bill 2345 5678 3 9876
Bill 2345 5678 3 4365
Bill 2345 5678 3 7698
169
Monday, April 20, 2009 169
Goldberg Machine
• Query result reveals a Cartesian Product:
account bug bug product
comment
name reported assigned assigned
Bill 3456 1234 1 6789
Bill 3456 1234 1 9876
Bill 3456 1234 1 4365
Bill 3456 1234 1 7698
Bill 3456 1234 3 6789
Bill 3456 1234 3 9876
Bill 3456 1234 3 4365
Bill 3456 1234 3 7698
170
Monday, April 20, 2009 170
Goldberg Machine
• Query result reveals a Cartesian Product:
account bug bug product
comment
name reported assigned assigned
Bill 3456 5678 1 6789
Bill 3456 5678 1 9876
Bill 3456 5678 1 4365
Bill 3456 5678 1 7698
Bill 3456 5678 3 6789
Bill 3456 5678 3 9876
Bill 3456 5678 3 4365
Bill 3456 5678 3 7698
171
Monday, April 20, 2009 171
Goldberg Machine
• Visualizing a Cartesian Product:
bugs
reported Bugs each assigned bug
applies to 2 products
3 × 4 × 4 = 48
comments Comments
172
Monday, April 20, 2009 172
Goldberg Machine
173
Monday, April 20, 2009 173
Antipattern Categories
Database Design Database Creation
Antipatterns Antipatterns
CREATE TABLE BugsProducts (
bug_id INTEGER REFERENCES Bugs,
product VARCHAR(100) REFERENCES Products,
PRIMARY KEY (bug_id, product)
);
Query Application
Antipatterns Antipatterns
SELECT b.product, COUNT(*) $dbHandle = new PDO(‘mysql:dbname=test’);
FROM BugsProducts AS b $stmt = $dbHandle->prepare($sql);
GROUP BY b.product; $result = $stmt->fetchAll();
174
Monday, April 20, 2009 174
Application Antipatterns
175
Monday, April 20, 2009 175
Parameter Facade
176
Monday, April 20, 2009 176
Parameter Facade
177
Monday, April 20, 2009 177
Parameter Facade
178
Monday, April 20, 2009 178
Parameter Facade
179
Monday, April 20, 2009 179
Parameter Facade
180
Monday, April 20, 2009 180
Parameter Facade
181
Monday, April 20, 2009 181
Parameter Facade
182
Monday, April 20, 2009 182
Parameter Facade
183
Monday, April 20, 2009 183
Parameter Facade
184
Monday, April 20, 2009 184
Parameter Facade
185
Monday, April 20, 2009 185
Parameter Facade
SELECT expr-list *
bug_id
parameter
placeholder
186
Monday, April 20, 2009 186
Parameter Facade
• Runs query
could invalidate
optimization plan
187
Monday, April 20, 2009 187
Parameter Facade
SELECT expr-list *
bug_id
1234
supplied
value
188
Monday, April 20, 2009 188
Parameter Facade
SELECT expr-list *
bug_id
1234
or 1=1
supplied
value
189
Monday, April 20, 2009 189
Parameter Facade
190
Monday, April 20, 2009 190
Parameter Facade
SELECT expr-list *
bug_id
1234
intended
value
191
Monday, April 20, 2009 191
Parameter Facade
SELECT expr-list *
bug_id
1234
WHERE expr OR
1
equality =
SQL injection 1
192
Monday, April 20, 2009 192
Parameter Facade
193
Monday, April 20, 2009 193
Parameter Facade
must supply
exactly four values
194
Monday, April 20, 2009 194
Parameter Facade
Scenario Value Interpolation Parameter
single value ‘1234’ SELECT * FROM bugs SELECT * FROM bugs
WHERE bug_id = $id; WHERE bug_id = ?;
multiple values ‘1234, 3456, 5678’ SELECT * FROM bugs SELECT * FROM bugs
WHERE bug_id IN ($list); WHERE bug_id IN ( ?, ?, ? );
195
Monday, April 20, 2009 195
Parameter Facade
• Solution:
• Use parameters only for individual values
196
Monday, April 20, 2009 196
Phantom Side Effects
Every program attempts to expand until it can read mail.
— Jamie Zawinsky
197
Monday, April 20, 2009 197
Phantom Side Effects
198
Monday, April 20, 2009 198
Phantom Side Effects
199
Monday, April 20, 2009 199
Phantom Side Effects
• External effects don’t obey ROLLBACK
1. Start transaction and INSERT
bug_id description
notify of
bug_id 1234
insert row
bug_id = 1234
200
Monday, April 20, 2009 200
Phantom Side Effects
• External effects don’t obey ROLLBACK
2. ROLLBACK
bug_id description
I got email,
discard but no row
row 1234?
201
Monday, April 20, 2009 201
Phantom Side Effects
• External effects don’t obey transaction
isolation
1. Start transaction and INSERT
bug_id description
notify of
bug_id 1234
insert row
bug_id = 1234
202
Monday, April 20, 2009 202
Phantom Side Effects
• External effects don’t obey transaction
isolation
2. Email is received before row is visible
bug_id description
I got email,
row but no row
pending 1234?
commit
203
Monday, April 20, 2009 203
Phantom Side Effects
• Auditing/logging confusion
204
Monday, April 20, 2009 204
Phantom Side Effects
205
Monday, April 20, 2009 205
Phantom Side Effects
206
Monday, April 20, 2009 206
Phantom Side Effects
• Solution:
• Operate only on database in triggers, stored
procedures, database functions
207
Monday, April 20, 2009 207
See No Evil
Everyone knows that debugging is twice as hard as writing
a program in the first place. So if you’re as clever as you
can be when you write it, how will you ever debug it?
— Brian Kernighan
208
Monday, April 20, 2009 208
See No Evil
209
Monday, April 20, 2009 209
See No Evil
• Antipatterns:
• Ignore errors in return status or exceptions.
210
Monday, April 20, 2009 210
See No Evil
• Ignoring errors in return status:
$sql = “SELECT * FROM Bugs”;
$result = $mysqli->query( $sql );
$rows = $result->fetch_all();
OK
211
Monday, April 20, 2009 211
See No Evil
• Ignoring errors in return status:
$sql = “SELECT * FROM Bugz”; returns FALSE
FAIL
212
Monday, April 20, 2009 212
See No Evil
• Ignoring exceptions:
$sql = “SELECT * FROM Bugz”;
$stmt = $pdo->query( $sql );
$rows = $stmt->fetchAll(); throws PDOException
NOT
REACHED
213
Monday, April 20, 2009 213
See No Evil
• Solution: check for error status.
$sql = “SELECT * FROM Bugz”;
$result = $mysqli->query( $sql );
if ($result === FALSE ) {
log($mysqli->error());
return FALSE;
} don’t let it go this far!
$rows = $result->fetchAll();
214
Monday, April 20, 2009 214
See No Evil
• Solution: handle exceptions.
$sql = “SELECT * FROM Bugz”;
try {
$stmt = $pdo->query( $sql );
} catch (PDOException $e) {
log($stmt->errorInfo());
return FALSE;
} don’t let it go this far!
$rows = $stmt->fetchAll();
215
Monday, April 20, 2009 215
See No Evil
• Troubleshooting code:
who wants to
$sql = ‘SELECT * FROM Bugs read this!?
WHERE summary LIKE \’%’
. $mysqli->quote( $feature )
. ‘ doesn\’t work 50\% of the time!%\’’;
$result = $mysqli->query( $sql );
$rows = $result->fetchAll();
216
Monday, April 20, 2009 216
See No Evil
• Solution: Look at the SQL, not the code!
$sql = ‘SELECT * FROM Bugs
WHERE summary LIKE \’%’
. $mysqli->quote( $feature )
. ‘ doesn\’t work 50\% of the time!%\’’;
the error
$firephp = FirePHP::getInstance(true); is now clear!
217
Monday, April 20, 2009 217
Diplomatic Immunity
Humans are allergic to change. They love to say,
“We’ve always done it this way.” I try to fight that.
— Adm. Grace Murray Hopper
218
Monday, April 20, 2009 218
Diplomatic Immunity
219
Monday, April 20, 2009 219
Diplomatic Immunity
220
Monday, April 20, 2009 220
Diplomatic Immunity
• Documentation
221
Monday, April 20, 2009 221
Diplomatic Immunity
• Functional testing
Tables, Views, • Verify presence of
Columns tables and views.
Constraints • Verify they contain
columns you expect.
Triggers
• Verify absence of
Stored Procedures tables, views, or
Bootstrap Data columns that you
dropped.
Queries
ORM Classes
222
Monday, April 20, 2009 222
Diplomatic Immunity
• Functional testing
Tables, Views, • Try to execute updates
Columns that ought to be denied
by constraints.
Constraints
• You can catch bugs
Triggers earlier by identifying
Stored Procedures constraints that are
failing.
Bootstrap Data
Queries
ORM Classes
223
Monday, April 20, 2009 223
Diplomatic Immunity
• Functional testing
Tables, Views, • Triggers can enforce
Columns constraints too.
Constraints • Triggers can perform
cascading effects,
Triggers transform values, log
Stored Procedures changes, etc.
Bootstrap Data • You should test these
scenarios.
Queries
ORM Classes
224
Monday, April 20, 2009 224
Diplomatic Immunity
• Functional testing
Tables, Views, • Code is more easily
Columns developed, debugged,
and maintained in the
Constraints application layer.
Triggers • Nevertheless, stored
Stored Procedures procedures are useful,
e.g. solving tough
Bootstrap Data bottlenecks.
Queries • You should test stored
ORM Classes procedure code.
225
Monday, April 20, 2009 225
Diplomatic Immunity
• Functional testing
Tables, Views, • Lookup tables need to
Columns be filled, even in an
“empty” database.
Constraints
• Test that the required
Triggers data are present.
Stored Procedures • Other cases exist for
Bootstrap Data initial data.
Queries
ORM Classes
226
Monday, April 20, 2009 226
Diplomatic Immunity
• Functional testing
Tables, Views, • Application code is
Columns laced with SQL queries.
Constraints • Test queries for result
set metadata, e.g.
Triggers columns, data types.
Stored Procedures • Test performance; good
Bootstrap Data queries can become
bottlenecks, as data and
Queries indexes grow.
ORM Classes
227
Monday, April 20, 2009 227
Diplomatic Immunity
• Functional testing
Tables, Views, • Like Triggers, ORM
Columns classes contain logic:
Constraints • Validation.
• Transformation.
Triggers
• Monitoring.
Stored Procedures
• You should test these
Bootstrap Data classes as you would
any other code.
Queries
ORM Classes
228
Monday, April 20, 2009 228
Diplomatic Immunity
• Documentation
Entity Relation-
ship Diagram
Tables, Columns
Relationships
Views, Triggers
Stored Procedures
SQL Privileges
Application Code
229
Monday, April 20, 2009 229
Diplomatic Immunity
• Documentation
Entity-Relationship • Purpose of each table,
Diagram each column.
Tables, Columns • Constraints, rules that
apply to each.
Relationships
• Sample data.
Views, Triggers
• List the Views, Triggers,
Stored Procedures Procs, Applications, and
Users that use each.
SQL Privileges
Application Code
230
Monday, April 20, 2009 230
Diplomatic Immunity
• Documentation
Entity-Relationship • Describe in text the
Diagram dependencies between
tables.
Tables, Columns
• Business rules aren’t
Relationships represented fully by
Views, Triggers declarative constraints.
Stored Procedures
SQL Privileges
Application Code
231
Monday, April 20, 2009 231
Diplomatic Immunity
• Documentation
Entity-Relationship • Purpose of Views;
Diagram who uses them.
Tables, Columns • Usage of updatable
Views.
Relationships
• Business rules enforced
Views, Triggers by Triggers:
Stored Procedures • Validation
SQL Privileges • Transformation
• Logging
Application Code
232
Monday, April 20, 2009 232
Diplomatic Immunity
• Documentation
Entity-Relationship • Document the Stored
Diagram Procedures as an API.
Tables, Columns • Especially side-effects.
Relationships • What problem is the
procedure solving?
Views, Triggers • Encapsulation
Stored Procedures • Performance
SQL Privileges • Privileged access
Application Code
233
Monday, April 20, 2009 233
Diplomatic Immunity
• Documentation
Entity-Relationship • Logins with specific
Diagram access purposes
(e.g. backup, reports).
Tables, Columns
• Sets of privileges
Relationships (roles) used for
Views, Triggers different scenarios.
Stored Procedures • Security measures.
SQL Privileges
Application Code
234
Monday, April 20, 2009 234
Diplomatic Immunity
• Documentation
Entity-Relationship • Data Access Layer:
Diagram • Connection params.
Tables, Columns • Client options.
Relationships • Driver versions.
Views, Triggers • Object-Relational
Mapping (ORM):
Stored Procedures • Validations, Logging,
SQL Privileges Transformations.
• Special find() methods.
Application Code
235
Monday, April 20, 2009 235
Diplomatic Immunity
• Source code control
• Keep database in synch with application code.
236
Monday, April 20, 2009 236
Diplomatic Immunity
• Source code control: “Migrations.”
• Migrations are like version-control for the
database instance.
237
Monday, April 20, 2009 237
Magic Beans
Essentially, all models are wrong, but some are useful.
— George E. P. Box
238
Monday, April 20, 2009 238
Magic Beans
239
Monday, April 20, 2009 239
Magic Beans
• Antipattern: equating “Model” in MVC
architecture with the Active Record pattern.
• The Golden Hammer of data access.
240
Monday, April 20, 2009 240
Magic Beans
• Antipattern: Model is-a Active Record.
Active
Record
inheritance (IS-A)
Controller View
aggregation (HAS-A)
241
Monday, April 20, 2009 241
Magic Beans
• Bad object-oriented design:
• “Model”
Active Record inheritance (IS-A)
242
Monday, April 20, 2009 242
Magic Beans
• Bad Model-View-Controller design “T.M.I.” !!
243
Monday, April 20, 2009 243
Magic Beans
• Bad testability design
• Model coupled to Active Record; tests are
slow
harder to test Model without database.
244
Monday, April 20, 2009 244
Magic Beans
• Solution: Model has-a Active Record(s).
Active
Record
inheritance (IS-A)
composition (HAS-A)
BugReport
Controller View
(Model)
aggregation (HAS-A)
245
Monday, April 20, 2009 245
Magic Beans
• Solution: Model has-a Active Record(s).
246
Monday, April 20, 2009 246
Magic Beans
• Solution: Model has-a Active Record(s).
247
Monday, April 20, 2009 247
Magic Beans
• Solution: Model has-a Active Record(s).
248
Monday, April 20, 2009 248
Antipattern Categories
Database Design Database Creation
Antipatterns Antipatterns
CREATE TABLE BugsProducts (
bug_id INTEGER REFERENCES Bugs,
product VARCHAR(100) REFERENCES Products,
PRIMARY KEY (bug_id, product)
);
Query Application
Antipatterns Antipatterns
SELECT b.product, COUNT(*) $dbHandle = new PDO(‘mysql:dbname=test’);
FROM BugsProducts AS b $stmt = $dbHandle->prepare($sql);
GROUP BY b.product; $result = $stmt->fetchAll();
249
Monday, April 20, 2009 249
Thank You
Copyright 2008-2009 Bill Karwin
www.karwin.com
Released under a Creative Commons 3.0 License:
http://creativecommons.org/licenses/by-nc-nd/3.0/