You are on page 1of 73

Best Design Practices - Part A

Building Scalable Enterprise Solutions
Architecture, Performance, Scalability, Migration, Knowledge Transfer

1
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A

Best Design Practices
For MySQL Architects, DBA’s and
Developers

Ronald Bradford
Principal
42SQL

ronald@42sql.com
2
(c) Copyright 2008
Version 1.0 3.Aug..2008 http://42sql.com
Best Design Practices - Part A

Introduction

❖ Lots of information to discuss
❖ Basic principles
❖ Rules

3
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A

Agenda

❖ Architecture
❖ Availability
❖ Scalability
❖ Accountability

4
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A
A
Architecture R
C
H
❖ Toolset I
❖ The performance equation T
E
❖ Data integrity C
T
U
R
E

5
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A
A
Your Toolset R
C
H
❖ Generics are inefficient I
❖ different RDBMS is not enough T
E
❖ You have chosen MySQL C
T
U
R
E

6
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A
A
Your MySQL Toolset R
C
H
❖ Maximize MySQL strengths I
❖ Scale out / HA Options T
❖ Different Storage Engines E
❖ Query Cache - Both good and bad C
T
U
R
E

7
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A
A
Your MySQL Toolset R
C
H
❖ Minimize MySQL weaknesses I
❖ No Online Alter T
❖ Backup Strategies E
❖ Instrumentation C
T
U
R
E

8
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A
A
Your MySQL Toolset R
Example
C
H
❖ Connector/J I
❖ Supports read/write splitting T
❖ Supports read balancing across slaves E
❖ Supports failover C
T
❖ Connection.setReadOnly(true | false); U
R
E
http://dev.mysql.com/doc/refman/5.0/en/connector-j-reference-replication-connection.html

9
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A
A
Your Toolset R
C
H
I
T

RTFM
E
C
T
U
R
E
Manual: http://dev.mysql.com/doc
Book: High Performance MySQL - 2nd Edition
10
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A
A
The performance equation R
C
H
Disk = Memory = Performance I
T
E
❖ Every single byte counts C
❖ Average saving of 25% - 30% T
U
❖ Better 60% ‫‏‬ R
❖ Best 78% E

11
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A
A
The performance equation R
C
H
❖ Favorite signs of poor design in data types I
❖ INT(1)‫‏‬ T
❖ BIGINT AUTO_INCREMENT E
❖ no UNSIGNED used C
❖ DECIMAL(31,0)‫‏‬
T
U
R
E

12
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A
A
Data integrity R
C
H
❖ MySQL historically has been very lax I
❖ Warnings (e.g. Truncations) are rarely every caught T
E
❖ sql_mode=strict_all_tables
C
T
U
R
E

http://ronaldbradford.com/blog/why-sql_mode-is-important-part-i-2008-07-17/
13
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A
A
Data integrity R
C
H
❖ With Schema I
❖ NOT NULL T
❖ ENUM E
❖ UNSIGNED C
T
U
R
E

http://ronaldbradford.com/blog/why-sql_mode-is-important-part-i-2008-07-17/
14
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A
A
Data integrity R
C
H
❖ Storage Engines I
❖ Always design for transactions T
❖ Always use transactions E
❖ Use a transactional storage engine C
T
U
R
E

15
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A
A
From Top 20 Design Tips R
C
H
I
1. Know your technology tools 11. Monitor Every SQL Statement
2. Know your disk footprint
T
12. The impact of Indexes
3. Choose your numeric data type 13. Index types for design
4. Other data type efficiencies 14. Minimizing internal MySQL Processing E
5. Application data type efficiencies 15. Transactions C
6. NOT NULL 16. Data Integrity is key T
7. Know about character sets 17. Leveraging the Query Cache U
8. When VARCHAR is bad 18. Create Object appropriately R
9. Be Wary of TEXT/BLOB
10. Know every SQL Statement
19. Naming Standards
E
20. Testing, Testing, Testing

http://ronaldbradford.com/presentations.htm#200804
16
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A
A
Further Discussion (Part B) R
C
H
❖ How to design for caching at multiple levels I
❖ Cloud computing considerations (MySQL/ Non MySQL) T
E
❖ Supporting 3rd party applications (REST/SOAP/XMMP) C
❖ CAP Theory (Consistency, Availability, Partition) T
U
❖ Balancing design with agile approaches
R
❖ Hardware requirements (Disk, Memory, Network, CPU) E
❖ Managing different Read/Write environments
17
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A
A
Site presence V
A
I
❖ What happens when your site is: L
❖ Unavailable A
❖ Under load B
❖ Something is broken I
❖ Undergoing maintenance
L
I
❖ Somebody elseʼs mistake (e.g. Data Center) T
Y

18
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A

19
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A

20
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A

Firefox Download day - Example 1 21
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A

Firefox Download day - Example 2 22
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A

23
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A

24
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A
A
Minimum Ideals V
A
I
❖ Static version of site L
❖ Support for Read Only & Write Enabled A
B
❖ Support for components being unavailable I
L
I
T
Y

25
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A
A
Further Discussion (Part B) V
A
I
❖ What is no downtime and how to achieve it? L
❖ Tracking referrer through site navigation A
❖ What is the minimum MySQL configuration? B
❖ The strengths and weaknesses of MySQL Topologies I
L
I
T
Y

26
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A
S
Scalability C
A
L
❖ Successful scaling is? A
B
I
L
I
T
Y

27
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A
S
Scalability C
A
L
❖ Successful scaling is? A
B
I
L
I
T
Y
Scale Out not Scale Up

28
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A
S
Scalability C
A
L
❖ To scale out is to? A
B
I
L
I
T
Y

29
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A
S
Scalability C
A
L
❖ To scale out is to? A
B
I
L
I
T
Y
Shard

30
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A
S
Scale out C
A
L
❖ The art of Sharding is? A
❖ A one day seminar (ask me for more) B
I
L
I
T
Y

31
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A
S
Scale out C
A
L
❖ A database table consists of A
❖ Columns B
❖ Rows I
L
I
T
Y

32
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A
S
Scale out Columns C
A
L
TABLE
row1
col1 col2 col3 col4 col5 col6 col7
A
row2 B
row3
I
L
row4
row5
row6
I
row7
row8 T
row9 Y
row10
row11
row12
row13
row14
33
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A
S
Scale out Columns C
A
L
TABLE
row1
col1 col2 col3 col4 col5 col6 col7
A
row2 B
row3
I
L
row4
row5
row6
I
row7
row8 T
row9 Y
row10
row11
row12
row13
row14
34
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A
S
Scale out Columns C
A
L
TABLE
row1
col1 col2 col3 col4 col5 col6 col7
A
row2 B
row3
I
L
row4
row5
row6
I
row7
row8 T
row9 Y
row10
row11
row12
row13
row14
35
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A
S
Scale out Rows C
A
L
TABLE
row1
col1 col2 col3 col4 col5 col6 col7
A
row2 B
row3
I
L
row4
row5
row6
I
row7
row8 T
row9 Y
row10
row11
row12
row13
row14
36
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A
S
Scale out Rows C
A
L
TABLE
row1
col1 col2 col3 col4 col5 col6 col7
A
row2 B
row3
I
L
row4
row5
row6
I
row7
row8 T
row9 Y
row10
row11
row12
row13
row14
37
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A
S
Scale out Rows C
A
L
TABLE
row1
col1 col2 col3 col4 col5 col6 col7
A
row2 B
row3
I
L
row4
row5
row6
I
row7
row8 T
row9 Y
row10
row11
row12
row13
row14
38
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A
S
Scale out C
A
L
❖ A database consists of A
❖ Tables B
❖ Table Rows I
L
I
T
Y

39
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A
S
Scale out Tables C
A
L
table1 table2 table3 table4 table5
A
B
I
L
I
T
Y

40
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A
S
Scale out Tables C
A
L
table1 table2 table3 table4 table5
A
B
I
L
I
T
Y

41
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A
S
Scale out Tables C
A
L
table1 table2 table3 table4 table5
A
B
I
L
I
T
Y

42
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A
S
Scale out Table Rows C
A
L
row1
table1 table2 table3 table4 table5
A
row2 B
row3
I
L
row4
row5
row6
I
row7
row8 T
row9 Y
row10
row11
row12
row13
row14
43
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A
S
Scale out Table Rows C
A
L
row1
table1 table2 table3 table4 table5
A
row2 B
row3
I
L
row4
row5
row6
I
row7
row8 T
row9 Y
row10
row11
row12
row13
row14
44
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A
S
Scale out Table Rows C
A
L
row1
table1 table2 table3 table4 table5
A
row2 B
row3
I
L
row4
row5
row6
I
row7
row8 T
row9 Y
row10
row11
row12
row13
row14
45
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A
S
Scale out C
A
L
❖ Partitioning is? A
❖ Grouping like rows in a table together B
❖ e.g. I
❖ By Date L
❖ By Local
❖ By parent grouping
I
T
Y

46
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A
S
Scale out Partition C
A
L
TABLE
row1
col1 col2 col3 col4 col5 col6 col7
A
row2 B
row3
I
L
row4
row5
row6
I
row7
row8 T
row9 Y
row10
row11
row12
row13
row14
47
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A
S
Scale out Partition C
A
L
TABLE
row1
col1 col2 col3 col4 col5 col6 col7
A
row2 B
row3
I
L
row4
row5
row6
I
row7
row8 T
row9 Y
row10
row11
row12
row13
row14
48
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A
S
Scale out Partition C
A
L
TABLE
row1
col1 col2 col3 col4 col5 col6 col7
A
row2 B
row3
I
L
row4
row5
row6
I
row7
row8 T
row9 Y
row10
row11
row12
row13
row14
49
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A
S
Scale out C
A
L
❖ Sharding is? A
❖ Grouping like tables together B
❖ optionally partition subsets of rows I
L
I
T
Y

50
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A
S
Scale out Table Group C
A
L
table1 table2 table3 table4 table5
A
B
I
L
I
T
Y

51
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A
S
Scale out Table Group C
A
L
table1 table2 table3 table4 table5
A
B
I
L
I
T
Y

52
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A
S
Scale out Table Group C
A
L
table1 table2 table3 table4 table5
A
B
I
L
I
T
Y

53
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A
S
Scale out Table Group + RowsC
A
L
table1 table2 table3 table4
A
table5

B
I
L
I
T
Y

54
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A
S
Scale out Table Group + RowsC
A
L
table1 table2 table3 table4
A
table5

B
I
L
I
T
Y

55
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A
S
Scale out Table Group + RowsC
A
L
table1 table2 table3 table4
A
table5

B
I
L
I
T
Y

56
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A
S
Scale out Table Group + RowsC
A
L
table1 table2 table3 table4
A
table5

B
I
L
I
T
Y

57
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A
S
Scale out C
A
L
❖ How do you group tables? A
B
I
L
I
T
Y

58
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A
S
Scale out C
A
L
❖ How do you group tables? A
B
I
L
I
T
Y
Thatʼs the secret
59
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A
S
Further Discussion (Part B) C
A
L
❖ The partition key is the key A
❖ Poor examples of sharding B
I
❖ The theory of ideal partition key L
❖ Balancing and re balancing I
T
Y

60
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A
S
Further Discussion (Part B) C
A
L
❖ Master/Master does not solve scalability A
❖ Store data in a database, donʼt always retrieve B
from the database
I
L
❖ Cache at multiple layers I
T
Y

61
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A
A
Accountability C
C
O
❖ The rule of everything U
❖ You should look at your logs N
T
A
B
I
L
I
T
Y
62
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A
A
The Rule of Everything C
C
O
❖ Everythingʼs are U
❖ Monitor N
❖ Measure T
❖ Graph A
❖ Automate
B
I
L
I
T
Y
63
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A
A
Easy Information to know C
C
O
❖ Know your error logs U
❖ System, Database, Application, RAID N
❖ Seen degraded RAID multiple times *** T
A
❖ donʼt ignore errors (such as Innodb) B
I
L
I
T
Y
64
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A
A
Further Discussion (Part B) C
C
O
❖ Testability U
❖ Donʼt think of testing as to know your software works, N
testing should be to break your software T
❖ Know every single SQL Statement A
B
❖ Your application should have detailed metrics I
L
I
T
Y
65
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A
A
Executive Summary R
C
H
Architecture I
T
E
❖ Learn and know MySQL specifics C
❖ Disk = Memory = Performance T
U
❖ Integrity is important, but how important is
R
consistency? E

66
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A
A
Executive Summary V
A
I
Availability L
A
B
❖ Test your site response under error cases I
❖ Disaster is inevitable. Donʼt wait for it L
I
❖ Try testing unavailability first
T
❖ Write failing test cases for your website first Y

67
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A
S
Executive Summary C
A
L
Scalability A
B
I
❖ Think low cost commodity hardware L
❖ like general cloud computing architecture I
❖ Understanding sharding from day 1 T
Y
❖ XMPP

68
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A
A
Executive Summary C
C
O
Accountability U
N
T
❖ Donʼt let users tell you when things donʼt work A
❖ You need numbers to predicate failure B
I
❖ You have to have history for trending
L
❖ Log, Review & Monitor all SQL I
T
Y
69
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A

Whatʼs next

❖ Any key point could be an hour discussion

❖ What 3 points are best topics for audience?

70
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A

Looking forward

A word on cloud computing

❖ Donʼt put all your eggs in one basket
❖ Keep your important data close

71
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A

Professional Help is Available

❖ 2 decades Expertise & Experience
❖ Architecture, Design, Performance & Scalability
❖ 9 years in MySQL

ronaldbradford.com
42sql.com
72
(c) Copyright 2008 http://42sql.com
Best Design Practices - Part A

Building Scalable Enterprise Solutions
Architecture, Performance, Scalability, Migration, Knowledge Transfer

73
(c) Copyright 2008 http://42sql.com