A NoSQL Rollercoaster

Saturday, November 6, 2010

Saturday, November 6, 2010
 IMS
The Hierarchical
Database

(1966)

Vern Watts
Saturday, November 6, 2010
 “A Relational
Model for Large
Shared
Databanks”

(1970)

Ted Codd
Saturday, November 6, 2010
 “"SEQUEL: A
Structured
English Query
Language"
Don Chamberlin & Ray
Boyce

(1974)

Don Chamberlin
Saturday, November 6, 2010
 ACID

(late 1970’s)

Jim Gray
Saturday, November 6, 2010
 “NoSQL”
coined

(1998)

Carlo Strozzi
Saturday, November 6, 2010
 Eric Evans

“NoSQL” Reintroduced
(2008)

Saturday, November 6, 2010

 select fun, profit from
real_world where
relational=false?

Saturday, November 6, 2010

 “In striving to make every
user happy, a technology an
actually leave the majority
unhappy.”

“Every good idea is

generalized to its level of
inapplicability.”

(Peter Principle)

Saturday, November 6, 2010

Saturday, November 6, 2010
 Why?

Saturday, November 6, 2010

 Internet Scale
• Massive data
collections

• Huge number of
requests

• Coming from
geographic areas
across the globe

• 24/7

Saturday, November 6, 2010

 Total Cost of Ownership

• The price of a license

• The price of support

• The price of hardware

Saturday, November 6, 2010

 Availability

Saturday, November 6, 2010

 Data Models

Saturday, November 6, 2010

 Column Oriented
Column Family ≈ Table Can grow “indefinitely”

named named named named named

key
column column column column column

Empty cells are

cheap (sparse
table)
No
Schemaless
secundary
indexes

Saturday, November 6, 2010

 Column Oriented + Super Columns
named named named named named
key
column column column column column

Super Columns
named named named
column column column
…
…

Saturday, November 6, 2010

 Key Value Store

1011
•Schemaless 0110
•Versioning

Saturday, November 6, 2010

 Graph Database

SPARQL?

Saturday, November 6, 2010

 Document Store
XML
<persons>
<person>
<name>Wilfred</name>
JSON
<surname>Springer
</person>
[{ "Name" :
…
"Wilfred",
</persons>
"Surname" :
"Springer"},
…
]

Improved
Indexing Serverside
Processing

Saturday, November 6, 2010

 Challenges

Saturday, November 6, 2010

 CAP
Theorem
Eric Brewer

Saturday, November 6, 2010

 Availability Consistency

Partition Pick two

Tolerance

Saturday, November 6, 2010

 Consistency
Consistency

Strong Weak
Consistency Consistency

Eventual
Other
Consistency

Read-your-
Causal Session Monotonic Read Monotonic Write
writes
Consistency Consistency Consistency Consistency
Consistency

Saturday, November 6, 2010

 Strong Consistency
1
0 value = "foo"

value = "bar" 2
B
A
value = "bar"

C
2 value = "bar"
value = "bar"

After the update, any subsequent access will return the

updated value.

Saturday, November 6, 2010

 Weak Consistency
B
0 value = "foo"

>1
1 value = "bar"

A
value = "bar" / "foo"

>1

C
value = "bar" / value = "bar" / "foo"
>1 "foo"

The system does not guarantee that at any given point in

the future subsequent access will return the updated
value

Saturday, November 6, 2010

 Eventual Consistency
B
0 value = "foo"

1 value = "bar" t

A
value = "bar"

C
value = "bar"
t value = "bar"

t≥1

If no updates are made to the object, eventually all

accesses will return the last updated value.

Saturday, November 6, 2010

 Session Consistency
B
Session 1
0 value = "foo"

1 value = "bar"

A 2 value = "bar"

2 value = "foo"
C
Session 2

Within the “session”, the system guarantees read-your-

writes consistency

Saturday, November 6, 2010

 Partition Tolerance
replicates
new value

reads new
value
writes new value

Saturday, November 6, 2010

 Partition Tolerance
replicates
new value

!
reads new
value
writes new value

Saturday, November 6, 2010

 Partition Tolerance
fails to
replicate
new value

reads old
value
writes new value

Saturday, November 6, 2010

 Partition Tolerance
fails to
replicate
new value

failing attempt
to write a new
value

Saturday, November 6, 2010

 Byzantine Failures

Saturday, November 6, 2010

 Faults that might
occur in a
distributed
system
•Lying
•Fabrication of messages
•Collisions
•Selective non-participation

Saturday, November 6, 2010

 Byzantine
Generals
Problem

Saturday, November 6, 2010

 Solution
Strategies

Saturday, November 6, 2010

 Strong
Consistency
1

Saturday, November 6, 2010

 Παξοί

Saturday, November 6, 2010

 Leader Proposer

Four Roles

Acceptor Learner

Saturday, November 6, 2010

 Paxos in a Nutshell

Request
Propose
Promise
Accept
Accepted
Response

Saturday, November 6, 2010

 Processors Consensus
combine based on
various majority vote
roles

Paxos guarantees
progress for 2F + 1
processors in face of
failure of F processors

Saturday, November 6, 2010

 “Either Paxos,
Paxos with
cruft, or
broken”

Mike Burrows

(Google Chubby)

Saturday, November 6, 2010

 Partitioning
2

Saturday, November 6, 2010

 Consistent Hashing
Key K
A
H B

G C

F D
E

Saturday, November 6, 2010

 High
Availability
3

Saturday, November 6, 2010

 Replication
A
H B

G C

F D
E

Saturday, November 6, 2010

 Replication Factor
W=3

N=4
R=2

Saturday, November 6, 2010

 Reconciliation during reads
W=3

N=4
Reconciliation
R=2

Saturday, November 6, 2010

 Handling
Temporary
Failures
4

Saturday, November 6, 2010

 Problem
Reconcilation Alice Ben Dave Cathy

Wednesday?

Thursday

Tuesday

What's it gonna be?

Tuesday ?X Thursday
? ? ?

Saturday, November 6, 2010

 Vector Clocks
date = Wednesday
vclock = Alice:1

date = Tuesday
vclock = Alice:1, Ben:1

date = Tuesday
vclock = Alice:1, Ben:1, Dave:1

date = Tuesday
vclock = Alice:1, Cathy:1

date = Thursday
vclock = Alice:1, Ben:1, Cathy:1, Dave:2

Saturday, November 6, 2010

 Conflict Resolution

date = Tuesday date = Thursday

vclock = Alice:1, Ben:1, Dave:1 vclock = Alice:1, Ben:1, Cathy:1, Dave:2

Saturday, November 6, 2010

 Recovering from
permanent
failures
5

Saturday, November 6, 2010

 What if nodes die permantly?
A
H B

G C

F D
E

Saturday, November 6, 2010

 Merkle Trees
# #

# # # #
# #
# #
# #
# #

Saturday, November 6, 2010

 Enough is enough!!!

Saturday, November 6, 2010

 Conclusions

Saturday, November 6, 2010

 Conclusions

Scale!!!

Saturday, November 6, 2010

 Conclusions
No free ride
Scale!!!

Saturday, November 6, 2010

 Conclusions
No free ride
Scale!!! You gain some

Saturday, November 6, 2010

 Conclusions
No free ride
Scale!!! You gain some
You lose some

Saturday, November 6, 2010

 Conclusions
No free ride
Scale!!! You gain some
You lose some
Choice

Saturday, November 6, 2010

 Conclusions
No free ride
Scale!!! You gain some
You lose some
Choice
Contr
ol

Saturday, November 6, 2010

 Conclusions
No free ride
Scale!!! You gain some
You lose some
Choice
Contr
ol
Take control

Saturday, November 6, 2010

 Conclusions
No free ride
Scale!!! You gain some
You lose some
Choice
Contr
ol Understand your options
Take control

Saturday, November 6, 2010

 Conclusions
No free ride
Scale!!! You gain some
You lose some
Choice
Contr
ol Understand your options
Take control
Fun!!!

Saturday, November 6, 2010

 Conclusions
No free ride
Scale!!! You gain some
You lose some
Choice
Contr
ol Understand your options
Take control
Fun!!!
Changes your way o
f thinking

Saturday, November 6, 2010

 Conclusions
No free ride
Scale!!! You gain some
You lose some
Choice
Contr
ol Understand your options
Take control
Fun!!!
Changes your way o
f thinking

Saturday, November 6, 2010

 No SQL!!!

wilfredspringer@gmail.com

Saturday, November 6, 2010

 BASE

• Basically Available

• Soft State

• Eventually Consistent

Saturday, November 6, 2010