You are on page 1of 28

Amazon's Dynamo DB

Aditya Dasgupta (adasgpt2) CS 598 Cloud Computing

SQL Era
Started around 1970 A Relational Model of Data for Large Shared Data Banks IBM -> System R -> DB2 Microsoft -> Microsoft SQL Server Open Source -> MySQL and PostgreSQL Relational Software -> Oracle

NoSQL Era
NoSQL was used by Carlo Strozzi to name his lightweight, open-source, relational DB Eric Evans, employee of Rackspace, reintroduced the term in 2009

What is NoSQL?
Distributed No joins No Schema Didn't attempt to provide ACID guarantees

Example of NoSQL
Key-value DB: Like a big hash table. One key and one value. o Dynamo (Amazon) o Redis o Voldemort (LinkedIn) Column(Family) based DB: Storing data by columns. o BigTable (Google) o Cassandra (Facebook) Document Based DB: Document based. Versioned. o MongoDB o CouchDB Graph DB: Graph based database. Think about relationship. o Neo4j o FlockDB (Twitter)

CAP Theorem: for large scaled distributed systems


CAP Theorem states that it is impossible for a distributed computer system to simultaneously provide all three of the following guarantees: Consistency - all nodes see the same data at the same time Availability - a guarantee that every request receives a response about whether it was successful or failed Partition Tolerance - the system continues to operate despite arbitrary message loss or system failure

So, what is consistency?


Strong Consistency - After the update completes, any subsequent access will return the updated value. Weak Consistency - The system does not guarantee that subsequent accesses will return the updated value. A number of conditions need to be met before the value will be returned. Eventual Consistency - A specific form of weak consistency. Guarantee that if no new updates are made to the object, eventually all accesses will return the last updated value.

And, why not consistency?

"Anything which needs agreement will eventually fail at scale." - Werner Vogels, CTO of Amazon

Other Systems

Dynamo: System Assumptions and Requirements


Query Model: simple read and write operations to a data item that is uniquely identified by a key. ACID Properties: Atomicity, Isolation, Durability. Efficiency: latency requirements measured at the 99.9th percentile of the distribution. Other Assumptions: operation environment non-hostile

Dynamo:Service Level Agreements (SLA)


Application:delivers its functionality in a bounded time Service guarantee- provide a response within 300ms for 99.9% of its requests

Dynamo:Design Consideration
Trades off strong consistency for availability Conflict resolution is executed during read instead of write, i.e. always writeable .

Dynamo: Key challenges


Key challenges addressed: 1.Incremental scalability 2.Availability 3.Eventual Consistency 4.Handling Failures

1.Incremental Scalability
Problem: Partition data over the set of servers to achieve incremental scalability. Given a key you want to figure out which machine stores the data. Simple way is to follow a modulo arithmetic approach: key mod N What if you add a machine. WIll key mod(N+1) give the answer?

1.Incremental Scalability
What if a node(server) becomes unavailable? How do you distribute the load? What if a node(server) becomes available again? How do you distribute the load?

Solution: Consistent Hashing


Solution: Consistent Hashing Consistent hashing: the output range of a hash function is treated as a fixed circular space or ring . Virtual Nodes : Each node responsible for more than one virtual node.

Solution: Consistent Hashing


Advantages of Consistent Hashing: Node unavailable: load handled by this node evenly dispersed across the remaining available nodes. Node available again: accepts a roughly equivalent amount of load from each of the other available nodes. # of virtual nodes a node is responsible: decided based on its capacity, accounting for heterogeneity in the physical infrastructure

2.Availability
Problem: Need to ensure that system is available during server and network failures.

Solution: Replication
Each data item is replicated at N hosts. Co-ordinator node replicates these keys at N-1 clockwise successor nodes Preference list : The list of nodes that is responsible for storing a particular key.

3.Eventual Consistency
Need to provide eventual consistency which allows for updates to be propagated to all replicas asynchronously

3.Eventual Consistency
A put() call may return to its caller before the update has been applied at all the replicas A get() call may return many versions of the same object. Challenge: an object having distinct version sub-histories, which the system will need to reconcile in the future. Solution: uses vector clocks in order to capture causality between different versions of the same object.

Solution: Vector Clock


A vector clock is a list of (node, counter) pairs. Every version of every object is associated with one vector clock. If the counters on the first object s clock are less-than-orequal to all of the nodes in the second clock, then the first is an ancestor of the second and can be forgotten.

Vector Clock Example

4.Handling Failures
Need to handle network and server failures for read & write operations to execute without latency

Solution: Sloppy Quorom


R/W is the minimum number of nodes that must participate in a successful read/write operation. Setting R + W > N yields a quorum-like system. In this model, the latency of a get (or put) operation is dictated by the slowest of the R (or W) replicas. For this reason, R and W are usually configured to be less than N, to provide better latency. Use preference list instead of first healthy N nodes

Solution: Hinted Handoff


Assume N = 3. When A is temporarily down or unreachable during a write, send replica to D. D is hinted that the replica is belong to A and it will deliver to A when A is recovered. Again: always writeable

Conclusion
Stands out amongst other databases in terms of availability Can still work in case of multiple node failures Robust failure management system. No single point of failure Can tune N,R and W.Unique to Dynamo Hence meet desired performance,durability and consistency SLAs

THANK YOU !!!

You might also like