You are on page 1of 26

Distributed Hash Tables

David Tam Patrick Pang

Presentation Outline
• What is DHT (Distributed Hash Table)?


• • • • • • • • • •

Why DHTs?
Applications How lookup works? Alternatives to DHTs Performance – Routing Performance – Load Balancing Security – Routing Attack Security – Inconsistent Behaviour Comparison to Other Facilities Current Research Projects Conclusion

What is DHT? Distributed application put(key. • Nodes uniformly distributed across key space • Nodes form an overlay network • Nodes maintain list of neighbours in routing table • Decoupled from physical network topology (Figure adopted from Frans Kaashoek) . data) get (key) Distributed hash table node …. data node node DHT provides the information look up service for P2P applications.

Why DHTs? Why Middleware? • Simplifies the development for large-scale distributed Apps • Better security and robustness • Simple API Why Do We Need DHTs? • Simplifies the development for large-scale distributed Apps • Better security and robustness • Simple API • Exploits P2P resources .

Applications • Anything that requires a hash table • Databases. storage. caching • Content distribution • Query & indexing • Naming systems • Communication primitives • Chat services • Application-layer multi-casting • Event notification services • Publish/subscribe systems ? . archival • Web serving. FSes.

6) [6.4) succ.] Finger Table for Node 2 15 14 13 12 4 5 10 7 9 8 6 0 1 2 3 start interval 3 [3.How lookup works? Example: Chord [Stoica et. al. 5 4 6 10 [4.2) 5 7 10 11 .10) [10.

2) [2.12) succ.10) 12 14 2 11 . 12 12 14 2 [12.14) [14.How lookup works? Example: Chord Finger Table for Node 10 15 14 13 12 4 5 10 7 9 8 6 0 1 2 3 start interval 11 [11.

10) 12 14 2 11 .How lookup works? Example: Chord Finger Table for Node 10 15 14 13 12 4 5 10 7 9 8 6 0 1 2 3 start interval 11 [11. 12 12 14 2 [12.12) succ.2) [2.14) [14.

15 0 2 6 [0.2) [2.How lookup works? Example: Chord Finger Table for Node 14 15 14 13 12 4 5 10 7 9 8 6 0 1 2 3 start interval 15 [15.0) succ.6) [6.13) 1 2 7 11 .

0) succ.13) 1 2 7 11 .2) [2.How lookup works? Example: Chord Finger Table for Node 14 15 14 13 12 4 5 10 7 9 8 6 0 1 2 3 start interval 15 [15.6) [6. 15 0 2 6 [0.

0 1 2 3 4 5 12 11 10 7 9 8 6 .How lookup works? Example: Chord 15 14 Now Node 2 can retrive information for key 0 from Node 1.

Alternatives to DHTs • Distributed file system • Centralized lookup • P2P flooding queries Server N1 N4 Target N2 N6 N3 Start N10 N9 Client N7 N8 Client Internet N1 N4 Client N2 DB N6 Target N3 N8 Start N10 Client Server N9 N7 (Figures adopted from Frans Kaashoek) .

try to get closer to locating target node • Ask a closer neighbour • Performance & scalability tied directly to lookup algorithm 2 Aspects to Performance • Path latency • Lookup path length (# hops) 2 Aspects to Scalability • size of routing table – O(log N) • lookup path length – O(log N) 3 Techniques • proximity lookup • proximity neighbour selection • geographic layout .to locate a target node •Each step.Lookup Purpose -.Performance -.

Performance -.Load Balancing Issues • Hot-spots • Content • Lookup • Heterogeneous nodes & paths • System flux Solution • Replication is the key • Also good for fault-tolerance • Cache lookup answers backwards along path .

5 5 7 10 0 15 14 1 2 3 4 5 13 12 11 10 7 9 8 Node 2 to Node 10: Please tell me how to reach key 0 ….2) succ. 6 . give a wrong answer Finger Table for Node 2 start 3 4 6 10 interval [3.10) [10.Security – Incorrect Lookup (1) • When asked for the “next hop”.4) [4.6) [6.

10) succ. give a wrong answer Finger Table for Node 10 start 11 12 14 2 interval [11.14) [14.12) [12.Security – Incorrect Lookup (2) • When asked for the “next hop”. Node 10 answers: ask Node 14 6 . 12 12 14 2 0 15 14 1 2 3 4 5 13 12 11 10 7 9 8 Node 2 to Node 10: Please tell me how to reach key 0 ….2) [2.

Node 14 answers: ask Node 10 6 .13) succ.Security – Incorrect Lookup (3) • When asked for the “next hop”.6) [6. 15 1 2 7 0 15 14 1 2 3 4 5 13 12 11 10 7 9 8 Node 2 to Node 14: Please tell me how to reach key 0 …. give a wrong answer Finger Table for Node 14 start 15 0 2 6 interval [15.0) [0.2) [2.

• The concept of “monotonically decreasing quantity” has been used in program construction guaranteeing total correctness. using an integral monotonically decreasing quantity to implement the idea of “progress”. [Parnas] .Security – Incorrect Lookup (4) Solution [Sit and Morris]: • “Define verifiable system invariant” • “Allow the querier to observe lookup progress” Our idea how this can be implemented: • Concretely.

i.e.. lie intelligibly • Sybil attack [Kaashoek] Solution 1: public key solution .Security – Inconsistent Behaviour • Inconsistent Behaviour.

e.. i.Security – Inconsistent Behaviour • Inconsistent Behaviour. lie intelligibly • Sybil attack [Kaashoek] Solution 1: public key solution Solution 2: Byzantine Protocol Byzantine Generals Problem: How to find out the traitors among the Generals? [Lamport] .

i.Security – Inconsistent Behaviour • Inconsistent Behaviour.e.. lie intelligibly • Sybil attack [Kaashoek] Solution 1: public key solution Solution 2: Byzantine Protocol Commander Byzantine Generals Problem: How to find out the traitors among the Generals? [Lamport] Lieutenant 1 “he said „retreat‟” attack attack Lieutenant 2 .

e..Security – Inconsistent Behaviour • Inconsistent Behaviour. i. lie intelligibly • Sybil attack [Kaashoek] Solution 1: public key solution Solution 2: Byzantine Protocol Commander Byzantine Generals Problem: How to find out the traitors among the Generals? [Lamport] Lieutenant 1 “he said „retreat‟” attack retreat Lieutenant 2 .

Comparison to Other Facilities Facility DHT Centralized Lookup P2P flooding queries Distributed FS Facility DHT Centralized Lookup Distributed FS Abstraction Easy Use/Prg Scalability high medium medium low high medium high medium high low low medium Load-Balance yes no no no Fault-Tolerance Self-Org Admin high low medium yes no yes no low medium low high P2P flooding queries depends .

incremental scalability .middleware platform for internet service construction -.cluster-based -.Research Projects Iris – security & fault-tolerance – US Gov‟t Chord – circular key space Pastry – circular key space Tapestry – hypercube space CAN – n-dimensional key space Kelips – n-dimensional key space DDS -.

Summary • Good middleware platform • Exploits P2P networks • An exciting new research area .

Security Considerations for Peerto-Peer Distributed Hash Tables • Kaashoek. Ion et. Connecting Theory to Practice: Software Engineering Programme . D.References • Lamport. Frans. Morris. al. robust distributed applications • Stoica. The Byzantine Generals Problem • Sit. Distributed Hash Tables – Building largesacle. L. al. Robert. Emil. Leslie et. Chord: A scalable peer-to-peer lookup service for Internet applications • Parnas.

.91472 W 5498!!309478 W 30.93307080.70.705.7 W 44/2//0.7.$:22..

-08 W ..39300307.8%.9.8   4330.55..73.0$419. 4309 .55.70 8.-05007 94 500744:5 807..70 330073!747.840 7.220 .:79438/07...38 897-:90/..25479 08009 .93%04794!7.8!74-02 W $9 2 4778 #4-079 $0. 47/8.8%.08 W .-08 :/3.9438 W $94.. %0..#010703..9438 W !.01473907309.0 74-:89/897-:90/.9438147!007 94 !007897-:90/.