Lecture 11: Distributed Graph

Algorithms with Limited

Communication

1
Distributed Graph Algorithms
• Network graph
• Every is a processor
• Unique identifier
• Knows neighbors
• Synchronous rounds:
1. Send messages to neighbors
2. Receive messages from neighbors
3. Transition to next state
Distributed Network Models
CONGEST model:
#bits per round
Today
• CONGEST algorithms:
• “Trivial” upper bound is because this is the full graph on nodes
• Minimum-weight spanning tree
• Spanners (approximate shortest paths)
• Lower bounds ?

4
The “Trivial” Upper Bound
• Let be a graph property
• How hard to decide ?

• Can we “collect” all edges in one node?

• We can, but we are limited to bits per round so we will have to find a way to
actually do it we will use pipelining

5
Pipelining
• Initially: each has tokens
• Goal: each should output
Denote as the tokens we learned so far, but didn’t send yet
Denote as the tokens we already processed and sent
• Algorithm:
• Initialize ,
• While :
• Choose
• Send to neighbors,
• Add to tokens received from neighbors
• Output
6
 • Initialize ,
Pipelining • While “not done”:
• Choose
• Send to neighbors
• Lemma:
• Let
• Let have
• Then by round , either
• , or
• already sent other tokens. distance
• Corollary: 𝑢 𝑣
• Suppose
• After rounds? 𝑡

7
Back to the “Trivial” Algorithm
• Initially: each knows
• In total:
• Using pipelining: learn in rounds

8
Minimum-Weight Spanning Tree

9
Problem Setup

• Note that will have to be send across the network, so it also limited to bits
• Each knows for each
• Goal: output minimum-weight spanning tree
• Each outputs its edges in

10
Boruvka’s MST Algorithm

Grow forest:
1. Each component chooses
lightest outgoing edge

2. Unify components
3. Repeat
Output

11
Boruvka’s MST Algorithm

Grow forest:
1. Each component chooses
lightest outgoing edge

2. Unify components

12
Boruvka’s MST Algorithm

Grow forest:
1. Each component chooses
lightest outgoing edge

2. Unify components

13
Boruvka’s MST Algorithm

Grow forest:
1. Each component chooses
lightest outgoing edge

2. Unify components
3. Repeat

14
Boruvka’s MST Algorithm

Grow forest:
1. Each component chooses
lightest outgoing edge

2. Unify components
3. Repeat

15
Boruvka’s MST Algorithm
After each round: #components #components/2

Grow forest:
1. Each component chooses
lightest outgoing edge

2. Unify components
3. Repeat
Output
iterations
16
Distributed Implementation
• Components
• Each :
• Connected
• Leader
• MST inside
• All nodes of know
• Iteration: in parallel for ,
• Find min-weight outgoing edge for rounds
• Merge components

17
Finding the Min-Weight Outgoing Edge
• Each finds
• How to find ?

• Case 1:

18
Finding the Min-Weight Outgoing Edge
• Each finds
• How to find ?

• Case 2:
• Step 1: propagate inside component for rounds

19
Finding the Min-Weight Outgoing Edge
• Each finds
• How to find ?

• Case 2:
• Step 1: propagate inside component for rounds
• Step 2: pipeline surviving pairs

20
Merging Components
• Components
• Each :
•
•
Connected
Leader
𝑆𝑖 𝑆 𝑗
• MST inside
• All nodes of know

21
Informing Merged Nodes
• Our worry:

𝑆1 𝑆 2 𝑆 3 … 𝑆𝑘
• Want to prevent “long chains”

22
Informing Merged Nodes
• Choose “Leaf” or “Center”:

𝑆1 𝑆 2 𝑆 3 … 𝑆𝑘
𝐿 𝐿 𝐶 𝐿

• #merges

23
Merging Star Shapes

L L
C

Need to disseminate:
(1)Coin value
L
(2)Leader ID
(3)Size of merged component

24
Conclusion
• Iteration: in parallel for , rounds
• Find min-weight outgoing edge for
• Merge components

• Is this running time “good”?

25
Spanners

26
Spanners
• is a -spanner of if:
• is a subgraph of
• For every :

• Additive spanner:
• Multiplicative spanner:

27
Today
• Additive +2 spanner of size
• Baswana-Sen spanner: multiplicative, , of size

• Erdős Girth Conjecture: there exists a graph with

• edges,
• Girth

28
Additive +2 Spanner
Idea:
• Choose centers at random
• Compute shortest paths from all
• Hope: every long path hits many centers

29
Additive +2 Spanner
Implementation:
• If : add all ’s edges to
• Choose centers : each included w.p.
• For each , in parallel: compute BFS tree, add to

• Expected size?

30
Bounding the Stretch
• Path in :

𝑢0
𝑢𝑘

• Case I: every has

31
 Bounding the Stretch
𝑑𝑖𝑠 𝑡 𝐻 ( 𝑐,𝑢𝑘 )
• Path in : 𝑑𝑖𝑠 𝑡 𝐺 ( 𝑐 ,𝑢 𝑘 ) ≤ 𝑑𝑖𝑠𝑡 𝐺 ( 𝑢 𝑖 ,𝑢 𝑘 ) +1
𝑐∈𝐶
𝑑𝑖𝑠 𝑡 𝐺 ( 𝑢0 , 𝑐 ) ≤ 𝑑𝑖𝑠 𝑡 𝐺 ( 𝑢0 ,𝑢𝑖 ) +1
⊆𝐻
𝑑𝑖𝑠 𝑡 𝐻 ( 𝑢0 ,𝑐 ) 𝑢0 ⊆𝐻 𝑢𝑖
𝑢𝑘

• Case I: every has

 • Case II: some has
𝑑𝑖𝑠 𝑡 𝐻 ( 𝑢0 , 𝑢𝑘 ) ≤ 𝑑𝑖𝑠𝑡 ( 𝑢0 ,𝑢 𝑘 ) +2
w.h.p.,

32
The Baswana-Sen Spanner
• Grow clusters (trees) around centers
• Initially:
• All nodes are centers
• Each node in its own cluster
• Gradually “kill off” centers & re-cluster nodes
The Baswana-Sen Spanner
• For :
• Each center survives w.p. * 𝐶 𝑘=∅
• If ’s cluster center died:
• If neighbor in surviving cluster: assign to ’s cluster
𝑣 𝑢
+ add to spanner

• Else: remove from𝑣 the graph + add one edge to each

previous adjacent
cluster
Analysis: Size of the Spanner
Edges are added when: 𝑢
𝑣
• Node is re-clustered:
• At most once per level edges total
• Node is removed:
• No adjacent cluster survived 𝑣
was adjacent to clusters (w.h.p.)
edges added (one per adjacent cluster)
Total:
Analysis: Stretch 2ℓ

• At level cluster diameter

• Let
• Suppose removed no later than :

Level :
𝑢 𝑣
Analysis: Stretch 2ℓ

• At level cluster diameter

• Let
• Suppose removed no later than :

Level :
Path of length
𝑢 𝑣

Stretch
Lower Bounds

39
The Plan
• 2-party communication complexity
• New problem: Set Disjointness
• Deterministic lower bound
• Diameter lower bound
• MST lower bound

40
Communication Complexity
=?

𝑛 𝑛
𝑋 ∈ { 0,1 } 𝑌 ∈ { 0,1 }
Set Disjointness
• Input:
• Output: ?
• Theorem [Kalyanasundaran, Schnitger ‘92, Razborov ‘92]:
• Randomized CC =
• Easy to see for deterministic protocols
 Deterministic Protocols

• A protocol specifies, at each point:

• Which player speaks next
• What should the player say
• When to halt and what to output
• W.L.O.G.,
• Players alternate
• Each player says one bit in their turn
Deterministic Lower Bounds
0 𝑛 … 1 𝑛
𝑛
0

#rectangles
…

Same
output
Lower bound strategy:
Prove that cannot be
partitioned into few
monochromatic rectangles
𝑛
1
Fooling Set Lower Bound
• -fooling set: such that (𝑥, 𝑦) (𝑥′, 𝑦 )
• for all
• If then (𝑥, 𝑦 ′ ) (𝑥′, 𝑦 ′)
• or

• Every input in needs its own rectangle

45
Set Disjointness Lower Bound
• Fooling set for set disjointness:

• Is it a fooling set?

46
CONGEST Lower Bound Example
• [Frischknecht, Holzer, Wattenhofer ’12]: “Networks cannot compute
their own diameter in sublinear time”
• In the CONGEST model with bandwidth ,
• Computing the diameter requires rounds, even when
• -approximation requires rounds
Diameter Lower Bound
• Reduction from DISJOINTNESS with elements
• Suppose is a distributed algorithm that computes the diameter
• Given ,
• Alice and Bob jointly construct a graph
• Simulate on
• From ’s output, learn whether
The Graph
• “Clique with spines”:
Spine nodes have
no other edges
The Graph
The Graph

Connect Connect
“top ” to “top ” to
“bottom ” “bottom ”

Edges to all clique nodes on Alice’s side Edges to all clique nodes on Bob’s side
The Graph
From Alice’s top to Bob’s bottom :
The Graph
From Alice’s top to Bob’s bottom :

if
The Graph
From Alice’s top to Bob’s bottom :

if
The Graph
From Alice’s top to Bob’s bottom :
Properties of the Graph

• If , then
• If , then
Simulating
bits per round of

Connect Connect
“top ” to “top ” to
“bottom ” “bottom ”

Edges to all clique nodes on Alice’s side Edges to all clique nodes on Bob’s side
Conclusion
• If runs for rounds…
• We solve DISJOINTNESS with elements in rounds
MST Lower Bound of
• Again: reduce from 2-party DISJOINTNESS

59
Our Reduction
• Let be an -round algorithm for MST
• Given inputs to Alice and Bob (resp.):
• Jointly “construct”
• Simulate on
• Deduce

60
Lower Bound Graph
To prove lower bound of :
• Low diameter ( )
• Weight of MST in answer to
• Can simulate efficiently:
• Go for rounds
• Simulation cost = bits per round

61
Input-Independent Part of

paths of length
Input-Independent Part of
Weight
Weight

paths of length
Input-Independent Part of
Weight
Weight

paths of length
Input-Independent Part of
Weight
Weight

paths of length
Input-Independent Part of
Weight
Weight

paths of length
Input-Dependent Edges
Weight
Weight

1
paths of length
1

0
Input-Dependent Edges
Weight
Weight

: :

0 0

1 0

1 1

0 1
Simulating on
Weight
Weight

: :

0 0

1 0

1 1

0 1
Simulating on
• Each is locally simulated by one or both players
• “Locally simulated”:
• Store current state of
• Each round: Need to know:
• Compute messages sends ’s neighbors +
• Feed to messages it receives edge weights
• Update
• Eventually: learn output
Need to know:
messages sent by ’s
What can we simulate using neighbors
messages per round?
Simulating Path Nodes
Weight
Weight

: :

0 0

1 0

1 1

0 1

round 0
Simulating Path Nodes
Weight
Weight

: :

0 0

1 0

1 1

0 1

round 1 round 0
Simulating Path Nodes
Weight
Weight

: :

0 0

1 0

1 1

0 1

round 2 round 1 round 0

Simulating Tree Nodes
Weight
Weight

: :

0 0

1 0

1 1

0 1
Conclusion
Weight
Weight

Accounting:
: :

0 0

1 0

1 1

0 1
-Freeness in CONGEST

76
Algorithm for
• The challenge:

77
Algorithm for
Step 1: finding light cycles
• is light if
• Find cycles of only light nodes:

78
Algorithm for
Step 2: finding non-light cycles
• is heavy if
• If contains a heavy node :
• Sample random
• Check if belongs to some
• BFS to depth 4 from

• Start depth-2 BFS from each neighbor of …

79
Algorithm for
• Start depth-2 BFS from all neighbors of
• Claim: receive only one BFS token!
• Suppose otherwise…

80
Linear Lower Bound for -detection
• Claim:
-round algorithm for detection
protocol for DISJ using bits
• Reduction outline:
• Alice and Bob get inputs
• Construct input graph on nodes, s.t. contains
• Simulate the run of -detection algorithm on
Construction of
1 1
2 2
3 3
4 4

1 1
2 2
3 3
4 4
Construction of
1 1
2 2
3 3
4 4

Top-bottom: Bottom-top:

1 1
2 2
3 3
4 4
Construction of
1 1
2 2
3 3
4 4

( 𝟐 , 𝟒 ) ∈ 𝑿 ∩𝒀

1 1
2 2
3 3
4 4
Simulating the Algorithm
• Alice simulates nodes, Bob simulates
• To simulate one round, each player:
• Locally computes message broadcast by each node it simulates
• Sends all messages to the other player
• Cost: per round
• Total cost:
What About ?
• More complicated…. Can contain regardless of

Solution: use
extremal -free
graph
Elements of DISJ
= edges of
Detecting Triangles
• Trivial upper bound: rounds
• Lower bound?
• 2-party (black box) reduction cannot prove it
• For each triangle, one player knows all 3 edges