Issues in Synchronization

Slides taken from

The Art of Multiprocessor Programming
by Maurice Herlihy & Nir Shavit
 Recall where we are
• Time no longer cures software bloat
– The “free ride” is over
• When you double your program’s path
length
– You can’t just wait 6 months
– Your software must somehow exploit
twice as much concurrency

2
 Traditional Scaling Process
7x
Speedup 3.6x
1.8x

User code

Traditional
Uniprocessors

Time: Moore’s law

3
 Multicore Scaling Process
7x
Speedup 3.6x
1.8x

User code

Multicores

Unfortunately, not so simple…

4
 Real-World Scaling Process

Speedup 2x 2.9x
1.8x

User code

Multicores

Parallelization and Synchronization

require great care…
5
 Sequential Computation

thread

memory

object object
6
 Concurrent Computation

s
ead
th r

memory

object object
7
 Asynchrony

• Sudden unpredictable delays

– Cache misses (short)
– Page faults (long)
– Scheduling quantum used up (really long)

8
 Model Summary
• Multiple threads
– Sometimes called processes
• Single shared memory
• Objects live in memory
• Unpredictable asynchronous delays

9
 Parallel Primality Testing
• Challenge
– Print primes from 1 to 1010
(in some arbitrary order)
• Given
– Ten-processor multiprocessor
– One thread per processor
• Goal
– Get ten-fold speedup (or close)
10
 Load Balancing
1 109 2·109 … 1010

P0 P1 … P9

• Split the work evenly

• Each thread tests range of 109

11
 Procedure for Thread i

void primePrint() {
int i = ThreadID.get(); // IDs in {0..9}
for (j = i*109+1; j < (i+1)*109; j++) {
if (isPrime(j))
print(j);
}
}

12
 Issues
• Higher ranges have fewer primes
• Yet larger numbers harder to test
• Thread workloads d
te
– Uneven
j ec
– Hard to predict re
• Need dynamic load balancing

13
Shared Counter

19

18 each thread
takes a number
17
14
 Procedure for Thread i

Counter counter = new Counter(1);

void primePrint() { Shared counter object

long j = 0;
while (j < 1010) {
j = counter.getAndIncrement();
if (isPrime(j))
print(j);
}
}

15
 Where Things Reside
void primePrint() {
int i = ThreadID.get(); // IDs in {0..9}
for (j = i*109+1, j<(i+1)*109; j++) {
if (isPrime(j))
print(j);

Local
}
}

variables
code

cache cache cache

Bus Bus

shared
1 memory

shared counter

16
 Procedure for Thread i
Counter counter = new Counter(1);

void primePrint() {
long j = 0;
while (j < 1010) { Stop when every
value is taken
j = counter.getAndIncrement();
if (isPrime(j))
print(j);
}
}

17
 Procedure for Thread i
Counter counter = new Counter(1);

void primePrint() {
long j = 0;
while (j < 1010) {
j = counter.getAndIncrement();
if (isPrime(j))
print(j);
}
} Increment & return
each new value
18
 Counter Implementation

public class Counter {

private long value;

public long getAndIncrement() {

,
return value++;
re ad
}
l e t h a ds
s i n g t h re
r
}
f o ren t
OK o n c u r
f or c
n o t
19
 What It Means

public class Counter {

private long value;

public long getAndIncrement() {

return value++; temp = value;
} value = value + 1;
} return temp;

20
 Not so good…
Value… 1 2 3 2

read write read write

1 2 2 3

read write
1 2

time
21
Is this problem inherent?
write
read

write read

If we could only glue reads and writes…

22
 Challenge

public class Counter {

private long value;

public long getAndIncrement() {

temp = value;
value = temp + 1;
return temp;
}
}

23
 Challenge

public class Counter {

private long value;

public long getAndIncrement() {

temp = value;
value = temp + 1;
return temp;
} Make these steps
} atomic (indivisible)
24
 Hardware Solution

public class Counter {

private long value;

public long getAndIncrement() {

temp = value;
value = temp + 1;
return temp;
}
} ReadModifyWrite()
instruction 25
 An Aside: Java™
public class Counter {
private long value;

public long getAndIncrement() {

synchronized {
temp = value;
value = temp + 1;
}
return temp;
}
}
26
 An Aside: Java™
public class Counter {
private long value;

public long getAndIncrement() {

synchronized {
temp = value;
value = temp + 1;
}
return temp;
} Synchronized block
}
27
 An Aside: Java™
public class Counter {
private long value;

public long getAndIncrement() {

synchronized {
temp = value;
value = temp + 1;
}
return temp; Mutual Exclusion
}
}
28
Mutual Exclusion or “Alice &
Bob share a pond”
A B

29
 Alice has a pet
A B

30
 Bob has a pet
A B

31
 The Problem
A B

The pets don’t

get along

32
 Formalizing the Problem
• Two types of formal properties in
asynchronous computation:
• Safety Properties
– Nothing bad happens ever
• Liveness Properties
– Something good happens eventually

33
 Formalizing our Problem
• Mutual Exclusion
– Both pets never in pond simultaneously
– This is a safety property
• No Deadlock
– if only one wants in, it gets in
– if both want in, one gets in.
– This is a liveness property

34
 Simple Protocol
• Idea
– Just look at the pond
• Gotcha
– Trees obscure the view

35
 Interpretation
• Threads can’t “see” what other
threads are doing
• Explicit communication required for
coordination

36
 Cell Phone Protocol
• Idea
– Bob calls Alice (or vice-versa)
• Gotcha
– Bob takes shower
– Alice recharges battery
– Bob out shopping for pet food …

37
 Interpretation
• Message-passing doesn’t work
• Recipient might not be
– Listening
– There at all
• Communication must be
– Persistent (like writing)
– Not transient (like speaking)

38
Can Protocol

cola
cola

39
 Bob conveys a bit
A B
cola

40
 Bob conveys a bit
A B

cola

41
 Can Protocol
• Idea
– Cans on Alice’s windowsill
– Strings lead to Bob’s house
– Bob pulls strings, knocks over cans
• Gotcha
– Cans cannot be reused
– Bob runs out of cans

42
 Interpretation
• Cannot solve mutual exclusion with
interrupts
– Sender sets fixed bit in receiver’s space
– Receiver resets bit when ready
– Requires unbounded number of interrupt
bits

43
 Flag Protocol
A B

44
Alice’s Protocol (sort of)
A B

45
Bob’s Protocol (sort of)

A BB

46
 Alice’s Protocol

• Raise flag
• Wait until Bob’s flag is down
• Unleash pet
• Lower flag when pet returns

47
 Bob’s Protocol

• Raise flag
• Wait until Alice’s flag is down
• Unleash pet

!
er
• Lower flag when pet returns

ng
da
48
 Bob’s Protocol
Bob defers
• Raise flag to Alice
• While Alice’s flag is up
– Lower flag
– Wait for Alice’s flag to go down
– Raise flag
• Unleash pet
• Lower flag when pet returns
49
 The Flag Principle
• Raise the flag
• Look at other’s flag
• Flag Principle:
– If each raises and looks, then
– Last to look must see both flags up

50
 Proof of Mutual Exclusion
• Assume both pets in pond
– Derive a contradiction
– By reasoning backwards
• Consider the last time Alice and Bob
each looked before letting the pets in
• Without loss of generality assume
Alice was the last to look…

51
 Proof
Bob last raised
flag Alice last raised her flag

Alice’s last look

Bob’s last
look

ED
time
Q
Alice must have seen Bob’s Flag. A Contradiction
52
 Proof of No Deadlock
• If only one pet wants in, it gets in.
• Deadlock requires both continually
trying to get in.
• If Bob sees Alice’s flag, he gives her
priority. (a gentleman…)

ED
Q
53
 Remarks
• Protocol is unfair
– Bob’s pet might never get in
• Protocol uses waiting
– If Bob is eaten by his pet, Alice’s pet
might never get in

54
 Moral of Story
• Mutual Exclusion cannot be solved by
– transient communication (cell phones)
– interrupts (cans)
• It can be solved by
– one-bit shared variables
– that can be read or written

55
 The Fable Continues
• Alice and Bob fall in love & marry
• Then they fall out of love & divorce
– She gets the pets
– He has to feed them
• Leading to a new coordination
problem: Producer-Consumer

57
Bob Puts Food in the Pond
A

58
Alice Releases her Pets to Eat

mmm… B
mmm…

59
 Producer/Consumer
• Alice and Bob can’t meet
– Each has restraining order on other
– So he puts food in the pond
– And later, she releases the pets
• Avoid
– Releasing pets when there’s no food
– Putting out food if uneaten food remains

60
 Producer/Consumer
• Need a mechanism so that
– Bob lets Alice know when food has been
put out
– Alice lets Bob know when to put out
more food

61
 Surprise Solution
A B
cola

62
 Bob Puts Food in Pond
A B
cola

63
 Bob Knocks over Can
A B

cola

64
 Alice Releases Pets
A yum… B
yum…

cola

65
Alice Resets Can when Pets are Fed

A B
cola

66
 Pseudocode

while (true) {
while (can.isUp()){};
pet.release();
Bob’s code
pet.recapture();
can.reset(); while (true) {
} while (can.isDown()){};
pond.stockWithFood();
can.knockOver();

Alice’s code
}

67
 Correctness
• Mutual Exclusion safety
– Pets and Bob never together in pond
• No Starvation liveness
– If Bob always willing to feed, and pets
always famished, then pets eat infinitely
often.
• Producer/Consumer safety
– The pets never enter pond unless there
is food, and Bob never provides food if
there is unconsumed food.
68
Could Also Solve Using Flags

A B

69
 Waiting
• Both solutions use waiting
– while(...){}
• Waiting is problematic
– If one participant is delayed
– So is everyone else
– But delays are common & unpredictable

70
 The Fable drags on …
• Bob and Alice still have issues
• So they need to communicate
• So they agree to use billboards …

71
Billboards are Large

B D Letter
A C E
2
Tiles
3
1 3
1 From Scrabble™ box
72
Write One Letter at a Time …

W A S
4 1 1

H
4

B D
A C E
3 2

1 3
1

73
 To post a message

W A S H T H E C A R
4 1 1 4 1 4 1 3 1 1

whew

74
Let’s send another message

L A
S E L L L A V A 1
1

M PS
1 1 1 1 1 1 4 1

3 3 1

75
 Uh-Oh

S E L L
1 1 1 1
T H E
1 4 1
C A R
3 1 1

L
1

OK
76
 Readers/Writers
• Devise a protocol so that
– Writer writes one letter at a time
– Reader reads one letter at a time
– Reader sees
• Old message or new message
• No mixed messages

77
Readers/Writers (continued)
• Easy with mutual exclusion
• But mutual exclusion requires waiting
– One waits for the other
– Everyone executes sequentially
• Remarkably, we can solve Readers/Writers
without mutual exclusion

78
 Why do we care?
• We want as much of the code as
possible to execute concurrently (in
parallel)
• A larger sequential part implies
reduced performance
• Amdahl’s law: this relation is not
linear…

79
 Amdahl’s Law

Speedup= OldExecutionTime
NewExecutionTime

…of computation given n CPUs instead of 1

80
 Amdahl’s Law
Sequential
Parallel
fraction

Speedup=
1 fraction

p
1−p +
Number of n
processors
81
 Example
• Ten processors
• 60% concurrent, 40% sequential
• How close to 10-fold speedup?

1
Speedup=2.17= 0.6
1 − 0.6 +
10

82
 Example
• Ten processors
• 80% concurrent, 20% sequential
• How close to 10-fold speedup?

1
Speedup=3.57= 0.8
1 − 0.8 +
10

83
 Example
• Ten processors
• 90% concurrent, 10% sequential
• How close to 10-fold speedup?

1
Speedup=5.26= 0.9
1 − 0.9 +
10

84
 Example
• Ten processors
• 99% concurrent, 1% sequential
• How close to 10-fold speedup?

1
Speedup=9.17= 0.99
1 − 0.99 +
10

85
 The Moral
• Making good use of our multiple
processors (cores) means
• Finding ways to effectively parallelize
our code
– Minimize sequential parts
– Reduce idle time in which threads wait
without doing useful work.

86
 Multicore Programming
• This is what makes multicore
programming challenging
– The % that is not easy to make parallel
may have a large impact on the overall
speedup
• Next module:
– Programming using pthreads
– A deeper look at mutual exclusion and
synchronization algorithms
87