Divide-and-Conquer

Divide-and-Conquer Most widespread application of divide-and-conquer.

■ Break up problem into two pieces of equal size.
■ Solve two sub-problems independently by recursion.
■ Combine two results in overall solution in linear time.
"Divide et impera"
"Veni, vidi, vici" Consequence.
■ Brute force / naïve solution: N2.

- Julius Caesar ■ Divide-and-conquer: N log N.

100BC - 44BC
Familiar example.
■ Mergesort.

This course.
■ Counting inversions, closest pair of points, order statistics, fast
matrix multiplication, fast integer multiplication, FFT.

Why Does It Matter? Orders of Magnitude

Seconds Equivalent Meters Per Imperial
Example
Run time Second Units
1.3 N3 10 N2 47 N log2N 48 N 1 1 second
(nanoseconds) 10-10 1.2 in / decade Continental drift
1000 1.3 seconds 10 msec 0.4 msec 0.048 msec 10 10 seconds
10-8 1 ft / year Hair growing
Time to 10,000 22 minutes 1 second 6 msec 0.48 msec 102 1.7 minutes
10-6 3.4 in / day Glacier
solve a 103 17 minutes
100,000 15 days 1.7 minutes 78 msec 4.8 msec 10-4 1.2 ft / hour Gastro-intestinal tract
problem
of size million 41 years 2.8 hours 0.94 seconds 48 msec 104 2.8 hours
10-2 2 ft / minute Ant
10 million 41 millennia 1.7 weeks 11 seconds 0.48 seconds 105 1.1 days
1 2.2 mi / hour Human walk
106 1.6 weeks
second 920 10,000 1 million 21 million 102 220 mi / hour Propeller airplane
Max size 107 3.8 months
problem minute 3,600 77,000 49 million 1.3 billion 104 370 mi / min Space shuttle
solved 108 3.1 years
hour 14,000 600,000 2.4 trillion 76 trillion 106 620 mi / sec Earth in galactic orbit
in one 109 3.1 decades
day 41,000 2.9 million 50 trillion 1,800 trillion 108 62,000 mi / sec 1/3 speed of light
1010 3.1 centuries
N multiplied by 10, ... forever 210 thousand
1,000 100 10+ 10
time multiplied by Powers
age of 220 million
1021 of 2
universe 230 billion
3 4
 A Useful Recurrence Relation Analysis of Recurrence: Recursion Tree
T(N) = worst case running time on input of size N. Assuming N is a power of 2. 0 if N = 1
T(N ) = 
■ Note: O(1) is "standard" abuse of notation. 2T ( N / 2 ) + cN otherwise

 O( 1 ) if N ≤ n 0
 T(N) cN
T( N ) ≤  T (  N / 2  ) + T (  N / 2 ) + O ( N ) otherwise
 1 4243 14243 123
solve left half solve right half combine
T(N/2) T(N/2) 2(cN/2)

Alternate informal form: T(N) ≤ T(N/2) + O(N).

■ Implicitly assumes N is a power of 2. T(N/4) T(N/4) T(N/4) T(N/4) 4(cN/4)
■ Implicitly assume T(N) ∈ O(1) for sufficiently small N. log2N
...
Solution. T(N / 2k) 2k (cN / 2k)
■ Any function satisfying above recurrence is ∈ O(N log2 N).
...
■ Equivalently, O(logb N) for any b > 1.
T(2) T(2) T(2) T(2) T(2) T(2) T(2) T(2) N/2 (2c)

cN log2N
5 6

Analysis of Recurrence Counting Inversions

0 if N = 1 Web site tries to match your preferences with others on Internet.

T( N ) ≤  T (  N / 2 ) + T (  N / 2  ) + cN otherwise ■ You rank N songs.
 1 4243 14243 {
solve left half solve right half combine ■ Web site consults database to find people with similar rankings.

⇒ T ( N ) ≤ cN  log 2 N 
Closeness metric.
■ My rank = { 1, 2, . . . , N }.
Proof by induction on N.
■ Your rank = { a1, a2, . . . , aN }.
■ Base case: N = 1.
■ Number of inversions between two preference lists.
■ Define n1 = n / 2 , n2 = n / 2.
■ Songs i and j inverted if i < j, but ai > aj
■ Induction step: assume true for 1, 2, . . . , N – 1.
T ( N ) ≤ T ( n1 ) + T ( n2 ) + cn Songs
n2 = n / 2
≤ cn1  log 2 n1  + cn2  log 2 n2  + cn
 
A B C D E
≤ 2 2 /2
log n Inversions
≤ cn1  log 2 n2  + cn2  log 2 n2  + cn
Me 1 2 3 4 5 {3, 2}, {4, 2}
= cn  log 2 n2  + cn ⇒ log 2 n2 ≤  log 2 n − 1 You 1 3 4 2 5
≤ cn(  log 2 n  − 1 ) + cn
= cn  log 2 n 
7 Inversion 8
 Counting Inversions Counting Inversions: Divide-and-Conquer
Brute-force solution. Divide-and-conquer.
■ Check all pairs i and j such that i < j.
■ Θ (N2) comparisons.

Note: there can be a quadratic number of inversions.

■ Asymptotically faster algorithm must compute total number
without even looking at each inversion individually.
1 5 4 8 10 2 6 9 12 11 3 7

9 10

Counting Inversions: Divide-and-Conquer Counting Inversions: Divide-and-Conquer

Divide-and-conquer. Divide-and-conquer.
■ Divide: separate list into two pieces. ■ Divide: separate list into two pieces.
■ Conquer: recursively count inversions in each half separately.

1 5 4 8 10 2 6 9 12 11 3 7 O(1) 1 5 4 8 10 2 6 9 12 11 3 7 O(1)

1 5 4 8 10 2 6 9 12 11 3 7 1 5 4 8 10 2 6 9 12 11 3 7 2T(N / 2)
5 blue-blue inversions 8 green-green inversions

11 12
 Counting Inversions: Divide-and-Conquer Counting Inversions: Divide-and-Conquer
Divide-and-conquer. Divide-and-conquer.
■ Divide: separate list into two pieces. ■ Divide: separate list into two pieces.
■ Conquer: recursively count inversions in each half. ■ Conquer: recursively count inversions in each half.
■ Combine: count inversions where ai and aj are in different halves. ■ Combine: count inversions where ai and aj are in different halves.
■ Return sum of three quantities.

1 5 4 8 10 2 6 9 12 11 3 7 O(1) 1 5 4 8 10 2 6 9 12 11 3 7 O(1)

1 5 4 8 10 2 6 9 12 11 3 7 2T(N / 2) 1 5 4 8 10 2 6 9 12 11 3 7 2T(N / 2)
5 blue-blue inversions 8 green-green inversions 5 blue-blue inversions 8 green-green inversions

9 blue-green inversions: O(N2) 9 blue-green inversions: O(N2)

{5-3, 4-3, 8-6, 8-3, 8-7, 10-6, 10-9, 10-3, 10-7} {5-3, 4-3, 8-6, 8-3, 8-7, 10-6, 10-9, 10-3, 10-7}

Total = 5 + 8 + 9 = 22. O(1)

13 14

Counting Inversions: Divide-and-Conquer Counting Inversions: Good Combine

Divide-and-conquer. Combine: count inversions where ai and aj are in different halves.
■ Divide: separate list into two pieces. ■ Key idea: easy if each half is sorted.
■ Conquer: recursively count inversions in each half. ■ Sort each half.
■ Combine: count inversions where ai and aj are in different halves. ■ Count inversions.
■ Return sum of three quantities.

1 5 4 8 10 2 6 9 12 11 3 7 O(1) 1 5 4 8 10 2 6 9 12 11 3 7

1 5 4 8 10 2 6 9 12 11 3 7 2T(N / 2)
1 2 4 5 8 10 3 6 7 9 11 12 O(N log N)
5 blue-blue inversions 8 green-green inversions

9 blue-green inversions: Can we do this step in 9 blue-green inversions: 4 + 2 + 2 + 1 + 0 + 0 O(N)

{5-3, 4-3, 8-6, 8-3, 8-7, 10-6, 10-9, 10-3, 10-7} sub-quadratic time?

Total = 5 + 8 + 9 = 22. O(1) T(N ) = T ( N / 2  ) + T ( N / 2  ) + O (N log N ) ⇒ T(N ) = O (N log2 N )

15 16
 Counting Inversions: Better Combine Closest Pair
Combine: count inversions where ai and aj are in different halves. Given N points in the plane, find a pair that is closest together.
■ Assume each half is pre-sorted. ■ For concreteness, we assume Euclidean distances.
■ Count inversions. ■ Foundation of then-fledgling field of computational geometry.
■ Merge two sorted halves into sorted whole. ■ Graphics, computer vision, geographic information systems,
molecular modeling, air traffic control.

Brute force solution.

3 7 10 14 18 19 2 11 16 17 23 25 ■ Check all pairs of points p and q.
■ Θ (N2) comparisons.
13 blue-green inversions: 6 + 3 + 2 + 2 + 0 + 0 O(N)

One dimensional version (points on a line).

2 3 7 10 11 14 16 17 18 19 23 25 O(N) ■ O(N log N) easy.

Assumption to make presentation cleaner.

■ No two points have same x coordinate.
T (N ) = T ( N / 2  ) + T ( N / 2  ) + O (N ) ⇒ T(N ) = O (N log N )

17 18

Closest Pair Closest Pair

Algorithm. Algorithm.
■ Divide: draw vertical line so that roughly N / 2 points on each side. ■ Divide: draw vertical line so that roughly N / 2 points on each side.
■ Conquer: find closest pair in each side recursively.

21

12

19 20
 Closest Pair
Algorithm.
■ Divide: draw vertical line so that roughly N / 2 points on each side.
■ Conquer: find closest pair in each side recursively.
■ Combine: find closest pair with one point in each side.
Closest Pair
Algorithm.
■ Divide: draw vertical line so that roughly N / 2 points on each side.
■ Conquer: find closest pair in each side recursively.
■ Combine: find closest pair with one point in each side.
■ Return best of 3 solutions.

8 8
21 21

12 12

21 22

Closest Pair Closest Pair

Algorithm. Key step: find closest pair with one point in each side.
■ Divide: draw vertical line so that roughly N / 2 points on each side. ■ Extra information: closest pair entirely in one side had distance δ.
■ Conquer: find closest pair in each side recursively.
■ Combine: find closest pair with one point in each side.
■ Return best of 3 solutions.

8
21

δ = min(12, 21)
12

23 24
 Closest Pair
Key step: find closest pair with one point in each side.
■ Extra information: closest pair entirely in one side had distance δ.
■ Observation: only need to consider points S within δ of line.
Closest Pair
Key step: find closest pair with one point in each side.
■ Extra information: closest pair entirely in one side had distance δ.
■ Observation: only need to consider points S within δ of line.
■ Sort points in strip S by their y coordinate.
– suffices to compute distances for pairs within constant number
of positions of each other in sorted list!

5
21 4 21

δ = min(12, 21) δ = min(12, 21)

12 12 3

2
1

δ 25 δ 26

Closest Pair Closest Pair

S = list of points in the strip sorted by their y coordinate. δ = ClosestPair (p1, p2 , . . . , pN )
Compute separation line x = xmed such that half the points
O(N log N)
Crucial fact: if p and q are in S, and if have x coordinate less than xmed, and half are greater.
d(p, q) < δ , then they are within 11
positions of each other in S. 39
2T(N / 2) δ1 = ClosestPair(left half)
■ No two points lie in same box. 31 δ2 = ClosestPair(right half)
δ = min (δ1 , δ2 )
■ Two points at least 2 rows apart
have distance ≥ 2δ / 2.
δ/2 O(N) Delete all points further than δ from separation line.
2 rows
30
δ/2 O(N log N) Sort remaining points in strip by y coordinate.
29

28 δ/2 O(N) Scan in y order, and compute distance between each point
27
and next 11 neighbors.

26
O(N) If any of these distances is less than δ , update δ .
25

T(N ) = T ( N / 2  ) + T ( N / 2  ) + O (N log N ) ⇒ T(N ) = O (N log2 N )

δ δ 27 28
 Closest Pair Integer Arithmetic
Can we achieve O(N log N)? Given two N-digit integers a and b, compute a + b.
■ Yes. Don’t sort points in strip from scratch each time. ■ O(N) bit operations.
■ Each recursive call should return two lists: all points sorted by y
coordinate, and all points sorted by x coordinate. Multiplication: given two N-digit integers a and b,
Sorting is accomplished by merging two already sorted lists. compute ab.
■
1 1 0 1 0 1 0 1
■ Brute force solution: Θ (N2) bit operations.
* 0 1 1 1 1 1 0 1

T (N ) = T ( N / 2  ) + T ( N / 2  ) + O (N )
1 1 0 1 0 1 0 1 0
⇒ T(N ) = O (N log N ) Application.
0 0 0 0 0 0 0 0 0
■ Cryptography.
1 1 0 1 0 1 0 1 0
1 1 0 1 0 1 0 1 0
1 1 1 1 1 1 0 1 1 1 0 1 0 1 0 1 0
1 1 0 1 0 1 0 1 1 1 0 1 0 1 0 1 0
+ 0 1 1 1 1 1 0 1 1 1 0 1 0 1 0 1 0
1 0 1 0 1 0 0 1 0 0 0 0 0 0 0 0 0 0
0 1 1 0 1 0 0 0 0 0 0 0 0 0 0 1 0
29 30

Divide-and-Conquer Multiplication: First Attempt Karatsuba Multiplication

To multiply two N-digit integers: To multiply two N-digit integers:
■ Multiply four N/2-digit integers. ■ Add two N/2 digit integers.
■ Add two N/2-digit integers, and shift to obtain result. ■ Multiply three N/2-digit integers.
■ Subtract two N/2-digit integers, and shift to obtain result.
3 3
123,456 × 987,654 = (10 w + x ) × (10 y + z )
123,456 × 987,654 = (103 w + x ) × (103 y + z )
= 106 (wy ) + 103 (wz + xy ) + 100 ( xz )
= 106 (wy ) + 103 (wz + xy ) + 100 ( xz )
= 106 (121,401) + 103 (80,442 + 450,072 ) + 100 (298,224 )
= 123 w = 123 = 106 ( p ) + 103 (r − p − q ) + 100 (q )
w = 121,401,29 9,224
x = 456 x = 456 = 106 (121,401) + 103 (950,139 - 121,401 - 298,224 ) + 100 (298,224 )
y = 987 y = 987 121,401,29 9,224
z = 654 z = 654
p = wy
(wz + xy ) = r − p − q
q = xz
ab = (10N / 2 w + x )(10N / 2 y + z )
r = (w + x )( y + z )
N is a power of 2 T(N ) =
142 43
4T (N / 2 ) +
recursive calls
1 42 43
Θ (N )
add, shift
⇒ T(N ) = Θ(N 2 )

31 32
 Karatsuba Multiplication: Analysis Matrix Multiplication
To multiply two N-digit integers: Given two N x N matrices A and B, compute C = AB.
■ Add two N/2 digit integers. N
■ Multiply three N/2-digit integers. cij = ∑ aik bkj
k =1
■ Subtract two N/2-digit integers, and shift to obtain result.
■ Brute force: Θ (N3) time.
Karatsuba-Ofman (1962). p = wy (wz + xy ) = r − p − q
O(N1.585) bit operations. q = xz  26 62 98  0 2 4  1 7 13 
■
     
 80 224 368  =  6 8 10  ×  3 9 15 
r = (w + x )( y + z )  134 386 638   12 14 16   5 11 17 
     

ab = (10N / 2 w + x )(10N / 2 y + z )  c11 c12 L c

c13   a11 a12 L aa13   b11 b12 L b
b13 
L c L a L b
1N 1N 1N
     
 c21 c22 c23   a21 a22 a23   b21 b22 b23 
14444)4+4T4( 44N2/ 24 4) +44T (414+ 4N4/ 243 )
T(N ) ≤ T ( N / 2 +
142 43
Θ(N )
L c L a L b
2N 2N 2N
c c32 c33  = a a32 a33  × b b32 b33 
M
 31
M M O M 
M
 31
M M O M 
M
 31
M M O M 
3N 3N 3N
recursive calls add, subtract, shift
     
⇒ T(N ) = O (N
log 2 3
)
c
 N 1 cN 2 c L c
N3

NN 
a
 N 1 aN 2 a L aN3

NN 
b
 N 1 bN 2 b L b
N3

NN 

Hard to imagine naïve algorithm can be improved upon.

33 34

Matrix Multiplication: Warmup Matrix Multiplication: Idea

Warmup: divide-and-conquer. Idea: multiply 2 x 2 matrices with only 7 scalar multiplications.
■ Divide: partition A and B into N/2 x N/2 blocks.
r s   a b  e g 
■ Conquer: multiply 8 N/2 x N/2 recursively.  =   
■ Combine: add appropriate products using 4 matrix additions. t u  c d   f h

P1 = a × (g − h ) r = P5 + P4 − P2 + P6
P2 = (a + b ) × h s = P1 + P2
 C11 C12   A11 A12   B11 B12  C11 = (A11 × B11 ) + (A12 × B21 ) = (c + d ) × e
  =     P3 t = P3 + P4
 C21 C22   A21 A22   B21 B22  C12 = (A11 × B12 ) + (A12 × B22 ) = d × (f − e )
P4 u = P5 + P1 − P3 − P7
C21 = (A21 × B11 ) + (A22 × B21 ) P5 = (a + d ) × (e + h )
C22 = (A21 × B12 ) + (A22 × B22 )
P6 = ( b − d ) × (f + h )
P7 = (a − c ) × (e + g )

T(N ) =
142 43
8T (N / 2 ) +
1 4 44 24443
Θ (N 2 )
recursive calls add, form submatrices
⇒ T(N ) = Θ(N 3 )
■ 7 multiplications.
■ 18 = 10 + 8 additions and subtractions.

Note: did not rely on commutativity of scalar multiplication.

35 36
 Matrix Multiplication: Strassen Beyond Strassen
Generalize to matrices. Can you multiply two 2 x 2 matrices with only 7 scalar multiplications?
■ Divide: partition A and B into N/2 x N/2 blocks. ! Yes! Strassen (1969). Θ(N log2 7) = O (N 2.81)
■ Compute: 14 N/2 x N/2 matrices via 10 matrix add/subtract.
■ Conquer: multiply 7 N/2 x N/2 recursively. Can you multiply two 2 x 2 matrix with only 6 scalar multiplications?
! Impossible (Hopcroft and Kerr, 1971).
■ Combine: 7 products into 4 terms using 8 matrix add/subtract. Θ(N log2 6) = O (N 2.59 )

 C11 C12   A11 A12   B11 B12  Two 3 x 3 matrices with only 21 scalar multiplications?
  =     ! Also impossible.
 C21 C22   A21 A22   B21 B22  Θ (N log3 21) = O (N 2.77 )

Two 70 x 70 matrices with only 143,640 scalar multiplications?

Analysis.
! Yes! (Pan, 1980).
■ Assume N is a power of 2. Θ (N log70 143640 ) = O (N 2.80 )
■ T(N) = # arithmetic operations. Decimal wars.
! December, 1979: O(N2.521813).
T(N ) =
142 43
7T (N / 2 ) +
1 42
4 43 4
Θ (N 2 )
recursive calls add, subtract
⇒ T(N ) = Θ(N log2 7 ) = O (N 2.81) ! January, 1980: O(N2.521801).

Coppersmith-Winograd (1987): O(N2.376).

37 38

Strassen in Practice? Order Statistics

Practical considerations. Given N linearly ordered elements, find ith smallest element.
■ Stop recursion around N = 100. ■ Minimum if i = 1.
■ Numerical stability. ■ Maximum if i = N.
■ Harder to parallelize. ■ Median:
■ Caching effects. – i = (N+1) / 2 if N is odd
– i = N/2 or i = N/2 + 1
■ Easy to do with O(N) comparisons if i or N – i is a constant.
■ Easy to do in general with O(N log2N) comparisons by sorting.

Can we do in worst-case O(N) comparisons?

■ Yes. (Blum, Floyd, Pratt, Rivest, Tarjan, 1973)
■ Cool and simple idea. Ahead of its time.

Assumption to make presentation cleaner.

■ All items have distinct values.

39 40
 Fast Select Fast Partition
Similar to quicksort, but throw away useless "half" at each iteration. FastPartition().
■ Select ith smallest element from a1, a2, . . . , aN. ■ Divide N elements into N/5 groups of 5 elements each, plus extra.

FastSelect (ith, N, a1, a2, . . . , aN)

x ← FastPartition(N, a1, a2, ..., aN) x = partition element

k ← rank(x) is kth smallest 29 10 38 37 02 55 18 24 34 35 36
if (i == k)
return x 22 44 52 11 53 12 13 43 20 04 27

else if (i < k) 28 23 06 26 40 19 01 46 31 49 08
b[] ← all items of a[] less than x
return FastSelect(ith, k-1, b1, b2, ..., bk-1)
14 09 05 03 54 30 48 47 32 51 21
else if (i > k)
c[] ← all items of a[] greater than x 45 39 50 15 25 16 41 17 33 07
return FastSelect((i-k)th, N-k, c1, c2, ..., cN-k)

N = 54
41 42

Fast Partition Fast Partition

FastPartition(). FastPartition().
■ Divide N elements into N/5 groups of 5 elements each, plus extra. ■ Divide N elements into N/5 groups of 5 elements each, plus extra.
■ Brute force sort each of the 5-element groups. ■ Brute force sort each of the 5-element groups.
■ Find x = "median of medians" using FastSelect() recursively.

29
14 10
9 38
05 37
03 02 55
12 18
01 24
17 34
20 35
04 36 14 9 05 03 02 12 01 17 20 04 36

22 44
10 52
06 11 53
25 12
16 13 43
24 20
31 04
07 27 22 10 06 11 25 16 13 24 31 07 27

28 23 06
38 26
15 40 19 01
18 46
43 31
32 49
35 08 28 23 38 15 40 19 18 43 32 35 08

14
29 09
39 05
50 03
26 54
53 30 48
41 47
46 32
33 51
49 21 29 39 50 26 53 30 41 46 33 49 21

45 39
44 50
52 15
37 25
54 16
53 41
48 17
47 33
34 07
51 45 44 52 37 54 53 48 47 34 51

43 44
 Fast Selection and Fast Partition Fast Selection Analysis
FastPartition(). Crux of proof: at least 25% of elements thrown away at each step.
■ Divide N elements into N/5 groups of 5 elements each, plus extra. ■ At least 1/2 of 5 element medians ≤ x
■ Brute force sort each of the 5-element groups. – at least  N / 5 / 2 = N / 10 medians ≤ x

■ Find x = "median of medians" using FastSelect() recursively.

FastSelect().
■ Call FastPartition(). Let x be partition element used, and let k
be its rank. 14 9 05 03 02 12 01 17 20 04 36
■ Call FastSelect() recursively to find ith smallest element. median of
medians 22 10 06 11 25 16 13 24 31 07 27
– return x if i = k
– return ith smallest on left side if i < k
28 23 38 15 40 19 18 43 32 35 08
– return (i-k)th smallest on right side if i > k
29 39 50 26 53 30 41 46 33 49 21

45 44 52 37 54 53 48 47 34 51

45 46

Fast Selection Analysis Fast Selection Analysis

Crux of proof: at least 25% of elements thrown away at each step. Crux of proof: at least 25% of elements thrown away at each step.
■ At least 1/2 of 5 element medians ≤ x ■ At least 1/2 of 5 element medians ≤ x
– at least  N / 5 / 2 = N / 10 medians ≤ x – at least  N / 5 / 2 = N / 10 medians ≤ x
■ At least 3 N / 10 elements ≤ x. ■ At least 3 N / 10 elements ≤ x.
■ At least 3 N / 10 elements ≥ x.

14 9 05 03 02 12 01 17 20 04 36 14 9 05 03 02 12 01 17 20 04 36
median of median of
medians 22 10 06 11 25 16 13 24 31 07 27 medians 22 10 06 11 25 16 13 24 31 07 27

28 23 38 15 40 19 18 43 32 35 08 28 23 38 15 40 19 18 43 32 35 08

29 39 50 26 53 30 41 46 33 49 21 29 39 50 26 53 30 41 46 33 49 21

45 44 52 37 54 53 48 47 34 51 45 44 52 37 54 53 48 47 34 51

47 48
 Fast Selection Analysis Fast Selection Analysis
Crux of proof: at least 25% of elements thrown away at each step. Analysis of recurrence.
■ At least 1/2 of 5 element medians ≤ x
c if N < 50
– at least  N / 5 / 2 = N / 10 medians ≤ x T(N ) ≤ 
 T ( N / 5  ) + T (N − 3 N / 10 ) + cN otherwise
■ At least 3 N / 10 elements ≤ x.
■ At least 3 N / 10 elements ≥ x.
⇒ FastSelect() called recursively with at most N - 3 N / 10
elements in last step Claim: T(N) ≤ 20cN.
■ Base case: N < 50.
T(N ) ≤
1T (42
N 43
median of medians
144424443 142
/ 5  ) + T (N − 3 N / 10  ) +
recursive select
43
O (N )
insertion sort
■ Inductive step: assume true for 1, 2, . . . , N-1.

⇒ T (N ) = O (N ). T (N ) ≤ T ( N / 5  ) + T (N − 3 N / 10  ) + cN
≤ 20c N / 5  + 20c ( N − 3 N / 10  ) + cN For n ≥ 50,
≤ 20c (N / 5 ) + 20c (N ) − 20c (N / 4 ) + cN 3 N / 10 ≥ N / 4.
= 20cN

49 50

Linear Time Median Finding Postmortem

Practical considerations.
■ Constant (currently) too large to be useful.
■ Practical variant: choose random partition element.
– O(N) expected running time ala quicksort.
■ Open problem: guaranteed O(N) with better constant.

Quicksort.
■ Worst case O(N log N) if always partition on median.
■ Justifies practical variants: median-of-3, median-of-5.

51