Lec3 PDF

Introduction to Algorithms
6.046J/18.401J LECTURE 3
Divide and Conquer Binary search Powering a number Fibonacci numbers Matrix multiplication Strassens algorithm VLSI tree layout Prof. Erik D. Demaine
September 14, 2005 Copyright 2001-5 Erik D. Demaine and Charles E. Leiserson L2.1
The divide-and-conquer design paradigm

1. Divide the problem (instance) into subproblems. 2. Conquer the subproblems by solving them recursively. 3. Combine subproblem solutions.
September 14, 2005
Copyright 2001-5 Erik D. Demaine and Charles E. Leiserson
L2.2
Merge sort
1. Divide: Trivial. 2. Conquer: Recursively sort 2 subarrays. 3. Combine: Linear-time merge.
September 14, 2005
L2.3
Merge sort
1. Divide: Trivial. 2. Conquer: Recursively sort 2 subarrays. 3. Combine: Linear-time merge. T(n) = 2 T(n/2) + (n) # subproblems subproblem size
September 14, 2005
work dividing and combining

L2.4
Master theorem (reprise)

T(n) = a T(n/b) + f (n) CASE 1: f (n) = O(nlogba ), constant > 0 T(n) = (nlogba) . CASE 2: f (n) = (nlogba lgkn), constant k 0 T(n) = (nlogba lgk+1n) . CASE 3: f (n) = (nlogba + ), constant > 0, and regularity condition T(n) = ( f (n)) .
September 14, 2005
L2.5
Master theorem (reprise)

T(n) = a T(n/b) + f (n) CASE 1: f (n) = O(nlogba ), constant > 0 T(n) = (nlogba) . CASE 2: f (n) = (nlogba lgkn), constant k 0 T(n) = (nlogba lgk+1n) . CASE 3: f (n) = (nlogba + ), constant > 0, and regularity condition T(n) = ( f (n)) . Merge sort: a = 2, b = 2 nlogba = nlog22 = n CASE 2 (k = 0) T(n) = (n lg n) .
Binary search
Find an element in a sorted array: 1. Divide: Check middle element. 2. Conquer: Recursively search 1 subarray. 3. Combine: Trivial.
September 14, 2005
L2.7
Binary search
Find an element in a sorted array: 1. Divide: Check middle element. 2. Conquer: Recursively search 1 subarray. 3. Combine: Trivial. Example: Find 9 3
September 14, 2005
12
15
L2.8
Binary search
September 14, 2005
12
15
L2.9
Binary search
September 14, 2005
12
15
L2.10
Binary search
September 14, 2005
12
15
L2.11
Binary search
September 14, 2005
12
15
L2.12
Binary search
September 14, 2005
12
15
L2.13
Recurrence for binary search

T(n) = 1 T(n/2) + (1) # subproblems subproblem size work dividing and combining
September 14, 2005
L2.14
Recurrence for binary search

T(n) = 1 T(n/2) + (1) # subproblems subproblem size work dividing and combining
nlogba = nlog21 = n0 = 1 CASE 2 (k = 0) T(n) = (lg n) .

Powering a number
Problem: Compute a n, where n N. Naive algorithm: (n).
September 14, 2005
L2.16
Powering a number
Problem: Compute a n, where n N. Naive algorithm: (n). Divide-and-conquer algorithm: an = a n/2 a n/2 a (n1)/2 a (n1)/2 a if n is even; if n is odd.
September 14, 2005
L2.17
Powering a number
Problem: Compute a n, where n N. Naive algorithm: (n). Divide-and-conquer algorithm: an = a n/2 a n/2 a (n1)/2 a (n1)/2 a if n is even; if n is odd.
T(n) = T(n/2) + (1) T(n) = (lg n) .

Fibonacci numbers
Recursive definition: 0 if n = 0; if n = 1; Fn = 1 Fn1 + Fn2 if n 2. 0 1 1 2 3 5 8 13 21 34 L
September 14, 2005
L2.19
Fibonacci numbers
Recursive definition: 0 if n = 0; if n = 1; Fn = 1 Fn1 + Fn2 if n 2. 0 1 1 2 3 5 8 13 21 34 L Naive recursive algorithm: ( n) (exponential time), where = (1 + 5) / 2 is the golden ratio.
Computing Fibonacci numbers

Bottom-up: Compute F0, F1, F2, , Fn in order, forming each number by summing the two previous. Running time: (n).
September 14, 2005
L2.21
Computing Fibonacci numbers

Bottom-up: Compute F0, F1, F2, , Fn in order, forming each number by summing the two previous. Running time: (n). Naive recursive squaring: Fn = n/ 5 rounded to the nearest integer. Recursive squaring: (lg n) time. This method is unreliable, since floating-point arithmetic is prone to round-off errors.
Recursive squaring
Fn +1 Theorem: Fn Fn 1 1 . = Fn 1 1 0 n
September 14, 2005
L2.23
Recursive squaring
Algorithm: Recursive squaring. Time = (lg n) .
September 14, 2005
L2.24
Recursive squaring
Algorithm: Recursive squaring. Time = (lg n) .
Proof of theorem. (Induction on n.) 1 F F 1 1 2 1 Base (n = 1): . = 1 0 F F 1 0

Recursive squaring
Inductive step (n 2):
Fn +1 F n
. Fn Fn Fn 1 1 1 = Fn 1 Fn 1 Fn 2 1 0 n1 1 1 1 1 = . 1 0 1 0 n 1 1 = 1 0
L2.26
September 14, 2005
Matrix multiplication
Input: A = [aij], B = [bij]. Output: C = [cij] = A B.
c11 c12 c c 21 22 M M c c n1 n 2 L c1n a11 a12 L c2 n a21 a22 = M O M M a a L cnn n1 n 2 n
i, j = 1, 2, , n.
L b1n L b2 n O M L bnn
L a1n b11 b12 L a2 n b21 b22 O M M M b b L ann n1 n 2
cij = aik bkj

k =1
Standard algorithm
for i 1 to n do for j 1 to n do cij 0 for k 1 to n do cij cij + aik bkj
September 14, 2005
L2.28
Standard algorithm
for i 1 to n do for j 1 to n do cij 0 for k 1 to n do cij cij + aik bkj
Running time = (n3)
September 14, 2005
L2.29
Divide-and-conquer algorithm
IDEA: nn matrix = 22 matrix of (n/2)(n/2) submatrices: r s a b e f t u = c d g h
C r s t u = ae + bg = af + bh = ce + dg = cf + dh
September 14, 2005
8 mults of (n/2)(n/2) submatrices 4 adds of (n/2)(n/2) submatrices

Copyright 2001-5 Erik D. Demaine and Charles E. Leiserson L2.30
Divide-and-conquer algorithm
IDEA: nn matrix = 22 matrix of (n/2)(n/2) submatrices: r s a b e f t u = c d g h
r s t u
= ae + bg = af + bh = ce + dh = cf + dg
September 14, 2005
C = A B recursive 8 mults of (n/2)(n/2) submatrices ^ 4 adds of (n/2)(n/2) submatrices

Copyright 2001-5 Erik D. Demaine and Charles E. Leiserson L2.31
Analysis of D&C algorithm

T(n) = 8 T(n/2) + (n2) # submatrices submatrix size work adding submatrices
September 14, 2005
L2.32

nlogba = nlog28 = n3 CASE 1 T(n) = (n3).
September 14, 2005
L2.33

nlogba = nlog28 = n3 CASE 1 T(n) = (n3).

No better than the ordinary algorithm.
Strassens idea
Multiply 22 matrices with only 7 recursive mults.
September 14, 2005
L2.35
Strassens idea
Multiply 22 matrices with only 7 recursive mults. P1 = a ( f h ) P2 = (a + b) h P3 = (c + d) e P4 = d ( g e ) P5 = (a + d) (e + h) P6 = (b d) (g + h) P7 = (a c) (e + f )
Strassens idea
September 14, 2005
r s t u
= P5 + P4 P2 + P6 = P1 + P2 = P3 + P4 = P5 + P1 P3 P7
L2.37
Strassens idea
September 14, 2005
r s t u
= P5 + P4 P2 + P6 = P1 + P2 = P3 + P4 = P5 + P1 P3 P7
7 7 mults, mults, 18 18 adds/subs. adds/subs. Note: Note: No No reliance reliance on on commutativity commutativity of of mult! mult!
L2.38
Strassens idea
September 14, 2005
r = P5 + P4 P2 + P6 = (a + d) (e + h) + d (g e) (a + b) h + (b d) (g + h) = ae + ah + de + dh + dg de ah bh + bg + bh dg dh = ae + bg
L2.39
Strassens algorithm
1. Divide: Partition A and B into (n/2)(n/2) submatrices. Form terms to be multiplied using + and . 2. Conquer: Perform 7 multiplications of (n/2)(n/2) submatrices recursively. 3. Combine: Form C using + and on (n/2)(n/2) submatrices.
September 14, 2005
L2.40
Strassens algorithm
1. Divide: Partition A and B into (n/2)(n/2) submatrices. Form terms to be multiplied using + and . 2. Conquer: Perform 7 multiplications of (n/2)(n/2) submatrices recursively. 3. Combine: Form C using + and on (n/2)(n/2) submatrices.
T(n) = 7 T(n/2) + (n2)

Analysis of Strassen
T(n) = 7 T(n/2) + (n2)
September 14, 2005
L2.42
T(n) = 7 T(n/2) + (n2) nlogba = nlog27 n2.81 CASE 1 T(n) = (nlg 7).
September 14, 2005
L2.43
T(n) = 7 T(n/2) + (n2) nlogba = nlog27 n2.81 CASE 1 T(n) = (nlg 7). The number 2.81 may not seem much smaller than 3, but because the difference is in the exponent, the impact on running time is significant. In fact, Strassens algorithm beats the ordinary algorithm on todays machines for n 32 or so.
September 14, 2005
L2.44
T(n) = 7 T(n/2) + (n2) nlogba = nlog27 n2.81 CASE 1 T(n) = (nlg 7). The number 2.81 may not seem much smaller than 3, but because the difference is in the exponent, the impact on running time is significant. In fact, Strassens algorithm beats the ordinary algorithm on todays machines for n 32 or so.
Best to date (of theoretical interest only): (n2.376L).
VLSI layout
Problem: Embed a complete binary tree with n leaves in a grid using minimal area.
September 14, 2005
L2.46
VLSI layout
W(n) H(n)
September 14, 2005
L2.47
VLSI layout
W(n) H(n) H(n) = H(n/2) + (1) = (lg n)

VLSI layout
W(n) H(n) H(n) = H(n/2) + (1) = (lg n)

September 14, 2005
W(n) = 2 W(n/2) + (1) = (n)

L2.49
VLSI layout
W(n) H(n) H(n) = H(n/2) + (1) W(n) = 2 W(n/2) + (1) = (lg n) = (n) Area = (n lg n)
H-tree embedding
L(n)
L(n)
September 14, 2005
L2.51
H-tree embedding
L(n)
L(n)
L(n/4) (1) L(n/4)

H-tree embedding
L(n) L(n) = 2 L(n/4) + (1) = ( n ) Area = (n) L(n/4) (1) L(n/4)
L(n)
Conclusion
Divide and conquer is just one of several powerful techniques for algorithm design. Divide-and-conquer algorithms can be analyzed using recurrences and the master method (so practice this math). The divide-and-conquer strategy often leads to efficient algorithms.
September 14, 2005
L2.54

Lec3 PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Lec3 PDF

Uploaded by

Copyright:

Available Formats

Introduction to Algorithms

The divide-and-conquer design paradigm

September 14, 2005

Copyright 2001-5 Erik D. Demaine and Charles E. Leiserson

September 14, 2005

Copyright 2001-5 Erik D. Demaine and Charles E. Leiserson

work dividing and combining

Copyright 2001-5 Erik D. Demaine and Charles E. Leiserson

Master theorem (reprise)

September 14, 2005

Copyright 2001-5 Erik D. Demaine and Charles E. Leiserson

Master theorem (reprise)

September 14, 2005

Copyright 2001-5 Erik D. Demaine and Charles E. Leiserson

Copyright 2001-5 Erik D. Demaine and Charles E. Leiserson

Copyright 2001-5 Erik D. Demaine and Charles E. Leiserson

Copyright 2001-5 Erik D. Demaine and Charles E. Leiserson

Copyright 2001-5 Erik D. Demaine and Charles E. Leiserson

Copyright 2001-5 Erik D. Demaine and Charles E. Leiserson

Copyright 2001-5 Erik D. Demaine and Charles E. Leiserson

Recurrence for binary search

September 14, 2005

Copyright 2001-5 Erik D. Demaine and Charles E. Leiserson

Recurrence for binary search

nlogba = nlog21 = n0 = 1 CASE 2 (k = 0) T(n) = (lg n) .

September 14, 2005

Copyright 2001-5 Erik D. Demaine and Charles E. Leiserson

September 14, 2005

Copyright 2001-5 Erik D. Demaine and Charles E. Leiserson

T(n) = T(n/2) + (1) T(n) = (lg n) .

September 14, 2005

Copyright 2001-5 Erik D. Demaine and Charles E. Leiserson

Computing Fibonacci numbers

September 14, 2005

Copyright 2001-5 Erik D. Demaine and Charles E. Leiserson

Computing Fibonacci numbers

September 14, 2005

Copyright 2001-5 Erik D. Demaine and Charles E. Leiserson

Algorithm: Recursive squaring. Time = (lg n) .

September 14, 2005

Copyright 2001-5 Erik D. Demaine and Charles E. Leiserson

Algorithm: Recursive squaring. Time = (lg n) .

Proof of theorem. (Induction on n.) 1 F F 1 1 2 1 Base (n = 1): . = 1 0 F F 1 0

September 14, 2005

Copyright 2001-5 Erik D. Demaine and Charles E. Leiserson

L a1n b11 b12 L a2 n b21 b22 O M M M b b L ann n1 n 2

cij = aik bkj

September 14, 2005

Copyright 2001-5 Erik D. Demaine and Charles E. Leiserson

Running time = (n3)

September 14, 2005

Copyright 2001-5 Erik D. Demaine and Charles E. Leiserson

8 mults of (n/2)(n/2) submatrices 4 adds of (n/2)(n/2) submatrices

C = A B recursive 8 mults of (n/2)(n/2) submatrices ^ 4 adds of (n/2)(n/2) submatrices

Analysis of D&C algorithm

September 14, 2005

Copyright 2001-5 Erik D. Demaine and Charles E. Leiserson

Analysis of D&C algorithm

nlogba = nlog28 = n3 CASE 1 T(n) = (n3).

September 14, 2005

Copyright 2001-5 Erik D. Demaine and Charles E. Leiserson

Analysis of D&C algorithm

nlogba = nlog28 = n3 CASE 1 T(n) = (n3).