Professional Documents
Culture Documents
Lec15 PDF
Lec15 PDF
6.046J/18.401J LECTURE 15
Dynamic Programming Longest common subsequence Optimal substructure Overlapping subproblems
Dynamic programming
Design technique, like divide-and-conquer. Example: Longest Common Subsequence (LCS) Given two sequences x[1 . . m] and y[1 . . n], find a longest subsequence common to them both.
November 7, 2005
L15.2
Dynamic programming
Design technique, like divide-and-conquer. Example: Longest Common Subsequence (LCS) Given two sequences x[1 . . m] and y[1 . . n], find a longest subsequence common to them both. a not the
November 7, 2005
L15.3
Dynamic programming
Design technique, like divide-and-conquer. Example: Longest Common Subsequence (LCS) Given two sequences x[1 . . m] and y[1 . . n], find a longest subsequence common to them both. a not the x: A B C B D A B y: B D C A B A
November 7, 2005
L15.4
Dynamic programming
Design technique, like divide-and-conquer. Example: Longest Common Subsequence (LCS) Given two sequences x[1 . . m] and y[1 . . n], find a longest subsequence common to them both. a not the x: A B C B D A B BCBA = LCS(x, y) y: B D C A B A functional notation, but not a function
November 7, 2005 Copyright 2001-5 by Erik D. Demaine and Charles E. Leiserson L15.5
November 7, 2005
L15.6
November 7, 2005
L15.8
November 7, 2005
L15.9
Recursive formulation
Theorem. c[i, j] = c[i1, j1] + 1 if x[i] = y[j], max{c[i1, j], c[i, j1]} otherwise.
November 7, 2005
L15.11
Recursive formulation
Theorem. c[i1, j1] + 1 if x[i] = y[j], c[i, j] = max{c[i1, j], c[i, j1]} otherwise. Proof. Case x[i] = y[ j]: x: y:
1 1 2 2 i m
L
j n
November 7, 2005
L15.12
Recursive formulation
Theorem. c[i1, j1] + 1 if x[i] = y[j], c[i, j] = max{c[i1, j], c[i, j1]} otherwise. Proof. Case x[i] = y[ j]: x: y:
1 1 2 2 i m
L
j n
Let z[1 . . k] = LCS(x[1 . . i], y[1 . . j]), where c[i, j] = k. Then, z[k] = x[i], or else z could be extended. Thus, z[1 . . k1] is CS of x[1 . . i1] and y[1 . . j1].
November 7, 2005 Copyright 2001-5 by Erik D. Demaine and Charles E. Leiserson L15.13
Proof (continued)
Claim: z[1 . . k1] = LCS(x[1 . . i1], y[1 . . j1]). Suppose w is a longer CS of x[1 . . i1] and y[1 . . j1], that is, | w | > k1. Then, cut and paste: w || z[k] (w concatenated with z[k]) is a common subsequence of x[1 . . i] and y[1 . . j] with | w || z[k] | > k. Contradiction, proving the claim.
November 7, 2005
L15.14
Proof (continued)
Claim: z[1 . . k1] = LCS(x[1 . . i1], y[1 . . j1]). Suppose w is a longer CS of x[1 . . i1] and y[1 . . j1], that is, | w | > k1. Then, cut and paste: w || z[k] (w concatenated with z[k]) is a common subsequence of x[1 . . i] and y[1 . . j] with | w || z[k] | > k. Contradiction, proving the claim. Thus, c[i1, j1] = k1, which implies that c[i, j] = c[i1, j1] + 1. Other cases are similar.
November 7, 2005 Copyright 2001-5 by Erik D. Demaine and Charles E. Leiserson L15.15
Dynamic-programming hallmark #1
Optimal substructure An optimal solution to a problem (instance) contains optimal solutions to subproblems.
November 7, 2005
L15.16
Dynamic-programming hallmark #1
Optimal substructure An optimal solution to a problem (instance) contains optimal solutions to subproblems. If z = LCS(x, y), then any prefix of z is an LCS of a prefix of x and a prefix of y.
November 7, 2005 Copyright 2001-5 by Erik D. Demaine and Charles E. Leiserson L15.17
November 7, 2005
L15.18
Recursion tree
m = 3, n = 4:
2,4 2,4 1,4 1,4 1,3 1,3 2,3 2,3 2,2 2,2 1,3 1,3 2,3 2,3 2,2 2,2 3,4 3,4 3,3 3,3 3,2 3,2
November 7, 2005
L15.20
Recursion tree
m = 3, n = 4:
2,4 2,4 1,4 1,4 1,3 1,3 2,3 2,3 2,2 2,2 1,3 1,3 2,3 2,3 2,2 2,2 3,4 3,4 3,3 3,3 3,2 3,2
m+ n
Recursion tree
m = 3, n = 4:
2,4 2,4 1,4 1,4 1,3 1,3 2,3 2,3 2,2 2,2 1,3 1,3 3,4 3,4 3,3 3,3 2,3 2,3 2,2 2,2 3,2 3,2
same subproblem
m+ n
, Height = m + n work potentially exponential. but were solving subproblems already solved!
November 7, 2005 Copyright 2001-5 by Erik D. Demaine and Charles E. Leiserson L15.22
Dynamic-programming hallmark #2
Overlapping subproblems A recursive solution contains a small number of distinct subproblems repeated many times.
November 7, 2005
L15.23
Dynamic-programming hallmark #2
Overlapping subproblems A recursive solution contains a small number of distinct subproblems repeated many times. The number of distinct LCS subproblems for two strings of lengths m and n is only mn.
November 7, 2005
L15.24
Memoization algorithm
Memoization: After computing a solution to a subproblem, store it in a table. Subsequent calls check the table to avoid redoing work.
November 7, 2005
L15.25
Memoization algorithm
Memoization: After computing a solution to a subproblem, store it in a table. Subsequent calls check the table to avoid redoing work.
LCS(x, y, i, j) if c[i, j] = NIL then if x[i] = y[j] then c[i, j] LCS(x, y, i1, j1) + 1 else c[i, j] max{ LCS(x, y, i1, j), LCS(x, y, i, j1)}
same as before
November 7, 2005
L15.26
Memoization algorithm
Memoization: After computing a solution to a subproblem, store it in a table. Subsequent calls check the table to avoid redoing work.
LCS(x, y, i, j) if c[i, j] = NIL then if x[i] = y[j] then c[i, j] LCS(x, y, i1, j1) + 1 else c[i, j] max{ LCS(x, y, i1, j), LCS(x, y, i, j1)}
same as before
Dynamic-programming algorithm
IDEA: Compute the table bottom-up. A B C B D 0 0 0 0 0 0 0 0 0 0 0 0 B 0 0 1 1 1 1 1 1 1 0 0 1 D 0 0 1 1 1 1 1 1 2 2 0 0 C 0 0 1 1 0 0 A 0 1 1 1 0 1 B 0 1 2 2 0 1 A 0 1 2 0 1 2
November 7, 2005
A B 0 0 0 0 1 1 1 1
2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 3 3 3 3 2 3 3 3 4 2 3 3 3 4 2 2 3 3 3 3 4 4 4 4
L15.28
Dynamic-programming algorithm
IDEA: Compute the table bottom-up. Time = (mn). A B C B D 0 0 0 0 0 0 0 0 0 0 0 0 B 0 0 1 1 1 1 1 1 1 0 0 1 D 0 0 1 1 1 1 1 1 2 2 0 0 C 0 0 1 1 0 0 A 0 1 1 1 0 1 B 0 1 2 2 0 1 A 0 1 2 0 1 2
November 7, 2005
A B 0 0 0 0 1 1 1 1
2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 3 3 3 3 2 3 3 3 4 2 3 3 3 4 2 2 3 3 3 3 4 4 4 4
L15.29
Dynamic-programming algorithm
IDEA: Compute the table bottom-up. Time = (mn). Reconstruct LCS by tracing backwards. A B C B D 0 0 0 0 0 0 0 0 0 0 0 0 B 0 0 1 1 1 1 1 1 1 1 0 0 D 0 0 1 1 1 1 1 1 2 2 0 0 C 0 0 1 1 0 0 A 0 1 1 1 0 1 B 0 1 2 2 0 1 A 0 1 2 0 1 2 A B 0 0 0 0 1 1 1 1
2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 3 3 3 3 2 3 3 3 4 2 3 3 3 4 2 2 3 3 3 3 4 4 4 4
L15.30
November 7, 2005
Dynamic-programming algorithm
IDEA: Compute the table bottom-up. Time = (mn). Reconstruct LCS by tracing backwards. Space = (mn). Exercise: O(min{m, n}).
November 7, 2005
A B C B D 0 0 0 0 0 0 0 0 0 0 0 0 B 0 0 1 1 1 1 1 1 1 1 0 0 D 0 0 1 1 1 1 1 1 2 2 0 0 C 0 0 1 1 0 0 A 0 1 1 1 0 1 B 0 1 2 2 0 1 A 0 1 2 0 1 2
A B 0 0 0 0 1 1 1 1
2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 3 3 3 3 2 3 3 3 4 2 3 3 3 4 2 2 3 3 3 3 4 4 4 4
L15.31