0 Up votes0 Down votes

20 views31 pagesOct 13, 2013

lec15

© Attribution Non-Commercial (BY-NC)

PDF, TXT or read online from Scribd

Attribution Non-Commercial (BY-NC)

20 views

lec15

Attribution Non-Commercial (BY-NC)

- Incident INC0381768
- Mathematics
- A2 1.3 Packet
- Mrunal.org-Strategy Mathematics Strategy Topicwise Booklist Sample Answers for UPSC Mains Exam
- Minning Assignment
- MS Curriculum-6 Dept.
- AITMO_2013 Individual.pdf
- dynamic programming
- Data Structures & Algorithms
- b value
- acpc2013
- TADM2E 2.3 - Algorithm Wiki FUNCTIONPRESTIFEROUS.pdf
- CutOff Report for RP 12042016 Web (1)
- Annex 4. List of Research Fields
- ECE 2ND & 3RD
- Consolidated[1]
- Fusion of Masses Defined on Infinite Countable Frames of Discernment
- GATE 2013 Answer Key.pdf
- cutoff
- IEEE-Chilecon2017-Program.pdf

You are on page 1of 31

6.046J/18.401J LECTURE 15

Dynamic Programming Longest common subsequence Optimal substructure Overlapping subproblems

November 7, 2005 Copyright 2001-5 by Erik D. Demaine and Charles E. Leiserson L15.1

Dynamic programming

Design technique, like divide-and-conquer. Example: Longest Common Subsequence (LCS) Given two sequences x[1 . . m] and y[1 . . n], find a longest subsequence common to them both.

November 7, 2005

L15.2

Dynamic programming

Design technique, like divide-and-conquer. Example: Longest Common Subsequence (LCS) Given two sequences x[1 . . m] and y[1 . . n], find a longest subsequence common to them both. a not the

November 7, 2005

L15.3

Dynamic programming

Design technique, like divide-and-conquer. Example: Longest Common Subsequence (LCS) Given two sequences x[1 . . m] and y[1 . . n], find a longest subsequence common to them both. a not the x: A B C B D A B y: B D C A B A

November 7, 2005

L15.4

Dynamic programming

Design technique, like divide-and-conquer. Example: Longest Common Subsequence (LCS) Given two sequences x[1 . . m] and y[1 . . n], find a longest subsequence common to them both. a not the x: A B C B D A B BCBA = LCS(x, y) y: B D C A B A functional notation, but not a function

November 7, 2005 Copyright 2001-5 by Erik D. Demaine and Charles E. Leiserson L15.5

Check every subsequence of x[1 . . m] to see if it is also a subsequence of y[1 . . n].

November 7, 2005

L15.6

Check every subsequence of x[1 . . m] to see if it is also a subsequence of y[1 . . n]. Analysis Checking = O(n) time per subsequence. 2m subsequences of x (each bit-vector of length m determines a distinct subsequence of x). Worst-case running time = O(n2m) = exponential time.

November 7, 2005 Copyright 2001-5 by Erik D. Demaine and Charles E. Leiserson L15.7

Simplification: 1. Look at the length of a longest-common subsequence. 2. Extend the algorithm to find the LCS itself.

November 7, 2005

L15.8

Simplification: 1. Look at the length of a longest-common subsequence. 2. Extend the algorithm to find the LCS itself. Notation: Denote the length of a sequence s by | s |.

November 7, 2005

L15.9

Simplification: 1. Look at the length of a longest-common subsequence. 2. Extend the algorithm to find the LCS itself. Notation: Denote the length of a sequence s by | s |. Strategy: Consider prefixes of x and y. Define c[i, j] = | LCS(x[1 . . i], y[1 . . j]) |. Then, c[m, n] = | LCS(x, y) |.

November 7, 2005 Copyright 2001-5 by Erik D. Demaine and Charles E. Leiserson L15.10

Recursive formulation

Theorem. c[i, j] = c[i1, j1] + 1 if x[i] = y[j], max{c[i1, j], c[i, j1]} otherwise.

November 7, 2005

L15.11

Recursive formulation

Theorem. c[i1, j1] + 1 if x[i] = y[j], c[i, j] = max{c[i1, j], c[i, j1]} otherwise. Proof. Case x[i] = y[ j]: x: y:

1 1 2 2 i m

L

j n

November 7, 2005

L15.12

Recursive formulation

Theorem. c[i1, j1] + 1 if x[i] = y[j], c[i, j] = max{c[i1, j], c[i, j1]} otherwise. Proof. Case x[i] = y[ j]: x: y:

1 1 2 2 i m

L

j n

Let z[1 . . k] = LCS(x[1 . . i], y[1 . . j]), where c[i, j] = k. Then, z[k] = x[i], or else z could be extended. Thus, z[1 . . k1] is CS of x[1 . . i1] and y[1 . . j1].

November 7, 2005 Copyright 2001-5 by Erik D. Demaine and Charles E. Leiserson L15.13

Proof (continued)

Claim: z[1 . . k1] = LCS(x[1 . . i1], y[1 . . j1]). Suppose w is a longer CS of x[1 . . i1] and y[1 . . j1], that is, | w | > k1. Then, cut and paste: w || z[k] (w concatenated with z[k]) is a common subsequence of x[1 . . i] and y[1 . . j] with | w || z[k] | > k. Contradiction, proving the claim.

November 7, 2005

L15.14

Proof (continued)

Claim: z[1 . . k1] = LCS(x[1 . . i1], y[1 . . j1]). Suppose w is a longer CS of x[1 . . i1] and y[1 . . j1], that is, | w | > k1. Then, cut and paste: w || z[k] (w concatenated with z[k]) is a common subsequence of x[1 . . i] and y[1 . . j] with | w || z[k] | > k. Contradiction, proving the claim. Thus, c[i1, j1] = k1, which implies that c[i, j] = c[i1, j1] + 1. Other cases are similar.

November 7, 2005 Copyright 2001-5 by Erik D. Demaine and Charles E. Leiserson L15.15

Dynamic-programming hallmark #1

Optimal substructure An optimal solution to a problem (instance) contains optimal solutions to subproblems.

November 7, 2005

L15.16

Dynamic-programming hallmark #1

Optimal substructure An optimal solution to a problem (instance) contains optimal solutions to subproblems. If z = LCS(x, y), then any prefix of z is an LCS of a prefix of x and a prefix of y.

November 7, 2005 Copyright 2001-5 by Erik D. Demaine and Charles E. Leiserson L15.17

LCS(x, y, i, j) if x[i] = y[ j] then c[i, j] LCS(x, y, i1, j1) + 1 else c[i, j] max{ LCS(x, y, i1, j), LCS(x, y, i, j1)}

November 7, 2005

L15.18

LCS(x, y, i, j) if x[i] = y[ j] then c[i, j] LCS(x, y, i1, j1) + 1 else c[i, j] max{ LCS(x, y, i1, j), LCS(x, y, i, j1)} Worst-case: x[i] y[ j], in which case the algorithm evaluates two subproblems, each with only one parameter decremented.

November 7, 2005 Copyright 2001-5 by Erik D. Demaine and Charles E. Leiserson L15.19

Recursion tree

m = 3, n = 4:

2,4 2,4 1,4 1,4 1,3 1,3 2,3 2,3 2,2 2,2 1,3 1,3 2,3 2,3 2,2 2,2 3,4 3,4 3,3 3,3 3,2 3,2

November 7, 2005

L15.20

Recursion tree

m = 3, n = 4:

2,4 2,4 1,4 1,4 1,3 1,3 2,3 2,3 2,2 2,2 1,3 1,3 2,3 2,3 2,2 2,2 3,4 3,4 3,3 3,3 3,2 3,2

m+ n

November 7, 2005 Copyright 2001-5 by Erik D. Demaine and Charles E. Leiserson L15.21

Recursion tree

m = 3, n = 4:

2,4 2,4 1,4 1,4 1,3 1,3 2,3 2,3 2,2 2,2 1,3 1,3 3,4 3,4 3,3 3,3 2,3 2,3 2,2 2,2 3,2 3,2

same subproblem

m+ n

, Height = m + n work potentially exponential. but were solving subproblems already solved!

November 7, 2005 Copyright 2001-5 by Erik D. Demaine and Charles E. Leiserson L15.22

Dynamic-programming hallmark #2

Overlapping subproblems A recursive solution contains a small number of distinct subproblems repeated many times.

November 7, 2005

L15.23

Dynamic-programming hallmark #2

Overlapping subproblems A recursive solution contains a small number of distinct subproblems repeated many times. The number of distinct LCS subproblems for two strings of lengths m and n is only mn.

November 7, 2005

L15.24

Memoization algorithm

Memoization: After computing a solution to a subproblem, store it in a table. Subsequent calls check the table to avoid redoing work.

November 7, 2005

L15.25

Memoization algorithm

Memoization: After computing a solution to a subproblem, store it in a table. Subsequent calls check the table to avoid redoing work.

LCS(x, y, i, j) if c[i, j] = NIL then if x[i] = y[j] then c[i, j] LCS(x, y, i1, j1) + 1 else c[i, j] max{ LCS(x, y, i1, j), LCS(x, y, i, j1)}

same as before

November 7, 2005

L15.26

Memoization algorithm

Memoization: After computing a solution to a subproblem, store it in a table. Subsequent calls check the table to avoid redoing work.

LCS(x, y, i, j) if c[i, j] = NIL then if x[i] = y[j] then c[i, j] LCS(x, y, i1, j1) + 1 else c[i, j] max{ LCS(x, y, i1, j), LCS(x, y, i, j1)}

same as before

November 7, 2005 Copyright 2001-5 by Erik D. Demaine and Charles E. Leiserson L15.27

Dynamic-programming algorithm

IDEA: Compute the table bottom-up. A B C B D 0 0 0 0 0 0 0 0 0 0 0 0 B 0 0 1 1 1 1 1 1 1 0 0 1 D 0 0 1 1 1 1 1 1 2 2 0 0 C 0 0 1 1 0 0 A 0 1 1 1 0 1 B 0 1 2 2 0 1 A 0 1 2 0 1 2

November 7, 2005

A B 0 0 0 0 1 1 1 1

2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 3 3 3 3 2 3 3 3 4 2 3 3 3 4 2 2 3 3 3 3 4 4 4 4

L15.28

Dynamic-programming algorithm

IDEA: Compute the table bottom-up. Time = (mn). A B C B D 0 0 0 0 0 0 0 0 0 0 0 0 B 0 0 1 1 1 1 1 1 1 0 0 1 D 0 0 1 1 1 1 1 1 2 2 0 0 C 0 0 1 1 0 0 A 0 1 1 1 0 1 B 0 1 2 2 0 1 A 0 1 2 0 1 2

November 7, 2005

A B 0 0 0 0 1 1 1 1

2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 3 3 3 3 2 3 3 3 4 2 3 3 3 4 2 2 3 3 3 3 4 4 4 4

L15.29

Dynamic-programming algorithm

IDEA: Compute the table bottom-up. Time = (mn). Reconstruct LCS by tracing backwards. A B C B D 0 0 0 0 0 0 0 0 0 0 0 0 B 0 0 1 1 1 1 1 1 1 1 0 0 D 0 0 1 1 1 1 1 1 2 2 0 0 C 0 0 1 1 0 0 A 0 1 1 1 0 1 B 0 1 2 2 0 1 A 0 1 2 0 1 2 A B 0 0 0 0 1 1 1 1

2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 3 3 3 3 2 3 3 3 4 2 3 3 3 4 2 2 3 3 3 3 4 4 4 4

L15.30

November 7, 2005

Dynamic-programming algorithm

IDEA: Compute the table bottom-up. Time = (mn). Reconstruct LCS by tracing backwards. Space = (mn). Exercise: O(min{m, n}).

November 7, 2005

A B C B D 0 0 0 0 0 0 0 0 0 0 0 0 B 0 0 1 1 1 1 1 1 1 1 0 0 D 0 0 1 1 1 1 1 1 2 2 0 0 C 0 0 1 1 0 0 A 0 1 1 1 0 1 B 0 1 2 2 0 1 A 0 1 2 0 1 2

A B 0 0 0 0 1 1 1 1

2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 3 3 3 3 2 3 3 3 4 2 3 3 3 4 2 2 3 3 3 3 4 4 4 4

L15.31

- Incident INC0381768Uploaded byabhishek agrawal
- MathematicsUploaded byashi1564
- A2 1.3 PacketUploaded byThomas Carder
- Mrunal.org-Strategy Mathematics Strategy Topicwise Booklist Sample Answers for UPSC Mains ExamUploaded byIlias
- Minning AssignmentUploaded byRaja Bilal
- MS Curriculum-6 Dept.Uploaded byNahéén de Große
- AITMO_2013 Individual.pdfUploaded byChandru Mirpuri
- dynamic programmingUploaded byShivam Yadav
- Data Structures & AlgorithmsUploaded bySidharth Shekhar
- b valueUploaded byIffa D'Galaphicos
- acpc2013Uploaded byMoncef Mhasni
- TADM2E 2.3 - Algorithm Wiki FUNCTIONPRESTIFEROUS.pdfUploaded byJessica Gomez
- CutOff Report for RP 12042016 Web (1)Uploaded bysambit
- Annex 4. List of Research FieldsUploaded bydpk
- ECE 2ND & 3RDUploaded byanon-938661
- Consolidated[1]Uploaded bylegenkiller
- Fusion of Masses Defined on Infinite Countable Frames of DiscernmentUploaded byRyanElias
- GATE 2013 Answer Key.pdfUploaded byamreshkr19
- cutoffUploaded bygeekyboygeekyboy
- IEEE-Chilecon2017-Program.pdfUploaded byLacides Ripoll Solano
- CS or CEUploaded byapi-3770072
- Assignment 3 SolutionsUploaded byAndrianMelmamBesy
- solutio generalisUploaded byIlham
- weelby resumeUploaded byapi-302028279
- Research PaperUploaded byTom
- ME-GATE-2012 questionUploaded byrobyoga
- 20-14 Iste Standards-cse PDFUploaded byDurga Prasad
- cs2013-strawmanUploaded bySatish Paruchuri
- cs2013-strawmanUploaded bydardanthaqi
- Project 3Uploaded byBrian Dean

- FREE EDUCATION in EUUploaded byshannuonly4u
- aiIntro-ch10 (1)Uploaded byAshok Kumar
- Teach yourself cncUploaded bytarnoscribd
- 879Uploaded byxionnipu
- Study in Germany Land of IdeasUploaded byRoshan Sajjad
- cvUploaded bySyed Zubair
- 9A12501 Automata & Compiler DesignUploaded bysivabharathamurthy
- UNIX PresentationUploaded byRahul Parashar
- PptUploaded byПартизан Мехта
- Surveying the Field of ComputingUploaded byBandile Mbele
- 10 03 SoftwareUploaded byPrateek Garg
- A History of C++: 1979-1991Uploaded byJeff Pratt
- 1. Processes and Threads (2013 1) 1x2Uploaded byMariella Porras
- Titlelist CopperUploaded byArun Kumar
- Airlines ReservationUploaded byPritam Debnath
- Assignment 6Uploaded bykiran1213
- Handout Preparation ModelUploaded byPrasadDam
- The Free Lunch is OverUploaded byRaziwan
- Chapter 5Uploaded bybebo_ha2008
- 1. Intro 8051 ArchUploaded byquasar1234
- SRIBD - Simple Batch SystemsUploaded byTro Jan Ong
- 58979208-Project-on-C-programming.docxUploaded byAnil Rawat
- Fifth Semester Question Paper CO OSY Sum15Uploaded byAkshayHanumante
- BDA Prac 8Uploaded byKaranjot Vilkhu
- ST-Lab8(1)Uploaded byMuhammad Asif
- Ieeestd 1228 1994 Software Safety PlansUploaded byCarlos Aguilera
- Qu QueryUploaded byLouis Davis
- scopus.pdfUploaded byHarish Venkatasubramanian
- Second Semester 2010-2011 - Schedue of Classes - As of October 26 2010Uploaded byNtheng DaVinci
- PHI LEARNING Computer Science IT Engineering Electrical Electronics Mechanical Civil Chemical Metallurgy and Agricultural Catalogue 2016Uploaded byPHI Learning Pvt. Ltd.

## Much more than documents.

Discover everything Scribd has to offer, including books and audiobooks from major publishers.

Cancel anytime.