You are on page 1of 9

# Dynamic Programming

Introduction Dynamic programming is a confusing name for a programming technique that dramatically reduces the runtime of algorithms: from exponential to polynomial. The basic idea is to try to avoid solving the same problem or subproblem twice. Here is a problem to demonstrate its power: Given a sequence of as many as 10,000 integers (0 < integer < 100,000), what is the maximum decreasing subsequence? Note that the subsequence does not have to be consecutive. Recursive Descent Solution The obvious approach to solving the problem is recursive descent. One need only find the recurrence and a terminal condition. Consider the following solution:
1# i n c l u d e< s t d i o . h > 2 l o n gn ,s e q u e n c e [ 1 0 0 0 0 ] ; 3 m a i n( ){ 4 F I L E* i n ,* o u t ; 5 i n ti ; 6 i n=f o p e n( " i n p u t . t x t " ," r " ) ; 7 o u t=f o p e n( " o u t p u t . t x t " ," w " ) ; 8 f s c a n f ( i n ," % l d " ,& n ) ; 9 f o r( i=0 ;i<n ;i + + )f s c a n f ( i n ," % l d " ,& s e q u e n c e [ i ] ) ; 1 0 f p r i n t f( o u t ," % d \ n " ,c h e c k( 0 ,0 ,9 9 9 9 9 9 ) ) ; 1 1 e x i t( 0 ) ; 1 2 } 1 3 c h e c k( s t a r t ,n m a t c h e s ,s m a l l e s t ){ 1 4 i n tb e t t e r ,i ,b e s t = n m a t c h e s ; 1 5 f o r( i=s t a r t ;i<n ;i + + ){ 1 6 i f( s e q u e n c e [ i ]<s m a l l e s t ){ 1 7 b e t t e r=c h e c k( i ,n m a t c h e s + 1 ,s e q u e n c e [ i ] ) ; 1 8 i f( b e t t e r>b e s t )b e s t=b e t t e r ; 1 9 } 2 0 } 2 1 r e t u r nb e s t ; 2 2 }

Lines 1-9 and and 11-12 are arguably boilerplate. They set up some standard variables and grab the input. The magic is in line 10 and the recursive routine `check'. The `check' routine knows where it should start searching for smaller integers, the length of the longest sequence so far, and the smallest integer so far. At the cost of an extra call, it terminates `automatically' when `start' is no longer within proper range. The `check' routine is simplicity itself. It traverses along the list looking for a smaller integer than the `smallest' so far. If found, `check' calls itself recursively to find more. The problem with the recursive solution is the runtime:
N S e c o n d s 6 0 0 . 1 3 0 7 0 0 . 3 6 0 8 0 2 . 3 9 0

& n u m [ i ] ) . Lines 12-20 run the O(N 2) algorithm in a fairly straightforward way with the `i' loop counting backwards and the `j' loop counting forwards.i ){ b e s t s o f a r [ i ]=1 .9 0 1 3 . Consider the sequence starting at the end. F I L E* i n . it's `1'." % l d " . Any time it's larger than an element closer to the end.* o u t . f o r( j=i + 1 . of length 1.j + + ){ i f( n u m [ j ]<n u m [ i ]& &b e s t s o f a r [ j ]> =b e s t s o f a r [ i ] ) { b e s t s o f a r [ i ]=b e s t s o f a r [ j ]+1 . it is often fruitful to trade a bit of storage for execution efficiency. f s c a n f ( i n . keeping track of the longest descending (sub-)sequence so far in an auxiliary variable. t x t " . the largest of the `bestsofar's is the length of the longest descending subsequence. l o n gn . Another program might work from the end of the sequence. Any sequence of length 1 meets all the criteria for a longest sequence. Check out its code: 1 2 3 4 5 6 7 8 9 1 0 1 1 1 2 1 3 1 4 1 5 1 6 1 7 1 8 1 9 2 0 2 1 2 2 2 3 # i n c l u d e< s t d i o . o u t=f o p e n( " o u t p u t . i n=f o p e n( " i n p u t . b e s t s o f a r [ n 1 ]=1 . 9 5 0 1 . this solution is of limited interest.j<n . Upon termination. } } } f p r i n t f ( o u t . 0 8 0 2 . One line longer then before.j . the runtime figures show better performance: N 1 0 0 0 2 0 0 0 3 0 0 0 4 0 0 0 5 0 0 0 6 0 0 0 7 0 0 0 S e c s 0 . that means looking at the end of the sequence first. Notate the `bestsofar' array as `1' for this case.b e s t s o f a r [ M A X N ] . lines 1-10 are boilerplate.i + + )f s c a n f ( i n . f o r( i=n 1 1 . 9 9 0 . In this case.i . 2 4 0 0 . i f( b e s t s o f a r [ i ]>l o n g e s t )l o n g e s t=b e s t s o f a r [ i ] . 0 8 0 0 ." w " ) . t x t " . Consider any element prior to the last two. 1 9 0 Since the particular problem of interest suggests that the maximum length of the sequence might approach six digits. it is often fruitful to approach the problem backward." % l d " ." r " ) .i> =0 . 4 5 0 2 . f o r( i=0 . then the `bestsofar' is 2 (which is 1 + `bestsofar' for the last number). This is fairly clearly an O(N 2) algorithm. Otherwise. } Again. If the penultimate number is larger than the last one. h > # d e f i n eM A X N1 0 0 0 0 m a i n( ){ l o n gn u m [ M A X N ] .i<n . its `bestsofar' element becomes at least one larger than that of the smaller element that was found. Consider the last two elements of the sequence. Starting At The End When solutions don't work by approaching them `forwards' or `from the front'.l o n g e s t=0 ." b e s t s o f a ri s% d \ n " . Line 11 sets up the end condition.& n ) . e x i t ( 0 ) . Additionally. 5 5 0 0 .l o n g e s t ) .

i .j> =0 . 7 0 0 6 . That inner loop (``search for any smaller number'') begs to have some storage traded for it.j ){ 1 9 i f( n u m [ i ]>b e s t r u n [ j ] ){ 2 0 i f( j= =h i g h e s t r u n-1| |n u m [ i ]<b e s t r u n [ j + 1 ] ) { . Implement an array `bestrun' whose index is the length of a long subsequence and whose value is the first (and. 1 1 b e s t r u n [ 0 ]=n u m [ n 1 ] . Because the `bestrun' array probably grows much less quickly than the length of the processed sequence. h > 2 # d e f i n eM A X N2 0 0 0 0 0 3 m a i n( ){ 4 F I L E* i n . 7 i n=f o p e n( " i n p u t . to the `bestrun' array grows: The `4' is not larger than the `6'.i> =0 .b e s t r u n [ M A X N ] . the `6' is larger than the `3'. 9 f s c a n f ( i n . 1 0 f o r( i=0 . 7 0 0 4 . so the `bestrun' array changes: The `9' extends the array: 0 : 3 1 : 4 2 : 9 0 : 3 1 : 4 0 : 3 1 : 6 0 : 3 1 089463 3 . The new integer might be a new `best head of sequence'.i<n .& n u m [ i ] ) ." w " ) . 1 6 c o n t i n u e .& n ) . longer sequence can potentially be created. Encountering an integer larger than one of the values in this array means that a new. 1 7 } 1 8 f o r( j=h i g h e s t r u n-1 . 6 l o n gn . 3 5 0 The `8' changes the array similar to the earlier case with the `4': 0 : 3 1 : 4 2 : 8 The `10' extends the array again: 0 : 3 1 : 4 2 : 8 3 : 1 0 and yields the answer: 4 (four elements in the array). `best') integer that heads that subsequence." r " ) ." % l d " . In practice. as it turns out. though it is larger than the `3'. Consider this sequence: Scanning from right to left (backward to front)." % l d " .8 0 0 0 9 0 0 0 1 0 0 0 0 1 1 0 0 0 The algorithm still runs too slow (for competitions) at N=9000. 5 l o n gn u m [ M A X N ] .* o u t .i ){ 1 4 i f( n u m [ i ]<b e s t r u n [ 0 ] ){ 1 5 b e s t r u n [ 0 ]=n u m [ i ] . 3 3 0 7 . 1 2 h i g h e s t r u n=1 . t x t " . t x t " .i + + )f s c a n f ( i n . this algorithm probabalistically runs much faster than the previous one.j . the `bestrun' array has but a single element after encountering the 3: Continuing the scan. A different set of values might best be stored in the auxiliary array. Here's a coding of this algorithm: 1 # i n c l u d e< s t d i o . 1 3 f o r( i=n 1 1 . 8 o u t=f o p e n( " o u t p u t . or it might not.h i g h e s t r u n=0 . the speedup is large.

k+1 ) . these lines only effect the `worst' case of the algorithm when the input is sorted `badly'. 0 6 0 0 . } } } } p r i n t f ( " b e s ti s% d \ n " .n .i + + ){ b e s t [ i ]=1 . y )( ( x ) > ( y ) ? ( x ) : ( y ) ) i n t b e s t [ S I Z E ] . lines 1-10 are boilerplate. 0 8 0 2 0 0 0 0 . 2 2 0 Marcin Mika points out that you can simplify this algorithm to this tiny little solution: # i n c l u d e< s t d i o . Again.b e s t [ k ]>x ." % d " . } p r i n t f( " b e s ti s% d \ n " . b r e a k .k + + ) . / /b e s t [ ]h o l d sv a l u e so ft h eo p t i m a ls u b s e q u e n c e i n t m a i n( v o i d ){ F I L E* i n =f o p e n( " i n p u t .k . b e s t [ k ]=x . Lines 18-26 are the meat that searches the bestrun list and contain all the exceptions and tricky cases (bigger than first element? insert in middle? extend the array?)." r " ) . F I L E* o u t=f o p e n( " o u t p u t . t x t " . 1 4 0 0 . 5 5 0 4 0 0 0 0 . e x i t ( 0 ) .i<n . 1 3 0 0 . 4 5 0 6 0 0 0 2 . 1 7 0 0 ." w " ) . 2 4 0 3 0 0 0 0 .s o l=1 . 0 3 0 0 . Mostly. 0 5 0 0 . 1 1 0 0 . 0 8 0 0 .& n ) . s o l=M A X( s o l .without memorizing it. Lines 14-17 are an optimization for a new `smallest' element. } . You should try to code this right now . 9 9 0 8 0 0 0 3 . i n t i . t x t " . / /N=h o wm a n yi n t e g e r st or e a di n f o r( i=0 . The speeds are impressive. 5 7 0 0 .x .h i g h e s t r u n ) . showing this algorithm worked for N well into five digits: N o r i g 1 0 0 0 0 . 0 9 0 0 . 0 8 0 7 0 0 0 2 . 1 6 0 0 ." % d " . 7 0 0 9 0 0 0 4 . They could have been moved after line 26. f s c a n f( i n .2 1 2 2 2 3 2 4 2 5 2 6 2 7 2 8 2 9 3 0 } b e s t r u n [ + + j ]=n u m [ i ] . f o r( k=0 . 2 9 0 2 .& x ) . The table below compares this algorithm with the previous one. Lines 11-12 are initialization. 2 9 0 0 . 3 5 0 2 0 0 0 0 4 0 0 0 0 6 0 0 0 0 8 0 0 0 0 1 0 0 0 0 0 I m p r o v e d 0 . 7 0 0 1 0 0 0 0 6 . r e t u r n0 .s o l ) . 9 1 0 1 . 9 5 0 5 0 0 0 1 . 3 3 0 1 1 0 0 0 7 . i f( j= =h i g h e s t r u n )h i g h e s t r u n + + . 0 3 0 0 . h > # d e f i n eS I Z E2 0 0 0 0 0 # d e f i n eM A X ( x . f s c a n f( i n .

& n ) .n .h i g h . . f o r( i=1 . t x t " . } } s o l=M A X( s o l .& x ) .& b e s t [ 0 ] ) . } / /c h e c ki fr i g h ts p o t i f ( x>b e s t [ k ]& &x<b e s t [ k 1 ] ) b e s t [ k ]=x .i<n .s o l ) .x . h i g h=s o l 1 . / /N=h o wm a n yi n t e g e r st or e a di n / /r e a di nt h ef i r s ti n t e g e r f s c a n f( i n . s o l=1 . b r e a k ." % d " ." r " ) . # i n c l u d e< s t d i o . The solutions above use a linear search to find the appropriate location in the 'bestrun' array to insert an integer. i f ( x<b e s t [ k ]& &x>b e s t [ k + 1 ] ) b e s t [ + + k ]=x . i f ( x> =b e s t [ 0 ] ){ k=0 . f s c a n f( i n . using binary search will make it run even faster. c o n t i n u e . ){ k=( i n t )( l o w+h i g h )/2 . f s c a n f( i n . However. } p r i n t f( " b e s ti s% d \ n " . } e l s e{ / /u s eb i n a r ys e a r c hi n s t e a d l o w=0 . } Summary . Tyler Lu points out the program below.Not to be outdone. c o n t i n u e . / /b e s t [ ]h o l d sv a l u e so ft h eo p t i m a ls u b s e q u e n c e i n t m a i n( v o i d ){ F I L E* i n =f o p e n( " i n p u t . h > # d e f i n eS I Z E2 0 0 0 0 0 # d e f i n eM A X ( x . r e t u r n0 .k+1 ) . i n tl o w .s o l . } / /g oh i g h e ri nt h ea r r a y i f ( x<b e s t [ k ]& &x<b e s t [ k + 1 ] ){ l o w=k+1 . because the auxiliary array is sorted. i n ti .i + + ){ b e s t [ i ]=1 . y )( ( x ) > ( y ) ? ( x ) : ( y ) ) i n t b e s t [ S I Z E ] ." % d " . f o r ( . / /g ol o w e ri nt h ea r r a y i f ( x>b e s t [ k ]& &x>b e s t [ k 1 ] ){ h i g h=k-1 .k . b e s t [ 0 ]=x ." % d " . decreasing the runtime to O(N log N). f c l o s e ( i n ) .

the solution is trivial (either the element is in the other sequence or it isn't). the main subproblem was: Find the largest decreasing subsequence (and its first value) for numbers to the `right' of a given element. There are only two possibilities. Two Dimensional DP It is possible to create higher dimension problems such as: Given two sequences of integers. For the previous programming challenge. Pseudocode Here's the pseudocode for this algorithm: 1 2 #t h et a i lo ft h es e c o n ds e q u e n c ei se m p t y f o re l e m e n t=1t ol e n g t h 1 l e n g t h [ e l e m e n t . the subproblems are the longest common subsequence of smaller sequences (where the sequences are the tails of the original subsequences).These programs demonstrate the main concept behind dynamic programming: build larger solutions based on previously found solutions. The other possibility results from some element in the tail of the second sequence matching the first element in tail of the first. First. The first element of the first tail might be in the longest common subsequence or it might not. Look at the problem of finding the longest common subsequence of the last i elements of the first sequence and the last j elements of the second sequences. Note that this sort of approach solves a class of problems that might be denoted ``one-dimensional''. and finding the longest common subsequence of the elements after those matched elements.l e n g t h 2 + 1 ]=0 #t h et a i lo ft h ef i r s ts e q u e n c eh a so n ee l e m e n t m a t c h e l e m=0 f o re l e m e n t=l e n g t h 2t o1 i fl i s t 1 [ l e n g t h 1 ]=l i s t 2 [ e l e m e n t ] 3 4 5 . what is the longest sequence which is a subsequence of both sequences? Here. The longest common sequence not containing the first element of the first tail is merely the longest common subsequence of the last i-1 elements of the first sequence and the last j elements of the second subsequence. if one of the sequences contains only one element. This building-up of solutions often yields programs that run very quickly.

e l e m e n t + 1 ]+1 l e n g t h [ l o c . but it's important what path you've taken (since you can't revisit cities on the return trip). e l e m e n t ]>m a x l e n m a x l e n=l e n g t h [ l o c + 1 . e l e m e n t ] #l o n g e s tc o m m o ns u b s e q u e n c ei n c l u d e sf i r s te l e m e n t i fl i s t 1 [ l o c ]=l i s t 2 [ e l e m e n t ]& & l e n g t h [ l o c + 1 . a bitonic can either increase-then-decrease or decrease-then-increase. according to the rules. with the flights that can be done (you can only fly between certain cities. what's longest sequence assuming the turn has not occurred yet. The obvious item to try to do dynamic programming on is your location and your direction. the subproblems are the longest bitonic sequence and the longest decreasing sequence of prefixes of the sequence (basically. Sometimes the subproblems are well hidden: You have just won a contest where the prize is a free vacation in Canada. only increase and then decrease will be considered). e l e m e n t ]=n m a t c h l e n #l o o po v e rt h eb e g i n n i n go ft h et a i lo ft h ef i r s ts e q u e n c e f o rl o c=l e n g t h 1 1t o1 m a x l e n=0 f o re l e m e n t=l e n g t h 2t o1 #l o n g e s tc o m m o ns u b s e q u e n c ed o e s n ' ti n c l u d ef i r s te l e m e n t i fl e n g t h [ l o c + 1 . where N and M are the respectively lengths of the sequences. l i s t 2 [ e l e m e n t ] ) l o c a t i o n 2=e l e m e n t+1 f l a g=T r u e b r e a kf o r l o c a t i o n 1=l o c a t i o n 1+1 The trick to dynamic programming is finding the subproblems to solve. and what's longest sequence starting here assuming the turn has already occurred).6 7 8 9 1 0 1 1 1 2 1 3 1 4 1 5 1 6 m a t c h e l e m=1 l e n g t h [ l e n g t h 1 . and the number of paths is too large to be able to solve (and store) the result of doing all of those subproblems. You must travel via air. . e l e m e n t + 1 ] + 1=l e n g t h [ l o c a t i o n 1 . given the length matrix. travel only east until you reach the furthest city east. Sometimes it involves multiple parameters: A bitonic sequence is a sequence which increases for the first part and decreases for the second part. you must start at the further city west. e l e m e n t + 1 ] + 1>m a x l e n m a x l e n=l e n g t h [ l o c . Given the order of the cities. In addition. In this case. e l e m e n t ]=m a x l e n This program runs in O(N x M) time. l o c a t i o n 2 ] o u t p u t( l i s t 1 [ l o c a t i o n 1 ] . Note that this algorithm does not directly calculate the longest common subsequence. of course). Find the longest bitonic sequence of a sequence of integers (technically. you may visit no city more than once (except the starting city. calculate the maximum number of cities you can visit. However. and the cities are ordered from east to west. you can determine the subsequence fairly quickly: 1 2 3 4 5 6 7 8 9 1 0 1 1 1 2 l o c a t i o n 1=1 l o c a t i o n 2=1 w h i l e( l e n g t h [ l o c a t i o n 1 . and then fly only west until you reach your starting location. l o c a t i o n 2 ]! =0 ) f l a g=F a l s e f o re l e m e n t=l o c a t i o n 2t ol e n g t h 2 i f( l i s t 1 [ l o c a t i o n 1 ]=l i s t 2 [ e l e m e n t ]A N D l e n g t h [ l o c a t i o n 1 + 1 . but for this problem. and just because you can fly from city A to city B does not mean you can fly the other direction). In addition.

instead of trying to find the path as described. . such as an integer." where he can travel from city A to city B if and only if there is a flight from city B to city A. . but the travelers may never be at the same city. you could do recursive descent to find the complete path. the path may not be important. . . Almost without fail. depending on how you look at it. Your goal is to completely describe the state of a solution in a small amount of data. dynamic programming solutions are applied to those solutions which would otherwise be exponential in time. . Imagine having two travelers who start in the western most city. Why this algorithm might yield the maximum number of cities is left as an exercise. you only pass a small amount of data. you should check to see if a dynamic programming solution exists if the inputs limits are over 30. . but that means you'd have to pass not only your location. Thus. For example. a pair of integers. . The travelers take turns traveling east. you know that the traveler y has not yet visited any city east of traveler x except the city traveler y is current at. for example: .However. and then taking the reverse of the other traveler's path back to the western-most city. any program you were thinking about doing recursive descent on. you will not be able to do dynamic programming unless the paths are very short. evaluate the simple term) across edges. there is a way to do the recursive descent such that at each step. but the cities you've visited already (either as a list or as a boolean array). if. Basically. Sample Problems Polygon Game [1998 IOI] Imagine a regular N-gon. That is. a boolean and an integer. either and the operators `+' or `*' on the edges. -8* -7. as otherwise traveler y must have moved once while x was west of y. unless it is either the first or the last city. as in the air travel problem. so if the bounds of the problem are too large to be able to be done in exponential time for any but a very small set of the input cases. replacing the edge and end points with node with value equal to value of end point combined by operations. Recognizing Problems solvable by dynamic programming Generally. in the air travel one. the two traveler's paths are disjoint. Put numbers on nodes. the subproblem will be the `tail-end' of a problem. recursing on the pair of cities as you travel east subject to the constant given is a very small amount of data to recurse on. . After that..g. However. Finding the Subproblems As mentioned before. where the next traveler to move is always the western-most. finding the subproblems to do dynamic programming over is the key. The first move is to remove an edge. it is found a different manner. then the number of states greatly decreases. However. look for a dynamic programming solution. -3+ -5* -7. by taking the normal traveler's path to the eastern-most city. However. combine (e. . . etc. Also. That's too much state for dynamic programming to work on. It's not too difficult to see that the paths of the two travelers can be combined to create a round-trip. If the path is important. one of the traveler is only allowed to make "reverse flights. when traveler x is moved.

For example.. 7} so that each partition has the same sum: {1. USACO Gateway | Comment or Question . Subset Sums [Spring 98 USACO] . .4. 3. Number Game [IOI 96. 2. -5 6. there are 4 ways to partition the set {1.6. one can partition the set into two sets whose sums are identical.2} This counts as a single partitioning (i. maybe] Given a sequence of no more than 100 integers (-32000.32000). reversing the order counts as the same partitioning and thus does not increase the count of partitions). .. if N=3.Given a labelled N-gon. determine the maximum winning score for the first player.6} {3.6} Given N. 3} in one way so that the sums of both subsets are identical: {3} and {1.7} and {1. . .3. your program should print the number of ways a set containing the integers from 1 through N can be partitioned into two sets whose sums are identical. Print 0 if there are no such ways. If N=7.5.5.4. maximize the final value computed. assuming the second player plays optimally.7} and {3.3.e.7} and {1..5.4.2. Given a sequence.7} and {2.2. .6} {1. two opponents alternate turns removing the leftmost or rightmost number from a sequence.5} {2. 2. one can partition the set {1. Each player's score at the end of the game is the sum of those numbers he or she removed.4. For many sets of consecutive integers from 1 through N (1 <= N <= 39)..