ch15 Rod Cuttingdp 01

CSE 373 Design and Analysis of Algorithms
Mirza Mohammad Lutfe Elahi
Department of Electrical and Computer Engineering

North South University
Dynamic Programming - Rod Cutting Problem

Chapter 15
Mirza Mohammad Lutfe Elahi (ECE@NSU) CSE 373 Design and Analysis of Algorithms 1 / 28
Dynamic Programming
Fibonacci Numbers
0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, ...

Computing the nth Fibonacci number recursively:
F(n) = F(n-1) + F(n-2)
F(0) = 0
F(1) = 1
Top-Down Approach
Fib(n)
1 if n = 0
2 return 0
3 if n = 1
4 return 1
5 return Fib(n − 1) + Fib(n − 2)
Dynamic Programming
Fibonacci Numbers
Dynamic Programming
Fibonacci Numbers
Time complexity between 2n/2 and 2n

T (n) = F (n) = O(1.6n )
Dynamic Programming
Fibonacci Numbers
Problem with the recursive FIB algorithm

Each subproblem was solved for many times!
Solution: avoid solving the same subproblem more than once
Compute on demand, but memorize the solution to avoid recomputing
Pre-compute all sub-problems that may be needed later
Dynamic Programming
Fibonacci Numbers
Computing the nth Fibonacci number using a top-down approach

(Memoization):
Fib(n)
1 if F [n] = −1
2 F [n] = Fib(n − 1) + Fib(n − 2)
3 return F [n]
Dynamic Programming
Fibonacci Numbers
Dynamic Programming
Fibonacci Numbers
0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, ...

Computing the nth Fibonacci number using a bottom-up approach:
F(0) =0
F(1) =1
F(2) =1+0
...
F(n) = F(n-1) + F(n-2)
Efficiency
Time - O(n)
Space - O(n)
Dynamic Programming
Fibonacci Numbers
Computing the nth Fibonacci number using a bottom-up approach

(Iterative):
Fib(n)
1 F [0] = 0
2 F [1] = 1
3 for i = 2 to n
4 F [i] = F [i − 1] + F [i − 2]
5
6 return F [n]
Dynamic Programming
Dynamic Programming
The term Dynamic Programming comes from Control Theory, not

computer science.
Dynamic Programming is an algorithm design technique for
optimization problems: often minimizing or maximizing. In such
problem there can be many solutions. Each solution has a value, and
we wish to find a solution with the optimal value.
It refers to the use of tables (arrays) to construct a solution.
Dynamic Programming
Dynamic Programming
Like divide and conquer, DP solves problems by combining solutions

to sub-problems.
Unlike divide and conquer, sub-problems are not independent.
Sub-problems may share sub-sub-problems
We solve the problem by solving sub-problems of increasing size and
saving each optimal solution in a table (usually).
The table is then used for finding the optimal solution to larger
problems.
In dynamic programming we usually reduce time by increasing the
amount of space.
Time is saved since each sub-problem is solved only once.
Dynamic Programming
Dynamic Programming
1 Characterize the structure of an optimal solution.

2 Recursively define the value of an optimal solution.
3 Compute the value of an optimal solution in bottom up fashion.
4 Construct an optimal solution from computed information.
Dynamic Programming
Rod Cutting Problem
You are given a rod of length n ≥ 0 (n in inches)

A rod of length i inches will be sold for pi dollars
Cutting is free (simplifying assumption).
Problem: given a table of prices pi determine the maximum revenue
rn obtainable by cutting up the rod and selling the pieces.
Dynamic Programming
Rod Cutting Problem
Greedy approach:
Select the length that has the maximum price/length
Reduce to problem to a smaller sub-problem
Does it work?
Use the greedy approach for length = 4
Greedy solution is 3, 1 with revenue 9
Optimal solution is 2, 2 with revenue 10
Dynamic Programming
Rod Cutting Problem

Step 1: Characterizing an Optimal Solution
Question: in how many different ways can we cut a rod of length n?
For a rod of length 4:
24−1 = 23 = 8
For a rod of length n: 2n−1 .

Exponential: we cannot try all possibilities for n - ”large”. The
obvious exhaustive approach won’t work.
Dynamic Programming
Rod Cutting Problem
Let us find a way to solve the problem recursively (we might be able
to modify the solution so that the maximum can be actually
computed):
Assume we have cut a rod of length n into 0 ≤ k ≤ n pieces of length
i1 , i2 ..., ik ,
n = i1 + i2 + ... + ik
with revenue
rn = pi1 + pi2 + ... + pik
Assume further that this solution is optimal.
How can we construct it?
Advice: when you don’t know what to do next, start with a simple
example and hope something will occur to you...
Dynamic Programming
Rod Cutting Problem
Begin by constructing (by hand) the optimal solutions for i = 1, 2, ..., 10:
r1 = 1 from solution 1 = 1 (no cuts)
r4 = 10 from solution 4 = 2 + 2
r7 = 18 from solution 7 = 1 + 6 or 7 = 2 + 2 + 3
Dynamic Programming
Rod Cutting Problem
Notice that in some cases rn = pn , while in other cases the optimal

revenue rn is obtained by cutting the rod into smaller pieces.
In ALL cases we have the recursion
rn = max (pn , r1 + rn−1 , r2 + rn−2 , ..., rn−1 + r1 )
exhibiting optimal substructure (meaning?)
A slightly different way of stating the same recursion, which avoids
repeating some computations, is
rn = max1≤i≤n (pi + rn−i )
And this latter relation can be implemented as a simple top-down
recursive procedure.
Dynamic Programming
Rod Cutting Problem
To-down recursive procedure:
Cut-Rod(p, n)
1 if n == 0
2 return 0
3
4 q = −∞
5
6 for i = 1 to n
7 q = max (q, p[i] + Cut-Rod(p, n − i))
8
9 return q
Dynamic Programming
Rod Cutting Problem
How to justify the step from:

rn = max (pn , r1 + rn−1 , r2 + rn−2 , ..., rn−1 + r1 )
to
rn = max1≤i≤n (pi + rn−i )
Every optimal partitioning of a rod of length n has a first cut – a
segment of, say, length i. The optimal revenue, rn , must satisfy
rn = pi + rn−i , where rn−i is the optimal revenue for a rod of length n
– i. If the latter were not the case, there would be a better
0
partitioning for a rod of length n–i, giving a revenue rn−i > rn−i and
0 0
a total revenue rn = pi + rn−i > pi + rn−i = rn .
Dynamic Programming
Rod Cutting Problem
Since we do not know which one of the leftmost cut positions

provides the largest revenue, we just maximize over all the possible
first cut positions.
We can also notice that all the items we choose the maximum of are
optimal in their own right: each substructure (max revenue for rods
of lengths 1, 2, ..., n − 1) is also optimal (again, optimal substructure
property).
Nevertheless, we are still in trouble: computing the recursion leads to
recomputing a number (= overlapping subproblems) of values – how
many?
Dynamic Programming
Rod Cutting Problem - Simple Case
Cut-Rod(p, 4)
The number of nodes for a tree corresponding to a rod of size n is:

n−1
X
T (0) = 1, T (n) = 1 + T (j) = 2n , n ≥ 1
j=0
Dynamic Programming
Rod Cutting Problem
Beyond Naive Time Complexity

We have a problem: ”reasonable size” problems are not solvable in
”reasonable time” (but, in this case, they are solvable in ”reasonable
space”).
Specifically:
Note that navigating the whole tree requires 2n stack-frame activations.
Note also that no more than n + 1 stack-frames are active at any one
time and that no more than n + 1 different values need to be computed
or used.
Can we exploit these observations?
A standard solution method involves saving the values associated with
each T (j), so that we compute each value only once (called
”memoizing” = writing yourself a memo).
Dynamic Programming
Rod Cutting Problem -Memoization

Memoized-Cut-Rod(p, n)
1 Let r [0..n] be a new array
2 for i = 1 to n
3 r [i] = −∞
4 return Memoized-Cut-Rod-Aux(p, n, r )
Memoized-Cut-Rod-Aux(p, n, r )
1 if r [n] ≥= 0
2 return r [n]
3 if n == 0
4 q=0
5 else q = −∞
6 for i = 1 to n
7 q = max (q, p[i] + Memoized-Cut-Rod-Aux(p, n − i, r ))
8 r [n] = q
9 return r [n]
Dynamic Programming
Rod Cutting Problem - Bottom-Up
Bottom-Up-Cut-Rod(p, n)
1 Let r [0..n] be a new array
2 r [0] = 0
3 for j = 1 to n
4 q = −∞
5 for i = 1 to j
6 q = max (q, p[i] + r [j − i])
7 r [j] = q
8 return r [n]
Dynamic Programming
Rod Cutting Problem - Time Complexity
Step 1: Characterizing an Optimal Solution Whether we solve the problem

in a top-down or bottom-up manner the asymptotic time is Θ(n2 ), the
major difference being recursive calls as compared to loop iterations.
Why??
Dynamic Programming
Rod Cutting Problem - Reconstructing Solution
Extended-Bottom-Up-Cut-Rod(p, n)
1 Let r [0..n] and s[0..n] be a new arrays
2 r [0] = 0
3 for j = 1 to n
4 q = −∞
5 for i = 1 to j
6 if q < p[i] + r [j − i]
7 q = p[i] + r [j − i]
8 s[j] = i
9 r [j] = q
10 return r and s
Dynamic Programming
Rod Cutting Problem - Reconstructing Solution
Print-Cut-Rod-Solution(p, n)
1 (r , s) = Extended-Bottom-Up-Cut-Rod(p, n)
2 while n > 0
3 print s[n]
4 n = n − s[n]

ch15 Rod Cuttingdp 01

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

ch15 Rod Cuttingdp 01

Uploaded by

Copyright:

Available Formats

CSE 373 Design and Analysis of Algorithms

Mirza Mohammad Lutfe Elahi

Department of Electrical and Computer Engineering

Dynamic Programming - Rod Cutting Problem

0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, ...

Time complexity between 2n/2 and 2n

Problem with the recursive FIB algorithm

Computing the nth Fibonacci number using a top-down approach

0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, ...

Computing the nth Fibonacci number using a bottom-up approach

The term Dynamic Programming comes from Control Theory, not

Like divide and conquer, DP solves problems by combining solutions

1 Characterize the structure of an optimal solution.

Rod Cutting Problem

You are given a rod of length n ≥ 0 (n in inches)

Rod Cutting Problem

Rod Cutting Problem

For a rod of length n: 2n−1 .

Rod Cutting Problem

Rod Cutting Problem

Rod Cutting Problem

Notice that in some cases rn = pn , while in other cases the optimal

Rod Cutting Problem

To-down recursive procedure:

Rod Cutting Problem

How to justify the step from:

Rod Cutting Problem

Since we do not know which one of the leftmost cut positions

Rod Cutting Problem - Simple Case

The number of nodes for a tree corresponding to a rod of size n is:

Rod Cutting Problem

Beyond Naive Time Complexity

Rod Cutting Problem -Memoization

Rod Cutting Problem - Bottom-Up

Rod Cutting Problem - Time Complexity

Step 1: Characterizing an Optimal Solution Whether we solve the problem

Rod Cutting Problem - Reconstructing Solution

Rod Cutting Problem - Reconstructing Solution

You might also like