Fall 2023 - Analysis and Design of Algorithms

Lecture 5: Dynamic Programming

Ahmed Kosba

Department of Computer and Systems Engineering

Faculty of Engineering
Alexandria University
Dynamic Programming
• As in divide-and-conquer, dynamic programming
solves a problem by breaking it into subproblems.
• In the divide-and-conquer approach, the subproblems
are independent (disjoint).
• On the other hand, dynamic programming is applied
when there is some overlap between the subproblems,
i.e., when the subproblems share subsubproblems.
• "Programming" in this context refers to planning or
using a tabular method to store the intermediate
results of the subsubproblems.
• Used for optimization problems in different areas.
2
Dynamic Programming
• Where did the name come from?

3
Examples covered in this lecture
• Fibonacci numbers
• Rod cutting problem
• Matrix-chain multiplication

4
Computing Fibonacci Numbers
• Fibonacci numbers: 0, 1, 1, 2, 3, 5, 8, …
F0 = 0, F1 = 1
Fi = Fi-1 + Fi-2 for i > 1
• Compute the nth Fibonacci number recursively (n ≥ 0):
Fib(n)
if (n == 0 or n == 1)
return n
return Fib(n-1) + Fib(n-2)
Running time? T(n) = T(n-1) + T(n-2) + θ(1)
Note: This is simplified. θ(1) is not accurate here, as the numbers grow 1 with n,
i.e., the addition won’t be constant-time.
1 5
The nth Fibonacci number is θ(n)-bit long.
Computing Fibonacci Numbers
• Based on the recurrence, the running time of the
naïve recursive algorithm is O(2n) and Ω(2n/2).
• Exercise: Verify the upper and lower bounds.
• What can you observe about the recursion tree of the
previous algorithm?
Subproblems appear multiple times.
Fib(n)

Fib(n-2) Fib(n-1)

Fib(n-3) Fib(n-4) Fib(n-2) Fib(n-3)

Fib(n-3) Fib(n-4)

6
Computing Fibonacci Numbers
• How to avoid the unnecessary cost of the previous
implementation?
• Two methods:
• Top-down approach: Use the recursive approach, but
with memoization.
• Store the solutions of subproblems.
• Bottom-up approach: Compute the solutions of the
smaller subproblems first.
• A dependency graph of the subproblems can be built and used
to infer which problems to do first.

7
Computing Fibonacci Numbers
• Top-down approach with memoization:
• Use an array F of size n to store the results of the
subproblems.
• Assume all F[i] are initialized to 0 at the beginning.

Fib(n)
if (n == 0 or n == 1)
return n
if (F[n] == 0)
F[n] = Fib(n-1) + Fib(n-2)
return F[n]

8
Computing Fibonacci Numbers
• Bottom-up approach:
• Compute the solutions to the small subproblems first,
and use them to compute the solutions of larger
subproblems.

Fib(n)
F[0] = 0
F[1] = 1
for(i = 2 to n)
F[i] = F[i-1] + F[i-2]
return F[n]
Note: The above solution can be done using constant space without
having to store all the previous solutions.
9
Computing Fibonacci Numbers
• There are other ways to compute Fibonacci
numbers but they are out of the context of this
lecture.
𝜑𝑛
• Method 1: 𝐹𝑛 = rounded to the nearest integer,
√5
where 𝜑 is the golden ratio. (Section 3.2 in CLRS)
• A problem with this approach is the floating-point errors.
• Method 2:
𝐹𝑛 𝑛 𝐹0
0 1
= .
𝐹𝑛+1 1 1 𝐹1

Exercise: How can exponentiations be done efficiently?

10
Rod Cutting Problem [CLRS 15.1]
• Given a rod of length n inches and a list of prices pi
for i = 1, 2, 3, … , n, find the best way to cut the rod
into pieces, such that the revenue is maximized
when selling them.
• pi represents the price of selling a rod of length i without
any cuts.
• The length of each piece must be an integer.
• A simpler variant of the problem is just to find the
maximum revenue, which we will focus first.

11
Rod Cutting Problem
• Example from CLRS notes:
n=8
i 1 2 3 4 5 6 7 8
pi 1 5 8 9 10 17 17 20

Solution?
Max possible revenue = 22
This is obtained by cutting the rod into pieces of lengths 2 and 6.

12
Rod Cutting Problem
• Attempt 1:
• Choose based on the maximum price per inch.
i 1 2 3 4 5 6 7 8
pi 1 5 8 9 10 17 17 20
pi / i 1 2.5 2.67 2.25 2 2.83 2.42 2.5

6 is selected at the beginning as it gives the highest pi / i.

A piece of 2 inches remains. Select the best cut for it using the same way.

i 1 2 3 4 5 6 7 8
pi 1 5 8 9 10 17 17 20
pi / i 1 2.5 2.67 2.25 2 2.83 2.42 2.5
So, the chosen cut is (6, 2)
What is the problem with this approach? 13
Rod Cutting Problem
• Counter example for attempt 1:
• Assume the rod was 4 inches only.
i 1 2 3 4
pi 1 5 8 9
pi / i 1 2.5 2.67 2.25

• If the piece with the maximum price per inch is selected first,
then (3,1) will be the proposed cut, but this won't be optimal.
• (2, 2) is the optimal solution here.
• The price of (3, 1) is 9, while the price for (2, 2) is 10.

• Using a simple greedy strategy won't always lead to

optimal solutions. 14
Rod Cutting Problem
• Given a rod of length 5, how many ways to cut it into pieces?
• (5)
• (4, 1), (1, 4)
• (3, 2), (2, 3)
• (3, 1, 1), (1, 3, 1), (1, 1, 3)
• (2, 2, 1), (2, 1, 2), (1, 2, 2)
• (2, 1, 1, 1), (1, 2, 1, 1), (1, 1, 2, 1), (1, 1, 1, 2)
• (1, 1, 1, 1, 1)
• If the order matters, the number of possible ways for
partitioning is 2n-1 for a rod of length n, e.g., 16 for n = 5.
• If the order does not matter, there is no closed form to get the
exact number, but it's still exponential.
• See the partition function (p(5) = 7).
• The number of unique partition ways can be approximated by an
exponential function, i.e., it won't be efficient to enumerate and check
them all. 15
Rod Cutting Problem
• Breaking the problem into subproblems:
• Let rn denote the maximum revenue of selling a rod of
length n.
• The maximum revenue when a cut at a position i is made =
ri + r n - i
Find max revenue for a rod of length i Find max revenue for a rod of length n - i

Cut at position i

• However, we don't know where the cut (i) should be made.

• Therefore, compare the maximum revenues for all possible
values of i, and return their maximum.
rn = max(pn, r1 + rn-1 , r2 + rn-2 , …, rn-1 + r1) 16
Rod Cutting Problem
• This problem has an optimal substructure.
Optimal solutions to the rod cutting problem incorporate
optimal solutions to related subproblems, which can be
solved independently.

17
Rod Cutting Problem
A simpler way to view the problem: Find max revenue for a
p1 +
rod of length n - 1
Every decomposition of the rod starts
with a first piece of length i followed
Find max revenue for a
by a decomposition of the remainder p2 +
rod of length n - 2
(n – i) inches.

p3 + Find max revenue for a

max rod of length n - 3
rn = max(pi + rn - i) for 1 ≤ i ≤ n
r0 = 0
.
.
pn

18
Rod Cutting Problem
Naïve Algorithm
Cut-Rod(p, n)
Pseudocode from CLRS
if n == 0
return 0
q = -∞
for(i = 1 to n) How efficient is this algorithm?
q= max(q, p[i] + Cut-Rod(p, n- i))
return q
4

Tracing the algorithm on n = 4. 3 2 1 0

2 1 0 1 0 0
As in the first example, the
subproblems overlap. 1 0 0 0
19
0
Rod Cutting Problem
Naïve Algorithm
• Runtime of the naïve recursive algorithm:
• Let 𝑇 𝑛 be the number of calls to Cut-Rod
𝑇 0 =1
𝑛−1
𝑇 𝑛 = 1 + Σ𝑗=0 𝑇(𝑗)
• It can be shown that 𝑇 𝑛 = 2n

• The naïve algorithm solves the same subproblems

unnecessarily. Each subproblem should be solved
once.
• Dynamic programming can be applied here.

20
Rod Cutting Problem
Bottom-up Approach
Pseudocode from CLRS

Start from the smallest subproblems and solve them first

What's the running time of BOTTOM-UP-CUT-ROD?

Θ(n2)

21
Rod Cutting Problem
Top-down approach with Memoization
Pseudocode from CLRS

Initialize memo.

Running time
of MEMOIZED-
Check if the problem was solved before. CUT-ROD:
Θ(n2)

Store the solution of the subproblem.

22
Rod Cutting Problem
• Subproblem Graph
• Shows which subproblems need to be solved first.
• Helps with identifying the running time of the dynamic
programming algorithm.
• Usually, the running time is linear in the number of edges
and vertices. 4

Subproblem graph for n = 4

2

23
0
Rod Cutting Problem
Reconstructing a solution:
Original goal: return the actual piece sizes (not just
the price).
Pseudocode from CLRS

Auxiliary data structure to hold data needed for solution

reconstruction.

Store the size of the first piece that would lead to the
maximum revenue for a rod of size j.

24
Rod Cutting Problem
• Trace the previous algorithm for n = 8, and p
defined as follows:
i 1 2 3 4 5 6 7 8
pi 1 5 8 9 10 17 17 20

i 0 1 2 3 4 5 6 7 8
r[i] 0 1 5 8 10 13 17 18 22
s[i] 0 1 2 3 2 2 6 1 2

To construct the solution, note that s[n] will have the length of the first piece
that would lead to the optimal solution for a rod of length n.
As s[8] = 2, the length of the first piece should be 2.
The next piece length can be found by s[8 - 2] = s[6] = 6.
In the general case, this continues till we reach s[0].
(Pseudocode of reconstruction is in CLRS – Simple while loop)
25
Dynamic Programming Paradigm
- Study the structure of the optimal solution.
- Find a recursive definition.
- Compute the value of the optimal solution using
either
- a bottom-up approach, or
- a top-down approach with memoization.
- Construct an optimal solution.

26
Examples covered in this lecture
• Fibonacci numbers
• Rod cutting problem
• Matrix-chain multiplication

27
Matrix-chain Multiplication [CLRS 15.2]
Given a sequence of n matrices A1, A2, .. , An, the goal is to
compute their product
A1A2 … An
Determine a way for multiplying the matrices with the lowest
possible cost (min. number of scalar multiplications).

Note that the order of multiplication has an impact on the

total cost.
Assume having three matrices with the following dimensions:
(10, 100), (100, 5), (5, 50)
(A1A2)A3 costs 7500 scalar multiplications.
A1 (A2A3) costs 75000 scalar multiplications.
28
Matrix-chain Multiplication
Attempt 1:
Why not check all the possible parenthesizations?
For example, if n = 4, the number of possible parenthesizations is 5.
A1(A2(A3A4))
A1((A2A3)A4)
(A1(A2A3))A4
(A1A2)(A3A4)
((A1A2)A3)A4
However, the number grows exponentially as n grows. (For n = 30,
there are 1002242216651368 possibilities).
The number of parenthesizations for multiplying n matrices equals
the (n-1)th Catalan number. This is exponential in n. 29
Matrix-chain Multiplication
• The optimal solution can be constructed from the
optimal solutions to subproblems.
• Suppose we made the parenthesization as follows
(𝐴1 𝐴2 … 𝐴𝑘 )(𝐴𝑘+1 … 𝐴𝑛 )
• The optimal solution for this case can be
constructed from the optimal solutions for
parenthesizations of 𝐴1 𝐴2 … 𝐴𝑘 and 𝐴𝑘+1 … 𝐴𝑛 .
• As before, we need to consider all possible values
of 𝑘.

30
Matrix-chain Multiplication
• Let m[i, j] be the minimum number of scalar
multiplications need to compute 𝐴𝑖 𝐴𝑖+1 … 𝐴𝑗
• Each 𝐴𝑖 is a 𝑝𝑖−1 × 𝑝𝑖 matrix.
• Suppose 𝐴𝑖 𝐴𝑖+1 … 𝐴𝑗 is parenthesized as
(𝐴𝑖 𝐴𝑖+1 … 𝐴𝑘 )(𝐴𝑘+1 𝐴𝑘+2 … 𝐴𝑗 )
The optimal cost for this will be
m[i, k] + m[k+1, j] + 𝑝𝑖−1 𝑝𝑘 𝑝𝑗
• There are possible j – i ways to do the
parenthesizations.
• Need to select k that would lead to the minimum
cost. 31
Matrix-chain Multiplication
Recursive definition for m[i, j]
m[i, j] = 0 if i = j
m[i, j] = min {m[i, k] + m[k+1, j] + 𝑝𝑖−1 𝑝𝑘 𝑝𝑗 } if i < j
i≤k<j

Example: m[1, 1] + m[2, 4] + 𝑝0 𝑝1 𝑝4

m[1, 2] + m[3, 4] + 𝑝0 𝑝2 𝑝4
m[1, 4] = min
m[1, 3] + m[4, 4] + 𝑝0 𝑝3 𝑝4

32
Matrix-chain Multiplication
Subproblem dependencies
[1, 4]

[1, 3] [2, 4]

[1, 2] [2, 3] [3, 4]

[1, 1] [2, 2] [3, 3] [4, 4]

To solve the problem using a bottom-up approach, solve the lower

problems first. 33
Matrix-chain Multiplication
Bottom-up approach
• Example: Find the optimal way to multiply A1A2A3A4
matrix A1 A2 A3 A4
dimension 30 x 35 35 x 15 15 x 5 5 x 10
j
1 2 3 4
1 m[1,4]: value of optimal solution will
appear here.
2
i
3
4 Basic subproblems with no dependencies
m[i,i] = 0
m

34
Matrix-chain Multiplication
Bottom-up approach
• Example: Find the optimal way to multiply A1A2A3A4
matrix A1 A2 A3 A4
dimension 30 x 35 35 x 15 15 x 5 5 x 10
j
1 2 3 4
1 0
2 0
i
3 0
4 0

35
Matrix-chain Multiplication
Bottom-up approach
• Example: Find the optimal way to multiply A1A2A3A4
matrix A1 A2 A3 A4
dimension 30 x 35 35 x 15 15 x 5 5 x 10
j
1 2 3 4
1 0 m[1,2] = m[1,1] + m[2,2] + p0p1p2

2 0 m[2,3] = m[2,2] + m[3,3] + p1p2p3

i
3 0 m[3,4] = m[3,3] + m[4,4] + p2p3p4
4 0

36
Matrix-chain Multiplication
Bottom-up approach
• Example: Find the optimal way to multiply A1A2A3A4
matrix A1 A2 A3 A4
dimension 30 x 35 35 x 15 15 x 5 5 x 10
j
1 2 3 4
1 0 15750

2 0 2625
i
3 0 750

4 0

37
Matrix-chain Multiplication
Bottom-up approach
• Example: Find the optimal way to multiply A1A2A3A4
matrix A1 A2 A3 A4
dimension 30 x 35 35 x 15 15 x 5 5 x 10
j
1 2 3 4
1 0 15750 m[1,3] = min( m[1,1] + m[2,3] + p0p1p3 ,
m[1,2] + m[3,3] + p0p2p3 )
2 0 2625
i
3 0 750
m[2,4] = min( m[2,2] + m[3,4] + p1p2p4 ,
4 0 m[2,3] + m[4,4] + p1p3p4 )

m
The colored expressions will lead to the minimum value for m[I, j] at the specified i, j.
38
As we will see later, it will help to keep track of which k led to the minimum.
Matrix-chain Multiplication
Bottom-up approach
• Example: Find the optimal way to multiply A1A2A3A4
matrix A1 A2 A3 A4
dimension 30 x 35 35 x 15 15 x 5 5 x 10
j
1 2 3 4
1 0 15750 7875

2 0 2625 4375
i
3 0 750

4 0

39
Matrix-chain Multiplication
Bottom-up approach
• Example: Find the optimal way to multiply A1A2A3A4
matrix A1 A2 A3 A4
dimension 30 x 35 35 x 15 15 x 5 5 x 10
j
1 2 3 4
1 0 15750 7875 m[1,4] = min( m[1, 1] + m[2, 4] + 𝑝0 𝑝1 𝑝4
m[1, 2] + m[3, 4] + 𝑝0 𝑝2 𝑝4
2 0 2625 4375
i m[1, 3] + m[4, 4] + 𝑝0 𝑝3 𝑝4 )
3 0 750

4 0

40
Matrix-chain Multiplication
Bottom-up approach
• Example: Find the optimal way to multiply A1A2A3A4
matrix A1 A2 A3 A4
dimension 30 x 35 35 x 15 15 x 5 5 x 10
j
1 2 3 4
1 0 15750 7875 9375

2 0 2625 4375
i How to construct the optimal solution?
3 0 750

4 0

41
Matrix-chain Multiplication
Bottom-up approach
• Example: Find the optimal way to multiply A1A2A3A4
matrix A1 A2 A3 A4
dimension 30 x 35 35 x 15 15 x 5 5 x 10
j
1 2 3 4
1 0 15750 7875 9375

2 0 2625 4375
i How to construct the optimal solution?
3 0 750
As before, we can maintain an auxiliary data
4 0 structure along the way.

42
Matrix-chain Multiplication
Bottom-up approach
• Example: Find the optimal way to multiply A1A2A3A4
matrix A1 A2 A3 A4
dimension 30 x 35 35 x 15 15 x 5 5 x 10
j j
1 2 3 4 1 2 3 4
1 0 15750 7875 9375 1 1 1 3
2 0 2625 4375 2 2 3
i i
3 0 750 3 3
4 0 4
m s
s[i,j] = the value of k that led to the optimal
value for m[i,j]
 Matrix-chain Multiplication
Bottom-up approach
• Example: Find the optimal way to multiply A1A2A3A4
matrix A1 A2 A3 A4
dimension 30 x 35 35 x 15 15 x 5 5 x 10

j
Given the matrix s, how to construct an optimal
1 2 3 4
solution for A1A2A3A4?
1 1 1 3 • Check s[1, 4] first. Since s[1,4] = 3, this means
2 2 3 that the best way to multiply would be
i (A1A2A3 )A4
3 3 • However, this does not answer how A1A2A3
4 should be multiplied. To find this, check s[1, 3].
Since s[1,3] = 1, the best way would be A1(A2A3)
s • Therefore, the best way would be (A1(A2A3))A4
Matrix-chain Multiplication
Bottom-up approach
• Note: The example in the CLRS book draws the
matrices in a different way, but it is the same
method.

Figures from CLRS 45

Matrix-chain Multiplication
Bottom-up approach
Pseudocode from CLRS

Work on small subproblems first. Start with chain length 2.

Loop over the possible values of i, j according to the chain length.

For example, when length = 2, find m[1,2], m[2,3], m[3,4], ….
when length = 3, find m[1,3], m[2,4], m[3,5], ….

Matches the definition of m[i, j]

listed before.

Keep track of k for reconstruction.

The cost can be shown to be θ(n3)

46
Exercise
• Write pseudocode for solving the matrix-chain
multiplication problem using the top-down
recursive approach with memoization.

47