Algorithms-I CSC 302 (July-Dec2022 IIITK) L43 - L44

Algorithms - I
CSC 302
SK Hafizul Islam
Department of CSE, IIIT Kalyani
October 21, 2022
SK Hafizul Islam (Department of CSE, IIIT Kalyani) Algorithms - I October 21, 2022 1 / 29
Agenda I
1 Dynamic Programming
0-1 Knapsack Problem
Naive Approach
Dynamic Programming Approach
An Example
2 Suggested Readings
We are given n objects and a knapsack.

Object Oi has a weight wi and cost ci .
The knapsack has a capacity W .
If we fill the knapsack with an object Oi , i.e., xi = 1, then a profit ci is earned.
The objective is to obtain a filling of the knapsack that maximizes the total profit earned.
Since the knapsack capacity is W , we require the total weight of all chosen objects to be
at most W .
Formally, the knapsack problem can be described as follows

n
X
Maximize c = xi ci (1)
i=1

n
X
i=1
n
X
Subject to xi wi ≤ W (2)
i=1

n
X
i=1
n
X
Subject to xi wi ≤ W (2)
i=1
and xi = 0 or xi = 1, wi > 0, 1 ≤ i ≤ n (3)
0-1 Knapsack Problem: Naive Approach
We can easily solve the 0-1 knapsack problem by enumerating all subsets of O =
{O1 , O2 , · · · , On } and selecting the one that has highest total benefit from among all
those with total weight not exceeding W .
T (n) = Θ(2n )
0-1 Knapsack Problem: Dynamic Programming
Step 1: The structure of an optimal solution

ZOKP(k, W ): 0-1 knapsack problem within capacity W for the first k objects.
Goal: ZOKP(n, W ).
OPT(k, W ) is an optimal solution to ZOKP(k, W ).
M[k, W ] = max profit subset of items 1, 2, · · · , k with weight limit W .
We will construct a table M[0 · · · n, 0 · · · W ]. For 1 ≤ k ≤ n, and 0 ≤ w ≤ W , the entry
M[k, w ] will store the maximum (combined) value of any subset of items {O1 , O2 , · · · , Ok } of
(combined) weight at most w .
M[n, W ] will contain the optimal solution.
Step 2: Recursively define the value of an optimal solution

Case 1: Optimal solution does not select object Ok .
Optimal solution selects best of {O1 , O2 , · · · , Ok−1 } using weight limit W .
OPT(k, W ) is an optimal solution of ZOKP(k − 1, W )
M[k, W ] = M[k − 1, W ]
Case 2: Optimal solution selects object Ok .
New weight limit: W − wk
Optimal solution selects best of {O1 , O2 , · · · , Ok−1 } using weight limit W − wk .
OPT(k, W ) − {Ok } is an optimal solution of ZOKP(k − 1, W − wk )
M[k, W ] = M[k − 1, W − wk ] + ck
Step 2: Recursively define the value of an optimal solution

Case 1: Optimal solution does not select object Ok .
Optimal solution selects best of {O1 , O2 , · · · , Ok−1 } using weight limit W .
OPT(k, W ) is an optimal solution of ZOKP(k − 1, W )
M[k, W ] = M[k − 1, W ]
Case 2: Optimal solution selects object Ok .
New weight limit: W − wk
Optimal solution selects best of {O1 , O2 , · · · , Ok−1 } using weight limit W − wk .
OPT(k, W ) − {Ok } is an optimal solution of ZOKP(k − 1, W − wk )
M[k, W ] = M[k − 1, W − wk ] + ck

0 if k = 0,


M[k, W ] = M[k − 1, W ] if wk > W ,

max M[k − 1, W ], M[k − 1, W − wk ] + ck
 otherwise
Step 3: Compute the optimal cost in a bottom-up fashion

Solve smaller subproblem first.
Fill up an (n + 1) × (W + 1) table
M[k, w ] 0 1 2 3 ··· w ··· W

0
1
2
..
.
k −1 M[k − 1, w ]
k M[k, w ]
..
.
n

Algorithm: ZOKP(n, w , W ) tabular, bottom-up method
1 Let M[0 · · · n, 0 · · · W ] be a table
2 for w ← 0 to W do
3 M[0, w ] := 0 ▷ w denotes different capacity
4 end
5 for k ← 1 to n do
7 if (wk > w ) then
8 M[k, w ] := M[k − 1, w ]
9 end
10 else
11 M[k, w ] := Max ck + M[k − 1, w − wk ], M[k − 1, w ]
12 end
13 end
14 end
15 Return(M[n, W ])

Algorithm: ZOKP(n, w , W ) tabular, bottom-up method T (n) = O(nW )
1 Let M[0 · · · n, 0 · · · W ] be a table
3 M[0, w ] := 0 ▷ w denotes different capacity
4 end
5 for k ← 1 to n do
7 if (wk > w ) then
8 M[k, w ] := M[k − 1, w ]
9 end
10 else
11 M[k, w ] := Max ck + M[k − 1, w − wk ], M[k − 1, w ]
12 end
13 end
14 end
15 Return(M[n, W ])
Step 4: Constructing an optimal solution

Algorithm: Solution-ZOKP(M, n, W )
1 S=ϕ
2 w := W
3 for k ← n to 1 do
4 if (M[k, w ] > M[k − 1, w ] then
5 w := w − wk
6 S := S ∪ {Ok }
7 end
8 end
9 Return(S)
T (n) = Θ(n)
0-1 Knapsack Problem: An Example
Example 1
Find the solution of the “0-1 Knapsack problem” using the Dynamic Programming approach
by considering the following instance: (i) n = 4, (ii) W = 8,
Object (O) O1 O2 O3 O4
Cost (ci ) 1 2 5 6
Weight (wi ) 2 3 4 5
M[k, w ] 0 1 2 3 4 5 6 7 8
0
1
2
3
4
For k = 0, and w = 0, 1, · · · , W , M[0, w ] = 0

For w = 0, and k = 1, 2, · · · , n, M[k, 0] = 0
M[k, w ] 0 1 2 3 4 5 6 7 8
0 0 0 0 0 0 0 0 0 0
1 0
2 0
3 0
4 0
For k = 1, w = 1, w1 = 2, w1 > w ,
M[1, 1] = M[0, 1]
= 0
For k = 1, c1 = 1, w = 2, w1 = 2, w1 = w ,

M[1, 2] = Max c1 + M[0, 0], M[0, 2]

= Max c1 + 0, 0
= 1
For k = 1, c1 = 1, w = 3, w1 = 2, w1 < w ,

M[1, 3] = Max c1 + M[0, 1], M[0, 3]

= Max c1 + 0, 0
= 1
For k = 1, w = 1, w1 = 2, w1 > w ,
M[k, w ] 0 1 2 3 4 5 6 7 8
M[1, 1] = M[0, 1] 0 0 0 0 0 0 0 0 0 0
= 0 1 0 0 1 1 1 1 1 1 1
For k = 1, c1 = 1, w = 2, w1 = 2, w1 = w , 2 0
3 0
4 0

M[1, 2] = Max c1 + M[0, 0], M[0, 2]

= Max c1 + 0, 0
= 1
For k = 1, c1 = 1, w = 3, w1 = 2, w1 < w ,

M[1, 3] = Max c1 + M[0, 1], M[0, 3]

= Max c1 + 0, 0
= 1
For k = 2, w = 1, w2 = 3, w2 > w ,
M[2, 1] = M[1, 1]
= 0
For k = 2, c2 = 2, w = 2, w2 = 3, w2 > w ,
M[2, 2] = M[1, 2]
= 1
For k = 2, c2 = 2, w = 3, w2 = 3, w2 = w ,

M[2, 3] = Max c2 + M[1, 0], M[1, 3]

= Max 2 + 0, 1
= 2
For k = 2, c2 = 2, w = 4, w2 = 3, w2 < w ,

M[2, 4] = Max c2 + M[1, 1], M[1, 4]

= Max 2 + 0, 1
= 2
For k = 2, c2 = 2, w = 5, w2 = 3, w2 < w ,

M[2, 5] = Max c2 + M[1, 2], M[1, 5]

= Max 2 + 1, 1
= 3
M[k, w ] 0 1 2 3 4 5 6 7 8
0 0 0 0 0 0 0 0 0 0
1 0 0 1 1 1 1 1 1 1
2 0 0 1 2 2 3 3 3 3
3 0
4 0
For k = 3, w = 1, w3 = 4, w3 > w ,
M[3, 1] = M[2, 1]
= 0
For k = 3, c3 = 5, w = 2, w3 = 4, w3 > w ,
M[3, 2] = M[2, 2]
= 1
For k = 3, c3 = 5, w = 3, w3 = 4, w3 > w ,
M[3, 3] = M[2, 3]
= 2
For k = 3, c3 = 5, w = 4, w3 = 4, w3 = w ,

M[3, 4] = Max c3 + M[2, 0], M[2, 4]

= Max 5 + 0, 3
= 5
For k = 3, c3 = 5, w = 5, w3 = 4, w3 < w ,

M[3, 5] = Max c3 + M[2, 1], M[2, 5]

= Max 5 + 0, 3
= 5
For k = 3, c3 = 5, w = 6, w3 = 4, w3 < w ,

M[3, 6] = Max c3 + M[2, 2], M[2, 6]

= Max 5 + 1, 3
= 6
For k = 3, c3 = 5, w = 7, w3 = 4, w3 < w ,

M[3, 7] = Max c3 + M[2, 3], M[2, 7]

= Max 5 + 2, 3
= 7
For k = 3, c3 = 5, w = 8, w3 = 4, w3 < w ,

M[3, 8] = Max c3 + M[2, 4], M[2, 8]

= Max 5 + 2, 3
= 7
M[k, w ] 0 1 2 3 4 5 6 7 8
0 0 0 0 0 0 0 0 0 0
1 0 0 1 1 1 1 1 1 1
2 0 0 1 2 2 3 3 3 3
3 0 0 1 2 5 5 6 7 7
4 0
For k = 4, c4 = 6, w = 1, w4 = 5, w4 > w ,
M[4, 1] = M[3, 1]
= 0
For k = 4, c4 = 6, w = 2, w4 = 5, w4 > w ,
M[4, 2] = M[3, 2]
= 1
For k = 4, c4 = 6, w = 3, w4 = 5, w4 > w ,
M[4, 3] = M[3, 3]
= 2
For k = 4, c4 = 6, w = 4, w4 = 5, w4 > w ,
M[4, 4] = M[3, 4]
= 5
For k = 4, c4 = 6, w = 5, w4 = 5, w4 = w ,

M[4, 5] = Max c4 + M[3, 0], M[3, 5]

= Max 6 + 0, 5
= 6
For k = 4, c4 = 6, w = 6, w4 = 5, w4 < w ,

M[4, 6] = Max c4 + M[3, 1], M[3, 6]

= Max 6 + 0, 6
= 6
For k = 4, c4 = 6, w = 7, w4 = 5, w4 < w ,

M[4, 7] = Max c4 + M[3, 2], M[3, 7]

= Max 6 + 1, 7
= 7
For k = 4, c4 = 6, w = 8, w4 = 5, w4 < w ,

M[4, 8] = Max c4 + M[3, 3], M[3, 8]

= Max 6 + 2, 8
= 8
M[k, w ] 0 1 2 3 4 5 6 7 8
0 0 0 0 0 0 0 0 0 0
1 0 0 1 1 1 1 1 1 1
2 0 0 1 2 3 3 3 3 3
3 0 0 1 2 5 5 6 7 7
4 0 0 1 2 5 6 6 7 8
M[k, w ] 0 1 2 3 4 5 6 7 8
0 0 0 0 0 0 0 0 0 0
1 0 0 1 1 1 1 1 1 1
2 0 0 1 2 3 3 3 3 3
3 0 0 1 2 5 5 6 7 7
4 0 0 1 2 5 6 6 7 8
M[k, w ] 0 1 2 3 4 5 6 7 8
0 0 0 0 0 0 0 0 0 0
1 0 0 1 1 1 1 1 1 1
2 0 0 1 2 3 3 3 3 3
3 0 0 1 2 5 5 6 7 7
4 0 0 1 2 5 6 6 7 8
w = 8 and S = ϕ
k = 4, M[k, w ] = M[4, 8] = 8, and M[k − 1, w ] = M[3, 8] = 7, M[k, w ] > M[k − 1, w ],
S = {O4 }, w := w − w4 = 3.
k = 3, M[k, w ] = M[3, 3] = 2, and M[k − 1, w ] = M[2, 3] = 2, M[k, w ] ≯ M[k − 1, w ],
S = {O4 }, w := 3.
k = 2, M[k, w ] = M[2, 3] = 2, and M[k − 1, w ] = M[1, 3] = 1, M[k, w ] > M[k − 1, w ],
S = {O2 , O4 }, w := w − w2 = 0.
M[k, w ] 0 1 2 3 4 5 6 7 8
0 0 0 0 0 0 0 0 0 0
1 0 0 1 1 1 1 1 1 1
2 0 0 1 2 3 3 3 3 3
3 0 0 1 2 5 5 6 7 7
4 0 0 1 2 5 6 6 7 8
M[k, w ] 0 1 2 3 4 5 6 7 8
0 0 0 0 0 0 0 0 0 0
1 0 0 1 1 1 1 1 1 1
2 0 0 1 2 3 3 3 3 3
3 0 0 1 2 5 5 6 7 7
4 0 0 1 2 5 6 6 7 8
w = 8 and S = ϕ
k = 1, M[k, w ] = M[1, 0] = 0, and M[k − 1, w ] = M[0, 0] = 0, M[k, w ] ≯ M[k − 1, w ],
S = {O2 , O4 }.
Optimal solution x1 = 0, x2 = 1, x3 = 0, x4 = 1
Profit=c2 + c4 = 2 + 6 = 8, Weight = w2 + w4 = 3 + 5 = 8
Suggested Readings
Chapter 15, Thomas H. Cormen, Charles E. Lieserson, Ronald L. Rivest and Clifford Stein,
Introduction to Algorithms, 3rd Edition, MIT Press/McGraw-Hill, 2009.
Chapter 05, E. Harowitz, S. Sahani, and S. Rajasekaran, Fundamentals of Computer
Algorithms, 2nd Edition, University Press, 2008.
Chapter 12, Michael T. Goodrich and Roberto Tamassia, Algorithm Design and Appli-
cations, Wiley, 2014.
Thank You

Algorithms-I CSC 302 (July-Dec2022 IIITK) L43 - L44

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Algorithms-I CSC 302 (July-Dec2022 IIITK) L43 - L44

Uploaded by

Copyright:

Available Formats

Algorithms - I

Department of CSE, IIIT Kalyani

October 21, 2022

We are given n objects and a knapsack.

Formally, the knapsack problem can be described as follows

Formally, the knapsack problem can be described as follows

Formally, the knapsack problem can be described as follows

and xi = 0 or xi = 1, wi > 0, 1 ≤ i ≤ n (3)

Step 1: The structure of an optimal solution

Step 2: Recursively define the value of an optimal solution

Step 2: Recursively define the value of an optimal solution

Step 3: Compute the optimal cost in a bottom-up fashion

M[k, w ] 0 1 2 3 ··· w ··· W

Step 3: Compute the optimal cost in a bottom-up fashion

Step 3: Compute the optimal cost in a bottom-up fashion

Step 4: Constructing an optimal solution

For k = 0, and w = 0, 1, · · · , W , M[0, w ] = 0

You might also like