Professional Documents
Culture Documents
CSCE 310J
Data Structures & Algorithms
Giving credit where credit is due:
Dynamic programming
0-1 Knapsack problem
http://www.cse.unl.edu/~goddard/Courses/CSCE310J
Dynamic Programming
Dynamic programming
Basic idea:
Variations:
Item #
1
2
3
Weight
1
3
5
Value
8
6
5
6
Knapsack problem
Items
This is a knapsack
Max weight: W = 20
W = 20
wi
Benefit value
bi
10
max
iT
bi subject to
iT
wi W
10
Can we do better?
Yes, with an algorithm based on dynamic
programming
We need to carefully identify the subproblems
12
Defining a Subproblem
Defining a Subproblem
w1 =2 w2 =4
b1 =3 b2 =5
w3 =5
b3 =8
w1 =2 w2 =4
b1 =3 b2 =5
w5 =9
b5 =10
For S5:
Total weight: 20
Maximum benefit: 26
13
Defining a Subproblem
(continued)
w3 =5
b3 =8
wi
bi
10
Item
Max weight: W = 20
For S4:
Total weight: 14;
Maximum benefit: 20
Weight Benefit
w4 =3
b4 =4
S4
S5
Solution for S4 is
not part of the
solution for S5!!!
15
Recursive Formula
B[ k , w ] =
14
16
B[ k 1, w ]
if wk > w
max{ B[ k 1, w], B[ k 1, w wk ] + bk } else
18
Running time
Example
for w = 0 to W
O(W)
B[0,w] = 0
for i = 1 to n
B[i,0] = 0
Repeat n
for i = 1 to n
for w = 0 to W
O(W)
< the rest of the code >
n = 4 (# of elements)
W = 5 (max weight)
Elements (weight, benefit):
(2,3), (3,4), (4,5), (5,6)
19
Example (2)
i\W 0
0
1
0
20
Example (3)
2
0
3
0
4
0
i\W 0
0
5
0
for w = 0 to W
B[0,w] = 0
1
0
2
0
3
0
4
0
5
0
for i = 1 to n
B[i,0] = 0
21
Example (4)
i\W 0
0
1
0
2
0
3
0
4
0
5
0
Items:
1: (2,3)
2: (3,4)
3: (4,5)
4: (5,6)
22
Example (5)
i=1
bi=3
wi=2
w=1
w-wi =-1
i\W 0
0
1
0
2
0
3
0
4
0
5
0
Items:
1: (2,3)
2: (3,4)
3: (4,5)
4: (5,6)
i=1
bi=3
wi=2
w=2
w-wi =0
24
Example (6)
i\W 0
0
1
0
2
0
3
0
4
0
5
0
Items:
1: (2,3)
2: (3,4)
3: (4,5)
4: (5,6)
Example (7)
i=1
bi=3
wi=2
w=3
w-wi =1
i\W 0
0
1
0
2
0
3
0
4
0
5
0
Items:
1: (2,3)
2: (3,4)
3: (4,5)
4: (5,6)
i=1
bi=3
wi=2
w=4
w-wi =2
Example (8)
i\W 0
0
1
0
2
0
3
0
4
0
5
0
Items:
1: (2,3)
2: (3,4)
3: (4,5)
4: (5,6)
26
Example (9)
i\W 0
0
i=1
bi=3
wi=2
w=5
w-wi =3
1
0
2
0
3
0
4
0
5
0
Items:
1: (2,3)
2: (3,4)
3: (4,5)
4: (5,6)
i=2
bi=4
wi=3
w=1
w-wi =-2
Example (10)
i\W 0
0
1
0
2
0
3
0
4
0
5
0
Items:
1: (2,3)
2: (3,4)
3: (4,5)
4: (5,6)
28
Example (11)
i=2
bi=4
wi=3
w=2
w-wi =-1
i\W 0
0
1
0
2
0
3
0
4
0
5
0
Items:
1: (2,3)
2: (3,4)
3: (4,5)
4: (5,6)
i=2
bi=4
wi=3
w=3
w-wi =0
30
Example (12)
i\W 0
0
1
0
2
0
3
0
4
0
5
0
3
Items:
1: (2,3)
2: (3,4)
3: (4,5)
4: (5,6)
Example (13)
i=2
bi=4
wi=3
w=4
w-wi =1
i\W 0
0
1
0
2
0
3
0
4
0
5
0
Items:
1: (2,3)
2: (3,4)
3: (4,5)
4: (5,6)
i=2
bi=4
wi=3
w=5
w-wi =2
Example (14)
i\W 0
0
1
0
2
0
3
0
4
0
5
0
Items:
1: (2,3)
2: (3,4)
3: (4,5)
4: (5,6)
32
Example (15)
i\W 0
0
i=3
bi=5
wi=4
w= 1..3
1
0
2
0
3
0
4
0
5
0
Items:
1: (2,3)
2: (3,4)
3: (4,5)
4: (5,6)
i=3
bi=5
wi=4
w= 4
w- wi=0
Example (16)
i\W 0
0
1
0
2
0
3
0
4
0
5
0
Items:
1: (2,3)
2: (3,4)
3: (4,5)
4: (5,6)
34
Example (17)
i=3
bi=5
wi=4
w= 5
w- wi=1
i\W 0
0
1
0
2
0
3
0
4
0
5
0
Items:
1: (2,3)
2: (3,4)
3: (4,5)
4: (5,6)
i=4
bi=6
wi=5
w= 1..4
36
Example (18)
i\W 0
0
1
0
2
0
3
0
4
0
5
0
Items:
1: (2,3)
2: (3,4)
3: (4,5)
4: (5,6)
Comments
This algorithm only finds the max possible value
that can be carried in the knapsack
i=4
bi=6
wi=5
w= 5
w- wi=0
38
i\W 0
0
1
0
2
0
3
0
4
0
5
0
Items:
1: (2,3)
2: (3,4)
3: (4,5)
4: (5,6)
i=4
k= 5
bi=6
wi=5
B[i,k] = 7
B[i1,k] =7
i=n, k=W
while i,k > 0
if B[i,k] B[i1,k] then
mark the ith item as in the knapsack
i = i1, k = k-wi
else
i = i1
39
1
0
2
0
3
0
4
0
5
0
Items:
1: (2,3)
2: (3,4)
3: (4,5)
4: (5,6)
40
i=4
k= 5
bi=6
wi=5
B[i,k] = 7
B[i1,k] =7
i\W 0
0
1
0
2
0
3
0
4
0
5
0
i=n, k=W
while i,k > 0
if B[i,k] B[i1,k] then
Items:
1: (2,3)
2: (3,4)
3: (4,5)
4: (5,6)
i=3
k= 5
bi=6
wi=4
B[i,k] = 7
B[i1,k] =7
i=n, k=W
while i,k > 0
if B[i,k] B[i1,k] then
else
else
i = i1
41
i = i1
42
Items:
1: (2,3)
2: (3,4)
3: (4,5)
4: (5,6)
1
0
2
0
3
0
4
0
5
0
i=n, k=W
while i,k > 0
if B[i,k] B[i1,k] then
i=2
k= 5
bi=4
wi=3
B[i,k] = 7
B[i1,k] =3
k wi=2
i\W 0
0
1
0
2
0
3
0
4
0
5
0
i=n, k=W
while i,k > 0
if B[i,k] B[i1,k] then
i = i1
43
i=1
k= 2
bi=3
wi=2
B[i,k] = 3
B[i1,k] =0
k wi=0
else
Items:
1: (2,3)
2: (3,4)
3: (4,5)
4: (5,6)
1
0
2
0
3
0
4
0
5
0
i=0
k= 0
Items:
1: (2,3)
2: (3,4)
3: (4,5)
4: (5,6)
44
1
0
2
0
3
0
4
0
5
0
i=n, k=W
while i,k > 0
if B[i,k] B[i1,k] then
Items:
1: (2,3)
2: (3,4)
3: (4,5)
4: (5,6)
The optimal
knapsack
should contain
{1, 2}
i=n, k=W
while i,k > 0
if B[i,k] B[i1,k] then
The optimal
knapsack
should contain
{1, 2}
else
else
i = i1
45
i = i1
46
47
48
Memoization
50
Conclusion
Dynamic programming is a useful technique of
solving certain kind of problems
When the solution can be recursively described in
terms of partial solutions, we can store these
partial solutions and re-use them as necessary
(memoization)
Running time of dynamic programming algorithm
vs. nave algorithm:
0-1 Knapsack problem: O(W*n) vs. O(2n)
51