Lecture 13: The Knapsack Problem

Outline of this Lecture

Introduction of the 0-1 Knapsack Problem.

A dynamic programming solution to this problem.

0-1 Knapsack Problem Informal Description: We have computed data les bytes that we want to store, and we have available of storage. File has size bytes and takes minutes to recompute. We want to avoid as much recomputing as possible, so we want to nd a subset of les to store such that The les have combined size at most . The total computing time of the stored les is as large as possible. We can not store parts of les, it is the whole le or nothing. How should we select the les?

0-1 Knapsack Problem Formal description: Given two numbers

-tuples of positive

  "!$# %     &!'#( and ) 0 , we wish to determine the subset and 1 2 354 768  9@BA (of les to store) that maximizes "F DC%E subject to HG  DCE
Remark: This is an optimization problem. Brute force: Try all

6 !

possible subsets

Question: Any solution better than the brute-force?

3

Recall of the Divide-and-Conquer

1. Partition the problem into subproblems.

2. Solve the subproblems.

3. Combine the solutions to solve the original one.

Remark: If the subproblems are not independent, i.e. subproblems share subsubproblems, then a divideand-conquer algorithm repeatedly solves the common subsubproblems. Thus, it does more work than necessary! Question: Any better solution? YesDynamic programming (DP)!
4

The Idea of Dynamic Programming Dynamic programming is a method for solving optimization problems. The idea: Compute the solutions to the subsub-problems once and store the solutions in a table, so that they can be reused (repeatedly) later. Remark: We trade space for time.

The Idea of Developing a DP Algorithm

Step1: Structure: Characterize the structure of an optimal solution. Decompose the problem into smaller problems, and nd a relation between the structure of the optimal solution of the original problem and the solutions of the smaller problems.

Step2: Principle of Optimality: Recursively dene the value of an optimal solution. Express the solution of the original problem in terms of optimal solutions for smaller problems.

The Idea of Developing a DP Algorithm

Step 3: Bottom-up computation: Compute the value of an optimal solution in a bottom-up fashion by using a table structure.

Step 4: Construction of optimal solution: Construct an optimal solution from computed information.

Steps 3 and 4 may often be combined.

Remarks on the Dynamic Programming Approach

Steps 1-3 form the basis of a dynamic-programming solution to a problem.

Step 4 can be omitted if only the value of an optimal solution is required.

Developing a DP Algorithm for Knapsack

Step 1: Decompose the problem into smaller problems. We construct an array I P QRS Q . 4 G G , and 0 G G , the entry For I PUV@ T will store the maximum (combined) 354 computing time of any subset of les 768   @WA of (combined) size at most . If we can compute all the entries of this array, then T will contain the maximum the array entry I PUS computing time of les that can t into the storage, that is, the solution to our problem.

0 T

Developing a DP Algorithm for Knapsack Step 2: Recursively dene the value of an optimal solution in terms of solutions to smaller problems.

Initial Settings: Set

THY 0 I P0 X S I P`V T Y a b

for for

0 G G c 0,

no item illegal

Recursive Step: Use

e fih I UP a 4  I PUV@ TSY d g  T @ @p I PU a 4 @  a TFq 0 4 for G G , G G .

10

Correctness of the Method for Computing Lemma: For

4 G G ,0 G G , r T Y d e g i f h a T a a T q 4 4 I PUV@ I PU   p I PU X   T Proof: To compute I P`V we note that we have only two choices for le :
Leave le : The best we can do with les 354 768   @ a 4 A and storage limit is I

I PUV@ T

a T 4 UP V  .

Take le (only possible if s G ): Then we gain of computing time, but have spent bytes of our storage. The best we can do with remaining 3t4 76u   a 4 A and storage h a q is les I PU a 4  a & T . a 4 X a & T . Totally, we get v p I PU

, then " p I PU Note that if so the lemma is correct in any case.

a 4 V a & THY a b
11

Developing a DP Algorithm for Knapsack Step 3: Bottom-up computing I not recursion). Bottom:

PUV T

(using iteration,

I P0   TrY 0

for all

0 G G

Bottom-up computation: Computing the table using

I P`V TrY d ewfih I P` a 4  T X xp I P` a 4  a TFq

V[i,w] w=0 i= 0 0 1 2 . . . n up
12

row by row.
1 0 2 0 3 0 ... ... ... W ... 0

bottom

Example of the Bottom-up computation

Y 0 4 Let

and

y D 0
1 2 3 4 0 0 0 0 1 0 0 0 0 0

" &
2 0 0 0 0 0

1 10 5
3 0 0 0 0 50

2 3 4 40 30 50 4 6 3
4 0 0 40 40 50 5 0 10 40 40 50 6 0 10 40 40 50 7 0 10 40 40 90 8 0 10 40 40 90 9 0 10 50 50 90 10 0 10 50 70 90

Remarks:

The nal output is

y  g

The method described does not tell which subset gives the in this example). optimal solution. (It is

13

The Dynamic Programming Algorithm

KnapSack(

d ! fe ) y ghi ; for ( to e ) for (  to ! ) for ( to e ) if (y Q'j ) Dk l mon y wp qhr Qts y w p qh p QQ ; else y Dk ys wp qh ; y ! ue ; return v g w h q.

Time complexity: Clearly,

14

Constructing the Optimal Solution

described in the previous slide does not keep record of which subset of items gives the optimal solution.

T The algorithm for computing I PUV

To compute the actual subset, we can add an T auxiliary boolean array x5yy@zBPUV{ which is 1 if we T decide to take the -th le in I P`V and 0 otherwise.

Question: How do we use all the values x5yy@zBP`V( to 1 determine the subset of les having the maximum computing time?

15

Constructing the Optimal Solution Question: How do we use the values x5yqy@z|PUV7 to 1 determine the subset of items having the maximum computing time? If keep[ S ] is 1, then a this argument for keep[ If keep[ r ] is 0, the } ~ a 4  ]. ment for keep[

} 4 1 . a We can now repeat ! ]. 1 

and we repeat the argu-

Therefore, the following partial program will output the 1 elements of :

 Y ; Y downto for ( 1)  H T Y Y 4 if (keep ` P V   Y  a PU T ;

output i;

16

The Complete Algorithm for the Knapsack Problem KnapSack(

d ! fe ) y wh ; for ( to e ) for (  to ! ) for ( to e ) Qts y g QQ y w Q j p  q h p p qh )) if (( ) and ( g ys Dh Qts y gp qh Q Q ; p D  ; keep v else g ys Dh y wp qh ; D ; keep v e ; for ( ! downto 1) if g (keep D t  ) output i; p Q ; v y ! e ; return
17