National Engineering School of Tunis

Discrete
Optimization
Course notes for 2AGI class: Industrial Engineering

Atidel B. Hadj-Alouane
15/09/2016
OPTIMIZATION 2AGI

TABLE OF CONTENTS

Chapter 1 Introduction to Integer Programming ..................................................................... 3

1.1 Introduction ........................................................................................................................................................ 3

1.2 Basic concepts and definitions .................................................................................................................... 4

1.3 Example................................................................................................................................................................. 6

1.4 Concept of unimodularity .............................................................................................................................. 7

Chapter 2 Integer programming Formulations ...................................................................... 12

2.1 The capital budgeting problem ............................................................................................................... 12

2.2 Fixed-Charge Problems ............................................................................................................................... 15

2.3 The set covering problem ........................................................................................................................... 17

2.4 Other types of formulations ....................................................................................................................... 19

Chapter 3 The Branch-and-Bound Approach .......................................................................... 21

3.1 Introduction ..................................................................................................................................................... 21

3.2 The concepts of branching and bounding ........................................................................................... 21

3.3 Branch-and-Bound based on linear programming ......................................................................... 23

3.4 Branch-and-Bound for scheduling ......................................................................................................... 28

Chapter 4 Dynamic Programming ............................................................................................... 32

4.1 Introduction ..................................................................................................................................................... 32

4.2 Dynamic Programming formulations ................................................................................................... 33

4.3 Characteristics of dynamic programming applications ............................................................... 34

4.4 Production and inventory control .......................................................................................................... 35

4.5 Probabilistic Inventory model .................................................................................................................. 42

4.6 Equipment replacement.............................................................................................................................. 45

4.7 Summary of general aspects of dynamic programming ............................................................... 49

2
OPTIMIZATION 2AGI

Chapter 1 Introduction to Integer Programming

1.1 Introduction
Linear programming allows for modeling a variety of real optimization problems by
expressing the related objective functions and constraints in terms of decision variables.
However, linear programming imposes certain assumptions that restrict its application
range. These assumptions are the following:

Proportionality: implies that cost terms and resource utilization are proportional to the
associated activity level.
Additivity: implies that the total consumption (or production) of an item is equal to the
sum of the various quantities of the item consumed (or produced) in carrying out each
individual activity at its specified level. The proportionality and the additivity assumptions
together are known as the linearity assumptions.
Divisibility: It is assumed that each variable in the model can take all the real values in its
range of variation.
Therefore, and due to divisibility assumption, linear programming does not handle
problem where decision variables represent indivisible entities such as an individual, a
building or a project. The class of mathematical programming where the linearity
assumption (proportionality and additivity) but allows for variables to be integers is called
Integer Programming (IP).

Applications of IP : Industrial and operations planning, Economics, Finance, …

• Assignment and scheduling
• Production planning
• Facility location
• Transportation and distribution
• Project selection
• Portfolio optimization, …

3
OPTIMIZATION 2AGI

1.2 Basic concepts and definitions

Definitions:
A linear program where all decision variables have to be integers is called a Pure Integer
Program or simply an Integer Program (IP).

A pure integer program can be written as follows:

Max cx
(IP) s. to Ax = b
x≥0
x integral
where A is (m x n) matrix (m ≤ n), and b ∈ ℜm.

Example 1.1
max z = 3 x1 + 2 x2
s.to x1 + x2 ≤ 6
x1, x2 ≥ 0
x 1 , x 2 integers

• A Linear program where only some decision variables have to be integers is called a
Mixed Integer Program.
• A Linear program where all decision variables have to be binary (0 or 1) is called a
Binary Integer Program or 0-1 Integer Program:
Max cx
(BIP) s. to Ax = b
x ∈ {0,1}

The Linear relaxation of an integer program (mixed or binary) consists of relaxing the
integrality constraints in order to obtain a linear (continuous) program.

The linear relaxation of (IP) gives the following linear program ( IP ):

Max cx

( IP ) s. to Ax = b
x≥0

4
OPTIMIZATION 2AGI

The linear relaxation of the binary integer program (BIP) is the following:
Max cx

( BIP ) s. to Ax = b
0≤ x ≤ 1
Notes :
• An integer program can be viewed as a linear program with added constraints (i.e.
integrality constraints). As a consequence, the feasible domain of IP is contained in

the feasible domain of IP :

D(IP) ⊂ D( IP ),

where D(IP) and D(LP) are the feasible domains of IP and IP , respectively.

• For any maximization program (IP), the optimal objective value of the relaxed

problem ( IP ) is larger than or equal to the optimal objective value of the integer
program (IP) :

Z(IP) ≤ Z( IP ) (1.1)

where Z(IP) and Z( IP ) are the optimal objective values of IP and IP , respectively. The
difference Z( IP )-Z(IP) is called the duality gap.

At this point, two major questions are raised:

Q1: Can we use linear programming algorithms (Simplex or interior point methods) to

solve integer programs? When can we say that an optimal solution to IP is also optimal to
IP?
Q2: If linear programming algorithms give non integral solutions, how can we solve integer
programs?
The application of linear programming algorithms may sometimes give an integral solution
which, in this case, would also be optimal to the integer program. In this case, there is no
duality gap (see the inequality (1.1)). Unfortunately, this is rarely the case as linear
programming solutions are usually fractional.
Rounding off fractional solutions to the nearest integer might work for some cases.
However, for most problems, it can result in solutions that are far from the optimum or
even non feasible. Consider the problem where an automobile manufacturer who needs to

5
OPTIMIZATION 2AGI

determine the number of cars to produce in a given year. If the solution of the
corresponding linear program is such that 150,000.4 cars are to be produced, we can
safely round off to 150,000.

However, for a problem where one has to decide whether or not to build an Electrical Plant
(binary variables), rounding off a fractional solution of 0.4 has no sense. It would make a
big difference whether the solution is rounded down to 0 or up to 1. Therefore, in this case,
rounding off is not appropriate.

1.3 Example
Consider the following integer program:
Max z = 21 x 1 + 11 x 2
S. to 7 x 1 + 4 x 2 ≤ 13
x1, x2 ≥ 0
x 1 , x 2 integers

x2 Optimal integral Solution

(0, 3) with z* = 33
3,0

2,5

2,0
7 x1 + 4 x2 = 13
1,5
Feasible Domain of IP
1,0

Optimal continuous solution

0,5
(13/7, 0) with z* = 39

0,5 1,0 1,5 2,0 x1

Figure 1.1 Geometric representation of an integer/linear program

According to figure 1.1, the feasible solutions to the problem are given by the set:
S = {(0, 0), (0, 1), (0, 2), (0, 3), (1, 0), (1, 1)},
and the optimal integer solution is (0,3) with z*=33. if we had to round off the optimal
continuous solution (13/7,0) we would obtain:

6
OPTIMIZATION 2AGI

− Round up: (2,0) infeasible

− Round down: (1,0) feasible but far from optimal (z(1,0) = 21 !)
Notes:
1. Contrary to linear programs, the feasible domain of an integer program is not
convex.

2. By computing and comparing the objective values, z, for each feasible solution in S,
we obtain the optimal solution (0, 3) for which z = 33. This is possible whenever the
feasible domain is bounded (or finite).

3. If the feasible domain of an IP is bounded, then it contains a finite number of feasible

solutions. In theory, such an IP could be solved by enumerating all points and
choosing the best. However, in practice, most integer programs have a prohibitively
large number of feasible solutions. Therefore, enumerating all possible solutions
may require a very large computing time.

4. For some integer programs, all round offs are infeasible. Consider the following
example:
Max z = 4 x 1 + x 2
S. to 2 x1 + x2 ≤ 5
2 x 1 + 3x 2 = 5
x1, x2 ≥ 0
x 1 , x 2 integers
The continuous optimal solution is (5/2,0). Rounding up gives (3,0) and rounding down
gives (2,0); and both solutions are infeasible!

In conclusion, even though the feasible domain of an IP is included in the feasible domain of
its corresponding linear relaxation, solving the IP is generally much more difficult than
solving the linear program. However, one can still raise the following question: When do
we expect to efficiently solve an IP? This question is addressed in the following section.

1.4 Concept of unimodularity

Consider the following linear program in the standard form:
Max cx
s. to Ax = b
x≥0

7
OPTIMIZATION 2AGI

Assume rank(A) = m. An extreme point of the polyhedral representing the feasible domain
of the linear program can be found by rewriting Ax = b in terms of a basis B:
B x B + N x N = b,
where B is mxm nonsingular and N is called the non basic matrix. Then, the extreme point
solution is x B = B-1 b; x N = 0. It is known in linear programming that if an optimal solution
exists there must be at least one extreme point that is optimal. It is therefore, interesting to
characterise the data of an integer program such that all extreme points are integral.
From the expression x B = B-1 b, it is easy to notice that if all elements of B-1 are integers and
b is an integral vector, than x B must be integral. The following results formalize this
observation.
Definition: a matrix A is totally unimodular ⇔ each square submatrix, B, of A has a
determinant equal to 0, +1 or –1.
Consequence
This implies that the elements of a totally unimodular matrix are themselves equal to 0, +1
or –1, since they are the determinants of submatrices of order 1.
The following theorem gives a sufficient condition related to the integrality of basic
solutions on an IP.
Theorem 1.1: [Hoffman-Kruskal, 1956]
Consider the linear program {min cx subject to: Ax = b}. If b is integral and A is totally
unimodular, then all basic solutions to this LP are integral.
Example 1.2
Max z = c1 x1 + c2 x2
S. to x1 + x2 ≤ b
x1, x2 ≥ 0
x 1 , x 2 integer

The matrix A = (1 1) is totally unimodular. If, in addition, b is integer than all extreme
points of the relaxed problem are integral.

8
OPTIMIZATION 2AGI

x2
2,0
x1 + x2 = b
1,5
b =1
1,0 b =2

0,5

0,5 1,0 1,5 2,0 2,5 3,0 x1

Figure 1.2 Extreme points for Example 1.2

Example 1.3
Max z = 4 x1 + 6 x2
S. to x1 + 2 x2 ≤ b
x1, x2 ≥ 0
x 1 , x 2 integer

If b = 3, the optimal solution of the relaxed problem is (0 ; 3/2).

If b = 2, all extreme points of the relaxed problem are integral even though the matrix
A = (1 2) is not totally unimodular.

x2
2,0
x1 + 2 x2 = b
1,5
b =1
1,0 b =2
b =3
0,5

0,5 1,0 1,5 2,0 2,5 3,0 x1

Figure 1.3 Extreme points for Example 1.3

9
OPTIMIZATION 2AGI

How do we know if A is totally unimodular? The following theorem gives sufficient (but not
necessary) conditions for the total unimodularity of A.
Theorem 1.2: (Hoffman-Gale)
An integral matrix A which elements are 0 or ±1 is totally unimodular if:
• No more than two nonzero elements appear in each column.
• The rows can be partitioned into 2 sets, R 1, R 2, such that:
- If a column contains two nonzero elements of the same sign, they are in the
opposite sets.
- If a column has two elements of different sign, they are in the same set.

Proof: As exercise (by induction)

Example 1.4 The transportation problem
In the classical transportation problem, there are p (i = 1, …p) supply sites and q (j = 1, …q)
demand sites. A product is shipped from the supply sites to the demand sites. The shipment
of a unit from the supply site i to demand site j costs c ij. The problem is to determine the
quantities to ship from each supply site to every demand site such as total shipment cost is
minimized while satisfying the following two constraint types:
- Supply capacity: the quantity shipped out of a supply site i cannot exceed its
capacity, a i units.
- Demand satisfaction: The quantity shipped to a demand site j has to cover its
demand of b j units.
Let x ij be the number of units shipped from i to j; i = 1, …, p and j = 1, …, q. The problem is
then formulated as follows:
p q
min z = ∑∑ cij x ij
i =1 j =1

q
S. to ∑x
j =1
ij = ai i = 1, …, p

∑x
i =1
ij = bj j = 1, …, q

x ij ≥ 0 i = 1, …, p j = 1, …, q
where:

10
OPTIMIZATION 2AGI

p q

∑a
i =1
i = ∑b
j =1
j : balance condition

c ij ≥ 0 i = 1, …, p j = 1, …, q
ai > 0 i = 1, …, p
bj > 0 j = 1, …, q

For a problem with 2 supply sites and 3 demand sites, the constraints are written as
follows:
x 11 + x 12 + x 13 = a1
x 21 + x 22 + x 23 = a2
x 11 + x 21 = b1
x 12 + x 22 = b2
x 13 + x 23 = b3
The constraint matrix, A, of the integer program is the following:

Basis x 11 x 12 x 13 x 21 x 22 x 23 Solution
Z c 11 c 12 c 13 c 21 c 22 c 23 0
1 1 1 0 0 0 a1
Supply sites (R1)
0 0 0 1 1 1 a2
1 0 0 1 0 0 b1
Demand sites (R2)
Redundant 0 1 0 0 1 0 b2
0 0 1 0 0 1 b3

The matrix A is totally unimodular. Hence, as long as a i and b j are integers for all i and j, the
optimum solution of the transportation problem obtained by the Simplex algorithm will be
an integer solution. This result makes integer programs of the transportation problem easy
to solve.

11
OPTIMIZATION 2AGI

Chapter 2 Integer programming Formulations

Introduction

This chapter shows, through practical examples, how to formulate real problems as integer
programs. It starts with simple and common problems, then builds up to more complicated
formulations.
Similarly to linear programming, formulating integer programs involves the following
three steps:
1. Define the decision variables;
2. Write the objective function in terms of the decision variables;
3. Write the constraints in terms of the decision variables.

2.1 The capital budgeting problem

Consider an investment selection problem where 4 possible projects can be selected. Each
project requires an initial cash outflow and yields a certain net present value (NPV), given
in Table 2.1 below:

Table 2.1 Data for an example of a capital budgeting problem

Project 1 2 3 4
Initial investment ($) 5000 7000 4000 3000
NPV ($) 16000 22000 12000 8000

Currently, a budget of $14 OOO can be made available for these investments. The problem
consists of selecting, among these 4 projects, those that maximize the total yield (NPV)
while satisfying the budget constraint.

1. Decision variables :

Define a binary variable for each project:

xi (i = 1, 2, 3, 4) = 1 if project i is selected
= 0 otherwise.

12
OPTIMIZATION 2AGI

2. Objective function:

The objective is to maximize the total yield:

z (x) = 16000 x 1 + 22000 x 2 + 12000 x 3 + 8000 x 4
For example, if x 1 = 1, the objective function will include the yield of related to project 1
($16 000), otherwise (x 1 = 0), project 1 will not contribute in the total yield. Therefore, for
any decision, the objective function will include only the yields related to selected projects.

3. Constraints :

Since the selection of a project requires an initial investment, the total investment cannot
exceed the available budget of $14 000. This is translated by the following linear
constraint:
5000 x 1 + 7000 x 2 + 4000 x 3 + 3000 x 4 ≤ 14000
For example, if the decision is to select projects 1, 3 and 4 (x 2 = 0 and x 1 = x 3 = x 4 = 1), the
total investment is 5000 + 4000 + 3000 = 12 000 ≤ 14 000 (feasible solution).
The resulting model is a binary integer program :

(BIP1) Max z(x) = 16000 x 1 + 22000 x 2 + 12000 x 3 + 8000 x 4

S. to 5 x 1 + 7 x 2 + 4 x 3 + 3 x 4 ≤ 14
x i = 0 or 1 (i = 1, 2, 3, 4)
Solving BIP1 (for example by complete enumeration) gives the following optimal solution:
x = (x 1 ,x 2 ,x 3 ,x 4 ) = (0,1,1,1) with z* = $42 000, which implies investing in projects 2, 3 and
4, and not in project 1.
If we compute the profitability of each project, which means the yield per dollar invested,
we find that project 1 is the most profitable (see Table 2.2 below).

Table 2.2 Profitability per project

Profitability
Project 1 3.2
Project 2 3.14
Project 3 3
Project 4 2.67

13
OPTIMIZATION 2AGI

Why does the optimal solution exclude the most profitable project?
Notice that combining project 1 with any other project (projects 1 and 2 or projects 1, 3
and 4) does not require more than $12000 as initial investment. This means that $2000
will remain non invested; Weheras selecting projects 2, 3 and 4 allows for a full use of the
budget and yields the maximum NPV.

Note:
An integer program of the same type as (BIP.1) is called the Knapsack problem (an IP in
which there is only a single constraint on the variables, other than the nonnegative integer
requirement). This problem relates to the situation where a camper has to fill a knapsack,
of a given capacity, with some useful items in order to go to camp. The question is what
should he put in the knapsack so as to maximize his satisfaction function without exceeding
the knapsack capacity?

Additional Constraints:
Suppose that for some practical reasons, the following restrictions are imposed:
• At most two projects can be selected.
• If project 2 is selected then project 1 has to be selected.
• If project 2 is selected than project 4 cannot be selected.

The formulation (BIP.1) should be modified by adding the following constraints.

Constraint (1): Simply add x 1 + x 2 + x 3 + x 4 ≤ 2

Constraint (2): Add x 2 ≤ x 1

This constraint implies that :
- if x 2 = 1, the constraint becomes 1 ≤ x 1
since x 1 = 0 or 1, then x 1 = 1
- if x 2 = 0, the constraint becomes 0 ≤ x 1 ; this allows x 1 to be 0 or 1.

Constraint (3): x 2 + x 4 ≤ 1
This constraint implies that either project 2 or project 4 can be selected, but not both. In
fact, we have:

14
OPTIMIZATION 2AGI

- if x 2 = 1, then 1 + x 4 ≤ 1 or x 4 ≤ 0
since x 4 = 0 ou 1, then x 4 = 0
- if x 2 = 0, then x 4 ≤ 1 , which implies no further restriction on the value of x 4 .

2.2 Fixed-Charge Problems

These problems involve a fixed cost that is associated with carrying out an activity
independently of its level. Examples facility location, machine set-up, equipment rentals, …
Consider the case of a clothing company that can fabricate 3 types of clothings: shirts,
shorts and pants. The machinery needed to manufacture each type of item must be rented
at a weekly price given on Table 2.3. The same table also shows the production costs, the
sales prices, the required labor and raw material (cloth) per unit.

Table 2.3 Data for the fixed charge example

Rental Production Sales Labor/unit Cloth

price/week ($) cost/unit ($) price/unit ($) (hours) (m2)

Shirts 200 6 12 3 4
Shorts 150 4 8 2 3
Pants 100 8 15 6 4

Each week, there are 150 hours of labor and 160 m2 of cloth available for production.
The problem is to determine the weekly production level for each item such that total
weekly profit is maximized.
1. Decision variables:
Define, for each type of clothing, an integer variable that denotes the number of unit to
produce per week.
x 1 : the number of shirts to produce per week.
x 2 : the number of shorts to produce per week.
x 3 : the number of pants to produce per week.

We notice that the machine rental cost depends on the clothing type to produce and not on
the quantity produced. Therefore, we define a decision variable that denotes whether or
not to produce a clothing type:

15
OPTIMIZATION 2AGI

y1 =1 if x 1 > 0 (if shirts are produced),

=0 otherwise,
y2 =1 if x 2 > 0 (if shorts are produced),
=0 otherwise,
y3 =1 if x 3 > 0 (if pants are produced),
=0 otherwise.
2. Objective function:
The company is interested in maximizing the weekly profit which can be written as follows:
(Weekly profit) = (weekly sales revenue) – (weekly variable costs)
- (weekly machine rental costs: fixed costs)

(Weekly sales revenue) = 12 x 1 + 8 x 2 + 15 x 3

(Weekly variable costs) = 6 x1 + 4 x2 + 8 x3
(Weekly machine rental costs) = 200 y 1 + 150 y 2 + 100 y 3
⇒ Maximize z = 6 x 1 + 4 x 2 + 7 x 3 - 200 y 1 - 150 y 2 - 100 y 3

3. Constraints:
The company is faced to two types of constraints:
Constraint 1 : at most 150 hours of labor are available each week,
3 x 1 + 2 x 2 + 6 x 3 ≤ 150
Constraint 2 : at most 160 m2 of cloth is available each week,
4 x 1 + 3 x 2 + 4 x 3 ≤ 160
The integer program is written as follows:

(IP.2) Max z = 6 x 1 + 4 x 2 + 7 x 3 - 200 y 1 - 150 y 2 - 100 y 3

S.to 3 x 1 + 2 x 2 + 6 x 3 ≤ 150
4 x 1 + 3 x 2 + 4 x 3 ≤ 160
x 1 , x 2 , x 3 ≥ 0 ; x 1 , x 2 , x 3 integers
y 1 , y 2 , y 3 = 0 or 1.

An optimal solution of the above integer program is : x 1 = 30, x 3 = 10, x 2 = y 1 = y 2 = y 3 = 0.

However, this solution is practically wrong since the company cannot produce shirts or

16
OPTIMIZATION 2AGI

pants without paying for machinery rental (the solution indicates x 1 >0 whereas y 1 =0 !).
This is because the formulation is incomplete. We need to specify the relationship betweent
x i and y i , otherwise nothing would stop us from setting y 1 = y 2 = y 3 = 0 (setting y i = 0 costs
less than y i = 1).
The following constraints have to be added to the formulation:
x1 ≤ M y1
x2 ≤ M y2
x3 ≤ M y3
where M is a very large number.

With the above constraints,

- if x i > 0 then 0< M y i . Since y i = 0 or 1 then we must have y i = 1.
This constraints becomes x i ≤ M, which does not restrict the value of x i .
- if x i = 0 then 0 ≤ M y i . This allows y i to be either 0 or 1, without any restriction. Since
y i = 0 costs less than yi = 1, the optimal solution would be such that y i = 0 if x i = 0.
What is an appropriate value for M ?
M has to be fixed at a value larger than or equal to the maximum value any xi could take. In
the clothing example, we know from the capacity constraints that:
160  160  160 
x1 ≤   ; x2 ≤   and x 3 ≤ 
 4   3   4 
We can safely set M=53 (or larger).

2.3 The set covering problem

This problem belongs to an important class in integer programming, or combinatorial
optimization. Set-covering involves a given set (Set 1) that has to be covered by elements of
another set (Set 2). The objectif is to cover the set 1 by a minimum number of elements
from Set 2. The set covering problem arises in many applications such as crew scheduling,
and public facility planning.

Consider the following example. There are six cities (cities 1-6) in Washtenaw County. The
county must determine where to build fire stations. The county wants to build the
minimum number of fire stations needed to ensure that at least one fire station is within 15

17
OPTIMIZATION 2AGI

minute (driving time) of each city. The times required to drive between the cities in
Washtenaw County are shown in Table 2.4.
The problem consists of finding how many fire stations to build and where to build them.

Table 2.4 Travel times (in minutes) between every pair of cities

From / To City 1 City 2 City 3 City 4 City 5 City 6

City 1 0 10 20 30 30 20
City 2 10 0 25 35 20 10
City 3 20 25 0 15 30 20
City 4 30 35 15 0 15 25
City 5 30 20 30 15 0 14
City 6 20 10 20 25 14 0

1. Decision variables:
At each city, the county has to decide whether or not to build a fire station. Then, define a
0-1 variable for each city i=1, …, 6 :
xi = 1 if a fire station is built at city i,
= 0 otherwise.

2. Objective function:
The number of fire stations has to be minimized:
Min z = x 1 + x 2 + x 3 + x 4 + x 5 + x 6

3. Constraints :
The county has to ensure that every city can be reached by at least one fire station, within
at most 15 minutes. From Table 2.4, we can deduce, for each city, those cities that are
within 15 minutes of driving time. The results are summarized on Table 2.5.

Table 2.5 Cities within 15 minutes of driving time

City 1 City 2 City 3 City 4 City 5 City 6

1; 2 1; 2; 6 3; 4 3; 4; 5 4; 5; 6 2; 5; 6

18
OPTIMIZATION 2AGI

For example, for city 1, the neighboring cities that are within 15 minutes are cities 1 and 2.
The corresponding constraint should state that one fire station has to be built either at city
1 or city 2:
x1 + x2 ≥ 1
The same logic leads to the following constraints for the remaining cities:
x1 + x2 + x6 ≥ 1 : City 2
x3 + x4 ≥ 1 : City 3
x3 + x4 + x5 ≥ 1 : City 4
x4 + x5 + x6 ≥ 1 : City 5
x2 + x5 + x6 ≥ 1 : City 6
The binary integer program for the above problem is:

(BIP.2) Min z = x1 + x2 + x3 + x4 + x5 + x6
S. to x1 + x2 ≥1
x1 + x2 + x6 ≥1
x3 + x4 ≥1
x3 + x4 + x5 ≥1
x4 + x5 + x6 ≥ 1
x2 + x5 + x6 ≥ 1
x i = 0 or 1, for i = 1,…, 6

2.4 Other types of formulations

2.4.1 "Either-Or" constraints (at least)

Consider a problem where at least one of the following two constraints is satisfied:
f(x 1 , x 2 , …., x n ) ≤ 0
g(x 1 , x 2 , …., x n ) ≤ 0
In a mathematical programming formulation, these constraints have to be included but not
necessarily both of them are satisfied. These constraints are called "disjunctive" or of the
"either-or" type.
One way to model this situation is to replace the either-or constraints by the following two
"conjunctive" constraints:

19
OPTIMIZATION 2AGI

f(x 1 , x 2 , …., x n ) ≤ M y ( 2.1)

g(x 1 , x 2 , …., x n ) ≤ M (1 – y) ( 2.2)

where y is a binary variable and M is a sufficiently large number such that f(x 1 , x 2 , …., x n ) ≤
M and g(x 1 , x 2 , …., x n ) ≤ M for any values of x 1 , x 2 , …., x n that satisfy the rest of the
constraints.

• For y = 0, the constraints (2.1) and (2.2) become: f(x) ≤ 0 and g(x) ≤ M and hence
the constraint on f(x) has to be verified but there is no restriction on g(x).
• For y = 1, the constraints become: f(x) ≤ M and g(x) ≤ 0. Therefore, the constraint on
g(x) has to be verified but there is no restriction on f(x).

2.4.2 "If-Then" constraints

In many applications, the following situation arises: We need to ensure that:

If the constraint f(x 1 , x 2 , …., x n ) > 0 is verified then g(x 1 , x 2 , …., x n ) ≤ 0 also has to be
verified. Otherwise g(x 1 , x 2 , …., x n ) ≤ 0 may or may not be verified (i.e. there is no
restriction on g(x)). This type of constraints is called if-then constraints.

To handle this situation, we first note that this condition is equivalent to the previous one :
Either f(x 1 , x 2 , …., x n ) ≤ 0 or g(x 1 , x 2 , …., x n ) ≤ 0. Therefore it can be replaced with the
system:

f(x 1 , x 2 , …., x n ) ≤ M y ( 2.3)

g(x 1 , x 2 , …., x n ) ≤ M (1 – y) ( 2.4)

y = 0 or 1.
Again, M is a sufficiently large number such that f(x 1 , x 2 , …., x n ) ≤ M and g(x 1 , x 2 , …., x n ) ≤ M
for any values of x 1 , x 2 , …., x n that satisfy the rest of the constraints.
• If f(x) > 0 the constraints (2.3) and (2.4) are verified only if y = 1. This implies that
g ≤ 0 ; which is the desired outcome.
• If f(x) > 0 is not verified then y = 0 or 1. In this case, there is no restriction on the
variables x 1 , x 2 , …., x n and g(x) < 0 or g(x) ≥ 0 are both possible.

20
OPTIMIZATION 2AGI

Chapter 3 The Branch-and-Bound Approach

3.1 Introduction
Consider the following IP: {max cx, s.to Ax = b, x ≥ 0 and x integer}. If the constraint matrix
A is not totally unimodular, or there are noninteger right hand sides (b i ) , it is possible for
the linear programming relaxation to have nonintegral extreme points.
In this case, linear programming algorithms, such as Simplex, are not appropriate for
solving the IP.

In this chapter, we present a well-known approach specially developed for solving integer
programs: Branch-and-Bound. It is an-enumeration based approach that consists of
schematically describing the solution set as a search tree, then intelligently searching this
tree to obtain an optimal solution.

3.2 The concepts of branching and bounding

Consider the following program:

max f ( x )


 x ∈D


Where f(x) is any real function of x ∈ ℜ n and D ⊂ ℜ n .

3.2.1 Branching procedure

It consists of breaking up the feasible domain, D, into sections ⇒ subproblems (See Figure
1.1).

D D2

D3 D

Figure 3.1 Breakdown of the feasible domain D into four sections

21
OPTIMIZATION 2AGI

Let fi* be the optimal solution for section i, and f* the optimal solution to the entire

problem. Then f ∗ = max f i ∗ .

i

Suppose one subproblem, say the 2nd, turns out to be difficult as well → break it further
into subproblems.

D21
D D23
D22

D D

Figure 3.2 Further breakdown of section 2 into three subsections

Now f* = max{f 1 *, f 21 *, f 22 *, f 23 *, f 3 *, f 4 *}

Figure 3.3 below shows a tree diagram related to the above problems.

Original Problem

Subproblem 1
Subproblem4
Subproblem 2 Subproblem 3

Subproblem21 Subproblem23
Subproblem22

Figure 3.3 Search tree corresponding to the branching process

22
OPTIMIZATION 2AGI

3.2.2 Bounding procedure

Suppose we found f 1 * and we discovered only an upper bound B 3 for subproblem 3 that is
worse than f 1 *, i.e., f 3 * ≤ B 3 and B 3 ≤ f 1 *. This implies that f 3 * ≤ f 1 * . Therefore,
there is no need to continue solving subproblem 3: The solutions in sub3 have been
implicitly enumerated. The process of evaluating the subproblems in order to compare it
with the best found solution is called "Bounding".

The combination of the breakup and bound computation is called branch-and-bound

(“Séparation et Evaluation Successive”).

Practically, this process of alternately branching and bounding requires finding a "good"
implicit evaluation of f(x) over all subsets D i and rapidely obtaining a "good" feasible
solution of f(x) over D.

As a consequence, when a feasible solution of IP is found, the implicit evaluation allows for
implicitly searching the integer branches of the constructed tree without necessarily
examining all subproblems (also called "nodes").

3.3 Branch-and-Bound based on linear programming

3.3.1 Basic Definitions:

Consider the following integer program:

max c x
(P) s.t. A x ≤ b
x≥0, x integral
A : mxn matrix.
• A branch-and-bound tree is a directed graph with nodes corresponding to a
subproblem of a "father" problem. An arc exists from a father problem to its
immediate offspring.
• An incumbent is the best (integer) solution found so far.

23
OPTIMIZATION 2AGI

• A node (partial solution) that we can prove has no partial completion with a better
objective value than the incumbent is considered completely explored (the node is
cut from the tree).
Note: if we solve a subproblem and find an integer solution better than the incumbent, the
newly discovered solution becomes the new incumbent and therefore a reference for
cutting nodes.
• A node that has been branched from and is not cut or explored is called a dangling
node.

3.3.2 Branch-and-bound algorithms for integer programs

Consider again:
max c x
(P) s.t. A x ≤ b
x≥0, x integral

Bounding: LP relaxation (Simplex)

Branching: Suppose we solved the LP relaxation and we got x 1 = 4.33. We could create two
subproblems to force:

(Sub1) (Sub2)
max c x max c x
s.t. A x ≤ b s.t. A x ≤ b
x1 ≤ 4 x1 ≥ 5
x≥0, x integral x≥0, x integral

Algorithm 3.1 (Dakin, for a maximization problem)

i
Let P be the linear relaxation of subproblem i, where i=0 designates the original problem
(P).

Step 1: (Initialization) Set z* = -∞ or some lower bound on the objective value. Solve P .
- If it is infeasible, so is P; stop.
- If the optimal solution is such that x is integral, then stop, it is optimal for P.
Otherwise, set i=0 and Go to step 2.

24
OPTIMIZATION 2AGI

Step 2:
i
Branching: From the optimal solution to P , pick x k that is not integer (that is, its value x̂ k
is not integer). Add the constraints:
x k ≤ x̂ k  (largest integer ≤ x̂ k )

x k ≥ x̂ k  + 1

to get two new subproblems.

Bounding:
- For each new node: if the optimal value to the linear program is ≤ z*, or the LP is
infeasible, cut it.
- For each new node, if the Linear relaxation has an integer solution, and value > z*,
record is as the new incumbent, update z*, and cut it.
Otherwise, the node is dangling.
Go to step 3.
Step 3:
- If there is no dangling node, stop. If an incumbent exists, it is optimal; otherwise P is
infeasible.
- If there are dangling nodes, go to step 4.
Step 4:
Choose the dangling node with the largest value and let let i be the index of the
corresponding subproblem. Go to step 2.

Example 3.1

Max 3 x 1 + 4 x 2

s.to x 1 + x 2 ≤ 4.5
x2 ≤ 2
x 1 , x 2 ≥ 0, integer

Simplex tableau for the root problem (linear relaxation of the integer program):

25
OPTIMIZATION 2AGI

x1 x2 x3 x4 RHS
x3 1 1 1 0 4,5
x4 0 1 0 1 2
Z 3 4 0 0 0

x1 x2 x3 x4 RHS x1 x2 x3 x4 RHS
x3 1 0 1 -1 2,5 x1 1 0 1 -1 2,5
x2 0 1 0 1 2 x2 0 1 0 1 2
Z 3 0 0 -4 -8 Z 0 0 -3 -1 -15,5

z* = -∞
(x 1 , x 2 )
z = 15,5 for x = (2,5 ; 2)
x1 ≤ 2 x1 ≥ 3

z* = 14 for x = (2 ; 2) z = 15 for x = (3 ; 1,5)

x2 ≤ 1 x2 ≥ 2

z = 14,5 for x = (3,5 ; 1) infeasible

x1 ≤ 3 x1 ≥ 4

z = 13 for x = (3 ; 1) z = 14 for x = (4 ; 0,5)

Figure 3.4 B&B tree for example 3.1

Example 3.2 (graphical resolution)

Max z = 3 x1 + 2 x2
s. to - 2 x1 + 2 x2 ≤ 7
2 x 1 + 3 x 2 ≤ 18
9 x 1 - 2 x 2 ≤ 36
x 1 , x 2 ≥ 0 and x 1 , x 2 integers.

26
OPTIMIZATION 2AGI

z* = -∞
(x 1 , x 2 )
z = 19+23/31 for x = C (4+20/31 ; 2+28/31)
x2 ≤ 2 x2 ≥ 3

(OEFD) : Point F (ABHG) : Point H

z* = 17+1/3 for x = (4+4/9 ; 2) z = 19+1/2 for x = (4+1/2 ; 3)
x1 ≤ 4 x1 ≥ 5

(ABIJG) : Point I
z = 18+2/3 for x = (4 ; 3+1/3) infeasible
x2 ≤ 3 x2 ≥ 4

([JG]) (BKL)
z* = 18 for x = J (4 ; 3) z = 17 for x = L (3 ; 4)

Figure 3.5 B&B tree for example 3.2

x2
-2x1+2x2=7
9x1-2x2=36
B

K L

A I
H
G J C (4+20/31 ; 2+28/31)

E (0 ; 2)
2x1+3x2=18

O x1
D (4 ; 0)
3.3.3 B&B exploration :

In Dakin's algorithm, the node selection is based on the based LP bound. In general there
two different strategies for node selection:
Progressive Selection: it is based on the best bound ort he best evaluation. It is also called
″breadth-first search″, since it generally leads to wide trees. This strategy is typically

27
OPTIMIZATION 2AGI

memory consuming, but are interesting when the problem at hand has an "easy" structure
and feasible good solutions are relatively easy to find.
Sequential Selection: It consists of choosing, among the non explored nodes, the node that
ist he closest to the most recently explored node (the node just branched on). This is
reduced to perform a "depth first" search or "backtracking" (or also LIFO). This strategy is
less memory consuming that the breadth-first search since data storing is limited to
offspring nodes of the considered node.

3.3.4 B&B for mixed integer programs

To solve mixed integer programs, Dakin's algorithm can be used by branching only on
variables restricted to be integers.
Example 3.3
Max z = 2 x1 + x2
s.to 5 x1 + 2 x2 ≤ 8
x1+ x2 ≤ 3
x 1 , x 2 ≥ 0 and x 1 integer.

z* = -∞ (x 1 , x 2 )
z = 11/3 for x = (2/3 ; 7/3)

x1 ≤ 0 x1 ≥ 1

z=3 z = 7/2
x1 = 0 x1 = 1
x2 = 3 x 2 = 3/2
First incumbent Last incumbent

3.4 Branch-and-Bound for scheduling

Here, we will look at a specific scheduling problem: the single machine tardiness problem.
Consider an example of 4 tasks to be processed on one machine. The processing time and
the due date for each task are shown on Table 3.1 below. The tardiness of a task is defined
as the difference between the actual completion time the due date, if the task is completed

28
OPTIMIZATION 2AGI

after the due date. If the task is completed before or on the due date, its tardiness is then
zero. We would like to determine the order in which the tasks have to be processed in a
way to minimize the total tardiness, denoted by T.

Table 3.1: Processing times and due dates of jobs

Processing times (days) Due dates
Task 1 6 End of day 8
Task 2 4 End of day 4
Task 3 5 End of day 12
Task 4 8 End of day 16

We may formulate this problem as an integer program, by defining the following variables:

x ij = 1 if task i is the jth to be processed,

= 0 otherwise.

Exercise: Continue the formulation of the above problem.

It turns out that we can develop a B&B algorithm that does not use the linear programming
relaxation. We may, however, use the definition of the variables to show how the
subproblems are constructed.
Since the problem consists of finding the optimal sequence of tasks, we define a
subproblem by fixing the positions of some tasks and computing a lower bound of what is
called of ″solution completion″. For example we can choose fixing the task at the last
position. In this case, we may finish the sequence by task 1, 2, 3 or 4. This means:
x 14 = 1 or x 24 = 1 or x 34 = 1 or x 44 = 1
This results in 4 subproblems or branches corresponding to the 4 considered tasks.
How to compute the bound of each branch ?
Take the branch corresponding to x 44 = 1, which means task 4 is processed last. We know
that, for any sequence, the last task will be completed at: 6 + 4 + 5 + 8 = 23. Then, if the last
task is task 4, its tardiness will be = 23 – 16 = 7 days. Therefore, any sequence that ends

29
OPTIMIZATION 2AGI

with task 4 has a total tardiness ≥ 7 days. Hence, the Lower bound of this branch will be LB
= 7 days. Proceeding this way, we obtain the following B&B tree.

x14 = 1 x44 = 1
x24 = 1 x34 = 1

Branch 1 Branch 4
Branch 2 Branch 3
LB = 15 LB = 7
LB = 19 LB = 11
x13 = 1
x33 = 1
x13 = 1 x23 = 1
x23 = 1 x43 = 1

Branch 10 Branch 11 Branch 12 Branch 5 Branch 6 Branch 7

LB = 21 LB = 25 LB = 13 LB = 14 LB = 18 LB = 10

x12 = 1
x22 = 1

Branch 8 Branch 9
T = 12 T = 16

Figure 3.6: B&B search tree for scheduling

To move across the B&B tree, we chose the ″Breadth First″. This means that the next
branch to explore is the one having the smallest lower bound. For example, after creating
the first 4 branches, we chose the explore Branch 4. To do so, we further decompose the
solution space corresponding to Branch 4 into 3 branches or subproblems : 5, 6 and 7.
For each branch, we compute the new lower bound. For example, for Branch 7, the 3rd task
will be finished at the end of day 15 (= 23-8) which will result in a tardiness of 15 – 12 = 3
additional days. Hence the total tardiness T will be at least = 7 + 3 = 10 days.
Then, the next branch to explore is Branch 7 since it has the lowest LB.

The process continues until the first complete solution is found. Since the LB corresponding
to a complete solution is actually equal to the corresponding T. Every branch of the tree
that has LB≥T can be crossed (or fathomed). When branch 7 is explored the solution
having T=12 will allow to fathom all branches except Branch 3. Here, we record Branch 8
as the best solution so far and continue to explore Branch 3. In this example, this branch

30
OPTIMIZATION 2AGI

resulted in three lower bound ≥ 12. These branches are fathomed because any further
exploration will be useless. Once all branches are explored, the best solution so far must be
the optimum. In this case, Branch 8 gives the optimal sequence: 2, 1, 3 and 4, with T=12.

31
OPTIMIZATION 2AGI

Chapter 4 Dynamic Programming

4.1 Introduction
Up to this point, the methods and models presented in this course have one unifying
characteristic which is their static aspect. They address optimisation problems over a fixed
planning period. The whole decision problem is considered in its totality for the whole
period.
However, this does not exclude the use of these models and methods for problems with a
dynamical nature or those involving sequencial decison processes. For example, a
production and inventory control problem over a horizon of several periods can be
modeled as an integer program and solved by branch-and-bound methods.

On the other hand, problems with sequential decision processes, where a decison at one
stage influences the outcome and the decisions of the flollowing stages are, in general,
naturally and best modeled using another technique called "dynamic Programming".

Definition 4.1
Dynamic programming is a mathematical programming technique based on implicit
enumeration, that is used to model optimisation problems involving sequencial decison
processes.
This technique was developed by R. Bellman in 1950 (USA). It consists in decomposing a
problem into subproblems or stages that can be easily solved.

Application examples:
1. Production and inventory control: It consists in planning the production in way that
satisfies demand while minimizing total production and holding cost.
2. Resource allocation: involves allocating limited resources to a set of activities.
3. Equipment replacement: deals with determining a minimum cost strategy that
specifies how long a machine or an equipment should be kept before it is replaced
(or traded in).

32
OPTIMIZATION 2AGI

Most of these problems can be solved using linear or integer programming. However,
dynamic programming is more efficient for problems with low dimensions (a small number
of resources to assign to a number of activities) and, for some problems with nonlinear cost
or profit functions, it may be the only sensible method.
• Dynamic programming can be used for problems having the following
characteristics:
• Continuous or discrete decision variables
• Continuous or discrete horizons (or time)
• Finite of infinite horizons
• Deterministic or stochastic data (such as demand in production planning problems)
In this course we consider only: Discrete, finite horizon and both deterministic and
stochastic dynamic programming.

4.2 Dynamic Programming formulations

4.2.1 General form of a dynamic program

A dynamic program can be written as:

T
Max/Min x1, x2,… xT ∑
t
gt (st , xt )
=1

(DP) s t+1 = ϕ t (s t , x t ) t = 1, …, T
S T+1 given
t : period.
x t : decision variables relative to period t,
s t : state variable describing the system at period t
ϕ t : a recursive function that transforms the state of the system from s t to s t+1 if decision x t
is made.

xT
x1 x2

g1(s1, x1) g2(s2, x2) gT (sT, xT)

s1 s2 … sT sT+1

Figure 4.1 Graphical representation of a dynamic program

33
OPTIMIZATION 2AGI

In general, a problem is decomposed into smaller subproblems. For example, a problem

having T variables can be decomposed into T subproblems of only one variable.

4.2.2 Backward formulation

For many other applications, it is easier to use a dynamic programming formulation using
backaward reasoning. This results into a backward formulation. Backward reasoning may
sometimes make a seemingly difficult problem easy to solve.
Example 4.1
Consider the following game. We have 30 matches. I begin by taking 1, 2 or 3 matches.
Then, comes my opponent's turn to also take 1, 2 or 3 matches. We continue this way until
there is only one match left. The player who takes the last match loses the game. As a first
player, how can I garantee to win the game ?

If I make sure that it is be my opponent's turn when there is only one match left, then I
would be sure of winning. Using the same reasoning by going backwards, if I make sure it is
be my opponent's turn when there are 5 matches left, then I would be sure of winning. The
reason is that no matter how many matches he will take, I can always make sure to get to
the situation where it is his turn when there is only one match left:
If he takes 1 I will take 3
If he takes 2 I will take 2
If he takes 3 I will take 1

Similarly, if if I make sure it is be my opponent's turn when there are 9, 13, 17, 21, 25 or 29
matches left, then I would be sure of winning. This way, I can garantee to win if I begin by
taking 1 match. Then, at each one of my subsequent turns, I will make sure to leave for my
opponent either 25, 21, 17, 13, 9 or 5 matches.

4.3 Characteristics of dynamic programming applications

Dynamic programming applications have certain commun characteristics such as:

1. The problem can be decomposed into stages or periods each representing a point of
decision.

34
OPTIMIZATION 2AGI

2. Each stage is associated with a number of states, where a state is the set of
information (state variables) that is necessary in order to make a feasible decision.

3. The state variables have to be defined such that, to make an optimal decision at each
stage, all we need to know is at which state we started that stage but not how we
reached the state. In other words, given a state of a current stage, the optimal
decision for each of the following stages has to be independent of the previous
decisions. This is the principle of optimality of Bellman.

4. The decision taken at a given stage describes the transition of the system from a
state at a given stage into a state of the next stage.

5. If a problem is formulated using the concept os stages, there must be a recursive

equation that determines the cost or the profit of a state at any stage t, t+1, …, T as a
function of the cost of the profit of states belonging to stages t+1, …, T

f t (i) = Min j {f t+1 (j) + c ij } where j belongs to stage t+1

f 4 (7) = 0 boundary conditions
Objective : find f 1 (1)

4.4 Production and inventory control

A company must decide how much to produce of a given product in order to satisfy
customers' demand over a horizon of T periods. The demand at a period t can be satisfied
either from production during the current period or from inventory that is available at the
beginning of the current period.
Production and inventory holding costs varie from one period to another. the problem is to
determine how much to produce during each period so as to minimize total production and
holding cost.

The following are the data given for each period t = 1,… , T :
- d t : demand of the period, know at the beginning of period 1,
- c t : fixed production cost during period t,
- p t : unit production cost during period t,
- h t : unit holding cost during period t; the quantity held in stock is the quantity left
unsold at the end the period t.
Let x t : be the quantity to produce during period t.

35
OPTIMIZATION 2AGI

The production and inventory control can be formulated as a linear or integer program
(depending on whether the quantities are continuous or discrete) and thus solved using
appropriate algorithms. However, one major assumption has to be made:
- the cost functions have to be linear in terms of the quantities produced or held in
stock.

In some practical settings, this assumption is too restrictive if the cost functions have to be
nonlinear. In this case the problem can be solved using dynamic programming.

Problem formulation
Decision variables:
x t : the quantity to produce during period t.
yt = 1 if x t >0 (decision to produce)
= 0 otherwise.

State variables (t,i t )

t: stage index
i t = inventory at hand at the beginning of period t.
Define f t (i t ) (also denoted by f(t,i t )) as the minimum cost production and inventory
plan for periods t,…, T if the initial inventory at period t is i t .

The recursive equation of the backward formulation is:

f t (i t )= min feasible xt {g t (i t , x t ) + f t+1 (i t +x t - d t )}
i t+1
where g t (i t , x t ) = c t y t + p t x t + h t (i t +x t - d t )
i t ∈ [0, d t + d t+1 + … + d T ]

Total demand left to satisfy

Boundary conditions:
F T (i T )= min feasible xT {g T (i T , x T )

Objective: find f 1 (i 1 ), where i 1 is the initial inventory (known).

36
OPTIMIZATION 2AGI

Example 4.2
A company knows the demands of the next 4 months:

Month t 1 2 3 4
Demand d t 1 3 2 4
At the beginning of each month t, the company has to determine the quantities to produce
x t , given that :
- Fixed production cost c = 3$,
- Unit production cost p = 1$/unit,
- Unit holding cost h = 0,5$/unit left at the end of the period.
- Production capacity ProdCap = 5 units/month
- Storage capacity StorCap = 4 units at the end of the month.

We suppose that there no unit at hand at the beginning of period 1.

To formulate the problem of finding the minimum cost production and inventory plan, one
has to define the states and if possible the stages of the problem. The stages have to defined
such that, if just one stage is left, the problem becomes easy to solve. In this case, if we are
at the beginning of the 4th month, the company can satisfy the demand of the month at a
minimal cost by producing just the necessary quantity based on the stock at the beginning
of the month and the known demand.
⇒ We can define the stage as the month since the problem corresponding to only one
month is easy to solve.

At the beginning of each month, the company must determine the quantity to produce.
Therefore, all is needed is to know the stock level at the beginning of the month.
⇒ The state variables correspond to the stock level at the beginning of the month, which is
nothing but the stock level at the end of the previous month.
i t = 0, 1, …, StorCap (= 4) for t = 2, 3, 4
i 1 = 0.
Recursive function of the backward formulation:
g t (x t , i t ) = 3 + 1 x t + 0.5 (i t + x t – d t ) if x t >0
0.5(i t - d t ) if x t =0

37
OPTIMIZATION 2AGI

Define f t (i t ) as the minimal cost production plan for months t,…, 4 if the inventory level at
the beginning of month t is i t .
f t (i t ) = min feasible xt {g t (x t , i t ) + f t+1 (i t +x t - d t )} for t=1,2,3
In order to take into account the capacity constraints, we have to ensure:
0 ≤ (x t = i t+1 - i t + d t ) ≤ 5 ⇔ i t - d t ≤ i t+1 ≤ 5 + i t - d t
0 ≤ it ≤ 4 ∀ t
Boundary conditions:
f 4 (i 4 ) = 3 + (d 4 – i 4 ) if i 4 ≤ d 4
f 4 (i 4 )= 0 if i 4 ≥ d 4

Objective : find f 1 (0)

Month 4 : the company must produce just the necessary quantity to satisfy the month 4
demand. Since the stock level is zero at the end of month 4, the holding cos twill be zero.
f 4 (i 4 ) = min x4 {g 4 (i 4 , x 4 )}
where
g 4 (i 4 , x 4 ) = 3 y 4 + x 4 ; i 5 = i 4 + x 4 – 4 = 0
0 ≤ x 4 ≤ 5 and 0 ≤ s 4 ≤ 4

f 4 (0) = 3 + 4 = 7$ x4 = 4
f 4 (1) = 3 + 3 = 6$ x4 = 3
f 4 (2) = 3 + 2 = 5$ x4 = 2
F 4 (3) = 3 + 1 = 4$ x4 = 1
f 4 (4) = 0 + 0 = 0$ x4 = 0

Month 3 : we have to detremine the quantities to produce during month 3 so as to

minimize the total cost for months 3 and 4: f 3 (i 3 )
f 3 (i 3 ) = min x3 {g 3 (i 3, x 3 ) + f 4 (i 3 + x 3 - d 3 )}
where
g 3 (i 3, x 3 ) = 3 y 3 + x 3 + 0,5 (i 3 + x 3 - 2)
i4 = i3 + x3 - 2
0 ≤ x3 ≤ 5 ; 0 ≤ i3≤ 4 ; 0 ≤ i4 ≤ 4

38
OPTIMIZATION 2AGI

i3 x3 i4 g3 f 3 (i 3 +x 3 -2) Total cost f 3 (i 3 ) and x 3 (i 3 )

0 2 0 0+5=5 7 5 + 7 = 12*
0 3 1 0,5 + 6 = 6,5 6 6,5 + 6 = 12,5 f 3 (0) = 12
0 4 2 1+7=8 5 8 + 5 = 13 x 3 (0) = 2
0 5 3 1,5 + 8 = 9,5 4 9,5 + 4 =13,5
1 1 0 0+4=4 7 4 + 7 = 11
1 2 1 0,5 + 5 = 5,5 6 5,5 + 6 = 11,5 f 3 (1) = 10
1 3 2 1+6=7 5 5 + 7 = 12 x 3 (1) = 5
1 4 3 1,5 + 7 = 8,5 4 8,5 + 4 = 12,5
1 5 4 2 + 8 = 10 0 10 + 0 =10*
2 0 0 0+0=0 7 0 + 7 = 7*
2 1 1 0,5 + 4 = 4,5 6 4,5 + 6 = 10,5 f 3 (2) = 7
2 2 2 1+5=6 5 6 + 5 = 11 x 3 (2) = 0
2 3 3 1,5 + 6 = 7,5 4 7,5 + 4 = 11,5
2 4 4 2+7=9 0 9+0=9
3 0 1 0,5 + 0 = 0,5 6 0,5 + 6 = 6,5*
3 1 2 1+4=5 5 5 + 5 = 10 f 3 (3) = 6,5
3 2 3 1,5 + 5 = 6,5 4 6,5 + 4 = 10,5 x 3 (3) = 0
3 3 4 2+6=8 3 8 + 3 = 11
4 0 2 1+0=1 5 1 + 5 = 6* f 3 (4) = 6
4 1 3 1,5 + 4 = 5,5 4 5,5 + 4 = 9,5 x 3 (4) = 0
4 2 4 2+5=7 0 7+0=7

Month 2 : (Exercise)
We have to determine the quantities to produce during month 2 so as to minimize the total
cost over the months 2, 3 and 4: f 2 (i 2 )
f 2 (i 2 ) = min x2 {g 2 (i 2 , x 2 ) + f 3 (i 2 + x 2 – d 2 )}
where
g 2 (i 2 , x 2 ) = 3 y 2 + x 2 + 0,5 (i 2 + x 2 - 3)
i3= i2 + x2 - 3
0 ≤ x2 ≤ 5 ; 0 ≤ s3 ≤ 4 ; 0 ≤ s2 ≤ 4

39
OPTIMIZATION 2AGI

i2 x2 i3 g2 f 3 (i 2 +x 2 -3) Total cost f 2 (i 2 ) and x 2 (i 2 )

0 3 0 0+6=6 12 18 f 2 (0) = 16
0 4 1 0,5 + 7 = 7,5 10 17,5 x 2 (0) = 5
0 5 2 1+8=9 7 16*
1 2 0 0+5=5 12 17
1 3 1 0,5 + 6 = 6,5 10 16,5 f 2 (1) = 15
1 4 2 1+7=8 7 15* x 2 (1) = 4
1 5 3 1,5 + 8 = 9,5 6,5 16
2 1 0 0+4=4 12 16
2 2 1 0,5 + 5 = 5,5 10 15,5 f 2 (2) = 14
2 3 2 1+6=7 7 14* x 2 (2) = 3
2 4 3 1,5 + 7 = 8,5 6,5 15
2 5 4 2 + 8 = 10 6 16
3 0 0 0+0=0 12 12*
3 1 1 0,5 + 4 = 4,5 10 4,5 + 10 = 14,5 f 2 (3) = 12
3 2 2 1+5=6 7 6 + 7 = 13 x 2 (3) = 0
3 3 3 1,5 + 6 = 7,5 6,5 7,5 + 6,5 = 14
3 4 4 2+7=9 6 9 + 6 = 15
4 0 1 0,5 + 0 = 0,5 10 10,5*
4 1 2 1+4=5 7 12 f 2 (4) = 10,5
4 2 3 1,5 + 5 = 6,5 6,5 13 x 2 (4) = 0
4 3 4 2+6=8 6 14

Month 1 : We have to determine the quantities to produce during month 1 so as to

minimize the total cost over the months 1, 2, 3 and 4: F 1 (0)
f 1 (0) = min x1 {g 1 (0,x 1 ) + f 2 (x 1 - d 1 )}
where
g 1 (0,x 1 ) = 3 y 1 + x 1 + 0,5 (0 + x 1 - 1)
s 2 = x 1 – 1; 1 ≤ x 1 ≤ 5 ; 0 ≤ s 2 ≤ 4

40
OPTIMIZATION 2AGI

i1 x1 I2 g1 f 2 (x 1 -1) Coût total f 1 (0) and x 1 (0)

0 1 0 0+4=4 16 20*
0 2 1 0,5 + 5 = 5,5 15 20,5 f 1 (0) = 20
0 3 2 1+6=7 14 21 x 1 (0) = 1
0 4 3 1,5 + 7 = 8,5 12 20,5
0 5 4 2 + 8 = 10 10,5 20,5

The cost of the optimal production and inventory plan 20$.

Optimal production plan:
Month t 1 2 3 4
Production x t 1 5 0 4
Exercise
This problem can be addressed as a shortest route problem in a network, which consists of
finding the shortest route from node (1, 0) to node (5, 0) represented in Figure 5.3.

Month Inventory level at the end of the horizon

x2 x3

2, 0 3, 0 4, 0

x4
x1
2, 1 3, 1 4, 1

1, 0 2, 2 3, 2 4, 2 5, 0

2, 3 3, 3 4, 3

2, 4 3, 4 4, 4

Figure 4.2 Network representation of the production and inventory control problem

41
OPTIMIZATION 2AGI

Theorem 4.1
The production and inventory control problem has a solution such that :
1. i t * x t = 0 t = 1,…, T
2. If xt > 0 then x t = ∑ j=t n d j t≤n≤T
3. If it > 0 then i t = ∑ j=t q d j t≤q≤T

Consequence
Using a backward procedure, if the the stock level at the beginning of a period is non zero,
then it is optimal not to produce at that period.
Example : (2, 3) (3, 1) and (3, 2) (4, 1)

4.5 Probabilistic Inventory model

Here we modify the previous inventory model to allow for uncertain demand. This will
illustrate the difficulties involved in solving Stochastic (or probabilistic) Dynamic
Programming (SDP) for which the state during the next stage is uncertain, given the
current state and current decision.

Consider the following 3-period inventory problem.

- Production cost c(0) and c (x) = 3+2x for x>0,
- Unit holding cost h = 1$/unit left at the end of the period,
- Production capacity : ProdCap = 4 units/month,
- Storage capacity : StorCap = 3 units at the end of the month,
- Demand : each period's demand is equally likely to be 1 or 2 units.

Demand has to be met on time and any inventory on hand at the end of period 3 can be sold
at 2$ per unit. Initial inventory = 1 unit.

Question : Use SDP to determine a production policy that minimizes the expected net cost
incurred during the 3 periods.

Define f t (i t ) to be the minimum expected net cost incurred during periods t, t+1, …, 3 when
the inventory at the beginning of period t is i t units. Then,

f 3 (i 3 ) = min x3 {c(x 3 ) + ½ (i 3 + x 3 - 1)+ ½( i 3 + x 3 – 2) – (½)2(i 3 + x 3 - 1)- (½)2( i 3 + x 3 – 2)}

where x 3 ∈{0,2,3,4} and
i 3 + x 3 ≥ 2 and i 3 + x 3 – 1 ≤ 4 ⇔ (2- i 3 ) ≤ x 3 ≤ (4- i 3 )
For t=1,2

42
OPTIMIZATION 2AGI

(t+1, it+ xt-1)

dt = 1 ft+1(it+ xt-1)
Prob = 1/2
Cost=( it+ xt-1)
Decision xt
(3, i3)
Cost= c(xt)
Cost= ( it+ xt-2)
dt = 2
ft+1( it+ xt-2)
Prob = 1/2 (t+1, it+ xt-2)

Therefore, f t (i t ) = min feasible xt { c(x t ) + ½ (i t + x t - 1)+ ½ f t+1 (i t + x t -1)

+ ½( i t + x t – 2)+½ f t+1 (i t + x t -2)}

Where feasible x t means x t ∈{0,2,3,4} and 2 ≤ x t + i t and x t + i t -1≤ 3 which means that
(2- i t ) ≤ x t ≤ (4- i t )

Generalizing this reasoning yields the following important observation concerning the
formulation of SDP : suppose the possible states during period t+1 are s 1 , s 2 , …s n and the
probability that the period t+1 state will be s k is p k . Then the minimum expected cost
incurred during periods t+1, t+2, …, T is

∑k=1,n pkft+1(sk)

where f t+1 (s k ) is the minimum expected cost incurred from period t+1 to the end of the
horizon, given that the state during period t+1 is s k .

We define x t (i t ) to be a period t production level attaining the min for the expression of
f t (i t ). We now work backwards until f 1 (1) is determined. Since each period's ending
inventory must be non-negative and cannot exceed 3 units, the state during each period t
must be either (t,0), (t,1), (t,2) or (t,3).

43
 OPTIMIZATION 2AGI

f3(0)=13/2 x3=2
f4(0)=0
x3(0) = 2 c(x3)=7 d3=2
d2=2
c(x2)=7 3, 0 4, 0
f2(0)=25/2 x3=3
x2=2
d2=1
x2(0) = 3 or 4 x3=4 c(x3)=9 d3=1
d1=2 x2=3 c(x3)=11
2, 0
1≤x1≤3 c(x2)=9
d2=2 f3(1)=9/2
c(x2)=11 f4(1)=-2
d1=1 f2(1)=21/2 x2=4
x1=1 x3(1) = 1
x2(1) = 2 or 3 4,1
c(x1)=5 d2=1 3, 1
2, 1

x1=2
d1=2 d2=2 ⋮
f3(1)=-1/2
1, 1
c(x1)=7 d1=1
f2(2)=6
x2(2) = 0
⋮ x3(1) =0 d3=2 f4(2)=-4
f1(1)=65/4
2, 2 4, 2
x1(1) = 3 c(x2)=0 d2=2
3, 2
x1=3 d1=2 d2=1 d3=1
c(x1)=9
x2=0
d1=1 d2=1 x3=0
2, 3 d2=2 c(x2)=0
x2=1 f4(2)=-6
d3=2
f2(3)=7/2 d2=1
c(x2)=5 3, 3
4, 3
x2(3) = 0
f3(3)=-3/2 x3=1 d3=1
x3(1) =0 c(x2)=5
For example,
7 + ½(0+0)+ ½ (1-2) = 13/2 → x 3 (0) = 2; f 3 (0) = 13/2
f 3 (0) = min 9 + ½(1-2) + ½(2-4)= 15/2
11 + ½(2-4) + ½(3-6) = 17/2

Notice that holding cost of i t units is attributed to each state demand outcome or state (t,i t ).

Solution: We begin by producing x 1 (1) = 3 units during period 1. We cannot, however,

determine period 2's production level until period 1's demand is observed. Also, period 3's
production level cannot be determined until period 2's demand is observed. Look at the
case where d 1 = d 2 = 2 (solution: x 1 (1) = 3; x 2 (2) = 0; x 3 (0) =2).

Important aspect of SDP: If a random factor (e.g., demand) influences transitions from the
period t state to the period t+1 state, the optimal action for period t cannot be determined
until period t's state is known.

44
OPTIMIZATION 2AGI

4.6 Equipment replacement

The problem is to determine when an equipment or a machine should be replaced over a
given horizon.
Example 4.3
A auto repair shop uses an engin analyser. The purchase cost of a new analyser is 1000 $.
The maintnance cost depends on the age of the analyser. We denote by m i the maintnance
cost of the anlyser at it ith year of utilisation.
The analyser can be used for up to 3 years. After its ith year, it can be replaced by a new
one. It can also be sold for an amount that is equal to its salvage value, denoted by s i . The
values of m i and s i are summarized below:
Year i 1 2 3
m i ($) 60 80 120
si ($) 800 600 500

The shop manager needs to determine a replacement policy that minimizes total cost over
the next 5 years, assuming that at the end the 5-year period the analyser will be sold for its
salvage value.
C = maintnance cost + purchase cost – salvage value
A1 A2 A3 A4 A5

T=0 T=1 T=2 T=3 T=4 T=5

After the purchase of a new analyser, the manager has to know when to replace it with a
new one.

4.6.1 A dynamic programming formulation with stages

Stage: t : the time of year t.

State variable: x t : age of the equipment at year t.
Decision variable: replace or keep the equipment.
Recursive function:
f t (x t ) : minimal cost incurred for years t, t+1, …, 5, starting year t with an equipment that
has been used for x t years.
Forr t = 1, 2, 3, 4, we compute f t (1), f t (2) and f t (3)

45
OPTIMIZATION 2AGI

f t (x t ) = min {m x+1 + f t+1 (x t+1 ) ; 1.000 - s x + m 1 + f t+1 (x t+1 )} pour x t = 1 ou 2

xt + 1 1
f t (3) = 1.000 – s 3 + m 1 + f t+1 (1)

Equation linking the state variables and the decision variables:

If the decision is to replace the equipment: x t+1 = 1
If the decision is to keep the equipment: x t+1 = x t + 1

Boundary conditions: f 5 (1) = - 800 ; f 5 (2) = -600 ; f 5 (3) = -500

Objective: Find f 0 (0) = 1.000 + 60 + f 1 (1)

To solve this dynamic program, we start by computing f 4 (x 4 ) then f 3 (x 3 ) then f 2 (x 2 ) then

f 1 (x 1 ) and last f 0 (0).

For t = 1, 2, 3, 4:
f t (1) = min {80+f t+1 (2): keep the equipment; 1.000-800+ 60+f t+1 (1): replace the
equipment}
f t (2) = min {120 + f t+1 (3): keep ; 1.000 - 600 + 60 + f t+1 (1): replace }
f t (3) =1.000 - 500 + 60 + f t+1 (1) : replace the equipment

For t = 4 :
f 4 (1) = min {80 + f 5 (2) = -520,1.000 - 800 + 60 + f 5 (1) = -540* }
⇒ f 4 (1) = -540 : replace the equipment

f 4 (2) = min {120 + f 5 (3) = -380*, 1.000 - 600 + 60 + f 5 (1) = -340 }

⇒ f 4 (2) = -380 : keep the equipment

f 4 (3) =1.000 - 500 + 60 + f 5 (1) = -240 : replace the equipment

For t = 3 :
f 3 (1) = min {80 + f 4 (2) = -300*, 1.000 - 800 + 60 + f 4 (1) = -280 }
⇒ f 3 (1) = -300 : keep the equipment

46
OPTIMIZATION 2AGI

f 3 (2) = min {120 + f 4 (3) = -120*, 1.000 - 600 + 60 + f 4 (1) = -80 }

⇒ f 3 (2) = -120 : keep the equipment

f 3 (3) =1.000 - 500 + 60 + f 4 (1) = 20 : replace the equipment

For t = 2 :
f 2 (1) = min {80 + f 3 (2) = -40*,1.000 - 800 + 60 + f 3 (1) = -40* }
⇒ f 2 (1) = -40: keep or replace the equipment

f 2 (2) = min {120 + f 3 (3) = 140*, 1.000 - 600 + 60 + f 3 (1) = 160 }

⇒ f 2 (2) = 140: keep the equipment

For t = 1 :
f 1 (1) = min {80 + f 2 (2) = 220*,1.000 - 800 + 60 + f 2 (1) = 220* }
⇒ f 1 (1) = 220: keep or replace the equipment

Therefore, f 0 (0) = 1000 + 60 + f 1 (1) = 1280.

The optimal solutions can be represented as follows:

2 3 4 5
0 1 2 3 4 5
3 4 5

keep the equipment replace the equipment

Note :
We have assumed that costs are the same over the years. This is just to simplify the
computation of the costs c tx , which usually depend on t rather than the difference de x-t.

4.6.2 A dynamic programming formulation without stages

We define :

47
OPTIMIZATION 2AGI

g(t) = Minimal cost incurred during periods t, t+1, …, 5, including purchase cost and
salvage, given a new equipment has been purchased at period t.
c tx = the net cost of purchasing a new equipment and using it until year x.

Backward formulation:
g(t) = min x {c tx + g(x)} t = 0, 1, 2, 3, 4
where t+1 ≤ x ≤ t+3
x≤5
Boundary conditions: g(5) = 0
Objective : Find g(0)
c tx dépends on the interval x-t
c tx = 1.000 + m 1 + … + m x-t - s x-t
c 01 = c 12 = c 23 = c 34 = c 45 = 1.000 + 60 - 800 = 260 MD
c 02 = c 13 = c 24 = c 35 = 1.000 + 60 + 80 - 600 = 540 MD
c 03 = c 14 = c 25 = 1.000 + 60 + 80 + 120 - 500 = 760 MD

g(5) = 0
g(4) = c 45 + g(5) = 260 MD
g(3) = min{c 34 + g(4) ; c 35 + g(5)} = min{260+260=520 ; 540+0+540} = 520
g(2) = min{c 23 + g(3) ; c 24 + g(4) ; c 25 + g(5)} = min{780 ; 800 ; 760} = 760
g(1) = min{c 12 + g(2) ; c 13 + g(3) ; c 14 + g(4)} = min{1020 ; 1060 ; 1020} = 1020
g(0) = min{c 01 + g(1) ; c 02 + g(2) ; c 03 + g(3)} = min{1280 ; 1300 ; 1280} = 1280

The optimal solution consists of purchasing a new equipment at t = 0, use it until year 1 or
3, then replace it with a new one.
If we replace it at year 1, we use it until year 2 or 4.
If we replace the equipment at year 2 or 4, we keep the newly purchased until year 5.
If we replace the equipment at year 3, we use it until year 4, then use the newly purchased
equipment until year 5.

0 → 1 → 2 → 5 or 0 → 1 → 4 → 5 or 0 → 3 → 4 → 5
The optimal cost is 1280 $.

48
OPTIMIZATION 2AGI

Exercise: network representation of the problem.

4.7 Summary of general aspects of dynamic programming

In general, in dynamic programming, a stage is composed of all states possible of the
system being in that stage.
Typically, a backward formulation is such that:
f t (i) = minimal cost from stage t until the last stage (T), given that at stage t, the
system is state i.
And, for a forward formulation:
f t (i) = minimal cost from stage 1 until the last stage t, given that at stage t, the
system is state i.

We can identify 3 different aspects of dynamic programming:

Aspect 1 : the set of possible decisions for a given state and a stage:
For the production problem:
If s t-1 is the stock level at the beginning of month t, the set of possible decisions at
stage t is x t such that :
0 ≤ xt ≤ 5 and 0 ≤ s t-1 + x t - d t ≤ 4 ⇒ - s t-1 + d t ≤ x t ≤ 4 - s t-1 + d t

For the equipment replacement problem:

If x t is the age of the equipment at stage t, the set of possible decisions at this stage
is:
Replace or keep the equipment if 1 ≤ x t < 3
Replace the equipment if x t = 3

For the reliability problem:

If y t is the available budget at the beginning of stage t, the set of possible decisions at
stage k t such that :
T t
1 ≤ kt ≤ 3 and ∑Ci(1) ≤ y t - C t (k t ) ≤ 10 -
i = t +1
∑C (1)
i =1
i

49
OPTIMIZATION 2AGI

Aspect 2 : We have to specify how the cost during the current stage t depends on the value
of t, the current state and the decision taken at that stage: c(t) = f(s t-1 , x t )

For the production problem: g t (x t , y t , s t-1 ) = c t y t + p t x t + h t (s t-1 + x t - d t )

For the equipment replacement problem: c tx = 1.000 + m 1 + … + m x-t - s x-t

For the reliability problem: R(i, k i ) = R i (k i )

Aspect 3 : We have to specify how the state of stage t depend on the value of t, the state at
the previous stage t-1 and the decision take at stage t.

For the production problem: s t = s t-1 + x t - d t

For the equipment replacement problem:
If the decision is to replace the equipment: x t+1 = 1
If the decision is to keep the equipment: x t+1 = x t + 1
For the reliability problem: y i+1 = y i - C i (k i )

Notes:
Not all aspects are necessarily found in all recursions. very often, when the stage contains
sufficient information, it is used to define the state (ex: equipment replacement problem).
For problems with a large number of states, dynamic programming becomes less efficient
than Integer programming techniques such as Branch-and-Bound.
dynamic programming is, in general, efficient for low dimensional problems.
There are techniques that make solving high dimensional problems possible with dynamic
programming (e.g. turnpike theorem for the knapsack problem).

50
OPTIMIZATION 2AGI

References

[1] D. Bertsekas, 2001. Dynamic Programming and Optimal Control, Athena Scientific,

2nd Edition.

[2] E. Denardo, 1982. Dynamic Programming : Theory and Applications, Prentice Hall.

[3] F. S. Hillier and G. J. Lieberman, 2001. Introduction to Operations Research. Holden-

Day, Inc.

[4] G. L. Nemhauser and L. A. Wolsey, 1988. Integer and Combinatorial Optimization.

Wiley Interscience.

[5] W. L. Winston, 1994. Operations Research : Applications and Algorithms, Duxbury

press.

51