Professional Documents
Culture Documents
Programming
Chapter 3
Integer Linear Programming
Contents
Simplex Method
Remarks
How to Branch?
Which Node to Select?
Rule of Fathoming
Le Hong Trang, Tran Van Hoai, Nguyen An Khuong, and Software for ILP
1 Motivated Examples
Linear Classification Problem
Facility Location Problem
2 Linear and Integer Linear Programs Contents
Motivated Examples
3 Simplex Method for Solving LP Classification
Facility Location
4 Branch & Bound Method for Solving ILP Linear and Integer
Linear Programs
Numerical Example 1 – IP
Simplex Method
Numerical Example 2 – Binary IP Branch & Bound
Strategy and Steps Method
Numerical Example 1
Exercise: 0-1 Knapsack Problem Numerical Example 2
Strategy and Steps
Remarks
How to Branch? How to Branch?
1 Motivated Examples
Linear Classification Problem
Facility Location Problem
2 Linear and Integer Linear Programs Contents
Motivated Examples
3 Simplex Method for Solving LP Classification
Facility Location
4 Branch & Bound Method for Solving ILP Linear and Integer
Linear Programs
Numerical Example 1 – IP
Simplex Method
Numerical Example 2 – Binary IP Branch & Bound
Strategy and Steps Method
Numerical Example 1
Exercise: 0-1 Knapsack Problem Numerical Example 2
Strategy and Steps
Remarks
How to Branch? How to Branch?
Statement
Given two sets of points in Rn , denoted by{x1 , . . . , xN } and
{y1 , . . . , yM }, we need to classify the points.
Contents
Motivated Examples
Classification
Facility Location
Simplex Method
Problems and
• Data mining, Homeworks
• ...
3.4
Integer Linear
Linear Classification Programming
Contents
Motivated Examples
Classification
Facility Location
Simplex Method
• Use a line (hyperplane in Rn ) aT x + b = 0, hence the Branch & Bound
Method
followings must be satisfied: Numerical Example 1
Numerical Example 2
Remarks
• Since the strict inequalities are homogeneous in a and b, they How to Branch?
Which Node to Select?
are feasible iff the set of nonstrict linear inequalities: Rule of Fathoming
Software for ILP
3.5
Integer Linear
Linear Classification Programming
Contents
Motivated Examples
Classification
Facility Location
Simplex Method
Remarks
where i = 1, . . . , N and j = 1, . . . , M . How to Branch?
Which Node to Select?
PN PM Rule of Fathoming
• The best one should be at minimizer of i=1 ui + j=1 vj . Software for ILP
Problems and
Homeworks
3.6
Integer Linear
Linear Classification Programming
N
X M
X
min ui + vj Contents
Motivated Examples
i=1 j=1
Classification
T
s.t. a xi + b ≥ 1 − ui , i = 1, . . . , N, Facility Location
Simplex Method
ui , vj ≥ 0, i = 1, . . . , N and j = 1, . . . , M. Branch & Bound
Method
or, in vector form Numerical Example 1
Numerical Example 2
Strategy and Steps
T T
min 1 u + 1 v Exercise: 0-1 KP
T Remarks
s.t. x a + 1b 1 − u, How to Branch?
Which Node to Select?
y T a + 1b −1 + v, Rule of Fathoming
Software for ILP
u, v 0. Problems and
Homeworks
3.7
Integer Linear
Linear Classification: numerically solving Programming
n = 2;
randn ( ’ state ’ ,2);
N = 50; M = 50;
Y = [1.5+0.9* randn (1 ,0.6* N ) , 1.5+0.7* randn (1 ,0.4* N ); Contents
2*( randn (1 ,0.6* N )+1) , 2*( randn (1 ,0.4* N ) -1)]; Motivated Examples
Classification
X = [ -1.5+0.9* randn (1 ,0.6* M ) , -1.5+0.7* randn (1 ,0.4* M );
Facility Location
2*( randn (1 ,0.6* M ) -1) , 2*( randn (1 ,0.4* M )+1)];
Linear and Integer
T = [ -1 1; 1 1]; Linear Programs
Y = T*Y; X = T*X; Simplex Method
3.8
Integer Linear
Linear Classification: numerically solving Programming
Contents
6
Motivated Examples
Classification
4 Facility Location
Remarks
−4 How to Branch?
Which Node to Select?
Rule of Fathoming
−6 Software for ILP
Problems and
−8 −6 −4 −2 0 2 4 6 8 10 Homeworks
3.9
Integer Linear
Linear Classification Programming
Contents
min 1T u + 1T v Motivated Examples
s.t. xT a + 1b 1 − u, Classification
Facility Location
u, v 0. Simplex Method
Problems and
Homeworks
3.10
Integer Linear
Uncapacitated Facility Location Programming
Statement
Given a set of supermarket locations, denoted by I, let J be the
set of candidate warehouse locations. The problem is to determine Contents
Simplex Method
Remarks
How to Branch?
Which Node to Select?
Rule of Fathoming
Software for ILP
3.11
Integer Linear
Uncapacitated Facility Location: variables and objective Programming
• A location j ∈ J is decided by
1 if j is chosen for a facility, Contents
xj =
0 otherwise. Motivated Examples
Classification
Facility Location
• The cost associated with constructing each warehouse: fj .
Linear and Integer
• The fraction of supply received by supermarket i from Linear Programs
Simplex Method
warehouse j: yij ≥ 0.
Branch & Bound
• The cost of transportation between candidate warehouse site Method
Numerical Example 1
Remarks
X XX How to Branch?
fj xj + cij yij . Which Node to Select?
Rule of Fathoming
j∈J j∈J i∈I
Software for ILP
Problems and
Homeworks
3.12
Integer Linear
Uncapacitated Facility Location: constraints Programming
Contents
• The sum of fraction from each warehouse for each Motivated Examples
supermarket: X
Classification
Facility Location
Remarks
How to Branch?
Which Node to Select?
Rule of Fathoming
Software for ILP
Problems and
Homeworks
3.13
Integer Linear
Uncapacitated Facility Location: model Programming
X XX Contents
Problems and
Homeworks
3.14
Integer Linear
Uncapacitated Facility Location: numerically solving Programming
Problems and
Homeworks
3.15
Integer Linear
Uncapacitated Facility Location: numerically solving Programming
Contents
Motivated Examples
Classification
Facility Location
Simplex Method
Remarks
How to Branch?
Which Node to Select?
Rule of Fathoming
Software for ILP
Problems and
Homeworks
3.16
Integer Linear
Uncapacitated Facility Location: numerically solving Programming
Contents
Motivated Examples
Classification
Facility Location
Simplex Method
Remarks
How to Branch?
Which Node to Select?
Rule of Fathoming
Software for ILP
Problems and
Homeworks
3.17
Integer Linear
Outline Programming
1 Motivated Examples
Linear Classification Problem
Facility Location Problem
2 Linear and Integer Linear Programs Contents
Motivated Examples
3 Simplex Method for Solving LP Classification
Facility Location
4 Branch & Bound Method for Solving ILP Linear and Integer
Linear Programs
Numerical Example 1 – IP
Simplex Method
Numerical Example 2 – Binary IP Branch & Bound
Strategy and Steps Method
Numerical Example 1
Exercise: 0-1 Knapsack Problem Numerical Example 2
Strategy and Steps
Remarks
How to Branch? How to Branch?
min cT x Contents
x
Motivated Examples
s.t. Classification
Facility Location
Ax = b, (5)
Linear and Integer
x ≥ 0, (6) Linear Programs
Simplex Method
n
where c ∈ R , A is an m × n matrix with full row rank, and Branch & Bound
Method
b ∈ Rm . Numerical Example 1
Numerical Example 2
• A polyhedron is a set of the form {x ∈ Rn |Bx ≥ d} for some Strategy and Steps
Exercise: 0-1 KP
matrix B. Remarks
3.19
Integer Linear
Linear Program in General form Programming
min cT x
Contents
x Motivated Examples
Classification
s.t. Facility Location
Remarks
Question How to Branch?
Is there any way to transform a general form to standard one? Which Node to Select?
Rule of Fathoming
Software for ILP
Problems and
Homeworks
3.20
Integer Linear
Integer Linear Program (ILP) Programming
min cT x
x Contents
Simplex Method
Why do we consider the problem with only equality constraints but Exercise: 0-1 KP
Remarks
not inequality ones? How to Branch?
Which Node to Select?
Rule of Fathoming
Remark Software for ILP
Often a mix is desired of integer and non-integer variables, called Problems and
Homeworks
Mixed Integer Linear Programs (MILP).
3.21
Integer Linear
Basic Solutions and Extreme Points Programming
0 Numerical Example 2
Strategy and Steps
Vector xB is called basic variables, xN is called nonbasic Exercise: 0-1 KP
variables. Remarks
How to Branch?
solution is also feasible, this is, B −1 b ≥ 0, then
• If thebasic Which Node to Select?
−1
B b Rule of Fathoming
0 Problems and
Homeworks
3.22
Integer Linear
Basic Solutions and Extreme Points Programming
Motivated Examples
basic variable. Classification
Facility Location
Theorem (Basic theorem of LP)
Linear and Integer
Linear Programs
Consider the linear program min{cT x|Ax = b, x ≥ 0}. If S has at Simplex Method
least one extreme point and there exists an optimal solution, then Branch & Bound
there exists an optimal solution that is an extreme point. Method
Numerical Example 1
Numerical Example 2
• The feasible set of standard form linear program has at least Strategy and Steps
Exercise: 0-1 KP
one extreme point.
Remarks
• Therefore, we claim that the optimal value of a linear How to Branch?
Which Node to Select?
program is either −∞, or is attained an extreme point (basic Rule of Fathoming
Problems and
Homeworks
3.23
Integer Linear
A Naive Algorithm for Solving Linear Program Programming
n Motivated Examples
many. Classification
Facility Location
−1
• Compute associated basic solution x = B b . Linear and Integer
Linear Programs
0
Simplex Method
• Return the one which has largest objective function value Branch & Bound
Method
among the feasible basic solutions. Numerical Example 1
Question Remarks
How to Branch?
Problems and
Homeworks
3.24
Integer Linear
Outline Programming
1 Motivated Examples
Linear Classification Problem
Facility Location Problem
2 Linear and Integer Linear Programs Contents
Motivated Examples
3 Simplex Method for Solving LP Classification
Facility Location
4 Branch & Bound Method for Solving ILP Linear and Integer
Linear Programs
Numerical Example 1 – IP
Simplex Method
Numerical Example 2 – Binary IP Branch & Bound
Strategy and Steps Method
Numerical Example 1
Exercise: 0-1 Knapsack Problem Numerical Example 2
Strategy and Steps
Remarks
How to Branch? How to Branch?
T
cTB xB + cTN xN
Strategy and Steps
c x= Exercise: 0-1 KP
(leaving variable), we can get a new basic feasible solution Linear and Integer
Linear Programs
whose objective value is less than that of the current basic Simplex Method
feasible solution x̂. Branch & Bound
Method
• Geometrically, the simplex method moves from one extreme Numerical Example 1
• Since there are only a finite number of extreme points, the Exercise: 0-1 KP
Remarks
method terminates finitely at an optimal solution or detects How to Branch?
Problems and
Homeworks
3.27
Integer Linear
Geometry of Simplex Method Programming
Contents
Motivated Examples
Classification
Facility Location
Simplex Method
Remarks
How to Branch?
Which Node to Select?
Rule of Fathoming
Software for ILP
Problems and
Homeworks
3.28
Integer Linear
Simplex Method Programming
Simplex Method
Otherwise, goto Step 3.
Branch & Bound
• Step 3: compute the step size Method
Numerical Example 1
Numerical Example 2
−B −1 aj
Which Node to Select?
Rule of Fathoming
Let x := x + λdj , where dj = . goto Step 1 by
ej Software for ILP
Problems and
replacing xr by xj in B to get the new basic. Homeworks
3.29
Integer Linear
Simplex Tableau Programming
Contents
xB xN rhs Motivated Examples
B N b Classification
I B −1 N B −1 b Numerical Example 1
Numerical Example 2
Remarks
How to Branch?
Which Node to Select?
Rule of Fathoming
Software for ILP
Problems and
Homeworks
3.30
Integer Linear
Simplex Method: An example Programming
Remarks
−1 2 1 0 0 4 How to Branch?
−7 −2 0 0 0 0 Problems and
Homeworks
3.31
Integer Linear
Simplex Method: example (cont.) Programming
1 Motivated Examples
1 1 0 0 −2 3 12 Classification
0 5 0 0 − 2 24 12
7 Facility Location
Remarks
obtained as below. How to Branch?
Motivated Examples
Therefore, the tableau is Classification
Problems and
Homeworks
3.33
Integer Linear
Outline Programming
1 Motivated Examples
Linear Classification Problem
Facility Location Problem
2 Linear and Integer Linear Programs Contents
Motivated Examples
3 Simplex Method for Solving LP Classification
Facility Location
4 Branch & Bound Method for Solving ILP Linear and Integer
Linear Programs
Numerical Example 1 – IP
Simplex Method
Numerical Example 2 – Binary IP Branch & Bound
Strategy and Steps Method
Numerical Example 1
Exercise: 0-1 Knapsack Problem Numerical Example 2
Strategy and Steps
Remarks
How to Branch? How to Branch?
Motivated Examples
Classification
Facility Location
Simplex Method
Remarks
How to Branch?
Which Node to Select?
Rule of Fathoming
Software for ILP
Problems and
Homeworks
3.35
Integer Linear
Branch & Bound Method – Introduction Programming
• The conquering part is done by estimate how good a solution Motivated Examples
Classification
we can get for each smaller problems. Facility Location
Simplex Method
Remarks
How to Branch?
Which Node to Select?
Rule of Fathoming
Software for ILP
Problems and
Homeworks
3.35
Integer Linear
Branch & Bound Method – Introduction Programming
• The conquering part is done by estimate how good a solution Motivated Examples
Classification
we can get for each smaller problems. Facility Location
• To do so, we may have to divide the problem further, until we Linear and Integer
Linear Programs
get a problem that we can handle, that is the “bound” part. Simplex Method
Remarks
How to Branch?
Which Node to Select?
Rule of Fathoming
Software for ILP
Problems and
Homeworks
3.35
Integer Linear
Branch & Bound Method – Introduction Programming
• The conquering part is done by estimate how good a solution Motivated Examples
Classification
we can get for each smaller problems. Facility Location
• To do so, we may have to divide the problem further, until we Linear and Integer
Linear Programs
get a problem that we can handle, that is the “bound” part. Simplex Method
• We will use the linear programming relaxation to estimate the Branch & Bound
Method
optimal solution of an integer programming. Numerical Example 1
Numerical Example 2
Strategy and Steps
Exercise: 0-1 KP
Remarks
How to Branch?
Which Node to Select?
Rule of Fathoming
Software for ILP
Problems and
Homeworks
3.35
Integer Linear
Branch & Bound Method – Introduction Programming
• The conquering part is done by estimate how good a solution Motivated Examples
Classification
we can get for each smaller problems. Facility Location
• To do so, we may have to divide the problem further, until we Linear and Integer
Linear Programs
get a problem that we can handle, that is the “bound” part. Simplex Method
• We will use the linear programming relaxation to estimate the Branch & Bound
Method
optimal solution of an integer programming. Numerical Example 1
Numerical Example 2
• For an integer programming model P, the linear programming Strategy and Steps
model we get by dropping the requirement that all variables Exercise: 0-1 KP
Remarks
must be integers is called the linear programming relaxation How to Branch?
Problems and
Homeworks
3.35
Integer Linear
Numerical Example 1: Integer Programming Programming
Contents
Simplex Method
5x1 + 10x2 ≤ 49 (14) Branch & Bound
Method
x1 ≤ 5 (15) Numerical Example 1
Numerical Example 2
Strategy and Steps
xi ≥ 0, xi ∈ Z ∀i ∈ {1, 2} (16) Exercise: 0-1 KP
Remarks
How to Branch?
Which Node to Select?
Rule of Fathoming
Software for ILP
Problems and
Homeworks
3.36
Integer Linear
Numerical Example 1 (cont’) : Linear programming relaxation Programming
Motivated Examples
max(Z = −x1 + 4x2 ) (17) Classification
Facility Location
Problems and
Homeworks
3.37
Integer Linear
Numerical Example 1 (cont’) Programming
Contents
Motivated Examples
Classification
Facility Location
Simplex Method
Remarks
How to Branch?
Which Node to Select?
Rule of Fathoming
Software for ILP
Problems and
Optimal solution of relaxation is (3.8, 3) with Z = 8.2. Homeworks
3.38
Integer Linear
Numerical Example 1 (cont’): Linear programming relaxation Programming
Contents
Motivated Examples
Classification
Facility Location
Simplex Method
Remarks
How to Branch?
Which Node to Select?
Since optimal solution of relaxation is (3.8, 3), we consider two Rule of Fathoming
Software for ILP
cases: x1 ≥ 4 and x1 ≤ 3.
Problems and
Homeworks
3.39
Integer Linear
Example 1 (cont’) – LP relaxation: x1 ≥ 4 Programming
Case x1 ≥ 4:
Contents
max(Z = −x1 + 4x2 ) (22) Motivated Examples
Classification
Subject to Facility Location
x2 ≥ 0 (27) Remarks
How to Branch?
Which Node to Select?
Rule of Fathoming
Software for ILP
Problems and
Homeworks
3.40
Integer Linear
Numerical Example 1 (cont’) – LP relaxation:x1 ≥ 4 Programming
Contents
Motivated Examples
Classification
Facility Location
Simplex Method
Remarks
How to Branch?
Which Node to Select?
has optimal solution at (4, 2.9) with Z = 7.6. Rule of Fathoming
Software for ILP
Then we consider two cases: x2 ≥ 3 and x2 ≤ 2.
Problems and
Homeworks
3.41
Integer Linear
Numerical Example 1 (cont’) – LP relaxation: x1 ≥ 4, x2 ≥ 3 Programming
Case x1 ≥ 4, x2 ≥ 3:
Contents
max(Z = −x1 + 4x2 ) (28) Motivated Examples
Classification
Facility Location
Subject to
Linear and Integer
−10x1 + 20x2 ≤ 22 (29) Linear Programs
Simplex Method
5x1 + 10x2 ≤ 49 (30) Branch & Bound
Method
4 ≤ x1 ≤ 5 (31) Numerical Example 1
Numerical Example 2
Strategy and Steps
x2 ≥ 3 (32) Exercise: 0-1 KP
Remarks
has no feasible solution (5x1 +10x2 ≥ 50) so the IP has no How to Branch?
Problems and
Homeworks
3.42
Integer Linear
Numerical Example 1 (cont’) – LP relaxation: x1 ≥ 4, x2 ≤ 2 Programming
Case x1 ≥ 4, x2 ≤ 2: Contents
Motivated Examples
max(Z = −x1 + 4x2 ) (33) Classification
Facility Location
Problems and
Homeworks
3.43
Integer Linear
Numerical Example 1 (cont’) – LP relaxation: - x1 ≥ 4, x2 ≤ 2 Programming
Contents
Motivated Examples
Classification
Facility Location
Simplex Method
Remarks
How to Branch?
3.44
Integer Linear
Numerical Example 1 (cont’) – LP relaxation: x1 ≤ 3 Programming
Motivated Examples
max(Z = −x1 + 4x2 ) (38) Classification
Facility Location
Problems and
Homeworks
3.45
Integer Linear
Numerical Example 1 (cont’): LP relaxation Programming
Contents
Motivated Examples
Classification
Facility Location
Simplex Method
Remarks
How to Branch?
Which Node to Select?
has an optimal solution at (3, 2.6) with Z = 7.4. Rule of Fathoming
Software for ILP
We branch out further to two cases: x2 ≥ 3 and x2 ≤ 2.
Problems and
Homeworks
3.46
Integer Linear
Example 1 (cont’) – LP relaxation: x1 ≤ 3, x2 ≥ 3 Programming
Case x1 ≤ 3, x2 ≥ 3:
Contents
max(Z = −x1 + 4x2 ) (43) Motivated Examples
Classification
Facility Location
Subject to
Linear and Integer
−10x1 + 20x2 ≤ 22 (44) Linear Programs
Simplex Method
5x1 + 10x2 ≤ 49 (45) Branch & Bound
Method
0 ≤ x1 ≤ 3 (46) Numerical Example 1
Numerical Example 2
Strategy and Steps
x2 ≥ 3 (47) Exercise: 0-1 KP
Remarks
has no feasible solution (−10x1 +20x2 ≥ 30). How to Branch?
Problems and
Homeworks
3.47
Integer Linear
Numerical Example 1 (cont’) – LP relaxation: - x1 ≤ 3, x2 ≤ 2 Programming
Case x1 ≤ 3, x2 ≤ 2: Contents
Motivated Examples
max(Z = −x1 + 4x2 ) (48) Classification
Facility Location
Problems and
Homeworks
3.48
Integer Linear
Numerical Example 1 (cont’) – LP relaxation: x1 ≤ 3, x2 ≤ 2 Programming
Contents
Motivated Examples
Classification
Facility Location
Simplex Method
Remarks
How to Branch?
Which Node to Select?
has an optimal at (1.8, 2) with Z = 6.2. Rule of Fathoming
Software for ILP
We branch further with two cases: x1 ≥ 2 or x1 ≤ 1
Problems and
Homeworks
3.49
Integer Linear
Numerical Example 1 (cont’) – LP relaxation: x1 ≥ 2, x2 ≤ 2 Programming
Case x1 ≥ 2, x2 ≤ 2: Contents
Motivated Examples
max(Z = −x1 + 4x2 ) (53) Classification
Facility Location
Problems and
Homeworks
3.50
Integer Linear
Numerical Example 1 (cont’) – LP relaxation: x1 ≥ 2, x2 ≤ 2 Programming
Contents
Motivated Examples
Classification
Facility Location
Simplex Method
Remarks
How to Branch?
3.51
Integer Linear
Numerical Example 1 (cont’) – LP relaxation: x1 ≤ 1, x2 ≤ 2 Programming
Case x1 ≤ 1, x2 ≤ 2: Contents
Motivated Examples
max(Z = −x1 + 4x2 ) (58) Classification
Facility Location
Problems and
Homeworks
3.52
Integer Linear
Numerical Example 1 (cont’) – LP relaxation: x1 ≤ 3, x2 ≤ 2 Programming
Contents
Motivated Examples
Classification
Facility Location
Simplex Method
Remarks
How to Branch?
with the value of Z greater than 5.4. This branch is fathomed. Problems and
Homeworks
3.53
Integer Linear
Numerical Example 1 (cont’) – LP relaxation: conclusion Programming
Motivated Examples
Classification
Facility Location
Simplex Method
Remarks
Z=6 (2, 2) Z=5.4 (1, 1.6) How to Branch?
Which Node to Select?
Rule of Fathoming
Software for ILP
Problems and
Homeworks
3.54
Integer Linear
Numerical Example 2 – Binary IP Problem Programming
Contents
max(Z = 9x1 + 5x2 + 6x3 + 4x4 ) (63) Motivated Examples
Classification
Subject to Facility Location
Problems and
Homeworks
3.55
Integer Linear
Numerical Example 2: LP Relaxation Programming
Motivated Examples
6x1 + 3x2 + 5x3 + 2x4 ≤ 10 (70) Classification
Facility Location
x3 + x4 ≤ 1 (71) Linear and Integer
Linear Programs
−x1 + x3 ≤ 0 (72) Simplex Method
3.56
Integer Linear
Numerical Example 2 – LP Relaxation: x1 = 0 Programming
Remarks
has the optimal solution at (0, 1, 0, 1) with Z = 9. (Current best How to Branch?
Problems and
Homeworks
3.57
Integer Linear
Numerical Example 2 – LP Relaxation: x1 = 1 Programming
Motivated Examples
Subject to Classification
Facility Location
x3 + x4 ≤ 1 Simplex Method
0 ≤xi Remarks
How to Branch?
Which Node to Select?
has the optimal solution at (1, 0.8, 0, 0.8) with Z = 16.2 Rule of Fathoming
Software for ILP
Branch: x2 = 0 or x2 = 1 Problems and
Homeworks
3.58
Integer Linear
Numerical Example 2 – LP Relaxation: x1 = 1, x2 = 0 Programming
Motivated Examples
Subject to Classification
Facility Location
x3 ≤ 1 Simplex Method
Remarks
• With x3 = 0 The optimal solution is (1, 0, 0, 0) with Z = 9 How to Branch?
Problems and
Homeworks
3.59
Integer Linear
Numerical Example 2 – LP Relaxation: x1 = 1, x2 = 0 Programming
Motivated Examples
Subject to Classification
Facility Location
x3 ≤ 1 Simplex Method
Remarks
• With x3 = 0 The optimal solution is (1, 0, 0, 0) with Z = 9 How to Branch?
Problems and
Homeworks
3.59
Integer Linear
Numerical Example 2 – LP Relaxation: x1 = 1, x2 = 1 Programming
Remarks
Problems and
Homeworks
3.60
Integer Linear
Numerical Example 2 – LP Relaxation: x1 = 1, x2 = 1 Programming
Contents
S.t. 5x3 + 2x4 ≤ 1 Motivated Examples
x3 + x4 ≤ 1 Classification
Facility Location
0 ≤ x4 ≤ 1 Simplex Method
Remarks
How to Branch?
Which Node to Select?
Rule of Fathoming
⇒ The current best solution (1, 1, 0, 0) with Z = 14 is the Software for ILP
Problems and
optimal solution. Homeworks
3.61
Integer Linear
Numerical Example 2 – LP Relaxation: x1 = 1, x2 = 1 Programming
Contents
S.t. 5x3 + 2x4 ≤ 1 Motivated Examples
x3 + x4 ≤ 1 Classification
Facility Location
0 ≤ x4 ≤ 1 Simplex Method
Remarks
How to Branch?
Which Node to Select?
Rule of Fathoming
⇒ The current best solution (1, 1, 0, 0) with Z = 14 is the Software for ILP
Problems and
optimal solution. Homeworks
3.61
Integer Linear
Numerical Example 2 – LP Relaxation: x1 = 1, x2 = 1 Programming
Contents
S.t. 5x3 + 2x4 ≤ 1 Motivated Examples
x3 + x4 ≤ 1 Classification
Facility Location
0 ≤ x4 ≤ 1 Simplex Method
⇒ The current best solution (1, 1, 0, 0) with Z = 14 is the Software for ILP
Problems and
optimal solution. Homeworks
3.61
Integer Linear
Numerical Example 2 – LP Relaxation: x1 = 1, x2 = 1 Programming
Contents
S.t. 5x3 + 2x4 ≤ 1 Motivated Examples
x3 + x4 ≤ 1 Classification
Facility Location
0 ≤ x4 ≤ 1 Simplex Method
⇒ The current best solution (1, 1, 0, 0) with Z = 14 is the Software for ILP
Problems and
optimal solution. Homeworks
3.61
Integer Linear
Numerical Example 2 (cont’) – LP relaxation: conclusion Programming
(0, 1, 0, 1) Z=9
(1, 0, 0, 0) Z = 9 Contents
Remarks
Problems and
Homeworks
3.62
Integer Linear
Branch & Bound: General Strategy Programming
Simplex Method
Remarks
How to Branch?
Which Node to Select?
Rule of Fathoming
Software for ILP
Problems and
Homeworks
3.63
Integer Linear
Branch & Bound: General Strategy Programming
Simplex Method
Remarks
How to Branch?
Which Node to Select?
Rule of Fathoming
Software for ILP
Problems and
Homeworks
3.63
Integer Linear
Branch & Bound: General Strategy Programming
Remarks
How to Branch?
Which Node to Select?
Rule of Fathoming
Software for ILP
Problems and
Homeworks
3.63
Integer Linear
Branch & Bound: General Strategy Programming
Remarks
How to Branch?
Which Node to Select?
Rule of Fathoming
Software for ILP
Problems and
Homeworks
3.63
Integer Linear
Branch & Bound: General Strategy Programming
Remarks
How to Branch?
Which Node to Select?
Rule of Fathoming
Software for ILP
Problems and
Homeworks
3.63
Integer Linear
Branch & Bound: General Strategy Programming
Remarks
How to Branch?
Which Node to Select?
Rule of Fathoming
Software for ILP
Problems and
Homeworks
3.63
Integer Linear
Branch & Bound: General Strategy Programming
Remarks
Repeat until there are no active subproblems. How to Branch?
Which Node to Select?
Rule of Fathoming
Software for ILP
Problems and
Homeworks
3.63
Integer Linear
Branch & Bound: Steps Programming
Motivated Examples
Classification
Facility Location
Simplex Method
Remarks
How to Branch?
Which Node to Select?
Rule of Fathoming
Software for ILP
Problems and
Homeworks
3.64
Integer Linear
Branch & Bound: Steps Programming
Simplex Method
Remarks
How to Branch?
Which Node to Select?
Rule of Fathoming
Software for ILP
Problems and
Homeworks
3.64
Integer Linear
Branch & Bound: Steps Programming
Simplex Method
Remarks
How to Branch?
Which Node to Select?
Rule of Fathoming
Software for ILP
Problems and
Homeworks
3.64
Integer Linear
Branch & Bound: Steps Programming
Simplex Method
Remarks
How to Branch?
Which Node to Select?
Rule of Fathoming
Software for ILP
Problems and
Homeworks
3.64
Integer Linear
Branch & Bound: Steps Programming
Remarks
How to Branch?
Which Node to Select?
Rule of Fathoming
Software for ILP
Problems and
Homeworks
3.64
Integer Linear
Branch & Bound: Steps Programming
Remarks
How to Branch?
Which Node to Select?
Rule of Fathoming
Software for ILP
Problems and
Homeworks
3.64
Integer Linear
Branch & Bound: Steps Programming
Problems and
Homeworks
3.64
Integer Linear
Branch & Bound: Steps Programming
- The LP problem has an optimal solution that are not all Which Node to Select?
Rule of Fathoming
integer, better than the incumbent. In this case we would Software for ILP
3.64
Integer Linear
Exercise: An 0-1 Knapsack Problem Instance Programming
Problems and
Homeworks
3.65
Integer Linear
Exercise: An 0-1 Knapsack Problem Instance Programming
Motivated Examples
Classification
Facility Location
Simplex Method
Problems and
Homeworks
3.66
Integer Linear
Outline Programming
1 Motivated Examples
Linear Classification Problem
Facility Location Problem
2 Linear and Integer Linear Programs Contents
Motivated Examples
3 Simplex Method for Solving LP Classification
Facility Location
4 Branch & Bound Method for Solving ILP Linear and Integer
Linear Programs
Numerical Example 1 – IP
Simplex Method
Numerical Example 2 – Binary IP Branch & Bound
Strategy and Steps Method
Numerical Example 1
Exercise: 0-1 Knapsack Problem Numerical Example 2
Strategy and Steps
Remarks
How to Branch? How to Branch?
Simplex Method
Branch the most fractional variable.
Branch & Bound
• We would like to choose the branching that minimizes the Method
Numerical Example 1
sum of the solution times of all the created subproblems. Numerical Example 2
Strategy and Steps
• How do we know how long it will take to solve each Exercise: 0-1 KP
subproblem? Remarks
How to Branch?
• Answer: We don’t. Which Node to Select?
• Idea: Try to predict the difficulty of a subproblem. Rule of Fathoming
Software for ILP
• A good branching rule: The value of the linear programming Problems and
Homeworks
relaxation changes a lot.
3.68
Integer Linear
Which Node to Select? Programming
Contents
• An important choice in branch and bound is the strategy for Motivated Examples
Classification
selecting the next subproblem to be processed. Facility Location
• Goals: (i) Minimizing overall solution time; (ii) Finding a Linear and Integer
Linear Programs
good feasible solution quickly. Simplex Method
Remarks
How to Branch?
Which Node to Select?
Rule of Fathoming
Software for ILP
Problems and
Homeworks
3.69
Integer Linear
The Best First Approach Programming
Motivated Examples
choosing the subproblem with the best bound (lowest lower Classification
feasible solutions are “more likely” to be found deep in the Branch & Bound
Method
tree. Numerical Example 1
• Node setup costs are high. The linear program being solved Numerical Example 2
Strategy and Steps
may change quite a bit from one node evaluation to the next. Exercise: 0-1 KP
the candidate list, since the tree can grow “broad”. How to Branch?
Which Node to Select?
Rule of Fathoming
Software for ILP
Problems and
Homeworks
3.70
Integer Linear
The Depth-First Approach Programming
of candidate nodes is minimized (saving memory). The node Linear and Integer
Linear Programs
set-up costs are minimized. Simplex Method
• LPs change very little from one iteration to the next. Feasible Branch & Bound
Method
solutions are usually found quickly. Numerical Example 1
• Drawback: If the initial lower bound is not very good, then we Numerical Example 2
Strategy and Steps
Remarks
• Hybrid Strategies: Go depth-first until you find a feasible How to Branch?
Problems and
Homeworks
3.71
Integer Linear
Rule of Fathoming Programming
Contents
with z < z ∗ where z ∗ is the current best solution; Linear and Integer
Linear Programs
Remarks
How to Branch?
Which Node to Select?
Rule of Fathoming
Software for ILP
Problems and
Homeworks
3.72
Integer Linear
Software for ILP Programming
Remarks
• Many open-source solvers were developed in the literature. How to Branch?
Which Node to Select?
Rule of Fathoming
Software for ILP
Problems and
Homeworks
3.73
Integer Linear
Software for ILP: Modeling Tools Programming
• CVX: cvxr.com/cvx
Contents
• YALMIP: yalmip.github.io Motivated Examples
Classification
• TOMLAB: tomopt.com/tomlab Facility Location
Remarks
→ See the next slide for an example of the use of CVX. How to Branch?
Which Node to Select?
Rule of Fathoming
Software for ILP
Problems and
Homeworks
3.74
Integer Linear
Example: Solving the Linear Classification Problem using CVX Programming
Statement
Contents
Given two sets of points in Rn , denoted by{x1 , . . . , xN } and Motivated Examples
{y1 , . . . , yM }, we need to classify the points. Classification
Facility Location
Simplex Method
Remarks
How to Branch?
Which Node to Select?
Rule of Fathoming
Software for ILP
Problems and
Homeworks
3.75
Integer Linear
Solving the Linear Classification Problem using CVX Programming
Contents
Remarks
How to Branch?
Which Node to Select?
Rule of Fathoming
Software for ILP
Problems and
Homeworks
3.76
Integer Linear
Solving the Linear Classification Problem using CVX Programming
n = 2;
randn ( ’ state ’ ,2);
N = 50; M = 50;
Y = [1.5+0.9* randn (1 ,0.6* N ) , 1.5+0.7* randn (1 ,0.4* N ); Contents
2*( randn (1 ,0.6* N )+1) , 2*( randn (1 ,0.4* N ) -1)]; Motivated Examples
Classification
X = [ -1.5+0.9* randn (1 ,0.6* M ) , -1.5+0.7* randn (1 ,0.4* M );
Facility Location
2*( randn (1 ,0.6* M ) -1) , 2*( randn (1 ,0.4* M )+1)];
Linear and Integer
T = [ -1 1; 1 1]; Linear Programs
Y = T*Y; X = T*X; Simplex Method
3.77
Integer Linear
Solving the Linear Classification Problem using CVX Programming
Contents
6
Motivated Examples
Classification
4 Facility Location
Remarks
−4 How to Branch?
Which Node to Select?
Rule of Fathoming
−6 Software for ILP
Problems and
−8 −6 −4 −2 0 2 4 6 8 10 Homeworks
3.78
Integer Linear
Outline Programming
1 Motivated Examples
Linear Classification Problem
Facility Location Problem
2 Linear and Integer Linear Programs Contents
Motivated Examples
3 Simplex Method for Solving LP Classification
Facility Location
4 Branch & Bound Method for Solving ILP Linear and Integer
Linear Programs
Numerical Example 1 – IP
Simplex Method
Numerical Example 2 – Binary IP Branch & Bound
Strategy and Steps Method
Numerical Example 1
Exercise: 0-1 Knapsack Problem Numerical Example 2
Strategy and Steps
Remarks
How to Branch? How to Branch?
Statement of an instance
Suppose a company has 2 factories F1 , F2 and 9 retail outlets
R1 , . . . , R9 :
• the total supply of the product from each factory F1 is ai ; Contents
• the total demand for the product at each outlet Rj is bj ; Motivated Examples
Classification
• The cost of sending one unit of the product from factory Fi Facility Location
Simplex Method
where i = 1, 2 and j = 1, 2, . . . , 9.
Branch & Bound
The problem is to determine a transportation scheme between the Method
factories and the outlets so as to minimize the total transportation Numerical Example 1
Numerical Example 2
cost, subject to the specified supply and demand constraints.. Strategy and Steps
Exercise: 0-1 KP
Remarks
How to Branch?
Which Node to Select?
Rule of Fathoming
Software for ILP
Problems and
Homeworks
3.80
Integer Linear
Transportation Problem — Linear Programming Model Programming
Statement of an instance
Suppose a company has 2 factories F1 , F2 and 9 retail outlets
R1 , . . . , R9 :
• the total supply of the product from each factory F1 is ai ; Contents
• the total demand for the product at each outlet Rj is bj ; Motivated Examples
Classification
• The cost of sending one unit of the product from factory Fi Facility Location
Simplex Method
where i = 1, 2 and j = 1, 2, . . . , 9.
Branch & Bound
The problem is to determine a transportation scheme between the Method
factories and the outlets so as to minimize the total transportation Numerical Example 1
Numerical Example 2
cost, subject to the specified supply and demand constraints.. Strategy and Steps
Exercise: 0-1 KP
Remarks
• Objective: minimum cost of transporting. How to Branch?
Which Node to Select?
• Constraint: total supply of the factories and, total demand for Rule of Fathoming
Problems and
• Variable: the size of the shipment from Fi to Rj , where Homeworks
i = 1, 2 and j = 1, 2, . . . , 9.
3.80
Integer Linear
Transportation Problem: Mathematical Formulation Programming
Motivated Examples
Classification
Facility Location
Simplex Method
Remarks
How to Branch?
Which Node to Select?
Rule of Fathoming
Software for ILP
Problems and
Homeworks
3.81
Integer Linear
Transportation Problem: Mathematical Formulation Programming
Remarks
How to Branch?
Which Node to Select?
Rule of Fathoming
Software for ILP
Problems and
Homeworks
3.81
Integer Linear
Transportation Problem: Mathematical Formulation Programming
Problems and
Homeworks
3.81
Integer Linear
Transportation Problem: Mathematical Formulation (cont.) Programming
X
min cij xij Contents
x
ij Motivated Examples
Classification
subject to Facility Location
xij ≥ 0, i = 1, 2, j = 1, . . . , 9. Remarks
How to Branch?
(83) Which Node to Select?
Rule of Fathoming
Software for ILP
Problems and
Homeworks
3.82
Integer Linear
Mathematical Formulation Programming
Contents
Motivated Examples
Classification
Steps Facility Location
Remarks
How to Branch?
Which Node to Select?
Rule of Fathoming
Software for ILP
Problems and
Homeworks
3.83
Integer Linear
Knapsack Problems — Integer Linear Programming Model Programming
Simplex Method
success of the tour;
Branch & Bound
Method
Numerical Example 1
Numerical Example 2
Strategy and Steps
Exercise: 0-1 KP
Remarks
How to Branch?
Which Node to Select?
Rule of Fathoming
Software for ILP
Problems and
Homeworks
3.84
Integer Linear
Knapsack Problems — Integer Linear Programming Model Programming
Simplex Method
success of the tour;
Branch & Bound
• The objects are independent from each other (e.g. can and Method
Numerical Example 1
can opener are not independent as any of them without the Numerical Example 2
Remarks
How to Branch?
Which Node to Select?
Rule of Fathoming
Software for ILP
Problems and
Homeworks
3.84
Integer Linear
Knapsack Problems — Integer Linear Programming Model Programming
Simplex Method
success of the tour;
Branch & Bound
• The objects are independent from each other (e.g. can and Method
Numerical Example 1
can opener are not independent as any of them without the Numerical Example 2
Problems and
Homeworks
3.84
Integer Linear
Knapsack Problems — Integer Linear Programming Model Programming
Simplex Method
success of the tour;
Branch & Bound
• The objects are independent from each other (e.g. can and Method
Numerical Example 1
can opener are not independent as any of them without the Numerical Example 2
• The strength of the tourist makes possible to bring only a Rule of Fathoming
Software for ILP
3.84
Integer Linear
Knapsack Problems — Integer Linear Programming Model Programming
Simplex Method
success of the tour;
Branch & Bound
• The objects are independent from each other (e.g. can and Method
Numerical Example 1
can opener are not independent as any of them without the Numerical Example 2
• The strength of the tourist makes possible to bring only a Rule of Fathoming
Software for ILP
b the maximal weight what the tourist can bring. Linear and Integer
Linear Programs
Simplex Method
Remarks
How to Branch?
Which Node to Select?
Rule of Fathoming
Software for ILP
Problems and
Homeworks
3.85
Integer Linear
Mathematical Formulation Programming
b the maximal weight what the tourist can bring. Linear and Integer
Linear Programs
Remarks
How to Branch?
Which Node to Select?
Rule of Fathoming
Software for ILP
Problems and
Homeworks
3.85
Integer Linear
Mathematical Formulation Programming
b the maximal weight what the tourist can bring. Linear and Integer
Linear Programs
Remarks
How to Branch?
• Notice that Which Node to Select?
Rule of Fathoming
wj if object j is present on the tour, Software for ILP
w j xj = Problems and
0 if object j isn’t present on the tour Homeworks
Simplex Method
Remarks
How to Branch?
Which Node to Select?
Rule of Fathoming
Software for ILP
Problems and
Homeworks
3.86
Integer Linear
Mathematical Formulation (cont.) Programming
Simplex Method
n
X Branch & Bound
Method
max vj x j (84) Numerical Example 1
x
j=1 Numerical Example 2
Strategy and Steps
Remarks
n
X How to Branch?
wj xj ≤ b, (85) Which Node to Select?
Rule of Fathoming
j=1 Software for ILP
3.86
Integer Linear
Solving the problem in this case Programming
Simplex Method
Remarks
v1 v2 vn How to Branch?
≥ ··· ≥ . (88) Which Node to Select?
w1 w2 wn Rule of Fathoming
Software for ILP
simplex method or any other LP algorithm to solve it, as its Simplex Method
Remarks
v1 v2 vn How to Branch?
≥ ··· ≥ . (88) Which Node to Select?
w1 w2 wn Rule of Fathoming
Software for ILP
Contents
Motivated Examples
Classification
Facility Location
Simplex Method
Remarks
How to Branch?
Which Node to Select?
Rule of Fathoming
Software for ILP
Problems and
Homeworks
3.88
Integer Linear
Remarks Programming
in the second one. But the opposite statement is not true. Facility Location
Simplex Method
Remarks
How to Branch?
Which Node to Select?
Rule of Fathoming
Software for ILP
Problems and
Homeworks
3.88
Integer Linear
Remarks Programming
in the second one. But the opposite statement is not true. Facility Location
is a proper subset of the feasible solutions of the second one. Simplex Method
Remarks
How to Branch?
Which Node to Select?
Rule of Fathoming
Software for ILP
Problems and
Homeworks
3.88
Integer Linear
Remarks Programming
in the second one. But the opposite statement is not true. Facility Location
is a proper subset of the feasible solutions of the second one. Simplex Method
upper bound of the optimal value of the Problem (84), (85), Numerical Example 2
Strategy and Steps
and (86). Exercise: 0-1 KP
Remarks
How to Branch?
Which Node to Select?
Rule of Fathoming
Software for ILP
Problems and
Homeworks
3.88
Integer Linear
Remarks Programming
in the second one. But the opposite statement is not true. Facility Location
is a proper subset of the feasible solutions of the second one. Simplex Method
upper bound of the optimal value of the Problem (84), (85), Numerical Example 2
Strategy and Steps
and (86). Exercise: 0-1 KP
• If the optimal solution of the Problem (84), (85), and (87) is Remarks
feasible in the Problem (84), (85), and (86) then it is the How to Branch?
Which Node to Select?
optimal solution of the latter problem as well. Rule of Fathoming
Software for ILP
Problems and
Homeworks
3.88
Integer Linear
Remarks Programming
in the second one. But the opposite statement is not true. Facility Location
is a proper subset of the feasible solutions of the second one. Simplex Method
upper bound of the optimal value of the Problem (84), (85), Numerical Example 2
Strategy and Steps
and (86). Exercise: 0-1 KP
• If the optimal solution of the Problem (84), (85), and (87) is Remarks
feasible in the Problem (84), (85), and (86) then it is the How to Branch?
Which Node to Select?
optimal solution of the latter problem as well. Rule of Fathoming
Software for ILP
• These properties are used in the course of the branch and Problems and
bound method intensively. Homeworks
3.88
Integer Linear
Sudoku Game Programming
Contents
Motivated Examples
Classification
Facility Location
Simplex Method
Some cells of subgrids are filled by digits in {1, 2, . . . , 9}. The How to Branch?
Which Node to Select?
objective is to fill remaining cells such that Rule of Fathoming
Software for ILP
• each column, each row, and each subgrid that compose the Problems and
Homeworks
grid, contain all of the digits from 1 to 9.
3.89
Integer Linear
Sudoku Game: Model Programming
1 if cell (i, j) contains k;
Variable xijk =
0 otherwise.
min 0T x Contents
x
Motivated Examples
subject to Classification
Facility Location
9
X Linear and Integer
xijk = 1, j, k = 1, . . . , 9, Linear Programs
9
X Branch & Bound
Method
xijk = 1, i, k = 1, . . . , 9, Numerical Example 1
Numerical Example 2
j=1
Strategy and Steps
9
X Exercise: 0-1 KP
xijk = 1, i, j = 1, . . . , 9, Remarks
How to Branch?
k=1 Which Node to Select?
Rule of Fathoming
3q 3p
X X Software for ILP
xijk = 1, k = 1, . . . , 9; p, q = 1, 2, 3, Problems and
Homeworks
j=3q+2 i=3p+2
Contents
Motivated Examples
Classification
Facility Location
Simplex Method
Remarks
How to Branch?
Which Node to Select?
Rule of Fathoming
Software for ILP
Problems and
Homeworks
3.91
Integer Linear
Vehicle Routing Programming
Problem statement
Let G = (V, E) be a complete graph, where V = 0, . . . , n is the
set of vertices and E is the set of edges.
• We assume that vertex 0 denotes the depot, whereas vertex Contents
i = 1, . . . , n is customer. Motivated Examples
with each edge of G that connects vertex i and j. Linear and Integer
Linear Programs
• Let S ⊆ V , we denote by r(S) the minimum number of Simplex Method
vehicles required for all customers in S. Branch & Bound
Method
Given a positive integer number k, the capacitated vehicle routing Numerical Example 1
Numerical Example 2
problem is asked for seeking k simple routes that minimizing a Strategy and Steps
cost function of the sum of cost of edges of the routes, and Exercise: 0-1 KP
(i) each route starts at the depot vertex, Which Node to Select?
Rule of Fathoming
(ii) each customer belongs exactly one route, Software for ILP
Problems and
(iii) the sum of demands of the vertices visited by a route is Homeworks
Motivated Examples
subject to Classification
Facility Location
X
xij = 1 for all j ∈ V \ {0}, Linear and Integer
Linear Programs
i∈V
X Simplex Method
xij = 1 for all i ∈ V \ {0}, Branch & Bound
Method
j∈V
Numerical Example 1
X Numerical Example 2
xi0 = k, Strategy and Steps
X Remarks
x0j = k, How to Branch?
Which Node to Select?
j∈V Rule of Fathoming
XX Software for ILP
xij ≥ r(S) for all S ⊆ V \ {0}, S 6= ∅, Problems and
/ j∈S
i∈S Homeworks
Contents
Motivated Examples
• Simplex Method: details. Classification
Facility Location
Simplex Method
Remarks
How to Branch?
Which Node to Select?
Rule of Fathoming
Software for ILP
Problems and
Homeworks
3.94
Integer Linear
Homeworks Programming
Contents
Motivated Examples
• Simplex Method: details. Classification
Facility Location
• Read carefully and do all Examples, Exercises in Chapter 7 of Linear and Integer
Linear Programs
[4]: F.R. Giordano, W.P. Fox and S.B. Horton, A First Course Simplex Method
in Mathematical Modeling, 5th ed., Cengage, 2014. Branch & Bound
Method
Numerical Example 1
Numerical Example 2
Strategy and Steps
Exercise: 0-1 KP
Remarks
How to Branch?
Which Node to Select?
Rule of Fathoming
Software for ILP
Problems and
Homeworks
3.94
Integer Linear
Homeworks Programming
Contents
Motivated Examples
• Simplex Method: details. Classification
Facility Location
• Read carefully and do all Examples, Exercises in Chapter 7 of Linear and Integer
Linear Programs
[4]: F.R. Giordano, W.P. Fox and S.B. Horton, A First Course Simplex Method
in Mathematical Modeling, 5th ed., Cengage, 2014. Branch & Bound
Method
• Do all Exercises in the file named "BT Chương 3.pdf". Numerical Example 1
Numerical Example 2
Strategy and Steps
Exercise: 0-1 KP
Remarks
How to Branch?
Which Node to Select?
Rule of Fathoming
Software for ILP
Problems and
Homeworks
3.94