Deterministic OR Models: Strong IP Formulations

C&O 370: Deterministic OR Models
Strong IP Formulations
Jochen Könemann
http://www.math.uwaterloo.ca/∼jochen
Jochen Könemann, March 19, 2007 CO 370 – Deterministic Operations Research Models - p. 1/45
Strong Formulations
● LP vs IP
● LP Relaxation
● Convex Hull
Valid Inequalities
Chvátal-Gomory Procedure
Cutting-Plane Algorithms
Guidelines for Strong Formulations
Gomory Cuts
LP vs IP
Strong Formulations
● LP vs IP
■ Good linear programming formulations have as few variables
● LP Relaxation
● Convex Hull
and constraints as possible.
Valid Inequalities Remember: Running time of LP solvers depends heavily on
Chvátal-Gomory Procedure number of variables and on number of constraints.
Gomory Cuts
LP vs IP
Strong Formulations
● LP vs IP
■ Good linear programming formulations have as few variables
● LP Relaxation
● Convex Hull
and constraints as possible.
Valid Inequalities Remember: Running time of LP solvers depends heavily on
Chvátal-Gomory Procedure number of variables and on number of constraints.
Cutting-Plane Algorithms ■ Different for IP!
Gomory Cuts ◆ Computational experiments suggest that the choice in
formulation crucially influences solution time and
sometimes solvability
◆ Feasible region of LP relaxation resembles convex hull of
feasible integer points closely
LP Relaxation
Strong Formulations
● LP vs IP
■ The important novelty over linear programs is that the
● LP Relaxation solution space is not any more convex.
● Convex Hull
Valid Inequalities
Gomory Cuts
LP Relaxation
Strong Formulations
● LP vs IP
■ The important novelty over linear programs is that the
● LP Relaxation solution space is not any more convex.
● Convex Hull
Valid Inequalities
■ Example
max 3x1 + 10x2 (IP)
Gomory Cuts
s.t. x1 + 4x2 ≤ 8
x1 + x2 ≤ 4
x1 , x2 ≥ 0
x1 , x2 integer
LP Relaxation
Strong Formulations
● LP vs IP
■ Geometric view:
● LP Relaxation
● Convex Hull
Valid Inequalities 5
4
Gomory Cuts
3
2
1
0
0 1 2 3 4 5
LP Relaxation
Strong Formulations
● LP vs IP
■ We obtain the linear programming relaxation of an integer
● LP Relaxation
● Convex Hull
program by dropping the integrality constraints
Valid Inequalities
Chvátal-Gomory Procedure 5
4
Gomory Cuts
3
2
1
0
0 1 2 3 4 5
Convex Hull
Strong Formulations
● LP vs IP
■ We have seen: optimal solution to LP relaxation is fractional.
● LP Relaxation
● Convex Hull
Can we write a different LP with the same set of feasible
integer solutions for which has an integral optimal solution?
Valid Inequalities
Gomory Cuts
Convex Hull
Strong Formulations
● LP vs IP
■ We have seen: optimal solution to LP relaxation is fractional.
● LP Relaxation
● Convex Hull
Can we write a different LP with the same set of feasible
integer solutions for which has an integral optimal solution?
Valid Inequalities
■ Yes! Let X be the set of all solutions to original IP. Then
define the convex hull of X as
X
Gomory Cuts n
CH(X) :={x ∈ R : x= λx̄ · x̄,
x̄∈X
X
λx̄ = 1
x̄∈X
λx̄ ≥ 0 ∀x̄ ∈ X}
Convex Hull
Strong Formulations
● LP vs IP
■ The convex hull CH(X) of feasible integer solutions X is the
● LP Relaxation
● Convex Hull
smallest polyhedron containing X:
Valid Inequalities
4
Gomory Cuts
3
2
1
0
0 1 2 3 4 5
Convex Hull
Strong Formulations
● LP vs IP
● LP Relaxation
● Convex Hull
Valid Inequalities
4
Gomory Cuts
3
2
1
0
0 1 2 3 4 5
■ If P is the feasible region of an LP relaxation then CH ⊆ P
Convex Hull
Strong Formulations
● LP vs IP
● LP Relaxation
● Convex Hull
Valid Inequalities
4
Gomory Cuts
3
2
1
0
0 1 2 3 4 5
■ If P is the feasible region of an LP relaxation then CH ⊆ P
■ Each vertex of the convex hull corresponds to an integer
solution!
Strong Formulations
Valid Inequalities
● Introduction
● Valid Inequalities
● Finding Valid Inequalities
Valid Inequalities
Gomory Cuts
Introduction
Strong Formulations ■ In this class, we are interested in integer programs of the

Valid Inequalities
● Introduction
following general form:
max{cT x : x ∈ X} (IP)
Cutting-Plane Algorithms and X = {x : Ax ≤ b, X ∈ Zn+ }.

Gomory Cuts
Introduction

Valid Inequalities
● Introduction

Gomory Cuts ■ We have seen: To be able to solve (IP) efficiently, we want
{x : Ax ≤ b} to be close to the convex hull CH(X) of the
feasible integer solutions.
Introduction

Valid Inequalities
● Introduction

■ Fact: There is Ae and eb such that
e ≤ eb}
CH(X) = {x : Ax
Introduction

Valid Inequalities
● Introduction

■ Fact: There is Ae and eb such that
e ≤ eb}
CH(X) = {x : Ax
■ e may be be huge! We will not be able to generate a

A
description of the convex hull in polynomial time for all
problems.
Valid Inequalities
Strong Formulations ■ A more tractable task: Find valid inequalities for

Valid Inequalities
● Introduction
X := {x : Ax ≤ b, x integer}.
● Finding Valid Inequalities An inequality
Chvátal-Gomory Procedure πx ≤ π0
is valid for X if is satisfied for all x ∈ X.
Gomory Cuts
Valid Inequalities
Strong Formulations ■ A more tractable task: Find valid inequalities for

Valid Inequalities
● Introduction
X := {x : Ax ≤ b, x integer}.
● Finding Valid Inequalities An inequality
Chvátal-Gomory Procedure πx ≤ π0
is valid for X if is satisfied for all x ∈ X.
Gomory Cuts
■ Recall the IP from last class:
max 3x1 + 10x2 (IP)

s.t. x1 + 4x2 ≤ 8
x1 + x2 ≤ 4
x1 , x2 ≥ 0
x1 , x2 integer
Introduction
Geometric view of LP relaxation:

Strong Formulations
5
Valid Inequalities
● Introduction
● Finding Valid Inequalities 4
Cutting-Plane Algorithms 3
Gomory Cuts
0
0 1 2 3 4 5
Introduction
Geometric view of LP relaxation:

Strong Formulations
5
Valid Inequalities
● Introduction
Gomory Cuts
0
0 1 2 3 4 5
Optimum solution: x1 = 8/3, x2 = 4/3.

Can you find a good valid inequality for this example?
Introduction
Geometric view:
Strong Formulations
5
Valid Inequalities
● Introduction
Gomory Cuts
0
0 1 2 3 4 5
Inequality x1 /3 + x2 ≤ 2 is valid for (IP)!
Introduction
Geometric view:
Strong Formulations
5
Valid Inequalities
● Introduction
Gomory Cuts
0
0 1 2 3 4 5
Inequality x1 /3 + x2 ≤ 2 is valid for (IP)!

Its addition to existing inequalities yields the convex hull of all
Introduction
Strong Formulations ■ In example, inequality x1 /3 + x2 ≤ 2 was useful as its

Valid Inequalities
● Introduction
addition to original constraints yielded CH(X).
● Finding Valid Inequalities Remember last class: Adding this inequality gives us the
Chvátal-Gomory Procedure optimum integer solution at once! No branch and bound
Cutting-Plane Algorithms search necessary!
Gomory Cuts
Introduction

Valid Inequalities
● Introduction
Gomory Cuts ■ What are the useful valid inequalities in general?
Introduction

Valid Inequalities
● Introduction
Gomory Cuts ■ What are the useful valid inequalities in general?
■ How do we find these inequalities? Are there systematic
ways?
Finding Valid Inequalities
Strong Formulations ■ Another set of integer solutions:

Valid Inequalities
● Introduction
X := {x ∈ {0, 1}5 : 3x1 − 4x2 + 2x3 − 3x4 + x5 ≤ −2}
Gomory Cuts

Valid Inequalities
● Introduction
X := {x ∈ {0, 1}5 : 3x1 − 4x2 + 2x3 − 3x4 + x5 ≤ −2}
Cutting-Plane Algorithms ■ Can there be a solution with x2 = x4 = 0?

Gomory Cuts

Valid Inequalities
● Introduction
X := {x ∈ {0, 1}5 : 3x1 − 4x2 + 2x3 − 3x4 + x5 ≤ −2}

Gomory Cuts
■ No! This implies that 3x1 + 2x3 + x5 ≤ −2. That is
impossible since all variables are in {0, 1}.

Valid Inequalities
● Introduction
X := {x ∈ {0, 1}5 : 3x1 − 4x2 + 2x3 − 3x4 + x5 ≤ −2}

Gomory Cuts
■ So all feasible solutions must satisfy
x2 + x4 ≥ 1

Valid Inequalities
● Introduction
X := {x ∈ {0, 1}5 : 3x1 − 4x2 + 2x3 − 3x4 + x5 ≤ −2}

Gomory Cuts
x2 + x4 ≥ 1
■ How about x1 = 1 and x2 = 0?

Valid Inequalities
● Introduction
X := {x ∈ {0, 1}5 : 3x1 − 4x2 + 2x3 − 3x4 + x5 ≤ −2}

Gomory Cuts
x2 + x4 ≥ 1

■ This implies 3 + 2x3 − 3x4 + x5 ≥ 3 − 3 = 0.

Valid Inequalities
● Introduction
X := {x ∈ {0, 1}5 : 3x1 − 4x2 + 2x3 − 3x4 + x5 ≤ −2}

Gomory Cuts
x2 + x4 ≥ 1

■ This implies 3 + 2x3 − 3x4 + x5 ≥ 3 − 3 = 0.
■ Implies: Whenever x1 = 1 then x2 must have value 1 as well.
Valid inequality:
x1 ≤ x2
Strong Formulations ■ Another IP:

Valid Inequalities
● Introduction
max(x − 5y) s.t. (x, y) ∈ X
● Finding Valid Inequalities with X := {(x, y) : x ≤ 100 · y, 0 ≤ x ≤ 5, y ∈ {0, 1}}
Gomory Cuts

Valid Inequalities
● Introduction
max(x − 5y) s.t. (x, y) ∈ X
Chvátal-Gomory Procedure ■ What is the LP relaxation of this IP?
Gomory Cuts

Valid Inequalities
● Introduction
max(x − 5y) s.t. (x, y) ∈ X
■ LP relaxation of above IP:
Gomory Cuts
max(x − 5y) s.t. (x, y) ∈ X

with X := {(x, y) : x ≤ 100 · y, 0 ≤ x ≤ 5, 0 ≤ y ≤ 1}

Valid Inequalities
● Introduction
max(x − 5y) s.t. (x, y) ∈ X
Gomory Cuts
max(x − 5y) s.t. (x, y) ∈ X

with X := {(x, y) : x ≤ 100 · y, 0 ≤ x ≤ 5, 0 ≤ y ≤ 1}
■ This relaxation is bad! The LP optimum is x = 5, y = .05 with
value 5 − .25 = 4.75.
IP optimum has value 0!

Valid Inequalities
● Introduction
max(x − 5y) s.t. (x, y) ∈ X
Gomory Cuts
max(x − 5y) s.t. (x, y) ∈ X

with X := {(x, y) : x ≤ 100 · y, 0 ≤ x ≤ 5, 0 ≤ y ≤ 1}
■ This relaxation is bad! The LP optimum is x = 5, y = .05 with
value 5 − .25 = 4.75.
IP optimum has value 0!
■ x ≤ 100 · y is a big-M constraint where the M is chosen
poorly.
Is there a good valid inequality? Can you find a better M?

Valid Inequalities
● Introduction
max(x − 5y) s.t. (x, y) ∈ X
Gomory Cuts

Valid Inequalities
● Introduction
max(x − 5y) s.t. (x, y) ∈ X
Chvátal-Gomory Procedure ■ The inequality
x ≤ 5y
Gomory Cuts
is valid! Variable x can only be positive if y = 1. Whenever
y = 1, x must have value at most 5.

Valid Inequalities
● Introduction
max(x − 5y) s.t. (x, y) ∈ X
x ≤ 5y
Gomory Cuts
is valid! Variable x can only be positive if y = 1. Whenever
y = 1, x must have value at most 5.
■ CH(X) = {(x, y) : x ≤ 5y, 0 ≤ y ≤ 1}.
Strong Formulations ■ One more example:

Valid Inequalities
● Introduction
X := {x ∈ Z4+ : 13x1 + 20x2 + 11x3 + 6x4 ≥ 72}
Gomory Cuts

Valid Inequalities
● Introduction
X := {x ∈ Z4+ : 13x1 + 20x2 + 11x3 + 6x4 ≥ 72}

Gomory Cuts
α · (13x1 + 20x2 + 11x3 + 6x4 ) ≥ α · 72
is valid for X for all α ≥ 0.

Valid Inequalities
● Introduction
X := {x ∈ Z4+ : 13x1 + 20x2 + 11x3 + 6x4 ≥ 72}

Gomory Cuts
α · (13x1 + 20x2 + 11x3 + 6x4 ) ≥ α · 72
is valid for X for all α ≥ 0.
1
■ Valid inequality for α = 11 :
13 20 11 6 72
x1 + x2 + x3 + x4 ≥
11 11 11 11 11
1
Strong Formulations
Valid Inequalities
● Introduction
13 20 11 6 72
● Finding Valid Inequalities x1 + x2 + x3 + x4 ≥
11 11 11 11 11
■ Rounding up all coefficients on left-hand side does not affect
Gomory Cuts
validity:
72
2x1 + 2x2 + x3 + x4 ≥
11
1
Strong Formulations
Valid Inequalities
● Introduction
13 20 11 6 72
● Finding Valid Inequalities x1 + x2 + x3 + x4 ≥
11 11 11 11 11
■ Rounding up all coefficients on left-hand side does not affect
Gomory Cuts
validity:
72
2x1 + 2x2 + x3 + x4 ≥
11
■ Left-hand side is integer! Can round up right-hand side:

2x1 + 2x2 + x3 + x4 ≥ 7
This inequality is valid for original set X.
Strong Formulations
Valid Inequalities
● Valid Inequalities for LP
● Strengthening Inequalities
● CG Procedure
● Discussion Chvátal-Gomory Procedure
Gomory Cuts
Valid Inequalities for LP
Strong Formulations ■ Back to IP example from last class:

Valid Inequalities
max 3x1 + 10x2 (IP)
● CG Procedure
● Discussion s.t. x ∈ P
P = {(x1 , x2 ) : x1 + 4x2 ≤ 8, (1)
Gomory Cuts
x1 + x2 ≤ 4, x ≥ 0}
x1 , x2 integer

Valid Inequalities
max 3x1 + 10x2 (IP)
● CG Procedure
P = {(x1 , x2 ) : x1 + 4x2 ≤ 8, (1)
Gomory Cuts
x1 + x2 ≤ 4, x ≥ 0}
x1 , x2 integer
■ Notice that the inequality

u1 (x1 + 4x2 ) + u2 (x1 + x2 ) ≤ 8u1 + 4u2
is valid for P for any u1 , u2 ≥ 0

Valid Inequalities
max 3x1 + 10x2 (IP)
● CG Procedure
P = {(x1 , x2 ) : x1 + 4x2 ≤ 8, (1)
Gomory Cuts
x1 + x2 ≤ 4, x ≥ 0}
x1 , x2 integer
■ Notice that the inequality

u1 (x1 + 4x2 ) + u2 (x1 + x2 ) ≤ 8u1 + 4u2
■ In fact: Any valid inequality for P can be obtained in this way.
Strong Formulations ■ Notice that the inequality

Valid Inequalities
u1 (x1 + 4x2 ) + u2 (x1 + x2 ) ≤ 8u1 + 4u2
● CG Procedure
● Discussion
■ Let’s try this with u1 = 2/3, u2 = 1/3:
Gomory Cuts 2 1 16 4
(x1 + 4x2 ) + (x1 + x2 ) ≤ +
3 3 3 3
Strong Formulations ■ Notice that the inequality

Valid Inequalities
u1 (x1 + 4x2 ) + u2 (x1 + x2 ) ≤ 8u1 + 4u2
● CG Procedure
● Discussion
■ Let’s try this with u1 = 2/3, u2 = 1/3:
Gomory Cuts 2 1 16 4
(x1 + 4x2 ) + (x1 + x2 ) ≤ +
3 3 3 3
■ . . . and this is equivalent to

20
x1 + 3x2 ≤
3
Geometric view:
Strong Formulations
5
Valid Inequalities
● Valid Inequalities for LP 4
● CG Procedure
● Discussion
3
Gomory Cuts
0
0 1 2 3 4 5
Red line is the inequality x1 + 3x2 ≤ 20 3 .

It is clearly satisfied by all points in P .
Strengthening Inequalities
Strong Formulations ■ Have seen that inequality

Valid Inequalities
20
x1 + 3x2 ≤ (1)
3
● CG Procedure
● Discussion is valid for P .
Gomory Cuts

Valid Inequalities
20
x1 + 3x2 ≤ (1)
3
● CG Procedure
Cutting-Plane Algorithms ■ Every feasible solution for the LP relaxation satisfies this
Gomory Cuts
inequality.
We haven’t gained anything, have we?

Valid Inequalities
20
x1 + 3x2 ≤ (1)
3
● CG Procedure
Gomory Cuts
inequality.
■ Well, if x1 , x2 are integer, then the left-hand side of (1) is
integer.

Valid Inequalities
20
x1 + 3x2 ≤ (1)
3
● CG Procedure
Gomory Cuts
inequality.
integer.
■ For every feasible integer solution in X, the left-hand side of
(1) has value at most ⌊20/3⌋ = 6.

Valid Inequalities
20
x1 + 3x2 ≤ (1)
3
● CG Procedure
Gomory Cuts
inequality.
integer.
■ Inequality x1 + 3x2 ≤ 6 is valid for CH(X) but not valid for P .

Valid Inequalities
20
x1 + 3x2 ≤ (1)
3
● CG Procedure
Gomory Cuts
inequality.
integer.
■ Inequality x1 + 3x2 ≤ 6 is valid for CH(X) but not valid for P .
■ We gained strength over the LP relaxation of (IP).
Geometric view:
Strong Formulations
5
Valid Inequalities
● Valid Inequalities for LP 4
● CG Procedure
● Discussion
3
Gomory Cuts
0
0 1 2 3 4 5
Red line is the inequality x1 + 3x2 ≤ 6.

Adding this inequality gives the convex hull CH(X) of all integer
solutions in X.
CG Procedure
Strong Formulations ■ Suppose you have a valid inequality for the polyhedron P
Valid Inequalities
given by the relaxation of your integer program:
● Valid Inequalities for LP n
X
● CG Procedure aj xj ≤ b
● Discussion
j=1
Gomory Cuts How can we strengthen this inequality to lead to a valid

inequality for X?
CG Procedure
Strong Formulations ■ The Chvátal-Gomory procedure:

Valid Inequalities
1. xi is non-negative for all i ∈ {1, . . . , n}. So the inequality
X
● CG Procedure ⌊aj ⌋xj ≤ b (1)
● Discussion
j=1
Gomory Cuts is valid for P as well.
CG Procedure
Strong Formulations ■ The Chvátal-Gomory procedure:

Valid Inequalities
1. xi is non-negative for all i ∈ {1, . . . , n}. So the inequality
X
● CG Procedure ⌊aj ⌋xj ≤ b (1)
● Discussion
j=1
Gomory Cuts is valid for P as well.

2. The left-hand side of (1) is integer for (x1 , . . . , xn ) ∈ X.
Therefore,
n
X
⌊aj ⌋xj ≤ ⌊b⌋
j=1
is a valid inequality for X.
Discussion
Strong Formulations ■ Notice that the linear program

Valid Inequalities
max 3x1 + 10x2
● CG Procedure
● Discussion s.t. x1 + 4x2 ≤ 8
x1 + x2 ≤ 4
Gomory Cuts
x1 + 3x2 ≤ 6
x1 , x2 ≥ 0
describes the convex hull CH(X) of all feasible integer

solutions for the original LP.
Discussion

Valid Inequalities
max 3x1 + 10x2
● CG Procedure
x1 + x2 ≤ 4
Gomory Cuts
x1 + 3x2 ≤ 6
x1 , x2 ≥ 0

■ Solving this LP gives us an integer solution right away. No
need for branch and bound!
Discussion

Valid Inequalities
max 3x1 + 10x2
● CG Procedure
x1 + x2 ≤ 4
Gomory Cuts
x1 + 3x2 ≤ 6
x1 , x2 ≥ 0

■ Solving this LP gives us an integer solution right away. No
need for branch and bound!
■ CG Procedure is a tool to strengthen valid inequalities for the
LP relaxation.
Discussion
Strong Formulations ■ Is adding more valid inequalities useful?

Valid Inequalities
● CG Procedure
● Discussion
Gomory Cuts
Discussion

Valid Inequalities
■ Advantages: More strong inequalities lead to a better
● Valid Inequalities for LP approximation of CH(X), the convex hull of integer solutions.
● CG Procedure
● Discussion
Hopefully this reduces the size of our branch & bound tree.
Gomory Cuts
Discussion

Valid Inequalities
● CG Procedure
● Discussion
Cutting-Plane Algorithms ■ Disadvantages: The size of the LP formulation may grow
Gomory Cuts quite dramatically. We need to solve an LP at each node in
the branch & bound tree.
Discussion

Valid Inequalities
● CG Procedure
● Discussion
Cutting-Plane Algorithms ■ Disadvantages: The size of the LP formulation may grow
Gomory Cuts quite dramatically. We need to solve an LP at each node in
the branch & bound tree.
■ There is no good answer here. Need to experiment!
Strong Formulations
Valid Inequalities
● General Framework
● General Framework Cutting-Plane Algorithms
Gomory Cuts
General Framework
Strong Formulations ■ Have seen how to find strong valid inequalities for a given IP.
Valid Inequalities
Gomory Cuts
General Framework
Valid Inequalities
■ Also know that there maybe too many such inequalities to
write them all out. What can we do?
Gomory Cuts
General Framework
Valid Inequalities
■ Also know that there maybe too many such inequalities to
write them all out. What can we do?
● General Framework ■ Cutting-Plane algorithms solve the LP relaxation of the given
integer program and add strong valid inequalities one by one.
Gomory Cuts
General Framework
Strong Formulations ■ Suppose you want to solve integer program

Valid Inequalities
max cT x (IP)
● General Framework s.t. x ∈ P0
Gomory Cuts
x integer
for some polyhedron P0 .
General Framework
Strong Formulations ■ Suppose you want to solve integer program

Valid Inequalities
max cT x (IP)
● General Framework s.t. x ∈ P0
Gomory Cuts
x integer
for some polyhedron P0 .

■ Solve the LP relaxation
max cT x (LP)
s.t. x ∈ P0
of (IP). Let x0 be the solution.
General Framework
Strong Formulations ■ We’re done if x0 is integral. Otherwise find a valid inequality

Valid Inequalities
a0 x ≤ b0
● General Framework for X such that
a0 x0 > b0
Gomory Cuts
General Framework

Valid Inequalities
a0 x ≤ b0
a0 x0 > b0
Gomory Cuts
■ Add this inequality to P0 :

P1 = P0 ∩ {x : a0 x ≤ b0 }
General Framework

Valid Inequalities
a0 x ≤ b0
a0 x0 > b0
Gomory Cuts

P1 = P0 ∩ {x : a0 x ≤ b0 }
■ Resolve LP relaxation with P0 replaced by P1 .
General Framework

Valid Inequalities
a0 x ≤ b0
a0 x0 > b0
Gomory Cuts

P1 = P0 ∩ {x : a0 x ≤ b0 }
■ Resolve LP relaxation with P0 replaced by P1 .

■ Continue this way until integral solution is found.
Strong Formulations
Valid Inequalities
Gomory Cuts
● The Idea
Gomory Cuts
● Gomory Cuts
● An Example
● Discussion
The Idea
Strong Formulations ■ Consider general IP of the form

Valid Inequalities
max{cx : Ax ≤ b, x ≥ 0 and integer}
Gomory Cuts
● The Idea
● Gomory Cuts
● An Example
● Discussion
The Idea

Valid Inequalities
Gomory Cuts
● The Idea ■ Bring to canonical form by adding slack variables:
● Gomory Cuts
● An Example
● Discussion
max{cx : Ax + Is = b, x ≥ 0 and integer, s ≥ 0}

Observe that slack variables must take on integral values if
A, b are integer!
The Idea

Valid Inequalities
Gomory Cuts
● The Idea ■ Bring to canonical form by adding slack variables:
● Gomory Cuts
● An Example
● Discussion
max{cx : Ax + Is = b, x ≥ 0 and integer, s ≥ 0}

Observe that slack variables must take on integral values if
A, b are integer!
■ We can therefore assume that the slack variables were part
of the original set of variables:
max{cx : Ax = b, x ≥ 0 and integer} (IP)
The Idea
Strong Formulations ■ We can therefore assume that the slack variables were part
Valid Inequalities
Gomory Cuts
● The Idea
● Gomory Cuts
● An Example ■ Solve the linear programming relaxation of (IP) via Simplex.
● Discussion
The Idea
Strong Formulations ■ We can therefore assume that the slack variables were part
Valid Inequalities
Gomory Cuts
● The Idea
● Gomory Cuts
● An Example ■ Solve the linear programming relaxation of (IP) via Simplex.
● Discussion
■ Gives a final tableau of the form
BV x1 ··· xj ··· xi ··· xn Value
z c1 cj ci cn z
.. .. .. .. ..
. . . 0 . .
xi ai1 aij 1 ain bi
.. .. .. .. ..
. . . 0 . .
The Idea
Strong Formulations ■ Final tableau of the form

Valid Inequalities
z c1 cj ci cn z
Gomory Cuts
● The Idea
.. .. .. .. ..
● Gomory Cuts
● An Example
. . . 0 . .
● Discussion
xi ai1 aij 1 ain bi
.. .. .. .. ..
. . . 0 . .
■ The optimal basis is B = {1, . . . , m} and the non-basis is
N = {1, . . . , n} \ B.
The Idea
Strong Formulations ■ Final tableau of the form

Valid Inequalities
z c1 cj ci cn z
Gomory Cuts
● The Idea
.. .. .. .. ..
● Gomory Cuts
● An Example
. . . 0 . .
● Discussion
xi ai1 aij 1 ain bi
.. .. .. .. ..
. . . 0 . .
■ The optimal basis is B = {1, . . . , m} and the non-basis is
N = {1, . . . , n} \ B.
■ Row of xi corresponds to:
X
xi + aij xj = bi
j∈N
Any feasible solution to (IP) must satisfy this equation!

Gomory Cuts
Strong Formulations ■ Row of xi corresponds to:

Valid Inequalities
X
Chvátal-Gomory Procedure xi + aij xj = bi (1)
Cutting-Plane Algorithms j∈N
Gomory Cuts
● The Idea Any feasible solution to (IP) must satisfy this equation!
● Gomory Cuts
● An Example
● Discussion
Gomory Cuts

Valid Inequalities
X
Gomory Cuts
● Gomory Cuts
● An Example
● Discussion
■ Assume that value bi of xi is not integer
Gomory Cuts

Valid Inequalities
X
Gomory Cuts
● Gomory Cuts
● An Example
● Discussion
■ Use Chvátal-Gomory procedure and conclude that any
feasible solution to (IP) must also satisfy
X
xi + ⌊aij ⌋xj ≤ ⌊bi ⌋
j∈N
Gomory Cuts

Valid Inequalities
X
Gomory Cuts
● Gomory Cuts
● An Example
● Discussion
■ Use Chvátal-Gomory procedure and conclude that any
feasible solution to (IP) must also satisfy
X
xi + ⌊aij ⌋xj ≤ ⌊bi ⌋
j∈N
■ From (1):
X
xi = bi − aij xj
j∈N
Gomory Cuts
Strong Formulations ■ Any feasible solution to (IP) must also satisfy

Valid Inequalities
X
Chvátal-Gomory Procedure xi + ⌊aij ⌋xj ≤ ⌊bi ⌋ (1)
Gomory Cuts
● The Idea
■ . . . and X
● Gomory Cuts
● An Example
xi = bi − aij xj (2)
● Discussion
j∈N
■ Combining (1) and (2) leads to a new valid inequality for (IP):
X
(aij − ⌊aij ⌋)xj ≥ bi − ⌊bi ⌋ (3)
j∈N
Gomory Cuts
Strong Formulations ■ Any feasible solution to (IP) must also satisfy

Valid Inequalities
X
Chvátal-Gomory Procedure xi + ⌊aij ⌋xj ≤ ⌊bi ⌋ (1)
Gomory Cuts
● The Idea
■ . . . and X
● Gomory Cuts
● An Example
xi = bi − aij xj (2)
● Discussion
j∈N
■ Combining (1) and (2) leads to a new valid inequality for (IP):
X
(aij − ⌊aij ⌋)xj ≥ bi − ⌊bi ⌋ (3)
j∈N
■ Notice that current optimum solution x does not satisfy (1) as

xj = 0 for all j ∈ N .
x therefore does not satisfy (3) either!
Gomory Cuts
Strong Formulations ■ The new valid inequality is called a Gomory Cut:

Valid Inequalities
X
Chvátal-Gomory Procedure (aij − ⌊aij ⌋)xj ≥ bi − ⌊bi ⌋
Gomory Cuts
● The Idea
● Gomory Cuts
● An Example
● Discussion
Gomory Cuts

Valid Inequalities
X
Gomory Cuts
● The Idea
■ Add this to optimum tableau and use dual simplex to
● Gomory Cuts
● An Example
re-optimize!
● Discussion
Gomory Cuts

Valid Inequalities
X
Gomory Cuts
● The Idea
■ Add this to optimum tableau and use dual simplex to
● Gomory Cuts
● An Example
re-optimize!
● Discussion
■ Repeat until optimum solution is integral.
An Example

Valid Inequalities
max 3x1 + 10x2 (IP)
Gomory Cuts
● The Idea
s.t. x ∈ P
● Gomory Cuts
● An Example P = {(x1 , x2 ) : x1 + 4x2 ≤ 8, (2)
● Discussion
x1 + x2 ≤ 4, x ≥ 0}
x1 , x2 integer
An Example

Valid Inequalities
max 3x1 + 10x2 (IP)
Gomory Cuts
● The Idea
s.t. x ∈ P
● Gomory Cuts
● An Example P = {(x1 , x2 ) : x1 + 4x2 ≤ 8, (2)
● Discussion
x1 + x2 ≤ 4, x ≥ 0}
x1 , x2 integer
■ Final tableau:
BV x1 x2 s1 s2 Value
z 0 0 7/3 2/3 64/3
x2 0 1 1/3 -1/3 4/3
x1 1 0 -1/3 4/3 8/3
An Example
Strong Formulations BV x1 x2 s1 s2 Value

Valid Inequalities
z 0 0 7/3 2/3 64/3
x2 0 1 1/3 -1/3 4/3
Gomory Cuts
● The Idea x1 1 0 -1/3 4/3 8/3
● Gomory Cuts
● An Example
● Discussion
■ Both variables x1 and x2 are fractional. What is the Gomory
cut for x1 row?
An Example
Strong Formulations BV x1 x2 s1 s2 Value

Valid Inequalities
z 0 0 7/3 2/3 64/3
x2 0 1 1/3 -1/3 4/3
Gomory Cuts
● The Idea x1 1 0 -1/3 4/3 8/3
● Gomory Cuts
● An Example
● Discussion
■ Both variables x1 and x2 are fractional. What is the Gomory
cut for x1 row?
■ Gomory cut formula is
X
(aij − ⌊aij ⌋)xj ≥ bi − ⌊bi ⌋
j∈N
and therefore cut is

2 1 2
s1 + s2 ≥
3 3 3
An Example
Strong Formulations ■ Add new slack-variable s3 and row

Valid Inequalities
2 1 2
− s1 − s2 + s3 = −
3 3 3
Gomory Cuts
● The Idea
to final tableau.
● Gomory Cuts
● An Example
● Discussion
An Example

Valid Inequalities
2 1 2
− s1 − s2 + s3 = −
3 3 3
Gomory Cuts
● The Idea
to final tableau.
● Gomory Cuts
● An Example
■ Tableau becomes:
● Discussion
BV x1 x2 s1 s2 s3 Value
z 0 0 7/3 2/3 0 64/3
x2 0 1 1/3 -1/3 0 4/3
x1 1 0 -1/3 4/3 0 8/3
s3 0 0 -2/3 -1/3 1 -2/3
An Example

Valid Inequalities
2 1 2
− s1 − s2 + s3 = −
3 3 3
Gomory Cuts
● The Idea
to final tableau.
● Gomory Cuts
● An Example
■ Tableau becomes:
● Discussion
z 0 0 7/3 2/3 0 64/3
x2 0 1 1/3 -1/3 0 4/3
x1 1 0 -1/3 4/3 0 8/3
s3 0 0 -2/3 -1/3 1 -2/3
■ Use dual simplex to remove infeasibility.
An Example
Strong Formulations
Valid Inequalities
Chvátal-Gomory Procedure z 0 0 1 0 2 20
Gomory Cuts
x2 0 1 1 0 -1 2
● The Idea
● Gomory Cuts x1 1 0 -3 0 4 0
● An Example
● Discussion s2 0 0 2 1 -3 2
■ All variables have integer values. Done!
Discussion
Strong Formulations ■ We were really lucky with the Gomory cut we chose but . . .
Valid Inequalities ◆ . . . in practice we’re often not that lucky and have to go
through many iterations.
Cutting-Plane Algorithms ◆ . . . fractional coefficients cause numerical instability.
Gomory Cuts
● The Idea
● Gomory Cuts
● An Example
● Discussion
Discussion
Gomory Cuts
● The Idea
■ There are often better cuts to add than Gomory cuts
● Gomory Cuts
◆ Chvàtal-Gomory cuts, specially tailored cuts, . . .
● An Example
● Discussion
Discussion
Gomory Cuts
● The Idea
■ There are often better cuts to add than Gomory cuts
● Gomory Cuts
◆ Chvàtal-Gomory cuts, specially tailored cuts, . . .
● An Example
● Discussion
■ Cuts are often used in Branch & Bound
◆ Add cuts while you go and reduce B&B tree size.

Deterministic OR Models: Strong IP Formulations

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Deterministic OR Models: Strong IP Formulations

Uploaded by

Copyright:

Available Formats

C&O 370: Deterministic OR Models

Strong Formulations ■ In this class, we are interested in integer programs of the

Cutting-Plane Algorithms and X = {x : Ax ≤ b, X ∈ Zn+ }.

Strong Formulations ■ In this class, we are interested in integer programs of the

Cutting-Plane Algorithms and X = {x : Ax ≤ b, X ∈ Zn+ }.

Strong Formulations ■ In this class, we are interested in integer programs of the

Cutting-Plane Algorithms and X = {x : Ax ≤ b, X ∈ Zn+ }.

Strong Formulations ■ In this class, we are interested in integer programs of the

Cutting-Plane Algorithms and X = {x : Ax ≤ b, X ∈ Zn+ }.

■ e may be be huge! We will not be able to generate a

Strong Formulations ■ A more tractable task: Find valid inequalities for

Strong Formulations ■ A more tractable task: Find valid inequalities for

max 3x1 + 10x2 (IP)

Geometric view of LP relaxation:

Geometric view of LP relaxation:

Optimum solution: x1 = 8/3, x2 = 4/3.

Inequality x1 /3 + x2 ≤ 2 is valid for (IP)!

Inequality x1 /3 + x2 ≤ 2 is valid for (IP)!

Strong Formulations ■ In example, inequality x1 /3 + x2 ≤ 2 was useful as its

Strong Formulations ■ In example, inequality x1 /3 + x2 ≤ 2 was useful as its

Strong Formulations ■ In example, inequality x1 /3 + x2 ≤ 2 was useful as its

Strong Formulations ■ Another set of integer solutions:

Strong Formulations ■ Another set of integer solutions:

Cutting-Plane Algorithms ■ Can there be a solution with x2 = x4 = 0?

Strong Formulations ■ Another set of integer solutions:

Cutting-Plane Algorithms ■ Can there be a solution with x2 = x4 = 0?

Strong Formulations ■ Another set of integer solutions:

Cutting-Plane Algorithms ■ Can there be a solution with x2 = x4 = 0?

Strong Formulations ■ Another set of integer solutions:

Cutting-Plane Algorithms ■ Can there be a solution with x2 = x4 = 0?

■ How about x1 = 1 and x2 = 0?

Strong Formulations ■ Another set of integer solutions:

Cutting-Plane Algorithms ■ Can there be a solution with x2 = x4 = 0?

■ How about x1 = 1 and x2 = 0?

Strong Formulations ■ Another set of integer solutions:

Cutting-Plane Algorithms ■ Can there be a solution with x2 = x4 = 0?

■ How about x1 = 1 and x2 = 0?

Strong Formulations ■ Another IP:

Strong Formulations ■ Another IP:

Strong Formulations ■ Another IP:

max(x − 5y) s.t. (x, y) ∈ X

Strong Formulations ■ Another IP:

max(x − 5y) s.t. (x, y) ∈ X

Strong Formulations ■ Another IP:

max(x − 5y) s.t. (x, y) ∈ X

Strong Formulations ■ Another IP:

Strong Formulations ■ Another IP:

Strong Formulations ■ Another IP:

Strong Formulations ■ One more example:

Strong Formulations ■ One more example:

Chvátal-Gomory Procedure ■ The inequality

Strong Formulations ■ One more example:

Chvátal-Gomory Procedure ■ The inequality

■ Left-hand side is integer! Can round up right-hand side:

Strong Formulations ■ Back to IP example from last class:

Strong Formulations ■ Back to IP example from last class:

■ Notice that the inequality

Strong Formulations ■ Back to IP example from last class:

■ Notice that the inequality

Strong Formulations ■ Notice that the inequality

Strong Formulations ■ Notice that the inequality

■ . . . and this is equivalent to