Lecture 6: Cutting Plane Method

and Strong Valid Inequalities

(3 units)
Outline

Valid inequality

Gomorys cutting plane method

Polyhedra, faces and facets

Cover inequalities for 0-1 knapsack set

Lifting and separation of cover inequalities

Branch-and-cut algorithm
1 / 35
Valid Inequality

Let X = {x | Ax b, x Z
n
+
}. Consider the following linear
program:
(CIP) min c
T
x
s.t. x conv(X).
Then (IP) is equivalent to (CIP).

Question: How to describe or approximate

conv(X) = {x |

Ax

b, x 0}?

An inequality
T
x
0
is said to be a valid inequality for X
if
T
x
0
for all x X.
2 / 35

Question:
(1) How to nd good or useful valid inequality?
(2) How to use valid inequality to solve an integer program?

Chvatal-Gomory procedure for VI. X = {x Z

n
+
| Ax b},
A = (a
1
, . . . , a
n
).
(i) (surrogate)

n
j =1

T
a
j
x
j

T
b, 0.
(ii) (round o)

n
j =1

T
a
j
x
j

T
b.
(iii)

n
j =1

T
a
j
x
j

T
b.

Theorem: every valid inequality of X can be obtained by

applying Chvatal-Gomory procedure for a nite number of
times.
3 / 35
Gomorys Cutting Plane Procedure for IP

Integer programming:
min{c
T
x | x Z
n
+
, Ax = b}.

Create the valid inequalities (cutting planes) directly from the

simplex tableau

Given an (optimal) LP basis B, write the (pure) IP as

min c
T
B
B
1
b +

j NB
c
j
x
j
s.t. (x
B
)
i
+

j NB
a
ij
x
j
=

b
i
, i = 1, 2, ...m,
x
j
Z
1
+
, j = 1, 2, ...n,
NB is the set of nonbasic variables.

c
j
0, j NB,

b
i
0, i = 1, . . . , m.
4 / 35

If the LP solution is not integral, then there exists some row i

with

b
i
Z.

The C-G cut for row i is

(x
B
)
i
+

j NB
a
ij
x
j

b
i
.

Substitute for (x
B
)
i
to get

j NB
(a
ij
a
ij
)x
j


b
i

b
i
.

Let f
ij
= a
ij
a
ij
, f
j
=

b
i

b
i
.

j NB
f
ij
x
j
f
i
.

Since the LP optimal solution x

j
= 0 for j NB, and
0 f
ij
< 1, 0 < f
i
< 1, this inequality cut o x

!
5 / 35

Example 1
min 5x
1
8x
2
s.t. x
1
+ x
2
6
5x
1
+ 9x
2
45
x
1
, x
2
Z
+

The feasible region is

6 / 35

Optimal Simplex Tableau:

x
1
x
2
x
3
x
4
b
1 0 2.25 0.25 2.25
0 1 1.25 0.25 3.75
0 0 1.25 0.75 41.25

The Gomory cut from the second row of the tableau is:
0.75x
3
+ 0.25x
4
0.75
7 / 35

The modied feasible set is

The fractional optimal solution x = (2.35, 3.75) violates the

cutting plane and is removed from the new feasible set.
8 / 35

Example 2
min 4x
1
+ x
2
s.t. 7x
1
2x
2
14
x
2
3
2x
1
2x
2
3
x
1
, x
2
Z
+
9 / 35

Optimal Simplex Tableau:

x
1
x
2
x
3
x
4
x
5
b
1 0
1
7
2
7
0
20
7
0 1 0 1 0 3
0 0
2
7
10
7
1
23
7
0 0
4
7
1
7
0
59
7
Cut from the rst row of the tableau is:
1
7
x
3
+
2
7
x
4

6
7
.
10 / 35

Reoptimization:
x
1
x
2
x
3
x
4
x
5
x
6
b
1 0 0 0 0 1 2
0 1 0 0
1
2
1
1
2
0 0 1 0 1 5 1
0 0 0 1
1
2
6
5
2
0 0 0 0
1
2
3
15
2
Cut from the second row of the tableau is:
1
2
x
5

1
2
.
11 / 35

Reoptimization:
x
1
x
2
x
3
x
4
x
5
x
6
x
7
b
1 0 0 0 0 1 0 2
0 1 0 0 0 1 1 1
0 0 1 0 0 5 2 2
0 0 0 1 0 6 1 2
0 0 0 0 1 0 1 1
0 0 0 0 0 3 1 7
Done! Optimal solution x

= (2, 1)
T
.
12 / 35
Linear algebra and convex analysis review

Ane independence: A nite collection of vectors x

1
, . . . ,
x
k
R
n
is anely independent if the unique solution to
k

i =1

i
x
i
= 0,
k

i =1

i
= 0,
is
i
= 0, i = 1, 2, ..., k.

Linear independence implies ane independence, but not vice

versa.

The following statements are equivalent:

1. x
1
, . . . , x
k
R
n
are anely independent.
2. x
2
x
1
, . . . , x
k
x
1
are linearly independent.
3. (x
1
, 1), . . . , (x
k
, 1) R
n+1
are linearly independent.
13 / 35

A polyhedron is a set of the form

{x R
n
| Ax b} = {x R
n
| a
i
x b
i
, i M}, where
A R
m@n
and b R
m
.

Dimension of Polyhedra: A polyhedron P is of dimension k,

denoted dim(P) = k, if the maximum number of anely
independent points in P is k + 1. A polyhedron P R
n
is
full-dimensional if dim(P) = n. Let
M = {1, ..., m},
M
=
= {i M | a
i
x = b
i
, x P}, (the equality set),
M

= M \ M
=
(the inequality set).
Let (A
=
, b
=
), (A

, b

) be the corresponding rows of (A, b).

If P R
n
, then dim(P) + rank(A
=
, b
=
) = n.
14 / 35
Valid Inequalities and Faces

The inequality denoted by (,

0
) is called a valid inequality
for P if x
0
, x P.

Note that (,
0
) is a valid inequality if and only if P lies in
the half-space {x R
n
| x
0
}.

If (,
0
) is a valid inequality for P and
F = {x P | x =
0
}, F is called a face of P and we say
that (,
0
) represents or denes F.

A face is said to be proper if F = and F = P. Note that a

face has multiple representations.

The face represented by (,

0
) is nonempty if and only if
max{x | x P} =
0
.

If the face F is nonempty, we say it supports P. Note that the

set of optimal solutions to an LP is always a face of the
feasible region.
15 / 35
Facets

Let P be a polyhedron with equality set M

=
. If
F = {x P | x =
0
} is nonempty, then F is a polyhedron.
We can get the polyhedron F by taking some of the
inequalities of P and making them equalities.

The number of distinct faces of P is nite.

A face F is said to be a facet of P if dim(F) = dim(P) 1.

The inequality corresponding to a facet is called a strong valid
inequality.

If F is a facet of P, then in any description of P, there exists

some inequality representing F. (By setting the inequality to
equality, we get F).

Every inequality that represents a face that is not a facet is

unnecessary in the description of P.
16 / 35
An example
17 / 35
18 / 35
19 / 35
20 / 35
0-1 Knapsack Inequalities

Valid Inequalities for the Knapsack Problem. We are

interested in valid inequalities for the knapsack set
S = {x {0, 1}
n
|
N

j =1
a
j
x
j
b}.
N = {1, 2, . . . , .n}. Assume that a
j
> 0, j N, a
j
< b, j N.
We are interested in nding facets of conv(S).

Simple facets. What is dim(conv(S))? 0, e

j
, j N are n + 1
anely independent points in conv(S), so dim(conv(S)) = n.

x
k
0 is a facet of conv(S).

Proof. 0, e
j
, j N \ {k} are n anely independent points
that satisfy x
k
= 0.
21 / 35

x
k
1 is a facet of conv(S) if a
j
+ a
k
b, j N \ {k}.

Proof. e
k
, e
j
+ e
k
, j N \ {k} are n anely independent
points that satisfy x
k
= 1.

A set C N is a cover if

j C
a
j
> b. A cover C is a
minimal cover if C \ {j } is not a cover j C.

If C N is a cover, then the cover inequality

j C
x
j
|C| 1
is a valid inequality for S.
22 / 35

Example:
S = {x B
7
| 11x
1
+6x
2
+6x
3
+5x
4
+5x
5
+4x
6
+x
7
19}.

Minimal covers:
C = {1, 2, 3}, C = {1, 2, 6},
C = {1, 5, 6}, C = {3, 4, 5, 6}.
23 / 35
Can we do better?

Are these inequalities the strongest ones we can come up

with? What does strongest mean? We all know that facets
are the strongest, but can we say anything else?

If x
0
and x
0
are two valid inequalities for P R
n
+
,
we say that x
0
dominates x
0
if u 0 such that
u,
0
u
0
and (,
0
) = u(,
0
).

If x
0
dominates x
0
, then
{x R
n
+
| x
0
} {x R
n
+
| x
0
}.
24 / 35

Strengthening cover inequalities. If C N is a minimal cover,

the extended cover E(C) is dened as
E(C) = C {j N | a
j
a
i
, i C}. If E(C) is an extended
cover for S, then the extended cover inequality

j E(C)
x
j
|C| 1
is a valid inequality for S.

The cover inequality x

3
+ x
4
+ x
5
+ x
6
3 is dominated by
the extended cover inequality x
1
+ x
2
+ x
3
+ x
4
+ x
5
+ x
6
3.
25 / 35

Let C be a minimal cover. If C = N, then

j C
x
j
|C| 1
is a facet of conv(S).

Proof R
k
= C \ {k}, k C. x
R
k
satises

j C
x
R
k
j
= |C| 1.
Also, x
R
1
, . . . , x
R
|C|
are anely independent. Since C = N,
there are n anely independent vectors satisfy

j C
x
R
k
j
= |C| 1 at equality.
26 / 35

Let C = {j
1
, . . . , j
r
} be a minimal cover. Let
p = min{j | j N \ E(C)}. If C = E(C), and

j C\{j
1
}
a
j
+ a
p
b, then

j C
x
j
|C| 1 is a facet of
conv(S).

Proof. T
k
= C \ {j
1
} {k}, k N \ E(C).
|T
k
E(C)| = |C| 1 and

j T
k
E(C)
x
T
k
j
= |C| 1.
|R
k
| +|T
k
| = N. x
R
k
, k C, x
T
k
, k N \ C, are n anely
independent vectors.
27 / 35

Example:
S =

{x {0, 1}
5
| 79x
1
+53x
2
+53x
3
+45x
4
+45x
5
178}.

Consider minimal cover C = {1, 2, 3}. The valid inequality is:

x
1
+ x
2
+ x
3
2.
C = E(C). p = 4, C \ {1} {4} = {2, 3, 4}.
53 + 53 + 45 = 151 < 178. So x
1
+ x
2
+ x
3
= 2 gives a facet
of conv(S).
28 / 35
Lifting Cover Inequalities

Question: Can we nd the the a valid inequality as strong as

possible?

Example: C = {3, 4, 5, 6}, the valid inequality for C is:

x
3
+ x
4
+ x
5
+ x
6
3.

Setting x
1
= x
2
= x
7
= 0, the cover inequalities
x
3
+ x
4
+ x
5
+ x
6
3 is valid for
{x {0, 1}
4
| 6x
3
+ 5x
4
+ 5x
5
+ 4x
6
19}.

If x
1
is not xed at 0, can we strengthen the inequality? For
what values of
1
is the inequality

1
x
1
+ x
3
+ x
4
+ x
5
+ x
6
3
valid for
P
2,7
= {x {0, 1}
5
| 11x
1
+ 6x
3
+ 5x
4
+ 5x
5
+ 4x
6
19}.
29 / 35

1
+ x
3
+ x
4
+ x
5
+ x
6
3 is valid for all x {0, 1}
4
satisfying 6x
3
+ 5x
4
+ 5x
5
+ 4x
6
19 11;

1
+ 3, where
= max{x
3
+ x
4
+ x
5
+ x
6
| 6x
3
+ 5x
4
+ 5x
5
+ 4x
6
8}.

= 1
1
2. Thus
1
= 2 gives the strongest inequality.

How to nd the best value

j
, j N \ C such that

j C
x
j
+

j N\C

j
x
j
|C| 1
is valid for S?
30 / 35

Lifting Procedure

Let j
1
, . . . , j
r
be an ordering of N \ C. Set t = 1.

The valid inequality

t1

i =1

j
i
x
j
i
+

j C
x
j
|C| 1
is given. Solve the following knapsack problem:

t
= max
t1

i =1

j
i
x
j
i
+

j C
x
j
s.t.
t1

i =1
a
j
i
x
j
i
+

j C
a
j
x
j
b a
j
t
x {0, 1}
|C|+t1
.

Set
j
t
= |C| 1
t
. Stop if t = r .
31 / 35

Example: C = {3, 4, 5, 6}, j

1
= 1, j
2
= 2, j
3
= 7.
1
= 2.
Consider x
2
, we have

2
= max 2x
1
+ x
3
+ x
4
+ x
5
+ x
6
s.t. 11x
1
+ 6x
3
+ 5x
4
+ 5x
5
+ 4x
6
19 6 = 13,
x {0, 1}
5
.
So
2
= 2 and
j
2
=
2
= 3 2 = 1.
Consider x
7
now, we have

3
= max 2x
1
+ x
2
+ x
3
+ x
4
+ x
5
+ x
6
s.t. 11x
1
+ 6x
2
+ 6x
3
+ 5x
4
+ 5x
5
+ 4x
6
19 1 = 18,
x {0, 1}
6
.
So
3
= 3 and
j
3
=
7
= 3 3 = 0. We obtain a valid
inequality:
2x
1
+ x
2
+ x
3
+ x
4
+ x
5
+ x
6
+ x
7
3.
32 / 35
Separation of Cover Inequalities

We often try to solve problems that have knapsack rows with

lots more variables than that... Obviously I do not want to
add all of those facets at once. What to do?

Given some P, nd an inequality of the form

j C
x
j
|C| 1 such that

j C
x

j
> |C| 1. This is
called a separation problem.

Note that

j C
x
j
|C| 1 can be rewritten as

j C
(1 x
j
) 1.
33 / 35

Separation Problem: Given a fractional LP solution x

, does
cover C N such that

j C
(1 x

j
) < 1? or is
= min
CN
{

j C
(1 x
j
) |

j C
aj > b} < 1?

If 1, then x

satises all the cover inequalities.

If < 1 with optimal solution z

R
, then

j R
x
j
|R| 1 is a
violated cover inequality.
34 / 35
Branch-and-Cut Method

Branch and cut is an LP-based branch and bound scheme in

which the linear programming relaxations combined with by
cutting plane method.

The valid inequalities are generated dynamically using

separation procedures.

At each node of the search tree, cuts are generated and used
to improve the LP relaxation.

Branch-and-cut method is very ecient for some hard integer

programming problems.
35 / 35