Column Generation

Column
Generation
Jacques Desrosiers
Ecole des HEC & GERAD

COLUMN GENERATION 2
Contents

The Cutting Stock Problem

Basic Observations

LP Column Generation

Dantzig-Wolfe Decomposition

Dantzig-Wolfe decomposition
vs Lagrangian Relaxation

Equivalencies

Alternative Formulations to
the Cutting Stock Problem

IP Column Generation

Branch-and- ...

Acceleration Techniques

Concluding Remarks
COLUMN GENERATION 3
A Classical Paper :
P.C. Gilmore &
R.E. Gomory

A Linear Programming
Approach to the Cutting
Stock Problem.

Oper. Res. 9, 849-859.
(1960)

: set of items
: number of times
item i is requested
: length of item i
: length of a standard roll
: set of cutting patterns
: number of times item i
is cut in pattern j
: number of times
pattern j is used
ij
a
i
l
j
Y
i
n
I
J
L
COLUMN GENERATION 4
The Cutting Stock
Problem ...
Set can be huge.

Solution of
the linear relaxation
of by column generation.

J j Y
J j Y
I i n Y a
Y P
j
j
J j
i j ij
J j
j
e
e >
e >

e
e
, integer
, 0
,
min :
J j Y
I i n Y a
Y LP
j
J j
i j ij
J j
j
e >
e >

e
e
, 0
,
min :
J
, P
, LP
Minimize the number
of standard rolls used
COLUMN GENERATION 5
The Cutting Stock
Problem ...
Given a subset
and the dual multipliers

the reduced cost of
any new patterns
must satisfy:

otherwise, is optimal.
J j Y
I i n Y a
Y LP
j
J j
i j ij
J j
j
e >
e >

e
e
, 0
,
min :
J J c '
, ), ' ( I i J
i i
e t t
; 0 1 min
' \

e
e
<
I i
i ij
J J j
a t
LP
COLUMN GENERATION 6
The Cutting Stock
Problem ...
Reduced costs for
are non negative,
hence:
is a decision variable:
the number of times
item i is selected in a
new pattern.
The Column Generator
is a Knapsack
Problem.
' J j e
I i a
L a l
a
a
a
i
I i
i i
I i
i i
I i
ij i
J j
I i
ij i
J J j
e >
s

e
e
e
e
e
e
integer, 0
1 min
1 min
1 min
' \
t
t
t
i
a
I i a
L a l
a
i
I i
i i
I i
i i
e >
s
e
e
integer, 0
max t
COLUMN GENERATION 7
Basic
Observations
Keep
the coupling
constraints at a superior
level, in a Master
Problem;

this with the goal
of obtaining a
Column Generator
which is rather easy to
solve.
At an inferior level,
solve the
Column Generator,
which is often
separable in several
independent sub-
problems;

use a specialized
algorithm that exploits
its particular structure.
COLUMN GENERATION 8
LP
Column Generation
Optimality Conditions: primal feasibility
complementary slackness
dual feasibility
MASTER PROBLEM

Columns Dual Multipliers

COLUMN GENERATOR
(Sub-problems)
COLUMN GENERATION 9
Historical
Perspective
G.B. Dantzig &
P. Wolfe

Decomposition Principle
for Linear Programs.

Oper. Res. 8, 101-111.
(1960)
Authors give credit to:
L.R. Ford & D.R. Fulkerson

A Suggested Computation
for Multi-commodity flows.

Man. Sc. 5, 97-101.
(1958)
COLUMN GENERATION 10
Historical Perspective :
a Dual Approach
J.E. Kelly The Cutting Plane Method
for Solving Convex Programs.
SIAM 8, 703-712.
(1960)
DUAL MASTER PROBLEM

Rows Dual Multipliers

ROW GENERATOR
(Sub-problems)
Dantzig-Wolfe Decomposition :
the Principle
) ( , 0
) ( , 0
1
: as rewritten ) (
) ( p
p
) ( ) (
O e >
- O e >
=
+ =
e

- O e
O e - O e
r
p
d x x
X Conv x
r
p
r
r r
p
p p
u
u
u
rays extreme : ) (
points extreme : ) (
sets define ), ( On
O
- O
X conv
X x
b Ax
cx P
e
=
min :
Substitution
) ( , 0
) ( , 0
1
) (
) ( min :
) (
) ( ) (
) ( ) (
O e >
- O e >
=
= +
+

- O e
- O e O e
- O e O e
r
p
b d x A
d x c P
r
p
p
p
p r
r r p p
p r
r r p p
u
u
u
u
X x
b Ax
cx P
e
=
min :
) ( , 0
) ( , 0
1
) ( ) (
) ( ) ( min :
) (
) ( ) (
) ( ) (
O e >
- O e >
=
= +
+

- O e
- O e O e
- O e O e
r
p
b Ad Ax
cd cx P
r
p
p
p
p r
r r p p
p r
r r p p
u
u
u
u
The Master Problem
) ( , 0
) ( , 0
1
) (
) ( min :
) (
) ( ) (
) ( ) (
O e >
- O e >
=
= +
+

- O e
- O e O e
- O e O e
r
p
b d x A
d x c P
r
p
p
p
p r
r r p p
p r
r r p p
u
u
u
u
The Master Problem
The Column Generator
Given the current
dual multipliers
for a subset of
columns :
coupling constraints

convexity constraint

generate (if possible)
new columns
with negative
reduced cost :
) (
) ( min
0
X conv x
Ax cx
e
t t
) (
, 0 ) (
if points extreme
otherwise
) (
, 0 ) (
if rays extreme
0
- O e
<
O e
<
p some f or
Ax cx
r some f or
d A c
p p
r
t t
t
) (t
o
t
) (
, ) (
if points extreme
otherwise
) (
, 0 ) (
if rays extreme
0
- O e
+ <
O e
<
p some f or
b b Ax cx
r some f or
d A c
p p
r
t t t
t
Remark
) (
, 0 ) (
if points extreme
otherwise
) (
, 0 ) (
if rays extreme
0
- O e
<
O e
<
p some f or
Ax cx
r some f or
d A c
p p
r
t t
t
Block Angular Structure
Exploits the structure of
many sub-problems.

Similar developments
& results.
K k X x
b x A
x c P
k k
K k
k k
K k
k k
e e
=

e
e
,
min :
Algorithm
Optimality Conditions: primal feasibility
complementary slackness
dual feasibility
MASTER PROBLEM

Columns Dual Multipliers

COLUMN GENERATOR
(Sub-problems)
Given the current
dual multipliers
(coupling constraints)

(convexity constraint),

a lower bound can
be computed
at each iteration,
as follows:
a Lower Bound
X x
b Ax cx
X x
Ax cx b
e
=
e
+ +
) ( min
) ( min ) (
0 0
t
t t t t
) (t
o
t
Current solution value
+ minimum
reduced cost column
). ( max using obtained is bound lower best The
. ), ( ) (
; ) ( , 0 ) (
: on bound lower a provides
) (
) ( min ) (
t
t
t
t t
t
L
otherwise p some f or b Ax cx
r some f or d A c if
P
X conv x
b Ax cx L
p p
r
- O e =
O e < =
e
=
Lagrangian Relaxation Computes
the Same Lower Bound
X x
b Ax
cx P
e
=
min :
Dantzig-Wolfe vs Lagrangian
Decomposition Relaxation
Essentially utilized
for Linear
Programs

Relatively difficult to
implement

Slow convergence

Rarely implemented
Essentially utilized
for Integer
Programs

Easy to implement with
subgradient adjustment
for multipliers t

No stopping rule !

~ 6% of OR papers
Equivalencies
Dantzig-Wolfe
Decomposition &

Lagrangian Relaxation

if both have the same
sub-problems
In both methods, coupling
or complicating constraints
go into a
DUAL MULTIPLIERS
ADJUSTMENT PROBLEM :
in DW : a
LP Master Problem
in Lagrangian Relaxation :
) ( max t
t
L
Equivalencies ...
Column Generation corresponds to the solution
process used in Dantzig-Wolfe decomposition.

This approach can also be used directly by
formulating a Master Problem and sub-problems
rather than obtaining them by decomposing a
Global formulation of the problem. However ...
Equivalencies ...
for any Column
Generation scheme,
there exits
a Global
Formulation that
can be decomposed by
using a
generalized
Dantzig-Wolfe
decomposition which
results in the same
Master and sub-
problems.
The definition of the
Global Formulation
is not unique.

A nice example:

The Cutting Stock
Problem
. , integer, 0
, 1 0
,
,
min
K k I i X
K k or Y
K k Y L X l
I i n X
Y
k
i
k
k
I i
k
i i
i
K k
k
i
K k
k
e e >
e =
e s
e >

e
e
e
The Cutting Stock Problem :
Kantorovich (1960/1939)
: set of
available rolls

: binary variable,
1 if roll k is cut,
0 otherwise

: number of times
item i is cut
on roll k
K
k
Y
k
i
X
Kantorovich ...
Kantorovichs LP
lower bound is
weak:

However, Dantzig-Wolfe
decomposition provides
the same bound as the
Gilmore-Gomory LP
bound if sub-problems
are solved as ...
integer Knapsack
Problems, (which
provide extreme point
columns).
Aggregation of
identical columns
in the Master
Problem.
Branch & Bound
performed on
| | K
.
k
i
X
. / L n l
I i
i i
e
Valerio de Carvalh (1996)
Network type formulation on graph
) , ( A N
} 1 ,..., 1 , 0 ), 1 , {( } , 0 : ) , {(
} , 1 ,..., 2 , 1 , 0 {
= + e = s < s =
=
L u u u I i l u v and L v u v u A
L L N
i

Example with , and 3 , 2
2 1
= = l l
5 = L
A v u X
z X
L v X X
z X
I i n X
Z
uv
L u
uL
v u u v
vu uv
v
v
i
A l u u
l u u
i
i
e >
=
e =
=
e >
e +
+
) , ( integer, 0
,
} 1 , , 2 , 1 { , 0
,
,
min
) , (
) , ( ) , (
) , 0 (
0
) , (
,

Valerio de Carvalh ...
Valerio de Carvalh ...
The sub-problem is a
shortest path
problem on a acyclic
network.

This Column Generator
only brings back
extreme ray
columns,
the single extreme point
being the null vector.
The Master Problem
appears without the
convexity constraint.

The correspondence
with Gilmore-Gomory
formulation is obvious.

Branch & Bound
performed on
.
uv
X
Desaulniers et al. (1998)
It can also be viewed as a
Vehicle Routing Problem
on a acyclic network
(multi-commodity flows):

Vehicles Rolls
Customers Items
Demands
Capacity
Column Generation tools
developed for Routing
Problems can be used.
Columns correspond to
paths visiting items
the requested number of
times.
Branch & Bound
performed on
i i
n l
L
.
k
ij
X
integer
min :
x
X x
b Ax
cx IP
e
=
IP
Column Generation
integer
) ( , 0
) ( , 0
1
) ( ) (
) ( ) ( min :
) ( ) (
) (
) ( ) (
) ( ) (
x
d x x
r
p
b Ad Ax
cd cx IP
r
r r
p
p p
r
p
p
p
p r
r r p p
p r
r r p p

O e - O e
- O e
- O e O e
- O e O e
+ =
O e >
- O e >
=
= +
+
u
u
u
u
u
Integrality
Property
The sub-problem
satisfies the
Integrality
Property

if it has an integer
optimal solution for
any choice of linear
objective function,
even if the integrality
restrictions on the
variables are relaxed.
In this case,

otherwise

i.e., the solution process
partially explores the
integrality gap.
) ( ) ( max LP v L = t
) ( ) ( max ) ( IP v L LP v s s t
Integrality
Property ...
In most cases, the
Integrality Property is a
undesirable property!

Exploiting the non trivial
integer structure reveals
that ...
some overlooked
formulations become
very good when a
Dantzig-Wolfe
decomposition process
is applied to them.

Localization Problems
Vehicle Routing
Problems ...
IP Column Generation :
Branch-and-...
Branch-and-Bound :

branching decisions
on a combination of the
original (fractional) variables

of a Global Formulation on
which Dantzig-Wolfe
Decomposition is applied.
Branch-and-Cut :

cutting planes defined
on a combination of the
original variables;

at the Master level, as
coupling constraints;
in the sub-problem,
as local constraints.
Branch-and-...
Branching &
Cutting decisions
integer
min :
x
X x
b Ax
x c IP
e
=
Dantzig-Wolfe decomposition
applied at all decision nodes
{ }
IP Column Generation:
Branch-and-...
Branch-and-Price :

a nice name

which hides
a well known
solution process
relatively easy to apply.
For alternative methods,
see the work of

S. Holm & J. Tind

C. Barnhart, E. Johnson,
G. Nemhauser, P.
Vance, M.
Savelsbergh, ...

F. Vanderbeck & L. Wolsey
Application to Vehicle Routing and
Crew Scheduling Problems (1981 - )
Global Formulation :
Non-Linear
Integer Multi-
Commodity Flows

Master Problem :
Covering & Other
Linking Constraints

Column Generator :
Resource Constrained
Shortest Paths
J. Desrosiers, Y. Dumas, F.
Soumis & M. Solomon
Time Constrained
Routing and Scheduling
Handbooks in OR & MS,
8 (1995)

G. Desaulniers et al. A
Unified Framework for
Deterministic Vehicle Routing
and Crew Scheduling Problems
T. Crainic & G. Laporte (eds) Fleet
Management & Logistics (1998)
Resource Constrained
Shortest Path Problem on G=(N,A)

e
e
e
+
N i
R r
r
i i
A j i
ij ij
T x c Min
) , (
d o i d i o i x x
A i j j
ij
A j i j
ij
, , 0 ; , 1 ; , 1
) , ( :
) , ( :
= = = =

e
e
R r A j i T T f x
r
j i ij ij
e e s , ) , ( , 0 ) ) ( (
A j i x
R r N i b x T a x
ij
r
i
A j i j
ij
r
i
r
i
A j i j
ij
e
e e s s

e e
) , ( , binary
, , ) ( ) (
) , ( : ) , ( :
P(N, A) :
Integer Multi-Commodity
Network Flow Structure

e
e
e
e
+
K k
N i
R r
kr
i
k
i
A j i
k
ij
k
ij
k k
T x c Min ) (
) , (

M m b T a x a
N i b x
m
K k
kr
i
N i
kr
i m
A j i
k
ij
k
ij m
K k
k
i
K k
A j i j
k
ij
k k
k
e s +
e =

e
e e
e e
e
, ) (
, ) (
,
) , (
,
) , ( :

K k A N P T x
k k k k
e e ), , ( ) , (
Vehicle Routing and Crew
Scheduling Problems ...
Sub-Problem
is strongly NP-
hard
It does not posses the
Integrality Property
Paths Extreme points
Master Problem results in
Set Partitioning/Covering
type Problems
Branching and Cutting decisions are
taken on the original network flow,
resource and supplementary variables
Acceleration Techniques
on the Column Generator

Master Problem

Global Formulation
With Fast Heuristics

Re-Optimizers

Pre-Processors
To get Primal
& Dual Solutions
Exploit all the Structures
Acceleration Techniques ...
Multiple Columns :
selected subset close to
expected optimal solution
Partial Pricing in case of
many Sub-Problems :
as in the Simplex Method
Early & Multiple Branching
& Cutting : quickly gets
local optima
Primal Perturbation &
Dual Restriction :
to avoid degeneracy and
convergence difficulties
Branching & Cutting :
on integer variables !
Branch-first, Cut-second
Approach :
exploit solution structures
Link all the Structures Be Innovative !
Stabilized Column Generation
0
min
>
=
x
b Ax
cx
c A
b
s t
t max
2 1
max
d d
c A
b
s s
s
t
t
t
Restricted Dual
2 2 1 1
2 1
0 , 0
0
min
c c s s s s
>
= +
y y
x
b y y Ax
cx
Perturbed Primal
2 2 1 1
2 1
2 2 1 1
0 , 0
0
min
c c s s s s
>
= +
+
y y
x
b y y Ax
y d y d cx
Stabilized Problem
Concluding Remarks
DW Decomposition is
an intuitive framework
that requires all tools
discussed to become
applicable
easier for IP
very effective in
several applications
Imagine what could be
done with theoretically
better methods such as
the Analytic Center
Cutting Plane Method
(Vial, Goffin, du Merle,
Gondzio, Haurie, et al.)

which exploits recent
developments in interior
point methods,
and is also compatible
with Column Generation.
Bridging Continents and Cultures
F. Soumis
M. Solomon
G. Desaulniers
P. Hansen
J.-L. Goffin
O. Marcotte
G. Savard
O. du Merle
O. Madsen
P.O. Lindberg
B. Jaumard
M. Desrochers
Y. Dumas
M. Gamache
D. Villeneuve
K. Ziarati
I. Ioachim
M. Stojkovic
G. Stojkovic
N. Kohl
A. Nu
et al.
Canada, USA, Italy,
Denmark, Sweden,
Norway, Ile Maurice,
France, Iran, Congo,
New Zealand, Brazil,
Australia, Germany,
Romania, Switzerland,
Belgium, Tunisia,
Mauritania, Portugal,
China, The Netherlands,
...

Column Generation

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Column Generation

Uploaded by

Copyright:

Available Formats

Column

The Cutting Stock Problem :

You might also like