Linear Programming

IE 5551, Spring 2011 Xi Chen

Outline
Background Definition (canonical form) Standard form Excel solver Branch-and-Bound algorithm

Background
Optimization (Wikipedia)
Choosing the best elements from some set of available alternatives Subfields: Convex programming (includes linear programming) Integer programming Nonlinear programming Stochastic programming Combinatorial optimization Components: Variables Objectives constraints

Linear Programming
A Linear Program (LP) is a mathematical optimization problem in which

the objective function and all constraints are linear.

Canonical form:
n

min
i 1 n

ci xi aij xi
i 1

min s.t.

cT x Ax b

s.t.
Uses:

bi ,

j 1, 2,..., m

Operations research Electrical engineering Portfolio management Heuristics

Linear Programming
Examples (LP?):

max s.t.

x1 x2 sin( x1 ) x2 2

max s.t.

x1 x1 x2 x1 x2 1 x1 0

max s.t.

x1

x2 2

max s.t.

x1

x2 2 0

x1 x2

x1 x2 x1 , x2

x2 {0,1}

Linear Programming
Standard form:
An LP is said to be in its standard form if it is of the form:

min s.t.

cT x Ax b x 0

Any LP can be written in the standard form using a few tricks

(Ronald L. Rardin, Optimization in Operations Research, 1998). Standard form is used to apply algorithm and use software

Linear Programming
Classes of LP problems:
Feasible and bounded

min s.t.

x1

x2 2 0

x1 x2 x1 , x2

Infeasible

min s.t.

x1 x2 x1 x2 x1 , x2 0 2

Unbounded

min s.t.

x1 x1

x2 x2 2

Linear Programming
Important theorems: Global optimal, local optimal
Unimodal Function:

Convex region: For any two points within the region, all the points on the line segment between them are all within the region

Unimodal Objective function & Convex feasible set implies Global optimal exists Local optimal is global optimal
LP has UOF & CFS Global optimal exists Local optimal is global optimal Global optimal are among the vertices

Linear Programming
Solving LP:
Graphical method (2-dimentional LP)

min s.t.

x1 x2 x1 2 x2 2 x1 x1 , x2 x2 0 3 3

Simplex method Interior point method Ellipsoid algorithm

Computer programming tools:

Excel Solver CPLEX

Excel Solver
Load Solver Add-In:
Menu -> Excel Options -> Add-Ins -> Manage Excel Add-Ins

-> select Solver Add-in

Example: min

x1 2 x2 3x3 x1 x2 x1 x2 x1 , x2 , x3 , x4 , x5 x3

x4 1 x4 0 1 x5 1.5

s.t.

Branch-and-Bound
Background Integer Programming
An integer program (IP) is just a linear program with the additional

constraint imposed that all variables must be integers.

min s.t. cT x Ax b x 0 x zn

Although a linear program is easily solved, the presence of the

integral make the problem much harder (NP-hard) One natural idea is to take the solution to the linear-programming relaxation and round it to the nearest integer point This may not be feasible, but even if it is, there is no guarantee that we're at an optimal point. The most commonly applied technique is branch-and-bound

Branch-and-Bound
Basic idea: solve the linear programming relaxation, and hope

that the solution is integral

Example:

max s.t.

17 x1 12 x2 10 x1 7 x2 x1 x2 5 Z
( 5 ,10 ) 3 3

40

x1 , x2

Solving the linear relaxation gives us: ( x1 , x2 ) We split the search space into two nodes:

P {( x1 , x2 ) :10 x1 7 x2 1 P2 {( x1 , x2 ) :10 x1 7 x2

40, x1 40, x1

x2 x2

5, x1 1} 5, x1 2}

Branch-and-Bound