ISM206 Optimization Scribing Notes

LECTURE: INTRO TO LINEAR PROGRAMMING AND

THE SIMPLEX METHOD, KEVIN ROSS
MARCH 31, 2005

DAVID L. BERNICK
dbernick@soe.ucsc.edu

1. Overview
• Typical Linear Programming problems
• Standard form and converting to Standard form
• Geometry of Linear Programming
• Extreme points, linear independence and bases
• Optimality conditions
• The Simplex method

2. Typical Linear programming Problems

2.1. Product Mix problem
Problem: How much beer and ale should be produced?

 
amt. item(pounds)
 480 corn 
available resources = 
 160

 (1)
hops
1190 malt

 
item corn hops malt
 (barrel) (pounds) (ounces) (pounds) 
required resources = 


 (2)
Ale 5 4 35
Beer 15 4 20

· ¸
Ale $13/barrel
prof it = (3)
Beer $23/barrel

1
ISM206 Optimization Scribing Notes

Analysis:

A = Ale(barrels)
B = Beer(barrels)
Objective f unction
M ax{P rof it = 13A + 23B}
Constraints :
Subject to(s.t)
5A + 15B ≤ 480
4A + 4B ≤ 160
35A + 20B ≤ 1190
A, B ≥ 0

In general:

ci = producti prof it
xi = producti count
bj = resourcej availability
ai,j = resourcej required f or producti
X
objective f unction = max{ ci xi }
i∈{products}
X
s.t. ai,j xi ≤ bj
i∈{products}
xi ≥ 0

2.2. Transportation Problem

Problem:

• Production of computers in Singapore and Oakland

• Distribution centers in Oakland, Hong Kong and Istanbul
ISM206 Optimization Scribing Notes

Supply, demand and cost summary:

 
ship.cost
 Oakland HongKong Istanbul Supply 
 
 
 Singapore 85 37 119 500 
Supply, demand, cost =  
 Hohboken 53 189 94 300 
 
 
Demand 350 250 200
(4)
Objective: meet demand with minimum total cost.
Analysis:
ci,j = ship.cost f rom i to j
Si = supply f rom sitei
Dj = demand f rom sitej
i ∈ {Singapore, Hohboken}
j ∈ {Oakland, HongKong, Istanbul}
xi,j = count shipped f rom i to j
X
objective = ci,j xi,j
i,j
X
s.t. xi,j = Si ∀i(supply)
j
X
xi,j = Dj ∀j(demand)
i
xi ≥ 0

3. Other LP examples
• Blending Problem
• Diet Problem
• Assignment Problem

3.1. Blending Problem

Problem: Consider a constraint where at least 10% of output must be from
1 variable.
We then have a constraint of the form: Pxixi ≥ 0.10. This is not a
i P
linear equation but we can change it to: xi ≥ 0.10 ∗ i xi and now it is a
linear form.
ISM206 Optimization Scribing Notes

4. Key elements of a LP
• Proportionality - the function depends on a proportion assigned to
a variable.
• Additivity
• Divisibility

4.1. Steps in building a LP

• Identify the activities.
• Identify the items.
• Identify the Input / Output coefficients.
• Write the constraints.
• Identify the coefficients of the objective function.

5. Geometry of a LP
Consider the plot of the solution space for the following:
objective f unction : max{x1 + x2 }
s.t. 3x1 − 5x2 ≤ 15
3x1 − 5x2 ≤ 12
x1 , x2 ≥ 0
As a quick way to find the regions specificd by the inequality, consider
weather (0,0) is a solution for the inequality or not. In this case, (0,0) is
part of the solution spacec for all constraints.
Geometrically, this problem can be viewed as Fig:??. As can be seen,
the solution is at a corner. This is true in general for Linear Programming
problems - one of the optimal points will be at a corner. This is the basis
of the Simplex method.

6. Standard Form
Types of LP descriptors:
Objective f unction :
max(c1 x1 + c2 x2 + . . . cN xN
s.t
a1,1 x1 + a1,2 x2 + . . . + a1,N xN = b1
a2,1 x1 + a2,2 x2 + . . . + a2,N xN = b2
xj ≥ 0, j = 1, N
ISM206 Optimization Scribing Notes

Figure 1. Graph of feasible region, bounded by all constraints, with family of curves of
x1 + x2 and max(x1 + x2 ).

or consisely:

max(c0 x)
s.t. Ax = b
x≥0

where A is an [m x n] matrix of N variables and m constraints.

All LP problems can be converted to standard form. The following
situations can arise, and are converted to standard form as follows:

• inequalities
• free variables
• MINimizations

6.1. Handling Inequalities

For example, the constraint x1 + x2 ≤ 4 can be handled by introducing
a new, surplus variable x3 . This constraint then becomes x1 + x2 + x3 =
4 s.t x3 ≥ 0.
ISM206 Optimization Scribing Notes

6.2. Handling Free variables

In the case where a variable is free (x1 is unconstrained), then we once
again introduce a new, surplus variable. For example:

x1 ∈ <(f ree variable)

objective f unction :
min(x1 + x2 )
s.t. x2 ≥ 0
x1 + x2 = 3
replace x1 = x+ −
1 − x1
x+ −
1 , x1 ≥ 0

rewritten in standard form:

objective f unction :
min x1 + x2 new variable does not appear in the objective function
s.t. x2 ≥0
x+
1 + −
x1 + x2 =3
x+ −
1 , x1 ≥0

6.3. Handling MIN objectives (vs. MAX)

To convert a MIN to a MAX, negate the terms. For example:

min x1 + 3x2 = max −x1 − 3x2

. Remember, any surplus variable that is introduced does not appear in

the objective function - only in constraints.

7. Solutions, Extreme points and Basis

• How many solutions are there to a set of linear equations?
• Convexity of a feasible region.
• Extreme Points
ISM206 Optimization Scribing Notes

7.1. How many solutions exist for a set of linear equations

How many linearly independent rows and columns exist in the set? In
matrix notation, the set of equations is described by
Ax = b
. If A is a square matrix (NxN) and det(A) 6= 0 then x = Ab is unique.
In general, A is rectangular (m x N), with many variable and few con-
straints.

7.2. Convexity of a feasible region

X ≡ a convex set iff

∀x1 , x2 ∈ X
λx1 + (1 − λ)x2 ∈ X , {0 ≤ λ ≤ 1}
or, for any two points in set X , any point between any point between those
2 points is also in the set.

7.3. Extreme Point of X

x̄ is an extreme point iff
f or distinct x1 , x2 ∈ X
if, x̄ = λx1 + (1 − λ)x2 , {0 ≤ λ ≤ 1}
=⇒ λ ∈ {0, 1}
then x̄ is an extreme point
or, if x̄ is not between 2 other points in the set, then it is an extreme point.

8. Linear independence of vectors

• Basis of a matrix
• A basic solution of an LP
• Basic Feasible solution (Corner Point Feasible)

8.1. Basic Feasible solution(bfs)

~x is an extreme point of a solution space iff it is a bfs of Ax = b, {x ≥ 0}.
Key fact: If an LP has an optimal solution, it exists at a corner of the
feasible region.
ISM206 Optimization Scribing Notes

8.2. Basis of a matrix

• Linear independence

Let V 1 , V 2 , . . . , V N be vectors.
They are linearly independent,

if : α1 V 1 + α2 V 2 . . . + αN V N = 0
=⇒ α1 = α2 = . . . αN = 0

Basis of matrix A is a maximal linearly independent set of columns of

A. For example:

· ¸
110
A= (constraints)
111
Basis(A) = 1st and 3rd columns of A, or 2nd and 3rd columns

Only a certain number of constraints need to be binding.

8.3. Rank of a Matrix

Rank of a matrix = number of independent columns in the matrix.
[m x n] matrix A is full rank if rank(A)=m. Or, if there are at least as
many idependent columns as there are constraints.

 
a1,1 a1,2 . . . a1, N
 
A =  ... 
am,1 am,2 . . . am,n

for m constraints and n variables.

ISM206 Optimization Scribing Notes

8.4. Linear Programming

Write A = [B, N ] where B is a basis of A.
· ¸
12011
A=
00111
· ¸
10
Basis of A = B =
01
· ¸
211
Lef tover or N ull = N =
011
max cT x
s.t. Ax = b
x≥0
A = [B, N ]
Rewrite constraints as:
µ ¶
xB
[B, N ] =b
xN
set xN = 0
xB = B −1 b
BxB + N xN = b
Set N xN = 0
XB = b
XB = B −1 b

9. Simplex Method - Overview

• Checks Corner Points
• Looks for better solutions at each iteration
Simplex Algorithm:
• Find a starting point - a corner.
• Test this point for optimality
• Stop if this point is optimal, otherwise repeat
One basic variable is replaced by another. Optimality test identifies the
basic variable to replace.