Chapter 4.

Solving Linear Programs

The Simplex Method
LP Model: The Augmented (Canonical) Form
max 3 x1 + 5 x2
s.t. x1 +x3 =4
2 x2 +x4 = 12
3 x1 + 2 x2 +x5 = 18
x1, x2, x3, x4, x5 >= 0
max cx 1 1
+ c2 x 2 +  + cn x n
s.t.
a x
11 1
+ a12
x 2
+  + a1n
x n
+ X n +1 =b1

a x
21 1
+a x 22 2
+ + a x 2n n
+ X n + 2 =b2
 
a x + a x + + a x
m1 1 m2 2 mn n
+ X n + m = bm

x ≥0 , x ≥0 ,  , x , X
1 2 n n +1
,  , X n + m ≥0

x : Slack Variable.
n +i
 x1 + x3 =4
2 x2 + x4 = 12
3 x1 + 2 x2 + x5 = 18

Augmented Solution: A solution for the original

variables that has been augmented by the corresponding
values of the slack variables.

In the standard form: (3, 2)

In the augmented form: (3, 2, 1, 8, 5)

How does the augmented corner solution look like?

 Rank of Matrix A
a a12 ... a1n 
 11 
 
a21 a22 ... a2n 
A = 
... ... ... ... 
 
 
a
 m1 am2 ... amn 


A has full rank if and only if

Ax =0 ⇒ x =0
and
AT y =0 ⇒ y =0.
 Basis of Matrix A
Let m × n matrixA have full rank and n > m. Then, B is a
basis of A, if B is a full - rank m × m sub - matrix of A
and its m columns are selected m columns of A.
Example :
a a12 ... a1m 
 11 
 
 a21 a22 ... a2m 
B = .
... ... ... ... 

 
a
 m1 am2 ... amm 

A = (B, N).
 Basic Solution of Ax=b
Consider t he matrix equation Ax = b.

For a selected basis B of A, the equation can be

decomposed as
BxB + Nx N =b.
The basic solution, corresponding to basis B, is
−1
xB = B b and x N = 0.
 A Basis Solution of the Augmented System
x1 + x3 =4
2 x2 + x4 = 12
3 x1 + 2 x2 + x5 = 18

1 0 1 0 0 1 0 0 
   
A = 0 2 0 1 0 . Select B = ( A•3 , A•4 , A•5 ) =  0 1 0 
3 2 0 0 1 0 0 1 
   
x 3   x1 
xB = x  and x N =
x  .
x 4
  2
 5 
4 
−1
  0 
x B =B b =12  and x N =
0 
.
   
8 
■ Basic Solution: An augmented corner-point solution.
Basic variables are x1, x2 and x5
In the standard form: (4, 6)
In the augmented form: (4, 6, 0, 0, -6)

■ Basic Feasible Solution: An augmented feasible corner-

point solution.

Basic variables are x2, x3 and x5

In the standard form: ( 0, 6)
In the augmented form: ( 0, 6, 4, 0, 6)
 Basic Solution of the Augmented Form

Basic Variable:
Variable
– A basic variable solution has at least n variables, original or
slack, whose value are zero. These variables are called non-
basic variables, and the others are called basic variables.
( 4, 6, 0, 0, -6)
( 0, 6, 4, 0, 6 )
– A basic solution can be obtained directly form the augmented
form by setting n ( non-basic ) variables equal to zero and
solving m equations for m ( basic ) variables
– The coefficient matrix of the m equations is called basis.
– If the values of the m basic variables are nonnegative, then it is
a basic feasible solution corresponding to a feasible corner-
point in the original form.
 Adjacent Basic Solutions

– Two basic solutions are adjacent if all but one of their

basic variables are the same. Two adjacent basic
solutions represent two neighbor corner-point in the
original form.

( 4, 6, 0, 0, -6 ) and ( 4, 3, 0, 6, 0 )

– Two basic feasible solutions are adjacent if all but one

of their basic variables are the same. Two adjacent
basic feasible solutions represent two feasible neighbor
corner-point in the original form.

( 4, 0, 0, 12, 6 ) and ( 4, 3, 0, 6, 0 )
 Algebraic Outline of the Simplex Method

Start from a basic feasible solution

NO
Is there a ‘better’ adjacent basic feasible solution exists?

YES

Move to a better adjacent basic feasible solution

– The movement involves converting one non-basic variable into a basic

variable ( called entering basic variable ) and simultaneously converting a
basic variable into a non-basic variable ( called the leaving basic variable ),
and then compute the new basic feasible solution.
 Z = 3x1 + 5x2
x1 + x3 =4
2x2 + x4 = 12
3x1 + 2x2 + x5 = 18

1. What is the criterion for selecting the entering basic

variable or to show that the current basic feasible solution
is optimal?

The candidate for the entering basic variables are the n current
nonbasic variables. The one chosen would be changed from nonbasic
into basic, so its value would be increased from zero to some positive
number and the others would be kept at zero. Since we require an
improvement, the rate of change in the objective function must be a
positive one.
 Z = 3x1 + 5x2
x1 + x3 =4
2x2 + x4 = 12
3x1 + 2x2 + x5 = 18

1. What is the criterion for selecting the entering basic

variable or to show that the current basic feasible solution
is optimal?

Express the objective function in terms of only nonbasic variables and

also the basic variables in terms of the nonbasic variables---called
Canonical Form. Then, the coefficient of the entering variable must be
positive. If the coefficients are all nonpositive, then no better adjacent
basic feasible solution exists and the current one is optimal.
2. How to select the leaving basic variable?
Z = 5 X2
+ X3 =4
2X2 +X4 = 12
2X2 + X5 = 18

X3 =4
X4 = 12 - 2X2
X5 = 18 - 2X2
2. How to select the leaving basic variable?
Z = 5 X2
+ X3 =4
2X2 +X4 = 12
2X2 + X5 = 18

X3 =4
X4 = 12 - 2X2
X5 = 18 - 2X2

3. How to compute the new basic feasible solution?

Z = 3X1 - (5/2) X4 +30

X1 + X3 =4
X2 + ( 1/2 ) X4 =6
X1 - X4 + X5 = 6
Iteration 2 for the example:
Z = 3 X1 + 30
X1 + X3 =4
X2 =6
3X1 + X5 = 6

Z= + 30 + 3X1
X3 = 4 - X1
X2 =6
X5 = 6 - 3X1

Z= - (5/2 ) X4 - X5 + 36
X3 + ( 1/3 ) X4 - ( 1/3 ) X5 = 2
X2 + ( 1/2 ) X4 =6
X1 - ( 1/3 ) X4 + ( 1/3 ) X5 =2
 Summary of the Simplex Method
1. Select a basic feasible solution

2. Express the basic variables in terms of the nonbasic variables, and

express the objective function in terms of only nonbasic variables.
variables
If all objective coefficients are non-positive, then stop: the current
basic feasible solution is optimal.
Otherwise, select the entering variable such that its coefficient is the
largest, and among basic variables select the leaving variable such
that it becomes zero first when increasing the entering variable and
keeping the other nonbasic variables at zero. If no basic variable
becomes zero, then stop: the objective function is unbounded.

3. Goto Step 2.
 The Original Simplex Method: Tabular Form
■ Gaussian Elimination

X1 + X3 = 4
X2 - X3 = 1
3X1 + 2X2 - 4X3 = -5

1 0 1 4
0 2 -1 1
3 2 -4 -5
Row 3 - 3 x Row 1:
1 0 1 4
0 2 -1 1
0 2 -7 -17
( Row 2 ) / 2:
1 0 1 4
0 1 -1/2 1/2
0 2 -7 -17
Row 3 - 2 x (Row 2):
1 0 1 4
0 1 -1/2 1/2
0 0 -6 -18

(Row 3 ) / (-6 ):
1 0 1 4
0 1 -1/2 1/2
0 0 1 3

Row 2 + ( Row 3 ) / 2:
1 0 1 4
0 1 0 2
0 0 1 3

Row 1 - Row 3:
1 0 0 1
0 1 0 2
0 0 1 3
 max c x +c x
1 1 2 2
+  + cn x n
s.t.
a x +a x
11 1 12 2
+  + a1n x n + X n +1 = b1

a x +a x
21 1 22 2
+  + a 2n x n
+ X n + 2 = b2
 
a x + a x + + a x
m1 1 m2 2 mn n
+ X n + m = bm

x ≥0 , x ≥0 ,  , x
1 2 n
, X n +1 ,  , X n + m ≥ 0

x n +i
: Slack Variable.

max 3 x1+ 5 x2
s.t. x1 + x3 =4
2 x2 + x4 = 12
3 x1 + 2 x2 + x5 = 18
x1, x2, x3, x4, x5 >= 0
 z - c x - c x -  -c x
1 1 2 2 n n
=0

a x + a x + + a x + X
11 1 12 2 1n n n +1 = b1

a x + a x + + a x + X
21 1 22 2 2n n n +2
= b2
 
a x +a x
m1 1 m2 2
+  + a m n x n + X n + m = bm

z - 3 x1 -5 x2 =0
x1 + x3 =4
2 x2 + x4 = 12
3 x1 + 2 x2 + x5 = 18
x1, x2, x3, x4, x5 >= 0
 Step 0. Initial Table

Basic Right-Hand
Var.. x x 1 2
 x n +1
... x n +m Side

Z - c1 - c 2  0  0 0

x n +1 a 11 a
12
 1  1 b1

x n +2 a 21 a 22
 0  0 b 2

   0  0 
x n +m a m1 a m2
 0  1 b m

Basic Variable RHS

Z -3 -5 0 0 0 0
3 1 0 1 0 0 4
4 0 2 0 1 0 12
5 3 2 0 0 1 18
 ■ Step 1. Optimality Test
The current basic feasible solution is optimal if and only if every
coefficient in Row 0 is nonnegative. If it is, stop; otherwise, go to
Step 2.

■ Step 2. Update the Basic Feasible Solution

1. Determine the variable by selecting the variable with the lowest
coefficient in Row 0. The corresponding column is called the pivot
column.

Basic Variable RHS

Z -3 -5 0 0 0 0
3 1 0 1 0 0 4
4 0 2 0 1 0 12
5 3 2 0 0 1 18
 Step 2. ( Continued )
2. Determine the leaving variable by
a) picking out each coefficient in the pivot column that is strictly
positive, ( if no positive coefficient exists, then stop; the
problem is unbounded. )
b) dividing each of these coefficients into RHS for the same
row,
c) identifying the row that has the smallest of these ratios, and
d) selecting the basic variable for this row, which is called the
pivot row.
row

Basic Variable RHS Ratio

Z -3 -5 0 0 0 0

3 1 0 1 0 0 4 ∞
4 0 2 0 1 0 12 6 = 12/2
5 3 2 0 0 1 16 9 = 18/2
 Step 2. ( Continued )

3. The intersection of the pivot column and pivot row is the pivot
number. Replace the leaving basic variable by the entering basic
variable in Column 0. Perform Gaussian elimination for the pivot
number. Goto Step 1.

Basic Variable RHS

Z -3 -5 0 0 0 0
3 1 0 1 0 0 4
2 0 2 0 1 0 12
5 3 2 0 0 1 18

( Row 2 ) / 2 :
Basic Variable RHS
Z -3 -5 0 0 0 0
3 1 0 1 0 0 4
4 0 1 0 1/2 0 6
5 3 2 0 0 1 18
Row 3 - 2 x Row 2

Basic Variable RHS

Z -3 -5 0 0 0 0
3 1 0 1 0 0 4
2 0 1 0 1/2 0 6
5 3 0 0 -1 1 6

Row 0 + 5 x Row 2

Basic Variable RHS

Z -3 0 0 5/2 0 30
3 1 0 1 0 0 4
2 0 1 0 1/2 0 6
5 3 0 0 -1 1 6
Iteration 2:
Basic Variable RHS Ratio
Z -3 0 0 5/2 0 30
3 1 0 1 0 0 4 4
2 0 1 0 1/2 0 6
5 3 0 0 -1 1 6 2
Basic Variable RHS
Z -3 0 0 5/2 0 30
3 1 0 1 0 0 4
2 0 1 0 1/2 0 6
1 1 0 0 -1/3 1/3 2

Basic Variable RHS

Z 0 0 0 3/2 1 36
3 0 0 1 1/3 -1/3 2
2 0 1 0 1/2 0 6
1 1 0 0 -1/3 1/3 2
 Initial BFS for General Form
Artificial Variable for Equality Constrains
maximize 3x1 + 5x2
subject to x1 =4
2x2 <= 12
3x1 + 2x2 <= 18
x1, x2 >= 0

maximize 3x1 + 5x2

subject to x1 =4
2x2 + x4 = 12
3x1 + 2x2 + x5 = 18
x1, x2, x3, x4 >= 0

maximize 3x1 + 5x2

subject to x1 + x3 =4
2x2 + x4 = 12
3x1 + 2x2 + x5 = 18
x1, x2, x3, x4, x5 >= 0
maximize 3x1 + 5x2 - Mx3
subject to x1 + x3 =4
2x2 + x4 = 12
3x1 + 2x2 + x5 = 18
x1, x2, x3, x4, x5 >= 0

Basic Variable RHS

Z -3 -5 100 0 0 0
3 1 0 1 0 0 4
4 0 2 0 1 0 12
5 3 2 0 0 1 18

Basic Variable RHS

Z -103 -105 0 0 0 -400
3 1 0 1 0 0 4
4 0 2 0 1 0 12
5 3 2 0 0 1 18
The Big M Method
1. Change every > = constraint to <= constraint by multiplying -1

2. Add every <= constraint a slack variable. If the right side of a <=
constraint is nonnegative, then the slack variable is the basic
variable for this row.

3. Multiply -1 to every constraint with the negative right side, and add
artificial variables to be the basic variables for these rows.

4. Add Big M as the objective coefficient for each artificial variable

in the original objective function.

5. Start the simplex method to optimize the new objective function.

function
The Two-Phase Method

1. Change every >= constraint to <= constraint by multiplying -1.

2. Add every <= constraint a slack variable. If the right side of a <= constraint is
nonnegative, then the slack variable is the basic variable for this row.
3. Multiply -1 to every constraint with the negative right side, and add artificial
variables to be the basic variables for these rows.
4. Minimize the sum of the artificial variables as the Phase-1 linear program.

minimize x3
subject to x1 + x3 =4
2x2 + x4 = 12
3x1 + 2x2 + x5 = 18
x1, x2, x3, x4, x5 >= 0
5. After find a BFS, optimize the original objective function as the Phase-2 linear
program.
 Tie Breaking

Degeneracy: The number of zero-value variables (including slacks) is

always greater than or equal to the number of original variables. In other
words, the number of positive-value variables is always less than or equal
to the number of inequality constraints. When they are not equal, we have
a degenerate corner or basic feasible solution.

Left-Right and Top-Bottom Rule

Max 2x1 - 3x2 + 5x3

s.t. 8x1 + 5x2 + 4x3 <= 24
x1 - 7x2 <= 17
5x1 + 3x2 + 10x3 <= 60
x1, x2, x3 >= 0
Simplex Method:
Basic Variable RHS Ratio
Z -2 3 -5 0 0 0 0
4 8 -5 4 1 0 0 24 6
5 1 -7 0 0 1 0 17
6 5 3 10 0 0 1 60 6

Basic Variable RHS Ratio

Z 8 -13/4 0 5/4 0 0 30
3 2 -5/4 1 1/4 0 0 6
5 1 -7 0 0 1 0 17
6 -15 31/2 0 -5/2 0 1 0 0

Basic Variable RHS Ratio

Z 301/62 0 0 45/62 0 13/62 30
3 49/62 0 1 3/62 0 5/62 6
5 -179/31 0 0 -35/31 1 14/31 17
2 -30/31 1 0 -5/31 0 2/31 0
 Cycling:
Degeneracy can cause the phenomenon called cycling, the
infinite repetition of a finite sequence of basic feasible
solutions producing non change in the value of the objective
and thereby preventing the finite termination of the
Simplex method.

Min - (3/4)x4 +20x5 - (1/2)x6 + 6x7

s.t. x1 + (1/4)x4 - 8x5 - x6 + 9x7 = 0
x2 + (1/2)x4 - 12x5 - (1/2)x6 + 3x7 = 0
x3 + x6 =1
x1, x2, x3, x4, x5, x6, x7 >= 0
 Methods Handling Degeneracy
■ Random Choice
– Chose the row randomly from among all those
that yield the minimum ratio
■ Perturbation of the RHS
– bi + ε i, i=1,…,m
■ Lexicographic Ordering
■ Double Least Index Rule
– Least indexed variable among all eligible
entering variables
– Least indexed variable among all rows that yield
the minimum ratio