Linear programming The Simplex Algorithm

The Simplex algorithm for solving LPs requires that all constraints are
equations (with exception of sign constraints on the variables) and all variables
be nonnegative. An LP in this form is said to be in standard form.

Any LP can be converted into an equivalent one in standard form.

Example:
max z 20 x1 15 x2

s.t. x1 100
x2 100
50 x1 35 x2 6000
20 x1 15 x2 2000
x1 , x2 0

max z 20 x1 15 x2

s.t. x1 s1 100
x2 s2 100
50 x1 35 x2 s3 6000
20 x1 15 x2 e4 2000

all variables 0

s variables are called slack variables

e variables are called excess or surplus variables

We usually refer to both the slack and surplus variables as slacks.

We can also convert an LP with unrestricted or nonpositive variables into

standard form.

1
Suppose we have an LP in standard form with n variables and m constraints.
For convenience, we label the variables x1 , x2 ,K , xn .

max (or min) z c1 x1 c2 x2 L cn xn

s.t.
a11 x1 a12 x2 L a1n xn b1
a21 x1 a22 x2 L a2 n xn b2
M
am1 x1 am 2 x2 L amn xn bm

x1 , x2 ,K , xn 0

x1
x
a11 a12 a1n 2 b1
a b2
a 22 a2 n
If we define A
21
, x , and b , then

M M

a m1 am 2 amn bm

xn

the equation constraints of the LP can be written in matrix form as Ax b .

Usually, A is called the constraint matrix and b is the right hand side vector.

We can write all the constraints of an LP in standard form as follows:

Ax b
x0

Before we can present the Simplex method for solving LPs we need to review
some fundamental definitions and results.

2
Consider here a system of equations Ax b with n variables and m constraints,
where we assume n m (i.e., more variables than equations).

Definition: A basic solution to the system of equations Ax b is obtained by:

1. setting n m variables equal to 0,

2. solving for the m remaining variables, and
3. confirming that the solution for these m remaining variables is unique.

Comment: condition 3 is equivalent to requiring that the m columns of matrix

A that correspond to these m variables be linearly independent. This means that
none of these m columns from A can be expressed as a linear combination of
the remaining m 1 columns.

The n m variables that are fixed equal to 0 are called nonbasic variables
(NBV). The remaining m variables are called basic variables (BV). Note that,
in general, different choices of nonbasic and basic variables will yield different
(basic) solutions to the system of equations Ax b .

Example: Find all basic solutions to the system

x1 x2 3
x2 x3 1

NBV = {x3} , BV= {x1 , x2 } x1 2, x2 1, x3 0

NBV = {x2 } , BV= {x1 , x3} x1 3, x2 0, x3 1
NBV = {x1} , BV= {x2 , x3} x1 0, x2 3, x3 2

Definition: A nonnegative basic solution is a called a basic feasible solution

(bfs) since it is feasible to Ax b, x 0 and basic for Ax b .

Theorem: the basic feasible solutions of Ax b are the extreme points of the
polyhedral set defined by Ax b, x 0 .

3
Example:

x1 x2 40
2 x1 x2 60
x1 , x2 0

By drawing the polyhedron in ( x1 , x2 ) space, we see that its extreme points are:
(0,0), (30,0), (20,20), (0,40).

Now we want to identify the basic feasible solutions and see their
correspondence with the extreme points. To do this, we need to convert the
constraint into standard form by adding slacks.

x1 x2 s1 40
2 x1 x2 s2 60

x1 , x2 , s1 , s2 0

BV NBV x1 x2 s1 s2 bfs?

Definition: Two basic feasible solutions are called adjacent if their sets of basic
variables differ in only one variable.

Theorem: If a linear program has an optimal solution, then there is one that is
an extreme point.

Corollary: we can solve an LP in standard form by considering only its basic

feasible solutions.

4
The Simplex Method

1. Convert the LP to standard form.

2. Obtain a bfs (if possible).

3. Determine whether the current bfs is optimal

4. If current bfs is not optimal, determine which nonbasic variables should

become basic and which basic variable should become nonbasic, so that a
new (adjacent) bfs with better objective function value is obtained.

5. Compute the values of the variables in the new bfs by using elementary
row operations (eros). Go back to step 3.

Example:

max z 60 x1 30 x2 20 x3

s.t.
8 x1 6 x2 x3 48
4 x1 2 x2 1.5 x3 20
2 x1 1.5 x2 0.5 x3 8
x2 5
x1 , x2 , x3 0

We first write the objective function in the following form:

z 60 x1 30 x2 20 x3 0

and we convert the constraints to standard form:

5
 Basic
variable
Row 0 z - 60 x1 - 30 x2 - 20 x3 = 0 z0
Row 1 8 x1 + 6 x2 + x3 + s1 = 48 s1 = 48
Row 2 4 x1 + 2 x2 + 1.5 x3 + s2 = 20 s2 = 20
Row 3 2 x1 + 1.5 x2 + 0.5 x3 + s3 = 8 s3 = 8
Row 4 x2 + s4 = 5 s4 = 5

We need an initial bfs. Note BV = { s1 , s2 , s3 , s4 }, NBV = { x1 , x2 , x3 } gives the

bfs s1 48, s2 20, s3 8, s4 5 , x1 x2 x3 0 . The value of this bfs is
z = 0.

The constraints in the above tabular formulation of our LP are not only in
standard form, they are also in Simplex tableau form with respect to the choice
we made of BV and NBV. Simplex tableau form means that each row is
associated to a unique basic variable which appears in its row with a coefficient
of 1 (and coefficient 0 in all other rows).

We will consider also z to be the basic variable associated with row 0 (however,
we dont require z to be nonnegative like the decision variables).

Is this bfs ( s1 48, s2 20, s3 8, s4 5 , x1 x2 x3 0 ) an optimal solution?

Note that, in this case, the value of z will improve if we can make x1 , x2 , or x3
positive. For a maximization LP, we will choose to increase the value of the
nonbasic variable with the most negative coefficient in row 0 (in this case, x1 ) .

We say we choose x1 to enter the set of basic variables. Note that we choose to
increase the value of only one nonbasic variable per iteration, the others will
remain fixed at value 0. This will have the effect of moving from one extreme
point of the feasible region to a (better) adjacent one.

6
We need to determine now which (currently) basic variable should become
nonbasic. Note that as we increase the value of x1 , the values of the current
basic variables must update according to:

8 x1 + s1 = 48
4 x1 + s2 = 20
2 x1 + s3 = 8
s4 = 5

That is,

s1 = 48 - 8 x1
s2 = 20 - 4 x1
s3 = 8 - 2 x1
s4 = 5

We need to maintain the nonnegativity of all variables that are currently basic:

s1 = 48 - 8 x1 0
s2 = 20 - 4 x1 0
s3 = 8 - 2 x1 0
s4 = 5 0 ( s4 is not affected here by changes in value of x1 )

To preserve the nonnegativity of s1 , s2 , and s3 ,

48
s1 nonnegative if: x1 6
8

20
s2 nonnegative if: x1 5
4

8
s3 nonnegative if: x1 4
2

7
 Ratio test:
48 20 8
The largest value we can assign to x1 now is the minimum of , , = 4.
8 4 2

Note that if we set x1 4 , then s3 becomes 0. Variable s3 is the one that we

select to leave the set of basic variables and become nonbasic.

Now we need to compute the values of the new bfs. In the same process we can
update our Simplex tableau to a canonical form that represents the new choice
of BV and NBV.

The row of the entering variable ( x1 ) in the new tableau will be the one of the
leaving variable ( s3 ).

We transform our current tableau to obtain the new one by a sequence of

elementary row operations.

x1 enters BV s3 leaves BV

Row 0 z - 60 x1 - 30 x2 - 20 x3 = 0
Row 1 8 x1 + 6 x2 + x3 + s1 = 48
Row 2 4 x1 + 2 x2 + 1.5 x3 + s2 = 20
Row 3 2 x1 + 1.5 x2 + 0.5 x3 + s3 = 8
Row 4 x2 + s4 = 5

The row of the leaving variable (in this case Row 3) is called the pivot row.
The coefficient of x1 in the pivot row is called the pivot term.

We will compute the values of the variables in the new basic feasible solution
and update the Simplex tableau to reflect the new choice of BV and NBV. We
will do this using elementary row operations (eros) on the current tableau.

8
Henceforth, we will save effort by dropping the variables from the Simplex
tableau and show only the coefficient entries:

z x1 x2 x3 s1 s2 s3 s4 rhs
Row 0 1 - 60 - 30 - 20 0 0 0 0 0 z0
Row 1 0 8 6 1 1 0 0 0 48 s1 48
Row 2 0 4 2 1.5 0 1 0 0 20 s2 20
Row 3 0 2 1.5 0.5 0 0 1 0 8 s3 8
Row 4 0 0 1 0 0 0 0 1 5 s4 5

Elementary row operations to update the Simplex tableau:

1. Divide all entries in the pivot row (here, row 3) by the pivot term so that
the entering variable (here, x1 ) has a coefficient of 1 in the pivot row.

(new row 3) = 0.5 * (old row 3)

z x1 x2 x3 s1 s2 s3 s4 rhs
Row 0 1 - 60 - 30 - 20 0 0 0 0 0
Row 1 0 8 6 1 1 0 0 0 48
Row 2 0 4 2 1.5 0 1 0 0 20
Row 3 0 1 .75 .25 0 0 .5 0 4
Row 4 0 0 1 0 0 0 0 1 5

2. Add appropriate multiples of the pivot row to eliminate the entering

variable from all other rows.

(new row 0) = (old row 0) + 60 * (new row 3)

(new row 1) = (old row 1) - 8 * (new row 3)

(new row 2) = (old row 2) - 4 * (new row 3)

(new row 4) = (old row 4) - 0 * (new row 3)

9
The (updated) Simplex tableau for BV={ s1 , s2 , x1 , s4 } is:

z x1 x2 x3 s1 s2 s3 s4 rhs
Row 0 1 0 15 -5 0 0 30 0 240 z 240
Row 1 0 0 0 -1 1 0 -4 0 16 s1 16
Row 2 0 0 -1 .5 0 1 -2 0 4 s2 4
Row 3 0 1 .75 .25 0 0 .5 0 4 x1 4
Row 4 0 0 1 0 0 0 0 1 5 s4 5

We have completed one iteration of the Simplex Method. Now we need to

determine if the current bfs is an optimal solution for the LP.

Since the coefficient of x3 in row 0 is negative, we can improve z if we make

x3 positive. Thus, we can not conclude that we have an optimal solution and we
decide to have x3 become basic.

We do the ratio test to determine which variable leaves the basis. We only
consider rows with positive entries in the column of the entering variable ( x3 ),
since these correspond to the basic variables that decrease as x3 increases.

x3 rhs ratios
Row 0
Row 1 ( s1 ) -1 16 no
Row 2 ( s2 ) .5 4 4/.5
Row 3 ( x1 ) .25 4 4/.25
Row 4 ( s4 ) 0 5 no

Minimum ratio attained at Row 2, thus s2 leaves basis.

New set of basic variables is: BV={ s1 , x3 , x1, s4 } . We use elementary row
operations to update the tableau (see next page).

10
 z x1 x2 x3 s1 s2 s3 s4 rhs
Row 0 1 0 5 0 0 10 10 0 280 z 280
Row 1 0 0 -2 0 1 2 -8 0 24 s1 24
Row 2 0 0 -2 1 0 2 -4 0 8 x3 8
Row 3 0 1 1.25 0 0 -.5 1.5 0 2 x1 2
Row 4 0 0 1 0 0 0 0 1 5 s4 5

Since the coefficients of the decision variables in row 0 are all nonnegative, we
can conclude that the current solution is optimal.

Remaining issues:

Finding an initial bfs:

It is easy for LPs were all original constraints are and the right hand side
data is nonnegative (as in the LP we just solved). In such case we can use
the slacks as initial BV. In the general case, finding an initial bfs is not
trivial and requires solving another LP (called the Phase I LP, for which we
find an initial bfs by construction).

Unbounded LPs:

We conclude that an LP is unbounded if at some iteration we select a variable to

enter the basis but there are no upper bound on its value since all coefficients in
its column has no positive coefficients (i.e., we cant do the ratio test)

Convergence:
n
There are at most possible basic solutions. Assuming we dont repeat a
m
basis, the Simplex method will terminate in a finite number of iterations.
Repeating a basis is called cycling and can only occur if the constraints are
degenerate. There are anti-cycling rules that can be used with the Simplex
method, but these are not implemented in commercial codes since cycling is
rarely a problem in practice.

11