The Idea Behind the Simplex Algorithm

Video: 2A-2C
Reading: Rao, Engineering Optimization: Theory and Practice, 2009, sections 3.1-3.8
The book can be accessed online via the JHU Library
https://onlinelibrary-wiley-com.proxy1.library.jhu.edu/doi/book/10.1002/9780470549124
In the last module, we solved two optimization problems graphically, one was non-linear and the
other one was linear. In a non-linear problem, any point of the feasible region can be an optimal
solution (this point is different for different objective functions). In a linear optimization
problem, only an extreme point of the feasible region can be an optimal solution. For this reason,
solving a linear optimization problem is easy, we can just find the extreme points and compare
the values of the objective function at these points.

With more than two decision variables, the feasible region is no longer two dimensional and
extreme points must be found algebraically rather than from the graph. Also, as the number of
decision variables and constraints grow, so does the number of extreme points. The simplex
method solves these two problems, it offers a way to find an extreme point and to move from this
extreme point to an extreme point with a better value of the objective function, eliminating the
need to evaluate all the extreme points.

To illustrate the idea behind the simplex algorithm, let’s start with a problem with two decision
variables that can be solved both graphically and algebraically,

min 2x1 − x2

s.t.

9 x1 + x2  6
x1 + 2 x2  −2
3 x1 − x2  −1
x1 , x2  0

Question 1: Plot the constraints in MATLAB. Please attach your plot here.

Question 2: How many extreme points does the feasible region have?
Next, let’s try to find the same extreme points algebraically. With two variables, an extreme point
is a point where two linearly independent constraints are active, and the remaining constraints
are satisfied. For example, if we chose the 9 x1 + x2  6 and x2  0 to be active, we’ll get

9 x1 + x2 = 6
x2 = 0
x1 + 2 x2  −2
3x1 − x2  −1
x1  0

Solving the system of two equations gives a candidate extreme point and, if the inequalities are
satisfied, this is an extreme point.

Question 3: Is (2/3, 0) an extreme point?

- yes
- no
The above is an algebraic solution, it can be applied to a problem with any number of decision
variables, not just two (the number of linearly independent active constraints is equal to the
number of decision variables). However, if we present this solution in a slightly different way,
we will be able to take a major shortcut.

Finding a solution to

9 x1 + x2  6
x1 + 2 x2  −2
3 x1 − x2  −1
x1 , x2  0

where two of the constraints are active and the remaining ones are satisfied is equivalent to
finding a solution to

9 x1 + x2 + x3 = 6
x1 + 2 x2 − x4 = −2
3 x1 − x2 − x5 = −1
x1 , x2 , x3 , x4 , x5  0

where at least two of x1 , x2 , x3 , x4 and x5 are equal to zero. For example, if x2 = x3 = 0 , the

constraints 9 x1 + x2  6 and x2  0 are active in the original problem. x3 , x4 and x5 are called slack
(or surplus) variables.

The transition from

9 x1 + x2  6 9 x1 + x2 + x3 = 6
x1 + 2 x2  −2 x1 + 2 x2 − x4 = −2
to
3 x1 − x2  −1 3 x1 − x2 − x5 = −1
x1 , x2  0 x1 , x2 , x3 , x4 , x5  0

only works because the original problem has a non-negativity constraint x1 , x2  0 , but it is easy

to convert any optimization problem into one with a non-negativity constraint (the specific
technics are shown in video 2A).

The advantage of the

9 x1 + x2 + x3 = 6
x1 + 2 x2 − x4 = −2
3 x1 − x2 − x5 = −1
x1 , x2 , x3 , x4 , x5  0

presentation of the problem is that we’re no longer limited to finding one extreme point at a time,
but can derive a new extreme point from the previous one and in such a way that the value of the
objective function will decrease (or stay the same) with each point, so, in most cases, we won’t
have to go through all the extreme points to find the optimal one.

To begin, we need to find an extreme point that corresponds to at least two of x1 , x2 , x3 , x4 and x5

being zeros. We can arbitrary choose x2 and x3 to be zeros (and call them non-basic variables).

It’s tempting to replace x2 and x3 by zeros right away, but, for the promised major shortcut to

work, we need to keep x2 and x3 . Let’s use elementary matrix operations to isolate the remaining

three variables (the basic variables) so that each one only appears in one equation and with a +1
coefficient:
 1 1 1?
x1 + x2 + x3 =
9 9 3
17 1 2?
− x2 + x3 + x4 =
9 9 3
4 1
x2 + x3 + x5 = 3?
3 3

Question 4: Replace the 1? with the correct number.

Question 5: Replace the 2? with the correct number.
Question 6: Replace the 3? with the correct number.
The above is a canonical form of the LP (except for the objective function). To find the
corresponding basic solution, we recall that x2 = x3 = 0 .

Question 7: The basic solution is (x1, 0, 0, x4, x5) with x1 =

Question 8: (continued from the previous question) x4 =
Question 9: (continued from the previous question) x5 =
We were lucky and all the variables in the basic solution are non-negative, so the non-negativity
constraint x1 , x2 , x3 , x4 , x5  0 is satisfied and the basic solution is feasible. If we chose a set of

three basic variables other than x1 , x4 and x5 , we wouldn’t necessarily be as lucky, and the basic

solution may have been not feasible.

Question 10: Is the basic solution with x1, x2 and x3 as basic variables feasible?
- yes
- no
Question 11: How many of the basic solutions are feasible?
Question 12: How many of the basic solutions are not feasible?
You have probably used the graphical solution for the original two variable problem to answer
the last three questions. However, with more than two variables, there is no graphical solution.
Sometimes, it’s easy to find a basis that corresponds to a feasible basic solution, otherwise, it
artificial variables need to be added on top of the slack/surplus variables.

Now that we have one feasible basic solution (extreme point), we can find a new one. Recall
that, earlier in the process, we resisted the temptation to set the non-basic variables to zero
x2 = x3 = 0 right away and pivoted the basic variables instead. Because we kept x2 and x3 , we
can now re-assign one of the non-basic variable to become a basic variable and one of the
existing basic variables will exit the basis. When only one pair of variables is to be switched, it is
possible to choose this pair in such a way that the new basic solution remains feasible and the
value of the objective function decreases (or stays the same).

Since the objective function z = 2 x1 − x2 matters now, we will add it to our system of equations,

2 x1 − x2 = z + 0
9 x1 + x2 + x3 = 6
x1 + 2 x2 − x4 = −2
3 x1 − x2 − x5 = −1
x1 , x2 , x3 , x4 , x5  0

and require the basic variables x1 , x4 and x5 to be excluded from the objective function

4? 5? 6?
− x2 − x3 = z −
9 9 3
1 1 1?
x1 + x2 + x3 =
9 9 3
17 1 2?
− x2 + x3 + x4 =
9 9 3
4 1
x2 + x3 + x5 = 3?
3 3

Question 13: Replace the 4? with the correct number.

Question 14: Replace the 5? with the correct number.
Question 15: Replace the 6? with the correct number.
x2 and x3 have negative coefficients in the objective function with the coefficient of x2 being the
most negative. Moving a variable from the non-basic set to a basic set is equivalent to increasing
its value from zero to a positive number (or keeping it zero) and the coefficient of x2 in the
objective function is negative, so increasing x2 will decrease z. Therefore, we chose x2 to enter
the basis.

Which variable should exit the basis, x1 , x4 or x5 ? We want to increase x2 as much as possible

because increasing it decreases the objective function. x3 is still zero, so we have

 1 2
x1 + x2 =
9 3
17 8
− x2 + x4 =
9 3
4
x2 + x5 = 3
3

Therefore,

2
x2  3 = 6
1
9
8
24
x2  − 3 = −
17 17
9
3 9
x2  =
4 4
3

9
The last constraint above is the most restrictive one and so x2 can only be increased to , at
4
which point x5 becomes zero and exits the basis. Therefore, the new basis is x1 , x2 and x4 and the

new canonical form is

1 11 17
x3 + x5 = z +
12 12 12
1 1 5
x1 + x3 − x5 =
12 12 12
7 17 83
x3 + x4 + x5 =
12 12 12
1 3 9
x2 + x3 + x5 =
4 4 4

The non-basic variables now have positive coefficients in the objective function, so the minimum
17  5 9 83 
possible value of the objective function is − and it is achieved at  , , 0, , 0  . The
12  12 4 12 
 5 9
optimal solution to the original two variable problem is  ,  .
 12 4 
Verifying the answer in MATLAB,

f = [2 -1];
A = [9 1; -1 -2; -3 1; -1 0; 0 -1];
b = [6 2 1 0 0];
linprog(f,A,b)
The process demonstrated above becomes the simplex algorithm when two changes are made to
it:

(1) the coefficients of the decision variables and the right-hand sides of the equations are
arranged in a tableau and
(2) if the first feasible basic solution is not obvious, artificial variables are used to obtain it.

The tableau is simply a different format and the artificial variables trick will be demonstrated in
the next example.

The Simplex Algorithm

Video: 2D-2F
Reading: Rao, Engineering Optimization: Theory and Practice, 2009, sections 3.9-3.11
The book can be accessed online via the JHU Library
https://onlinelibrary-wiley-com.proxy1.library.jhu.edu/doi/book/10.1002/9780470549124
Consider the following problem: a beach town council consists of three council members, one
per each of three districts.

Due to frequent inclement weather, the last year town budget was all used up for boring things,
such as road maintenance and snow removal. The town residents are unhappy and are not going
to vote for the incumbents in the upcoming election.
The councilors, however, want to stay at power and not let even one of them be replaced, so they
decide to invest their own money into the city infrastructure to make the town residents happy.
They can buy e-scooters for the park, cable cars for the mountain district and sand for the beach.
Each $1,000 of investment results in the following number of votes gained or lost:

Investment Park District Mountain District Lake District

E-scooters 7 -5 1

Cable Cars -2 6 3

Sand 1 1 3

The councilors need to get the majority in each of the three districts while keeping the total
investment minimum, so the optimization problem to solve is

min z = 1000 x1 + 1000 x2 + 1000 x3

7 x1 − 2 x2 + x3  1000
−5 x1 + 6 x2 + x3  1500
x1 + 3x2 + 3x3  500
x1 , x2 , x3  0

or, with slack variables,

min z = 1000 x1 + 1000 x2 + 1000 x3

7 x1 − 2 x2 + x3 − x4 = 1000
−5 x1 + 6 x2 + x3 − x5 = 1500
x1 + 3x2 + 3x3 − x6 = 500
x1 , x2 , x3 , x4 , x5 , x6  0

We start with obtaining any feasible basic solution. In the previous example, we arbitrary chose a
set of basic variables and got lucky, the corresponding basic solution was feasible. Sometimes, a
feasible basic solution is obvious, for example, if x4, x5 and x6 had the coefficients +1 in the
constraints,
7 x1 − 2 x2 + x3 + x4 = 1000
−5 x1 + 6 x2 + x3 + x5 = 1500
x1 + 3x2 + 3x3 + x6 = 500

then the LP would be in a canonical form and (0, 0, 0, 1000, 1500, 500) would be a feasible basic
solution.

However, with x4, x5 and x6 having coefficients -1, there is no obvious feasible basic solution and
we need to add artificial variables, one per each equation,

7 x1 − 2 x2 + x3 − x4 + x7 = 1000
−5 x1 + 6 x2 + x3 − x5 + x8 = 1500
x1 + 3x2 + 3x3 − x6 + x9 = 500
x1 , x2 , x3 , x4 , x5 , x6 , x7 , x8 , x9  0

The artificial variables are isolated by design and (0, 0, 0, 0, 0, 0, 1000, 1500, 500) is a feasible
basic solution.

This is a solution to the new LP with nine variables, not the original LP with six variables.
However, if we find an optimal feasible basic solution to

min w = x7 + x8 + x9
7 x1 − 2 x2 + x3 − x4 + x7 = 1000
−5 x1 + 6 x2 + x3 − x5 + x8 = 1500
x1 + 3x2 + 3x3 − x6 + x9 = 500
x1 , x2 , x3 , x4 , x5 , x6 , x7 , x8 , x9  0

then the values of x1 to x6 in this optimal feasible solution will constitute a (perhaps, non-
optimal) feasible solution to the original problem, provided a feasible solution to the original
problem exists.

In fact, if a feasible basic solution to the six variable LP exists, then the same solution combined
with x7=x8=x9=0 is a feasible basic solution to the nine variable LP and it is optimal because x7,
x8 and x9 are non-negative and so the minimum possible value of w is 0.

Therefore, we start with optimizing w in the nine variable LP.

Let’s use a tableau to save space:

 Basis x1 x2 x3 x4 x5 x6 x7 x8 x9 RHS

-z 1000 1000 1000 0 0 0 0 0 0 0

-w 0 0 0 0 0 0 1 1 1 0

x7 7 -2 1 -1 0 0 1 0 0 1000

x8 -5 6 1 0 -1 0 0 1 0 1500

x9 1 3 3 0 0 -1 0 0 1 500

We’ll be optimizing w first, not z, but it is customary to keep and update both w and z in the
tableau. As before, z is 0+z, “+z” is omitted from the RHS column and “-z” is added to the Basis
column, although it’s not a basis variable. The same applies to w.

The first feasible basic solution to the nine variable LP is (0, 0, 0, 0, 0, 0, 1000, 1500, 500) and
the basis includes x7, x8 and x9. For the LP to be in a canonical form, the basic variables must
have coefficients zero in the objective function, so we need to change this system of five
equations into an equivalent one,

Basis x1 x2 x3 x4 x5 x6 x7 x8 x9 RHS

-z 1000 1000 1000 0 0 0 0 0 0 0

-w -3 -7 -5 1 1 1 0 0 0 -3000

x7 7 -2 1 -1 0 0 1 0 0 1000

x8 -5 6 1 0 -1 0 0 1 0 1500

x9 1 3 3 0 0 -1 0 0 1 500

Next, we move from the first feasible basic solution to the second one.

Question 16: Which variable should enter the basis in accordance with the simplex algorithm?
(please enter the index of this variable)
Question 17: Which variable should exit the basis? (please enter the index of this variable)
The tableau after isolating the variable that has entered the basis is

Basis x1 x2 x3 x4 x5 x6 x7 x8 x9 RHS
 -z 1? 0 0 0 0 1000 0 0 1000 500000
− −
3 3 3 3

-w 2? 0 2 1 1 4 0 0 7 5500
− − −
3 3 3 3

3? 0 3 -1 0 2 1 0 2 4?
−
3 3 3 3

-7 0 -5 0 -1 2 0 1 -2 5?

1 1 1 0 0 1 0 0 1 6?
−
3 3 3 3

Question 18: Replace the 1? with the correct number.

Question 19: Replace the 2? with the correct number.
Question 20: Replace the 3? with the correct number.
Question 21: Replace the 4? with the correct number.
Question 22: Replace the 5? with the correct number.
Question 23: Replace the 6? with the correct number.

 7? 8? 
The second feasible basic solution is  0, , 0, 0, 0, 0, ,9?, 0  .
 3 3 

Question 24: Replace the 7? with the correct number.

Question 25: Replace the 8? with the correct number.
Question 26: Replace the 9? with the correct number.
The third feasible basic solution is (0, 250, 0, 0, 0, 10?, 11?, 0, 0).

Question 27: Replace the 10? with the correct number.

Question 28: Replace the 11? with the correct number.
After one more step, all the coefficients in the expression for w are non-negative and we have the
 1125 3875 9875 
optimal solution to the nine variable LP,  , , 0, 0, 0, , 0, 0, 0  .
 4 8 8 

w is now optimized and a feasible basic solution for the six variable LP is
 1125 3875 9875 
 , , 0, 0, 0, .
 4 8 8 

Generally, the next step would be optimizing z, but the coefficients of variables x1 to x6 in z in
1375 1125
the last tableau are 0, 0, 12?, , and 0, they are all non-negative (x7, x8 and x9 are zero
4 4
now, their coefficients don’t matter), so z happened to be already optimized.

Question 29: Replace the 12? with the correct number.

Question 30: How much money (in dollars) should the councilors invest into e-scooters?
Question 31: How much money (in dollars) should the councilors invest into cable cars?
Question 32: How much money (in dollars) should the councilors invest into sand?
Question 33: How much (in dollars) will it cost the councilors to win the election?
Verifying the answer in MATLAB,
f = [1000 1000 1000];
A = [-7 2 -1; 5 -6 -1; -1 -3 -3; -1 0 0; 0 -1 0; 0 0 -1];
b = [-1000 -1500 -500 0 0 0];
linprog(f,A,b)