IE 311 – Operations Research I

Part 3: Analysis of Linear Programming

3b. Simplex Method for ‘Structured’ LPs

Spring 2021
Recall: Linear Programming (Standard Form)

• Recall that any Linear Program can be written as below:

• Now, we will start understanding how LPs can be solved.

IE 311 2
 How Does Gurobi Solve LPs?

IE 311 3
Main Steps of the (Primal) Simplex Method

The simplex algorithm proceeds as follows:

• Step 1 Convert the LP to standard form.
• Step 2 Obtain a BFS (if possible) from the standard form.
• Step 3 Determine whether the current BFS is optimal.
• Step 4 If the current BFS is not optimal, then determine which
nonbasic variable should become a basic variable and which basic
variable should become a nonbasic variable to find a new BFS with a
better objective function value (or determine if the problem is unbounded).
• Step 5 Use EROs to find a new BFS with a better objective function
value. Go back to step 3.
IE 311 4
 Roadmap

• Although the Simplex Method can solve any LP in principle, we will first
look at well-structured LPs (those with an obvious initial BFS) and then
proceed with unstructured LPs (those without an obvious initial BFS).
• In particular, we will first learn how to solve maximization problems with
only ≤ constraints and a nonnegative right-hand-side (RHS) 𝒃 vector.
• In this case, an obvious BFS is available (WHY?)
• Then, we will discuss how to solve minimization problems with
minor modifications.
• Finally, we will learn how to incorporate ≥ and = constraints with an
arbitrary RHS vector.
• In these cases, some preliminary operations (Big-M or Two-Phase
methods) will be needed to obtain an initial BFS.
IE 311 5
A First Example

• We will first go over the following example LP and explain the steps of
𝑥2
the Simplex Method.

0
3 2
2

𝑥1
0 3
0 0
• For convenience, we defined a new variable 𝑥0 (column-0), which
represents the objective function value (row-0).

IE 311 6
A First Example: Step 1

• In Step 1, we convert the LP to standard form:

IE 311 7
 A First Example: Step 2
𝑥2
• In Step 2, we obtain an initial BFS:

0
3 2
2

𝑥1
0 3
0 0
• Convention: Variable 𝑥0 will ALWAYS be basic.
• Note: Since the original LP has only ≤ constraints with nonnegative
RHS, the slack variables 𝑥3 and 𝑥4 are chosen as basic variables.
• RHS column in this tableau gives the current values of the basic variables.
IE 311 8
 A First Example: Step 3

• In Step 3, we check whether the current BFS is optimal (look at row-0):

• Idea: If we move to an adjacent BFS, can we improve the objective

function variable 𝑥0 ?
• Moving to an adjacent BFS means that we make one of the nonbasic variables
basic while making one of the basic variables nonbasic.
• Note that increasing the value of 𝒙𝟏 or 𝒙𝟐 would make 𝒙𝟎 larger (WHY?)
• Therefore, the current BFS is NOT proven to be optimal.
IE 311 9
 A First Example: Step 4

• In Step 4, we first decide which nonbasic variable to make basic (look at

row-0):

• Idea: Intuitively, choosing the most negative coefficient in row-0

might improve the objective largest (although any nonbasic variable
with negative coefficient can be chosen).
• In this case, we can choose the nonbasic variable 𝒙𝟐 and make it basic.
• Such a variable will be called the entering variable.

IE 311 10
 A First Example: Step 4
• In Step 4, we next decide which basic variable to make nonbasic (look at
the column of the entering variable and RHS):

• Idea: What is the adjacent BFS obtained by making 𝑥2 basic?

• We need to perform the so-called minimum ratio test:
• Obtain the entrywise ratio of the RHS and the column of the entering
variable among the rows that have positive entry in the column of the
entering variable.
• Choose the basic variable that gives the minimum ratio as the leaving
variable (if there is no such variable, the LP is unbounded):
6 6
IE 311
The leaving variable is 𝑥4 since min , = 3. 11
1 2
A First Example: Step 5

• In Step 5, we use EROs to find a new BFS with a

better objective function value (pivoting):
𝑥2

0
3
2
• Idea: How can we make the column of the 2
entering variable a unit vector by pivoting?
𝑥1
0 3
0 0

IE 311 12
 A First Example: Step 3 (Again)

• In Step 3, we check whether the current BFS is optimal (look at row-0):

• Idea: If we move to an adjacent BFS, can we improve the objective

function variable 𝑥0 ?
• Note that increasing the value of 𝒙𝟏 would make 𝒙𝟎 larger.
• Therefore, the current BFS is NOT proven to be optimal.

IE 311 13
 A First Example: Step 4 (Again)

• In Step 4, we first decide which nonbasic variable to make basic (look at

row-0):

• Idea: Intuitively, choosing the most negative coefficient in row-0

might improve the objective largest (although any nonbasic variable
with negative coefficient can be chosen).
• In this case, there is only one candidate: the nonbasic variable 𝒙𝟏
enters the basis.
IE 311 14
A First Example: Step 4 (Again)

• In Step 4, we next decide which basic variable to make nonbasic (look

at the column of the entering variable and RHS):

• Idea: What is the adjacent BFS obtained by making 𝑥2 basic?

• We perform the minimum ratio test:
3 3
The leaving variable is 𝑥3 since min , = 2.
3/2 1/2

IE 311 15
A First Example: Step 5 (Again)

• In Step 5, we use EROs to find a new BFS with a better

objective function value (pivoting): 𝑥2

0
3
2
2
• Idea: How can we make the column of the
entering variable a unit vector by pivoting? 𝑥1
0 3
0 0

IE 311 16
 A First Example: Step 3 (Again)

• In Step 3, we check whether the current BFS is optimal (look at row-0):

• Idea: If we move to an adjacent BFS, can we improve the objective

function variable 𝑥0 ?
• Note that increasing the value of 𝒙𝟑 or 𝒙𝟒 would make 𝒙𝟎 smaller.
• Therefore, the current BFS is OPTIMAL with objective value 14.
• Optimal solution: 𝑥1 = 𝑥2 = 2.
IE 311 17
Main Steps of the Simplex Method: Structured LPs

• Consider the following structured LP with 𝒃 ≥ 𝟎:

IE 311 18
Main Steps of the Simplex Method: Structured LPs

• Step 1 Convert the LP to standard form.

IE 311 19
Main Steps of the Simplex Method: Structured LPs

• Step 2 Obtain a BFS from the standard form.

• Note that the slack variables 𝑥𝑛+1 , … , 𝑥𝑛+𝑚 constitute an obvious basis
(since 𝑏 ≥ 0).
• We construct the following simplex tableau:

IE 311 20
Main Steps of the Simplex Method: Structured LPs

• Step 3 Determine whether the current BFS is optimal.

• Look at the coefficient in the row-0 of the current simplex
tableau (except 𝑥0 ).
• If all the entries are nonnegative, then STOP. The current BFS is
optimal.
• Otherwise, we cannot prove the optimality of the current BFS.
Proceed with Step 4.

IE 311 21
Main Steps of the Simplex Method: Structured LPs

• Step 4 Determine which nonbasic variable should become a basic

variable.
• Any nonbasic variable with negative coefficient in row-0 can be
selected as the entering variable (this will guarantee the objective value to
improve-or at least not to deteriorate).
• As a common practice, we will use the following rule: choose the
variable with the most negative coefficient.
• If there is a tie, we can choose the variable with the smaller index.

IE 311 22
Main Steps of the Simplex Method: Structured LPs

• Step 4 Determine which basic variable should become a nonbasic

variable.
• We need to employ the minimum ratio test (this will guarantee the next
BFS to remain feasible) to find the leaving variable:
• Let 𝐶𝑂𝐿 be the column of the entering variable and 𝑅𝐻𝑆 be the right-hand-
side in the current tableau (ignore row-0 elements).
𝑅𝐻𝑆𝑖
• Find a row 𝑖∗ such that 𝑖∗ ∈ arg min .
𝑖:𝐶𝑂𝐿𝑖 >0 𝐶𝑂𝐿 𝑖
• If there is no row with 𝐶𝑂𝐿𝑖 > 0, then STOP. The LP is unbounded!
• Otherwise, choose the basic variable in row 𝑖 ∗ as the leaving variable.
• If there is a tie, we can choose the variable with the smaller index.

IE 311 23
Main Steps of the Simplex Method: Structured LPs

• Step 5 Use EROs to find a new BFS with a better objective function
value.
• Use the entry in the intersection of the column of the entering
variable and the row of the leaving variable as pivot.
• Apply EROs to obtain the new BFS.
• Go to Step 3.

IE 311 24
Some Useful Observations

• The value of the entering variable in the next BFS is equal to the
minimum ratio.
• The change in the objective value is equal to the minimum ratio
times (minus) the objective coefficient of the entering variable.
• If there is a tie in the minimum ratio, the next BFS is degenerate.
• If the objective coefficient of a nonbasic variable is 0 in the optimal
tableau, then there exist multiple (or alternative) optima.
• If the LP is unbounded, the column of the entering variable can be
used to determine a direction of unboundedness which makes
positive inner product with the objective vector.

IE 311 25
Example 2

• Solve the following LP using the simplex method:

• Before pivoting, state the value of the entering variable in the

next BFS and the change in the objective function value.
IE 311 26
 Example 2: Iteration 1

• Entering Variable: 𝑥1 (most negative coefficient = −6)

• Leaving Variable: 𝑥6 (minimum ratio = 4)

IE 311 27
 Example 2: Iteration 2

• Entering Variable: 𝑥3 (most negative coefficient = −1/2)

• Leaving Variable: 𝑥5 (minimum ratio = 8)

IE 311 28
Example 2: Iteration 3

• STOP: This is the optimal tableau (there is a unique optimal solution

since objective coefficients of all nonbasic variable are positive).

IE 311 29
Degenerate BFS: Example 3

• If there is a tie in the minimum ratio, the next BFS is degenerate.

• Solve the following LP using the simplex method:
𝑥2

0
4 2
2

𝑥1
0 5/2
0 0

IE 311 30
 Example 3: Iteration 1

𝑥2

0
4 2
• Entering Variable: 𝑥1 (most negative coefficient = −5) 2
• Leaving Variable: 𝑥4 (minimum ratio = 5/2)
𝑥1
0 5/2
0 0

IE 311 31
 Example 3: Iteration 2

𝑥2

0
4 2
• Entering Variable: 𝑥2 (most negative coefficient = −7/4) 2
• Leaving Variable: 𝑥3 (minimum ratio = 2 – there is a tie!)
𝑥1
0 5/2
0 0

IE 311 32
 Example 3: Iteration 3

𝑥2

0
4 2
• Entering Variable: 𝑥4 (most negative coefficient = −1/2) 2
• Leaving Variable: 𝑥5 (minimum ratio = 0 – degenerate pivot)
𝑥1
0 5/2
0 0

IE 311 33
Example 3: Iteration 4

• STOP: This is the optimal tableau (there is a unique optimal solution

since objective coefficients of all nonbasic variable are positive).

IE 311 34
Multiple (or Alternative) Optima: Example 4

• If the objective coefficient of a nonbasic variable is 0 in the optimal

tableau, then there exist multiple (or alternative) optima.
• Solve the following LP using the simplex method:
𝑥 2

56/17
45/17

0
1
𝑥1
0 7
0 0

IE 311 35
Example 4: Iteration 1

𝑥2

56/17
45/17

• Entering Variable: 𝑥2 (most negative coefficient = −6) 0

• Leaving Variable: 𝑥4 (minimum ratio = 1) 1
𝑥1
0 7
0 0

IE 311 36
Example 4: Iteration 2

𝑥2

56/17
45/17

• STOP: This is an optimal tableau (multiple optima exist). 0

• What happens if we make 𝒙𝟏 enter the basis? 1
𝑥1
0 7
0 0

IE 311 37
Example 4: Follow-up

• The set of all optimal solutions lie in the 𝑥2

following line segment: 56/17

45/17

0
1
𝑥1
0 7
0 0

IE 311 38
Unbounded LPs: Example 5

• If the LP is unbounded, the column of the entering variable

can be used to determine a direction of unboundedness which
makes positive inner product with the objective vector.
• Solve the following LP using the simplex method:
𝑥2

0
3

𝑥1
0 4
0 0

IE 311 39
Example 5: Iteration 1

𝑥2

0
• Entering Variable: 𝑥2 (most negative coefficient = −3) 3
• Leaving Variable: 𝑥4 (minimum ratio = 3)
𝑥1
0 4
0 0

IE 311 40
Example 5: Iteration 2

𝑥2

0
3

𝑥1
0 4
• Entering Variable: 𝑥1 (most negative coefficient = −7) 0 0
• Leaving Variable: NONE! LP is unbounded.

• How to find the direction of unboundedness from the tableau?

IE 311 41
 Example 5: Follow-up

• Suppose 𝑥1 ‘enters’ the basis and its value is increased by 1 unit. Then,
• due to row-1, 𝑥3 is increased by 3 units,
• due to row-2, 𝑥2 is increased by 2 units,
• 𝑥4 is unaffected since it is nonbasic.
𝑑1 1
𝑑2 2
• ⇒ Direction of unboundedness (standard form): =
𝑑3 3
𝑑4 0
𝑑1 1
• ⇒ Direction of unboundedness (original variables): =
IE 311 𝑑2 2 42
Minimization Problems

• If we have a minimization LP, there are two options:

1. We can maximize the negative of the original objective
(no modification is needed in the simplex algorithm).
2. We can modify the simplex algorithm to solve a
minimization problem as follows:
• Step 3: If all the coefficients in row-0 are nonpositive, then we
STOP. This is an optimal tableau.
• Step 4: If this is not the case, we select the variable with the
most positive coefficient in row-0 as the entering variable.

IE 311 43
Tableau Implementation of the Simplex Method
with Initial BFS: Python Code

•Use the Python file

SimplexTableauWithBV.py
uploaded to Sucourse to solve
Examples 1-5.

IE 311 44