The Revised Simplex Method

Combinatorial Problem Solving (CPS)

Javier Larrosa Albert Oliveras Enric Rodrı́guez-Carbonell

March 24, 2023

Tableau Simplex Method

■ The simplex method we have seen so far is called

tableau simplex method

■ Some observations:

◆ At each iteration we update the full tableau

xB = B −1 b − B −1 RxR
for the new basis

◆ But ...
■ For pricing only one negative reduced cost is needed
■ For ratio test, only
◆ the column of the chosen non-basic variable in the tableau, and
◆ the current basic solution
are needed
2 / 17
Revised Simplex Method
■ Idea: do not keep a representation of the full tableau, only B −1
■ Advantages over the tableau version:

◆ Time and space are saved

◆ Errors due to floating-point arithmetic are easier to control

3 / 17
Revised Simplex Method
■ Idea: do not keep a representation of the full tableau, only B −1
■ Advantages over the tableau version:

◆ Time and space are saved

◆ Errors due to floating-point arithmetic are easier to control

■ We will revise the algorithm and express it in terms of B −1

■ For simplicity, let us revise the version of the algorithm for LPs of the form
min z = cT x
Ax = b
x≥0

3 / 17
Basic Solution
■ Let us see how the basic solution is expressed in terms of B −1

■ For any basis B,

values of basic variables can be expressed in terms of non-basic variables:

BxB + RxR = b
BxB = b − RxR
xB = B −1 b − B −1 RxR

4 / 17
Basic Solution
■ Let us see how the basic solution is expressed in terms of B −1

■ For any basis B,

values of basic variables can be expressed in terms of non-basic variables:

BxB + RxR = b
BxB = b − RxR
xB = B −1 b − B −1 RxR

■ By definition, the basic solution corresponds to

assigning null values to all non-basic variables: xR = 0
Then xB = B −1 b

■ We will denote the basic solution (projected on basic variables) with

β := B −1 b

4 / 17
Optimality Condition
■ Let us see now how to express the reduced costs in terms of B −1

■ Recall the equation of basic variables in terms of non-basic variables:

xB = B −1 b − B −1 RxR

■ Cost function can be split: cT x = cTB xB + cTR xR , where

cTB are the costs of basic variables,
cTR are the costs of non-basic variables

5 / 17
Optimality Condition
■ Let us see now how to express the reduced costs in terms of B −1

■ Recall the equation of basic variables in terms of non-basic variables:

xB = B −1 b − B −1 RxR

■ Cost function can be split: cT x = cTB xB + cTR xR , where

cTB are the costs of basic variables,
cTR are the costs of non-basic variables

■ We can express the cost function in terms of non-basic variables:

cT x =
cTB xB + cTR xR =
cTB (B −1 b − B −1 RxR ) + cTR xR =
cTB B −1 b − cTB B −1 RxR + cTR xR =
cTB B −1 b + (cTR − cTB B −1 R)xR

5 / 17
Optimality Condition

■ We found that cT x = cTB B −1 b + (cTR − cTB B −1 R)xR

■ The part that depends on non-basic variables is (cTR − cTB B −1 R)xR

■ Let aj be the column in A corresponding to variable xj ∈ xR .

The coefficient of xj in (cTR − cTB B −1 R)xR is cj − cTB B −1 aj

We will denote the reduced cost of xj with dj := cj − cTB B −1 aj

■ Optimality condition: dj ≥ 0 for all j ∈ R

6 / 17
Cost at Basic Solution
■ Let’s see how to express the value of the cost function at the basic solution

■ We found that cT x = cTB B −1 b + dTR xR , where dj = cj − cTB B −1 aj

■ We will denote the value of the cost function at the basic solution with z

■ Taking xR = 0 in the above equation: z := cTB B −1 b

7 / 17
Cost at Basic Solution
■ Let’s see how to express the value of the cost function at the basic solution

■ We found that cT x = cTB B −1 b + dTR xR , where dj = cj − cTB B −1 aj

■ We will denote the value of the cost function at the basic solution with z

■ Taking xR = 0 in the above equation: z := cTB B −1 b

■ To avoid repeating computations:

Let us define the simplex multiplier as π := (B T )−1 cB

Then π T = cTB B −1

So dj = cj − π T aj
(and z = π T b)

7 / 17
Improving a Non-Optimal Solution

■ Let us assume the optimality condition is violated

■ Let xq be a non-basic variable such that its reduced cost is dq < 0

■ Current value of xq is 0.
We can improve by increasing only this value
while non-negativity constraints of basic variables are satisfied.

8 / 17
Improving a Non-Optimal Solution

■ Let us assume the optimality condition is violated

■ Let xq be a non-basic variable such that its reduced cost is dq < 0

■ Current value of xq is 0.
We can improve by increasing only this value
while non-negativity constraints of basic variables are satisfied.

■ Let t ≥ 0 be the new value for xq .

Let xB (t) be the values of basic variables in terms of t
Let xR (t) be the values of non-basic variables in terms of t
Note that xq (t) = t, and xp (t) = 0 if p ∈ R and p 6= q

8 / 17
Improving a Non-Optimal Solution

■ Let us assume the optimality condition is violated

■ Let xq be a non-basic variable such that its reduced cost is dq < 0

■ Current value of xq is 0.
We can improve by increasing only this value
while non-negativity constraints of basic variables are satisfied.

■ Let t ≥ 0 be the new value for xq .

Let xB (t) be the values of basic variables in terms of t
Let xR (t) be the values of non-basic variables in terms of t
Note that xq (t) = t, and xp (t) = 0 if p ∈ R and p 6= q

So xB (t) = B −1 b − B −1 RxR (t) = B −1 b − B −1 aq t = β − tαq

where β = B −1 b is the basic solution

and we denote the column in the tableau of xq as αq := B −1 aq
8 / 17
Improving a Non-Optimal Solution
■ How much do we improve?
How does the objective value change as a function of t?

z(t) =
cT x(t) =
cTB xB (t) + cTR xR (t) =
cTB xB (t) + cq t =
cTB β − tcTB αq + cq t =
cTB β − tcTB B −1 aq + cq t =
z + tdq

■ As expected, the improvement in cost is ∆z = z(t) − z = tdq

9 / 17
Improving a Non-Optimal Solution
■ Recall that xB (t) = β − tαq
■ How can we satisfy the non-negativity constraints of basic variables?

10 / 17
Improving a Non-Optimal Solution
■ Recall that xB (t) = β − tαq
■ How can we satisfy the non-negativity constraints of basic variables?
■ Basic variables have indices B = (k1 , ..., km )
■ Let i ∈ {1, ..., m}. The i-th basic variable is xki
■ Value of xki as a function of t is the i-th component of xB (t): βi − tαqi ,
where βi is the i-th component of β and αqi is the i-th component of αq

10 / 17
Improving a Non-Optimal Solution
■ Recall that xB (t) = β − tαq
■ How can we satisfy the non-negativity constraints of basic variables?
■ Basic variables have indices B = (k1 , ..., km )
■ Let i ∈ {1, ..., m}. The i-th basic variable is xki
■ Value of xki as a function of t is the i-th component of xB (t): βi − tαqi ,
where βi is the i-th component of β and αqi is the i-th component of αq
■ We need βi − tαqi ≥ 0 ⇐⇒ βi ≥ tαqi

◆ If αqi ≤ 0 the constraint is satisfied for all t ≥ 0

βi
◆ If αqi > 0 we need αiq
≥t

10 / 17
Improving a Non-Optimal Solution
■ Recall that xB (t) = β − tαq
■ How can we satisfy the non-negativity constraints of basic variables?
■ Basic variables have indices B = (k1 , ..., km )
■ Let i ∈ {1, ..., m}. The i-th basic variable is xki
■ Value of xki as a function of t is the i-th component of xB (t): βi − tαqi ,
where βi is the i-th component of β and αqi is the i-th component of αq
■ We need βi − tαqi ≥ 0 ⇐⇒ βi ≥ tαqi

◆ If αqi ≤ 0 the constraint is satisfied for all t ≥ 0

βi
◆ If αqi > 0 we need αiq
≥t

■ The best improvement is achieved with the strongest of the upper bounds:

θ := min{ αβii | αqi > 0}

q

βp
■ We say the p-th basic variable xkp is blocking or tight when θ = αpq
.
Then αqp is the pivot 10 / 17
Improving a Non-Optimal Solution
1. If θ = +∞
(there is no upper bound, i.e., no i such that 1 ≤ i ≤ m and αqi > 0):

Value of objective function can be decreased infinitely.

LP is unbounded.

11 / 17
Improving a Non-Optimal Solution
1. If θ = +∞
(there is no upper bound, i.e., no i such that 1 ≤ i ≤ m and αqi > 0):

Value of objective function can be decreased infinitely.

LP is unbounded.

2. If θ < +∞ and the p-th basic variable xkp is blocking:

When setting xq = θ, the non-negativity of basic variables is respected

In particular the value of xkp , i.e. the p-th component of xB (t), is

βp − θαqp = 0

We can make a basis change:

xq enters the basis and xkp leaves, where B = (k1 , ..., km )

11 / 17
Update
■ New basic indices: B̄ = (k1 , . . . , kp−1 , q, kp+1 . . . , km )

Before the p-th basic variable was xkp , now it is xq

■ New basis: B̄ = B + (aq − akp )eTp

p
z}|{
T
where ep = (0, ..., 0, 1 , 0, ..., 0) is the p-th unit vector
| {z }
m

The p-th column of the basis (which was akp ) is replaced by aq .

■ New basic solution: β̄p = θ, β̄i = βi − θαqi if i 6= p

Note that before the p-th component of β corresponded to xkp , now to xq

■ New objective value: z̄ = z + θdq

12 / 17
Algorithmic Description

1. Initialization: Find an initial feasible basis B

Compute B −1 , β = B −1 b, z = cTB β

2. Pricing: Compute π T = cTB B −1 and dj = cj − π T aj .

If for all j ∈ R, dj ≥ 0 then return OPTIMAL
Else let q be such that dq < 0. Compute αq = B −1 aq

3. Ratio test: Compute I = {i | 1 ≤ i ≤ m, αqi > 0}.

If I = ∅ then return UNBOUNDED
β
Else compute θ = mini∈I ( αβii ) and p such that θ = αpp
q q

4. Update:
B̄ = B − {kp } ∪ {q} B̄ = B + (aq − akp )eTp
β̄p = θ, β̄i = βi − θαqi if i 6= p z̄ = z + θdq
Go to 2.

13 / 17
Updating Matrix Inverse
■ Actually what we really care about is B −1 , not B
We need it for computing π = cTB B −1 and αq = B −1 aq at each step
(and also β = B −1 b in the initialization)

■ Recomputing B −1 at each iteration is too expensive

(e.g. O(m3 ) arithmetic operations with Gaussian elimination!)

■ Next slides: a more efficient way of computing B̄ −1 using B −1

14 / 17
Updating Matrix Inverse
■ Let us make a diversion into linear algebra
■ Let b1 , ..., bm be the columns of an invertible matrix B
Pm
Let a, α be such that a = Bα = i=1 αi bi
Let p be such that 1 ≤ p ≤ m
■ Ba = (b1 , . . . , bp−1 , a, bp+1 , . . . , bm ). Want to compute Ba−1

15 / 17
Updating Matrix Inverse
■ Let us make a diversion into linear algebra
■ Let b1 , ..., bm be the columns of an invertible matrix B
Pm
Let a, α be such that a = Bα = i=1 αi bi
Let p be such that 1 ≤ p ≤ m
■ Ba = (b1 , . . . , bp−1 , a, bp+1 , . . . , bm ). Want to compute Ba−1
■ Note αp 6= 0 as otherwise rank(Ba ) < m.
P   P  
Then a = αp bp + i6=p αi bi ⇒ bp = α1p a + i6=p −α
αp
i
bi
        T
−α1 −αp−1 −αp+1 −αm
■ Let η T = αp ,..., αp , α1p , αp ,..., αp .
Then bp = Ba η

■ Let E = (e1 , . . . , ep−1 , η, ep+1 , . . . , em )

where eq is the q-th unit vector for 1 ≤ q ≤ m.
Then Ba E = B =⇒ E −1 Ba−1 = B −1 =⇒ Ba−1 = EB −1
15 / 17
Updating Matrix Inverse
■ Application to the simplex algorithm:
a = aq , α = αq , where xq is entering variable
Thus to update the inverse we can reuse already computed data!

■ Using this update: B −1 is not actually represented as a square table,

but as follows

■ Assume initial basis is B0 (e.g., unit matrix I).

Then at the k-th iteration of the simplex algorithm
the inverse matrix is B −1 = Ek Ek−1 · · · E2 E1 B0−1 ,
where Ei is the E matrix of the i-th iteration
■ Each E matrix can be stored compactly (vector η + column index p)
■ We can represent B −1 as the list Ek , Ek−1 , . . . , E2 , E1 , B0−1 :
Product Form of the Inverse (PFI)
■ When the list is long we reset: the inverse is computed (reinversion)
■ Other ways of representing B −1 : LU factoritzation
16 / 17
Tableau vs. Revised Simplex
■ Time is saved:
✗ Tableau: all dk , all αk are computed
✓ Revised: no. of non-basic variables xk for which
dk , αk are computed can be adjusted

■ Space is saved:
✗ Tableau: even if A sparse, tableau tends to get filled
✓ Revised: sparsity of A can be exploited for storage,
and pivots can be chosen to represent B −1 compactly

■ Better numerical behaviour:

✗ Tableau: errors due to floating-point arithmetic accumulate at each
pivoting step
✓ Revised: reinversion (PFI representation of B −1 ) or
refactorization (LU representation of B −1 ) can be used for resetting

17 / 17