3.

The Revised Simplex Method

Javier Larrosa Albert Oliveras Enric Rodr guez-Carbonell Problem Solving and Constraint Programming (RPAR)

Session 3 p.1/34

Tableau Simplex Method

The simplex algorithm we have seen so far is called tableau simplex algorithm Some observations: At each iteration we update the tableau xB = B 1 b B 1 RxR for the new basis Note: as A contains a unit matrix of slack variables, this involves computing B 1 For pricing only one negative reduced cost is needed For ratio test, only the column of the chosen non-basic variable in the tableau and the current basic solution are needed
Session 3 p.2/34

Revised Simplex Method

Idea: do not keep representation of full tableau, only B 1 Advantages over the tableau version: Time and space are saved Errors due to oating-point are easier to control We will develop revised algorithm for LPs of the form
min z = cT x Ax = b x0

Session 3 p.3/34

Basic Solution
For any basis B , the values of basic variables can be expressed in terms of non-basic variables:
BxB + RxR = b BxB = b RxR xB = B 1 b B 1 RxR

By denition, basic solution corresponds to assigning null values to all non-basic variables Basic solution: := B 1 b

Session 3 p.4/34

Optimality Conditions (1)

Basic variables in terms of non-basic variables:
xB = B 1 b B 1 RxR
T x , where x + c Cost function can be split: cT x = cT B R R B

cT B are the costs of basic variables, cT R are the costs of non-basic variables

Expressing cost function in terms of non-basic variables

cT x =
Tx = x + c cT B R R B 1 b B 1 Rx ) + cT x = cT ( B R B R R 1 b cT B 1 Rx + cT x = B cT R R R B B 1 b + ( c T cT B 1 R ) x B cT R B R B
Session 3 p.5/34

Optimality Conditions (2)

1 b + ( c T cT B 1 R ) x B c T x = cT R B R B

Let aj be the column in A corresponding to variable xj . 1 a B Let us dene reduced cost of xj as dj := cj cT j B

T cT B 1 R, so cT x = cT B 1 b + dT x = c Note that dT B R R R R B

Thus this notion of reduced cost matches with that used in the tableau version of the simplex method

Session 3 p.6/34

Optimality Conditions (3)

1 b + dT x , where d = c cT B 1 a B c T x = cT j j j R R B B

Value z of cost function at current basic solution: 1 b B make xR = 0 in above equation, i.e., z := cT B To avoid repeating computations: Let us dene the simplex multiplier as := (B T )1 cB , 1 B or equivalently, T := cT B Then dj = cj T aj and z = T b (note z = cT B holds too) Optimality condition: dj 0 for all j R

Session 3 p.7/34

Improving a Non-Optimal Solution (1)

Let us assume reduced cost of non-basic variable xq violates the optimality condition: dq < 0 Current value of xq is 0. We can improve by increasing only this value while sign constraints of basic variables are satised. Let t 0 be new value for xq . Then: (recall: the value of xp is null if p R and p = q )
xB (t) = B 1 b B 1 RxR = B 1 b B 1 aq t = tq

where = B 1 b and q = B 1 aq

Session 3 p.8/34

Improving a Non-Optimal Solution (2)

How much do we improve? How does the objective value change as a function of t?
z ( t) = cT x ( t ) =
T x ( t) = x ( t ) + c cT R R B B

cT B x B ( t ) + cq t =
T + c t = cT tc q B B q T B 1 a + c t = tc cT q q B B

z + tdq

Hence the improvement in cost is z = z (t) z = tdq

Session 3 p.9/34

Improving a Non-Optimal Solution (3)

How can we satisfy sign constraints of basic variables?
i Let i {1, ..., m}. Then i-th component of xB (t) is i tq i 0 We need i tq i 0 the constraint is satised for all t 0 If q i > 0 we need If q i i q

The best improvement is achieved with

i i := min{ i | q > 0} q

We say the p-th basic variable is blocking when = p Then q is the pivot

p . p q

Session 3 p.10/34

Improving a Non-Optimal Solution (4)

i > 0): 1. If = + (there is no i such that 1 i m and q

Value of objective function can be decreased innitely. LP is unbounded. 2. If < + and the p-th basic variable is blocking: When setting xq = sign of basic variables is respected
p In particular, the p-th component of xB (t) is p q =0

We can make a basis change: xq enters the basis and xkp leaves, where B = (k1 , ..., km )

Session 3 p.11/34

Update
= B {kp } {q } New basic indices: B p = , i = i i if i = p New basic solution: q = B + (aq ak )eT New basis: B p p
T + (c c )eT = c New basic costs: c T q kp p B B p) T = (0 , ..., 0 , 1 , 0 , ..., 0) where eT p m)

New objective value: z = z + dq

Session 3 p.12/34

Algorithmic Description
1. Initialization: Find an initial feasible basis B Compute B 1 , = B 1 b, z = cT B
1 and d = c T a . B 2. Pricing: Compute = cT j j j B If for all j R, dj 0 then return OPTIMAL Else let q be such that dq < 0. Compute q = B 1 aq i > 0}. 3. Ratio test: Compute I = {i | 1 i m, q If I = then return UNBOUNDED p i Else compute = miniI ( and p such that = ) i p q q

4. Update:
= B {kp } {q } B p = , i = i i if i = p q = B + (aq ak )eT B p p z = z + dq
Session 3 p.13/34

Go to 2.

Updating Matrix Inverse (1)

Actually what we really care about is B 1 , not B 1 and = B 1 a B We need it for computing = cT q q B Recomputing B 1 at each iteration is too expensive (e.g. cubic with Gaussian elimination!)
1 using B 1 There is efcient way of computing B

Session 3 p.14/34

Updating Matrix Inverse (2)

Let b1 , ..., bm be the columns of B Let a be such that a = Bv = m i=1 vi bi Let p be such that 1 p m
1 Ba = (b1 , . . . , bp1 , a, bp+1 , . . . , bm ). Want to compute Ba

Note vp = 0 as otherwise rank(Ba ) < m. So

a = v p bp +
i = p v i bi v1 vp

bp = ,
1 vp

1 vp

a+
vp+1 vp

i =p

vi vp vm vp

bi
T

Let T =

,...,

vp1 vp

,...,

E = (e1 , . . . , ep1 , , ep+1 , . . . , em )

1 = EB 1 Then Ba E = B = Ba

Session 3 p.15/34

Updating Matrix Inverse (3)

Application to the simplex algorithm: a = aq , v = q , where xq is entering variable Thus updating inverse reuses already computed data! Assume initial basis is B0 (e.g., unit matrix I ). Then at the k -th iteration of the simplex algorithm 1 the inverse matrix is B 1 = Ek Ek1 E2 E1 B0 , where Ei is the E matrix of the i-th iteration Each E matrix can be stored compactly
1 Can represent B 1 as list Ek Ek1 E2 E1 plus B0 : Product Form of the Inverse (PFI)

When list is long, matrix is recomputed (reinversion) Other ways of representing B 1 : LU factoritzation
Session 3 p.16/34

Bounded Variables
We want to solve with revised simplex LPs of the form
min cT x Ax = b xu ( k uk +)

In practice internally variables translated so that = or = 0 to save arithmetic operations Variable xk is lower bounded if k > Variable xk is upper bounded if uk < + Variable xk is range if k > and uk < + Variable xk is xed if range and k = uk Variable xk is free if k = and uk = +
Session 3 p.17/34

Basic Solution
For any basis B , the values of basic variables can be expressed in terms of non-basic variables:
xB = B 1 b B 1 RxR

Values a non-basic variable xj can be assigned to: If lower bounded: k (it is at lower bound; xj L) If upper bounded: uk (it is at upper bound; xj U ) 0 (it is at zero level; xj Z ) If free: Basic solution:
:= B 1 b
j L

B 1 aj j
j U

B 1 aj uj
Session 3 p.18/34

Optimality Conditions
Cost function in terms of non-basic variables:
1 b + d x B cT x = cT R R B 1 a for all variable x B where dj = cj cT j j B

If xj L: cannot improve if dj 0 If xj U : cannot improve if dj 0 If xj Z : cannot improve if dj = 0 Optimality condition: no improving non-basic variable

Session 3 p.19/34

Objective Function
Cost function in terms of non-basic variables:
1 b + d x B cT x = cT R R B 1 a for all variable x B where dj = cj cT j j B

Value z of cost function at current basic solution:

1 B b+ z = cT B j L

dj j +
j U

dj uj

Session 3 p.20/34

Improving a Non-Optimal Solution (1)

Let xq be non-basic variable that can improve objective value by increasing its value. This can happen when xq is lower bounded and xq L; or xq is free (so xq Z ) Since increasing xq can improve objective value: dq < 0 Let t 0 be difference of new value xq wrt old value
x B ( t) = B 1 b B 1 RxR = B 1 b tB 1 aq = tq
1 a B j j j L 1 a u B j j j U

where = B 1 b and q = B 1 aq
Session 3 p.21/34

Improving a Non-Optimal Solution (2)

How does the objective value change as a function of t?
z (t) = cT x(t) =
T cT B xB (t) + cR xR (t) =

cT B xB (t) + tcq +
j L 1 B b+ cT B 1 b+ cT BB j L

c j j +
j U

cj uj =
1 1 (c j c T aj )uj + tcq tcT aq = BB BB j U

1 (c j c T aj ) j + BB j L

dj j +
j U

1 dj uj + tcq tcT aq = BB

1 z + tcq tcT B aq = B

z + tdq

Hence the improvement in cost is z = z (t) z = tdq

Session 3 p.22/34

Improving a Non-Optimal Solution (3)

i Let i {1, ..., m}. Then i-th component of xB (t) is i tq Let xki be the i-th basic variable. Let i := ki , i := uki i We need i i tq i i > 0: ti If q i i q i < 0: ti If q i i q

i i i q i i i q

t t

But we need xq (t) uq too! Best improvement achieved with

i i > 0}, min{ i i | i < 0}) | := min(uq q , min{ i i q q i q q

If = + we have unboundedness Else if = uq q we have a bound ip: no pivot needed!

Session 3 p.23/34

Improving a Non-Optimal Solution (4)

i i > 0}, min{ i i | i < 0}) := min(uq q , min{ i | i q q i q q

Assume = +, = uq q . Thus variable xq enters basis and variable xkp leaves

i i > 0}, x leaves basis at lower bound If = min{ i | i kp q q i i < 0}, x leaves basis at upper bound If = min{ i | i kp q q

Session 3 p.24/34

Improving a Non-Optimal Solution (5)

Let xq be non-basic variable that can improve objective value by decreasing its value. This can happen when
xq is upper bounded and xq U ; or xq is free (so xq Z )

Since decreasing xq can improve objective value: dq > 0 Let t 0 be difference of new value xq wrt old value Again xB (t) = tq , where = B 1 b and q = B 1 aq Again the improvement in cost is z = z (t) z = tdq
Session 3 p.25/34

Improving a Non-Optimal Solution (6)

i Let i {1, ..., m}. Then i-th component of xB (t) is i tq Let xki be the i-th basic variable. Let i := ki , i := uki i We need i i tq i i > 0: ti If q i i q i < 0: ti If q i i q

i i i q i i i q

t t

But we need q xq (t) too! Best improvement achieved with

i i < 0}, max{ i i | i > 0}) | := max(q uq , max{ i i q q i q q

If = we have unboundedness Else if = q uq we have a bound ip: no pivot needed!

Session 3 p.26/34

Improving a Non-Optimal Solution (7)

i i < 0}, max{ i i | i > 0}) := max(q uq , max{ i | i q q i q q

Assume = , = q uq . Thus variable xq enters basis and variable xkp leaves

i i < 0}, x leaves basis at lower bound If = max{ i | i kp q q i i > 0}, x leaves basis at upper bound If = max{ i | i kp q q

Session 3 p.27/34

Update
New objective value: z = z + dq If bound ip Flip status of xq (i.e., x q L xq U ) = q New basic solution: Else
= B {kp } {q } New basic indices: B p = xq + , i = i i if i = p New basic solution: q 1 = EB 1 New basis inverse: B If entering variable comes from lower bound = L {xq } else U = U {xq } L If leaving variable leaves to lower bound =L {xk } else U =U {xk } L p p
Session 3 p.28/34

Tableau vs. Revised Simplex

Time is saved: # Tableau: all dk , all k are computed  Revised: no. of non-basic variables xk for which dk , k are computed can be adjusted Space is saved: # Tableau: even if A sparse, tableau tends to get lled  Revised: sparsity of A can be exploited for storage, and pivots can be chosen to represent B 1 compactly Better numerical behaviour: # Tableau: errors due to oating-point arithmetic accumulate at each pivoting step  Revised: reinversion (PFI representation of B 1 ) or refactorization (LU representation of B 1 ) for resetting
Session 3 p.29/34

Original vs. Bounds Simplex

Time is saved: # Original: no special treatment of bounds  Bounds: bound ips are much cheaper than pivoting steps in simplex iterations Space is saved: # Original: each bound constraint becomes a row  Bounds: bounds are stored cheaply in arrays

Session 3 p.30/34

Sensitivity Analysis
Data may be inaccurate (observations, estimations): small changes mean completely different solutions? Sensitivity analysis = study of the stability of optimal solution with respect to small changes in problem data Let B be a basis for
min cT x Ax = b x0 B gives an optimal feasible solution iff

Feasibility: B 1 b 0 T cT B 1 R 0 = c Optimality: dT B R R
Session 3 p.31/34

Changes in Cost Function

Assume ck changes to ck + k Feasibility is not affected If k R, only optimality for xk is affected:
1 a d B 0 ck + k c T k k k B

If k B and xk is p-th basic variable, for any j R

T ) B 1 a = d e T B 1 a = d p + e 0 cj ( c T j j j j k j k p k p B p k j dj

k
j

dj p j

p if j > 0,

k
j

dj p j

p if j <0

dj dj p p Thus max{ p | < 0} k min{ p | > 0} j j

Session 3 p.32/34

Changes in Right-Hand Side (1)

Assume bp , the p-th component of b, changes to bp + p Optimality is not affected Let p be the p-th column of B 1 and = B 1 b. The basic solution is feasible iff
0 B 1 (b + p ep ) = + p p
i for all 1 i m 0 i + p p i i p i if p > 0, p i i p i if p < 0 p

i i i i Thus max{ i | p > 0} p min{ i | p < 0} p p

Session 3 p.33/34

Changes in Right-Hand Side (2)

Assume bi , the i-th component of b, changes to bi + i , so that the optimal basis does not change How does this affect the objective function? Value of objective function at basic solution is
z ( b ) = cT B 1 b = T b ,
1 are simplex multipliers where T = cT B B

z ( b + i e i ) z ( b) = i T e i = i i

Can be used to determine price/benet of changing a resource, capacity, limit, etc. In economics, are called shadow prices: marginal (by one unit) cost of strengthening a constraint
Session 3 p.34/34