# 3.

The Revised Simplex Method
Javier Larrosa Albert Oliveras Enric Rodr´ ıguez-Carbonell Problem Solving and Constraint Programming (RPAR)

Session 3 – p.1/34

Tableau Simplex Method
The simplex algorithm we have seen so far is called tableau simplex algorithm Some observations: At each iteration we update the tableau xB = B −1 b − B −1 RxR for the new basis Note: as A contains a unit matrix of slack variables, this involves computing B −1 For pricing only one negative reduced cost is needed For ratio test, only the column of the chosen non-basic variable in the tableau and the current basic solution are needed
Session 3 – p.2/34

Revised Simplex Method
Idea: do not keep representation of full tableau, only B −1 Advantages over the tableau version: Time and space are saved Errors due to ﬂoating-point are easier to control We will develop revised algorithm for LP’s of the form
min z = cT x Ax = b x≥0

Session 3 – p.3/34

Basic Solution For any basis B . basic solution corresponds to assigning null values to all non-basic variables Basic solution: β := B −1 b Session 3 – p.4/34 . the values of basic variables can be expressed in terms of non-basic variables: BxB + RxR = b BxB = b − RxR xB = B −1 b − B −1 RxR By deﬁnition.

where x + c Cost function can be split: cT x = cT B R R B cT B are the costs of basic variables. cT R are the costs of non-basic variables Expressing cost function in terms of non-basic variables cT x = Tx = x + c cT B R R B −1 b − B −1 Rx ) + cT x = cT ( B R B R R −1 b − cT B −1 Rx + cT x = B cT R R R B B − 1 b + ( c T − cT B − 1 R ) x B cT R B R B Session 3 – p.5/34 .Optimality Conditions (1) Basic variables in terms of non-basic variables: xB = B −1 b − B −1 RxR T x .

so cT x = cT B −1 b + dT x = c Note that dT B R R R R B Thus this notion of reduced cost matches with that used in the tableau version of the simplex method Session 3 – p.6/34 . −1 a B Let us deﬁne reduced cost of xj as dj := cj − cT j B T − cT B −1 R.Optimality Conditions (2) − 1 b + ( c T − cT B − 1 R ) x B c T x = cT R B R B Let aj be the column in A corresponding to variable xj .

where d = c − cT B −1 a B c T x = cT j j j R R B B Value z of cost function at current basic solution: −1 b B make xR = 0 in above equation. i.7/34 .. π T := cT B Then dj = cj − π T aj and z = π T b (note z = cT B β holds too) Optimality condition: dj ≥ 0 for all j ∈ R Session 3 – p.e.Optimality Conditions (3) −1 b + dT x . z := cT B To avoid repeating computations: Let us deﬁne the simplex multiplier as π := (B T )−1 cB . −1 B or equivalently.

Then: (recall: the value of xp is null if p ∈ R and p = q ) xB (t) = B −1 b − B −1 RxR = B −1 b − B −1 aq t = β − tαq where β = B −1 b and αq = B −1 aq Session 3 – p.Improving a Non-Optimal Solution (1) Let us assume reduced cost of non-basic variable xq violates the optimality condition: dq < 0 Current value of xq is 0.8/34 . Let t ≥ 0 be new value for xq . We can improve by increasing only this value while sign constraints of basic variables are satisﬁed.

9/34 .Improving a Non-Optimal Solution (2) How much do we improve? How does the objective value change as a function of t? z ( t) = cT x ( t ) = T x ( t) = x ( t ) + c cT R R B B cT B x B ( t ) + cq t = Tα + c t = cT β − tc q B B q T B −1 a + c t = β − tc cT q q B B z + tdq Hence the improvement in cost is ∆z = z (t) − z = tdq Session 3 – p.

Then i-th component of xB (t) is βi − tαq i ≥0 We need βi − tαq i ≤ 0 the constraint is satisﬁed for all t ≥ 0 If αq i > 0 we need If αq βi αi q ≥t The best improvement is achieved with βi i θ := min{ α i | αq > 0} q We say the p-th basic variable is blocking when θ = p Then αq is the pivot βp .Improving a Non-Optimal Solution (3) How can we satisfy sign constraints of basic variables? i Let i ∈ {1. . m}. αp q Session 3 – p...10/34 ..

km ) Session 3 – p. the p-th component of xB (t) is βp − θαq =0 We can make a basis change: xq enters the basis and xkp leaves.11/34 .. LP is unbounded. where B = (k1 . If θ < +∞ and the p-th basic variable is blocking: When setting xq = θ sign of basic variables is respected p In particular. 2. .Improving a Non-Optimal Solution (4) i > 0): 1.. If θ = +∞ (there is no i such that 1 ≤ i ≤ m and αq Value of objective function can be decreased inﬁnitely..

..... β ¯i = βi − θαi if i = p New basic solution: β q ¯ = B + (aq − ak )eT New basis: B p p T + (c − c )eT = c New basic costs: c ¯T q kp p B B p) T = (0 . . 0 .. 0 . 1 . 0) where eT p m) New objective value: z ¯ = z + θdq Session 3 – p.Update ¯ = B − {kp } ∪ {q } New basic indices: B ¯p = θ.12/34 ..

Update: ¯ = B − {kp } ∪ {q } B ¯p = θ. β ¯i = βi − θαi if i = p β q ¯ = B + (aq − ak )eT B p p z ¯ = z + θdq Session 3 – p. z = cT Bβ −1 and d = c − π T a . αq If I = ∅ then return UNBOUNDED βp βi Else compute θ = mini∈I ( α and p such that θ = ) i αp q q 4. .Algorithmic Description 1.13/34 Go to 2. Pricing: Compute π = cT j j j B If for all j ∈ R. Initialization: Find an initial feasible basis B Compute B −1 . B 2. Compute αq = B −1 aq i > 0}. β = B −1 b. dj ≥ 0 then return OPTIMAL Else let q be such that dq < 0. Ratio test: Compute I = {i | 1 ≤ i ≤ m. 3.

Updating Matrix Inverse (1) Actually what we really care about is B −1 .14/34 .g. cubic with Gaussian elimination!) ¯ −1 using B −1 There is efﬁcient way of computing B Session 3 – p. not B −1 and α = B −1 a B We need it for computing π = cT q q B Recomputing B −1 at each iteration is too expensive (e.

. . bm ). ep−1 . ep+1 . So a = v p bp + i = p v i bi −v1 vp ⇒ bp = .. bp−1 . a.. .. Want to compute Ba Note vp = 0 as otherwise rank(Ba ) < m. η. bm be the columns of B Let a be such that a = Bv = m i=1 vi bi Let p be such that 1 ≤ p ≤ m −1 Ba = (b1 .. .. .Updating Matrix Inverse (2) Let b1 .. . em ) −1 = EB −1 Then Ba E = B =⇒ Ba Session 3 – p..15/34 . −vp−1 vp . . . .. . . . bp+1 . E = (e1 . . . . . 1 vp 1 vp a+ −vp+1 vp i =p −vi vp −vm vp bi T Let η T = . . . ...

. matrix is recomputed (reinversion) Other ways of representing B −1 : LU factoritzation Session 3 – p.Updating Matrix Inverse (3) Application to the simplex algorithm: a = aq . unit matrix I ). v = αq . where xq is entering variable Thus updating inverse reuses already computed data! Assume initial basis is B0 (e.16/34 .g. where Ei is the E matrix of the i-th iteration Each E matrix can be stored compactly −1 Can represent B −1 as list Ek Ek−1 · · · E2 E1 plus B0 : Product Form of the Inverse (PFI) When list is long. Then at the k -th iteration of the simplex algorithm −1 the inverse matrix is B −1 = Ek Ek−1 · · · E2 E1 B0 .

Bounded Variables We want to solve with revised simplex LP’s of the form min cT x Ax = b ℓ≤x≤u (−∞ ≤ ℓk ≤ uk ≤ +∞) In practice internally variables translated so that ℓ = −∞ or ℓ = 0 to save arithmetic operations Variable xk is lower bounded if ℓk > −∞ Variable xk is upper bounded if uk < +∞ Variable xk is range if ℓk > −∞ and uk < +∞ Variable xk is ﬁxed if range and ℓk = uk Variable xk is free if ℓk = −∞ and uk = +∞ Session 3 – p.17/34 .

Basic Solution For any basis B . xj ∈ L) If upper bounded: uk (it is at upper bound.18/34 . xj ∈ Z ) If free: Basic solution: β := B −1 b − j ∈L B −1 aj ℓj − j ∈U B −1 aj uj Session 3 – p. xj ∈ U ) 0 (it is at zero level. the values of basic variables can be expressed in terms of non-basic variables: xB = B −1 b − B −1 RxR Values a non-basic variable xj can be assigned to: If lower bounded: ℓk (it is at lower bound.

19/34 .Optimality Conditions Cost function in terms of non-basic variables: −1 b + d x B cT x = cT R R B −1 a for all variable x B where dj = cj − cT j j B If xj ∈ L: cannot improve if dj ≥ 0 If xj ∈ U : cannot improve if dj ≤ 0 If xj ∈ Z : cannot improve if dj = 0 Optimality condition: no improving non-basic variable Session 3 – p.

20/34 .Objective Function Cost function in terms of non-basic variables: −1 b + d x B cT x = cT R R B −1 a for all variable x B where dj = cj − cT j j B Value z of cost function at current basic solution: −1 B b+ z = cT B j ∈L dj ℓ j + j ∈U dj uj Session 3 – p.

21/34 . This can happen when xq is lower bounded and xq ∈ L. or xq is free (so xq ∈ Z ) Since increasing xq can improve objective value: dq < 0 Let t ≥ 0 be difference of new value xq wrt old value x B ( t) = B −1 b − B −1 RxR = B −1 b − tB −1 aq − = β − tαq −1 a ℓ − B j j j ∈L −1 a u B j j j ∈U where β = B −1 b and αq = B −1 aq Session 3 – p.Improving a Non-Optimal Solution (1) Let xq be non-basic variable that can improve objective value by increasing its value.

22/34 .Improving a Non-Optimal Solution (2) How does the objective value change as a function of t? z (t) = cT x(t) = T cT B xB (t) + cR xR (t) = cT B xB (t) + tcq + j ∈L −1 B b+ cT B −1 b+ cT BB j ∈L c j ℓj + j ∈U cj uj = −1 −1 (c j − c T aj )uj + tcq − tcT aq = BB BB j ∈U −1 (c j − c T aj ) ℓ j + BB j ∈L dj ℓj + j ∈U −1 dj uj + tcq − tcT aq = BB −1 z + tcq − tcT B aq = B z + tdq Hence the improvement in cost is ∆z = z (t) − z = tdq Session 3 – p.

Improving a Non-Optimal Solution (3) i Let i ∈ {1. min{ βi −µi | αi < 0}) | α θ := min(uq − ℓq .. min{ βiα i q q αi q q If θ = +∞ we have unboundedness Else if θ = uq − ℓq we have a bound ﬂip: no pivot needed! Session 3 – p.. m}. µi := uki i ≤µ We need λi ≤ βi − tαq i i > 0: β − tαi ≥ λ If αq i i q i < 0: β − tαi ≤ µ If αq i i q ⇒ ⇒ β i −λi αi q β i − µi αi q ≥t ≥t But we need xq (t) ≤ uq too! Best improvement achieved with −λi i > 0}. . Let λi := ℓki . Then i-th component of xB (t) is βi − tαq Let xki be the i-th basic variable..23/34 .

x leaves basis at upper bound If θ = min{ βiα | α i kp q q Session 3 – p.24/34 .Improving a Non-Optimal Solution (4) −λi i > 0}. Thus variable xq enters basis and variable xkp leaves −λi i > 0}. min{ βiα | α i q q αi q q Assume θ = +∞. min{ βi −µi | αi < 0}) θ := min(uq − ℓq . θ = uq − ℓq . x leaves basis at lower bound If θ = min{ βiα | α i kp q q − µi i < 0}.

or xq is free (so xq ∈ Z ) Since decreasing xq can improve objective value: dq > 0 Let t ≤ 0 be difference of new value xq wrt old value Again xB (t) = β − tαq . This can happen when xq is upper bounded and xq ∈ U .25/34 .Improving a Non-Optimal Solution (5) Let xq be non-basic variable that can improve objective value by decreasing its value. where β = B −1 b and αq = B −1 aq Again the improvement in cost is ∆z = z (t) − z = tdq Session 3 – p.

µi := uki i ≤µ We need λi ≤ βi − tαq i i > 0: β − tαi ≤ µ If αq i i q i < 0: β − tαi ≥ λ If αq i i q ⇒ ⇒ β i − µi αi q β i −λi αi q ≤t ≤t But we need ℓq ≤ xq (t) too! Best improvement achieved with −λi i < 0}. max{ βi −µi | αi > 0}) | α θ := max(ℓq − uq ... Then i-th component of xB (t) is βi − tαq Let xki be the i-th basic variable. . m}. max{ βiα i q q αi q q If θ = −∞ we have unboundedness Else if θ = ℓq − uq we have a bound ﬂip: no pivot needed! Session 3 – p.. Let λi := ℓki .26/34 .Improving a Non-Optimal Solution (6) i Let i ∈ {1.

max{ βi −µi | αi > 0}) θ := max(ℓq − uq .Improving a Non-Optimal Solution (7) −λi i < 0}. x leaves basis at lower bound If θ = max{ βiα | α i kp q q − µi i > 0}. x leaves basis at upper bound If θ = max{ βiα | α i kp q q Session 3 – p.27/34 . θ = ℓq − uq . Thus variable xq enters basis and variable xkp leaves −λi i < 0}. max{ βiα | α i q q αi q q Assume θ = −∞.

e. β ¯i = βi − θαi if i = p New basic solution: β q ¯ −1 = EB −1 New basis inverse: B If entering variable comes from lower bound ˆ = L − {xq } else U ˆ = U − {xq } L If leaving variable leaves to lower bound ¯=L ˆ ∪ {xk } else U ¯=U ˆ ∪ {xk } L p p Session 3 – p. x ¯ q ∈ L ⇔ xq ∈ U ) ¯ = β − θαq New basic solution: β Else ¯ = B − {kp } ∪ {q } New basic indices: B ¯p = xq + θ..28/34 .Update New objective value: z ¯ = z + θdq If bound ﬂip Flip status of xq (i.

of non-basic variables xk for which dk . tableau tends to get ﬁlled  Revised: sparsity of A can be exploited for storage. Revised Simplex Time is saved: # Tableau: all dk . αk are computed can be adjusted Space is saved: # Tableau: even if A sparse.29/34 . all αk are computed  Revised: no. and pivots can be chosen to represent B −1 compactly Better numerical behaviour: # Tableau: errors due to ﬂoating-point arithmetic accumulate at each pivoting step  Revised: reinversion (PFI representation of B −1 ) or refactorization (LU representation of B −1 ) for resetting Session 3 – p.Tableau vs.

30/34 .Original vs. Bounds Simplex Time is saved: # Original: no special treatment of bounds  Bounds: bound ﬂips are much cheaper than pivoting steps in simplex iterations Space is saved: # Original: each bound constraint becomes a row  Bounds: bounds are stored cheaply in arrays Session 3 – p.

31/34 . estimations): small changes mean completely different solutions? Sensitivity analysis = study of the stability of optimal solution with respect to small changes in problem data Let B be a basis for min cT x Ax = b x≥0 B gives an optimal feasible solution iff Feasibility: B −1 b ≥ 0 T − cT B − 1 R ≥ 0 = c Optimality: dT B R R Session 3 – p.Sensitivity Analysis Data may be inaccurate (observations.

for any j ∈ R T ) B −1 a = d − ∆ e T B −1 a = d − ∆ α p + ∆ e 0 ≤ cj − ( c T j j j j k j k p k p B p ⇐⇒ ∆k αj ≤ dj ⇐⇒ ∆k ≤ j dj αp j p if αj > 0. ∆k ≥ j dj αp j p if αj <0 dj dj p p Thus max{ α p | α < 0} ≤ ∆k ≤ min{ p | α > 0} j j α Session 3 – p.32/34 . only optimality for xk is affected: −1 a ⇐⇒ ∆ ≥ −d B 0 ≤ ck + ∆ k − c T k k k B If k ∈ B and xk is p-th basic variable.Changes in Cost Function Assume ck changes to ck + ∆k Feasibility is not affected If k ∈ R.

changes to bp + ∆p Optimality is not affected Let γp be the p-th column of B −1 and β = B −1 b. p βi i ∆p ≤ − γ i if γp < 0 p βi βi i i Thus max{− γ i | γp > 0} ≤ ∆p ≤ min{− γ i | γp < 0} p p Session 3 – p.Changes in Right-Hand Side (1) Assume bp . the p-th component of b. The basic solution is feasible iff 0 ≤ B −1 (b + ∆p ep ) = β + ∆p γp ⇐⇒ i for all 1 ≤ i ≤ m 0 ≤ βi + ∆p γp βi i ⇐⇒ ∆p ≥ − γ i if γp > 0.33/34 .

the i-th component of b. changes to bi + ∆i . In economics. −1 are simplex multipliers where π T = cT B B z ( b + ∆ i e i ) − z ( b) = ∆ i π T e i = ∆ i π i Can be used to determine price/beneﬁt of changing a resource. etc.Changes in Right-Hand Side (2) Assume bi . π are called shadow prices: marginal (by one unit) cost of strengthening a constraint Session 3 – p.34/34 . so that the optimal basis does not change How does this affect the objective function? Value of objective function at basic solution is z ( b ) = cT B − 1 b = π T b . limit. capacity.