LP Duality & Sensitivity

Mohamed Kobeissi
American University of Beirut

8 mars 2024

Mohamed Kobeissi LP Duality & Sensitivity

Duality

The concept of duality is best motivated by an example.

Consider the following transportation problem. Some good is
available at location A at no cost and may be transported to
locations B, C , and D according to the following directed
graph :
4 B
1
4
!
A 3
=
D
5
2 
*
C

Mohamed Kobeissi LP Duality & Sensitivity

On each of the unidirectional channels, the unit cost of
transportation is cj for j = 1, . . . , 5. At each of the vertices bα
units of the good are sold, where α = B, C , D. How can the
transport be done most efficiently ?

Mohamed Kobeissi LP Duality & Sensitivity

A first, and arguably most obvious way of quantifying
efficiency would be to state the question as a minimization
problem for the total cost of transportation. If xj denotes the
amount of good transported through channel j, we arrive at
the following linear programming program :

minimize c1 x1 + · · · + c5 x5 (1a)

subject to

x1 − x3 − x4 = bB (1b)
x2 + x3 − x5 = bC (1c)
x4 + x5 = bD (1d)

The three equality constraints state that nothing gets lost at

nodes B, C , and D except what is sold

Mohamed Kobeissi LP Duality & Sensitivity

There is, however, a second, seemingly equivalent way of
characterizing efficiency of transportation. Instead of looking
at minimizing the cost of transportation, we seek to maximize
the income from selling the good. Letting yα denote the unit
price of the good at node α = A, . . . , D with yA = 0 by
assumption, the associated linear programming problem is the
following :
maximize yB bB + yC bC + yD bD (2a)

subject to

yB − yA ≤ c1 (2b)
yC − yA ≤ c2 (2c)
yC − yB ≤ c3 (2d)
yD − yB ≤ c4 (2e)
yD − yC ≤ c5 (2f)
Mohamed Kobeissi LP Duality & Sensitivity
The inequality constraints encode that, in a free market, we
can only maintain a price difference that does not exceed the
cost of transportation. If we charged a higher price, then
“some local guy” would immediately be able to undercut our
price by buying from us at one end of the channel, using the
channel at the same fixed channel cost, then selling at a price
lower than ours at the high-price end of the channel

Mohamed Kobeissi LP Duality & Sensitivity

Setting
 
x1
   
yB 1 0 −1 −1 0
x = .
 ..  , y = yC  , and A = 0 1 1 0 −1
x5 yD 0 0 0 1 1

Mohamed Kobeissi LP Duality & Sensitivity

Primal

we can write program (1) as the abstract primal problem

maximize c T x (3a)
subject to Ax ≤ b , x ≥ 0 (3b)

Mohamed Kobeissi LP Duality & Sensitivity

Dual

Likewise, program (2) can be written as the dual problem

minimize b T y (4a)
subject to A y ≥ c , y ≥ 0
T
(4b)

We shall prove in the following that the minimal cost and the
maximal income coincide, i.e., that the two problems are
equivalent

Mohamed Kobeissi LP Duality & Sensitivity

Introduction to sensitivity

The unknowns x in the primal formulation of the problem not

only identify the vertex of the feasible region at which the
optimum is reached, but they also act as sensitivity parameters
with regard to small changes in the cost coefficients c . Indeed,
when the linear programming problem is non-degenerate, i.e.
has a unique optimal solution, changing the cost coefficients
from c to c + ∆c with |∆c | sufficiently small will not make
the optimal vertex jump to another corner of the feasible
region, as the cost depends continuously on c

Mohamed Kobeissi LP Duality & Sensitivity

If xi is nonbasic, the cost will not react at all to small changes
in ci , whereas if xi is large, then the cost will be sensitive to
changes in ci . This information is often important because it
gives an indication where to best spend resources if the
parameters of the problem—in the example above, the cost of
transportation—are to be improved

Mohamed Kobeissi LP Duality & Sensitivity

Likewise, the solution vector y to the dual problem provides
the sensitivity of the total income to small changes in b . Here,
b is representing the number of sales at the various vertices of
the network ; if the channels were capacity constrained, the
channel limits were also represented as components of b .
Thus, the dual problem is providing the answer to the question
”if I were to invest in raising sales, where should I direct this
investment to achieve the maximum increase in income ?”

Mohamed Kobeissi LP Duality & Sensitivity

Weak duality

Theorem
Assume that x is a feasible vector for the primal problem, and
y is a feasible vector for the dual problem. Then
1 y T b ≤ cT x
2 if (1) holds with equality, then x and y are optimal for
their respective linear programming problems
3 the primal problem does not have a finite minimum if and
only if the feasible region of the dual problem is empty ;
vice versa, the dual problem does not have a finite
maximum if and only if the feasible region of the primal
problem is empty

Mohamed Kobeissi LP Duality & Sensitivity

Strong duality

Theorem
The primal problem has a non-degenerate solution x if and
only if the dual problem has a non-degenerate solution y ; in
this case y T b = c T x

Mohamed Kobeissi LP Duality & Sensitivity

Equivalently ...

P\D feasible not feasible

feasible both have solutions ; values equal P unbounded
not feasible D unbounded possible

This table shows that if you know that both problems are
feasible, then you know that neither problem is unbounded or,
equivalently, that both have solutions. More than that, the
values of the solutions are equal

Mohamed Kobeissi LP Duality & Sensitivity

Using the Duality Theorem

Consider the following pair of problems :

max x1 + x2
subject to −3x1 + 2x2 ≤ −1
x1 − x2 ≤ 2
x ≥ 0
The dual is :
min −y1 + 2y2
subject to −3y1 + y2 ≥ 1
2y1 − y2 ≥ 1
y ≥ 0

Mohamed Kobeissi LP Duality & Sensitivity

 Primal is feasible ((x1 , x2 ) = (1, 0) satisfies the
constraints)
Dual is infeasible.(Add the two resource constraints in the
dual together you get : −y1 ≥ 2. This inequality implies
that y1 ≤ −2, which is inconsistent with the
non-negativity constraint)
Conclude Primal is unbounded !

Mohamed Kobeissi LP Duality & Sensitivity

Dual example

To find the dual of the program below

max z = 5x1 + x2 / x1 − x2 ≤8
2x1 + x2 ≥6
x1 + x2 =4
4x1 + 2x2 ≥1
x1 ≥0
x2 ∈R

first put it in ”canonical” form

Mohamed Kobeissi LP Duality & Sensitivity

As follows
max z = 5x1 + x2+ − x2− / x1 − x2+ + x2− ≤8
−2x1 − x2+ + x2− ≤ −6
x1 + x2+ − x2− ≤4
−x1 − x2+ + x2− ≤ −4
−4x1 − 2x2+ + 2x2− ≤ −1
x1 ≥0
x2+ , x2− ≥0

Mohamed Kobeissi LP Duality & Sensitivity

And its dual is

min w = 8y1 − 6y2 + 4y3 − 4y4 − y5

subject to
y1 − 2y2 + y3 − y4 − 4y5 ≥ 5
−y1 − y2 + y3 − y4 − 2y5 ≥ 1
y1 + y2 − y3 + y4 + 2y5 ≥ −1
yi ≥ 0

Mohamed Kobeissi LP Duality & Sensitivity

By letting y3 − y4 = y6 , y2′ = −y2 , and y5′ = −y5 , and
observing that constraints 2 and 3 in the above program can
be written in one constarint, we obtain :

min w = 8y1 − 6y2 + 4y6 − y5

subject to
y1 + 2y2′ + y6 + 4y5′ ≥ 5
y1 − y2′ − y6 − 2y5′ = −1
y1 ≥ 0
y2′ , y5′ ≤ 0
y6 ∈ R

Mohamed Kobeissi LP Duality & Sensitivity

Primal ↔ Dual

The primal-dual transformation rules are given in the following

table

Mohamed Kobeissi LP Duality & Sensitivity

By using the rules of primal-dual transformation, we can write
the dual directly :

min w = 8y1 + 6y2 + 4y3 + y4

subject to
y1 + 2y2 + y3 + 4y5 ≥ 5
−y1 + y2 + y3 + 2y5 = 1
y1 ≥ 0
y2 ≤ 0
y3 ∈ R
y4 ≤ 0

Mohamed Kobeissi LP Duality & Sensitivity

The slackness theorem

Theorem
Assume problem (P) has a solution x ∗ and problem (D) has a
solution y ∗
1 If xj∗ > 0, then the jth constraint in (D) is binding
2 If the jth constraint in (D) is not binding, then xj∗ = 0
3 If yi∗ > 0, then the ith constraint in (P) is binding
4 If the ith constraint in (P) is not binding, then yi∗ = 0

Mohamed Kobeissi LP Duality & Sensitivity

Using Complementary Slackness to Solve Duals

Consider the following linear program :

max 2x1 + 4x2 + 3x3 + x4

subject to 3x1 + x2 + x3 + 4x4 ≤ 12
x1 − 3x2 + 2x3 + 3x4 ≤ 7
2x1 + x2 + 3x3 − x4 ≤ 10
x ≥ 0

The solution to this problem is

x1 = 0; x2 = 10.4; x3 = 0; x4 = 0.4

Mohamed Kobeissi LP Duality & Sensitivity

And its dual
min 12y1 + 7y2 + 10y3
subject to 3y1 + y2 + 2y3 ≥ 2
y1 − 3y2 + y3 ≥ 4
y1 + 2y2 + 3y3 ≥ 3
4y1 + 3y2 − y3 ≥ 1
y ≥ 0

Mohamed Kobeissi LP Duality & Sensitivity

Algebra

Step 1, check feasibility

First constraint : 0 + 10.4 + 0 + 1.6 = 12. Binding. Tells
you nothing about y1
Second constraint : 0 − 31.2 + 0 + 1.2 = −30. Not
binding (but satisfied). y2 = 0
Third constraint : 0 + 10.4 + 0 − .4 = 10. Binding. Tells
you nothing about y3
So x satisfies constraints

Mohamed Kobeissi LP Duality & Sensitivity

Step 2, implications of positive variables
1 x1 = 0 tells you nothing
2 x2 > 0 tells you second dual constraint binds at solution :

y1 − 3y2 + y3 = 4
3 x3 = 0 tells you nothing
4 x4 > 0 tells you fourth dual constraint binds :

4y1 + 3y2 − y3 = 1
So solution to dual must satisfy y2 = 0, y1 + y3 = 4, and
4y1 − y3 = 1
This means that candidate for solution to (D) is y = (1, 0, 3)

Mohamed Kobeissi LP Duality & Sensitivity

Is it a solution ?

Need to check constraints we ignored :

1 Non negativity
2 Dual Constraints 1 and 3
These constraints do hold. It is a solution. We can check they
have equal values

Mohamed Kobeissi LP Duality & Sensitivity

What Happened ?

1 Begin with a ”guess” for Primal

2 Check feasibility. If not, stop. It cannot be a solution
3 If so, note positive variables (implies binding dual
constraints) and constraints with slack (implies zero dual
values)
4 Use non-zero dual variables to solve the equations in dual
(Typically equal number of equations and unknowns)
5 Check feasibility of this solution (are variables
nonnegative ? Are the dual constraints corresponding to
zero primal variables satisfied ?) If yes, then you have
solutions to both Primal and Dual. If not, original guess is
not a solution

Mohamed Kobeissi LP Duality & Sensitivity

What if guess is not a solution ?

Take the LP :

max x1 − x2
subject to −2x1 + x2 ≤ 2
x1 − 2x2 ≤ 2
x1 + x2 ≤ 5
x ≥ 0
The dual is :
min 2y1 + 2y2 + 5y3
subject to −2y1 + y2 + y3 ≥ 1
y1 − 2y2 + y3 ≥ −1
y ≥ 0

Mohamed Kobeissi LP Duality & Sensitivity

Does (1, 4) solves Primal ?
(1, 4) is feasible
First constraint binds. This tells you nothing about dual
variables
Second constraint holds with slack. Hence y2 = 0 must
hold in solution to dual
(1, 4) satisfies both non-negativity constraints strictly
Hence both dual constraints bind
So dual ”reduces” to :

−2y1 + y3 = 1
y1 + y3 = −1
y2 = 0
Conclude : y1 = − 23 and y3 = − 13
Mohamed Kobeissi LP Duality & Sensitivity
Punchline

Assuming that (1, 4) solves the primal leads to the

conclusion that y = (− 23 , 0, − 13 ) solves the dual
But : this choice of y is not feasible for the dual (it
violates the non-negative constraints)
Hence (1, 4) did not solve the primal after all
Using the same kind of reasoning, you can check the
solution to the primal is (4, 1). The dual solution is
y = (0, 23 , 31 ). The value of both problems is 3

Mohamed Kobeissi LP Duality & Sensitivity

Generally

1 Start with a ”guess” for the primal

2 Check feasibility
3 If infeasible, then it cannot be a solution
4 If feasible, use CS conditions to reduce dual to a system
of equations
5 Solve these equations to get guesses for dual variables
6 Check feasibility of the dual guess.
7 If the dual guess is feasible, then both the original primal
guess and the dual guess actually solve their problems
8 If the dual guess is not feasible, then neither guess is a
solution

Mohamed Kobeissi LP Duality & Sensitivity