SC 607: Optimization Spring 2020

Lecture 15: March 13, 2020

Instructor: Ankur A. Kulkarni Scribes: Harsha Priyanka Guntaka, Komal Agnihotri

Note: LaTeX template courtesy of UC Berkeley EECS dept.

Disclaimer: These notes have not been subjected to the usual scrutiny reserved for formal publications.
They may be distributed outside this class only with the permission of the Instructor.

Recap

Dual
Primal
min cT x
max cT x λ
x
AT λ ≥ c
Ax ≤ b
λ≥0
x≥0

where A ∈ Rmxn , c ∈ Rn , b ∈ Rm , λ ∈ Rm .

1. Let ΩP = {x|Ax ≤ b, x ≥ 0} be the set of all points satisfying the constraints of the Primal problem.
Similarly, let ΩD = {λ|AT λ ≥ c, λ ≥ 0} be the set of all points satisfying the constraints of the Dual
problem.
2. Weak Duality: Let xinΩP , λ ∈ ΩD . Then cT x ≤ (λ)T Ax ≤ (λ)T b
3. If x ∈ ΩP , λ ∈ ΩD such that cT x∗ = (λ∗ )T b then x∗ is optimal for Primal and λ∗ is optimal for Dual.
4. Strong Duality: Suppose ΩP 6= φ and ΩD 6= φ. Then there exists x∗ which is optimal for Primal
and λ∗ which is optimal for Dual. Furthermore, cT x∗ = (λ∗ )T b.
5. Suppose ΩP 6= φ then there exists x∗ which is optimal for Primal if and only if ΩD 6= φ and vice-versa.
6. Every constraint in the primal linear programming(LP) has a corresponding variable in the dual and
vice-versa.

15.1 Complementary Slackness

Theorem 15.1. x∗ ∈ ΩP is optimal if and only if there exists λ∗ ∈ ΩD such that

m
X
aij x∗j < bi implies λ∗i = 0,
j=1

and (15.1)
m
X
λ∗i aij < cj implies x∗j = 0.
i=1

15-1
15-2 Lecture 15: March 13, 2020

Proof. For x∗ ∈ ΩP and λ∗ ∈ ΩD , the complementary slackness conditions(15.1) are equivalent to

(λ∗ )T (Ax ∗ −b) = 0, and (AT λ∗ − c)T x∗ = 0 (15.2)

Necessity: Suppose x∗ ∈ ΩP is optimal for the primal linear programming(LP), then that implies ΩD 6= φ,
so that by strong duality condition ∃λ∗ ∈ ΩD such that cT x∗ = (λ∗ )T b. By weak duality condition we have
cT x∗ ≤ (λ∗ )T Ax∗ ≤ (λ∗ )T b so that we must have cT x∗ = (λ∗ )T Ax∗ = (λ∗ )T b. (15.2) is just an equivalent
rearrangement of these two equalities.
Sufficiency: Suppose (15.2) holds for some x∗ ∈ ΩP and λ∗ ∈ ΩD . The first equality in (15.2) yields
(λ∗ )T b = (λ∗ )T Ax∗ = (AT λ∗ )T x∗ while second yields (AT λ∗ )T x∗ = cT x∗ so that cT x∗ = (λ∗ )T b. This
implies x∗ is optimal.

15.2 Tangent Cone :

For x ∈ S and S ⊆ Rn the tangent cone T(x;S) is defined as

T (x? ; S) = {d|∃xk ∈ S, xk → x? andτk ↓ 0}such that d = lim xk − x? /τk (15.3)

k→∞

Fig :
Tangent cone of different sets

Theorem 15.2. If x? is local minima of minx f (x) such that x ∈ S then :

5f (x? )T .d ≥ 0 ∀ d ∈ T (x? : S) (15.4)

Proof. Suppose this is not true i.e. ∃d ∈ T (x? : S) such that

5f (x? )T .d < 0 (15.5)

since d ∈ T (x? : S)∃xk → x? , x ∈ S and τk ↓ 0 such that

τk .d =k −x? + o(τn ) (15.6)

Using of Taylor’s series approximation, we get

Lecture 15: March 13, 2020 15-3

f (xk ) = f (x? )+5f (x? )T .(xk −x? )+o(||xk − x? ||)f (xk ) = f (x? )+5f (x? )T .d.τk +o(τk )f (xk ) = f (x? )+τk (5f (x? )T ).d+o(τ
(15.7)
For large k ⇒f(xk ) < f (x? )
So it can not be x? a local minima. Hence it is a contradiction.

References

[1] Variya, Pravin (1998). “Lecture Notes on Optimization”. https://people.eecs.berkeley.edu/ varaiya/Download/Varaiya

Optimization.pdf. Berkeley, California