You are on page 1of 37

# Convex Analysis And Optimization

Convex Analysis And Optimization
Chap.1 Basic Convexity Concepts Section1.4 Relative Interior, Closure, and Continuity

Naoki Ito

July 8, 2012

.

.

.

.

.

.

Convex Analysis And Optimization Basic Convexity Concepts Relative Interior, Closure, and Continuity

The proof of Proposition 1.4.1
Proposition 1.4.1 C : Nonempty convex set
a

(Line Segment Principle) x ∈ ri(C), x ∈ cl(C) ⇒ αx + (1 − α)¯ ∈ ri(C) for ∀α ∈ (0, 1] ¯ x

Proof) (i)The case in x ∈ C ¯ Let x = αx + (1 − α)¯ ˆ x (α ∈ (0, 1]) Since x ∈ ri(C), ∃δ > 0 s.t. B(x, δ) ∩ aﬀ(C) ⊂ C The each point of B(xα , αδ) ∩ aﬀ(C) is a convex combination of x and some point of B(x, δ) ∩ aﬀ(C) ¯ Therefore, by the covexity of C, B(xα , αδ) ∩ aﬀ(C) ⊂ C, implying that xα ∈ ri(C) . . . . .

.

Convex Analysis And Optimization Basic Convexity Concepts Relative Interior, Closure, and Continuity

The proof of Proposition 1.4.1(a)(cont)
(ii)The case in x ∈ C ¯/ ¯ Let xk ⊂ C −→ x

.

.

.

.

.

.

Convex Analysis And Optimization Basic Convexity Concepts Relative Interior, Closure, and Continuity

The proof of Proposition 1.4.1(a)(cont)
(ii)The case in x ∈ C ¯/ ¯ Let xk ⊂ C −→ x Let xk,α = αx + (1 − α)xk xk,α → xα

(α ∈ (0, 1])

.

.

.

.

.

.

Convex Analysis And Optimization Basic Convexity Concepts Relative Interior, Closure, and Continuity

The proof of Proposition 1.4.1(a)(cont)
(ii)The case in x ∈ C ¯/ ¯ Let xk ⊂ C −→ x Let xk,α = αx + (1 − α)xk xk,α → xα As (i), we see that ∃δ > 0 s.t.

(α ∈ (0, 1])

B(x, δ) ∩ aﬀ(C) ⊂ C for allk ≥ 0

and B(xk,α , αδ) ∩ aﬀ(C) ⊂ C

.

.

.

.

.

.

Convex Analysis And Optimization Basic Convexity Concepts Relative Interior, Closure, and Continuity

The proof of Proposition 1.4.1(a)(cont)
(ii)The case in x ∈ C ¯/ ¯ Let xk ⊂ C −→ x Let xk,α = αx + (1 − α)xk xk,α → xα As (i), we see that ∃δ > 0 s.t.

(α ∈ (0, 1])

B(x, δ) ∩ aﬀ(C) ⊂ C for allk ≥ 0

and B(xk,α , αδ) ∩ aﬀ(C) ⊂ C Since xk,α → xα ∃N ≥ 0, ∀k ≥ N αδ 2
.

s.t. B xα ,

⊂ B(xk,α , αδ)

It follows B(xα , αδ ) ∩ aﬀ(C) ⊂ C 2

.

.

.

.

.

Convex Analysis And Optimization Basic Convexity Concepts Relative Interior, Closure, and Continuity

Proposition 1.4.1(b)
Proposition 1.4.1 C : Nonempty convex set
b

(Nonemptiness of Relative interior) ri(C) is a nonempty convex set aﬀ(C)=aﬀ(ri(C)) Furthermore... dim(aﬀ(C)):= m m>0 ⇒ ∃x0 , x1 , ..., xm ∈ ri(C) = {subspace parallel to aﬀ(C)}
. . . . . .

s.t. span{x1 − x0 , ..., xm − x0 }

Convex Analysis And Optimization Basic Convexity Concepts Relative Interior, Closure, and Continuity

The proof of proposition 1.4.1(b)

Proof) (Convexity of ri(C)) Convexity of ri(C) follows from the Line Segment Principle. ri(C) ⊂ C ⊂ cl(C)

.

.

.

.

.

.

Convex Analysis And Optimization Basic Convexity Concepts Relative Interior, Closure, and Continuity

The proof of proposition 1.4.1(b)(cont)

( Nonemptiness of ri(C) and aﬀ(C)=aﬀ(ri(C)) ) We assume without loss of generality that 0 ∈ C dim(aﬀ(C)) := m (i)The case in m = 0

.

.

.

.

.

.

Convex Analysis And Optimization Basic Convexity Concepts Relative Interior, Closure, and Continuity

The proof of proposition 1.4.1(b)(cont)

( Nonemptiness of ri(C) and aﬀ(C)=aﬀ(ri(C)) ) We assume without loss of generality that 0 ∈ C dim(aﬀ(C)) := m (i)The case in m = 0 C = φ → C = {0}

.

.

.

.

.

.

Convex Analysis And Optimization Basic Convexity Concepts Relative Interior, Closure, and Continuity

The proof of proposition 1.4.1(b)(cont)

( Nonemptiness of ri(C) and aﬀ(C)=aﬀ(ri(C)) ) We assume without loss of generality that 0 ∈ C dim(aﬀ(C)) := m (i)The case in m = 0 C = φ → C = {0} aﬀ(C) = {0}

.

.

.

.

.

.

Convex Analysis And Optimization Basic Convexity Concepts Relative Interior, Closure, and Continuity

The proof of proposition 1.4.1(b)(cont)

( Nonemptiness of ri(C) and aﬀ(C)=aﬀ(ri(C)) ) We assume without loss of generality that 0 ∈ C dim(aﬀ(C)) := m (i)The case in m = 0 C = φ → C = {0} aﬀ(C) = {0} ri(C) = {0}, thus, ri(C) = φ

.

.

.

.

.

.

Convex Analysis And Optimization Basic Convexity Concepts Relative Interior, Closure, and Continuity

The proof of proposition 1.4.1(b)(cont)

( Nonemptiness of ri(C) and aﬀ(C)=aﬀ(ri(C)) ) We assume without loss of generality that 0 ∈ C dim(aﬀ(C)) := m (i)The case in m = 0 C = φ → C = {0} aﬀ(C) = {0} ri(C) = {0}, thus, ri(C) = φ thus, aﬀ(C)=aﬀ(ri(C)) aﬀ(ri(C)) = {0}

.

.

.

.

.

.

Convex Analysis And Optimization Basic Convexity Concepts Relative Interior, Closure, and Continuity

The proof of proposition 1.4.1(b)(cont)
(ii)The case in m > 0 We can take m linearly independent vectors z1 , ..., zm ∈ C that span aﬀ(C)

.

.

.

.

.

.

Convex Analysis And Optimization Basic Convexity Concepts Relative Interior, Closure, and Continuity

The proof of proposition 1.4.1(b)(cont)
(ii)The case in m > 0 We can take m linearly independent vectors z1 , ..., zm ∈ C that span aﬀ(C) Consider the set X = {x|x = m αi zi , i=1 ( X is open. X ∈ C)
m i=1 αi

< 1, αi > 0 (i = 1, ..., m)}

.

.

.

.

.

.

Convex Analysis And Optimization Basic Convexity Concepts Relative Interior, Closure, and Continuity

The proof of proposition 1.4.1(b)(cont)
(ii)The case in m > 0 We can take m linearly independent vectors z1 , ..., zm ∈ C that span aﬀ(C) Consider the set X = {x|x = m αi zi , i=1 ( X is open. X ∈ C) We will show X is open relative to aﬀ(C) ∀¯ ∈ X, ∃δ > 0 x s.t. B(¯, δ) ∩ aﬀ(C) ⊂ X x
m i=1 αi

< 1, αi > 0 (i = 1, ..., m)}

.

.

.

.

.

.

Convex Analysis And Optimization Basic Convexity Concepts Relative Interior, Closure, and Continuity

The proof of proposition 1.4.1(b)(cont)
Fix x ∈ X and let x ∈aﬀ(C) ¯ We have unique m-dimensional vectors α, α ¯ such that x = Z α, x = Zα (where Z := [z1 , ..., zm ]) ¯ ¯

.

.

.

.

.

.

Convex Analysis And Optimization Basic Convexity Concepts Relative Interior, Closure, and Continuity

The proof of proposition 1.4.1(b)(cont)
Fix x ∈ X and let x ∈aﬀ(C) ¯ We have unique m-dimensional vectors α, α ¯ such that x = Z α, x = Zα (where Z := [z1 , ..., zm ]) ¯ ¯ Since Z Z is symmetric and positive deﬁnite, we have for some γ > 0, which is independent of x and x, ¯ x−x ¯
2

= (α − α) Z Z(α − α) ≤ γ α − α ¯ ¯ ¯

2

.

.

.

.

.

.

Convex Analysis And Optimization Basic Convexity Concepts Relative Interior, Closure, and Continuity

The proof of proposition 1.4.1(b)(cont)
Fix x ∈ X and let x ∈aﬀ(C) ¯ We have unique m-dimensional vectors α, α ¯ such that x = Z α, x = Zα (where Z := [z1 , ..., zm ]) ¯ ¯ Since Z Z is symmetric and positive deﬁnite, we have for some γ > 0, which is independent of x and x, ¯ x−x ¯
2

= (α − α) Z Z(α − α) ≤ γ α − α ¯ ¯ ¯

2

Since x ∈ X, α lies in the open set ¯ ¯
m

A=

(α1 , ..., αm )|
i=1

αi < 1, αi > 0 (i = 1, ..., m)

.

.

.

.

.

.

Convex Analysis And Optimization Basic Convexity Concepts Relative Interior, Closure, and Continuity

The proof of proposition 1.4.1(b)(cont)
Take the δ such that the α where γ α − α ¯ A. (i.e. x = Zα belongs to X)
2

< δ belongs to

.

.

.

.

.

.

Convex Analysis And Optimization Basic Convexity Concepts Relative Interior, Closure, and Continuity

The proof of proposition 1.4.1(b)(cont)
Take the δ such that the α where γ α − α ¯ A. (i.e. x = Zα belongs to X)
2

< δ belongs to

Such the δ can be constructed as below. Since A is open set, ∃ > 0 s.t. B(α, ) ⊂ A ¯ δ := γ 2 then, γ α − α 2 < δ = γ 2 ⇒ α − α ¯ ¯

2

<

2

.

.

.

.

.

.

Convex Analysis And Optimization Basic Convexity Concepts Relative Interior, Closure, and Continuity

The proof of proposition 1.4.1(b)(cont)
Take the δ such that the α where γ α − α ¯ A. (i.e. x = Zα belongs to X)
2

< δ belongs to

Such the δ can be constructed as below. Since A is open set, ∃ > 0 s.t. B(α, ) ⊂ A ¯ δ := γ 2 then, γ α − α 2 < δ = γ 2 ⇒ α − α ¯ ¯

2

<

2

...implying that x ∈ X

.

.

.

.

.

.

Convex Analysis And Optimization Basic Convexity Concepts Relative Interior, Closure, and Continuity

The proof of proposition 1.4.1(b)(cont)
Take the δ such that the α where γ α − α ¯ A. (i.e. x = Zα belongs to X)
2

< δ belongs to

Such the δ can be constructed as below. Since A is open set, ∃ > 0 s.t. B(α, ) ⊂ A ¯ δ := γ 2 then, γ α − α 2 < δ = γ 2 ⇒ α − α ¯ ¯

2

<

2

...implying that x ∈ X Hence X contains B(¯)∩aﬀ(C), so X is open relative to x aﬀ(C).

.

.

.

.

.

.

Convex Analysis And Optimization Basic Convexity Concepts Relative Interior, Closure, and Continuity

The proof of proposition 1.4.1(b)(cont)
Take the δ such that the α where γ α − α ¯ A. (i.e. x = Zα belongs to X)
2

< δ belongs to

Such the δ can be constructed as below. Since A is open set, ∃ > 0 s.t. B(α, ) ⊂ A ¯ δ := γ 2 then, γ α − α 2 < δ = γ 2 ⇒ α − α ¯ ¯

2

<

2

...implying that x ∈ X Hence X contains B(¯)∩aﬀ(C), so X is open relative to x aﬀ(C). It follows that ∀¯ ∈ X, x ∈ ri(C), so that ri(C) = φ. x ¯

.

.

.

.

.

.

Convex Analysis And Optimization Basic Convexity Concepts Relative Interior, Closure, and Continuity

The proof of proposition 1.4.1(b)(cont)
Take the δ such that the α where γ α − α ¯ A. (i.e. x = Zα belongs to X)
2

< δ belongs to

Such the δ can be constructed as below. Since A is open set, ∃ > 0 s.t. B(α, ) ⊂ A ¯ δ := γ 2 then, γ α − α 2 < δ = γ 2 ⇒ α − α ¯ ¯

2

<

2

...implying that x ∈ X Hence X contains B(¯)∩aﬀ(C), so X is open relative to x aﬀ(C). It follows that ∀¯ ∈ X, x ∈ ri(C), so that ri(C) = φ. x ¯ Also, aﬀ(X) = aﬀ(C) (since by construction), X ⊂ri(C)(⊂ C), we see that aﬀ(ri(C))=ri(C).
. . . . . .

Convex Analysis And Optimization Basic Convexity Concepts Relative Interior, Closure, and Continuity

Proposition 1.4.1(b) (written again)

m>0

∃x0 , x1 , ..., xm ∈ ri(C) = {subspace parallel to aﬀ(C)}

s.t. span{x1 − x0 , ..., xm − x0 }

Proof) consider vectors
m

x0 = α
i=1

zi ,

xi = x0 + αzi ,

i = 1, ..., m

where α is positive scalar that α(m + 1) < 1

.

.

.

.

.

.

Convex Analysis And Optimization Basic Convexity Concepts Relative Interior, Closure, and Continuity

Proposition 1.4.1(b) (written again)

m>0

∃x0 , x1 , ..., xm ∈ ri(C) = {subspace parallel to aﬀ(C)}

s.t. span{x1 − x0 , ..., xm − x0 }

Proof) consider vectors
m

x0 = α
i=1

zi ,

xi = x0 + αzi ,

i = 1, ..., m

where α is positive scalar that α(m + 1) < 1 xi ∈ X ⊂ ri(C) (i = 0, 1, ..., m)
. . . . . .

Convex Analysis And Optimization Basic Convexity Concepts Relative Interior, Closure, and Continuity

Proposition 1.4.1(b) (written again)

m>0

∃x0 , x1 , ..., xm ∈ ri(C) = {subspace parallel to aﬀ(C)}

s.t. span{x1 − x0 , ..., xm − x0 }

Proof) consider vectors
m

x0 = α
i=1

zi ,

xi = x0 + αzi ,

i = 1, ..., m

where α is positive scalar that α(m + 1) < 1 xi ∈ X ⊂ ri(C) (i = 0, 1, ..., m) xi − x0 = αzi (i = 0, 1, ..., m) span aﬀ(C). . .

.

.

.

.

Convex Analysis And Optimization Basic Convexity Concepts Relative Interior, Closure, and Continuity

Proposition 1.4.1 (c)
Proposition 1.4.1
c

x ∈ ri(C) ⇔ ∀¯ ∈ C, ∃γ > 1 x

s.t. x + (γ − 1)(x − x) ∈ C ¯

Proof) ⇒) (trivial) ⇐) Since ri(C) = φ, there exists some x ∈ ri(C) ˆ

.

.

.

.

.

.

Convex Analysis And Optimization Basic Convexity Concepts Relative Interior, Closure, and Continuity

Proposition 1.4.1 (c)
Proposition 1.4.1
c

x ∈ ri(C) ⇔ ∀¯ ∈ C, ∃γ > 1 x

s.t. x + (γ − 1)(x − x) ∈ C ¯

Proof) ⇒) (trivial) ⇐) Since ri(C) = φ, there exists some x ∈ ri(C) ˆ If x = x, we are done. So, we assume x = x. ˆ ˆ

.

.

.

.

.

.

Convex Analysis And Optimization Basic Convexity Concepts Relative Interior, Closure, and Continuity

Proposition 1.4.1 (c)
Proposition 1.4.1
c

x ∈ ri(C) ⇔ ∀¯ ∈ C, ∃γ > 1 x

s.t. x + (γ − 1)(x − x) ∈ C ¯

Proof) ⇒) (trivial) ⇐) Since ri(C) = φ, there exists some x ∈ ri(C) ˆ If x = x, we are done. So, we assume x = x. ˆ ˆ By the given condition, since x ∈ C, ˆ there exists a γ > 1 s.t. y = x + (γ − 1)(x − x) ∈ C ˆ

.

.

.

.

.

.

Convex Analysis And Optimization Basic Convexity Concepts Relative Interior, Closure, and Continuity

Proposition 1.4.1 (c)
Proposition 1.4.1
c

x ∈ ri(C) ⇔ ∀¯ ∈ C, ∃γ > 1 x

s.t. x + (γ − 1)(x − x) ∈ C ¯

Proof) ⇒) (trivial) ⇐) Since ri(C) = φ, there exists some x ∈ ri(C) ˆ If x = x, we are done. So, we assume x = x. ˆ ˆ By the given condition, since x ∈ C, ˆ there exists a γ > 1 s.t. y = x + (γ − 1)(x − x) ∈ C ˆ
1 Thus, we have x = γ−1 x + γ y γ ˆ By using Line Segment Principle, we obtain x ∈ ri(C)
. . . . . .

Convex Analysis And Optimization Basic Convexity Concepts Relative Interior, Closure, and Continuity

Proposition 1.4.2 X : Nonempty convex subset of Rn f : X → R , concave X ∗ := {x∗ ∈ X|f (x∗ ) = inf x∈X f (x)} X ∗ ∩ ri(X) = φ ⇒ f must be constant over X, i.e., X ∗ = X

.

.

.

.

.

.

Convex Analysis And Optimization Basic Convexity Concepts Relative Interior, Closure, and Continuity

Proposition 1.4.3 ¯ C, C: Nonempty convex sets.
a b c

cl(C) = cl(ri(C)) ri(C) = ri(cl(C)) Following three conditions are equivalent.
i ii iii

¯ ri(C) = ri(C) ¯ cl(C) = cl(C) ¯ ⊂cl(C) ri(C) ⊂ C

.

.

.

.

.

.

Convex Analysis And Optimization Basic Convexity Concepts Relative Interior, Closure, and Continuity

Proposition 1.4.4 C : Nonempty convex subset of Rn A : m × n matrix
a b

A · ri(C) = ri(A · C) A · cl(C) ⊂ cl(A · C) Furthermore, if C is bounded A · cl(C) = cl(A · C)

.

.

.

.

.

.

Convex Analysis And Optimization Basic Convexity Concepts Relative Interior, Closure, and Continuity

Proposition 1.4.5 C1 , C2 : Nonempty convex sets
a

ri(C1 ) ∩ ri(C2 ) ⊂ ri(C1 ∩ C2 ), cl(C1 ∩ C2 ) ⊂ cl(C1 ) ∩ ri(C2 ) Furthermore, if ri(C1 ) ∩ ri(C2 ) = φ, then ri(C1 ) ∩ ri(C2 ) = ri(C1 ∩ C2 ), cl(C1 ∩ C2 ) = cl(C1 ) ∩ ri(C2 ) ri(C1 + C2 ) ⊂ ri(C1 ) + ri(C2 ), cl(C1 ) + cl(C2 ) ⊂ cl(C1 + C2 ) Furthermore, if at least one of the sets C1 and C2 is bounded, then cl(C1 ) + cl(C2 ) = cl(C1 + C2 )

b

.

.

.

.

.

.

Convex Analysis And Optimization Basic Convexity Concepts Relative Interior, Closure, and Continuity

Proposition 1.4.6 f : Rn → R is convex f : Rn ⇒ f is continuous.

→ (−∞, ∞] is a proper convex function ⇒ f is continuous over the ri(dom(f )).

.

.

.

.

.

.