You are on page 1of 126

Convex Analysis

Dorel I. Duca
Contents

1. Introduction 1
Chapter 1. Convex Analysis 3
1. Notations and Preliminary Results 3
2. Linear Subspaces. A ne Sets 7
3. Convex Sets 10
4. Separation Theorems for Convex Sets 23
5. Cones 29
6. The Dual (Polar) of a Set 38
7. Convex Functions 50
8. Strictly Convex Functions 59
9. Pseudoconvex Functions 62
10. Quasiconvex Functions 65
Chapter 2. Optimality Conditions 71
1. Optimization Problems 71
2. Some General Results of Optimization 73
3. Necessary Optimality Criteria and Su cient Optimality
Criteria 79
4. Necessary Conditions for the Optimal Solutions of the
Optimization Problems with Dierentiable Hypothesis
about the Functions 91
5. Su cient Conditions for the Optimal Solutions of the
Optimization Problems with Dierentiable Hypothesis
about the Functions 111
Bibliography 119

iii
1 1

1. Introduction
The book ends with the list of references.
CHAPTER 1

Convex Analysis

1. Notations and Preliminary Results


We will denote by R the corp of real numbers, by Rn the n th
dimensional linear real space over R and by Rm n the linear space of
the real matrices of the type (m; n), over R.
An element (x1 ; :::; xn ) 2 Rn will be denoted by x or (xj ) and an
element
2 3
a11 ::: a1n
4 ::: ::: ::: 5 2 Rm n
am1 ::: amn
by A or [akj ] :
Since the spaces Rn and R1 n are isomorphic we will identify Rn
with the space R1 n .
If
2 3
a11 ::: a1n
4
A = ::: ::: ::: 5 2 Rm n
;
am1 ::: amn
then, for each x = (x1 ; :::; xn ) 2 Rn ; we will denote by A(x) the point
!
X n X
n
A(x) = a1j xj ; :::; amj xj 2 Rm :
j=1 j=1

The set fe1 ; :::; en g; where


e1 = (1; 0; 0; :::; 0) ; e2 = (0; 1; 0; :::; 0) ; :::; en = (0; 0; :::; 0; 1) 2 Rn

is a base of the linear space Rn : Then each point


z = (z1 ; z2 ; :::; zn ) 2 Cn

can be represented as follows


x = x1 e1 + x2 e2 + ::: + xn en :

For x = (x1 ; :::; xn ); v = (v1 ; :::; vn ) 2 Rn we will denote by


3
4 1. CONVEX ANALYSIS

hz; vi = z1 v1 + ::: + zn vn = v(z)

the inner product of x and v: In this way Rn becomes a Banach space


if we consider the hermitian norm
p
(1.1.1) kxk = hx; xi; for all x 2 Rn :

If x 2 Rn and r 2]0; +1[; then we denote by


B(x; r) = fv 2 Rn : kx vk < rg

the open ball with centre x and radius r and by


B[x; r] = fv 2 Rn : kx vk 5 rg

the closed ball with centre x and radius r:


Endowed with the topology induced by the norm (1.1.1), Rn be-
comes a linear topological space for which the set
fB(x; r) : r > 0g

represents a basis of neighborhoods for the point x 2 Rn :


If M is a subset of the space Rn , then we denote by
intM - the interior of M
and by
clM - the closure of M:
If x = (x1 ; :::; xn ); y = (y1 ; :::; yn ) 2 Rn , then we use the following
notations:

x 5 y if and only if xj 5 yj for each j 2 f1; :::; ng;

x < y if and only if xj < yj for each j 2 f1; :::; ng;

x 6 y if and only if x 5 y and x 6= y:

Moreover, we also use the notations:


Rn+ = fx 2 Rn : 0 5 xg
and
R+ = R1+ = [0; +1[:
1. NOTATIONS AND PRELIMINARY RESULTS 5

If a 2 Rn n f0g; then
H= (a) = fx 2 Rn : hx; ai = 0g;
H5 (a) = fx 2 Rn : hx; ai 5 0g;
H> (a) = fx 2 Rn : hx; ai > 0g;
H< (a) = fx 2 Rn : hx; ai < 0g
and if a1 ; :::; aq 2 Rn n f0g and
K = \fH= (ak ) : k 2 f1; :::; qgg;

then for each b 2 Rn we denote


K(b) = \fH= (ak ) : k 2 Eg;

where
E = fk 2 f1; :::; qg : hb; ak i = 0g:
Let us remark that
hA(x); vi = hz; AT (v)i;
for all A 2 Rm n
; x 2 Rn and v 2 Rm :

Denition 1.1.1 The matrix A 2 Rn n is said to be:


a) symmetric, if AT = A
b) antisymmetric (or skew-symmetric), if AT = A;
c) positive semidenite, if hA(x); xi = 0; for all x 2 Rn ;
d) positive denite if hA(x); xi > 0, for all x 2 Rn n f0g;
e) negativ (semi)denite, if the matrix ( A) is positive (semi)de-
nite.
If A 2 Rm n
; then we will denote by
R(A) = fy 2 Rm : there exists x 2 Rn such that y = A(x)g =
= A(Rn ) - the range of the matrix A;
and by
N (A) = fx 2 Rn : A(x) = 0g - the null space of the matrix A:
We dene now the concept of continuity for functions. From sev-
eral possible approaches, we have chosen one that uses convergent se-
quences.

Denition 1.1.2 Let f : M ! Rm be a function, where M is a


nonempty subset of Rn : The function f is said to be continuous at
a point x0 2 M if for each sequence (xk )k2N of points from M that
6 1. CONVEX ANALYSIS

converges to x0 ; the sequence (f (xk ))n2N of points from Rm converges


to f (x0 ):
If f is continuous at all the points of M , then f is said to be
continuous on M:
An important example of a continuous function is the norm function
k k : Rn ! R dened by
k k (x) = kxk ; for all x 2 Rn :
A particularly important example of a continuous function is the
distance function dM : Rn ! R of a nonempty subset M of Rn . The
function dM associates to each point x 2 Rn its distance dM (x) from
M: Formally, dM is dened as follows:
Denition 1.1.3 Let M be a nonempty subset of Rn : The function
dM : Rn ! R dened by
dM (x) = inffkx vk : v 2 M g; for all x 2 Rn ;
is called the distance function of the set M:
If M Rn is the singleton set fag; then
dM (x) = kx ak ; for all x 2 Rn :
In particular, if a = 0; then
dM (x) = kxk ; for all x 2 Rn :

Theorem 1.1.4 Let M be a nonempty subset of Rn : Then the distance


function dM : Rn ! R of the set M satises the Lipschitz condition:
jdM (x) dM (v)j 5 kx vk ; for all x; v 2 Rn :
Proof. Let x; v 2 Rn : From the denition of the distance function
dM ; it follows that, for each " > 0; there exists a point u in M such
that
kx uk < dM (x) + ":
By the triangle inequality,
dM (v) 5 kv uk 5 kv xk + kx uk < kv xk + dM (x) + ":
Since " > 0 is arbitrary,
dM (v) 5 kv xk + dM (x):
Interchanging x and v in the last inequality, we nd that
dM (x) 5 kx vk + dM (v);
2. LINEAR SUBSPACES. AFFINE SETS 7

whence dM satises the Lipschitz condition.


From this it follows the following statement.

Corollary 1.1.5 Let M be a nonempty subset of Rn : Then the distance


function dM : Rn ! R of the set M is continuous on Rn :
It follows that, a point x 2 Rn lies in the closure clM of M if and
only if its distance dM (x) from M is zero.

Remark 1.1.6 In general, the inf in the denition of dM cannot


be replaced by min : To this end, suppose that M = Rn n f0g: Then
dM (0) = 0; but there is no x 2 M = Rn n f0g such that k0 ak = 0:

2. Linear Subspaces. A ne Sets


In this section we introduce the concepts of linear subspace and
a ne set, which are generalizations to Rn of the terms point, line and
plane from elementary geometry.

Denition 1.2.1 Let M be a nonempty subset of Rn : We say that the


set M is a linear subspace of Rn if, for each s; t 2 R and each x; y 2 M;
we have
sx + ty 2 M:

Example 1.2.2 The sets f0g and Rn are linear subspaces of Rn :

Example 1.2.3 The sets


M1 = fx = (x1 ; :::; xn ) 2 Rn : x1 = ::: = xn g;
M2 = fx = (x1 ; :::; xn ) 2 Rn : x1 = 0g;
M3 = fx = (x1 ; :::; xn ) 2 Rn : x1 = x2 = ::: = xn 1 g
are linear subspaces of Rn :

Example 1.2.4 The set


M = fx = (x1 ; :::; xn ) 2 Rn : x1 = 1g

is not a linear subspace of Rn :

Theorem 1.2.5 If M and L are two linear subspaces of Rn and a;


b 2 R, then the set
aM + bL = fx 2 Rn : there exist u 2 M and v 2 L such that
x = au + bvg
is a linear subspace of Rn :
8 1. CONVEX ANALYSIS

Proof. It is immediate from Denition 1.

Theorem 1.2.6 The intersection of an arbitrary family of linear sub-


spaces of Rn is a linear subspace of Rn :
Proof. Let (Mj )k2J be a family of linear subspaces of Rn : Let
x; y 2 \fMj : j 2 Jg and s; t 2 R. From
x; y 2 \fMj : j 2 Jg;

it follows that x; y 2 Mj ; for all j 2 J: Since each set Mj (j 2 J) is a


linear subspace of Rn , we have that

sx + ty 2 Mj ; for all j 2 J:

Thus
sx + ty 2 \fMj : j 2 Jg;

which show that the intersection is a linear subspace of Rn :

Denition 1.2.7 Let M be a nonempty subset of Rn . The intersection


of all linear subspaces of Rn ; containing the set M , is called the linear
hull of the set M; and is denoted by sp(M ).

Theorem 1.2.8 If M is a nonempty subset of Rn ; then the following


statements are true:
10 The set sp(M ) is a linear subspace of Rn :
20 The inclusion M sp(M ) holds.
30 If M L Rn and L is a linear subspace of Rn ; then sp(M )
L.
40 The set sp(M ) is, with respect to the inclusion relation, the small-
est linear subspace of Rn containing M .
50 The equality
M = sp (M )
holds if and only if the set M is a linear subspace of Rn :
60 The equality
sp (sp (M )) = sp (M )
holds.
Proof. Statements 10 ; 20 ; 30 follow immedialtely from the deni-
tion and from Theorem 6. Statements 40 and 50 follow from statements
10 ; 20 ; 30 : Statement 60 follows from statements 10 and 50 above:
2. LINEAR SUBSPACES. AFFINE SETS 9

Denition 1.2.9 Let M be a subset of Rn . We say that the set M is


a ne if for each x; y 2 M and each t 2 R we have
(1 t) x + ty 2 M:

Example 1.2.10 Each linear subspace of Rn is an a ne set.

Example 1.2.11 The set


M = f(x1 ; :::; xn ) 2 Rn : x1 = 1g

is a ne, but is not a linear subspace of Rn .


Consequently, there exist a ne sets which are not linear subspaces:

Example 1.2.12 The sets


B[0; 1] = fx 2 Cn : kxk 5 1g; B(0; 1) = fx 2 Cn : kxk < 1g;

are not a ne.


Indeed, the points x = 0 2 Rn and y = (1=2)e1 = (1=2; 0; :::; 0) 2
n
R belong to the sets B[0; 1] and B(0; 1), while the point u = (1
4)x + 4y = 2e1 does not belong to B[0; 1] and B(0; 1):
Now, we examine how a ne sets behave with respect to the oper-
ations of addition and scalar multiplication.

Theorem 1.2.13 Let M and L be two subsets of Cn and a; b 2 C: If


the sets M and L are a ne, then the set

aM +bL = fz 2 Cn : there exist u 2 M; v 2 L such that z = au+bvg

is a ne.
Proof. It is immediate from Denition 9.

Theorem 1.2.14 If the subsets M of Rn and L of Rm are a ne, then


the subset M L of Rn+m is a ne.
Proof. It is immediate from Denition 9.
The next result expresses one of the most important properties of
the family of a ne sets, namely that it is closed under the formation
of arbitrary intersections.

Theorem 1.2.15 If (Mj )j2J is a family of a ne subsets of Rn , then


the set
M = \fMj : j 2 Jg
is a ne, too.
10 1. CONVEX ANALYSIS

(The intersection of an arbitrary family of a ne subsets of Rn is


an a ne set.)
Proof. Let x; y 2 M and t 2 R: From x; y 2 M it follows that
x; y 2 Mj ; for all j 2 J: Since each Mj (j 2 J) is a ne, we have
(1 t)x + ty 2 Mj ; for all j 2 J:

Thus
(1 t)x + ty 2 M;
which shows that the set M is a ne:

Denition 1.2.16 Let M be a subset of Rn : The intersection of all


a ne subsets of Rn ; containing M; is called the a ne hull of the set
M; and is denoted by a(M ).

Theorem 1.2.17 If M is a subset of Rn ; then the following statements


are true:
10 The set a(M ) is a ne.
20 The inclusion M a(M ) holds:
30 If M L Rn and L is a ne, then a(M ) L:
40 The set a(M ) is, with respect to the inclusion relation, the
smallest a ne set of Rn containing the set M .
50 The equality
M = a (M )
holds if and only if the set M is a ne.
60 The equality
a (a (M )) = a (M )
holds:
Proof. Statements 10 ; 20 ; 30 follow immediately from the deni-
tion and from Theorem 15. Statements 40 and 50 follow from state-
ments 10 ; 20 ; 30 : Statement 60 follows from statements 10 and 50 above:

3. Convex Sets
The notion of a convex set is central to optimization theory. The
concept of convex set is a simple one. A set in space is convex if when-
ever it contains two points, it also contains the line segment joining
them.
The purpose of this section is to introduce the fundamental concept
of convex set and to present some properties of these sets.
3. CONVEX SETS 11

Denition 1.3.1 Let M be a subset of Rn . We say that the set M is


convex if for each x; y 2 M and each t 2 [0; 1] we have
(1 t)x + ty 2 M:

Example 1.3.2 Each a ne subset of Rn is convex.


Example 1.3.3 The sets
B (0; 1) = fx 2 Rn : kxk < 1g
and
B[0; 1] = fx 2 Rn : kxk 5 1g
are convex but not a ne.
Consequently, there exist convex sets which are not a ne.
Example 1.3.4 The set M = Rn n f0g is not convex.
Indeed, the points
x = e1 = (1; 0; :::; 0) and y = e1 = ( 1; 0; :::; 0)
belong to the set M and the point
(1 (1=2)) x + (1=2)y = 0 2
= M:

Example 1.3.5 Let a 2 Rn : Then the sets


H= (a) = fx 2 Rn : hx; ai = 0g;

H5 (a) = fx 2 Rn : hx; ai 5 0g;


H> (a) = fx 2 Rn : hx; ai > 0g;
H< (a) = fx 2 Rn : hx; ai < 0g;
H(a) = fx 2 Rn : hx; ai = 0g
are convex.
Example 1.3.6 Let A 2 Rn m
: Then the set
M = fy 2 Rn : there exists x 2 Rm
+ such that y = A(x)g
is convex.
Theorem 1.3.7 Let M be a subset of Rn . The following statements
are equivalent:
10 The set M is convex.
20 For each t 2 [0; 1] the equality
(1 t)M + tM = M
12 1. CONVEX ANALYSIS

holds.
30 For each s; t 2 [0; +1[ the equality
sM + tM = (s + t)M
holds.
Proof. (10 =) 20 ) Assume that the set M is convex. Then, for
each t 2 [0; 1]; we have
(1 t) M + tM M:

On the other hand, for each r 2 R, we have


M = 1 M = (1 r + r)M (1 r)M + rM:

Consequently,
(1 t)M + tM = M; for all t 2 [0; 1]:

(20 =) 30 ) Let s and t be two positive real numbers. If at least one


of these numbers is zero, then obviously the following equality holds
sM + tM = (s + t)M:

If both numbers s and t are nonzero, then s+t > 0 and t=(s+s) 2]0; 1[.
From assertion 20 , we have

t t
1 M+ M = M;
s+t s+t

or, after multiplying by s + t, it results


tM + sM = (t + s)M:

(30 =) 10 ) Let x; y 2 M , t 2 [0; 1] and s = 1 t. Then, by assertion


30 ; we deduce that
(1 t)x + ty 2 (1 t)M + tM = (1 t + t)M = M:

Consequently, the set M is convex.

Theorem 1.3.8 Let M be a nonempty subset of Rn . The set M is


convex if and only if, for each x 2 M and each d 2 Rn ; the set

x;d = f 2 [0; +1[ : x + d 2 M g


is convex.
3. CONVEX SETS 13

Proof. Necessity. Assume that the set M is convex and let x 2


M and d 2 Rn . In order to prove that the set x;d is convex, let
; 2 x;d and t 2 [0; 1]. Obviously
(1 t) + t 2 [0; +1[:

Now, by the fact that ; 2 x;d it follows that the points u = x + d


and v = x + d belong to the set M: From the convexity of M; we
obtain
(1 t)u + tv 2 M:
It follows that
x + ((1 t) + t )d = (1 t)(x + d) + t(x + d) =
= (1 t)u + tv 2 M:
Consequently, the set x;d is convex.
Su ciency. Assume that, for each x 2 M and each d 2 Rn , the
set x;d is convex. In order to prove that the set M is convex, let
x; y 2 M and t 2 [0; 1]: Since
x + 0(y x) = x 2 M
and
x + 1 (y x) = y 2 M;
we deduce that the points 0 and 1 belong to the set x;y x : By the
convexity of the set x;y x ; it follows that t 2 x;y x . Consequently,
(1 t)x + ty = x + t(y x) 2 M:

The proof is complete.


Below we examine how convex sets behave with respect to the op-
erations of addition and scalar multiplication.

Theorem 1.3.9 Let M and L be subsets of Rn and a; b 2 R. If the


sets M and L are convex, then the set
aM + bL = fx 2 Rn : there are u 2 M; v 2 L such that x = au + bvg

is convex, too.
Proof. Let x; y 2 aM + bL and t 2 [0; 1]. Then
(1 t)x + ty 2 (1 t)(aM + bL) + t(aM + bL) =
= a ((1 t)M + tM ) + b ((1 t)L + tL) =
= aM + bL:
Consequently, the set aM + bL is convex.
14 1. CONVEX ANALYSIS

Corollary 1.3.10 If M and L are two convex subsets of Rn and a is


a real number, then the sets M + L, M L, aM , M are convex, too.
Proof. It is immediate from Theorem 9.

Theorem 1.3.11 Let M; a subset of Rn and L; a subset of Rm be two


convex sets. Then the set M L is convex, too.
Proof. Let x = (x1 ; x2 ) ; y = (y1 ; y2 ) 2 M L; and t 2 [0; 1] :
From the convexity of M and L we have 2 (1 t) x1 + ty1 2 M and
(1 t) x2 + ty2 2 L: It follows that
(1 t) x + ty = ((1 t) x1 + ty1 ; (1 t) x2 + ty2 ) 2 M L:
Consequently, the set M L is convex.

Theorem 1.3.12 Let M and L be two convex subsets of Rn+m and


P = fx 2 Rn+m : there exist u 2 Rn and v; w 2 Rm such that
(u; v) 2 M; (u; w) 2 L and x = (u; v + w)g:
Then the set P is convex, too.
Proof. Let x1 ; x2 2 P and t 2 [0; 1]: From x1 ; x2 2 P we deduce
that there exist the points u1 ; u2 2 Rn and v 1 ; v 2 ; w1 ; w2 2 Rm such
that
(u1 ; v 1 ) 2 M; (u1 ; w1 ) 2 L; x1 = (u1 ; v 1 + w1 )
and
(u2 ; v 2 ) 2 M; (u2 ; w2 ) 2 L; x2 = (u2 ; v 2 + w2 ):
From the convexity of the sets M and L we obtain
((1 t) u1 + tu2 ; (1 t)v 1 + tv 2 ) = (1 t)(u1 ; v 1 ) + t(u2 ; v 2 ) 2 M
and
((1 t) u1 + tu2 ; (1 t)w1 + tw2 ) = (1 t)(u1 ; w1 ) + t(u2 ; w2 ) 2 L:
Let now
u = (1 t)u1 + tu2 ; v = (1 t)v 1 + tv 2 ; w = (1 t)w1 + tw2 :

This shows that (u; v) 2 M; (u; w) 2 L and then


(1 t) x1 + tx2 =
= (1 t) u1 + tu2 ; (1 t) v 1 + tv 2 + (1 t) w1 + tw2 =
= (u; v + w) 2 P:
Consequently, P is convex.
3. CONVEX SETS 15

Remark 1.3.13 The reunion of two convex sets might not be convex.
Indeed, the sets
M = fx 2 R : x = 1g and L = fx 2 R : x 5 1g

are convex, while the set M [ L is not convex.


The next result expresses one of the most important properties of
the family of convex sets, namely that it is closed under the formation
of arbitrary intersections.

Theorem 1.3.14 If (Mj )j2J is a family of convex subsets of Rn ; then


their intersection
M = \fMj : j 2 Jg
is a convex set, too.
(The intersection of an arbitrary family of convex subsets is a convex
set.)
Proof. If M = ;, then the statement is obvious. If M 6= ;, let
x; y 2 M and t 2 [0; 1]: Then x; y 2 Mj ; for all j 2 J: The set Mj being
convex, for each j 2 J, it follows that
(1 t)x + ty 2 Mj ; for all j 2 J:
Thus
(1 t)x + ty 2 M:
Consequently, the set M is convex.

Corollary 1.3.15 Let A be a nonempty subset of Rn : Then the sets


M1 = \fH= (a) : a 2 Ag; M2 = \fH5 (a) : a 2 Ag;
M3 = \fH> (a) : a 2 Ag; M4 = \fH< (a) : a 2 Ag;
M5 = \fH(a) : a 2 Ag
are convex.
Proof. Invoke for it the theorem above and example 5.

Denition 1.3.16 Let M be a subset of Rn . The convex hull of M;


denoted by conv(M ), is the intersection of all convex subsets of Rn
containing M .

Theorem 1.3.17 Let M be a subset of Rn : Then the following state-


ments are true:
10 The set convM is convex.
20 The inclusion M convM holds:
16 1. CONVEX ANALYSIS

30 If M L Rn and the set L is convex, then convM L:


40 The set convM is, with respect to the inclusion relation, the
smallest convex set in Rn containing M .
50 The equality
M = convM
holds if and only if the set M is convex.
60 The equality
conv(convM ) = convM
holds.
Proof. The rst three assertions of the theorem are immediate
from the denition and Theorem 14. Statements 40 and 50 follow from
statements 20 and 30 : Statement 60 follows from statements 10 and
50 :

Example 1.3.18 The convex hull of the unit sphere


S(0; 1) = fx 2 Rn : kxk = 1g
is the closed unit ball
B[0; 1] = fx 2 Rn : kxk 5 1g;
i.e.
convfx 2 Rn : kzk = 1g = fx 2 Rn : kxk 5 1g:
Indeed, the closed unit ball B[0; 1] Rn is convex and contains the
unit sphere S(0; 1); so
convS(0; 1) B[0; 1]:

In order to prove that B[0; 1] convS(0; 1); let x 2 B[0; 1]:


If x = 0; then for u = (1; 0; :::; 0) 2 Rn and v = u; which belong
to S(0; 1); we have
x = (1=2)u + (1=2)v:
Since conv (S(0; 1)) is convex and
S(0; 1) convS(0; 1)

it follows that x 2 convS(0; 1):


If x 6= 0; then 0 < kxk 5 1 and hence t = (1 kxk) =2 2 [0; 1] : On
the other hand u = (1= kxk) x and v = ( 1= kxk) x belong to S (0; 1)
and x can be written as follows
x = (1 t) u + tv:
3. CONVEX SETS 17

Since conv (S(0; 1)) is convex and


S(0; 1) conv (S(0; 1))

it follows that x 2 conv (S(0; 1)) : Consequently,


B[0; 1] convS(0; 1):
Thus
convS(0; 1) B[0; 1] and B[0; 1] convS(0; 1)
as desired.

Denition 1.3.19 Let p = 1 be an integer number and x; x1 ; :::; xp be


p + 1 points from Rn : The point x is said to be a convex combination
of the points x1 ; :::; xp if there exist p real numbers t1 ; :::; tp 2 [0; +1[
such that t1 + ::: + tp = 1 and
x = t1 x1 + ::: + tp xp :

Denition 1.3.20 Let M be a nonempty subset of Rn : A point x 2 Rn


is said to be a convex combination of points from M if there exists an
integer number p = 1 and p points x1 ; :::; xp 2 M such that x is a
convex combination of the points x1 ; :::; xp :
The convex hull of a set was dened by means of convex sets contain-
ing that set. By contrast, the following theorem expresses the convex
hull of a set in terms of the points of the set themselves. The following
representation theorem of the convex hull of a set holds.

Theorem 1.3.21 The convex hull of a nonempty subset M of Rn is


the set of all convex combinations of points from M:
Proof. Let L denote the latter set, that is,
L = fx 2 Rn : there exists p 2 N, there exist x1 ; :::; xp 2 M;

there exist t1 ; :::; tp 2 [0; +1[ with t1 + ::: + tp = 1

such that x = t1 x1 + ::: + tp xp g:


Obviously, M L: Moreover, it is also obvious that the set L is convex.
Then, in virtue of assertion 30 of Theorem 17, it follows that
convM L:

We next show that


L convM:
18 1. CONVEX ANALYSIS

Let x 2 L; it follows that there exist an integer number p = 1, p points


x1 ; :::; xp 2 M and p real numbers t1 ; :::; tp 2 [0; +1[ with t1 +:::+tp = 1
such that
x = t1 x1 + ::: + tp xp :
We argue by induction on p, that x 2 conv (M ) : When p = 1; the
assertion is trivial. Assume now that each convex combination of p
1 = 1 points from M belongs to convM and we show that x; which
is a convex combination of p points from M; belongs to convM: We
distinguish two cases according to weather tp is equal or not to 1:
a) If tp = 1; then t1 = ::: = tp 1 = 0 and hence
x = tp xp = xp 2 M convM:

b) If tp 6= 1; then let
t1 tp 1
s1 = ; :::; sp 1 = :
1 tp 1 tp
Obviously, s1 ; :::; sp 2 [0; +1[ and
1
t1 + ::: + tp 1 1 tp
s1 + ::: + sp 1 = = = 1:
1 tp 1 tp
Then the point
v = s1 x1 + ::: + sp 1 xp 1
2 convM:
Moreover,
x = (1 tp )v + tp xp
and since the set convM is convex and M convM; we deduce that
x 2 convM: The theorem is proved.

Theorem 1.3.22 The subset M of Rn is convex if and only if M is


the set of all convex combinations of points from M:
Proof. Apply Theorems 21 and 17.

Theorem 1.3.23 Let x1 ; :::; xp 2 Rn : Then


convfx1 ; :::; xp g =

= ft1 x1 + ::: + tp xp : t1 ; :::; tp 2 [0; +1[ with t1 + ::: + tp = 1g =


= fx 2 Rn : there exist t1 ; :::; tp 2 [0; +1[ with t1 + ::: + tp = 1
such that x = t1 x1 + ::: + tp xp g:
3. CONVEX SETS 19

Proof. It is immediate from Theorem 21.


For example, the convex hull of three points in R2 is the triangle
having these points as vertices.
In the above results we considered all convex combinations, that is,
for all numbers p of points and for arbitrary selections of these points.
Both p and the points selected can be restricted. We now show that
the number of points p need not be larger than n + 1:

Theorem 1.3.24 (Caratheodory0s theorem) If M is a subset of Rn ;


then every element of convM is a convex combination of at most n + 1
points of M:
Proof. Let x 2 convM ; then there exists p 2 N, there exist p
points x1 ; :::; xp 2 M; there exist p real numbers t1 ; :::; tp 2 [0; +1[
with t1 + ::: + tp = 1 such that x = t1 x1 + ::: + tp xp :
If p n + 1; then theorem is proved.
If p > n + 1, then we shall show that p can be reduced by one. If
there exists j 2 f1; :::; pg such thet tj = 0; then we can delete the jth
point and we are done. So let, tk > 0; for all k 2 f1; :::; pg; then let
y 1 = (x1 ; 1) ; :::; y p = (xp ; 1) from Rn+1 : Then there exist p real numbers
s1 ; :::; sp not all equal 0; so that
s1 y 1 + + sp y p = 0;
or equivalent
(1.3.1) s1 x 1 + + sp xp = 0 and s1 + + sp = 0:
Obviously, there exists al least k 2 f1; :::; pg such that sk > 0; then the
set J = fj 2 f1; :::; pg : sj > 0g is nonempty: Let
tj
q = min : j2J ; and rk = tk qsk ; for all k 2 f1; :::; pg:
sj
Then, by (1.3.1) we still have
r1 + ::: + rp = (t1 + ::: + tp ) q (s1 + ::: + sp ) = 1;
and
r1 x1 + ::: + rp xp = t1 x1 + ::: + tp xp q s1 x1 + ::: + sp xp = x:
By the denition of q; at least one rj = 0 and then we can delete the
jth point.
Continuiting in this way, we can reduce the number of points to
n + 1:
20 1. CONVEX ANALYSIS

Theorem 1.3.25 In Rn , the convex hull of an open set is an open


set.
Proof. Let M be an open subset of Rn . If x 2 convM; then
there exist p 2 N, the points x1 ; :::; xp 2 M and the real numbers
t1 ; :::; tp 2 [0; +1[ with
t1 + ::: + tp = 1
such that
x = t1 x1 + ::: + tp xp :

Since M is open and x1 ; :::; xk 2 M; there exist p real numbers r1 >


0; :::; rp > 0 such that
B(x1 ; r1 ) M; :::; B(xp ; rp ) M:
Let
r = minfr1 ; :::; rp g:
Then r > 0 and
B(x1 ; r) B(x1 ; r1 ) M; :::; B(xp ; r) B(xp ; rp ) M:

We show that
B(x; r) convM:

Let u 2 B(x; r); then ku xk < r:


Moreover, for each k 2 f1; :::; pg; the point
uk = x k + u x

lies in B(xk ; r) and hence in M: Since the point u can be written as


u = t1 u1 + ::: + tp up ;

we deduce that u 2 convM: Thus


B(x; r) convM:

Consequently, each point of convM is an interior point of convM; i.e.


convM is an open set.

Theorem 1.3.26 If M is a convex subset of Rn ; then its interior


intM and its closure clM are convex.
3. CONVEX SETS 21

Proof. Let x; y 2 intM and t 2 [0; 1] : From x; y 2intM; it follows


that exists a real number r > 0 such that B (x; r) M and B (y; r)
M: Then
B ((1 t) x + ty; r) = f(1 t) x + tyg + B (0; r) =
= (1 t) (fxg + B (0; r)) + t (fyg + B (0; r)) =
= (1 t) B (x; r) + tB (y; r)
(1 t) M + tM = M;
because the sets B (0; r) and M are convex. Consequently, (1 t) x +
ty 2 intM; hence intM is convex.
To prove the second part of the theorem, let x; y 2 clM; and t 2
[0; 1]: From x; y 2 clM; it follows that exist two sequences xk k 1 and
y k k 1 such that xk ; y k 2 M; for all k 2 N, and xk ! x and y k ! y;
when k ! 1: The set M is convex, and hence, for each k 2 N, we have
(1 t) xk +ty k 2 M: On the other hand (1 t) xk +ty k ! (1 t) x+ty;
when k ! 1: It follows that (1 t) x + ty 2clM; hence the set clM is
convex.

Theorem 1.3.27 In Rn , the convex hull of a bounded set is a bounded


set.
Proof. Let M be a bounded subset of Rn ; then there exists a real
number r > 0 such that
M B[0; r]:

Since B[0; r] is convex, in view of Theorem 17, we have


convM B[0; r]:

Consequently, convM is bounded.


The next theorem is more explicit. We give a denition before.

Denition 1.3.28 Let M be a nonempty bounded subset of Rn . The


real number
diamM = supfkx yk : x; y 2 M g

is called the diameter of the set M:

Theorem 1.3.29 Let M be a nonempty bounded subset of Rn : Then


M and convM have the same diameter.
22 1. CONVEX ANALYSIS

Proof. Assume that the set M has the diameter r: If x; y 2


convM; then there exist p; q 2 N and
x1 ; :::; xp ; y 1 ; :::; y q 2 M ; t1 ; :::; tp ; s1 ; :::; sq = 0;
with
t1 + ::: + tp = 1; s1 + ::: + sq = 1;
such that
x = t1 x1 + ::: + tp xp , y = s1 y 1 + ::: + sq y q :
Thus p q p q
X X X X
k
x= tk si x ; y= tk si y i ;
k=1 i=1 k=1 i=1

whence, by the triangle inequality,


p q
X X
kx yk = tk si (xk yi) 5
k=1 i=1
p q p q
XX X X
5 tk si xk yi 5 tk si r = r:
k=1 i=1 k=1 i=1
Hence the diameter of the set convM does not exceed r: Since M
convM; the diameter of the set convM is at least r: Thus the set convM
has the diameter r: The theorem is proved.
The next result, while not having the same basic importance as
others in this section, does have its own intrinsic interest and will be
needed later in the chapter.
Theorem 1.3.30 Let M be a nonempty bounded subset of Rn , N be
a closed convex subset of Rn and L Rn be a set such that
M +L M + N:
Then
L N:
Proof. Let x0 2 M: If L = ;; then we have L N and the
theorem is proved. If L 6= ;; then let x 2 L: Since
x0 + x 2 M + L M + N;
we deduce that there exist two points x1 2 M and u1 2 N such that
x0 + x = x1 + u1 :
Similarly, from
x1 + x 2 M + L M + N;
4. SEPARATION THEOREMS FOR CONVEX SETS 23

there exist two points x2 2 M and u2 2 N such that


x1 + x = x2 + u2 :

Going on in this way, it follows that there exist the sequences xk k2N
;
uk k2N which, for each k 2 N; satisfy the relations:
(1.3.2) x k 2 M ; uk 2 N ; x k 1
+ x = x k + uk :

From (1.3.2) we deduce that, for each k 2 N; we have


(1.3.3) x0 + kx = xk + (u1 + ::: + uk ):

Since N is convex, the point


1 1
vk = (u + ::: + uk )
k
belongs to the set N; for all k 2 N:
On the other hand, from (1.3.3) we obtain that
1 k
(1.3.4) x vk = x x0 ; for all k 2 N.
k
Since the set M is bounded and xk 2 M; for each k 2 N, we deduce
that there exists the limit
1 k
lim x x0 = 0:
k!1 k

From this and (1.3.4), it follows that the sequence v k k2N converges
to x: Since the set N is closed, it follows that x 2 N: Consequently,
L N:
The theorem is proved.

4. Separation Theorems for Convex Sets


The results that we establish in this section on the separation of
convex sets are among the most important ones in convexity. From
several possible ways of approach, we have chosen that which uses the
distance function. The distance function dM : Rn ! R of a nonempty
subset M of Rn is dened by
dM (x) = inffkx uk : u 2 M g; for all x 2 Rn
(see Denition 3).
In general, the inf in the denition of dM cannot be replaced by
min : To this end, assume that M = Rn n f0g: Then dM (0) = 0; but
there exists no a 2 M = Rn n f0g such that k0 ak = 0:
24 1. CONVEX ANALYSIS

We have the following result.

Theorem 1.4.1 Let M be a nonempty closed subset of Rn and x 2 Rn :


Then there exists an element x0 2 M such that
dM (x) = x x0 :

Proof. From the denition of dM (x) it follows that there exists a


sequence xk k2N of points from M such that
(1.4.1) lim x xk = dM (x):
k!1

From this it follows that the sequence x xk k2N


is bounded, hence
there exists a real number r > 0 such that
x xk 5 r; for all k 2 N.

The inequalities
xk 5 x xk + kxk 5 r + kxk ; for all k 2 N

show that the sequence xk k2N is bounded. Then it contains a subse-


quence (xkj )j2N which converges to a point x0 2 Rn : Since the set M
is closed, it follows that x0 2 M:
Moreover, from (1.4.1), it follows that
lim x xkj = dM (x):
j!1

But
lim x xk = x x0 ;
k!1
hence
dM (x) = x x0
and the theorem is proved.
For obvious reasons, the point x0 in the statement of Theorem 1 is
called a nearest point of M to x:
In what follows we need the following result.

Theorem 1.4.2 Let x; u 2 Rn : If there exists a real number >0


such that
(1.4.2) kx + tuk = kxk ; for all t 2]0; [;
then
hx; ui = 0:
4. SEPARATION THEOREMS FOR CONVEX SETS 25

Proof. Since
kx + tuk2 = hx + tu; x + tui = kxk2 + 2t hx; ui + t2 kuk2 ;

from (1.4.2) we deduce that, for each t 2]0; [; we have


t
hx; ui + kuk2 = 0:
2
Passing to limit t ! 0+ in the last inequality, we deduce that
hx; ui = 0:
The theorem is proved.
Theorem 1 shows that, given a nonempty closed subset M of Rn
and a point x of Rn , there exists a point x0 of M which is nearest to
x: We show below that, if M is also convex, then this nearest point x0
of M to x is unique and that, for each point u of M; the points x x0
and u u0 make a nonacute angle with each other.

Theorem 1.4.3 Let M be a nonempty closed convex subset of Rn and


x 2 Rn : Then there exists a unique point x0 2 M such that
dM (x) = x x0 :
Moreover,
(1.4.3) x x0 ; u x0 5 0; for all u 2 M:

Proof. By Theorem 1, there exists a point x0 2 M such that


dM (x) = x x0 :

In order to prove (1.4.3), let u 2 M: By the convexity of the set M we


deduce that
(1 t) x0 + tu 2 M; for all t 2]0; 1]
and the property of x0 ; we get that
x x0 5 x (1 t) x0 + tu = x x0 + t x0 u ;

for all t 2]0; 1]: From this, in view of Theorem 2, we have


x x0 ; u x0 5 0:

Hence (1.4.3) is true.


To prove the unicity of x0 ; let x1 2 M be such that
dM (x) = x x1 :
26 1. CONVEX ANALYSIS

Then, by what we have just proved,


x x 0 ; x1 x0 5 0 and x x 1 ; x0 x1 5 0:

Adding these two last inequalities, we deduce that


2
x0 x1 = x0 x 1 ; x0 x1 = x0 x+x x 1 ; x0 x1 =
= x0 x; x0 x1 + x x 1 ; x0 x1 5 0;
which proves that x0 = x1 : The proof is complete:
Relation (1.4.3) characterizies the point x0 with the property that
dM (x) = x x0 :
The following statement is true.

Theorem 1.4.4 Let M be a nonempty closed convex subset of Rn ;


x 2 Rn n M and x0 2 M: Then the following statements are equivalent:
10 The equality
x x0 = dM (x)
holds:
20 For each u 2 M; we have
x x0 ; u x0 5 0:
Proof. If statement 10 is true; then, in view of Theorem 3, state-
ment 20 is true:
Assume now that statement 20 is true and let u be an arbitrary
point from M: Then
2 2
kx uk2 x x0 = kuk2 + 2 x; x0 u x0 =
0 2
= kuk2 + 2 x0 ; x0 u x =
2
= kuk2 2 x0 ; u + x0 =
0 2
= u x = 0:
The theorem is proved.
A convex closed set and a point outside of it can be separated by a
plane.

Theorem 1.4.5 (strict separation theorem) Let M be a closed convex


set of Rn and u 2 Rn n M: Then there exist a point a 2 Rn n f0g and a
real number " > 0 such that
ha; ui + " 5 ha; xi ; for all x 2 M:
4. SEPARATION THEOREMS FOR CONVEX SETS 27

Proof. By Theorem 3, there exists u0 2 M such that


(1.4.4) u u0 ; x u0 5 0; for all x 2 M:
Dene a = u0 u . Note that a is not zero, because u 2
= M: Then,
for each x 2 M; inequality (1.4.4) implies
ha; xi = a; u0 = ha; ui + a; u0 u =
2
= ha; ui + kak :

Setting " = kak2 ; we obtain the assertion of the theorem.


If the set M is not closed, the point u 2 Rn n M may be a boundary
point of M and we have a weaker version of the above result.

Theorem 1.4.6 (separation theorem) Let M be a convex set of Rn


and u 2 Rn n M: Then there exists a point a 2 Rn n f0g such that
(1.4.5) ha; ui 5 ha; xi ; for all x 2 M:
Proof. Let uk k=1 be a sequence of points uk 2 M n clM; (k = 1)
converges to u: By Theorem 5, for each k = 1 there exists a point
ak 2 Rn n f0g such that
ak ; uk 5 ak ; x ; for all x 2 M:
Since ak 6= 0; for all k = 1; without loss of generality, we may
assume that ak = 1; for all k = 1; because we can divide both sides
of (1.4.5) by ak :
Then the sequence ak contains at least convergent subsequence
akp p=1 . Let a 2 Rn the limit of the subsequence akp : Obviously,
kak = 1 and
(1.4.6) akp ; ukp 5 akp ; x ; for all x 2 M and p = 1:
Passing to the limit in (1.4.6) ; we obtain the required result.
Separation is possible also when both sets being separated are con-
vex sets.

Theorem 1.4.7 Let L and M be nonempty subsets of Rn : If L is closed


and M is compact, then there exist two points u0 2 L and x0 2 M such
that
(1.4.7) u0 x0 = inffku xk : u 2 L; x 2 M g:
28 1. CONVEX ANALYSIS

Proof. The distance function dL : Rn ! R of L is continuous on


n
R . Then there exists a point x0 2 M such that
dL (x0 ) = inffdL (x) : x 2 M g:

By Theorem 1, t to be used since L is closed, there exists a point


u0 2 L such that
dL (x0 ) = u0 x0 :
Moreover, for each u 2 L and each x 2 M; we have
ku xk = inffkv xk : v 2 Lg = dL (x) =
= inffdL (z) : z 2 M g = dL (x0 ) =
= u0 x0 ;
which, since u0 2 L; x0 2 M; shows that
u0 x0 = inffku xk : u 2 L; x 2 M g:
The theorem is proved.
We refer to the points u0 and x0 in the statement of Theorem 7 as
nearest points of L and M: These points u0 and x0 are not unique, in
general.

Theorem 1.4.8 Let L and M be two nonempty disjoint convex subsets


of Rn : If L is closed and M is compact, then there exist a point a 2
Rn n f0g and a real number " > 0 such that, for each u 2 L and each
x 2 M; we have
(1.4.8) ha; ui + " 5 ha; xi :

Proof. In view of Theorem 7, there exist two points u0 2 L and


0
x 2 M such that
u0 x0 = inffku xk : u 2 L; x 2 M g:

Since L and M are disjoint, we have u0 6= x0 : Let now


2
a = x0 u0 and " = x0 u0 :

Obviously, a 2 Rn n f0g and " > 0: In order to prove (1.4.8), let u 2 L


and x 2 M: In view of Theorem 3, we have
x0 u0 ; u u0 5 0 and u0 x0 ; x x0 5 0:

Then
2
ha; xi ha; ui = a; x0 a; u0 = a; x0 u0 = x0 u0 = ":
5. CONES 29

The theorem is proved.

Remark 1.4.9 The boundedness assumption in Theorem 8 is essen-


tial. The closed convex sets
M = fx = (x1 ; x2 ) 2 R2 : x2 5 0g
and
L = fx = (x1 ; x2 ) 2 Rn : x2 = exp ( x1 )g
cannot be strictly separated.
If the sets are not bounded, then we have a weaker version of The-
orem 8.

Theorem 1.4.10 If L and M are two nonempty disjoint convex sub-


sets of Rn ; then there exists a point a 2 Rn n f0g such that, for each
u 2 L and each x 2 M; we have
ha; ui 5 ha; xi :
Proof. Dene N = L M: Since 0 does not belong to N; we can
use Theorem 6. Then there exists a point a 2 Rn n f0g such that
ha; vi 5 ha; 0i = 0; for all v 2 N:
Since v 2 N has the form v = u x with u 2 L and x 2 M; the
theorem is proved.

5. Cones
The purpose of this section is to introduce the concept of cone and
to present some properties of these sets.

Denition 1.5.1 Let K be a nonempty subset of Rn . We say that


the set K is a cone if, for each x 2 K and each t 2]0; +1[; we have
tx 2 K:

Example 1.5.2 Each linear subspace of Rn is a cone.

Example 1.5.3 The set K = Rn n f0g is a cone.


Let us remark that the set Rn n f0g is not a subspace of the linear
space Rn , therefore not every cone is a linear subspace of Rn .
Let us also remark that the set Rn n f0g is not a convex set, hence
a cone might not be a convex set.

Example 1.5.4 The sets


B(0; 1) = fx 2 Rn : kxk < 1g and B[0; 1] = fx 2 Rn : kxk 5 1g
30 1. CONVEX ANALYSIS

are not cones.


Let us remark that the sets B(0; 1) and B[0; 1] are convex. Conse-
quently, there exist convex sets which are not cones.

Remark 1.5.5 All cones are unbounded, with the exception of the
trivial cone f0g.

Theorem 1.5.6 The nonempty subset K of Rn is a cone if and


only if
tK = K; for all t 2]0; +1[:

Proof. Necessity. If K is a cone and t 2]0; +1[, then by the


denition of a cone we deduce that
tK K:

In order to prove the converse inclusion, let x 2 K and t 2]0; +1[:


Since 1=t 2]0; +1[, by the fact that K is a cone, it follows that
(1=t)x 2 K:

Then, we have
x = t((1=t)x) 2 tK
and hence we showed that
K tK:

Su ciency. Let t 2]0; +1[ and x 2 K: Then


tx 2 tK = K

and hence tx 2 K, therefore K is a cone.

Theorem 1.5.7 If K and L are cones in Rn and a; b 2 R, then the


set
aK + bL
is a cone, too.
Proof. Let x 2 aK + bL and t 2]0; +1[. From x 2 aK + bL we
deduce that there exist s 2 K and v 2 L such that x = as + bv: Since
K and L are cones, we have that ts 2 K and tv 2 L and hence
tx = t(as + bv) = a(ts) + b(tv) 2 aK + bL:
The theorem is proved.
5. CONES 31

Theorem 1.5.8 If (Kj )j2J is a nonempty family of cones in Rn , then


the set
K := [fKj : j 2 Jg
is a cone.
Proof. Let x 2 K and t 2]0; +1[. From x 2 K we deduce that
there exists a point i 2 J such that x 2 Ki : Then, by the fact that Ki
is a cone and Ki K; it follows that
tx 2 tKi = Ki K:
The proof is complete.

Theorem 1.5.9 If K; a subset of Rn and L, a subset of Rm are cones,


then the set
K L Rn+m
is a cone, too.
Proof. Let x 2 K L; then there exist s 2 K and v 2 L such
that x = (s; v) : Then for each t 2]0 2; +1[; we have tx = t (s; v) =
(ts; tv) 2 K L; because K and L are cones.

Theorem 1.5.10 If (Kj )j2J is a nonempty family of cones in Rn and


the set
K := \fKj : j 2 Jg
is nonempty, then the set K is a cone.
Proof. Let x 2 K and t 2]0; +1[: From x 2 K; we deduce that
x 2 Kj , for all j 2 J: Then tx 2 Kj ; for all j 2 J: Consequently,
tx 2 K and hence K is a cone.

Denition 1.5.11 Let K be a nonempty subset of Rn : We say that


the set K is a convex cone if it is a cone and is convex.

Example 1.5.12 Each subspace of Rn is a convex cone.

Example 1.5.13 The positive orthant of Rn ; i.e.the set


Rn+ = fx = (x1 ; :::; xn ) 2 Rn : xj = 0; for all j 2 f1; :::; ngg
is a convex cone.

Example 1.5.14 The norm cone, i.e. the set


K = f(x; t) 2 Rn+1 : kxk 5 tg;
32 1. CONVEX ANALYSIS

is a convex cone.

Example 1.5.15 The set K = Rn n f0g is a cone without being a


convex cone.

Example 1.5.16 The sets B(0; 1) and B[0; 1] in Rn are convex without
being convex cones.

Example 1.5.17 Let a 2 Rn n f0g: The sets


H= (a) = fx 2 Rn : hx; ai = 0g;

H> (a) = fx 2 Rn : hx; ai > 0g;

H5 (a) = fx 2 Rn : hx; ai 5 0g;

H< (a) = fx 2 Rn : hx; ai < 0g;

H(a) = fx 2 Rn : hx; ai = 0g

are convex cones.

Example 1.5.18 Let A 2 Rn m


: The set
K := fy 2 Rn : there exists x 2 Rm
+ such that y = A(x)g

is a convex cone.

Theorem 1.5.19 The cone K in Rn is convex if and only if


K + K = K:

Proof. Necessity. If K is a convex cone, then by Theorems 7


and 6 it follows that
K + K = (1=2)K + (1=2)K = (1=2 + 1=2)K = K:

Su ciency. Let t 2]0; 1[: Then, in view of Theorem 6, we have


(1 t) K + tK = K + K = K:
Invoke for it Theorem 7.
5. CONES 33

Example 1.5.20 The set


K = fx = (x1 ; x2 ; x3 ) 2 R3 : x1 = 0; x2 = 0; x1 x2 = (x3 )2 g

is a convex cone.
Indeed, it is clear that the set K is a cone. In order to prove that
K is a convex cone, we invoke Theorem 19. Let x = (x1 ; x2 ; x3 ) and
u = (u1 ; u2 ; u3 ) from K: Then
x1 + u1 = 0; x2 + u2 = 0;
and
(x1 + u1 )(x2 + u2 ) = x1 x2 + x2 u1 + x1 u2 + u1 u2 =
p
= x1 x2 + 2 x1 x2 u1 u2 + u1 u2 = (x3 )2 + 2x3 u3 + (u3 )2 =
= (x3 + u3 )2 ;

hence x + u 2 K: We deduce that


K +K K:
The inclusion
K K +K

being obvious, we obtain that


K + K = K:

In view of Theorem 19, it follows that K is a convex cone.

Theorem 1.5.21 The nonempty subset K of Rn is a convex cone if


and only if the following two statements are true:
(i) tK = K; for all t 2]0; +1[;
(ii) K + K = K:
Proof. Invoke for it Theorems 6 and 19.

Theorem 1.5.22 If K and L are convex cones in Rn and a; b 2 R,


then the set
aK + bL

is a convex cone, too.


Proof. Invoke for it Theorems 9 and 7.
34 1. CONVEX ANALYSIS

Remark 1.5.23 If K and L are convex cones in Rn , then the set


K [ L may not be a convex cone.
Indeed the sets
K = fx = (x1 ; x2 ) 2 R2 : x2 = 0g = R f0g
and
L = fx = (x1 ; x2 ) 2 R2 : x1 = 0g = f0g R
are convex cones in R, but the reunion K [ L is not a convex cone
(K [ L is a cone, but K [ L is not convex).

Theorem 1.5.24 If K and L are convex cones in Rn respectively Rm ,


then the set K L is a convex cone in Rn+m .
Proof. Invoke for it Theorems 11 and 9:

Theorem 1.5.25 If K is a convex cone in Rn with nonempty interior,


then
K + intK intK:

Proof. Since
K + intK K + K;
by Theorem 19, it follows that
K + intK K:

Let now x 2 K; then the set fxg + intK is open, because intK is an
open set. Since
K + intK = [ffxg + int (K) : x 2 Kg;

we deduce that the set K +intK is open. Now by the fact that intK is,
with respect to the inclusion relation, the greatest open set containing
K; we deduce that
K + intK intK:
The theorem is proved.
It is a routine matter to show that the intersection of each family
of convex cones in Rn is a convex cone.

Theorem 1.5.26 If (Kj )j2J is a nonempty family of convex cones in


Rn and the set
K := \fKj : j 2 Jg
is nonempty, then K is a convex cone, too.
5. CONES 35

Proof. Invoke for it Theorems 14 and 10.

Denition 1.5.27 Let K be a nonempty subset of Rn . The intersec-


tion of all convex cones in Rn containing the set K is called the convex
cone generated by K (or, convex cone hull) and is denoted by conK:
Theorem 1.5.28 Let K be a nonempty subset of Rn : The following
statements are true:
10 The set conK is a convex cone.
20 The inclusion C conC holds:
30 If L K Rn and K is a convex cone, then conL K:
40 The set conK is, with respect to the inclusion relation, the small-
est convex cone in Rn containing K.
50 The equality
K = conK
holds if and only if the set K is a convex cone.
60 The equality
con (conK) = conK
holds:
Proof. The rst three assertions of the theorem are immediate
from Denition 27 and Theorem 26. Statements 40 and 50 follow from
statements 20 and 30 : Statement 60 is deduced from statements 10 and
50 :

Theorem 1.5.29 Let K be a closed convex cone in Rn and u 2 Rn nC:


Then there exists a point a 2 Rn n f0g such that
hx; ai = 0; for all x 2 K
and
hu; ai < 0:

Proof. Invoke for it Corollary ??.

Denition 1.5.30 Let K be a nonempty subset of Rn : We say that


the set K is a pointed convex cone if it is a convex cone and
K \ ( K) f0g:

Example 1.5.31 The set


fx 2 Rn : x = 0g
is a pointed convex cone.
36 1. CONVEX ANALYSIS

Example 1.5.32 The entire space Rn is a convex cone which is not


pointed.

Theorem 1.5.33 If K is a pointed convex cone in Rn ; then K n f0g


is a pointed convex cone, too.
Proof. Let x 2 K n f0g and t > 0: Since K is a cone we have that
tx 2 K. Moreover, tx 6= 0 because t 6= 0 and x 6= 0: Consequently,
K n f0g is a cone.
We establish now the convexity of the set K n f0g. To this end, let
x; u 2 K n f0g and t 2]0; 1[: Since C is convex, it follows that
(1 t) x + tu 2 K:

We must show that


(1 t) x + tu 6= 0:
Assume that
(1 t) x + tu = 0;
then
u = ((t 1)=t)x
and u 2 C: From u 2 C; it follows that
u 2 K \ ( K) ;

hence u = 0 because the cone K is pointed. But u 2 K n f0g; therefore


u 6= 0; which is a contradiction. Consequently, K n f0g is a convex
cone.
Since K n f0g C; it follows that
(K n f0g) \ ( (K n f0g)) K \ ( K) f0g;

hence K n f0g is a pointed convex cone.

Denition 1.5.34 Let K be a nonempty subset of Rn . We say that


the set K is a polyhedral cone if there exist an integer number q = 1
and q points a1 ; :::; aq 2 Rn such that
K = \fH= (ak ) : k 2 f1; :::; qgg;
where
H= (ak ) = fx 2 Rn : hx; ak i = 0g; for all k 2 f1; :::; qg:

Example 1.5.35 Each subspace of Rn is a polyhedral cone.


5. CONES 37

Theorem 1.5.36 If K; a subset of Rn and L; a subset of Rm are


polyhedral cones, then K L Rn+m is a polyhedral cone.
Proof. Since K and L are polyhedral cones, it follows that there
exist two positive integer numbers p and q and p + q points a1 ; :::; ap 2
Rn ; b1 ; :::; bq 2 Rm ; such that
K = \fH= (ak ) : k 2 f1; :::; pgg
and
L = \fH= (bj ) : j 2 f1; :::; qgg:
Then
K L=
k
= \fH= (a ) : k 2 f1; :::; pgg \fH= (bj ) : j 2 f1; :::; qgg =
= \fH= (cs ) : s 2 f1; :::; p + qgg;

where (
(as ; 0); s 2 f1; :::; pg
s
c =
(0; bs p ); s 2 fp + 1; :::; p + qg:
Consequently, K L is a polyhedral cone.

Theorem 1.5.37 Each polyhedral cone is a closed set.


Proof. First, we will show that, for each a 2 Rn ; the set
H= (a) = fx 2 Rn : hx; ai = 0g

is closed. In order to prove this, let xk k2N be a sequence of points


from H= (a); convergent to a point x 2 Rn . Since, for each k 2 N, we
have
xk ; a = 0;
we deduce that
hx; ai = h lim xk ; ai = lim hxk ; ai = 0:
k!1 k!1

Consequently, x 2 H= (a) and hence the set H= (a) is closed.


Let now K Rn be a polyhedral cone; then there exist an integer
number q = 1 and q points a1 ; :::; aq 2 Rn such that
K = \fH= (aj ) : j 2 f1; :::; qgg:

Since, for each j 2 f1; :::; qg; the set H= (aj ) is closed, we deduce that
the set K is closed.
38 1. CONVEX ANALYSIS

6. The Dual (Polar) of a Set


In this section, we show how to associate to each set in Rn another
set called its dual (or polar). The study of the relationship between
a set and its dual proves to be a fruitful concept in several areas of
convexity.

Denition 1.6.1 Let K be a nonempty subset of Rn : The set


K = fu 2 Rn : hu; xi = 0; for all x 2 Kg

is called the dual (or polar) of the set K:

Remark 1.6.2 The dual K of the nonempty subset K of Rn always


contains the point 0 2 Rn ; hence K 6= ;:

Example 1.6.3 The dual of the set f0g is the entire space Rn ; i.e
f0g = Rn :

Example 1.6.4 The dual of the set Rn is the set f0g; i.e.
(Rn ) = f0g:

Example 1.6.5 The dual of the positive orthant of Rn is the positive


orthant of Rn , i.e.
Rn+ = Rn+ :

Example 1.6.6 If K is the Euclidian norm cone, i.e.


K = f(x; t) 2 Rn+1 : kxk2 5 tg;
then
(K) = K:

Example 1.6.7 If K is the Minkowski norm cone, i.e.


K = f(x; t) 2 Rn+1 : kxk1 5 tg;
then its dual is the Tchebishev norm cone, i.e.
f(x; t) 2 Rn+1 : kxk1 5 tg = f(x; t) 2 Rn+1 : kxk1 5 tg:

Example 1.6.8 If a 2 Rn n f0g and


H(a) = fx 2 Rn : ha; xi = 0g;

H= (a) = fx 2 Rn : ha; xi = 0g;


6. THE DUAL OF A SET 39

H> (a) = fx 2 Rn : ha; xi > 0g;


then
(H(a)) = fu 2 Rn : there exists s 2 R such that u = sag = Rfag;

H= (a) = fu 2 Rn : there exists s 2 R+ such that u = sag = R+ fag;

(H> (a)) = H= (a) :

Example 1.6.9 If r 2 ]0; +1[; then


(B (0; r)) = (B[0; r]) = f0g:

Example 1.6.10 If K is a subspace of Rn ; then


K = K ? = fu 2 Rn : hu; xi = 0; for all x 2 Kg:

Theorem 1.6.11 Let K be a nonempty subset of Rn : Then:


10 K ? K :
20 ( K) = (K ) :
0
3 If K L; then L K :
Proof. 10 Obviously,
K ? = fu 2 Rn : hu; xi = 0; for all x 2 Kg
fu 2 Rn : hu; xi = 0; for all x 2 Kg = K :
20 We have
( K) = fu 2 Rn : hu; xi = 0; for all x 2 Kg =
= fu 2 R : hu; vi = 0; for all v 2 Kg =
n

= fu 2 Rn : h u; vi = 0, for all v 2 Kg =
= f w 2 Rn : hw; vi = 0, for all v 2 Kg =
= fw 2 Rn : hw; vi = 0; for all v 2 Kg =
= (K ) :
30 Let u 2 L ; then
hu; xi = 0; for all x 2 L:
From K L we deduce that
hu; xi = 0; for all x 2 K:
Consequently, u 2 K :
40 1. CONVEX ANALYSIS

Theorem 1.6.12 If K; a subset of Rn ; and L; a subset of Rm ; are


nonempty, then
(K L) = K L:
Proof. We have
(K L) = f(u; v) 2 Rn Rm : h(u; v); (x; y)i = 0;
for all (x; y) 2 K Lg =

= f(u; v) 2 Rn Rm : hu; xi + hv; yi = 0;


for all x 2 K and y 2 Lg =

= fu 2 Rn : hu; xi = 0; for all x 2 Kg


fv 2 Rm : hv; yi = 0; for all y 2 Lg =
=K L;

because 0 2 K \ L :

Theorem 1.6.13 If K is a nonempty subset of Rn , then the set K


is a closed convex cone.
Proof. Let u 2 K and t 2]0; +1[: Then, for each x 2 K; we
have
htu; xi = t hu; xi = 0;

hence tu 2 K : Consequently, K is a cone.


Let now u; v 2 K and t 2]0; 1[: Then, for each x 2 K; we have
h(1 t) u + tv; xi = (1 t) hu; xi + t hv; xi = 0;

because
hu; xi = 0; hv; xi = 0

and t 2]0; 1[: Consequently, (1 t) u + tv 2 K ; hence K is a convex


set.
To nish the proof it actually remains to show that K is a closed
set. Let u 2clK ; then there exists a sequence uk k2N of points uk
from K ; (k 2 N) convergent to u: From uk 2 K ; (k 2 N) it follows
that
hu; xi = h lim uk ; xi = lim uk ; x = 0:
k!1 k!1

Hence, u 2 K : Consequently, K is a closed set.


6. THE DUAL OF A SET 41

The dual K of a nonempty subset K of Rn is a nonempty subset


of Rn : This suggests that we should examine the double dual K of
an arbitrary subset K of Rn and see how it is related to K: Since the
dual of each nonempty subset of Rn is always a closed convex cone, a
necessary condition for the equality of the sets K and K is that K
shoud be a closed convex cone. We now show that this condition is
also su cient and establish the exact relationship between K and K:
We dene
K = (K ) :
We can also dene the dual K of the K ; as
K = (K ) :

The following statement holds.

Theorem 1.6.14 Let K be a nonempty subset of Rn : Then


10 The inclusion K K holds:
20 The equality K = K holds :
30 The equality K = (clK) holds:
Proof. 10 Let x 2 K: Then
hu; xi = hx; ui = 0; for all u 2 K ;
hence
x 2 (K ) = K :
Consequently, K K :
20 From statement 10 we obtain that K K : According to state-
ment 30 of Theorem 11, we deduce that
K K :

The converse inclusion


K (K ) =K

follows from statement 10 :


30 From K clK and statement 30 of Theorem 11; we deduce that
(clK) K :

In order to establish the converse inclusion, let x 2clK: Then, there


exists a sequence xk k2N of points xk 2 K; (k 2 N) such that
lim xk = x:
k!1
42 1. CONVEX ANALYSIS

Since
hu; xi = hu; lim xk i = lim hu; xk i = 0;
k!1 k!1

because
hxk ; ui = 0; for all k 2 N;
we obtain that u 2 (clK) : Consequently,
K (clK) :

The proof is complete.

Theorem 1.6.15 Let K and L be two nonempty subsets of Rn : Then:


10 The equality (K [ L) = K \ L holds:
20 If K \ L 6= ;; then K [ L (K \ L) :
30 The inclusion K \ L (K + L) holds:
40 If 0 2 K \ L; then (K + L) = K \ L :
50 If K \ L 6= ;; then the inclusion K + L (K \ L) holds.
Proof. 10 From K K [ L; in view of Theorem 11, we deduce
that
(K [ L) K :
In a similar way, we obtain that
(K [ L) L:
Consequently,
(K [ L) K \L :
Let now u 2 K \ L and x 2 K [ L: If x 2 K; then hu; xi = 0;
because u 2 K and if x 2 L; then also hu; xi = 0; because u 2 L :
Consequently, u 2 (K [ L) and the equality is established.
20 From K \ L K; according to Theorem 11, we deduce that
K (K \ L) :
Analogously, we obtain
L (K \ L) :
Consequently,
K [L (K \ L) :
30 Let u 2 K \ L : We will show that
hu; xi = 0; for all x 2 K + L:
6. THE DUAL OF A SET 43

Let x 2 K + L; then there exist s 2 K and t 2 L such that x = s + t.


From u 2 K \ L ; s 2 K; t 2 L; we deduce that hu; si = 0 and
hu; ti = 0: It follows that
hu; xi = hu; si + hu; ti = 0:

Consequently, u 2 (K + L) :
40 If 0 2 K; then
L = f0g + L K +L
and hence
(K + L) L:
Similarly,
(K + L) K :
Consequently, if 0 2 K \ L; then
(K + L) K \L :

Now, by statement 30 , we get statement 40 :


50 Let u 2 K + L ; then there exist s 2 K and t 2 L such that
u = s + t: From this we deduce that for each x 2 K \ L;
hu; xi = hs; xi + ht; xi = 0;
because
hs; xi = 0 and ht; xi = 0:
Consequently, u 2 (K \ L) and the proof is complete.

Remark 1.6.16 In statement 50 of Theorem 15, the equality might


not hold.

Theorem 1.6.17 (bipolar theorem) The nonempty subset C of Rn is


a closed convex cone if and only if the equality
K=K
holds:
Proof. Necessity. Assume that the set K is a closed convex cone.
In view of statement 10 of Theorem 14, it su ces to show that K K:
Suppose by contradiction that there exists a point x 2 K such that
x2= K: Then according to Theorem 29, there exists a point a 2 K nf0g
such that
ha; xi < 0:
Consequently, x 2
= K , but this is impossible.
44 1. CONVEX ANALYSIS

Su ciency. If K = K ; then in view of Theorem 13, the set K is


a closed convex cone.

Theorem 1.6.18 If the nonempy subset K of Rn is a convex cone,


then
clK = K :

Proof. The set clK being a closed convex cone, in view of Theo-
rem 17, we have that
clK = (clK) :
Since
K = (clK) ;
using statement 30 of Theorem 14, the proof is complete.

Theorem 1.6.19 Let K and L be two closed convex cones in Rn :


Then
(K \ L) = cl (K + L ) :

Proof. The sets K and L being closed convex cones, from Theo-
rem 17, it follows that
K=K and L = L :
Since
02K \L ;
according to Theorems 14, 15, and 17, we have
(K \ L) = (K \ L ) = (K + L ) = cl (K + L ) :

Theorem 1.6.20 Let K be a cone in Rn ; and u 2 Rn be such that the


function f : K ! R dened by
f (x) = hu; xi; for all x 2 K;
is bounded from below. Then u 2 K :
Proof. Assume that u 2 = K : Then there exists x 2 K such that
hu; xi < 0: Since K is a cone, the point tx 2 K; for all t 2]0; +1[: But
then f (tx) = hu; txi = thu; xi tends to 1 when t tends to +1; hence
f is not bounded from below, which contradicts our assumption.
6. THE DUAL OF A SET 45

Dual cones to pre-images of convex cones under linear transforma-


tions can be calculated in an explicit form.

Theorem 1.6.21 Let L be a closed convex cone in Rm ; A be an m n


matrix, and
K = fx 2 Rn : A (x) 2 Lg:
Then K is a closed convex cone, and
K = fy 2 Rn : there exists v 2 L such that y = AT (v)g =
= fAT (v) : v 2 L g:
Proof. The set K is a closed convex cone by construction.
Let
N = fy 2 Rn : there exists v 2 L such that y = AT (v)g =
= fAT (v) : v 2 L g::
Obviously, the set N is closed and convex.
Now, if y 2 N; then there exists v 2 L such that y = AT (v) and
hence, for each x 2 K; we have
hy; xi = AT (v) ; x = hv; A (x)i = 0;
because A (x) 2 L: It follows that
N K :
In order to prove that K N; we assume, by contradiction that
there exists a point u 2 K n N: By Theorem ???, there exist a point
a 2 Rn n f0g and a real number " > 0 such that
ha; ui + " 5 ha; xi ; for all x 2 N;
which implies
(1.6.1) ha; ui + " 5 a; AT (v) ; for all v 2 L ;
It follows that the function f : L ! R dened by
f (v) = a; AT (v) = hA (a) ; vi ; for all v 2 L ;
is bounded from below. Then, by Theorem 20, we have A (a) 2 (L ) =
L : Since L is a closed convex cone, bipolar theorem (Theorem 17)
implies A (a) 2 L and therefore a 2 K: But u 2 K ; and hence ha; ui =
0:
On the other hand, from (1.6.1); for v = 0 2 L we obtain
ha; ui + " 5 a; AT (0) = 0;
which contradicts ha; ui = 0:
46 1. CONVEX ANALYSIS

Theorem 21 allows us to explicitly calculate dual cones of cones


dened by system of linear inequalities.
For example, setting
L = fy 2 Rm : y = 0g = Rm
+

we obtain the following statement.

Theorem 1.6.22 (Farkas theorem) Let A be an m n matrix, and


K = fx 2 Rn : A (x) = 0g:
Then
K = fy 2 Rn : there exists v 2 Rm T
+ such that y = A (v)g:

The above fact is frequently formulated as an alternative: exactly


one of the following two systems has a solution, either
A (x) = 0
(i)
hc; xi < 0
or
AT (v) = c
(ii)
v = 0:
Indeed, system (i) is solvable if and only if c 2 K ; which is equiv-
alent to (ii) ; by Farkas theorem.

Theorem 1.6.23 Let K be a convex cone in Rn : If x 2 intK; then


hu; xi > 0; for all u 2 K n f0g:

Proof. Since x 2 K; we have hu; xi = 0; for all u 2 K n f0g:


Assume that there exists u 2 K n f0g such that hu; xi = 0: From x 2
intK; we deduce that there exists r 2]0; +1[ such that v = x ru 2 K:
Then
hv; ui = hx; ui r kuk2 = r kuk2 < 0;
a contradiction with u 2 K n f0g:

Theorem 1.6.24 Let K1 ; K2 ; :::; Kp be p convex cones in Rn : If


K1 \ K2 \ ::: \ Kp = ;;
then there exist u1 2 K1 ; u2 2 K2 ; :::; up 2 Kp ; not all equal 0; such
that
u1 + u2 + ::: + up = 0:
6. THE DUAL OF A SET 47

Proof. Let us dene two cones in Rn Rn ::: Rn = Rpn :


L = fv = v 1 ; :::; v p 2 Rpn : v 1 2 K1 ; :::; v p 2 Kp g = K1 ::: Kp ;
K = fw = (x; x; :::; x) 2 Rpn : x 2 Rn g:
Since K1 \ K2 \ ::: \ Kp = ;; we deduce that L \ K = ;: Then, by
Theorem ??? there exists u = (u1 ; :::; up ) 2 Rpn n f0g such that
hu; wi 5 hu; vi ;
for all v = (v 1 ; :::; v p ) 2 L and w = (x; x; :::; x) 2 K: From this it
follows that
(1.6.2) u1 + :::up ; x 5 u1 ; v 1 + ::: + hup ; v p i ;
for all x 2 Rn , and all v 1 2 K1 ; :::; v p 2 Kp :
Setting x = 0 in (1.6.2) ; we obtain that
u1 ; v 1 + ::: + hup ; v p i = 0; for all v 1 2 K1 ; :::; v p 2 Kp ;
hence the function f : K1 ::: Kp ! R dened by
f v 1 ; :::; v p = u1 ; v 1 + ::: + hup ; v p i ; for all v 1 2 K1 ; :::; v p 2 Kp ;
is bounded from below, which implies that each of the functions f1 :
K1 ! R, ..., fp : Kp ! R dened by
fi v i = ui ; v i ; for all v i 2 Ki ;
for i 2 f1; :::; pg; is bounded from below. Then, by Theorem 20, it
follows that u1 2 K1 ; :::; up 2 Kp :
On the other hand, (1.6.2) implies that the function g : Rn ! R
dened by
g (x) = u1 + ::: + up ; x ; for all x 2 Rn ;
is bounded from above, which is possible only when u1 + :::up = 0: The
theorem is proved.

Theorem 1.6.25 Let K1 ; K2 ; :::; Kp be p convex cones in Rn : If


(1.6.3) K1 \ (intK2 ) \ (intK3 ) \ ::: \ (intKp ) 6= ;;
then
(K1 \ K2 \ ::: \ Kp ) = K1 + K2 + ::: + Kp :
Proof. Let
K = K1 \ K2 \ ::: \ Kp :
If u 2 K1 ; u 2 K2 ; :::; up 2 Kp ; then, for each x 2 K we have
1 2

u1 + u2 + ::: + up ; x = u1 ; x + u2 ; x + ::: + hup ; xi = 0;


48 1. CONVEX ANALYSIS

so u1 + u2 + ::: + up 2 K : Consequently
(1.6.4) K1 + K2 + ::: + Kp K :
In order to prove the inverse inclusion of (1.6.4) ; let u 2 K :
Let us dene the cone
L = fx 2 Rn : hu; xi < 0g:
We have
L = fs 2 Rn : there exists t 2 [0; +1[ such that s = tug:
(see Example 8):
Since hu; xi = 0; for all x 2 L; it follows that L \ K = ;; and hence
L \ K1 \ K2 \ ::: \ Kp = ;: Then, by Theorem 24, there exist the points
s 2 L ; u1 2 K1 ; u2 2 K2 ; :::; up 2 Kp ; not all equal zero, such that
(1.6.5) s + u1 + u2 + ::: + up = 0:
From s 2 L it follows that there exists a real number t = 0 such that
s = tu: Then (1.6.5) becomes
(1.6.6) tu = u1 + u2 + ::: + up :
We will show that t > 0: Assume that t = 0; it follows that
(1.6.7) u1 + u2 + ::: + up = 0:
The hypothesis (1.6.3) ; implies there exists a point
x 2 K1 \ (intK2 ) \ (intK3 ) \ ::: \ (intKp ) :
Then
(1.6.8) 0 = u1 + u2 + ::: + up ; x = u1 ; x + u2 ; x + ::: + hup ; xi :
Since hu1 ; xi = 0; hu2 ; xi = 0; :::; hup ; xi = 0; from (1.6.8) ; we deduce
that
ui ; x = 0; for all i 2 f1; :::; pg:
But, x 2 (intK2 )\(intK3 )\:::\(intKp ) ; and then Theorem 23, implies
ui = 0; for all i 2 f1; :::; pg: Now, Equality (1.6.7) yiels u1 = 0; and
hence u1 = u2 = ::: = up = 0; a contradiction.
Thus t > 0: Dividing both sides of (1.6.6) by t 6= 0; we get
1 1 1
u = u1 + u2 + ::: + up 2 K1 + K2 + ::: + Kp :
t t t
Consequently K K1 + K2 + ::: + Kp : Theorem is proved.
6. THE DUAL OF A SET 49

Remark 1.6.26 When all cones are polyhedral, the regularity as-
sumption (1.6.3) is not needed.

Theorem 1.6.27 Let L Rn and M Rm be two closed convex


cones, A be an m n matrix, and
K = fx 2 L : A (x) 2 M g:
If
(1.6.9) 0 2 intfA (x) y : x 2 L; y 2 M g;
then
K = L + fAT (v) : v 2 M g:
Proof. Let us dene the set
N = fx 2 Rn : A (x) 2 M g:
Then, by Theorem 21, we have
N = fAT (v) : v 2 M g =
= fy 2 Rn : there exists vM such that y = AT (v)g:
Obviously
K = L \ N;
and hence
L +N (L \ N ) = K :
In order to prove L + N K ; let u 2 K :
Let us dene the cone
C = fx 2 Rn : hu; xi < 0g:
Then
C = fs 2 Rn : there exists t 2 [0; +1[ such that s = tug
(see Example 8):
On the other hand C \ K = ;; because hu; xi = 0; for all x 2 K:
Then, by Theorem 24, there exist s 2 C ; w 2 L ; z 2 N ; not all
equal 0; such that
(1.6.10) s + w + z = 0:
From z 2 N , it follows that there exists u 2 M so that z = AT (u) ;
and from s 2 C ; it follows that there exists t 2 [0; +1[ such that
s = tu:
We will show that t > 0:
50 1. CONVEX ANALYSIS

Assume that t = 0: Then (1.6.10) yields w = z: It follows that


AT (v) 2 L : Thus, for each x 2 L we obtain
x; AT (v) = 0;
that is
x; AT (v) 5 0:
As v 2 M ; we also have hy; vi = 0; for all y 2 M: Hence
hA (x) y; vi = hA (x) ; vi hy; vi = x; AT (v) hy; vi 5 0;
for all x 2 L and y 2 M: Since the set
fA (x) y : x 2 L; y 2 M g
containts a neighborhood of 0 by assumption, we get v = 0: This yields
z = 0 and w = 0 which is a contradiction.
Consequently t > 0: Dividind (1.6.10) by t 6= 0 we obtain
1 1 1 1
u = w + AT (v) 2 L + N = L + N :
t t t t
Thus K L + M : The theorem is proved.

Remark 1.6.28 When all cones are polyhedral, the regularity as-
sumption (1.6.9) is not needed.

7. Convex Functions
The convex functions and the generalizations of convex functions
play an important role in the study of optimization problems.
In this section we present some of the basic properties of con-
vex, concave, pseudoconvex, pseudoconcave, quasiconvex, quasiconcave
functions and obtain some fundamental theorems involving these func-
tions. These fundamental theorems will be used to derive the important
necessary optimality conditions of the optimization problem.

Denition 1.7.1 Let M be a nonempty subset of Rn ; x0 be a point


from M , K be a subset of Rm and f : M ! Rm be a function.
We say that the function f is convex at the point x0 with respect
to K if, for each x 2 M n fx0 g and each t 2]0; 1[ with the property that
(1 t) x0 + tx 2 M;
we have
(1 t) f x0 + tf (x) f (1 t)x0 + tx 2 K:

We say that the function f is concave at the point x0 with respect


to K if the function ( f ) is convex at the point x0 with respect to K:
7. CONVEX FUNCTIONS 51

Denition 1.7.2 Let M be a nonempty convex subset of Rn ; K be a


nonempty subset of Rm and f : M ! Rm be a function.
We say that the function f is convex on M with respect to K if the
function f is convex at each x0 2 M with respect to K:
We say that the function f is concave on M with respect to K if
the function ( f ) is convex on M with respect to K:

Example 1.7.3 Let A 2 Rm n and b 2 Rm : The function f : Rn !


Rm dened by
f (x) = A(x) + b, for all x 2 Rn ;
is convex on Rn with respect to K = f0g:
Indeed, for each x0 ; x 2 Rn and each t 2]0; 1[; we have
(1 t) f x0 + tf (x) f (1 t)x0 + tx = 0:

Example 1.7.4 The function f : Rn ! R dened by


f (x) = kxk ; for all x 2 Rn ;
is convex on Rn with respect to R+ :
Indeed, for each x0 ; x 2 Rn and each t 2]0; 1[; we have
(1 t) x0 + t kxk (1 t) x0 + tx =
= (1 t) x0 + t kxk (1 t) x0 t kxk = 0;
it follows that
(1 t) f x0 + tf (x) f (1 t)x0 + tx 2 R+ :

Therefore, the function f is convex on Rn with respect to R+ :


Many close connections exist between convex functions and convex
sets. Here we study one of these; others will be discussed later.

Example 1.7.5 If M is a nonempty convex subset of Rn ; then the


distance function dM : Rn ! R to M dened by
dM (x) = inffkx uk : u 2 M g; for all x 2 Rn

is convex on Rn with respect to R+ :


Indeed, let x0 ; x 2 Rn and t 2]0; 1[: Then, for each " > 0 there exist
two points u0 ; u 2 M such that
x0 u0 < dM (x0 ) + "
and
kx uk < dM (x) + ":
52 1. CONVEX ANALYSIS

Since M is convex, we have


(1 t)u0 + tu 2 M
and so
dM ((1 t)x0 + tx) 5 (1 t)x0 + tx (1 t)u0 tu
5 (1 t) x0 u0 + t kx uk
5 (1 t)(dM (x0 ) + ") + t(dM (x) + ")
= (1 t)dM (x0 ) + tdM (x) + ":
But " > 0 is arbitrary, whence
dM ((1 t)x0 + tx) 5 (1 t)dM (x0 ) + tdM (x):
Consequently,
(1 t)dM (x0 ) + tdM (x) dM ((1 t)x0 + tx) 2 R+ ;
which shows that dM is a convex function on Rn with respect to K =
R+ .
Remark 1.7.6 Let M be a nonempty subset of Rn ; x0 2 M and
f : M ! Rm be a function. Let K be a nonempty subset of Rm and
L K:
If the function f is convex (respectively concave) at x0 with respect
to K; then the function f is convex (respectively concave) at x0 with
respect to L:
The following four theorems give some other connections that exist
between the convex functions and convex sets.
Theorem 1.7.7 Let M be a nonempty convex subset of Rn ; K be a
convex cone in Rm and f : M ! Rm be a function.
10 If the function f is convex on M with respect to K, then the set
epi (f ) = f(x; u) 2 M Rm : u f (x) 2 Kg
is convex.
20 If the function f is concave on M with respect to K, then the set
hyp (f ) = f(x; u) 2 M Rm : u + f (x) 2 Kg
is convex.
Proof. 10 Let (x0 ; u0 ) ; (x; u) 2 epi (f ) and t 2]0; 1[: Then
u0 f x0 2 K and u f (x) 2 K:
Since M is a convex set we have
(1 t) x0 + tx 2 M:
7. CONVEX FUNCTIONS 53

Furthermore, the function f being convex on M with respect to K;


we have
(1 t) u0 + tu f ((1 t)x0 + tx) =
= (1 t) [u0 f (x0 )] + t [u f (x)] +
+ (1 t) f (x0 ) + tf (x) f ((1 t) x0 + tx) 2
2 (1 t) K + tK + K = K;

hence
(1 t) x0 ; u0 + t (x; u) = (1 t) x0 + tx; (1 t) u0 + tu 2 epi (f ) :

Consequently, the set epi (f ) is convex.


Analogously one proves the statement 20 :

Theorem 1.7.8 Let M be a nonempty subset of Rn ; K be a subset of


Rm with the property that 0 2 K and f : M ! Rm be a function.
10 If the set
epi (f ) = f(x; u) 2 M Rm : u f (x) 2 Kg

is convex, then the set M is convex and the function f is convex on M


with respect to K:
20 If the set
hyp (f ) = f(x; u) 2 M Rm : u + f (x) 2 Kg

is convex, then the set M is convex and the function f is concave on


M with respect to K:
Proof. 10 Assume that the set epi (f ) is convex and let x0 ; x 2 M;
x0 6= x and t 2]0; 1[: Since 0 2 K we have (x0 ; f (x0 )) and (x; f (x))
belong to epi (f ). Then
((1 t) x0 + tx; (1 t) f (x0 ) + tf (x)) =
= (1 t) (x0 ; f (x0 )) + t (x; f (x)) 2 epi (f ) ;

because epi (f ) is convex. From this it follows that


(1 t) x0 + tx 2 M
and
(1 t) f x0 + tf (x) f (1 t)x0 + tx 2 K;
hence the set M is convex and the function f is convex on M with
respect to K:
Analogously one proves the statement 20 :
54 1. CONVEX ANALYSIS

Theorem 1.7.9 Let M be a nonempty convex subset of Rn ; K be a


convex cone in Rm with property that 0 2 K and f : M ! Rm be a
function.
10 The function f is convex on M with respect to K if and only if
the set
epi (f ) = f(x; u) 2 M Rm : u f (x) 2 Kg
is convex.
20 The function f is concave on M with respect to K if and only if
the set
hyp (f ) = f(x; u) 2 M Rm : u + f (x) 2 Kg
is convex.

Proof. Apply Theorems 7 and 8.

Theorem 1.7.10 Let M be a nonempty convex subset of Rn ; K be a


convex cone in Rm and f : M ! Rm be a function. Then the following
statements are true:
10 If the function f is convex on M with respect to K; then, for
each u 2 Rm ; the set

Au = fx 2 M : u f (x) 2 Kg

is convex.
20 If the function f is concave on M with respect to K; then, for
each u 2 Rm ; the set

Bu = fx 2 M : u + f (x) 2 Kg

is convex.

Proof. 10 Let u 2 Rm : Let x0 ; x 2 Au and t 2]0; 1[: The set M


being convex it follows that

(1 t) x0 + tx 2 M:

If x0 = x, obviously

u f (1 t) x0 + tx = u f (x0 ) 2 K:
7. CONVEX FUNCTIONS 55

Let x0 6= x: The function f being convex on M with respect to K; and


K being a convex cone, we have
u f ((1 t) x0 + tx) =
= (1 t) [u f (x0 )] + t[u f (x)]+
0
+ (1 t) f (x ) + tf (x) f ((1 t) x0 + tx) 2
2 (1 t) K + tK + K =
= K + K = K;

hence
(1 t) x + tv 2 Au :

Consequently, the set Au is convex.


Analogously we prove the statement for the set Bu :

Below we examine how convex (respectively concave) functions be-


have with respect to the operations of addition and scalar multiplica-
tion.

Theorem 1.7.11 Let M be a nonempty subset of Rn ; x0 2 M; K


be a convex cone in Rm and f : M ! Rm and g : M ! Rm be two
functions.
10 If the functions f and g are convex at x0 with respect to K; then
for each a; b 2 [0; +1[ with a + b > 0; the function af + bg is convex
at x0 with respect to K:
20 If the functions f and g are concave at x0 with respect to K; then
for each a; b 2 [0; +1[ with a + b > 0; the function af + bg is concave
at x0 with respect to K:

Proof. We prove the theorem only in the case when the functions
f and g are convex at x0 with respect to K: Analogously one proves it
when f and g are concave at x0 with respect to K:
Let x 2 M n fx0 g and t 2]0; 1[ for which
(1 t) x0 + tx 2 M:
Then
(1 t) f x0 + tf (x) f (1 t)x0 + tx 2 K
and
(1 t) g x0 + tg (x) g (1 t)x0 + tx 2 K:
56 1. CONVEX ANALYSIS

From this it follows that


(1 t) (af + bg) (x0 ) + t(af + bg) (x) (af + bg) ((1 t)x0 + tx) =
= t [(1 t) f (x0 ) + tf (x) f ((1 t)x0 + tx)] +
+ b [(1 t) g (x0 ) + tg (x) g ((1 t)x0 + tx)] 2
2 aK + bK = K;
the last equality holds in view of Theorem 21. Consequently, the func-
tion af + bg is convex at x0 with respect to K:
From Theorem 11, it follows.

Theorem 1.7.12 Let M be a nonempty convex subset of Rn ; K be


a convex cone in Rm and f : M ! Rm and g : M ! Rm be two
functions.
10 If the functions f and g are convex on M with respect to K, then
for each a; b 2 [0; +1[ with a + b > 0; the function af + bg is convex
on M with respect to K:
20 If the functions f and g are concave on M with respect to K,
then for each a; b 2 [0; +1[ with a + b > 0; the function af + bg is
concave on M with respect to K:
Proof. Apply Theorem 11.

Theorem 1.7.13 Let M be a nonempty subset of Rn ; x0 be an interior


point of M; K be a closed convex cone in Rm ; and f : M ! Rm be a
dierentiable function at x0 :
10 If the function f is convex at x0 with respect to K; then
(1.7.1) f (x) f (x0 ) rf (x0 ) (x x0 ) 2 K;
for all x 2 M:
20 If the function f is concave at x0 with respect to K; then
(1.7.2) f (x0 ) f (x) rf (x0 ) (x0 x) 2 K;
for all x 2 M:
Proof. 10 Obviously relation (1.7.1) holds for x = x0 : Let x 2
M n fx0 g: Since x0 is an interior point of M; there exists a real number
r 2 ]0; +1[ such that
B x0 ; r = fx 2 Rn : x x0 < rg M:

Then, for each t 2]0; 1[ for which t < r= kx x0 k ; we have


(1.7.3) w = (1 t) x0 + tx = x0 + t x x0 2 B x0 ; r M:
7. CONVEX FUNCTIONS 57

The function f is convex at x0 with respect to K; then, for each t 2]0; 1[;
t < r= kx x0 k ; we have
(1 t) f (x0 ) + tf (x) f (1 t) x0 + tx 2 K:

From this, it follows that


1 1
(1.7.4) f (x) f x0 f x0 + t(x x0 ) f x0 2 K = K;
t t
for all t 2]0; 1[; t < r= kx x0 k ; the last equality in (1.7.4) holds
because K is a convex cone.
Furthermore, the function f is dierentiable at x0 ; then there exists
1
lim ff x0 + t(x x0 ) f x0 g = rf (x0 ) (x x0 ):
t!0 t
Then, taking the limit of expression (1.7.4) and taking into account
the fact that K is a closed set, we deduce that
(1.7.5) f (x) f x0 rf (x0 ) (x x0 ) 2 K:

Statement 20 is established in a similar way.

Theorem 1.7.14 Let M be a nonempty open convex subset of Rn ;


K be a closed convex cone in Rm and f : M ! Rm be a dierentiable
function on M:
10 If the function f is convex on M with respect to K; then

f (x) f (x0 ) rf (x0 ) (x x0 ) 2 K;

for all x; x0 2 M:
20 If the function f is concave on M with respect to K; then

f (x0 ) f (x) rf (x0 ) (x0 x) 2 K;

for all x; x0 2 M:
Proof. Apply Theorem 13.

Theorem 1.7.15 Let M be a nonempty open convex subset of Rn ; K


be a nonempty convex subset of Rm and f : M ! Rm be a dierentiable
function on M:
10 If
f (x) f (x0 ) rf (x0 ) (x x0 ) 2 K;
58 1. CONVEX ANALYSIS

for all x; x0 2 M; x = 6 x0 ; then the function f is convex on M with


respect to K:
20 If
f (x0 ) f (x) rf (x0 ) (x0 x) 2 K;
for all x; x0 2 M; x 6= x0 ; then the function f is concave on M with
respect to K:
Proof. 10 Let x; x0 2 M; x 6= x0 and t 2]0; 1[: Since the set M is
convex, the point
u = (1 t) x0 + tx 2 M:
Moreover, x0 6= u 6= x: Then we have
f (x0 ) f (u) t [rf (u)] (x0 x) 2 K
and
f (x) f (u) + (1 t) [rf (u)] (x0 x) 2 K:
From this, we deduce that
(1 t) f (x0 ) + tf (x) f ((1 t) x0 + tx) =
= (1 t) ff (x0 ) f (u) t [rf (u)] (x0 x)+
+tff (x) f (u) + (1 t) [rf (u)] (x0 x) 2
2 (1 t) K + tK = K:

Consequently, the function f is convex on M with respect to K:


Statement 20 is established in a similar way.
From Theorems 14 and 15 follows:

Theorem 1.7.16 Let M be a nonempty open convex subset of Rn ;


K be a closed convex cone in Rm and f : M ! Rm be a dierentiable
function on M: Then the following statements are true:
10 The function f is convex on M with respect to K if and only if

f (x) f (x0 ) rf (x0 ) (x x0 ) 2 K;


for all x; x0 2 M:
20 The function f is concave on M with respect to K if and only if

f (x0 ) f (x) rf (x0 ) (x0 x) 2 K;


for all x; x0 2 M

Example 1.7.17 Let A 2 Rn n be a positive semidenite matrix and


b 2 Rn : Then the function f : Rn ! R dened by
8. STRICTLY CONVEX FUNCTIONS 59

1
f (x) = hA(x); xi + b(x); for all x 2 Rn
2
is convex on Rn with respect to K = R+ , because the function f is
dierentiable on Rn and
1
f (x) f (x0 ) rf (x0 ) x x0 = hA x x0 ; x x0 i = 0;
2
for all x; x0 2 Rn :
8. Strictly Convex Functions
If K Rm has the property that 0 2 K; then obviously the im-
plication from the denitions of the convex, concave functions are true
for x = x0 and t = 0 and t = 1: Consequently, in this case, it is not
necessary to require that x 6= x0 ; t 6= 0 and t 6= 1:
Denition 1.8.1 Let M be a nonempty subset of Rn ; x0 be a point
from M; K be a subset of Rm with nonempty interior and f : M ! Rm
be a function.
We say that the function f is strictly convex at x0 with respect to
K; if the function f is convex at x0 with respect to intK:
We say that the function f is strictly concave at x0 with respect to
K; if the function f is concave at x0 with respect to intK:
Denition 1.8.2 Let M be a nonempty convex subset of Cn ; K be a
subset of Rm with nonempty interior and f : M ! Rm be a function.
We say that the function f is strictly convex on M with respect
to K if the function f is strictly convex at each point x0 2 M with
respect to K:
We say that the function f is strictly concave on M with respect
to K if the function f is strictly concave at each point x0 2 M with
respect to K:
Remark 1.8.3 From Remark 6, it follows that if the function f :
M ! Rm is strictly convex (respectively strictly concave) at x0 2 M
with respect to K Rm ; then the function f is convex (respectively
concave) at x0 with respect to K:
Bellow we examine how strictly convex (respectively strictly concave)
functions behave with respect to the operations of addition and scalar
multiplication.
Theorem 1.8.4 Let M be a nonempty subset of Rn ; x0 be a point
from M; K be a convex cone in Rm with nonempty interior and a;
b 2 [0; +1[ with b > 0:
60 1. CONVEX ANALYSIS

10 If the function f : M ! Rm is convex at x0 with respect to K


and the function g : M ! Rm is strictly convex at x0 with respect to
K; then the function af + bg is strictly convex at x0 with respect to K:
20 If the function f : M ! Rm is concave at x0 with respect to K
and the function g : M ! Rm is strictly concave at x0 with respect to
K; then the function af + bg is strictly concave at x0 with respect to K:
Proof. The proof of this theorem is similar to the proof of Theo-
rem 11.

Theorem 1.8.5 Let M be a nonempty subset of Rn ; x0 be an interior


point of M; K be a closed convex cone of Rm with nonempty interior
and f : M ! Rm be a dierentiable function at x0 :
10 If the function f is strictly convex at x0 with respect to K; then
(1.8.1) f (x) f (x0 ) [rf (x0 )](x x0 ) 2 intK;

for all x 2 M n fx0 g:


20 If the function f is strictly concave at x0 with respect to K; then
f (x0 ) f (x) [rf (x0 )](x0 x) 2 intK;

for all x 2 M n fx0 g:


Proof. 10 Let x 2 M n fx0 g: Since x0 is an interior point of M;
there exists a real number t 2]0; 1[ such that
u = (1 t) x0 + tx = x0 + t x x0 2 M:

Evidently, x0 6= u 6= x: Since f is strictly convex at x0 with respect


to K; we have
(1 t)f (x0 ) + tf (x) f (u) 2 intK:

In view of Remark 3, the function f is convex at x0 with respect to K;


then according to Theorem 13
f (u) f (x0 ) [rf (x0 )](u x0 ) 2 K:

Now, we have
f (x) f (x0 ) [rf (x0 )](x x0 ) =
= 1t f(1 t) f (x0 ) + tf (x) f (u)g +
+ 1t ff (u) f (x0 ) [rz f (x0 )](u x0 )g 2
2 (1=t) intK + (1=t) K = intK + K = intK:

20 Statement 20 is established in a similar way.


8. STRICTLY CONVEX FUNCTIONS 61

From Theorem 5 follows the following statement.

Theorem 1.8.6 Let M be a nonempty open convex subset of Rn ; K be


a closed convex cone in Rm with nonempty interior and f : M ! Rm
be a dierentiable function on M:
10 If the function f is strictly convex on M with respect to K; then
f (x) f (x0 ) [rf (x0 )](x x0 ) 2 intK;

for all x; x0 2 M; x 6= x0 :
20 If the function f is strictly concave on M with respect to K; then
f (x0 ) f (x) [rf (x0 )](x0 x) 2 intK;

for all x; x0 2 M; x 6= x0 :

Theorem 1.8.7 Let M be a nonempty open convex subset of Rn ; K


be a convex subset of Rm with nonempty interior and f : M ! Rm be
a dierentiable function on M:
10 If
f (x) f (x0 ) [rf (x0 )](x x0 ) 2 intK;
for all x; x0 2 M; x 6= x0 ; then the function f is strictly convex on M
with respect to K:
20 If
f (x0 ) f (x) [rf (x0 )](x0 x) 2 intK;
for all x; x0 2 M; x 6= x0 ; then the function f is strictly concave on M
with respect to K:
Proof. 10 Let x; x0 2 M; x 6= x0 and t 2]0; 1[: Since the set M is
convex, the point
u = (1 t)x0 + tx 2 M:
Moreover x0 6= u 6= x: Then we have
f (x0 ) f (u) t[rf (u)](x0 x) 2 intK;
and
f (x) f (u) + (1 t)[rf (u)](x0 x) 2 intK:
From this, we obtain that
(1 t)f (x0 ) + tf (x) f ((1 t)x0 + tx) =
= (1 t)ff (x0 ) f (u) t[rz f (u)](x0 x)g+
+tff (x) f (u) + (1 t)[rf (u)](x0 x)g 2

2 (1 t)intK + tintK = intK:


62 1. CONVEX ANALYSIS

Consequently, the function f is strictly convex on M with respect to


K:
One proves statement 20 in a similar way.
From Theorems 6 and 7 follows.

Theorem 1.8.8 Let M be a nonempty open convex subset of Rn ; K be


a closed convex cone in Rm with nonempty interior and f : M ! Rm
be a dierentiable function on M:
10 The function f is strictly convex on M with respect to K if and
only if
f (x) f (x0 ) [rf (x0 )](x x0 ) 2 intK;
for all x; x0 2 M; x 6= x0 :
20 The function f is strictly concave on M with respect to K; if and
only if
f (x0 ) f (x) [rf (x0 )](x0 x) 2 intK;
for all x; x0 2 M; x 6= x0 :

9. Pseudoconvex Functions

Denition 1.9.1 Let M be a nonempty subset of Rn ; x0 be an interior


point of M; K be a nonempty subset of Rm and f : M ! Rm be a
dierentiable function at x0 :
We say that the function f is pseudoconvex at x0 with respect to
K if, for each x 2 M n fx0 g such that
rf (x0 ) x x0 2 K;
we have
f (x) f x0 2 K:

We say that the function f is pseudoconcave at x0 with respect to


K if the function ( f ) is pseudoconvex at x0 with respect to K:

Denition 1.9.2 Let M be a nonempty open subset of Rn ; K be a


nonempty subset of Rm and f : M ! Rm be a dierentiable function
on M:
We say that the function f is pseudoconvex on M with respect to
K if the function f is pseudoconvex at each point x0 2 M with respect
to K:
We say that the function f is pseudoconcave on M with respect to
K if the function ( f ) is pseudoconvex on M with respect to K:
9. PSEUDOCONVEX FUNCTIONS 63

We will relate now pseudoconvex (respectively pseudoconcave) func-


tions to convex (respectively concave) functions.

Theorem 1.9.3 Let M be a nonempty subset of Rn ; x0 be an interior


point of M; K be a closed convex cone in Rm and f : M ! Rm be a
dierentiable function at x0 :
10 If the function f is convex at x0 with respect to K; then the
function f is pseudoconvex at x0 with respect to K:
20 If the function f is concave at x0 with respect to K; then the
function f is pseudoconcave at x0 with respect to K:
Proof. We shall establish the theorem for the convex case. The
concave case follows from it.
10 By the convexity of the function f at x0 with respect to K; in
view of Theorem 13, we have
f (x) f x0 rf (x0 ) x x0 2 K;

for all x 2 M:
Let now x 2 M n fx0 g be such that
u = rf (x0 ) x x0 2 K:
Then
f (x) f x0 2 K + fug K + K = K:
Consequently, the function f is pseudoconvex at x0 with respect to
K:

Remark 1.9.4 The converse of Theorem 3 is not necessarily true as


it can be seen from the following example: the function f : R ! R
dened by
f (x) = exp x2 ; for all x 2 R
is pseudoconvex at x0 = 100 with respect to K = R+ ; but it is not
convex at x0 = 100 with respect to K:
From Theorem 3 follows immediatly the next statement:

Theorem 1.9.5 Let M be a nonempty open convex subset of Rn ; K


be a closed convex cone in Rm and f : M ! Rm be a dierentiable
function on M:
10 If the function f is convex on M with respect to K; then the
function f is pseudoconvex on M with respect to K:
20 If the function f is concave on M with respect to K; then the
function f is pseudoconcave on M with respect to K:
64 1. CONVEX ANALYSIS

Remark 1.9.6 If K Rm has the property that 0 2 K; then ob-


viously the implication from the denitions of the pseudoconvex and
pseudoconcave functions are true for x = x0 and t = 0 and t = 1:
Consequently, in this case, it is not necessary to require that x 6= x0 :

Denition 1.9.7 Let M be a nonempty subset of Rn ; x0 be an interior


point of M; K be a subset of Rm with nonempty interior and f : M !
Rm be a dierentiable function at x0 :
We say that the function f is strictly pseudoconvex (respectively
strictly pseudoconcave) at x0 with respect to K if the function f is
pseudoconvex (respectively pseudoconcave) at x0 with respect to int
K:

Denition 1.9.8 Let M be a nonempty open subset of Rn ; K be a


subset of Rm with nonempty interior and f : M ! Rm be a dieren-
tiable function on M:
We say that the function f is strictly pseudoconvex (respectively
strictly pseudoconcave) on M with respect to K if the function f is
strictly pseudoconvex (respectively strictly pseudoconcave) at each z 2
M with respect to K:

Theorem 1.9.9 Let M be a nonempty subset of Rn ; x0 be an interior


point of M; K be a closed convex cone in Rm with nonempty interior
and f : M ! Rm be a dierentiable function at x0 :
10 If the function f is convex at x0 with respect to K; then the
function f is strictly pseudoconvex at x0 with respect to K:
20 If the function f is concave at x0 with respect to K; then the
function f is strictly pseudoconcave at x0 with respect to K:
Proof. We shall establish the theorem for the convex case. The
concave case follows from it.
10 By the convexity of the function f at x0 with respect to K; in
view of Theorem 13, we have
f (x) f (x0 ) [rf (x0 )](x x0 ) 2 K;

for all x 2 M n fx0 g:


Let now x 2 M n fx0 g be such that
t = [rf (x0 )](x x0 ) 2 intK:

Then
f (x) f x0 2 ftg + intK K + intK = intK:
10. QUASICONVEX FUNCTIONS 65

Consequently, the function f is strictly pseudoconvex at x0 with


respect to K:
From Theorem 9 follows.

Theorem 1.9.10 Let M be a nonempty open convex subset of Rn ; K


be a closed convex cone in Rm with nonempty interior and f : M ! Rm
be a dierentiable function on M:
10 If the function f is convex on M with respect to K; then the
function f is strictly pseudoconvex on M with respect to K:
20 If the function f is concave on M with respect to K; then the
function f is strictly pseudoconcave on M with respect to K:

10. Quasiconvex Functions

Denition 1.10.1 Let M be a nonempty subset of Rn ; x0 be a point


from M; K be a nonempty subset of Rm and f : M ! Rm be a function.
We say that the function f is quasiconvex at x0 with respect to K;
if, for each x 2 M n fx0 g with the property that
f x0 f (x) 2 K
and, for each t 2]0; 1[ such that (1 t) x0 + tx 2 M; we have
f x0 f (1 t) x0 + tx 2 K:

We say that the function f is quasiconcave at x0 with respect to K;


if the function ( f ) is quasiconvex at x0 with respect to K:

Denition 1.10.2 Let M be a nonempty convex subset of Rn ; K be


a nonempty subset of Rm and f : M ! Rm be a function.
We say that the function f is quasiconvex on M with respect to K
if the function f is quasiconvex at each x0 2 M with respect to K:
We say that the function f is quasiconcave on M with respect to
K if the function ( f ) is quasiconvex on M with respect to K:

Example 1.10.3 The function f : R ! R dened by f (x) = x3 ; for


all x 2 R is qusiconvex on R with respect to R+ :
We will relate now quasiconvex (respectively quasiconcave) func-
tions to convex (respectively concave) functions.
The following statement is true.

Theorem 1.10.4 Let M be a nonempty subset of Rn ; x0 be a point


from M; K be a convex cone in Rm and f : M ! Rm be a function.
66 1. CONVEX ANALYSIS

10 If the function f is convex at x0 with respect to K; then the


function f is quasiconvex at x0 with respect to K:
20 If the function f is concave at x0 with respect to K; then the
function f is quasiconcave at x0 with respect to K:
Proof. 10 Let x 2 M n fx0 g for which
f x0 f (x) 2 K

and t 2]0; 1[ for which (1 t) x0 + tx 2 M: Then, by the convexity of


f at x0 with respect to K and by the fact that K is a convex cone, we
have
f x0 f (1 t) x0 + tx =

= (1 t) f (x0 ) + tf (x) f (1 t) x0 + tx + t[f x0 f (x)] 2

2 K + tK K + K = K:
The concave case is established in a similar way.

Remark 1.10.5 The converse of Theorem 4 is not true; there are


quasiconvex functions at x0 with respect to K which are not convex at
x0 with respect to K:
Indeed, the function f : R ! R dened by
f (x) = x + x3 ; for all x 2 R

is quasiconvex at x0 = 0 with respect to R+ ; but it is not convex at


x0 = 0 with respect to K = R+ :
From Theorem 4 follows immediately the next statement:

Theorem 1.10.6 Let M be a nonempty convex subset of Rn ; K be a


convex cone in Rm and f : M ! Rm be a function.
10 If the function f is convex on M with respect to K; then the
function f is quasiconvex on M with respect to K:
20 If the function f is concave on M with respect to K; then the
function f is quasiconcave on M with respect to K:

Theorem 1.10.7 Let M be a nonempty subset of Rn ; x0 be an interior


point of M; K be a closed convex cone in Rm and f : M ! Rm be a
dierentiable function at x0 :
10 If the function f is quasiconvex at x0 with respect to K; then,
for each x 2 M n fx0 g with the property that
f x0 f (x) 2 K;
10. QUASICONVEX FUNCTIONS 67

we have
rf (x0 ) x0 x 2 K:
20 If the function f is quasiconcave at x0 with respect to K; then,
for each z 2 M with the property that
f (x) f x0 2 K;
we have
rf (x0 ) x x0 2 K:

Proof. 10 Let x 2 M n fx0 g with property that


f x0 f (x) 2 K:
Since x0 is an interior point of M; there exists a real number r 2 ]0; +1[
such that
B x0 ; r := fx 2 Rn : x x0 < rg M:

Then, for each t 2]0; 1[ for which


t < r= x x0 ;
we have
(1.10.1) w = (1 t) x0 + tx = x0 + t x x0 2 B x0 ; r M:

The function f is quasiconvex at x0 with respect to K; then, for each


t 2]0; 1[; t < r= kx x0 k ; we have
f (x0 ) f (1 t) x0 + tx 2 K:

From this, it follows that


1 1
(1.10.2) f x0 + t(x x0 ) f x0 2 K = K;
t t
for all t 2]0; 1[; t < r= kx x0 k ; the last equality in (1.10.2) holds
because K is a convex cone.
Furthermore, the function f is dierentiable at x0 ; then there exists
1
lim ff x0 + t(x x0 ) f x0 g = rf (x0 ) (x x0 ):
t!0 t
Then, taking the limit of expression (1.10.2) and taking into account
the fact that K is a closed set, we deduce that
(1.10.3) rf (x0 ) (x x0 ) 2 K:

Statement 20 is established in a similar way.


68 1. CONVEX ANALYSIS

From this theorem follows immediately:

Theorem 1.10.8 Let M be a nonempty open convex subset of Rn ;


K be a closed convex cone in Rm and f : M ! Rm be a dierentiable
function on M:
10 If the function f is quasiconvex on M with respect to K; then,
for each x; x0 2 M with the property that
f x0 f (x) 2 K;
we have
rf (x0 ) x0 x 2 K:
20 If the function f is quasiconcave on M with respect to K; then,
for each x; x0 2 M with the property that
f (x) f x0 2 K;
we have
rf (x0 ) x x0 2 K:

Remark 1.10.9 If K Rm has the property that 0 2 K; then ob-


viously the implication from the denitions quasiconvex and quasicon-
cave functions are true for x = x0 and t = 0 and t = 1: Consequently, in
this case, it is not necessary to require that x 6= x0 ; t 6= 0 and t 6= 1:

Denition 1.10.10 Let M be a nonempty subset of Rn ; x0 be a point


from M; K be a subset of Rm with nonempty interior and f : M ! Rm
be a function.
We say that the function f is strictly quasiconvex (respectively
strictly quasiconcave) at x0 with respect to K; if the function f is
quasiconvex (respectively quasiconcave) at x0 with respect to intK:

Denition 1.10.11 Let M be a nonempty convex subset of Rn ; K be


a subset of Rm with nonempty interior and f : M ! Rm be a function.
We say that the function f is strictly quasiconvex (respectively
strictly quasiconcave) on M with respect to K; if the function f is
strictly quasiconvex (respectively strictly quasiconcave) at each point
x0 2 M with respect to K:

Theorem 1.10.12 Let M be a nonempty subset of Rn ; x0 be a point


from M; K be a convex cone in Rm with nonempty interior and f :
M ! Rm be a function.
10 If the function f is convex at x0 with respect to K; then the
function f is strictly quasiconvex at x0 with respect to K:
10. QUASICONVEX FUNCTIONS 69

20 If the function f is concave at x0 with respect to K; then the


function f is strictly quasiconcave at x0 with respect to K:
Proof. 10 Let x 2 M n fx0 g with property that
f x0 f (x) 2 intK

and t 2]0; 1[ such that (1 t) x0 + tx 2 M: By the convexity of the


function f at x0 with respect to K; we have
(1 t)f (x0 ) + tf (x) f ((1 t)x0 + tx) 2 K:
Then
f (x0 ) f ((1 t)x0 + tx) =
= [(1 t)f (x0 ) + tf (x) f ((1 t)x0 + tx)] + t [f (x0 ) f (x)] 2
2 K + tintK = intK:

Consequently, the function f is strictly quasiconvex at x0 with respect


to K:
Statement 20 is established in a similar way.
From Theorem 12 follows.
Theorem 1.10.13 Let M be a nonempty convex subset of Rn ; K be
a convex cone in Rm with nonempty interior and f : M ! Rm be a
function.
10 If the function f is convex on M with respect to K; then the
function f is strictly quasiconvex on M with respect to K:
20 If the function f is concave on M with respect to K; then the
function f is strictly quasiconcave on M with respect to K:
CHAPTER 2

Optimality Conditions

1. Optimization Problems
By a minimization problem we understand a problem of the follow-
ing type:
Being given:
a) a nonempty subset M of the space Rn ;
b) a nonempty subset K of the space Rm ;
c) the functions f : M ! R and g : M ! Rm ;
nd all (locally) minimum points of the function f on the set
X = fx 2 M : g(x) 2 Kg:

Formally, the minimization problem in complex space is written in


the literature as follows
(2.1.1) min f (x)
subject to
x 2 X;
or explicitly
min f (x)
subject to (
x 2 M;
g(x) 2 K:
The function f is called the criteria function, the set X is called
the set of the feasible solutions (or the feasible set) of Problem (2.1.1) ;
the relation g (x) 2 K is called the constraint of the problem and the
function g is called the constraint function.
Minimization problem (2.1.1) is said to be consistent if X 6= ; and
inconsistent if X = ;: All points x from the feasible set X are called
feasible points, or feasible solution of the minimization problem (2.1.1).
We say that the point x0 2 X is a solution of minimization problem
(2.1.1) if
f x0 = minff (x) : x 2 Xg.

71
72 2. OPTIMALITY CONDITIONS

We say that the point x0 2 X is a local solution of minimization


problem (2.1.1) if there exists a neigborhood V of the point x0 such
that
f x0 = minff (x) : x 2 X \ Vg:
We say that the minimization problem (2.1.1) does not have solu-
tions if
inf ff (x) : x 2 Xg > 1
0
and an x 2 X such that
inf ff (x) : x 2 Xg = f x0
does not exist.
Problem (2.1.1) does not have nite solutions if there exists a se-
quence xk k2N with xk 2 X; for all k 2 N, such that
lim f xk = 1:
k!1

Similarly, we can dene a maximization problemx. It is written in


the literature as follows
max f (x)
subject to
x 2 X;
or explicitly
max f (x)
subject to (
x 2 M;
g(x) 2 K:
Minimization problems and maximization problems are called op-
timization problems.
Since
max ff (x) : x 2 Xg = min f( f (x)) : x 2 Xg ;

we shall study from now on only problems of the form:


min f (x)
subject to
x 2 X:
According to the properties of the sets M and K and of the func-
tions f and g which are used in order to dene Problem (2.1.1) one
may distinguish between dierent classes of optimization problems.
2. DOREL I. DUCA 73

Two types of questions can appear when working with the opti-
mization problem (2.1.1) :
(i) determine necessary, respectively su cient conditions for a
point x0 2 Rn to be a solution or a local solution of Problem (2.1.1) ;
and
(ii) discover some numerical methods for determining a solution or
a local solution of Problem (2.1.1) :
Even though for question (i) above one can give both necessary and
su cient conditions for a point x0 2 Rn to be a solution of Problem
(2.1.1) ; question (ii) is far from being answered.

2. Some General Results of Optimization


During this section we shall consider the optimization problem:

(2.2.1) min f (x)


subject to
x 2 X;
where X is a nonempty subset of Rn and f : X ! R is a function.
We establish now a basic result concerning the set of solutions of
the optimization problem (2.2.1).

Theorem 2.2.1 Let X be a nonempty convex subset of Rn ; and f :


X ! R be a quasiconvex function on X with respect to R+ : Then the
set of the solutions of Problem (2.2.1) is convex.
Proof. Let x0 and x1 be two solutions of Problem (2.2.1) : It is
obvious that x0 ; x1 2 X and
f x0 = f x1 = min ff (x) : x 2 Xg

and therefore
(2.2.2) f x0 f x1 0:

Let now t 2 ]0; 1[ : Since X is a convex set, it follows that


xt = (1 t) x0 + tx1 2 X:

Because the function f is quasiconvex on X with respect to R+ ; from


(2.2.2) it follows that
f x0 f x t 2 R+ ;

or equivalently,
f x0 = f xt :
74 2. OPTIMALITY CONDITIONS

Hence, xt 2 X is also a solution of Problem (2.2.1) and thus the set


of the solutions of the same problem is convex.

Theorem 2.2.2 Let X be a nonempty convex subset of Rn , f : X ! R


be a function and x0 be a solution of Problem (2.2.1) : If f is strictly
convex at x0 with respect to R+ , then x0 is the unique solution of Prob-
lem (2.2.1) :
Proof. Let x1 6= x0 be another solution of Problem (2.2.1) : Then
x ; x1 2 X and
0

f z0 f z 1 = 0:

Let now t 2 ]0; 1[ : Since X is a convex set, it follows that


xt = (1 t) x0 + tx1 2 X:

Moreover, since f is strictly convex at z 0 with respect to R+ ; it follows


that
(1 t) f x0 + tf x1 f xt > 0

and therefore
f z t < (1 t) f x0 + tf x1 =
= f x0 t f x0 f x1 =
= f x0 ;
which is a contradiction to the fact that x0 is a solution of Problem
(2.2.1) : Thus, x1 cannot be dierent from x0 :

Theorem 2.2.3 If x0 is a solution of Problem (2.2.1); then x0 is a


local solution of Problem (2.2.1):
Proof. It is immediate.
The converse of Theorem 3 does not generally hold. The follow-
ing theorem gives a su cient condition for the converse statement of
Theorem 3 to hold.

Theorem 2.2.4 Let X be a nonempty convex subset of Rn and f :


X ! R be a convex function on X with respect to R+ : If x0 is a local
solution of Problem (2.2.1), then x0 is a solution of Problem (2.2.1) :
2. DOREL I. DUCA 75

Proof. Let x0 be a local solution of Problem (2.2.1) : Then there


exists r 2 R; r > 0 such that
f x0 5 f (x) ; for all x 2 X \ B x0 ; r :

Let us suppose that there exists a point x1 with the property that
(2.2.3) f x1 < f x0 :

It is obvious that x1 6= x0 : Since X is a convex set, it follows that


xt = (1 t) x0 + tx1 2 X; for all t 2 ]0; 1[ :

Let us choose t 2 ]0; 1[ such that


r
0<t< :
kx0 x1 k

Then
xt x0 = (1 t) x0 + tx1 x0 = t x0 x1 < r

and thus
xt 2 X \ B x0 ; r :

Therefore
(2.2.4) f xt = f x0 :

Moreover, since the function f is convex on X with respect to R+ ;


taking into account (2.2.3) ; it follows that
f xt 5 (1 t) f x0 + tf x1 <
< (1 t) f x0 + tf x0 =
= f x0 ;
which is a contradiction to (2.2.4) :

Theorem 2.2.5 Let X be a nonempty convex subset of Rn and f :


X ! R be a concave function on X with respect to R+ ; under the
assumption that it is not a constant. If x0 is a solution of Problem
(2.2.1) ; then x0 belongs to the boundary of X:
76 2. OPTIMALITY CONDITIONS

Proof. If intX = ;; then x0 is a point of the boundary of the set


X:
Let us consider the case when intX 6= ;: Because f is not a constant
it follows that there exists x1 2 X such that
f x0 < f x1 :

Let now z 2 intX; then there exists a real number r > 0 such that
B (x; r) X:

Let us denote
kx x1 k + 1
t= :
kx x1 k + 1 + r
It is obvious that x 2 ]0; 1[ :
Let
1 1
(2.2.5) w = x+ 1 x1 :
t t
Since
1 r
kx wk = 1 x x1 = x x1 < r;
t kx x1 k + 1
it follows that w 2 B (x; r) and thus w 2 X: We deduce then that
f (w) = f x0 :

From (2.2.5) it follows that


x = (1 t) x1 + tw

and because X is a convex set, x1 2 X; w 2 B (x; r) X and t 2 ]0; 1[


it follows that x 2 X:
Moreover, since the function f is concave on X with respect to R+ ;
we obtain that
f (x) = f (1 t) x1 + tw =
= (1 t) f x1 + tf (w) >
> (1 t) f x0 + tf x0 =
= f x0 :
Hence, if x 2 intX; then
f (x) > f x0 :

Thus, f cannot attain its minimum at an interior point of X:


2. DOREL I. DUCA 77

Theorem 2.2.6 Let X be a nonempty convex subset of Rn and f :


X ! R be a concave function on X with respect to R+ : Then the set of
the points x 2 X which are not solutions of Problem (2.2.1) is convex.
Proof. Let x1 ; u1 2 X be two points which are not solutions of
Problem (2.2.1) ; then there exists x0 2 X such that
f x0 < f x1 and f x0 < f u1 :

Let t 2 ]0; 1[ : Since X is a convex set, it holds


xt = (1 t) x1 + tu1 2 X:
Because the function f is concave on X with respect to R+ ; it follows
that
f (x ) = (1 t) f x1 + tf u1 >
> (1 t) f x0 + tf x0 = f x0 :
Consequently, the set of the points form X which are not solutions of
Problem (2.2.1) is convex.

Theorem 2.2.7 Let M be a subset of Rn ; X be a convex subset of


M; x0 2 intX and f : M ! R be a dierentiable function at x0 : A
necessary condition, for x0 to be a solution of Problem (2.2.1) which
is also su cient if the function f is pseudoconvex at x0 with respect to
R+ ; is
(2.2.6) hrf (x0 ); x x0 i = 0; for all x 2 X:

Proof.

Necessity. Let x 2 X: Let us consider the function g : [0; 1] ! R


dened by
g (t) = f x0 + t x x0 ; for all t 2 [0; 1] :
It is obvious that t = 0 is a minimum point of the function g; then
d
g (t) jt=0 = 0:
dt
Because
d
g (t) jt=0 = rf z 0 z z 0 + rz f z 0 z z0 ;
dt
78 2. OPTIMALITY CONDITIONS

it follows that (2.2.6) holds.


Proof. Su ciency. Inequality (2.2.6) may be written as
T
(2.2.7) rz f z 0 z z 0 + rz f z 0 z z 0 2 CR+ ;

for all z 2 X, where


CR+ = fv 2 C : Re v = 0g :

Let now z 2 X: Because f has pseudoconvex real part at z 0 with


respect to R+ ; from (2.2.7) ; it follows that
f (z) f z 0 2 CR+ ;

or equivalently,
Re f (z) = Re f z 0 :

Hence, z 0 is a solution of Problem (2.2.1) :

Corollary 2.2.8 Let X be a nonempty subset of Rn ; x0 be an interior


point of X and f : M ! R be a dierentiable function at xz 0 : A
necessary condition for x0 to be a solution of Problem (2.2.1) is that:
rf (x0 ) = 0:

Proof. From x0 2 intX, we deduce that there exists a real number


r > 0 such that
B(x0 ; r) X:

Moreover, by Theorem 7, we have that inequality (2.2.6) holds; then


hrf (x0 ); vi = 0; for all v 2 B(0; r):

Therefore
rf (x0 ) 2 (B(0; r)) :

and since
(B(0; r)) = f0g;

the corollary is proved.


3. NECESSARY AND SUFFICIENT OPTIMALITY CRITERIA 79

Corollary 2.2.9 Let M be nonempty open subset of Rn , X be a non-


empty convex subset of M; x0 2 X and f : M ! R be a dierentiable
function at x0 and pseudoconvex at z 0 with respect to R+ : If
rf (x0 ) = 0;

then x0 is a solution of Problem (2.2.1) :


Proof. It uses Theorem 7.

3. Necessary Optimality Criteria and Su cient Optimality


Criteria
The purpose of this section is to derive optimality criteria of the
saddle point type for optimization problem. We shall develop the opti-
mality criteria of this section without any dierentiability assumptions
imposed on the functions involved. In the next section will establish
optimality criteria that involve dierentiable functions.
Let M be a nonempty subset of Rn ; K be a nonempty subset of Rm
and f : M ! R and g : M ! Rm be two functions. We shall consider
the following optimization problem:
(2.3.1) min f (x)

subject to (
x2M
g(x) 2 K:
For each u 2 R we attach to Problem (2.3.1) the function Lu :
M Rm ! R dened by
Lu (x; v) := uf (x) hg (x) ; vi ; for all (x; v) 2 M Rm :

Under such circumstances, the following F. John saddle point theorem


holds.

Theorem 2.3.1 Let M be a nonempty convex subset of Rn ; K be a


polyhedral cone in Rm with nonempty interior, f : M ! R be a convex
function on M with respect to R+ and g : M ! Rm be a concave
function on M with respect to K.
If x0 is a solution of Problem (2.3.1), then there exist u0 2 R and
v 2 Rm such that:
0

(2.3.2) u0 2 R+ ; v 0 2 K ; (u0 ; v 0 ) 6= (0; 0);

(2.3.3) g(x0 ); v 0 = 0
80 2. OPTIMALITY CONDITIONS

and
(2.3.4) Lu0 (x0 ; v) 5 Lu0 (x0 ; v 0 ) 5 Lu0 (x; v 0 );

for all (x; v) 2 M K :


Proof. Since x0 is a solution of Problem (2.3.1) it follows that the
system
8 0
< f (x) f (x ) < 0
>
(2.3.5) g(x) 2 K
>
:
x2M

is inconsistent.
System (2.3.5) can be written as:
8 0
< f (x ) f (x) 2 intR+
>
(2.3.6) g(x) 2 K
>
:
x 2 M:

Taking into account Remark ?? and Corollary ??, it follows that there
exist u0 2 R and v 0 2 Rm such that
(2.3.7) u 0 2 R+ = R + ; v 0 2 K ; u0 ; v 0 6= (0; 0)

and
(2.3.8) f (x0 ) f (x); u0 + g(x); v 0 5 0; for all x 2 M:

Choosing x = x0 2 M in (2.3.8) we get that


g(x0 ); v 0 5 0:

Because v 0 2 K and g (x0 ) 2 K we obtain that


g(x0 ); v 0 = 0

and therefore
g(x0 ); v 0 = 0;
i.e. equality (2.3.3) :
Now, from (2.3.8), (2.3.3) and the fact that x0 is a solution of
Problem (2.3.1) ; it follows that
u0 f (x0 ) g(x0 ); v 0 5 u0 f (x) g(x); v 0 ; for all x 2 M;

which is exactly the second inequality in (2.3.4) :


3. NECESSARY AND SUFFICIENT OPTIMALITY CRITERIA 81

In order to show that the rst inequality in (2.3.4) is satised, let


us remark that from
g x0 2 K;
we obtain that
g(x0 ); v = 0, for all v 2 K
and therefore
u0 f (x0 ) g(x0 ); v 5 u0 f (x0 ) g(x0 ); v 0 ; for all v 2 K ;

because
g(x0 ); v 0 = 0:

Example 2.3.2 Let us consider the following optimization problem


in complex space:
(2.3.9) min f (x1 ; x2 ) := x21 + x22

subject to
(
(x1 ; x2 ) 2 M := f(x1 ; x2 ) 2 R2 : x1 = 0g
g (x1 ; x2 ) := x1 x2 1 2 K := R+ :

One can easily notice that the hypotheses of Theorem 1 are fullled
and it can easily be shown that x0 = (1; 1) is a solution of Problem
(2.3.9) ;
u 0 = 1 2 R+
and
v 0 = 0 2 K = R+
verify (2.3.2) (2.3.4) :

Example 2.3.3 Let us consider the following optimization problem:


(2.3.10) min x

subject to
x 2 M := R
g(x) := x2 2 K := R+ :
It can be easily seen that the solution of Problem (2.3.10) is x0 =
0 2 R and the hypotheses of Theorem 1 are satised. Then, there exist
u0 2 R and
v 0 2 K = R+ ;
82 2. OPTIMALITY CONDITIONS

such that (2.3.2) (2.3.4) hold.


Let us suppose that u0 is dierent from 0: Then, from the second
inequality in (2.3.4) it follows that, for each x 2 M; it holds
(2.3.11) 0 5 u0 x + x 2 v 0 :

If v 0 were 0; then inequality (2.3.11) would become


0 5 u0 x; for all x 2 R;

which obviously, holds if and only if u0 = 0, which is a contradiction to


the fact that (u0 ; v 0 ) 6= (0; 0) : If v 0 were dierent from 0, then taking
in (2.3.11)
x = u0 = 2v 0 2 M = R
we would obtain
2
(u0 )
u0 x + v 0 x 2 = < 0;
(4v 0 )

which is a contradiction to (2.3.11) : Therefore, for Problem (2.3.10) ;


u0 cannot be dierent from 0.

Denition 2.3.4 Let M be a nonempty subset of Rn ; K be a subset


of Rm , g : M ! Rm be a function and
X = fx 2 M : g (x) 2 Kg :

We say that the function g satises :


10 Slaterss constraint qualication with respect to X if intK 6= ;
and there exists a point x1 2 M with the property that
g x1 2 intK:
20 The strict constraint qualication with respect to X if intK 6= ;;
and there exist two dierent points x0 ; x1 2 X and a real number
t 2 ]0; 1[ such that
(1 t) x0 + tx1 2 M

and
g (1 t) x0 + tx1 (1 t) g x0 tg x1 2 intK:

30 Karlins constraint qualication with respect to X if it does not


exist v 2 K n f0g such that
hg (x) ; vi 5 0; for all x 2 M:
3. NECESSARY AND SUFFICIENT OPTIMALITY CRITERIA 83

Example 2.3.5 Let M be the entire space R; K = R+ ; g : R ! R be


the function dened by
g (x) = x, for all x 2 R
and
X = fx 2 M : g (x) 2 Kg = R+ :
Since x1 = 1 2 M = R satises the condition
g x1 = 1 2 intK = fx 2 R : x > 0g;
it follows that the function g satises Slaters constraint qualication
with respect to X:
The function g also satises Karlins complex constraint qualica-
tion with respect to X: Indeed, since
K = R+ ;
we obtain that, for each v 2 K n f0g = fx 2 R : x > 0g; the point
x = v 2 M satises the equality
hg (x) ; vi = v 2 > 0;
and consequently it does not exist v 2 K n f0gsuch that
hg (x) ; vi 5 0; for all x 2 M:
Let now x0 ; x1 2 X and t 2 ]0; 1[ : Obviously,
(1 t) x0 + tx1 2 M = R;
meanwhile
g (1 t) x0 + tx1 (1 t) g x0 tg t1 = 0 62 intK
and therefore the function g satises the strict constraint qualication
with respect to X:
Example 2.3.6 Let M be the entire space Rn ; K = f0g, g : M ! Rn
be the function dened by
g(x) = x, for all x 2 M
and
X = fx 2 M : g(x) 2 Kg:
On one hand, since intK = ;; it follows that the function g does
not satises Slaters constraint qualication with respect to X: Also,
the function g does not satisfy the strict constraint qualication with
respect to X:
84 2. OPTIMALITY CONDITIONS

On the other hand, for each v 2 K = Rn ; v 6= 0; the point x =


v 2 M has the property that
hg(x); vi = kvk2 > 0;

hence there exists no v 2 K n f0g such that


hg(x); vi 5 0; for all x 2 M:

Therefore, the function g satises the Karlins constraint qualica-


tion with respect to X:

Example 2.3.7 Let M be the entire space R, K := R+ ; g : M ! R


be the function dened by
g(x) := x2 ; for all x 2 M

and
X = fx 2 M : g(x) 2 Kg:
Obviously, intK 6= ;: We have
g(x) = x2 5 0; for all x 2 R;

hence Slaters constraint qualication with respect to X is not satised.


Since X = f0g; if follows that there exist no points x0 ; x1 2 X;
x0 6= x1 and no real number t 2]0; 1[ such that
(1 t)x0 + tx1 2 M

and
g((1 t)x0 + tx1 ) (1 t)g(x0 ) tg(x1 ) 2 intK:
Therefore, the function g does not satisfy the strict constraint quali-
cation with respect to X:
Obviously, for each x 2 R and each v 2 K = R+ ; we have
hg(x); vi = x2 v 5 0:

Therefore, the function g does not satises Karlins constraint quali-


cation with respect to X:

Example 2.3.8 Let M be the entire space Rn ; K = R+ ; g : M 2 R


be the function dened by
g(x) = kxk , for all x 2 M

and
X = fx 2 M : g(x) 2 Kg:
3. NECESSARY AND SUFFICIENT OPTIMALITY CRITERIA 85

The function g satises Slaters, Karlins and the strict constraint


qualications with respect to X:
Relations between the above constraint qualications are estab-
lished in the following theorem.

Theorem 2.3.9 Let M be a nonempty subset of Rn ; K be a closed


convex cone in Rm , g : M ! Rm be a function and
X = fx 2 M : g(x) 2 Kg:

10 If the function g satises Slaters constraint qualication with re-


spect to X; then the function g satises Karlins constraint qualication
with respect to X:
20 If the function g satises the strict constraint qualication with
respect to X; then the function g satises Slaters constraint qualica-
tion with respect to X:
30 If, in addition, the set M is convex, K is a polyhedral cone with
nonempty interior and the function g is concave on M with respect to
K; then the function g satises Karlins constraint qualication with
respect to X if and only if the function g satises Slaters constraint
qualication with respect to X:
Proof. 10 Assume that the function g satises Slaters constraint
qualication with respect to X: Then there exists a point x1 2 M such
that
g(x1 ) 2 intK:
Now, if v 2 K n f0g; we have
g(x1 ); v > 0;

and hence there exists no v 2 K n f0g such that


hg(x); vi 5 0, for all x 2 M:

Therefore, the function g satises Karlins constraint qualication with


respect to X:
20 Assume that the function g satises the strict constraint qual-
ication with respect to X: Then there exist two points x0 ; x1 2 X;
x0 6= x1 and a real number t 2]0; 1[ such that
(1 t)x0 + tx1 2 M

and
(2.3.12) g((1 t)x0 + tx1 ) (1 t)g(x0 ) tg(x1 ) 2 int K:
86 2. OPTIMALITY CONDITIONS

Since g(x0 ); g(x1 ) 2 K; t 2]0; 1[ and K is a cone, it follows that


(1 t)g(x0 ) 2 K and tg(x1 ) 2 K:

From (2.3.12) and from the fact that K is a convex cone, we have
g((1 t)x0 + tx1 ) 2 intK + K + K = intK + K intK:

Therefore, the function g satises Slaters constraint qualication with


respect to X:
30 Assume that the function g satises Karlins constraint quali-
cation with respect to X: Then there exists no v 2 K n f0g such
that
hg(x); vi 5 0, for all x 2 M:
From this, we deduce that the system
(
g(x) 2 intK
x2M

is consistent, hence the function g satises Slaters constraint quali-


cation with respect to X:
From this and from statement 10 above, it follows that the function
g satises Karlins constraint qualication with respect to X if and only
if the function g satises Slaters constraint qualication with respect
to X.

Remark 2.3.10 Example 7 proves the existence of a function g :


M ! Rm ; where M Rn and of a set K Rm for which none of the
strict, Slaters, or Karlins constraint qualications with respect to
X = fx 2 M : g (x) 2 Kg
is satised.
We derive now one of the most important saddlepoint theorems of
Kuhn-Tucker type.

Theorem 2.3.11 Let M be a nonempty convex subset of Rn ; K Rm


be a polyhedral cone with nonempty interior, f : M ! R be a convex
function on M with respect to R+ ; g : M ! Rm be a concave function
on M with respect to K; and
X = fx 2 M : g(x) 2 Kg:

In addition, assume that one of the following conditions is satised:


i) The function g satises Slaters constraint qualication with re-
spect to X; or
3. NECESSARY AND SUFFICIENT OPTIMALITY CRITERIA 87

ii) The function g satises the strict constraint qualication with


respect to X; or
iii) The function g satises Karlins constraint qualication with
respect to X:
If x0 is a solution of Problem (2.3.1), then there exists a point
v 2 Cm such that
0

(2.3.13) v0 2 K ;

(2.3.14) g(x0 ); v 0 = 0
and
(2.3.15) L1 (x0 ; v) 5 L1 (x0 ; v 0 ) 5 L1 (x; v 0 ); for all (x; v) 2 M K :

Proof. In view of Theorem 9 above we only need to establish the


theorem under assumption i).
u0 ; vb0 ) 2 R Rm
By Theorem 1, it results that there exists a point (b
such that

(2.3.16) b0 2 R+ ; vb0 2 K ; (b
u u0 ; vb0 ) 6= (0; 0);

(2.3.17) g(x0 ); vb0 = 0


and
(2.3.18)
Lub0 (x0 ; v) 5 Lub0 (x0 ; vb0 ) 5 Lub0 (x; vb0 ); for all (x; v) 2 M K :
We will show now by contradiction that u b0 6= 0: Suppose that u
b0 =
0; then from (2.3.16) ; if follows that vb0 6= 0: From (2.3.17) and the
second inequality in (2.3.18) we obtain
(2.3.19) 05 g(x); vb0 ; for all x 2 M:
Since the function g satises Slaters constraint qualication with re-
spect to X; there exists a point x1 2 M such that
g(x1 ) 2 intK;
then
g(x1 ); v > 0; for all v 2 K n f0g:
Since vb0 2 K n f0g; it results that
g(x1 ); vb0 > 0;

which contradicts (2.3.19) for x = x1 2 M: Hence u


b0 6= 0:
88 2. OPTIMALITY CONDITIONS

b0 > 0 and making the notation


Dividing (2.3.17) and (2.3.18) by u
v 0 = (1=b
u0 )b
v0

we obtain (2.3.13) (2.3.15) :

Example 2.3.12 Let us consider the following optimization problem:


(2.3.20) min f (x) := x21 + x22 + ::: + x2n

subject to
(
x := (x1 ; x2 ; :::; xn ) 2 M = Rn
g(x) := hx; ei 1 2 K = R+ ;

where e = (1; :::; 1) 2 Rn : Obviously, the set M is convex. Moreover,


the function f is convex on Rn with respect to R+ and the function
g is concave on Rn with respect to K and satises Slaters constraint
qualication with respect to
X = fx 2 M : g(x) 2 Kg:

Hence the hypotheses of Theorem 11 are satised. It is easy to verify


that the above problem, (2.3.20) ; has the solution
x0 = (1=n)e 2 Rn

and that p
v0 = 1=n 2 R
satises (2.3.13) (2.3.15) :
The converse of Theorem 11 holds under wider assumptions.

Theorem 2.3.13 Let M be a nonempty subset of Rn ; K be a closed


convex cone in Rm ; f : M ! R and g : M ! Rm be two functions. If
there exist x0 2 Rn ; u0 2 R; v 0 2 Rm such that
(2.3.21) x0 2 M; u0 > 0; v 0 2 K
and
(2.3.22)
Lu0 (x0 ; v) 5 Lu0 (x0 ; v 0 ) 5 Lu0 (x; v 0 ); for all (x; v) 2 M K ;

then x0 is a solution of Problem (2.3.1) :


3. NECESSARY AND SUFFICIENT OPTIMALITY CRITERIA 89

Proof. From the rst inequality of (2.3.22) ; it follows that


(2.3.23) g x0 ; v v 0 = 0; for all v 2 K ;

or
g x0 ; w = 0; for all w 2 K ;
from which we deduce that
g x0 2 (K ) = K;

because K is a closed convex cone. Consequently x0 2 M is a feasible


solution of Problem (2.3.1) :
Moreover, since g (x0 ) 2 K and v 0 2 K it follows that
g x0 ; v 0 = 0

and hence
(2.3.24) g x0 ; v 0 = 0;

because, (2.3.23) for v = 0 2 K ; becomes


g x0 ; v 0 5 0:
Let now x 2 M be a feasible solution of Problem (2.3.1) : Then
(2.3.25) g (x) ; v 0 = 0:

From (2.3.25), the second inequality of (2.3.22) and (2.3.24) ; it


follows that
u0 f (x) = u0 f (x) hg(x); v 0 i =
= u0 f (x0 ) hg(x0 ); v 0 i =
= u0 f (x0 ):
Since u0 > 0 we deduce that
f (x) = f x0

and consequently x0 is a solution of Problem (2.3.1) :

Example 2.3.14 Let us consider the following optimization problem:


(2.3.26) min f (x1 ; x2 ) := x1

subject to
(x1 ; x2 ) 2 M := R2
g (x1 ; x2 ) := x1 ; (x1 )3 + x2 2 K;
90 2. OPTIMALITY CONDITIONS

where
K := f(v1 ; v2 ) 2 R2 : v1 = 0; v2 = 0g:

Obviously, K is a closed convex cone and K = K: The points x0 =


(0; 0) 2 R2 ; u0 = 1 2 R and v 0 = (1; 0) 2 R2 verify (2.3.21) and
(2.3.22) : Consequently, x0 is a solution of Problem (2.3.1) :

Remark 2.3.15 The condition for the set K to be a closed convex


cone in Theorem 13 is essential. If K is not a closed convex cone we
cannot guarantee that x0 is a feasible solution of Problem (2.3.1) :
Indeed, let us consider the following optimization problem in com-
plex space:
(2.3.27) min f (x1 ; x2 ) := x1

subject to
(
(x1 ; x2 ) 2 M := R2
g(x1 ; x2 ) := (x1 ; x2 ) 2 K := f(x1 ; x2 ) 2 R2 : x2 < x1 ; x2 < x1 g:

For x0 = (0; 0) ; u0 = 1 > 0 and

v 0 = (1; 0) 2 K = f(x1 ; x2 ) 2 R2 : x2 < x1 ; x2 < x1 g:;

the conditions (2.3.21) (2.3.22) are satised. Still, x0 is not a solution


of Problem (2.3.27); Problem (2.3.27) does not have solutions. Let us
remark that K is a convex cone but it is not a closed set.

Remark 2.3.16 In Theorem 13 the condition u0 > 0 is essential. If


u0 < 0 and the other hypotheses are satised, then it can be proved
that x0 is a solution of Problem
max f (x)

subject to
(
x2M
g (x) 2 K;

and if u0 = 0; then the criteria function does not appear anywhere.


4. NECESSARY OPTIMALITY CRITERIA 91

4. Necessary Conditions for the Optimal Solutions of the


Optimization Problems with Dierentiable Hypothesis
about the Functions
The purpose of this section is to derive necessary optimality criteria
for optimization problems. In the necessary optimality conditions de-
rived here, convexity does not play a crucial role. The dierentiability
property of the functions is used to linearize the optimization problem
and then theorems of the alternative are used to obtain the necessary
optimality conditions. In order to derive the most important necessary
optimality conditions, constraint qualications are needed.
Let us consider the optimization problem:
(2.4.1) min f (x)
subject to
x2M
g (x) 2 K;
where M Rn and K Rm are two nonempty sets and f : M ! R
and g : M ! Rm are two functions.
In what follows we use the following notations: If M is a nonempty
subset of Rn , x0 is a point from M; q is a positive integer number,
a1 ; :::; aq are points from Rm n f0g,
S := \fH= (ak ) : k 2 f1; :::; qgg;
where
H= (ak ) := fv 2 Rm : hv; ak i = 0g; k 2 f1; :::; qg

and g : M ! Rm is a function, then


(2.4.2) E := fk 2 f1; :::; qg : hg(x0 ); ak i = 0g;

P := fk 2 E : g is pseudoconvex at x0 with respect to H= (ak )g;

Q := E n P; N := f1; :::; qg n E
and
K(g(x0 )) := \fH= (ak ) : k 2 Eg:

We begin with the following linearization theorem, which estab-


lishes the nonexistence of a solution of a system of linear inequalities
whenever the optimization problem has a solution.
92 2. OPTIMALITY CONDITIONS

Theorem 2.4.1 Let M be a nonempty subset of Rn ; x0 be an interior


point of M; f : M ! R and g : M ! Rm be two dierentiable functions
at x0 and
K := \fH= (ak ) : k 2 f1; :::; qgg
be a polyhedral cone in Rm :
If x0 2 Rn is a local solution of the optimization problem (2.4.1),
then the system:
8
>
> [rf (x0 )] (x) < 0
>
>
< [rg(x0 )](z); ak > 0, k 2 Q
(2.4.3)
>
>
> [rg(x0 )](z); ak = 0, k 2 P
>
:
x 2 Rn
has no solution x 2 Rn .
Proof. Since x0 2 M is a local solution of Problem (2.4.1); there
exists a real number re > 0 such that if
(2.4.4) B(x0 ; re) := fx 2 Rn : x x0 < reg
and
X := fx 2 M : g(x) 2 Kg;
then the system
(
f (x0 ) < f (x)
(2.4.5)
x 2 X \ B(x0 ; re);

is inconsistent.
We shall show that if x 2 Rn satises System (2.4.3), then a con-
tradiction follows.
Let x 2 Rn be a solution of System (2.4.5). Then, since x0 is an
interior point of M , there exists a real number rb 2]0; re] such that

x0 + tx 2 M
for all t 2 [0; rb[:,
Since f and g are dierentiable at x0 ; it follows that, for all t 2 [0; rb[;
we have:

f x0 + tx = f x0 + t rf x0 (x) + t x x0 "f x0 + tx; x0 ;


and
g(x0 + tx) = g(x0 ) + t rg(x0 ) (x) + t x x0 "g (x0 + tx; x0 );

where "f (:; x0 ) : M ! R and "g (:; x0 ) : M ! Rm have the properties:


4. NECESSARY OPTIMALITY CRITERIA 93

(2.4.6) lim "f (x0 + tx; x0 ) = "f (x0 ; x0 ) = 0


t!0
and
(2.4.7) lim "g (x0 + tx; x0 ) = "g (x0 ; x0 ) = 0:
t!0

From (??) and (2.4.7), we have


hg(x0 + tx); ak i = hg(x0 ); ak i + t [rz g(x0 )] (x); ak +
(2.4.8)
+ t kx x0 k h"g (x0 + tx; x0 ); ak i;

for all k 2 f1; :::; qg and all t 2 [0; rb[ and

(2.4.9) limh"g (x0 + tx; x0 ); ak i = 0; for all k 2 f1; :::; qg:


t!0

i) From (2.4.3), (??); (2.4.6) and (??); we deduce that there exists
a real number r0 2]0; rb] such that

(2.4.10) f (x0 + tx) < f (x0 ); for all t 2]0; r0 [:

ii) From (2.4.8)-(2.4.9); (??) and (2.4.3) it follows that, for each
k 2 Q; there exists a real number rk 2]0; rb] such that

hg(x0 + tx); ak i > hg(x0 ); ak i = 0; for all t 2]0; rk [:

Consequently,
(2.4.11) g(x0 + tx) 2 H= (ak ); for all t 2 [0; rk [ and all k 2 Q:

iii) Let k 2 P: Since g is pseudoconvex at x0 with respect to H= (ak ),


from (2.4.3), we obtain that

hg(x0 + tx) g(x0 ); ak i = 0; for all t 2 [0; rb[:

Consequently,
(2.4.12) g(x0 + tx) 2 H= (ak ); for all t 2 [0; rb[ and all k 2 P;

because
hg(x0 ); ak i = 0; for all k 2 P E:
iv) From (2.4.8) and (2.4.9), it follows that, for each k 2 N =
f1; :::; qg n E; there exists a real number rk 2]0; rb] such that

hg(x0 + tx); ak i = 0; for all t 2 [0; rk [:

Consequently,
94 2. OPTIMALITY CONDITIONS

(2.4.13) g(x0 + tx) 2 H= (ak ); for all t 2 [0; rk [ and all k 2 N:

Let r be the minimum of all positive numbers

r0 ; rk (k 2 N [ Q)

dened above: Obviously, r > 0: Then, from (2.4.4), (??), (2.4.10)-


(2.4.13), it follows that
r
x1 = x0 + x 2 X \ B(x0 ; re)
2
and
f (x1 ) < f (x0 ).
Consequently, system (2.4.5) has the solution x = x1 which is a con-
tradiction.
We establish now a necessary optimality criteria based on the above
linearization theorem.

Theorem 2.4.2 Let M be a nonempty subset of Rn and x0 be an inte-


rior point of M: Let f : M ! R and g : M ! Rm be two dierentiable
functions at x0 ;

K := \fH= (ak ) : k 2 f1; :::; qgg

be a polyhedral cone in Rm with nonempty interior and E; P; Q; N;


K(g(x0 )) be the sets dened by (2.4.2 ):
If x0 is a local solution of the optimization problem (2.4.1), then
there exist the points u0 2 R, v 0 2 Rm and 2 Rq such that:

(2.4.14) u0 = 0; v 0 2 (K g(x0 ) ) K , u0 ; v 0 6= (0; 0);

(2.4.15) = ( k ) 2 Rq+ ;

(2.4.16) k = 0; for all k 2 N;

q
X
0 k
(2.4.17) v = ka ;
k=1

(2.4.18) if Q = ;; then u0 6= 0;

(2.4.19) if Q 6= ; and Q = ( k )k2Q then u0 ; Q 6= (0; 0) ;


4. NECESSARY OPTIMALITY CRITERIA 95

(2.4.20) g(x0 ); v 0 = 0;

(2.4.21) [rf x0 ]T (u0 ) [rg z 0 ]T (v 0 ) = 0:

Proof. By Theorem 1, system (2.4.3) has no solution x 2 Rn .


A) If P = Q = ;; then system (2.4.3) becomes:
(
[rf (x0 )] (x) < 0
x 2 Rn

and, in view of Corollary ??, system (2.4.3) is inconsistent if and only


if the system
(
[rf (x0 ) (w) = 0
w 2 R+ n f0g

has a solution w: Then the theorem holds with u0 = w; v 0 = 0 and


= 0 2 Rq :
B) If Q = ; and P 6= ;; then E = P and system (2.4.3) becomes
8 0
< [rf (x )] (x) < 0
>
[rg(x0 )](ak )(x) = 0, k 2 P
>
:
x 2 Rn :

If
rf x0 = 0;
then the theorem holds with u0 = 1 2 R, v 0 = 0 2 Rm and = 0 2 Rq :
If
rf x0 6= 0;
then by Corollary ??, the system
T
Xh T
iT
(2.4.22) rf x0 (u) rg x0 (ak ) (yk ) = 0
k2E

has a solution (u0 ; yE ) (0; 0) with


(2.4.23) u0 > 0 and yE = (yk )k2E = 0:

Let us dene
q
X
0 k
(2.4.24) v = ka and = ( 1 ; :::; q) 2 Rq ;
k=1
96 2. OPTIMALITY CONDITIONS

where
(
yk ; if k 2 E
(2.4.25) k =
0; if k 2 N = f1; :::; qg n E:

Since
q
X X
0 k
(2.4.26) v = ka = y k ak ;
k=1 k2E

we have that
(2.4.27) v 0 2 K(g(x0 ) ) K :

From (2.4.23) (2.4.27), it follows that (2.4.14)-(2.4.18) are true. From


(2.4.24), (2.4.25) (2.4.26) and (2.4.2) it follows that equality (2.4.20)
holds and from (2.4.22), (2.4.24); (2.4.25) and (2.4.26) follows equality
(2.4.21).
C ) Let Q 6= ;: If
rf x0 = 0;
then the theorem holds with u0 = 1 2 R; v 0 = 0 2 Rm and = 0 2 Rq :
If
rf x0 6= 0;
then by Corollary ??, the system
T
X T
(2.4.28) rf x0 (u) rg x0 (ak )(yk ) = 0
k2E

has a solution
(2.4.29) u0 ; yE > (0; 0)
with
(2.4.30) u0 ; yQ > (0; 0);
where
yE = (yk )k2E and yJ = (yk )k2J :
Let us dene
q
X
0 k
(2.4.31) v = ka and = ( 1 ; :::; q) 2 Rq ;
k=1

where
(
yk ; if k 2 E
(2.4.32) k =
0; if k 2 N = f1; :::; qg n E:
4. NECESSARY OPTIMALITY CRITERIA 97

Obviously,
q
X X
0 k
(2.4.33) v = ka = yk ak 2 (K(g(x0 ))) :
k=1 k2E

Statements (2.4.14)-(2.4.17) are true. From (2.4.31), (2.4.32) (2.4.33)


and (2.4.2) it follows that equality (2.4.20) holds and from (2.4.28),
(2.4.31) and (2.4.32) follows equality (2.4.21). It remains to show that
(u0 ; v 0 ) 6= (0; 0) and (u0 ; Q ) 6= (0; 0): If u0 = 0; then from (2.4.30) it
follows that Q 0; hence (u0 ; Q ) 6= (0; 0):
If u0 = 0; since Q 0; from (2.4.31)-(2.4.32) we have that the
system
8 P q
< k
ka = 0
k=1
:
= ( k) > 0
has a solution = 2 Rq : By Corollary ??, the system
(
w; ak > 0; k 2 f1; :::; qg
w 2 Cm

has no solution w 2 Rm ; which contradicts intK 6= ;: Consequently,


v 0 6= 0 and hence (u0 ; v 0 ) 6= (0; 0): This completes the proof.
We derive now the Fritz John type saddle point theorem.

Theorem 2.4.3 Let M be a nonempty subset of Rn , x0 be an interior


point of M and K Rm be a polyhedral cone with nonempty interior.
Let also f : M ! R be a dierentiable function at x0 and convex at x0
with respect to R+ and g : M ! Rm be a dierentiable function at x0
and concave at x0 with respect to K:
If x0 is a local solution of Problem (??), then there exist u0 2 R
and v 0 2 Rm such that

(2.4.34) u0 2 R+ , v 0 2 K g x 0 K ; u0 ; v 0 6= (0; 0) ;

(2.4.35) g(x0 ); v 0 = 0;

(2.4.36) Lu0 (x0 ; v) 5 Lu0 (x0 ; v 0 ) 5 Lu0 (x; v 0 );

for all (x; v) 2 M K ; where Lu0 : M Rm ! R is the Lagrangian


of Problem (??) dened by
98 2. OPTIMALITY CONDITIONS

Lu0 (x; v) = u0 f (x) hg(x); vi ; for all (x; v) 2 M Rm :

Proof. If x0 is a local solution of Problem (??) then, in view of


Theorem 2, there exist u0 2 R and v 0 2 Rm such that (2.4.14)-(2.4.21)
are true. Hence (2.4.34) and (2.4.35) hold. Moreover, since g(x0 ) 2 K;
we have
hg(x0 ); vi = 0; for all v 2 K

and then

Lu0 (x0 ; v 0 ) = [u0 f (x0 ) hg(x0 ); v 0 i] = [u0 f (x0 )] =


= [u0 f (x0 ) hg (x0 ) ; vi] = Lu0 (x0 ; v); for all v 2 K :

The function f is dierentiable at x0 and convex at x0 with respect


to R+ ; then

f (x) f (x0 ) = rf (x0 ) x x0 ;

for all x 2 M:
The function g is dierentiable at x0 and concave at x0 with respect
to K; then

g(x0 ) g(x) + rg(x0 ) x x0 2 K;

for all x 2 M: Then

hg(x0 ) g(x) + rg(x0 ) x x0 ; v 0 i = 0;

for all x 2 M; because v 0 2 K : From this it follows that

hg(x) g(x0 ); v 0 i 5 h[rg(x0 )] (x x0 ) ; v 0 i =


(2.4.37) D E
T
= x x0 ; [rg(x0 )] (v 0 ) ;

for all x 2 M:
From (??), (2.4.37) and (2.4.21), it follows that

Lu0 (x; v 0 ) Lu0 (x0 ; v 0 ) = [u0 (f (x) f (x0 ))] [hg(x) g(x0 ); v 0 i] =
= hx
T T
x0 ; [rf (x0 )] (u0 ) [rg(x0 )] (v 0 )i = 0 for all x 2 M:
4. NECESSARY OPTIMALITY CRITERIA 99

Again, just as in the case of the saddle point necessary optimality


criteria, there is no guarantee that u0 > 0 in Theorem 2 above. In the
case when u0 = 0; the function f disappears from (2.4.14)-(2.4.21) and
hence from (??), therefore we have a degenerate case. As it was done
in section 3, it is possible to exclude such cases by introducing restric-
tions on the constraints. These restrictions are again called constraint
qualications. In addition to Slaters constraint qualication, Kar-
lins constraint qualication and the strict constraint qualication, we
shall introduce the Arrow-Hurwicz-Uzawa constraint qualication, the
reverse-concave constraint qualication, the Kuhn-Tucker constraint
qualication and the weak constraint qualication.

Denition 2.4.4 Let M be a nonempty subset of Rn , x0 be an interior


point of M and

K := fH= (ak ) : k 2 f1; :::; qgg

be a polyhedral cone in Rm : The function g : M ! Rm which denes


the set
X := fx 2 M : g(x) 2 Kg
is said to satisfy:
10 The Arrow-Hurwicz-Uzawa constraint qualication at x0 with
respect to X if g is dierentiable at x0 and the system
8 0 k
< [rg(x )] (x) ; a > 0; k 2 Q
>
(2.4.38) [rg(x0 )] (x) ; ak = 0; k 2 P
>
:
x 2 Rn ;

where E; Q; P and K(g(x0 )) are given by (2.4.2); is consistent.


20 The reverse-concave constraint qualication at x0 with respect
to X if g is dierentiable at x0 and pseudoconvex at x0 with respect to
K(g(x0 )):
30 The Kuhn-Tucker constraint qualication at x0 with respect to
X if g is dierentiable at x0 and for each x 2 Rn nf0g with the property
that

(2.4.39) rg(x0 ) (x) 2 K(g(x0 ));

there exists a real number " > 0 and a function : [0; "[! Rn dieren-
tiable at 0 such that
d
(0) = x0 ; (0) = x
dt
100 2. OPTIMALITY CONDITIONS

and
(t) 2 M; g( (t)) 2 K; for all t 2 [0; "[:
40 The weak constraint qualication at x0 with respect to X if g is
dierentiable at x0 and the system
8 0 T
>
< [rg(x )] (v) = 0
(2.4.40) hg(x0 ); vi = 0
>
:
v2K

has the unique solution v = 0:


Some relations between the constraint qualications are established
in what follows.

Theorem 2.4.5 Let M be a nonempty subset of Rn ; x0 be an interior


point of M;
K := \fH= (ak ) : k 2 f1; :::; qgg
be a polyhedral cone in Rn , g : M ! Rm be a function and

x0 2 X =: fx 2 M : g(x) 2 Kg:

10 If the function g satises the reverse-concave constraint quali-


cation at x0 with respect to X; then the function g satises the Arrow-
Hurwicz-Uzawa constraint qualication at x0 with respect to X:
20 If the function g satises the reverse-concave constraint quali-
cation at x0 with respect to X; then the function g satises the Kuhn-
Tucker constraint qualication at x0 with respect to X:
30 Let g be dierentiable at x0 and concave at x0 with respect to
K(g(x0 )): If the function g satises Slaters constraint qualication
with respect to X; then the function g satises the Arrow-Hurwicz-
Uzawa constraint qualication at x0 with respect to X:
40 Let g be dierentiable at x0 and concave at x0 with respect to K:
If the function g satises Slaters constraint qualication with respect
to X; then the function g satises the weak constraint qualication with
respect to X:
50 Let K be with nonempty interior. If the function g satises the
weak constraint qualication at x0 with respect to X; then the function
g satises the Arrow-Hurwicz-Uzawa constraint qualication at x0 with
respect to X:
60 Let g be dierentiable at x0 and concave at x0 with respect to
K(g(x0 )): If the function g satises the strict constraint qualication
4. NECESSARY OPTIMALITY CRITERIA 101

with respect to X; then the function g satises the Arrow-Hurwicz-


Uzawa constraint qualication at x0 with respect to X:
70 Let g be dierentiable at x0 and concave at x0 with respect to K:
If the function g satises the strict constraint qualication with respect
to X; then the function g satises the weak constraint qualication at
x0 with respect to X:
80 Let M be convex, g be concave on M with respect to K; intK 6= ;
and g dierentiable at x0 : If the function g satises Karlins constraint
qualication with respect to X; then the function g satises the Arrow-
Hurwicz-Uzawa constraint qualication at x0 with respect to X:
Proof. 10 Assume that the function g satises the reverse-concave
constraint qualication at x0 with respect to X: Since g is pseudoconvex
at x0 with respect to K(g(x0 )); we have Q = ;: Then system (2.4.38)
becomes
(
[rg(x0 )] (x) ; ak = 0; k 2 P
x 2 Rn ;
which, obviously, is consistent (x = 0 2 Rn is a solution). Hence the
function g satises the Arrow-Hurwicz-Uzawa constraint qualication
at x0 with respect to X:
20 Assume that the function g satises the reverse-concave con-
straint qualication at x0 with respect to X and let x 2 Rn n f0g
satises (2.4.39) : Let : R ! Rn be the function dened by

(t) = x0 + tx; for all t 2 R.

Obviously, the function is dierentiable at the point t = 0 and


d
(2.4.41) (0) = x0 and (0) = x:
dt
We will show that there exists a real number " > 0 such that

(t) 2 M and g ( (t)) 2 K; for all t 2 [0; "[:

Since x0 is an interior point of M; there exists a real number r > 0


such that:

(2.4.42) (t) = x0 + tx 2 M; for all t 2 [0; r[:

From (2.4.42) ; it follows that

(2.4.43) rg(x0 ) (t) x0 2 K(g(x0 ));


102 2. OPTIMALITY CONDITIONS

for all t 2 [0; r[:


Since g is pseudoconvex at x0 with respect to K(g(x0 )); from (2.4.43)
we obtain

g( (t)) g x0 2 K(g(x0 )), for all t 2 [0; r[

and hence

(2.4.44) g( (t)) 2 K(g(x0 )); for all t 2 [0; r[;

because
g(x0 ) 2 K(g(x0 ))
and K(g(x0 )) is a convex cone.
If E = f1; :::; qg; the assertion is trivial. Assume that E 6= f1; :::; qg:
The function g being dierentiable at x0 ; we have

g( (t)) = g(x0 + tx) = g(x0 ) + tf rg(x0 ) (x) +


+ t kxk "(x0 + tx; x0 );
for all t 2 [0; r[; where

lim "(x0 + tx; x0 ) = "(x0 ; x0 ) = 0:


t!0

From this it follows that, for each k 2 f1; :::; qg; we have

g( (t)); ak = g(x0 ); ak + t [rg(x0 )] (x) ; ak +


(2.4.45)
+ t kxk "(x0 + tx; x0 ); ak ; for all t 2 [0; r[
and
(2.4.46) lim "(x0 + tx; x0 ); ak = "(x0 ; x0 ); ak = 0:
t!0

If
k 2 N = fk 2 f1; :::; qg : g(x0 ); ak > 0g;
then, from (2.4.45) and (2.4.46) ; it follows that there exists a real
number rk 2]0; r0 [ such that

(2.4.47) g( (t)); ak = 0;

for all t 2 [0; rk [; and all k 2 N:


If we denote by
" = minfrk : k 2 N g;
then " > 0 and, by (2.4.47) ; it follows that
4. NECESSARY OPTIMALITY CRITERIA 103

(2.4.48) g( (t)) 2 \fH= (ak ) : k 2 N g; for all t 2 [0; "[:

Since
K = K(g(x0 )) \ \fH= (ak ) : k 2 P g ;

from (2.4.44) and (2.4.48) ; we obtain

g( (t)) 2 K; for all t 2 [0; "[

and hence the Kuhn-Tucker constraint qualication at x0 with respect


to X is satised.
30 Assume that the function g satises Slaters constraint quali-
cation with respect to X; then there exists x1 M such that g (x1 ) 2
intS. Since the function g is dierentiable at x0 and concave at x0 with
respect to K(g(x0 )); we have

0 < g(x1 ); ak 5 g(x0 ); ak + rg(x0 ) (x1 x0 ); ak ;

for all k 2 E and hence

0< rg(x0 ) (x1 x0 ); ak ,

for all k 2 E; because

g x0 ; ak = 0; for all k 2 E:

Therefore, system (2.4.38) is consistent (x = x1 x0 is a solution) and


hence the Arrow-Hurwicz-Uzawa constraint qualication at x0 with
respect to X is satised.
40 Assume that the function g satises Slaters constraint quali-
cation with respect to X; then there exists x1 2 M such that g(x1 ) 2
intS: It follows that

(2.4.49) g(x1 ); v > 0; for all v 2 K n f0g:

If the weak constraint qualication at x0 with respect to X is not


satised, then there exists a point v 0 2 K n f0g such that
( T
[rg(x0 )] (v 0 ) = 0
(2.4.50)
hg(x0 ); v 0 i = 0:

The function g being dierentiable at x0 and concave at x0 with respect


to K; we have
104 2. OPTIMALITY CONDITIONS

hg(x1 ); vi 5 hg (x0 ) ; vi + h[rg(x0 )] (x1 x0 ); vi =


(2.4.51) D E
0 1 0 0 T
= hg(x ); vi + x x ; [rg(x )] (v) ;

for all v 2 K : If we choose v = v 0 2 K in (2.4.51) and use (2.4.50) ;


the inequality (2.4.51) becomes

g(x1 ); v 0 5 0;

which contradicts (2.4.49) if we set v = v 0 2 K n f0g:


50 Assume, by contradiction, that the function g does not satisfy
the Arrow-Hurwicz-Uzawa constraint qualication at x0 with respect
to X: Then system (2.4.38) is inconsistent. Obviously, in this case
Q 6= ;; otherwise z = 0 2 Rn is a solution of system (2.4.38), hence g
satises the Arrow-Hurwicz-Uzawa constraint qualication at x0 with
respect to X:
We distingwish two cases:
i) On one hand, for each k 2 Q; we have
T
(2.4.52) rg(x0 ) ak = 0

Let us denote X
v= ak 2 R m :
k2Q

On the other hand, adding equalities (2.4.52) gives


T
(2.4.53) rg(x0 ) (v) = 0:

Obviously,
(2.4.54) v2K

and hence
X
(2.4.55) g(x0 ); v = g(x0 ); ak :
k2Q

We will now show by contradiction that v 6= 0: Assume that v = 0;


then the system
8 P k
> ka = 0
>
< k2Q
> ( k )k2Q = 0
>
:
k 2 R, k 2 Q;
4. NECESSARY OPTIMALITY CRITERIA 105

is consistent ( k = 1; (k 2 Q) is a solution): Consequently, in view of


Corollary ??, the system
(
w; ak > 0; k 2 Q
w 2 Rm

is inconsistent. It follows that the system


(
w; ak > 0; k 2 f1; :::; qg
w 2 Rm

is inconsistent, too. But this contradicts intS 6= ;: Hence


(2.4.56) v 6= 0:
From (2.4.53) (2.4.56) it follows that system (2.4.40) has nonzero
solutions. Consequently, the function g does not satysfy the weak con-
straint qualication at x0 with respect to X; which contradicts the
hypothesis.
ii) There exists k 2 Q such that
T
rg(x0 ) ak 6= 0:

Then, because system (2.4.38) is inconsistent, in view of Corollary ??,


there exist the real numbers k , (k 2 E) such that
(2.4.57) ( k )k2E > 0;
and
T
(2.4.58) rg(x0 ) (v) 6= 0:

where
q
(
X k, if k 2 E
k
v= ka and k =
k=1
0; if k 2 f1; :::; qg n E:

Obviously
(2.4.59) v2K
and
X
(2.4.60) g(x0 ); v = k g(x0 ); ak = 0:
k2E
106 2. OPTIMALITY CONDITIONS

We will show by contradiction that v 6= 0: Suppose that v = 0; then


the system 8 P k
>
> ka = 0
< k2E
> ( k )k2E > 0
>
:
k 2 R, k 2 E

is consistent and, in view of Corollary ??, it follows that the system


(
w; ak > 0; k 2 E
w 2 Rm ;

is inconsistent, too. Consequently, the system


(
w; ak > 0; k 2 f1; :::; qg
w 2 Rm ;

is inconsistent, too. But this contradicts intS 6= ;: It follows that v 6= 0;


but this, together with (2.4.58) (2.4.60) contradicts the fact that the
function g satises the weak constraint qualication at x0 with respect
to X:
60 Assertion 60 follows immediately from assertion 20 of Theorem
9 and the above assertion 30 :
70 Apply assertion 20 of Theorem 9 and the above assertion 40 .
80 Apply assertion 30 of Theorem 9 and the above assertions 30 and
0
4:

Theorem 2.4.6 Let M be a nonempty subset of Rn , x0 be an interior


point of M; and K be a polyhedral cone in Rm . Let also A 2 Rm n ;
b 2 Rm and g : M ! Rm be the function dened by
g(x) := A (x) + b; for all x 2 M:
If
x0 2 X := fx 2 M : g(x) 2 Kg;
then the function g satises the reverse-concave constraint qualication
at x0 with respect to X:
Proof. Obviously the function g is dierentiable at x0 and
rg(x0 ) = A:

The function g being pseudoconvex with respect to K(g(x0 )) at the


point x0 ; it follows that g satises the reverse-concave constraint qual-
ication at x0 with respect to X:
4. NECESSARY OPTIMALITY CRITERIA 107

Theorem 2.4.7 Let M be a nonempty subset of Rn , x0 be an interior


point of M and K be a polyhedral cone in Rm . Let also A 2 Rm n ;
b 2 Rm and g : M ! Rm be the function dened by
g(x) =: A (x) + b for all x 2 M:
If
X =: fx 2 M : g(x) 2 Kg;
and x0 2 X, then the function g satises the Kuhn-Tucker constraint
qualication at x0 with respect to X:
Proof. Apply Theorem 6 and Theorem 5.
We derive now the fundamental necessary optimality criterion of
the optimization problem.

Theorem 2.4.8 Let M be a nonempty subset of Rn ; x0 be an interior


point of M and K be a polyhedral cone in Rm . Let also f : M ! R
and g : M ! Rm be two dierentiable functions at x0 : Moreover, let us
assume that one of the following conditions fullled:
10 The function g satises the Arrow-Hurwicz-Uzawa constraint
qualication at x0 with respect to X and intK 6= ;; or
20 The function g satises the Kuhn-Tucker constraint qualication
at x0 with respect to X; or
30 The function g satises the reverse-concave constraint qualica-
tion at x0 with respect to X; or
40 The function g satises the weak constraint qualication at x0
with respect to X and intS 6= ;; or
50 The function g is concave at x0 with respect to K(g(x0 )); and
satises the Slater constraint qualication with respect to X; or
60 The function g is concave at x0 with respect to K(g(x0 )); and
satises the strict constraint qualication with respect to X; or
70 The set M is convex, intS 6= ;; g is concave on M with respect
to K; and satises Karlins constraint qualication with respect to X:
If x0 is a local solution of Problem (2.4.1); then there exists a point
v 2 Rm such that:
0

(2.4.61) v 0 2 (K(g(x0 ))) K ;

(2.4.62) g(x0 ); v 0 = 0;
and
T
(2.4.63) rf (x0 ) rg(x0 ) v 0 = 0:
108 2. OPTIMALITY CONDITIONS

Proof. The cone K Rm being polyhedral, there exists an integer


number q = 1 and q points a1 ; :::; aq 2 Rm n f0g such that
K = \fH= (ak ) : k 2 f1; :::; qgg;
where
H= (ak ) = fv 2 Rm : v; ak = 0g; for all k 2 f1; :::; qg:

In view of Theorem 5 above we only need to establish the proof under


the assumptions 10 or 20 :
10 Since the hypotheses of Theorem 2 are satised there exist u1 2
R, v 1 2 Rm and = ( k ) 2 Rq such that

(2.4.64) u1 = 0; v 1 2 (K g(x0 ) ) K , u1 ; v 1 6= (0; 0);

(2.4.65) = ( k ) 2 Rq+ ;

(2.4.66) k = 0; for all k 2 N;

q
X
1 k
(2.4.67) v = ka ;
k=1

(2.4.68) if Q = ;; then u1 6= 0;

(2.4.69) if Q 6= ; and Q = ( k )k2Q then u1 ; Q 6= (0; 0) ;

(2.4.70) g(x0 ); v 1 = 0;

(2.4.71) [rf x0 ]T (u1 ) [rg x0 ]T (v 1 ) = 0:

We will show that u1 6= 0: Indeed


i) If Q = ;; then (2.4.68) shows that u1 6= 0:
ii) If Q 6= ;; since g satises the Arrow-Hurwicz-Uzawa constraint
qualication at x0 with respect to X, system (2.4.38) is consistent. Let
x1 be a solution of system (2.4.38) : Assume that u1 = 0; then from
(2.4.69) it follows that
(2.4.72) Q = ( k )k2Q > 0;

and by (2.4.71) we obtain that

(2.4.73) [rg x0 ]T (v 1 ) = 0:
4. NECESSARY OPTIMALITY CRITERIA 109

Now, from (2.4.73) ; (2.4.66) ; (2.4.67) ; (2.4.72) and (2.4.38) we de-


duce that
0 = hx1 ; [rg (x0 )]T (v 1 ))i = h[rg(x0 )] (x1 ) ; v 1 i =
P q P
= [rg(x0 )] (x1 ) ; ka
k
= [rg(x0 )] (x1 ) ; ka
k
=
k=1 k2E
P P
= [rg(x0 )] (x1 ) ; ak = k [rg(x0 )] (x1 ) ; ak > 0;
k2E k2Q

which is a contradiction. Hence u1 > 0:


Since v 1 2 (K(g(x0 ))) K and since (K(g(x0 ))) and K are
1
cones and u > 0; it follows that
(1=u1 )v 1 2 (K(g(x0 ))) K :

Multiplying (2.4.70) and (2.4.71) by (1=u1 ) > 0 and denoting


v 0 = (1=u1 )v 1

it results that there exists v 1 2 Rm such that (2.4.61) (2.4.63) hold.


20 Let x 2 Rn n f0g with the property that
(2.4.74) [rg(x0 )] (x) 2 K(g(x0 )):

Then, since g satises the Kuhn-Tucker constraint qualication at x0


with respect to X; there exists a real number " > 0 and a function
: [0; "[! Rm dierentiable at the point t = 0 such that
d
(2.4.75) (0) = x0 ; (0) = x
dt
and
(t) 2 M; g( (t)) 2 K, for all t 2 [0; "[:
Since x0 is a local solution of Problem (2.4.1) ; we have
d
f ( (t)) jt=0 = 0;
dt
hence
(2.4.76) [rf (x0 )] (x) = 0:

Therefore, each x 2 Rn which satises (2.4.74) satises (2.4.76) ; too.


Hence the system
8
< [rf (x0 )] (z) < 0
(2.4.77) h i
: [rg(x0 )]T ak (x) = 0; k 2 E
110 2. OPTIMALITY CONDITIONS

is inconsistent.
If
rf (x0 ) = 0;
then the theorem holds with v 0 = 0 2 Rm: :
If
rf (x0 ) 6= 0;
then system (2.4.77) being inconsistent, it follows that the system

T
Xh T
iT
(2.4.78) rf (x0 ) (u) rg(x0 ) ak (yk ) = 0;
k2E

has a solution (u1 ; yE ) > 0 with u1 > 0; where yE = (yk )k2E : Now from
(2.4.78) ; we obtain
T
rf (x0 ) rg(x0 ) v 0 = 0;

where X yk
v0 = ak :
k2E
u1
By dening
(
(yk )= (u1 ) , if k 2 E
k =
0; if k 2 f1; :::; qg n f0g

we have that
q
X
0 k
v = ka and v 0 2 (K(g(x0 ))) K :
k=1

Moreover X
g(x0 ); v 0 = k g(x0 ); ak = 0
k2E

and the theorem is proved.


We give the following Kuhn-Tucker type saddlepoint theorem as a
consequence of Theorem 8.

Theorem 2.4.9 Let M be a nonempty subset of Rn ; x0 be an interior


point of M and K be a polyhedral cone in Rm . Let also f : M ! R
be a dierentiable function at x0 convex at x0 with respect to R+ and
let g : M ! Rm be a dierentiable function at x0 concave at x0 with
respect to K: In addition, let one of the following conditions hold:
5. SUFFICIENT CONDITIONS 111

10 The function g satises the Arrow-Hurwicz-Uzawa complex con-


straint qualication at x0 with respect to X and intK 6= ;; or
20 The function g satises the Kuhn-Tucker complex constraint
qualication at x0 with respect to X; or
30 The function g satises the reverse-concave complex constraint
qualication at x0 with respect to X; or
40 The function g satises the weak complex constraint qualication
at x0 with respect to X and intK 6= ;; or
50 The function g is concave at x0 with respect to K(g(z 0 )) and
satises the Slater complex constraint qualication with respect to X;
or
60 The function g is concave at x0 with respect to K(g(z 0 )) and
satises the strict complex constraint qualication with respect to X; or
70 The set M is convex, intK 6= ;; g is concave on M with respect to
K and satises the Karlin complex constraint qualication with respect
to X:
If x0 is a local solution of Problem (2.4.1) ; then there exists a point
v 2 Rm such that:
0

v 0 2 (K(g(z 0 ))) K ;

g(x0 ); v 0 = 0;

and
L1 (x0 ; v) 5 L1 (x0 ; v 0 ) 5 L1 (x; v 0 );

for all (x; v) 2 M K ; where L1 : M K ! R is the Lagrangian of


Problem (2.4.1) given by the formula
L1 (x; v) = f (x) hg(x); vi for all (x; v) 2 M K :

Proof. The proof is similar to that of Theorem 3.

5. Su cient Conditions for the Optimal Solutions of the


Optimization Problems with Dierentiable Hypothesis
about the Functions
The purpose of this paragraph is to give some su cient conditions
for the solution of an optimization problem
Consider the optimization problem:
(2.5.1) min f (x)
112 2. OPTIMALITY CONDITIONS

subject to (
x2M
g(x) 2 K;
where M is a nonempty subset of Rn ; K is a nonempty subset of Rm
and f : M ! R and g : M ! Rm are two functions.
Theorem 2.5.1 Let M be a nonempty subset of Rn and x0 be an
interior point of M: Let also f : M ! R and g : M ! Rm be two
dierentiable functions at x0 and K be a closed convex cone in Rm :
A su cient condition for
x0 2 X := fx 2 M : g(x) 2 Kg;
to be a solution of Problem (2.5.1) is that there exists a point v 0 2 Rm
such that the function L : M ! R dened by
(2.5.2) L(x) := f (x) hg(x); v 0 i; for all x 2 M;
is pseudoconvex at x0 with respect to R+ and
(2.5.3) v0 2 K ;

(2.5.4) hg(x0 ); v 0 i = 0;
and
(2.5.5) hx x0 ; rf (x0 ) [rg(x0 )]H v 0 i = 0;
for all x 2 X:
Proof. Obviously, the function L is dierentiable at x0 and
rL(x0 ) = rf (x0 ) [rg(x0 )]T v 0 ;

Then (2.5.5) can be written as


(2.5.6) x x0 ; rL(x0 ) = 0; for all x 2 X:

Hence x0 2 X; L is a dierentiable function at x0 and x pseudoconvex


at x0 with respect to R+ : Moreover, (2.5.6) is true. Therefore, the
hypotheses of Theorem 7 are satised. Then, it follows that
(2.5.7) L(x) = L(x0 ); for all x 2 X:
Let now x 2 X: Since v 0 2 K and g(x) 2 K we have
hg(x); v 0 i = 0:
Taking into account that (2.5.7) and (2.5.4) hold, we obtain
5. SUFFICIENT CONDITIONS 113

f (x) = f (x) hg(x); v 0 i = L(x) = L(x0 ) = f (x0 ) hg(x0 ); v 0 i =


= f (x0 ); for all x 2 X:
Therefore x0 is a solution of Problem (2.5.1).

Example 2.5.2 Consider the optimization problem:


(2.5.8) min f (x1 ; x2 ) := x21 + x22
subject to
(
(x1 ; x2 ) 2 R2
g (x1 ; x2 ) := z 2 + 2zz + z 2 2iz + 2iz 2 S;

where K = f0g R. Obviously,


f (x1 ; x2 ) = x21 + x22 ; for all (x1 ; x2 ) 2 R2
and
g(x1 ; x2 ) = (x1 )2 + (x2 )2 ; for all (x1 ; x2 ) 2 R2 :
For v 0 = 0 2 R, the function L : R2 ! R dened by (2.5.2) becomes
L (x1 ; x2 ) = x21 + x22 ; for all (x1 ; x2 ) 2 R2

and is convex on R2 with respect to R+ . At the point x0 = (0; 0) the


hypotheses of Theorem 1 are satised, then x0 = (0; 0) is a solution of
Problem (2.5.8).

Remark 2.5.3 Problem (2.5.8) does not satisfy the hypotheses of


Corollary ??.

Theorem 2.5.4 Let M be a nonempty subset of Rn ; x0 be an interior


point of M and K be a polyhedral cone in Rm . Let also f : M ! R be a
dierentiable function at x0 which is pseudoconvex at x0 with respect to
R+ and g : M ! Rm be a dierentiable function at x0 and quasiconcave
at x0 with respect to K(g(x0 )):
A su cient condition for
x0 2 X := fx 2 M : g(x) 2 Kg

to be a solution of Problem (2.5.1) is that there exists a point v 0 2 Rm


such that
(2.5.9) v 0 2 K g x0

and
(2.5.10) hx x0 ; rf (x0 ) [rg(x0 )]H v 0 i = 0; for all x 2 X:
114 2. OPTIMALITY CONDITIONS

Proof. Let x 2 X; then


(2.5.11) g(x) 2 K K(g(x0 ))

and
(2.5.12) g(x0 ) 2 K(g(x0 )):

Since K(g(x0 )) is a convex cone, from (2.5.11) and (2.5.12) ; we obtain


(2.5.13) g (x) g(x0 ) 2 K(g(x0 )):

The function g being qusiconcave at x0 with respect to K(g(x0 )), from


(2.5.13) ; it results that
(2.5.14) [rg(x0 )] x x0 2 K(g(x0 )):

Now from (2.5.9) and (2.5.14) ; it follows that


[rg(x0 )] x x0 ; v 0 = 0;

or, if we take into account the properties of the inner product we have
(2.5.15) x x0 ; [rg(x0 )]H v 0 = 0:
Adding (2.5.10) and (2.5.15) we obtain
x x0 ; rf (x0 ) = 0;
which, leads us to
f (x) = f (x0 )
if we take into account the fact that the function f is pseudoconvex at
z 0 with respect to R+ : Therefore x0 is a solution of Problem (2.5.1) :

Theorem 2.5.5 Let M be a nonempty subset of Rn ; x0 be an interior


point of M , f : M ! R and g : M ! Rm be twice dierentiable
functions at x0 and
K =: \fH= (ak ) : k 2 f1; :::; qgg

be a polyhedral cone in Rm :
Let
(2.5.16) ( k ) 2 Rq+
such that
q
X
0 k
(2.5.17) v = ka 2K
k 1
5. SUFFICIENT CONDITIONS 115

satises
(2.5.18) g x0 ; v 0 = 0

and
(2.5.19) rf (x0 ) [rg(x0 )]H v 0 = 0:

If, for each solution d 2 Rn n f0g of the system


(
h[rg(x0 )] (d) ; ak i = 0; k 2 fk 2 E : k > 0g
(2.5.20)
[rg(x0 )] (d) 2 K(g(x0 ));

we have
(2.5.21) hd; [r2 f (x0 ) r2 g(x0 )u0 ]di > 0;

then x0 is a local solution of Problem (2.5.1):


Proof. Assume that x0 is not a local solution of Problem (2.5.1):
Then there exists a sequence (xj )j2N from X n fx0 g which converges to
x0 such that
(2.5.22) f (xj ) < f (x0 ); for all j 2 N.

For each j 2 N we denote by


1 j
tj = xj x0 and dj = (x x0 ):
tj
Then we have
tj > 0 and xj = x0 + tj dj ; for all j 2 N

and
lim tj = 0:
j!1

Since
dj = 1; for all j 2 N;
it follows that the sequence (dj )j2N contains a convergent subsequence.
Without loss of generality we can assume that the sequence itself is
convergent. Let
d = lim dj :
j!1
Then
kdk = 1:
From
xj 2 X; for all j 2 N;
116 2. OPTIMALITY CONDITIONS

we have
g(xj ) 2 K; for all j 2 N
and hence
g(xj ) g(x0 ) 2 K(g(x0 )); for all j 2 N.
Moreover
1
d x0 + tj dj g x0 2 K:
tj
From this, by passing to limit, we obtain that
(2.5.23) [rg(x0 )] (d) 2 K(g(x0 ));

or, equivalently
h[rg(x0 )] (d) ; ak i = 0, for all k 2 E:

From (2.5.22) we deduce that


1
[f (x0 + tj dj ) f (x0 )] < 0; for all j 2 N.
tj

From this, by passing to limit, we obtain that


(2.5.24) [rf (x0 )] (d) 5 0:

We will show, by contradiction, that


(2.5.25) h[rg(x0 )] (d) ; ak i = 0 for all k 2 fk 2 E : k > 0g :

Assume that there exists k0 2 E with k0 > 0 such that


(2.5.26) h[rg(x0 )] (d) ; ak0 i > 0:

Then, from (2.5.16) ; (2.5.17) ; (2.5.18) ; (2.5.26) ; we obtain

P
q
h[rg(x0 )] (d) ; v 0 i = h[rg(x0 )] (d) ; ka
k
i=
k=1
P
q P
= k h[rg(x
0
)] (d) ; ak i = k h[rg(x
0
)] (d) ; ak i =
k=1 k2E

= k0 h[rg(x
0
)] (d) ; ak0 i > 0:

Hence
(2.5.27) h[rg(x0 )] (d) ; v 0 i > 0:
5. SUFFICIENT CONDITIONS 117

On the other hand, from (2.5.19) and (2.5.24), we have


h[rg(x0 )] (d) ; v 0 i = hd; [rg(x0 )]H (v 0 )i =
= hd; [rf (x0 )]i = [rf (x0 )] (d) 5 0;

which contradicts (2.5.27):


From (2.5.25) and (2.5.23); it follows that d is a solution of system
(??): Since d 2 Rn n f0g; we have
(2.5.28) hd; [r2 f (x0 ) r2 g(x0 )u0 ]di > 0:

If we denote by F : M ! R the function


F (x) = f (x) hg(x); u0 i for all x 2 M;
then, from (2.5.28) we obtain
(2.5.29) hd; [r2 F (x0 )]d > 0:
We have
F (xj ) = f (xj ) hg(xj ); v 0 i 5 f (x0 ) =
5 f (x0 ) hg(x0 ); v 0 i = F (x0 ) for all j 2 N,
because
g(xj ) 2 K for all j 2 N,
v 0 2 K and hg(x0 ); v 0 i = 0:
Since
X
m
0 0
rF (x ) = rf (x ) vk0 rg(x0 );
k=1
it follows that
F (x0 + tj dj ) F (x0 ) tj hdj ; rF (x0 )i =
= f (x0 + tj dj ) hg(x0 + tj dj ); v 0 i f (x0 ) + hg(x0 ); v 0 i
P m
tj hdj ; rf (x0 ) vk0 rgk (x0 ) i =
k=1
0 j 0
= f (x + tj d ) f (x ) hg(x0 + tj dj ); v 0 i
f (x0 + tj dj ) f (x0 ) for all j 2 N.

Hence
1
[F (x0 + tj dj ) F (x0 ) tj hdj ; rF (x0 )i] 5 0 for all j 2 N.
t2j

Then, by Theorem , we have


hr2 F (x0 )i 5 0
118 2. OPTIMALITY CONDITIONS

which contradicts (2.5.29):


Therefore x0 is a local solution of Problem (2.5.1):
Bibliography

[1] J.-P. Aubin: Optima and Equilibria: an Introduction to Nonlinear Analysis,


Springer-Verlag, Berlin, 1993
[2] V. Barbu and Th. Precupanu: Convexity and Optimization in Banach
Spaces, Publishing House of Roumanian Academy and Reidel Publishing
Comp., 1986
[3] A. Ben-Israel: Linear Equations and Inequalities on Finite Dimensional, real
or complex, vector space: a Unied Theory, J. Math. Anal. Appl., 27(1969),
no.2, 367-389
[4] J.M. Borwein and A.S. Lewis: Convex Analysis and Nonlinear Optimiza-
tion, Springer-Verlag, New York, Berlin, Heidelberg, 2000
[5] W.W. Breckner: Introducere n teoria problemelor de optimizare convexa cu
restrictii, partea I, Editura Dacia, Cluj, 1974
[6] W.W. Breckner: Cercetare operationala, Universitatea Babes-Bolyai, Fa-
cultatea de Matematic a, Cluj-Napoca, 1981
[7] W.W. Breckner si D.I. Duca: Culegere de probleme de cercetare operatio-
nala, Universitatea din Cluj-Napoca, Facultatea de Matematic a, Cluj-Napoca,
1983
[8] F.H. Clarke: Optimization and Nonsmooth Analysis, John Wiley&Sons, New
York, 1983
[9] M. Dragomirescu si M. Mali ta: Programare neliniara, Editura stiintic
a,
Bucuresti, 1972
[10] D.I. Duca: Constraint Qualications in Nonlinear Programming in Complex
Space, Studia Univ. Babes-Bolyai, Math., 23(1978), no.1, 61-65
[11] Duca D.I.: The Vectorial Programming Problem in Complex Space, Proceed-
ing of the Third Colloquim on Operations Research, Cluj-Napoca, october 20-
21, 1978, Babes-Bolyai University of Cluj-Napoca, Cluj-Napoca, 1979, pp.82-
89
[12] D.I. Duca: On Vectorial Programming Problem in Complex Space, Studia
Univ. Babes-Bolyai, Math., 24(1979), no.1, 51-56
[13] Duca D.I.: Proper E ciency in the Complex Vectorial Programing, Studia
Univ. Babes-Bolyai, Math., 25(1980), no.1, 73-80
[14] Duca D.I.: O teorem a de punct sa n programarea matematic a n domeniul
complex, Lucr arile Seminarului itinerant de ecuatii functionale, aproximare si
convexitate, Cluj-Napoca, mai, 1980, Univ. Babes-Bolyai, Cluj-Napoca, 1980,
pp. 35-39
[15] Duca D.I.: Necessary Optimality Criteria in NonlinearProgramming in Com-
plex Space with Dierentiability, Mathematica-LAnalyse numerique et la the-
orie de lapproximation, 9(1980), no.2, 163-179

119
120 BIBLIOGRAPHY

[16] Duca D.I.: Saddlepoint Optimality Criteria of Nonlinear Programming in


Complex Space without Dierentiability, Studia Univ. Babes-Bolyai, Math.,
25(1980), no.4, 39-46
[17] Duca D.I.: On Su cient Optimality Conditions in Nonlinear Programming
in Complex Space, Mathematica, 22(45)(1980), no.2, 263-267
[18] Duca D.I.: On some Types of Optimization Problems in Complex Space,
LAnalyse numerique et la theorie de lapproximation, 10(1981), no.1, 11-16
[19] Duca D.I.: Linear Optimality Criteria in Nonlinear Programming in Complex
Space, Studia Univ. Babes-Bolyai, Math., 26(1981), no.3, 73-79
[20] Duca D.I.: Asupra unui rezultat din teoria dualit atii in programarea ma-
tematica n domeniul complex, Lucr arile Seminarului itinerant de ecuatii
functionale, aproximare si convexitate (Cluj-Napoca, decembrie 1981), 119-
126, Univ. Babes-Bolyai, Cluj-Napoca, 1981
[21] Duca D.I.: Mathematical Programming in Complex Space, Doctoral Thesis,
University of Cluj-Napoca, 1981
[22] Duca D.I.: (2,1)-Order Duality in Nonlinear Programming in Complex Space,
Lucrarile Seminarului Th. Angheluta (Cluj-Napoca, iunie 1983), 99-104,
Preprint, Institutul Politehnic din Cluj-Napoca, Cluj-Napoca, 1983
[23] Duca D.I.: E ciency Criteria in Vectorial Programming in Complex Space,
Itinerant Seminar on Functional Equations, approximation and Convexity
(Cluj-Napoca, 1983), 51-54, Preprint 83-2,Univ. Babes-Bolyai,Cluj-Napoca,
1983
[24] Duca D.I.: On Duality in Nonlinear Programming in Complex Space, Itin-
erant Seminar on Functional, Approximation and Convexity (Cluj-Napoca,
1984), 45-48, Preprint 84-6, Univ. Babes-Bolyai, Cluj-Napoca, 1984
[25] Duca D.I.: E ciency Criteria in Vectorial Programming in Complex Space
without Convexity, Cahiers du Centre Etudes Rech. Oper., 26(1984), no.3-4,
217-226
[26] Duca D.I.: Duality in Mathematical Programming in Complex Space.
Converse Theorems, LAnalyse numerique et la theorie de lapproximation,
13(1984), no.1, 15-22
[27] Duca D.I.: Second-Order Duality in Nonlinear Programming in Complex
Space, Proceedings of the Colloquium on Approximation and Optimization
(Cluj-Napoca, October, 1984), 213-222, Univ. Cluj-Napoca, Cluj-Napoca, 1985
[28] Duca D.I.: On the Higher-Order in Nonlinear Programming in Complex
Space, Seminar on Optimization Theory (Cluj-Napoca, 1985), 39-50, Preprint
85-5, Univ. Babes-Bolyai, Cluj-Napoca, 1985
[29] Duca D.I.: Vectorial Programming in Complex Space, Seminar on Opti-
mization Theory (Cluj-Napoca, 1986), 3-82, Preprint 86-8, Univ Babes-Bolyai,
Cluj-Napoca, 1986
[30] Duca D.I.: On the Farkas Type Theorem for Complex Linear Equations and
Inequalities, Itinerant Seminar on Functional Equations, Approximation and
Convexity (Cluj-Napoca, 1987), 143-148, Preprint 87-6, Univ. Babes-Bolyai,
Cluj-Napoca, 1987
[31] Duca D.I.: On Theorems of the Alternative for Nonhomogeneous Complex
Linear Equations and Inequalities, Seminar on Optimization Theory (Cluj-
Napoca, 1987), 29-40, Preprint 87-8, Babes-Bolyai University, Cluj-Napoca,
1987
BIBLIOGRAPHY 121

[32] Duca D.I.: Saddlepoint Necessary Criteria for Nonlinear Programming


in Complex Space, LAnalyse numerique et la theorie de lapproximation,
16(1987), no.1, 29-33
[33] Duca D.I.: Theorem of Motzkins Alternative for Nonhomogeneous Com-
plex Linear Equations and Inequalities, LAnalyse numerique et la theorie de
lapproximation, 16(1987), no.2, 117-126
[34] Duca D.I.: On Theorems of the Alternative for Complex Linear Equations
and Inequalities, Seminar on Optimization Theory (Cluj-Napoca, 1989), 35-48,
Preprint 89-8, Babes-Bolyai University, Cluj-Napoca, 1989
[35] Duca D.I.: Theorems of the Alternative for Nonhomogeneous Complex Linear
Equations and Inequalities, Mathematica, 31(54)(1989), no.2, 127-142
[36] Duca D.I.: The Dual of the Dual in Mathematical Programming in Complex
Space, LAnalyse numerique et la theorie de lapproximation, 19(1990), no.1,
35-43
[37] Duca D.I., Duca E. and Lup sa L.: On the Structure of the Set of Points
Dominated and Nondominated in a Vectorial Optimization Problem, Seminar
on Functional Equations, Approximation and Convexity (Cluj-Napoca, 1993),
20-26, Preprint 93-6, Babes-Bolyai University, Cluj-Napoca, 1993
[38] D.I. Duca and L. Lup sa: On the E-Epigraph of an E-Convex Function,
Journal of Optimization Theory and Applications, 129(2006), 2, 341-348
[39] Duca D.I. and Stancu-Minasian I.M.: Mathematical Programming in
Complex Space: A Comprehensive Bibliography, LAnalyse numerique et la
theorie de lapproximation, 17(1988), no.2, 133-140
[40] Duca E. si Duca D.I.: Asupra structurii multimii punctelor eciente ntr-o
problema de programare vectorial a n domeniul complex, Lucrarile Seminarului
itinerant de ecuatii functionale aproximare si convexitate, Cluj-Napoca, mai
1979, Univ. Babes-Bolyai, Cluj-Napoca, 1979, pp. 41-47
[41] Farkas J.: Theorie der einfachen Ungleichungen, Journal fur die reine und
angewandte Mathematik, 124(1902), 1-27
[42] Gale D.: A Geometric Duality Theorem with Economic Applications, Review
of Economic Studies, 34(1967), 19-24
[43] Gordan P.: Uber die Auosung linearer Gleichungen mit reelen Coe cienten,
Mathematische Annalen, 6(1873), 23-28
[44] Karlin S.: Mathematical Methods and Theory in Games, Programming and
Economics, McGraw-Hill, New York, 1960
[45] Karush W.: Minima of Functions of Several Variables with Inequalities as
Side Conditions, Masters thesis, University of Chicago, 1939
[46] Kuhn H.W. and Tucker A.W.: Nonlinear Programming, In: Proceedings of
the Second Berkeley Symposium on Mathematical Statistics and Probability,
University of California Press, Berkeley,1951
[47] P.J. Laurent: Approximation et optimisation, Herman, Paris, 1972
[48] D.T. Luc: Theory of Vector Optimization, Lecture Notes in Econ. and Math.
Systems, vol. 319, Springer-Verlag, Berlin, 1989
[49] Lup sa L., Duca E. and Duca D.I.: On the Structure of the Set of Points
Dominated and Nondominated in an Optimization Problem, Revue dAnalyse
Numerique et de Theorie de lApproximation, 22(1993), no.2, 193-199
[50] O.L. Mangasarian: Nonlinear Programming, McGrow-Hill Book Company,
New York, 1969
122 BIBLIOGRAPHY

[51] T.S. Motzkin: Beitrage zur Theorie der linearen Ungleichungen, Inaugural
Dissertation, Basel 1933; Jerusalem Azriel 1936
[52] A. Ruszczynski: Nonlinear Optimization, Priceton University Press, Prince-
ton and Oxford, 2006
[53] I.M. Stancu-Minasian: Fractional Programming. Theory, Methods and Ap-
plications, Kluwer Academic Publishers Group, Dordrecht, 1997
[54] Stancu-Minasian I.M., Duca D.I. and Nishida T.: Multiple Objective
Linear Fractional Optimization in Complex Space, Math. Japonica, 35(1990),
no.1, 195-203
[55] J. Werner: Optimization Theory and Applications, Friendr. Vieweg&Sohn,
Braunschweig, 1984
[56] Zalinescu C.: A Generalization of the Farkas Lemma and Applications to
Convex Programming, J. Math. Anal. Appl., 66(1978), 651-678
[57] C. Zalinescu: Programare matematica n spatii normate innit dimensionale,
Editura Academiei Romne, Bucuresti, 1998