You are on page 1of 126

# Convex Analysis

Dorel I. Duca
Contents

1. Introduction 1
Chapter 1. Convex Analysis 3
1. Notations and Preliminary Results 3
2. Linear Subspaces. A ne Sets 7
3. Convex Sets 10
4. Separation Theorems for Convex Sets 23
5. Cones 29
6. The Dual (Polar) of a Set 38
7. Convex Functions 50
8. Strictly Convex Functions 59
9. Pseudoconvex Functions 62
10. Quasiconvex Functions 65
Chapter 2. Optimality Conditions 71
1. Optimization Problems 71
2. Some General Results of Optimization 73
3. Necessary Optimality Criteria and Su cient Optimality
Criteria 79
4. Necessary Conditions for the Optimal Solutions of the
Optimization Problems with Dierentiable Hypothesis
5. Su cient Conditions for the Optimal Solutions of the
Optimization Problems with Dierentiable Hypothesis
Bibliography 119

iii
1 1

1. Introduction
The book ends with the list of references.
CHAPTER 1

Convex Analysis

## 1. Notations and Preliminary Results

We will denote by R the corp of real numbers, by Rn the n th
dimensional linear real space over R and by Rm n the linear space of
the real matrices of the type (m; n), over R.
An element (x1 ; :::; xn ) 2 Rn will be denoted by x or (xj ) and an
element
2 3
a11 ::: a1n
4 ::: ::: ::: 5 2 Rm n
am1 ::: amn
by A or [akj ] :
Since the spaces Rn and R1 n are isomorphic we will identify Rn
with the space R1 n .
If
2 3
a11 ::: a1n
4
A = ::: ::: ::: 5 2 Rm n
;
am1 ::: amn
then, for each x = (x1 ; :::; xn ) 2 Rn ; we will denote by A(x) the point
!
X n X
n
A(x) = a1j xj ; :::; amj xj 2 Rm :
j=1 j=1

## The set fe1 ; :::; en g; where

e1 = (1; 0; 0; :::; 0) ; e2 = (0; 1; 0; :::; 0) ; :::; en = (0; 0; :::; 0; 1) 2 Rn

## is a base of the linear space Rn : Then each point

z = (z1 ; z2 ; :::; zn ) 2 Cn

## can be represented as follows

x = x1 e1 + x2 e2 + ::: + xn en :

## For x = (x1 ; :::; xn ); v = (v1 ; :::; vn ) 2 Rn we will denote by

3
4 1. CONVEX ANALYSIS

## the inner product of x and v: In this way Rn becomes a Banach space

if we consider the hermitian norm
p
(1.1.1) kxk = hx; xi; for all x 2 Rn :

## If x 2 Rn and r 2]0; +1[; then we denote by

B(x; r) = fv 2 Rn : kx vk < rg

## the open ball with centre x and radius r and by

B[x; r] = fv 2 Rn : kx vk 5 rg

## the closed ball with centre x and radius r:

Endowed with the topology induced by the norm (1.1.1), Rn be-
comes a linear topological space for which the set
fB(x; r) : r > 0g

## represents a basis of neighborhoods for the point x 2 Rn :

If M is a subset of the space Rn , then we denote by
intM - the interior of M
and by
clM - the closure of M:
If x = (x1 ; :::; xn ); y = (y1 ; :::; yn ) 2 Rn , then we use the following
notations:

## Moreover, we also use the notations:

Rn+ = fx 2 Rn : 0 5 xg
and
R+ = R1+ = [0; +1[:
1. NOTATIONS AND PRELIMINARY RESULTS 5

If a 2 Rn n f0g; then
H= (a) = fx 2 Rn : hx; ai = 0g;
H5 (a) = fx 2 Rn : hx; ai 5 0g;
H> (a) = fx 2 Rn : hx; ai > 0g;
H< (a) = fx 2 Rn : hx; ai < 0g
and if a1 ; :::; aq 2 Rn n f0g and
K = \fH= (ak ) : k 2 f1; :::; qgg;

## then for each b 2 Rn we denote

K(b) = \fH= (ak ) : k 2 Eg;

where
E = fk 2 f1; :::; qg : hb; ak i = 0g:
Let us remark that
hA(x); vi = hz; AT (v)i;
for all A 2 Rm n
; x 2 Rn and v 2 Rm :

## Denition 1.1.1 The matrix A 2 Rn n is said to be:

a) symmetric, if AT = A
b) antisymmetric (or skew-symmetric), if AT = A;
c) positive semidenite, if hA(x); xi = 0; for all x 2 Rn ;
d) positive denite if hA(x); xi > 0, for all x 2 Rn n f0g;
e) negativ (semi)denite, if the matrix ( A) is positive (semi)de-
nite.
If A 2 Rm n
; then we will denote by
R(A) = fy 2 Rm : there exists x 2 Rn such that y = A(x)g =
= A(Rn ) - the range of the matrix A;
and by
N (A) = fx 2 Rn : A(x) = 0g - the null space of the matrix A:
We dene now the concept of continuity for functions. From sev-
eral possible approaches, we have chosen one that uses convergent se-
quences.

## Denition 1.1.2 Let f : M ! Rm be a function, where M is a

nonempty subset of Rn : The function f is said to be continuous at
a point x0 2 M if for each sequence (xk )k2N of points from M that
6 1. CONVEX ANALYSIS

## converges to x0 ; the sequence (f (xk ))n2N of points from Rm converges

to f (x0 ):
If f is continuous at all the points of M , then f is said to be
continuous on M:
An important example of a continuous function is the norm function
k k : Rn ! R dened by
k k (x) = kxk ; for all x 2 Rn :
A particularly important example of a continuous function is the
distance function dM : Rn ! R of a nonempty subset M of Rn . The
function dM associates to each point x 2 Rn its distance dM (x) from
M: Formally, dM is dened as follows:
Denition 1.1.3 Let M be a nonempty subset of Rn : The function
dM : Rn ! R dened by
dM (x) = inffkx vk : v 2 M g; for all x 2 Rn ;
is called the distance function of the set M:
If M Rn is the singleton set fag; then
dM (x) = kx ak ; for all x 2 Rn :
In particular, if a = 0; then
dM (x) = kxk ; for all x 2 Rn :

## Theorem 1.1.4 Let M be a nonempty subset of Rn : Then the distance

function dM : Rn ! R of the set M satises the Lipschitz condition:
jdM (x) dM (v)j 5 kx vk ; for all x; v 2 Rn :
Proof. Let x; v 2 Rn : From the denition of the distance function
dM ; it follows that, for each " > 0; there exists a point u in M such
that
kx uk < dM (x) + ":
By the triangle inequality,
dM (v) 5 kv uk 5 kv xk + kx uk < kv xk + dM (x) + ":
Since " > 0 is arbitrary,
dM (v) 5 kv xk + dM (x):
Interchanging x and v in the last inequality, we nd that
dM (x) 5 kx vk + dM (v);
2. LINEAR SUBSPACES. AFFINE SETS 7

## whence dM satises the Lipschitz condition.

From this it follows the following statement.

## Corollary 1.1.5 Let M be a nonempty subset of Rn : Then the distance

function dM : Rn ! R of the set M is continuous on Rn :
It follows that, a point x 2 Rn lies in the closure clM of M if and
only if its distance dM (x) from M is zero.

## Remark 1.1.6 In general, the inf in the denition of dM cannot

be replaced by min : To this end, suppose that M = Rn n f0g: Then
dM (0) = 0; but there is no x 2 M = Rn n f0g such that k0 ak = 0:

## 2. Linear Subspaces. A ne Sets

In this section we introduce the concepts of linear subspace and
a ne set, which are generalizations to Rn of the terms point, line and
plane from elementary geometry.

## Denition 1.2.1 Let M be a nonempty subset of Rn : We say that the

set M is a linear subspace of Rn if, for each s; t 2 R and each x; y 2 M;
we have
sx + ty 2 M:

## Example 1.2.3 The sets

M1 = fx = (x1 ; :::; xn ) 2 Rn : x1 = ::: = xn g;
M2 = fx = (x1 ; :::; xn ) 2 Rn : x1 = 0g;
M3 = fx = (x1 ; :::; xn ) 2 Rn : x1 = x2 = ::: = xn 1 g
are linear subspaces of Rn :

## Example 1.2.4 The set

M = fx = (x1 ; :::; xn ) 2 Rn : x1 = 1g

## Theorem 1.2.5 If M and L are two linear subspaces of Rn and a;

b 2 R, then the set
aM + bL = fx 2 Rn : there exist u 2 M and v 2 L such that
x = au + bvg
is a linear subspace of Rn :
8 1. CONVEX ANALYSIS

## Theorem 1.2.6 The intersection of an arbitrary family of linear sub-

spaces of Rn is a linear subspace of Rn :
Proof. Let (Mj )k2J be a family of linear subspaces of Rn : Let
x; y 2 \fMj : j 2 Jg and s; t 2 R. From
x; y 2 \fMj : j 2 Jg;

## it follows that x; y 2 Mj ; for all j 2 J: Since each set Mj (j 2 J) is a

linear subspace of Rn , we have that

sx + ty 2 Mj ; for all j 2 J:

Thus
sx + ty 2 \fMj : j 2 Jg;

## Denition 1.2.7 Let M be a nonempty subset of Rn . The intersection

of all linear subspaces of Rn ; containing the set M , is called the linear
hull of the set M; and is denoted by sp(M ).

## Theorem 1.2.8 If M is a nonempty subset of Rn ; then the following

statements are true:
10 The set sp(M ) is a linear subspace of Rn :
20 The inclusion M sp(M ) holds.
30 If M L Rn and L is a linear subspace of Rn ; then sp(M )
L.
40 The set sp(M ) is, with respect to the inclusion relation, the small-
est linear subspace of Rn containing M .
50 The equality
M = sp (M )
holds if and only if the set M is a linear subspace of Rn :
60 The equality
sp (sp (M )) = sp (M )
holds.
Proof. Statements 10 ; 20 ; 30 follow immedialtely from the deni-
tion and from Theorem 6. Statements 40 and 50 follow from statements
10 ; 20 ; 30 : Statement 60 follows from statements 10 and 50 above:
2. LINEAR SUBSPACES. AFFINE SETS 9

## Denition 1.2.9 Let M be a subset of Rn . We say that the set M is

a ne if for each x; y 2 M and each t 2 R we have
(1 t) x + ty 2 M:

## Example 1.2.11 The set

M = f(x1 ; :::; xn ) 2 Rn : x1 = 1g

## is a ne, but is not a linear subspace of Rn .

Consequently, there exist a ne sets which are not linear subspaces:

## Example 1.2.12 The sets

B[0; 1] = fx 2 Cn : kxk 5 1g; B(0; 1) = fx 2 Cn : kxk < 1g;

## are not a ne.

Indeed, the points x = 0 2 Rn and y = (1=2)e1 = (1=2; 0; :::; 0) 2
n
R belong to the sets B[0; 1] and B(0; 1), while the point u = (1
4)x + 4y = 2e1 does not belong to B[0; 1] and B(0; 1):
Now, we examine how a ne sets behave with respect to the oper-
ations of addition and scalar multiplication.

## Theorem 1.2.13 Let M and L be two subsets of Cn and a; b 2 C: If

the sets M and L are a ne, then the set

## aM +bL = fz 2 Cn : there exist u 2 M; v 2 L such that z = au+bvg

is a ne.
Proof. It is immediate from Denition 9.

## Theorem 1.2.14 If the subsets M of Rn and L of Rm are a ne, then

the subset M L of Rn+m is a ne.
Proof. It is immediate from Denition 9.
The next result expresses one of the most important properties of
the family of a ne sets, namely that it is closed under the formation
of arbitrary intersections.

## Theorem 1.2.15 If (Mj )j2J is a family of a ne subsets of Rn , then

the set
M = \fMj : j 2 Jg
is a ne, too.
10 1. CONVEX ANALYSIS

## (The intersection of an arbitrary family of a ne subsets of Rn is

an a ne set.)
Proof. Let x; y 2 M and t 2 R: From x; y 2 M it follows that
x; y 2 Mj ; for all j 2 J: Since each Mj (j 2 J) is a ne, we have
(1 t)x + ty 2 Mj ; for all j 2 J:

Thus
(1 t)x + ty 2 M;
which shows that the set M is a ne:

## Denition 1.2.16 Let M be a subset of Rn : The intersection of all

a ne subsets of Rn ; containing M; is called the a ne hull of the set
M; and is denoted by a(M ).

## Theorem 1.2.17 If M is a subset of Rn ; then the following statements

are true:
10 The set a(M ) is a ne.
20 The inclusion M a(M ) holds:
30 If M L Rn and L is a ne, then a(M ) L:
40 The set a(M ) is, with respect to the inclusion relation, the
smallest a ne set of Rn containing the set M .
50 The equality
M = a (M )
holds if and only if the set M is a ne.
60 The equality
a (a (M )) = a (M )
holds:
Proof. Statements 10 ; 20 ; 30 follow immediately from the deni-
tion and from Theorem 15. Statements 40 and 50 follow from state-
ments 10 ; 20 ; 30 : Statement 60 follows from statements 10 and 50 above:

3. Convex Sets
The notion of a convex set is central to optimization theory. The
concept of convex set is a simple one. A set in space is convex if when-
ever it contains two points, it also contains the line segment joining
them.
The purpose of this section is to introduce the fundamental concept
of convex set and to present some properties of these sets.
3. CONVEX SETS 11

## Denition 1.3.1 Let M be a subset of Rn . We say that the set M is

convex if for each x; y 2 M and each t 2 [0; 1] we have
(1 t)x + ty 2 M:

## Example 1.3.2 Each a ne subset of Rn is convex.

Example 1.3.3 The sets
B (0; 1) = fx 2 Rn : kxk < 1g
and
B[0; 1] = fx 2 Rn : kxk 5 1g
are convex but not a ne.
Consequently, there exist convex sets which are not a ne.
Example 1.3.4 The set M = Rn n f0g is not convex.
Indeed, the points
x = e1 = (1; 0; :::; 0) and y = e1 = ( 1; 0; :::; 0)
belong to the set M and the point
(1 (1=2)) x + (1=2)y = 0 2
= M:

## Example 1.3.5 Let a 2 Rn : Then the sets

H= (a) = fx 2 Rn : hx; ai = 0g;

## H5 (a) = fx 2 Rn : hx; ai 5 0g;

H> (a) = fx 2 Rn : hx; ai > 0g;
H< (a) = fx 2 Rn : hx; ai < 0g;
H(a) = fx 2 Rn : hx; ai = 0g
are convex.
Example 1.3.6 Let A 2 Rn m
: Then the set
M = fy 2 Rn : there exists x 2 Rm
+ such that y = A(x)g
is convex.
Theorem 1.3.7 Let M be a subset of Rn . The following statements
are equivalent:
10 The set M is convex.
20 For each t 2 [0; 1] the equality
(1 t)M + tM = M
12 1. CONVEX ANALYSIS

holds.
30 For each s; t 2 [0; +1[ the equality
sM + tM = (s + t)M
holds.
Proof. (10 =) 20 ) Assume that the set M is convex. Then, for
each t 2 [0; 1]; we have
(1 t) M + tM M:

## On the other hand, for each r 2 R, we have

M = 1 M = (1 r + r)M (1 r)M + rM:

Consequently,
(1 t)M + tM = M; for all t 2 [0; 1]:

## (20 =) 30 ) Let s and t be two positive real numbers. If at least one

of these numbers is zero, then obviously the following equality holds
sM + tM = (s + t)M:

If both numbers s and t are nonzero, then s+t > 0 and t=(s+s) 2]0; 1[.
From assertion 20 , we have

t t
1 M+ M = M;
s+t s+t

## or, after multiplying by s + t, it results

tM + sM = (t + s)M:

## (30 =) 10 ) Let x; y 2 M , t 2 [0; 1] and s = 1 t. Then, by assertion

30 ; we deduce that
(1 t)x + ty 2 (1 t)M + tM = (1 t + t)M = M:

## Theorem 1.3.8 Let M be a nonempty subset of Rn . The set M is

convex if and only if, for each x 2 M and each d 2 Rn ; the set

## x;d = f 2 [0; +1[ : x + d 2 M g

is convex.
3. CONVEX SETS 13

## Proof. Necessity. Assume that the set M is convex and let x 2

M and d 2 Rn . In order to prove that the set x;d is convex, let
; 2 x;d and t 2 [0; 1]. Obviously
(1 t) + t 2 [0; +1[:

## Now, by the fact that ; 2 x;d it follows that the points u = x + d

and v = x + d belong to the set M: From the convexity of M; we
obtain
(1 t)u + tv 2 M:
It follows that
x + ((1 t) + t )d = (1 t)(x + d) + t(x + d) =
= (1 t)u + tv 2 M:
Consequently, the set x;d is convex.
Su ciency. Assume that, for each x 2 M and each d 2 Rn , the
set x;d is convex. In order to prove that the set M is convex, let
x; y 2 M and t 2 [0; 1]: Since
x + 0(y x) = x 2 M
and
x + 1 (y x) = y 2 M;
we deduce that the points 0 and 1 belong to the set x;y x : By the
convexity of the set x;y x ; it follows that t 2 x;y x . Consequently,
(1 t)x + ty = x + t(y x) 2 M:

## The proof is complete.

Below we examine how convex sets behave with respect to the op-
erations of addition and scalar multiplication.

## Theorem 1.3.9 Let M and L be subsets of Rn and a; b 2 R. If the

sets M and L are convex, then the set
aM + bL = fx 2 Rn : there are u 2 M; v 2 L such that x = au + bvg

is convex, too.
Proof. Let x; y 2 aM + bL and t 2 [0; 1]. Then
(1 t)x + ty 2 (1 t)(aM + bL) + t(aM + bL) =
= a ((1 t)M + tM ) + b ((1 t)L + tL) =
= aM + bL:
Consequently, the set aM + bL is convex.
14 1. CONVEX ANALYSIS

## Corollary 1.3.10 If M and L are two convex subsets of Rn and a is

a real number, then the sets M + L, M L, aM , M are convex, too.
Proof. It is immediate from Theorem 9.

## Theorem 1.3.11 Let M; a subset of Rn and L; a subset of Rm be two

convex sets. Then the set M L is convex, too.
Proof. Let x = (x1 ; x2 ) ; y = (y1 ; y2 ) 2 M L; and t 2 [0; 1] :
From the convexity of M and L we have 2 (1 t) x1 + ty1 2 M and
(1 t) x2 + ty2 2 L: It follows that
(1 t) x + ty = ((1 t) x1 + ty1 ; (1 t) x2 + ty2 ) 2 M L:
Consequently, the set M L is convex.

## Theorem 1.3.12 Let M and L be two convex subsets of Rn+m and

P = fx 2 Rn+m : there exist u 2 Rn and v; w 2 Rm such that
(u; v) 2 M; (u; w) 2 L and x = (u; v + w)g:
Then the set P is convex, too.
Proof. Let x1 ; x2 2 P and t 2 [0; 1]: From x1 ; x2 2 P we deduce
that there exist the points u1 ; u2 2 Rn and v 1 ; v 2 ; w1 ; w2 2 Rm such
that
(u1 ; v 1 ) 2 M; (u1 ; w1 ) 2 L; x1 = (u1 ; v 1 + w1 )
and
(u2 ; v 2 ) 2 M; (u2 ; w2 ) 2 L; x2 = (u2 ; v 2 + w2 ):
From the convexity of the sets M and L we obtain
((1 t) u1 + tu2 ; (1 t)v 1 + tv 2 ) = (1 t)(u1 ; v 1 ) + t(u2 ; v 2 ) 2 M
and
((1 t) u1 + tu2 ; (1 t)w1 + tw2 ) = (1 t)(u1 ; w1 ) + t(u2 ; w2 ) 2 L:
Let now
u = (1 t)u1 + tu2 ; v = (1 t)v 1 + tv 2 ; w = (1 t)w1 + tw2 :

## This shows that (u; v) 2 M; (u; w) 2 L and then

(1 t) x1 + tx2 =
= (1 t) u1 + tu2 ; (1 t) v 1 + tv 2 + (1 t) w1 + tw2 =
= (u; v + w) 2 P:
Consequently, P is convex.
3. CONVEX SETS 15

Remark 1.3.13 The reunion of two convex sets might not be convex.
Indeed, the sets
M = fx 2 R : x = 1g and L = fx 2 R : x 5 1g

## are convex, while the set M [ L is not convex.

The next result expresses one of the most important properties of
the family of convex sets, namely that it is closed under the formation
of arbitrary intersections.

## Theorem 1.3.14 If (Mj )j2J is a family of convex subsets of Rn ; then

their intersection
M = \fMj : j 2 Jg
is a convex set, too.
(The intersection of an arbitrary family of convex subsets is a convex
set.)
Proof. If M = ;, then the statement is obvious. If M 6= ;, let
x; y 2 M and t 2 [0; 1]: Then x; y 2 Mj ; for all j 2 J: The set Mj being
convex, for each j 2 J, it follows that
(1 t)x + ty 2 Mj ; for all j 2 J:
Thus
(1 t)x + ty 2 M:
Consequently, the set M is convex.

## Corollary 1.3.15 Let A be a nonempty subset of Rn : Then the sets

M1 = \fH= (a) : a 2 Ag; M2 = \fH5 (a) : a 2 Ag;
M3 = \fH> (a) : a 2 Ag; M4 = \fH< (a) : a 2 Ag;
M5 = \fH(a) : a 2 Ag
are convex.
Proof. Invoke for it the theorem above and example 5.

## Denition 1.3.16 Let M be a subset of Rn . The convex hull of M;

denoted by conv(M ), is the intersection of all convex subsets of Rn
containing M .

## Theorem 1.3.17 Let M be a subset of Rn : Then the following state-

ments are true:
10 The set convM is convex.
20 The inclusion M convM holds:
16 1. CONVEX ANALYSIS

## 30 If M L Rn and the set L is convex, then convM L:

40 The set convM is, with respect to the inclusion relation, the
smallest convex set in Rn containing M .
50 The equality
M = convM
holds if and only if the set M is convex.
60 The equality
conv(convM ) = convM
holds.
Proof. The rst three assertions of the theorem are immediate
from the denition and Theorem 14. Statements 40 and 50 follow from
statements 20 and 30 : Statement 60 follows from statements 10 and
50 :

## Example 1.3.18 The convex hull of the unit sphere

S(0; 1) = fx 2 Rn : kxk = 1g
is the closed unit ball
B[0; 1] = fx 2 Rn : kxk 5 1g;
i.e.
convfx 2 Rn : kzk = 1g = fx 2 Rn : kxk 5 1g:
Indeed, the closed unit ball B[0; 1] Rn is convex and contains the
unit sphere S(0; 1); so
convS(0; 1) B[0; 1]:

## In order to prove that B[0; 1] convS(0; 1); let x 2 B[0; 1]:

If x = 0; then for u = (1; 0; :::; 0) 2 Rn and v = u; which belong
to S(0; 1); we have
x = (1=2)u + (1=2)v:
Since conv (S(0; 1)) is convex and
S(0; 1) convS(0; 1)

## it follows that x 2 convS(0; 1):

If x 6= 0; then 0 < kxk 5 1 and hence t = (1 kxk) =2 2 [0; 1] : On
the other hand u = (1= kxk) x and v = ( 1= kxk) x belong to S (0; 1)
and x can be written as follows
x = (1 t) u + tv:
3. CONVEX SETS 17

## Since conv (S(0; 1)) is convex and

S(0; 1) conv (S(0; 1))

## it follows that x 2 conv (S(0; 1)) : Consequently,

B[0; 1] convS(0; 1):
Thus
convS(0; 1) B[0; 1] and B[0; 1] convS(0; 1)
as desired.

## Denition 1.3.19 Let p = 1 be an integer number and x; x1 ; :::; xp be

p + 1 points from Rn : The point x is said to be a convex combination
of the points x1 ; :::; xp if there exist p real numbers t1 ; :::; tp 2 [0; +1[
such that t1 + ::: + tp = 1 and
x = t1 x1 + ::: + tp xp :

## Denition 1.3.20 Let M be a nonempty subset of Rn : A point x 2 Rn

is said to be a convex combination of points from M if there exists an
integer number p = 1 and p points x1 ; :::; xp 2 M such that x is a
convex combination of the points x1 ; :::; xp :
The convex hull of a set was dened by means of convex sets contain-
ing that set. By contrast, the following theorem expresses the convex
hull of a set in terms of the points of the set themselves. The following
representation theorem of the convex hull of a set holds.

## Theorem 1.3.21 The convex hull of a nonempty subset M of Rn is

the set of all convex combinations of points from M:
Proof. Let L denote the latter set, that is,
L = fx 2 Rn : there exists p 2 N, there exist x1 ; :::; xp 2 M;

## such that x = t1 x1 + ::: + tp xp g:

Obviously, M L: Moreover, it is also obvious that the set L is convex.
Then, in virtue of assertion 30 of Theorem 17, it follows that
convM L:

## We next show that

L convM:
18 1. CONVEX ANALYSIS

## Let x 2 L; it follows that there exist an integer number p = 1, p points

x1 ; :::; xp 2 M and p real numbers t1 ; :::; tp 2 [0; +1[ with t1 +:::+tp = 1
such that
x = t1 x1 + ::: + tp xp :
We argue by induction on p, that x 2 conv (M ) : When p = 1; the
assertion is trivial. Assume now that each convex combination of p
1 = 1 points from M belongs to convM and we show that x; which
is a convex combination of p points from M; belongs to convM: We
distinguish two cases according to weather tp is equal or not to 1:
a) If tp = 1; then t1 = ::: = tp 1 = 0 and hence
x = tp xp = xp 2 M convM:

b) If tp 6= 1; then let
t1 tp 1
s1 = ; :::; sp 1 = :
1 tp 1 tp
Obviously, s1 ; :::; sp 2 [0; +1[ and
1
t1 + ::: + tp 1 1 tp
s1 + ::: + sp 1 = = = 1:
1 tp 1 tp
Then the point
v = s1 x1 + ::: + sp 1 xp 1
2 convM:
Moreover,
x = (1 tp )v + tp xp
and since the set convM is convex and M convM; we deduce that
x 2 convM: The theorem is proved.

## Theorem 1.3.22 The subset M of Rn is convex if and only if M is

the set of all convex combinations of points from M:
Proof. Apply Theorems 21 and 17.

## Theorem 1.3.23 Let x1 ; :::; xp 2 Rn : Then

convfx1 ; :::; xp g =

## = ft1 x1 + ::: + tp xp : t1 ; :::; tp 2 [0; +1[ with t1 + ::: + tp = 1g =

= fx 2 Rn : there exist t1 ; :::; tp 2 [0; +1[ with t1 + ::: + tp = 1
such that x = t1 x1 + ::: + tp xp g:
3. CONVEX SETS 19

## Proof. It is immediate from Theorem 21.

For example, the convex hull of three points in R2 is the triangle
having these points as vertices.
In the above results we considered all convex combinations, that is,
for all numbers p of points and for arbitrary selections of these points.
Both p and the points selected can be restricted. We now show that
the number of points p need not be larger than n + 1:

## Theorem 1.3.24 (Caratheodory0s theorem) If M is a subset of Rn ;

then every element of convM is a convex combination of at most n + 1
points of M:
Proof. Let x 2 convM ; then there exists p 2 N, there exist p
points x1 ; :::; xp 2 M; there exist p real numbers t1 ; :::; tp 2 [0; +1[
with t1 + ::: + tp = 1 such that x = t1 x1 + ::: + tp xp :
If p n + 1; then theorem is proved.
If p > n + 1, then we shall show that p can be reduced by one. If
there exists j 2 f1; :::; pg such thet tj = 0; then we can delete the jth
point and we are done. So let, tk > 0; for all k 2 f1; :::; pg; then let
y 1 = (x1 ; 1) ; :::; y p = (xp ; 1) from Rn+1 : Then there exist p real numbers
s1 ; :::; sp not all equal 0; so that
s1 y 1 + + sp y p = 0;
or equivalent
(1.3.1) s1 x 1 + + sp xp = 0 and s1 + + sp = 0:
Obviously, there exists al least k 2 f1; :::; pg such that sk > 0; then the
set J = fj 2 f1; :::; pg : sj > 0g is nonempty: Let
tj
q = min : j2J ; and rk = tk qsk ; for all k 2 f1; :::; pg:
sj
Then, by (1.3.1) we still have
r1 + ::: + rp = (t1 + ::: + tp ) q (s1 + ::: + sp ) = 1;
and
r1 x1 + ::: + rp xp = t1 x1 + ::: + tp xp q s1 x1 + ::: + sp xp = x:
By the denition of q; at least one rj = 0 and then we can delete the
jth point.
Continuiting in this way, we can reduce the number of points to
n + 1:
20 1. CONVEX ANALYSIS

## Theorem 1.3.25 In Rn , the convex hull of an open set is an open

set.
Proof. Let M be an open subset of Rn . If x 2 convM; then
there exist p 2 N, the points x1 ; :::; xp 2 M and the real numbers
t1 ; :::; tp 2 [0; +1[ with
t1 + ::: + tp = 1
such that
x = t1 x1 + ::: + tp xp :

## Since M is open and x1 ; :::; xk 2 M; there exist p real numbers r1 >

0; :::; rp > 0 such that
B(x1 ; r1 ) M; :::; B(xp ; rp ) M:
Let
r = minfr1 ; :::; rp g:
Then r > 0 and
B(x1 ; r) B(x1 ; r1 ) M; :::; B(xp ; r) B(xp ; rp ) M:

We show that
B(x; r) convM:

## Let u 2 B(x; r); then ku xk < r:

Moreover, for each k 2 f1; :::; pg; the point
uk = x k + u x

## lies in B(xk ; r) and hence in M: Since the point u can be written as

u = t1 u1 + ::: + tp up ;

B(x; r) convM:

## Consequently, each point of convM is an interior point of convM; i.e.

convM is an open set.

## Theorem 1.3.26 If M is a convex subset of Rn ; then its interior

intM and its closure clM are convex.
3. CONVEX SETS 21

## Proof. Let x; y 2 intM and t 2 [0; 1] : From x; y 2intM; it follows

that exists a real number r > 0 such that B (x; r) M and B (y; r)
M: Then
B ((1 t) x + ty; r) = f(1 t) x + tyg + B (0; r) =
= (1 t) (fxg + B (0; r)) + t (fyg + B (0; r)) =
= (1 t) B (x; r) + tB (y; r)
(1 t) M + tM = M;
because the sets B (0; r) and M are convex. Consequently, (1 t) x +
ty 2 intM; hence intM is convex.
To prove the second part of the theorem, let x; y 2 clM; and t 2
[0; 1]: From x; y 2 clM; it follows that exist two sequences xk k 1 and
y k k 1 such that xk ; y k 2 M; for all k 2 N, and xk ! x and y k ! y;
when k ! 1: The set M is convex, and hence, for each k 2 N, we have
(1 t) xk +ty k 2 M: On the other hand (1 t) xk +ty k ! (1 t) x+ty;
when k ! 1: It follows that (1 t) x + ty 2clM; hence the set clM is
convex.

## Theorem 1.3.27 In Rn , the convex hull of a bounded set is a bounded

set.
Proof. Let M be a bounded subset of Rn ; then there exists a real
number r > 0 such that
M B[0; r]:

convM B[0; r]:

## Consequently, convM is bounded.

The next theorem is more explicit. We give a denition before.

## Denition 1.3.28 Let M be a nonempty bounded subset of Rn . The

real number
diamM = supfkx yk : x; y 2 M g

## Theorem 1.3.29 Let M be a nonempty bounded subset of Rn : Then

M and convM have the same diameter.
22 1. CONVEX ANALYSIS

## Proof. Assume that the set M has the diameter r: If x; y 2

convM; then there exist p; q 2 N and
x1 ; :::; xp ; y 1 ; :::; y q 2 M ; t1 ; :::; tp ; s1 ; :::; sq = 0;
with
t1 + ::: + tp = 1; s1 + ::: + sq = 1;
such that
x = t1 x1 + ::: + tp xp , y = s1 y 1 + ::: + sq y q :
Thus p q p q
X X X X
k
x= tk si x ; y= tk si y i ;
k=1 i=1 k=1 i=1

## whence, by the triangle inequality,

p q
X X
kx yk = tk si (xk yi) 5
k=1 i=1
p q p q
XX X X
5 tk si xk yi 5 tk si r = r:
k=1 i=1 k=1 i=1
Hence the diameter of the set convM does not exceed r: Since M
convM; the diameter of the set convM is at least r: Thus the set convM
has the diameter r: The theorem is proved.
The next result, while not having the same basic importance as
others in this section, does have its own intrinsic interest and will be
needed later in the chapter.
Theorem 1.3.30 Let M be a nonempty bounded subset of Rn , N be
a closed convex subset of Rn and L Rn be a set such that
M +L M + N:
Then
L N:
Proof. Let x0 2 M: If L = ;; then we have L N and the
theorem is proved. If L 6= ;; then let x 2 L: Since
x0 + x 2 M + L M + N;
we deduce that there exist two points x1 2 M and u1 2 N such that
x0 + x = x1 + u1 :
Similarly, from
x1 + x 2 M + L M + N;
4. SEPARATION THEOREMS FOR CONVEX SETS 23

## there exist two points x2 2 M and u2 2 N such that

x1 + x = x2 + u2 :

Going on in this way, it follows that there exist the sequences xk k2N
;
uk k2N which, for each k 2 N; satisfy the relations:
(1.3.2) x k 2 M ; uk 2 N ; x k 1
+ x = x k + uk :

## From (1.3.2) we deduce that, for each k 2 N; we have

(1.3.3) x0 + kx = xk + (u1 + ::: + uk ):

## Since N is convex, the point

1 1
vk = (u + ::: + uk )
k
belongs to the set N; for all k 2 N:
On the other hand, from (1.3.3) we obtain that
1 k
(1.3.4) x vk = x x0 ; for all k 2 N.
k
Since the set M is bounded and xk 2 M; for each k 2 N, we deduce
that there exists the limit
1 k
lim x x0 = 0:
k!1 k

From this and (1.3.4), it follows that the sequence v k k2N converges
to x: Since the set N is closed, it follows that x 2 N: Consequently,
L N:
The theorem is proved.

## 4. Separation Theorems for Convex Sets

The results that we establish in this section on the separation of
convex sets are among the most important ones in convexity. From
several possible ways of approach, we have chosen that which uses the
distance function. The distance function dM : Rn ! R of a nonempty
subset M of Rn is dened by
dM (x) = inffkx uk : u 2 M g; for all x 2 Rn
(see Denition 3).
In general, the inf in the denition of dM cannot be replaced by
min : To this end, assume that M = Rn n f0g: Then dM (0) = 0; but
there exists no a 2 M = Rn n f0g such that k0 ak = 0:
24 1. CONVEX ANALYSIS

## Theorem 1.4.1 Let M be a nonempty closed subset of Rn and x 2 Rn :

Then there exists an element x0 2 M such that
dM (x) = x x0 :

## Proof. From the denition of dM (x) it follows that there exists a

sequence xk k2N of points from M such that
(1.4.1) lim x xk = dM (x):
k!1

## From this it follows that the sequence x xk k2N

is bounded, hence
there exists a real number r > 0 such that
x xk 5 r; for all k 2 N.

The inequalities
xk 5 x xk + kxk 5 r + kxk ; for all k 2 N

## show that the sequence xk k2N is bounded. Then it contains a subse-

quence (xkj )j2N which converges to a point x0 2 Rn : Since the set M
is closed, it follows that x0 2 M:
Moreover, from (1.4.1), it follows that
lim x xkj = dM (x):
j!1

But
lim x xk = x x0 ;
k!1
hence
dM (x) = x x0
and the theorem is proved.
For obvious reasons, the point x0 in the statement of Theorem 1 is
called a nearest point of M to x:
In what follows we need the following result.

## Theorem 1.4.2 Let x; u 2 Rn : If there exists a real number >0

such that
(1.4.2) kx + tuk = kxk ; for all t 2]0; [;
then
hx; ui = 0:
4. SEPARATION THEOREMS FOR CONVEX SETS 25

Proof. Since
kx + tuk2 = hx + tu; x + tui = kxk2 + 2t hx; ui + t2 kuk2 ;

## from (1.4.2) we deduce that, for each t 2]0; [; we have

t
hx; ui + kuk2 = 0:
2
Passing to limit t ! 0+ in the last inequality, we deduce that
hx; ui = 0:
The theorem is proved.
Theorem 1 shows that, given a nonempty closed subset M of Rn
and a point x of Rn , there exists a point x0 of M which is nearest to
x: We show below that, if M is also convex, then this nearest point x0
of M to x is unique and that, for each point u of M; the points x x0
and u u0 make a nonacute angle with each other.

## Theorem 1.4.3 Let M be a nonempty closed convex subset of Rn and

x 2 Rn : Then there exists a unique point x0 2 M such that
dM (x) = x x0 :
Moreover,
(1.4.3) x x0 ; u x0 5 0; for all u 2 M:

dM (x) = x x0 :

## In order to prove (1.4.3), let u 2 M: By the convexity of the set M we

deduce that
(1 t) x0 + tu 2 M; for all t 2]0; 1]
and the property of x0 ; we get that
x x0 5 x (1 t) x0 + tu = x x0 + t x0 u ;

x x0 ; u x0 5 0:

## Hence (1.4.3) is true.

To prove the unicity of x0 ; let x1 2 M be such that
dM (x) = x x1 :
26 1. CONVEX ANALYSIS

## Then, by what we have just proved,

x x 0 ; x1 x0 5 0 and x x 1 ; x0 x1 5 0:

## Adding these two last inequalities, we deduce that

2
x0 x1 = x0 x 1 ; x0 x1 = x0 x+x x 1 ; x0 x1 =
= x0 x; x0 x1 + x x 1 ; x0 x1 5 0;
which proves that x0 = x1 : The proof is complete:
Relation (1.4.3) characterizies the point x0 with the property that
dM (x) = x x0 :
The following statement is true.

## Theorem 1.4.4 Let M be a nonempty closed convex subset of Rn ;

x 2 Rn n M and x0 2 M: Then the following statements are equivalent:
10 The equality
x x0 = dM (x)
holds:
20 For each u 2 M; we have
x x0 ; u x0 5 0:
Proof. If statement 10 is true; then, in view of Theorem 3, state-
ment 20 is true:
Assume now that statement 20 is true and let u be an arbitrary
point from M: Then
2 2
kx uk2 x x0 = kuk2 + 2 x; x0 u x0 =
0 2
= kuk2 + 2 x0 ; x0 u x =
2
= kuk2 2 x0 ; u + x0 =
0 2
= u x = 0:
The theorem is proved.
A convex closed set and a point outside of it can be separated by a
plane.

## Theorem 1.4.5 (strict separation theorem) Let M be a closed convex

set of Rn and u 2 Rn n M: Then there exist a point a 2 Rn n f0g and a
real number " > 0 such that
ha; ui + " 5 ha; xi ; for all x 2 M:
4. SEPARATION THEOREMS FOR CONVEX SETS 27

## Proof. By Theorem 3, there exists u0 2 M such that

(1.4.4) u u0 ; x u0 5 0; for all x 2 M:
Dene a = u0 u . Note that a is not zero, because u 2
= M: Then,
for each x 2 M; inequality (1.4.4) implies
ha; xi = a; u0 = ha; ui + a; u0 u =
2
= ha; ui + kak :

## Setting " = kak2 ; we obtain the assertion of the theorem.

If the set M is not closed, the point u 2 Rn n M may be a boundary
point of M and we have a weaker version of the above result.

## Theorem 1.4.6 (separation theorem) Let M be a convex set of Rn

and u 2 Rn n M: Then there exists a point a 2 Rn n f0g such that
(1.4.5) ha; ui 5 ha; xi ; for all x 2 M:
Proof. Let uk k=1 be a sequence of points uk 2 M n clM; (k = 1)
converges to u: By Theorem 5, for each k = 1 there exists a point
ak 2 Rn n f0g such that
ak ; uk 5 ak ; x ; for all x 2 M:
Since ak 6= 0; for all k = 1; without loss of generality, we may
assume that ak = 1; for all k = 1; because we can divide both sides
of (1.4.5) by ak :
Then the sequence ak contains at least convergent subsequence
akp p=1 . Let a 2 Rn the limit of the subsequence akp : Obviously,
kak = 1 and
(1.4.6) akp ; ukp 5 akp ; x ; for all x 2 M and p = 1:
Passing to the limit in (1.4.6) ; we obtain the required result.
Separation is possible also when both sets being separated are con-
vex sets.

## Theorem 1.4.7 Let L and M be nonempty subsets of Rn : If L is closed

and M is compact, then there exist two points u0 2 L and x0 2 M such
that
(1.4.7) u0 x0 = inffku xk : u 2 L; x 2 M g:
28 1. CONVEX ANALYSIS

## Proof. The distance function dL : Rn ! R of L is continuous on

n
R . Then there exists a point x0 2 M such that
dL (x0 ) = inffdL (x) : x 2 M g:

## By Theorem 1, t to be used since L is closed, there exists a point

u0 2 L such that
dL (x0 ) = u0 x0 :
Moreover, for each u 2 L and each x 2 M; we have
ku xk = inffkv xk : v 2 Lg = dL (x) =
= inffdL (z) : z 2 M g = dL (x0 ) =
= u0 x0 ;
which, since u0 2 L; x0 2 M; shows that
u0 x0 = inffku xk : u 2 L; x 2 M g:
The theorem is proved.
We refer to the points u0 and x0 in the statement of Theorem 7 as
nearest points of L and M: These points u0 and x0 are not unique, in
general.

## Theorem 1.4.8 Let L and M be two nonempty disjoint convex subsets

of Rn : If L is closed and M is compact, then there exist a point a 2
Rn n f0g and a real number " > 0 such that, for each u 2 L and each
x 2 M; we have
(1.4.8) ha; ui + " 5 ha; xi :

## Proof. In view of Theorem 7, there exist two points u0 2 L and

0
x 2 M such that
u0 x0 = inffku xk : u 2 L; x 2 M g:

## Since L and M are disjoint, we have u0 6= x0 : Let now

2
a = x0 u0 and " = x0 u0 :

## Obviously, a 2 Rn n f0g and " > 0: In order to prove (1.4.8), let u 2 L

and x 2 M: In view of Theorem 3, we have
x0 u0 ; u u0 5 0 and u0 x0 ; x x0 5 0:

Then
2
ha; xi ha; ui = a; x0 a; u0 = a; x0 u0 = x0 u0 = ":
5. CONES 29

## Remark 1.4.9 The boundedness assumption in Theorem 8 is essen-

tial. The closed convex sets
M = fx = (x1 ; x2 ) 2 R2 : x2 5 0g
and
L = fx = (x1 ; x2 ) 2 Rn : x2 = exp ( x1 )g
cannot be strictly separated.
If the sets are not bounded, then we have a weaker version of The-
orem 8.

## Theorem 1.4.10 If L and M are two nonempty disjoint convex sub-

sets of Rn ; then there exists a point a 2 Rn n f0g such that, for each
u 2 L and each x 2 M; we have
ha; ui 5 ha; xi :
Proof. Dene N = L M: Since 0 does not belong to N; we can
use Theorem 6. Then there exists a point a 2 Rn n f0g such that
ha; vi 5 ha; 0i = 0; for all v 2 N:
Since v 2 N has the form v = u x with u 2 L and x 2 M; the
theorem is proved.

5. Cones
The purpose of this section is to introduce the concept of cone and
to present some properties of these sets.

## Denition 1.5.1 Let K be a nonempty subset of Rn . We say that

the set K is a cone if, for each x 2 K and each t 2]0; +1[; we have
tx 2 K:

## Example 1.5.3 The set K = Rn n f0g is a cone.

Let us remark that the set Rn n f0g is not a subspace of the linear
space Rn , therefore not every cone is a linear subspace of Rn .
Let us also remark that the set Rn n f0g is not a convex set, hence
a cone might not be a convex set.

## Example 1.5.4 The sets

B(0; 1) = fx 2 Rn : kxk < 1g and B[0; 1] = fx 2 Rn : kxk 5 1g
30 1. CONVEX ANALYSIS

## are not cones.

Let us remark that the sets B(0; 1) and B[0; 1] are convex. Conse-
quently, there exist convex sets which are not cones.

Remark 1.5.5 All cones are unbounded, with the exception of the
trivial cone f0g.

## Theorem 1.5.6 The nonempty subset K of Rn is a cone if and

only if
tK = K; for all t 2]0; +1[:

## Proof. Necessity. If K is a cone and t 2]0; +1[, then by the

denition of a cone we deduce that
tK K:

## In order to prove the converse inclusion, let x 2 K and t 2]0; +1[:

Since 1=t 2]0; +1[, by the fact that K is a cone, it follows that
(1=t)x 2 K:

Then, we have
x = t((1=t)x) 2 tK
and hence we showed that
K tK:

tx 2 tK = K

## Theorem 1.5.7 If K and L are cones in Rn and a; b 2 R, then the

set
aK + bL
is a cone, too.
Proof. Let x 2 aK + bL and t 2]0; +1[. From x 2 aK + bL we
deduce that there exist s 2 K and v 2 L such that x = as + bv: Since
K and L are cones, we have that ts 2 K and tv 2 L and hence
tx = t(as + bv) = a(ts) + b(tv) 2 aK + bL:
The theorem is proved.
5. CONES 31

## Theorem 1.5.8 If (Kj )j2J is a nonempty family of cones in Rn , then

the set
K := [fKj : j 2 Jg
is a cone.
Proof. Let x 2 K and t 2]0; +1[. From x 2 K we deduce that
there exists a point i 2 J such that x 2 Ki : Then, by the fact that Ki
is a cone and Ki K; it follows that
tx 2 tKi = Ki K:
The proof is complete.

## Theorem 1.5.9 If K; a subset of Rn and L, a subset of Rm are cones,

then the set
K L Rn+m
is a cone, too.
Proof. Let x 2 K L; then there exist s 2 K and v 2 L such
that x = (s; v) : Then for each t 2]0 2; +1[; we have tx = t (s; v) =
(ts; tv) 2 K L; because K and L are cones.

## Theorem 1.5.10 If (Kj )j2J is a nonempty family of cones in Rn and

the set
K := \fKj : j 2 Jg
is nonempty, then the set K is a cone.
Proof. Let x 2 K and t 2]0; +1[: From x 2 K; we deduce that
x 2 Kj , for all j 2 J: Then tx 2 Kj ; for all j 2 J: Consequently,
tx 2 K and hence K is a cone.

## Denition 1.5.11 Let K be a nonempty subset of Rn : We say that

the set K is a convex cone if it is a cone and is convex.

## Example 1.5.13 The positive orthant of Rn ; i.e.the set

Rn+ = fx = (x1 ; :::; xn ) 2 Rn : xj = 0; for all j 2 f1; :::; ngg
is a convex cone.

## Example 1.5.14 The norm cone, i.e. the set

K = f(x; t) 2 Rn+1 : kxk 5 tg;
32 1. CONVEX ANALYSIS

is a convex cone.

## Example 1.5.15 The set K = Rn n f0g is a cone without being a

convex cone.

Example 1.5.16 The sets B(0; 1) and B[0; 1] in Rn are convex without
being convex cones.

## Example 1.5.17 Let a 2 Rn n f0g: The sets

H= (a) = fx 2 Rn : hx; ai = 0g;

## H< (a) = fx 2 Rn : hx; ai < 0g;

H(a) = fx 2 Rn : hx; ai = 0g

## Example 1.5.18 Let A 2 Rn m

: The set
K := fy 2 Rn : there exists x 2 Rm
+ such that y = A(x)g

is a convex cone.

K + K = K:

## Proof. Necessity. If K is a convex cone, then by Theorems 7

and 6 it follows that
K + K = (1=2)K + (1=2)K = (1=2 + 1=2)K = K:

## Su ciency. Let t 2]0; 1[: Then, in view of Theorem 6, we have

(1 t) K + tK = K + K = K:
Invoke for it Theorem 7.
5. CONES 33

## Example 1.5.20 The set

K = fx = (x1 ; x2 ; x3 ) 2 R3 : x1 = 0; x2 = 0; x1 x2 = (x3 )2 g

is a convex cone.
Indeed, it is clear that the set K is a cone. In order to prove that
K is a convex cone, we invoke Theorem 19. Let x = (x1 ; x2 ; x3 ) and
u = (u1 ; u2 ; u3 ) from K: Then
x1 + u1 = 0; x2 + u2 = 0;
and
(x1 + u1 )(x2 + u2 ) = x1 x2 + x2 u1 + x1 u2 + u1 u2 =
p
= x1 x2 + 2 x1 x2 u1 u2 + u1 u2 = (x3 )2 + 2x3 u3 + (u3 )2 =
= (x3 + u3 )2 ;

K +K K:
The inclusion
K K +K

K + K = K:

## Theorem 1.5.21 The nonempty subset K of Rn is a convex cone if

and only if the following two statements are true:
(i) tK = K; for all t 2]0; +1[;
(ii) K + K = K:
Proof. Invoke for it Theorems 6 and 19.

then the set
aK + bL

## is a convex cone, too.

Proof. Invoke for it Theorems 9 and 7.
34 1. CONVEX ANALYSIS

## Remark 1.5.23 If K and L are convex cones in Rn , then the set

K [ L may not be a convex cone.
Indeed the sets
K = fx = (x1 ; x2 ) 2 R2 : x2 = 0g = R f0g
and
L = fx = (x1 ; x2 ) 2 R2 : x1 = 0g = f0g R
are convex cones in R, but the reunion K [ L is not a convex cone
(K [ L is a cone, but K [ L is not convex).

## Theorem 1.5.24 If K and L are convex cones in Rn respectively Rm ,

then the set K L is a convex cone in Rn+m .
Proof. Invoke for it Theorems 11 and 9:

## Theorem 1.5.25 If K is a convex cone in Rn with nonempty interior,

then
K + intK intK:

Proof. Since
K + intK K + K;
by Theorem 19, it follows that
K + intK K:

Let now x 2 K; then the set fxg + intK is open, because intK is an
open set. Since
K + intK = [ffxg + int (K) : x 2 Kg;

we deduce that the set K +intK is open. Now by the fact that intK is,
with respect to the inclusion relation, the greatest open set containing
K; we deduce that
K + intK intK:
The theorem is proved.
It is a routine matter to show that the intersection of each family
of convex cones in Rn is a convex cone.

## Theorem 1.5.26 If (Kj )j2J is a nonempty family of convex cones in

Rn and the set
K := \fKj : j 2 Jg
is nonempty, then K is a convex cone, too.
5. CONES 35

## Denition 1.5.27 Let K be a nonempty subset of Rn . The intersec-

tion of all convex cones in Rn containing the set K is called the convex
cone generated by K (or, convex cone hull) and is denoted by conK:
Theorem 1.5.28 Let K be a nonempty subset of Rn : The following
statements are true:
10 The set conK is a convex cone.
20 The inclusion C conC holds:
30 If L K Rn and K is a convex cone, then conL K:
40 The set conK is, with respect to the inclusion relation, the small-
est convex cone in Rn containing K.
50 The equality
K = conK
holds if and only if the set K is a convex cone.
60 The equality
con (conK) = conK
holds:
Proof. The rst three assertions of the theorem are immediate
from Denition 27 and Theorem 26. Statements 40 and 50 follow from
statements 20 and 30 : Statement 60 is deduced from statements 10 and
50 :

## Theorem 1.5.29 Let K be a closed convex cone in Rn and u 2 Rn nC:

Then there exists a point a 2 Rn n f0g such that
hx; ai = 0; for all x 2 K
and
hu; ai < 0:

## Denition 1.5.30 Let K be a nonempty subset of Rn : We say that

the set K is a pointed convex cone if it is a convex cone and
K \ ( K) f0g:

## Example 1.5.31 The set

fx 2 Rn : x = 0g
is a pointed convex cone.
36 1. CONVEX ANALYSIS

pointed.

## Theorem 1.5.33 If K is a pointed convex cone in Rn ; then K n f0g

is a pointed convex cone, too.
Proof. Let x 2 K n f0g and t > 0: Since K is a cone we have that
tx 2 K. Moreover, tx 6= 0 because t 6= 0 and x 6= 0: Consequently,
K n f0g is a cone.
We establish now the convexity of the set K n f0g. To this end, let
x; u 2 K n f0g and t 2]0; 1[: Since C is convex, it follows that
(1 t) x + tu 2 K:

## We must show that

(1 t) x + tu 6= 0:
Assume that
(1 t) x + tu = 0;
then
u = ((t 1)=t)x
and u 2 C: From u 2 C; it follows that
u 2 K \ ( K) ;

## hence u = 0 because the cone K is pointed. But u 2 K n f0g; therefore

u 6= 0; which is a contradiction. Consequently, K n f0g is a convex
cone.
Since K n f0g C; it follows that
(K n f0g) \ ( (K n f0g)) K \ ( K) f0g;

## Denition 1.5.34 Let K be a nonempty subset of Rn . We say that

the set K is a polyhedral cone if there exist an integer number q = 1
and q points a1 ; :::; aq 2 Rn such that
K = \fH= (ak ) : k 2 f1; :::; qgg;
where
H= (ak ) = fx 2 Rn : hx; ak i = 0g; for all k 2 f1; :::; qg:

5. CONES 37

## Theorem 1.5.36 If K; a subset of Rn and L; a subset of Rm are

polyhedral cones, then K L Rn+m is a polyhedral cone.
Proof. Since K and L are polyhedral cones, it follows that there
exist two positive integer numbers p and q and p + q points a1 ; :::; ap 2
Rn ; b1 ; :::; bq 2 Rm ; such that
K = \fH= (ak ) : k 2 f1; :::; pgg
and
L = \fH= (bj ) : j 2 f1; :::; qgg:
Then
K L=
k
= \fH= (a ) : k 2 f1; :::; pgg \fH= (bj ) : j 2 f1; :::; qgg =
= \fH= (cs ) : s 2 f1; :::; p + qgg;

where (
(as ; 0); s 2 f1; :::; pg
s
c =
(0; bs p ); s 2 fp + 1; :::; p + qg:
Consequently, K L is a polyhedral cone.

## Theorem 1.5.37 Each polyhedral cone is a closed set.

Proof. First, we will show that, for each a 2 Rn ; the set
H= (a) = fx 2 Rn : hx; ai = 0g

## is closed. In order to prove this, let xk k2N be a sequence of points

from H= (a); convergent to a point x 2 Rn . Since, for each k 2 N, we
have
xk ; a = 0;
we deduce that
hx; ai = h lim xk ; ai = lim hxk ; ai = 0:
k!1 k!1

## Consequently, x 2 H= (a) and hence the set H= (a) is closed.

Let now K Rn be a polyhedral cone; then there exist an integer
number q = 1 and q points a1 ; :::; aq 2 Rn such that
K = \fH= (aj ) : j 2 f1; :::; qgg:

Since, for each j 2 f1; :::; qg; the set H= (aj ) is closed, we deduce that
the set K is closed.
38 1. CONVEX ANALYSIS

## 6. The Dual (Polar) of a Set

In this section, we show how to associate to each set in Rn another
set called its dual (or polar). The study of the relationship between
a set and its dual proves to be a fruitful concept in several areas of
convexity.

## Denition 1.6.1 Let K be a nonempty subset of Rn : The set

K = fu 2 Rn : hu; xi = 0; for all x 2 Kg

## Remark 1.6.2 The dual K of the nonempty subset K of Rn always

contains the point 0 2 Rn ; hence K 6= ;:

Example 1.6.3 The dual of the set f0g is the entire space Rn ; i.e
f0g = Rn :

Example 1.6.4 The dual of the set Rn is the set f0g; i.e.
(Rn ) = f0g:

## Example 1.6.5 The dual of the positive orthant of Rn is the positive

orthant of Rn , i.e.
Rn+ = Rn+ :

## Example 1.6.6 If K is the Euclidian norm cone, i.e.

K = f(x; t) 2 Rn+1 : kxk2 5 tg;
then
(K) = K:

## Example 1.6.7 If K is the Minkowski norm cone, i.e.

K = f(x; t) 2 Rn+1 : kxk1 5 tg;
then its dual is the Tchebishev norm cone, i.e.
f(x; t) 2 Rn+1 : kxk1 5 tg = f(x; t) 2 Rn+1 : kxk1 5 tg:

## Example 1.6.8 If a 2 Rn n f0g and

H(a) = fx 2 Rn : ha; xi = 0g;

## H= (a) = fx 2 Rn : ha; xi = 0g;

6. THE DUAL OF A SET 39

## H> (a) = fx 2 Rn : ha; xi > 0g;

then
(H(a)) = fu 2 Rn : there exists s 2 R such that u = sag = Rfag;

## Example 1.6.9 If r 2 ]0; +1[; then

(B (0; r)) = (B[0; r]) = f0g:

## Example 1.6.10 If K is a subspace of Rn ; then

K = K ? = fu 2 Rn : hu; xi = 0; for all x 2 Kg:

## Theorem 1.6.11 Let K be a nonempty subset of Rn : Then:

10 K ? K :
20 ( K) = (K ) :
0
3 If K L; then L K :
Proof. 10 Obviously,
K ? = fu 2 Rn : hu; xi = 0; for all x 2 Kg
fu 2 Rn : hu; xi = 0; for all x 2 Kg = K :
20 We have
( K) = fu 2 Rn : hu; xi = 0; for all x 2 Kg =
= fu 2 R : hu; vi = 0; for all v 2 Kg =
n

= fu 2 Rn : h u; vi = 0, for all v 2 Kg =
= f w 2 Rn : hw; vi = 0, for all v 2 Kg =
= fw 2 Rn : hw; vi = 0; for all v 2 Kg =
= (K ) :
30 Let u 2 L ; then
hu; xi = 0; for all x 2 L:
From K L we deduce that
hu; xi = 0; for all x 2 K:
Consequently, u 2 K :
40 1. CONVEX ANALYSIS

## Theorem 1.6.12 If K; a subset of Rn ; and L; a subset of Rm ; are

nonempty, then
(K L) = K L:
Proof. We have
(K L) = f(u; v) 2 Rn Rm : h(u; v); (x; y)i = 0;
for all (x; y) 2 K Lg =

## = f(u; v) 2 Rn Rm : hu; xi + hv; yi = 0;

for all x 2 K and y 2 Lg =

## = fu 2 Rn : hu; xi = 0; for all x 2 Kg

fv 2 Rm : hv; yi = 0; for all y 2 Lg =
=K L;

because 0 2 K \ L :

## Theorem 1.6.13 If K is a nonempty subset of Rn , then the set K

is a closed convex cone.
Proof. Let u 2 K and t 2]0; +1[: Then, for each x 2 K; we
have
htu; xi = t hu; xi = 0;

## hence tu 2 K : Consequently, K is a cone.

Let now u; v 2 K and t 2]0; 1[: Then, for each x 2 K; we have
h(1 t) u + tv; xi = (1 t) hu; xi + t hv; xi = 0;

because
hu; xi = 0; hv; xi = 0

## and t 2]0; 1[: Consequently, (1 t) u + tv 2 K ; hence K is a convex

set.
To nish the proof it actually remains to show that K is a closed
set. Let u 2clK ; then there exists a sequence uk k2N of points uk
from K ; (k 2 N) convergent to u: From uk 2 K ; (k 2 N) it follows
that
hu; xi = h lim uk ; xi = lim uk ; x = 0:
k!1 k!1

## Hence, u 2 K : Consequently, K is a closed set.

6. THE DUAL OF A SET 41

## The dual K of a nonempty subset K of Rn is a nonempty subset

of Rn : This suggests that we should examine the double dual K of
an arbitrary subset K of Rn and see how it is related to K: Since the
dual of each nonempty subset of Rn is always a closed convex cone, a
necessary condition for the equality of the sets K and K is that K
shoud be a closed convex cone. We now show that this condition is
also su cient and establish the exact relationship between K and K:
We dene
K = (K ) :
We can also dene the dual K of the K ; as
K = (K ) :

## Theorem 1.6.14 Let K be a nonempty subset of Rn : Then

10 The inclusion K K holds:
20 The equality K = K holds :
30 The equality K = (clK) holds:
Proof. 10 Let x 2 K: Then
hu; xi = hx; ui = 0; for all u 2 K ;
hence
x 2 (K ) = K :
Consequently, K K :
20 From statement 10 we obtain that K K : According to state-
ment 30 of Theorem 11, we deduce that
K K :

K (K ) =K

## follows from statement 10 :

30 From K clK and statement 30 of Theorem 11; we deduce that
(clK) K :

## In order to establish the converse inclusion, let x 2clK: Then, there

exists a sequence xk k2N of points xk 2 K; (k 2 N) such that
lim xk = x:
k!1
42 1. CONVEX ANALYSIS

Since
hu; xi = hu; lim xk i = lim hu; xk i = 0;
k!1 k!1

because
hxk ; ui = 0; for all k 2 N;
we obtain that u 2 (clK) : Consequently,
K (clK) :

## Theorem 1.6.15 Let K and L be two nonempty subsets of Rn : Then:

10 The equality (K [ L) = K \ L holds:
20 If K \ L 6= ;; then K [ L (K \ L) :
30 The inclusion K \ L (K + L) holds:
40 If 0 2 K \ L; then (K + L) = K \ L :
50 If K \ L 6= ;; then the inclusion K + L (K \ L) holds.
Proof. 10 From K K [ L; in view of Theorem 11, we deduce
that
(K [ L) K :
In a similar way, we obtain that
(K [ L) L:
Consequently,
(K [ L) K \L :
Let now u 2 K \ L and x 2 K [ L: If x 2 K; then hu; xi = 0;
because u 2 K and if x 2 L; then also hu; xi = 0; because u 2 L :
Consequently, u 2 (K [ L) and the equality is established.
20 From K \ L K; according to Theorem 11, we deduce that
K (K \ L) :
Analogously, we obtain
L (K \ L) :
Consequently,
K [L (K \ L) :
30 Let u 2 K \ L : We will show that
hu; xi = 0; for all x 2 K + L:
6. THE DUAL OF A SET 43

## Let x 2 K + L; then there exist s 2 K and t 2 L such that x = s + t.

From u 2 K \ L ; s 2 K; t 2 L; we deduce that hu; si = 0 and
hu; ti = 0: It follows that
hu; xi = hu; si + hu; ti = 0:

Consequently, u 2 (K + L) :
40 If 0 2 K; then
L = f0g + L K +L
and hence
(K + L) L:
Similarly,
(K + L) K :
Consequently, if 0 2 K \ L; then
(K + L) K \L :

## Now, by statement 30 , we get statement 40 :

50 Let u 2 K + L ; then there exist s 2 K and t 2 L such that
u = s + t: From this we deduce that for each x 2 K \ L;
hu; xi = hs; xi + ht; xi = 0;
because
hs; xi = 0 and ht; xi = 0:
Consequently, u 2 (K \ L) and the proof is complete.

not hold.

## Theorem 1.6.17 (bipolar theorem) The nonempty subset C of Rn is

a closed convex cone if and only if the equality
K=K
holds:
Proof. Necessity. Assume that the set K is a closed convex cone.
In view of statement 10 of Theorem 14, it su ces to show that K K:
Suppose by contradiction that there exists a point x 2 K such that
x2= K: Then according to Theorem 29, there exists a point a 2 K nf0g
such that
ha; xi < 0:
Consequently, x 2
= K , but this is impossible.
44 1. CONVEX ANALYSIS

## Su ciency. If K = K ; then in view of Theorem 13, the set K is

a closed convex cone.

## Theorem 1.6.18 If the nonempy subset K of Rn is a convex cone,

then
clK = K :

Proof. The set clK being a closed convex cone, in view of Theo-
rem 17, we have that
clK = (clK) :
Since
K = (clK) ;
using statement 30 of Theorem 14, the proof is complete.

## Theorem 1.6.19 Let K and L be two closed convex cones in Rn :

Then
(K \ L) = cl (K + L ) :

Proof. The sets K and L being closed convex cones, from Theo-
rem 17, it follows that
K=K and L = L :
Since
02K \L ;
according to Theorems 14, 15, and 17, we have
(K \ L) = (K \ L ) = (K + L ) = cl (K + L ) :

## Theorem 1.6.20 Let K be a cone in Rn ; and u 2 Rn be such that the

function f : K ! R dened by
f (x) = hu; xi; for all x 2 K;
is bounded from below. Then u 2 K :
Proof. Assume that u 2 = K : Then there exists x 2 K such that
hu; xi < 0: Since K is a cone, the point tx 2 K; for all t 2]0; +1[: But
then f (tx) = hu; txi = thu; xi tends to 1 when t tends to +1; hence
f is not bounded from below, which contradicts our assumption.
6. THE DUAL OF A SET 45

## Dual cones to pre-images of convex cones under linear transforma-

tions can be calculated in an explicit form.

## Theorem 1.6.21 Let L be a closed convex cone in Rm ; A be an m n

matrix, and
K = fx 2 Rn : A (x) 2 Lg:
Then K is a closed convex cone, and
K = fy 2 Rn : there exists v 2 L such that y = AT (v)g =
= fAT (v) : v 2 L g:
Proof. The set K is a closed convex cone by construction.
Let
N = fy 2 Rn : there exists v 2 L such that y = AT (v)g =
= fAT (v) : v 2 L g::
Obviously, the set N is closed and convex.
Now, if y 2 N; then there exists v 2 L such that y = AT (v) and
hence, for each x 2 K; we have
hy; xi = AT (v) ; x = hv; A (x)i = 0;
because A (x) 2 L: It follows that
N K :
In order to prove that K N; we assume, by contradiction that
there exists a point u 2 K n N: By Theorem ???, there exist a point
a 2 Rn n f0g and a real number " > 0 such that
ha; ui + " 5 ha; xi ; for all x 2 N;
which implies
(1.6.1) ha; ui + " 5 a; AT (v) ; for all v 2 L ;
It follows that the function f : L ! R dened by
f (v) = a; AT (v) = hA (a) ; vi ; for all v 2 L ;
is bounded from below. Then, by Theorem 20, we have A (a) 2 (L ) =
L : Since L is a closed convex cone, bipolar theorem (Theorem 17)
implies A (a) 2 L and therefore a 2 K: But u 2 K ; and hence ha; ui =
0:
On the other hand, from (1.6.1); for v = 0 2 L we obtain
ha; ui + " 5 a; AT (0) = 0;
which contradicts ha; ui = 0:
46 1. CONVEX ANALYSIS

## Theorem 21 allows us to explicitly calculate dual cones of cones

dened by system of linear inequalities.
For example, setting
L = fy 2 Rm : y = 0g = Rm
+

## Theorem 1.6.22 (Farkas theorem) Let A be an m n matrix, and

K = fx 2 Rn : A (x) = 0g:
Then
K = fy 2 Rn : there exists v 2 Rm T
+ such that y = A (v)g:

## The above fact is frequently formulated as an alternative: exactly

one of the following two systems has a solution, either
A (x) = 0
(i)
hc; xi < 0
or
AT (v) = c
(ii)
v = 0:
Indeed, system (i) is solvable if and only if c 2 K ; which is equiv-
alent to (ii) ; by Farkas theorem.

## Theorem 1.6.23 Let K be a convex cone in Rn : If x 2 intK; then

hu; xi > 0; for all u 2 K n f0g:

## Proof. Since x 2 K; we have hu; xi = 0; for all u 2 K n f0g:

Assume that there exists u 2 K n f0g such that hu; xi = 0: From x 2
intK; we deduce that there exists r 2]0; +1[ such that v = x ru 2 K:
Then
hv; ui = hx; ui r kuk2 = r kuk2 < 0;
a contradiction with u 2 K n f0g:

## Theorem 1.6.24 Let K1 ; K2 ; :::; Kp be p convex cones in Rn : If

K1 \ K2 \ ::: \ Kp = ;;
then there exist u1 2 K1 ; u2 2 K2 ; :::; up 2 Kp ; not all equal 0; such
that
u1 + u2 + ::: + up = 0:
6. THE DUAL OF A SET 47

## Proof. Let us dene two cones in Rn Rn ::: Rn = Rpn :

L = fv = v 1 ; :::; v p 2 Rpn : v 1 2 K1 ; :::; v p 2 Kp g = K1 ::: Kp ;
K = fw = (x; x; :::; x) 2 Rpn : x 2 Rn g:
Since K1 \ K2 \ ::: \ Kp = ;; we deduce that L \ K = ;: Then, by
Theorem ??? there exists u = (u1 ; :::; up ) 2 Rpn n f0g such that
hu; wi 5 hu; vi ;
for all v = (v 1 ; :::; v p ) 2 L and w = (x; x; :::; x) 2 K: From this it
follows that
(1.6.2) u1 + :::up ; x 5 u1 ; v 1 + ::: + hup ; v p i ;
for all x 2 Rn , and all v 1 2 K1 ; :::; v p 2 Kp :
Setting x = 0 in (1.6.2) ; we obtain that
u1 ; v 1 + ::: + hup ; v p i = 0; for all v 1 2 K1 ; :::; v p 2 Kp ;
hence the function f : K1 ::: Kp ! R dened by
f v 1 ; :::; v p = u1 ; v 1 + ::: + hup ; v p i ; for all v 1 2 K1 ; :::; v p 2 Kp ;
is bounded from below, which implies that each of the functions f1 :
K1 ! R, ..., fp : Kp ! R dened by
fi v i = ui ; v i ; for all v i 2 Ki ;
for i 2 f1; :::; pg; is bounded from below. Then, by Theorem 20, it
follows that u1 2 K1 ; :::; up 2 Kp :
On the other hand, (1.6.2) implies that the function g : Rn ! R
dened by
g (x) = u1 + ::: + up ; x ; for all x 2 Rn ;
is bounded from above, which is possible only when u1 + :::up = 0: The
theorem is proved.

## Theorem 1.6.25 Let K1 ; K2 ; :::; Kp be p convex cones in Rn : If

(1.6.3) K1 \ (intK2 ) \ (intK3 ) \ ::: \ (intKp ) 6= ;;
then
(K1 \ K2 \ ::: \ Kp ) = K1 + K2 + ::: + Kp :
Proof. Let
K = K1 \ K2 \ ::: \ Kp :
If u 2 K1 ; u 2 K2 ; :::; up 2 Kp ; then, for each x 2 K we have
1 2

## u1 + u2 + ::: + up ; x = u1 ; x + u2 ; x + ::: + hup ; xi = 0;

48 1. CONVEX ANALYSIS

so u1 + u2 + ::: + up 2 K : Consequently
(1.6.4) K1 + K2 + ::: + Kp K :
In order to prove the inverse inclusion of (1.6.4) ; let u 2 K :
Let us dene the cone
L = fx 2 Rn : hu; xi < 0g:
We have
L = fs 2 Rn : there exists t 2 [0; +1[ such that s = tug:
(see Example 8):
Since hu; xi = 0; for all x 2 L; it follows that L \ K = ;; and hence
L \ K1 \ K2 \ ::: \ Kp = ;: Then, by Theorem 24, there exist the points
s 2 L ; u1 2 K1 ; u2 2 K2 ; :::; up 2 Kp ; not all equal zero, such that
(1.6.5) s + u1 + u2 + ::: + up = 0:
From s 2 L it follows that there exists a real number t = 0 such that
s = tu: Then (1.6.5) becomes
(1.6.6) tu = u1 + u2 + ::: + up :
We will show that t > 0: Assume that t = 0; it follows that
(1.6.7) u1 + u2 + ::: + up = 0:
The hypothesis (1.6.3) ; implies there exists a point
x 2 K1 \ (intK2 ) \ (intK3 ) \ ::: \ (intKp ) :
Then
(1.6.8) 0 = u1 + u2 + ::: + up ; x = u1 ; x + u2 ; x + ::: + hup ; xi :
Since hu1 ; xi = 0; hu2 ; xi = 0; :::; hup ; xi = 0; from (1.6.8) ; we deduce
that
ui ; x = 0; for all i 2 f1; :::; pg:
But, x 2 (intK2 )\(intK3 )\:::\(intKp ) ; and then Theorem 23, implies
ui = 0; for all i 2 f1; :::; pg: Now, Equality (1.6.7) yiels u1 = 0; and
hence u1 = u2 = ::: = up = 0; a contradiction.
Thus t > 0: Dividing both sides of (1.6.6) by t 6= 0; we get
1 1 1
u = u1 + u2 + ::: + up 2 K1 + K2 + ::: + Kp :
t t t
Consequently K K1 + K2 + ::: + Kp : Theorem is proved.
6. THE DUAL OF A SET 49

Remark 1.6.26 When all cones are polyhedral, the regularity as-
sumption (1.6.3) is not needed.

## Theorem 1.6.27 Let L Rn and M Rm be two closed convex

cones, A be an m n matrix, and
K = fx 2 L : A (x) 2 M g:
If
(1.6.9) 0 2 intfA (x) y : x 2 L; y 2 M g;
then
K = L + fAT (v) : v 2 M g:
Proof. Let us dene the set
N = fx 2 Rn : A (x) 2 M g:
Then, by Theorem 21, we have
N = fAT (v) : v 2 M g =
= fy 2 Rn : there exists vM such that y = AT (v)g:
Obviously
K = L \ N;
and hence
L +N (L \ N ) = K :
In order to prove L + N K ; let u 2 K :
Let us dene the cone
C = fx 2 Rn : hu; xi < 0g:
Then
C = fs 2 Rn : there exists t 2 [0; +1[ such that s = tug
(see Example 8):
On the other hand C \ K = ;; because hu; xi = 0; for all x 2 K:
Then, by Theorem 24, there exist s 2 C ; w 2 L ; z 2 N ; not all
equal 0; such that
(1.6.10) s + w + z = 0:
From z 2 N , it follows that there exists u 2 M so that z = AT (u) ;
and from s 2 C ; it follows that there exists t 2 [0; +1[ such that
s = tu:
We will show that t > 0:
50 1. CONVEX ANALYSIS

## Assume that t = 0: Then (1.6.10) yields w = z: It follows that

AT (v) 2 L : Thus, for each x 2 L we obtain
x; AT (v) = 0;
that is
x; AT (v) 5 0:
As v 2 M ; we also have hy; vi = 0; for all y 2 M: Hence
hA (x) y; vi = hA (x) ; vi hy; vi = x; AT (v) hy; vi 5 0;
for all x 2 L and y 2 M: Since the set
fA (x) y : x 2 L; y 2 M g
containts a neighborhood of 0 by assumption, we get v = 0: This yields
z = 0 and w = 0 which is a contradiction.
Consequently t > 0: Dividind (1.6.10) by t 6= 0 we obtain
1 1 1 1
u = w + AT (v) 2 L + N = L + N :
t t t t
Thus K L + M : The theorem is proved.

Remark 1.6.28 When all cones are polyhedral, the regularity as-
sumption (1.6.9) is not needed.

7. Convex Functions
The convex functions and the generalizations of convex functions
play an important role in the study of optimization problems.
In this section we present some of the basic properties of con-
vex, concave, pseudoconvex, pseudoconcave, quasiconvex, quasiconcave
functions and obtain some fundamental theorems involving these func-
tions. These fundamental theorems will be used to derive the important
necessary optimality conditions of the optimization problem.

## Denition 1.7.1 Let M be a nonempty subset of Rn ; x0 be a point

from M , K be a subset of Rm and f : M ! Rm be a function.
We say that the function f is convex at the point x0 with respect
to K if, for each x 2 M n fx0 g and each t 2]0; 1[ with the property that
(1 t) x0 + tx 2 M;
we have
(1 t) f x0 + tf (x) f (1 t)x0 + tx 2 K:

## We say that the function f is concave at the point x0 with respect

to K if the function ( f ) is convex at the point x0 with respect to K:
7. CONVEX FUNCTIONS 51

## Denition 1.7.2 Let M be a nonempty convex subset of Rn ; K be a

nonempty subset of Rm and f : M ! Rm be a function.
We say that the function f is convex on M with respect to K if the
function f is convex at each x0 2 M with respect to K:
We say that the function f is concave on M with respect to K if
the function ( f ) is convex on M with respect to K:

## Example 1.7.3 Let A 2 Rm n and b 2 Rm : The function f : Rn !

Rm dened by
f (x) = A(x) + b, for all x 2 Rn ;
is convex on Rn with respect to K = f0g:
Indeed, for each x0 ; x 2 Rn and each t 2]0; 1[; we have
(1 t) f x0 + tf (x) f (1 t)x0 + tx = 0:

## Example 1.7.4 The function f : Rn ! R dened by

f (x) = kxk ; for all x 2 Rn ;
is convex on Rn with respect to R+ :
Indeed, for each x0 ; x 2 Rn and each t 2]0; 1[; we have
(1 t) x0 + t kxk (1 t) x0 + tx =
= (1 t) x0 + t kxk (1 t) x0 t kxk = 0;
it follows that
(1 t) f x0 + tf (x) f (1 t)x0 + tx 2 R+ :

## Therefore, the function f is convex on Rn with respect to R+ :

Many close connections exist between convex functions and convex
sets. Here we study one of these; others will be discussed later.

## Example 1.7.5 If M is a nonempty convex subset of Rn ; then the

distance function dM : Rn ! R to M dened by
dM (x) = inffkx uk : u 2 M g; for all x 2 Rn

## is convex on Rn with respect to R+ :

Indeed, let x0 ; x 2 Rn and t 2]0; 1[: Then, for each " > 0 there exist
two points u0 ; u 2 M such that
x0 u0 < dM (x0 ) + "
and
kx uk < dM (x) + ":
52 1. CONVEX ANALYSIS

## Since M is convex, we have

(1 t)u0 + tu 2 M
and so
dM ((1 t)x0 + tx) 5 (1 t)x0 + tx (1 t)u0 tu
5 (1 t) x0 u0 + t kx uk
5 (1 t)(dM (x0 ) + ") + t(dM (x) + ")
= (1 t)dM (x0 ) + tdM (x) + ":
But " > 0 is arbitrary, whence
dM ((1 t)x0 + tx) 5 (1 t)dM (x0 ) + tdM (x):
Consequently,
(1 t)dM (x0 ) + tdM (x) dM ((1 t)x0 + tx) 2 R+ ;
which shows that dM is a convex function on Rn with respect to K =
R+ .
Remark 1.7.6 Let M be a nonempty subset of Rn ; x0 2 M and
f : M ! Rm be a function. Let K be a nonempty subset of Rm and
L K:
If the function f is convex (respectively concave) at x0 with respect
to K; then the function f is convex (respectively concave) at x0 with
respect to L:
The following four theorems give some other connections that exist
between the convex functions and convex sets.
Theorem 1.7.7 Let M be a nonempty convex subset of Rn ; K be a
convex cone in Rm and f : M ! Rm be a function.
10 If the function f is convex on M with respect to K, then the set
epi (f ) = f(x; u) 2 M Rm : u f (x) 2 Kg
is convex.
20 If the function f is concave on M with respect to K, then the set
hyp (f ) = f(x; u) 2 M Rm : u + f (x) 2 Kg
is convex.
Proof. 10 Let (x0 ; u0 ) ; (x; u) 2 epi (f ) and t 2]0; 1[: Then
u0 f x0 2 K and u f (x) 2 K:
Since M is a convex set we have
(1 t) x0 + tx 2 M:
7. CONVEX FUNCTIONS 53

## Furthermore, the function f being convex on M with respect to K;

we have
(1 t) u0 + tu f ((1 t)x0 + tx) =
= (1 t) [u0 f (x0 )] + t [u f (x)] +
+ (1 t) f (x0 ) + tf (x) f ((1 t) x0 + tx) 2
2 (1 t) K + tK + K = K;

hence
(1 t) x0 ; u0 + t (x; u) = (1 t) x0 + tx; (1 t) u0 + tu 2 epi (f ) :

## Consequently, the set epi (f ) is convex.

Analogously one proves the statement 20 :

## Theorem 1.7.8 Let M be a nonempty subset of Rn ; K be a subset of

Rm with the property that 0 2 K and f : M ! Rm be a function.
10 If the set
epi (f ) = f(x; u) 2 M Rm : u f (x) 2 Kg

## is convex, then the set M is convex and the function f is convex on M

with respect to K:
20 If the set
hyp (f ) = f(x; u) 2 M Rm : u + f (x) 2 Kg

## is convex, then the set M is convex and the function f is concave on

M with respect to K:
Proof. 10 Assume that the set epi (f ) is convex and let x0 ; x 2 M;
x0 6= x and t 2]0; 1[: Since 0 2 K we have (x0 ; f (x0 )) and (x; f (x))
belong to epi (f ). Then
((1 t) x0 + tx; (1 t) f (x0 ) + tf (x)) =
= (1 t) (x0 ; f (x0 )) + t (x; f (x)) 2 epi (f ) ;

## because epi (f ) is convex. From this it follows that

(1 t) x0 + tx 2 M
and
(1 t) f x0 + tf (x) f (1 t)x0 + tx 2 K;
hence the set M is convex and the function f is convex on M with
respect to K:
Analogously one proves the statement 20 :
54 1. CONVEX ANALYSIS

## Theorem 1.7.9 Let M be a nonempty convex subset of Rn ; K be a

convex cone in Rm with property that 0 2 K and f : M ! Rm be a
function.
10 The function f is convex on M with respect to K if and only if
the set
epi (f ) = f(x; u) 2 M Rm : u f (x) 2 Kg
is convex.
20 The function f is concave on M with respect to K if and only if
the set
hyp (f ) = f(x; u) 2 M Rm : u + f (x) 2 Kg
is convex.

## Theorem 1.7.10 Let M be a nonempty convex subset of Rn ; K be a

convex cone in Rm and f : M ! Rm be a function. Then the following
statements are true:
10 If the function f is convex on M with respect to K; then, for
each u 2 Rm ; the set

Au = fx 2 M : u f (x) 2 Kg

is convex.
20 If the function f is concave on M with respect to K; then, for
each u 2 Rm ; the set

Bu = fx 2 M : u + f (x) 2 Kg

is convex.

## Proof. 10 Let u 2 Rm : Let x0 ; x 2 Au and t 2]0; 1[: The set M

being convex it follows that

(1 t) x0 + tx 2 M:

If x0 = x, obviously

u f (1 t) x0 + tx = u f (x0 ) 2 K:
7. CONVEX FUNCTIONS 55

## Let x0 6= x: The function f being convex on M with respect to K; and

K being a convex cone, we have
u f ((1 t) x0 + tx) =
= (1 t) [u f (x0 )] + t[u f (x)]+
0
+ (1 t) f (x ) + tf (x) f ((1 t) x0 + tx) 2
2 (1 t) K + tK + K =
= K + K = K;

hence
(1 t) x + tv 2 Au :

## Consequently, the set Au is convex.

Analogously we prove the statement for the set Bu :

## Below we examine how convex (respectively concave) functions be-

have with respect to the operations of addition and scalar multiplica-
tion.

## Theorem 1.7.11 Let M be a nonempty subset of Rn ; x0 2 M; K

be a convex cone in Rm and f : M ! Rm and g : M ! Rm be two
functions.
10 If the functions f and g are convex at x0 with respect to K; then
for each a; b 2 [0; +1[ with a + b > 0; the function af + bg is convex
at x0 with respect to K:
20 If the functions f and g are concave at x0 with respect to K; then
for each a; b 2 [0; +1[ with a + b > 0; the function af + bg is concave
at x0 with respect to K:

Proof. We prove the theorem only in the case when the functions
f and g are convex at x0 with respect to K: Analogously one proves it
when f and g are concave at x0 with respect to K:
Let x 2 M n fx0 g and t 2]0; 1[ for which
(1 t) x0 + tx 2 M:
Then
(1 t) f x0 + tf (x) f (1 t)x0 + tx 2 K
and
(1 t) g x0 + tg (x) g (1 t)x0 + tx 2 K:
56 1. CONVEX ANALYSIS

## From this it follows that

(1 t) (af + bg) (x0 ) + t(af + bg) (x) (af + bg) ((1 t)x0 + tx) =
= t [(1 t) f (x0 ) + tf (x) f ((1 t)x0 + tx)] +
+ b [(1 t) g (x0 ) + tg (x) g ((1 t)x0 + tx)] 2
2 aK + bK = K;
the last equality holds in view of Theorem 21. Consequently, the func-
tion af + bg is convex at x0 with respect to K:
From Theorem 11, it follows.

## Theorem 1.7.12 Let M be a nonempty convex subset of Rn ; K be

a convex cone in Rm and f : M ! Rm and g : M ! Rm be two
functions.
10 If the functions f and g are convex on M with respect to K, then
for each a; b 2 [0; +1[ with a + b > 0; the function af + bg is convex
on M with respect to K:
20 If the functions f and g are concave on M with respect to K,
then for each a; b 2 [0; +1[ with a + b > 0; the function af + bg is
concave on M with respect to K:
Proof. Apply Theorem 11.

## Theorem 1.7.13 Let M be a nonempty subset of Rn ; x0 be an interior

point of M; K be a closed convex cone in Rm ; and f : M ! Rm be a
dierentiable function at x0 :
10 If the function f is convex at x0 with respect to K; then
(1.7.1) f (x) f (x0 ) rf (x0 ) (x x0 ) 2 K;
for all x 2 M:
20 If the function f is concave at x0 with respect to K; then
(1.7.2) f (x0 ) f (x) rf (x0 ) (x0 x) 2 K;
for all x 2 M:
Proof. 10 Obviously relation (1.7.1) holds for x = x0 : Let x 2
M n fx0 g: Since x0 is an interior point of M; there exists a real number
r 2 ]0; +1[ such that
B x0 ; r = fx 2 Rn : x x0 < rg M:

## Then, for each t 2]0; 1[ for which t < r= kx x0 k ; we have

(1.7.3) w = (1 t) x0 + tx = x0 + t x x0 2 B x0 ; r M:
7. CONVEX FUNCTIONS 57

The function f is convex at x0 with respect to K; then, for each t 2]0; 1[;
t < r= kx x0 k ; we have
(1 t) f (x0 ) + tf (x) f (1 t) x0 + tx 2 K:

## From this, it follows that

1 1
(1.7.4) f (x) f x0 f x0 + t(x x0 ) f x0 2 K = K;
t t
for all t 2]0; 1[; t < r= kx x0 k ; the last equality in (1.7.4) holds
because K is a convex cone.
Furthermore, the function f is dierentiable at x0 ; then there exists
1
lim ff x0 + t(x x0 ) f x0 g = rf (x0 ) (x x0 ):
t!0 t
Then, taking the limit of expression (1.7.4) and taking into account
the fact that K is a closed set, we deduce that
(1.7.5) f (x) f x0 rf (x0 ) (x x0 ) 2 K:

## Theorem 1.7.14 Let M be a nonempty open convex subset of Rn ;

K be a closed convex cone in Rm and f : M ! Rm be a dierentiable
function on M:
10 If the function f is convex on M with respect to K; then

## f (x) f (x0 ) rf (x0 ) (x x0 ) 2 K;

for all x; x0 2 M:
20 If the function f is concave on M with respect to K; then

## f (x0 ) f (x) rf (x0 ) (x0 x) 2 K;

for all x; x0 2 M:
Proof. Apply Theorem 13.

## Theorem 1.7.15 Let M be a nonempty open convex subset of Rn ; K

be a nonempty convex subset of Rm and f : M ! Rm be a dierentiable
function on M:
10 If
f (x) f (x0 ) rf (x0 ) (x x0 ) 2 K;
58 1. CONVEX ANALYSIS

## for all x; x0 2 M; x = 6 x0 ; then the function f is convex on M with

respect to K:
20 If
f (x0 ) f (x) rf (x0 ) (x0 x) 2 K;
for all x; x0 2 M; x 6= x0 ; then the function f is concave on M with
respect to K:
Proof. 10 Let x; x0 2 M; x 6= x0 and t 2]0; 1[: Since the set M is
convex, the point
u = (1 t) x0 + tx 2 M:
Moreover, x0 6= u 6= x: Then we have
f (x0 ) f (u) t [rf (u)] (x0 x) 2 K
and
f (x) f (u) + (1 t) [rf (u)] (x0 x) 2 K:
From this, we deduce that
(1 t) f (x0 ) + tf (x) f ((1 t) x0 + tx) =
= (1 t) ff (x0 ) f (u) t [rf (u)] (x0 x)+
+tff (x) f (u) + (1 t) [rf (u)] (x0 x) 2
2 (1 t) K + tK = K:

## Consequently, the function f is convex on M with respect to K:

Statement 20 is established in a similar way.
From Theorems 14 and 15 follows:

## Theorem 1.7.16 Let M be a nonempty open convex subset of Rn ;

K be a closed convex cone in Rm and f : M ! Rm be a dierentiable
function on M: Then the following statements are true:
10 The function f is convex on M with respect to K if and only if

## f (x) f (x0 ) rf (x0 ) (x x0 ) 2 K;

for all x; x0 2 M:
20 The function f is concave on M with respect to K if and only if

## f (x0 ) f (x) rf (x0 ) (x0 x) 2 K;

for all x; x0 2 M

## Example 1.7.17 Let A 2 Rn n be a positive semidenite matrix and

b 2 Rn : Then the function f : Rn ! R dened by
8. STRICTLY CONVEX FUNCTIONS 59

1
f (x) = hA(x); xi + b(x); for all x 2 Rn
2
is convex on Rn with respect to K = R+ , because the function f is
dierentiable on Rn and
1
f (x) f (x0 ) rf (x0 ) x x0 = hA x x0 ; x x0 i = 0;
2
for all x; x0 2 Rn :
8. Strictly Convex Functions
If K Rm has the property that 0 2 K; then obviously the im-
plication from the denitions of the convex, concave functions are true
for x = x0 and t = 0 and t = 1: Consequently, in this case, it is not
necessary to require that x 6= x0 ; t 6= 0 and t 6= 1:
Denition 1.8.1 Let M be a nonempty subset of Rn ; x0 be a point
from M; K be a subset of Rm with nonempty interior and f : M ! Rm
be a function.
We say that the function f is strictly convex at x0 with respect to
K; if the function f is convex at x0 with respect to intK:
We say that the function f is strictly concave at x0 with respect to
K; if the function f is concave at x0 with respect to intK:
Denition 1.8.2 Let M be a nonempty convex subset of Cn ; K be a
subset of Rm with nonempty interior and f : M ! Rm be a function.
We say that the function f is strictly convex on M with respect
to K if the function f is strictly convex at each point x0 2 M with
respect to K:
We say that the function f is strictly concave on M with respect
to K if the function f is strictly concave at each point x0 2 M with
respect to K:
Remark 1.8.3 From Remark 6, it follows that if the function f :
M ! Rm is strictly convex (respectively strictly concave) at x0 2 M
with respect to K Rm ; then the function f is convex (respectively
concave) at x0 with respect to K:
Bellow we examine how strictly convex (respectively strictly concave)
functions behave with respect to the operations of addition and scalar
multiplication.
Theorem 1.8.4 Let M be a nonempty subset of Rn ; x0 be a point
from M; K be a convex cone in Rm with nonempty interior and a;
b 2 [0; +1[ with b > 0:
60 1. CONVEX ANALYSIS

## 10 If the function f : M ! Rm is convex at x0 with respect to K

and the function g : M ! Rm is strictly convex at x0 with respect to
K; then the function af + bg is strictly convex at x0 with respect to K:
20 If the function f : M ! Rm is concave at x0 with respect to K
and the function g : M ! Rm is strictly concave at x0 with respect to
K; then the function af + bg is strictly concave at x0 with respect to K:
Proof. The proof of this theorem is similar to the proof of Theo-
rem 11.

## Theorem 1.8.5 Let M be a nonempty subset of Rn ; x0 be an interior

point of M; K be a closed convex cone of Rm with nonempty interior
and f : M ! Rm be a dierentiable function at x0 :
10 If the function f is strictly convex at x0 with respect to K; then
(1.8.1) f (x) f (x0 ) [rf (x0 )](x x0 ) 2 intK;

## for all x 2 M n fx0 g:

20 If the function f is strictly concave at x0 with respect to K; then
f (x0 ) f (x) [rf (x0 )](x0 x) 2 intK;

## for all x 2 M n fx0 g:

Proof. 10 Let x 2 M n fx0 g: Since x0 is an interior point of M;
there exists a real number t 2]0; 1[ such that
u = (1 t) x0 + tx = x0 + t x x0 2 M:

## Evidently, x0 6= u 6= x: Since f is strictly convex at x0 with respect

to K; we have
(1 t)f (x0 ) + tf (x) f (u) 2 intK:

## In view of Remark 3, the function f is convex at x0 with respect to K;

then according to Theorem 13
f (u) f (x0 ) [rf (x0 )](u x0 ) 2 K:

Now, we have
f (x) f (x0 ) [rf (x0 )](x x0 ) =
= 1t f(1 t) f (x0 ) + tf (x) f (u)g +
+ 1t ff (u) f (x0 ) [rz f (x0 )](u x0 )g 2
2 (1=t) intK + (1=t) K = intK + K = intK:

## 20 Statement 20 is established in a similar way.

8. STRICTLY CONVEX FUNCTIONS 61

## Theorem 1.8.6 Let M be a nonempty open convex subset of Rn ; K be

a closed convex cone in Rm with nonempty interior and f : M ! Rm
be a dierentiable function on M:
10 If the function f is strictly convex on M with respect to K; then
f (x) f (x0 ) [rf (x0 )](x x0 ) 2 intK;

for all x; x0 2 M; x 6= x0 :
20 If the function f is strictly concave on M with respect to K; then
f (x0 ) f (x) [rf (x0 )](x0 x) 2 intK;

for all x; x0 2 M; x 6= x0 :

## Theorem 1.8.7 Let M be a nonempty open convex subset of Rn ; K

be a convex subset of Rm with nonempty interior and f : M ! Rm be
a dierentiable function on M:
10 If
f (x) f (x0 ) [rf (x0 )](x x0 ) 2 intK;
for all x; x0 2 M; x 6= x0 ; then the function f is strictly convex on M
with respect to K:
20 If
f (x0 ) f (x) [rf (x0 )](x0 x) 2 intK;
for all x; x0 2 M; x 6= x0 ; then the function f is strictly concave on M
with respect to K:
Proof. 10 Let x; x0 2 M; x 6= x0 and t 2]0; 1[: Since the set M is
convex, the point
u = (1 t)x0 + tx 2 M:
Moreover x0 6= u 6= x: Then we have
f (x0 ) f (u) t[rf (u)](x0 x) 2 intK;
and
f (x) f (u) + (1 t)[rf (u)](x0 x) 2 intK:
From this, we obtain that
(1 t)f (x0 ) + tf (x) f ((1 t)x0 + tx) =
= (1 t)ff (x0 ) f (u) t[rz f (u)](x0 x)g+
+tff (x) f (u) + (1 t)[rf (u)](x0 x)g 2

## 2 (1 t)intK + tintK = intK:

62 1. CONVEX ANALYSIS

## Consequently, the function f is strictly convex on M with respect to

K:
One proves statement 20 in a similar way.
From Theorems 6 and 7 follows.

## Theorem 1.8.8 Let M be a nonempty open convex subset of Rn ; K be

a closed convex cone in Rm with nonempty interior and f : M ! Rm
be a dierentiable function on M:
10 The function f is strictly convex on M with respect to K if and
only if
f (x) f (x0 ) [rf (x0 )](x x0 ) 2 intK;
for all x; x0 2 M; x 6= x0 :
20 The function f is strictly concave on M with respect to K; if and
only if
f (x0 ) f (x) [rf (x0 )](x0 x) 2 intK;
for all x; x0 2 M; x 6= x0 :

9. Pseudoconvex Functions

## Denition 1.9.1 Let M be a nonempty subset of Rn ; x0 be an interior

point of M; K be a nonempty subset of Rm and f : M ! Rm be a
dierentiable function at x0 :
We say that the function f is pseudoconvex at x0 with respect to
K if, for each x 2 M n fx0 g such that
rf (x0 ) x x0 2 K;
we have
f (x) f x0 2 K:

## We say that the function f is pseudoconcave at x0 with respect to

K if the function ( f ) is pseudoconvex at x0 with respect to K:

## Denition 1.9.2 Let M be a nonempty open subset of Rn ; K be a

nonempty subset of Rm and f : M ! Rm be a dierentiable function
on M:
We say that the function f is pseudoconvex on M with respect to
K if the function f is pseudoconvex at each point x0 2 M with respect
to K:
We say that the function f is pseudoconcave on M with respect to
K if the function ( f ) is pseudoconvex on M with respect to K:
9. PSEUDOCONVEX FUNCTIONS 63

## We will relate now pseudoconvex (respectively pseudoconcave) func-

tions to convex (respectively concave) functions.

## Theorem 1.9.3 Let M be a nonempty subset of Rn ; x0 be an interior

point of M; K be a closed convex cone in Rm and f : M ! Rm be a
dierentiable function at x0 :
10 If the function f is convex at x0 with respect to K; then the
function f is pseudoconvex at x0 with respect to K:
20 If the function f is concave at x0 with respect to K; then the
function f is pseudoconcave at x0 with respect to K:
Proof. We shall establish the theorem for the convex case. The
concave case follows from it.
10 By the convexity of the function f at x0 with respect to K; in
view of Theorem 13, we have
f (x) f x0 rf (x0 ) x x0 2 K;

for all x 2 M:
Let now x 2 M n fx0 g be such that
u = rf (x0 ) x x0 2 K:
Then
f (x) f x0 2 K + fug K + K = K:
Consequently, the function f is pseudoconvex at x0 with respect to
K:

## Remark 1.9.4 The converse of Theorem 3 is not necessarily true as

it can be seen from the following example: the function f : R ! R
dened by
f (x) = exp x2 ; for all x 2 R
is pseudoconvex at x0 = 100 with respect to K = R+ ; but it is not
convex at x0 = 100 with respect to K:
From Theorem 3 follows immediatly the next statement:

## Theorem 1.9.5 Let M be a nonempty open convex subset of Rn ; K

be a closed convex cone in Rm and f : M ! Rm be a dierentiable
function on M:
10 If the function f is convex on M with respect to K; then the
function f is pseudoconvex on M with respect to K:
20 If the function f is concave on M with respect to K; then the
function f is pseudoconcave on M with respect to K:
64 1. CONVEX ANALYSIS

## Remark 1.9.6 If K Rm has the property that 0 2 K; then ob-

viously the implication from the denitions of the pseudoconvex and
pseudoconcave functions are true for x = x0 and t = 0 and t = 1:
Consequently, in this case, it is not necessary to require that x 6= x0 :

## Denition 1.9.7 Let M be a nonempty subset of Rn ; x0 be an interior

point of M; K be a subset of Rm with nonempty interior and f : M !
Rm be a dierentiable function at x0 :
We say that the function f is strictly pseudoconvex (respectively
strictly pseudoconcave) at x0 with respect to K if the function f is
pseudoconvex (respectively pseudoconcave) at x0 with respect to int
K:

## Denition 1.9.8 Let M be a nonempty open subset of Rn ; K be a

subset of Rm with nonempty interior and f : M ! Rm be a dieren-
tiable function on M:
We say that the function f is strictly pseudoconvex (respectively
strictly pseudoconcave) on M with respect to K if the function f is
strictly pseudoconvex (respectively strictly pseudoconcave) at each z 2
M with respect to K:

## Theorem 1.9.9 Let M be a nonempty subset of Rn ; x0 be an interior

point of M; K be a closed convex cone in Rm with nonempty interior
and f : M ! Rm be a dierentiable function at x0 :
10 If the function f is convex at x0 with respect to K; then the
function f is strictly pseudoconvex at x0 with respect to K:
20 If the function f is concave at x0 with respect to K; then the
function f is strictly pseudoconcave at x0 with respect to K:
Proof. We shall establish the theorem for the convex case. The
concave case follows from it.
10 By the convexity of the function f at x0 with respect to K; in
view of Theorem 13, we have
f (x) f (x0 ) [rf (x0 )](x x0 ) 2 K;

## for all x 2 M n fx0 g:

Let now x 2 M n fx0 g be such that
t = [rf (x0 )](x x0 ) 2 intK:

Then
f (x) f x0 2 ftg + intK K + intK = intK:
10. QUASICONVEX FUNCTIONS 65

## Consequently, the function f is strictly pseudoconvex at x0 with

respect to K:
From Theorem 9 follows.

## Theorem 1.9.10 Let M be a nonempty open convex subset of Rn ; K

be a closed convex cone in Rm with nonempty interior and f : M ! Rm
be a dierentiable function on M:
10 If the function f is convex on M with respect to K; then the
function f is strictly pseudoconvex on M with respect to K:
20 If the function f is concave on M with respect to K; then the
function f is strictly pseudoconcave on M with respect to K:

## Denition 1.10.1 Let M be a nonempty subset of Rn ; x0 be a point

from M; K be a nonempty subset of Rm and f : M ! Rm be a function.
We say that the function f is quasiconvex at x0 with respect to K;
if, for each x 2 M n fx0 g with the property that
f x0 f (x) 2 K
and, for each t 2]0; 1[ such that (1 t) x0 + tx 2 M; we have
f x0 f (1 t) x0 + tx 2 K:

## We say that the function f is quasiconcave at x0 with respect to K;

if the function ( f ) is quasiconvex at x0 with respect to K:

## Denition 1.10.2 Let M be a nonempty convex subset of Rn ; K be

a nonempty subset of Rm and f : M ! Rm be a function.
We say that the function f is quasiconvex on M with respect to K
if the function f is quasiconvex at each x0 2 M with respect to K:
We say that the function f is quasiconcave on M with respect to
K if the function ( f ) is quasiconvex on M with respect to K:

## Example 1.10.3 The function f : R ! R dened by f (x) = x3 ; for

all x 2 R is qusiconvex on R with respect to R+ :
We will relate now quasiconvex (respectively quasiconcave) func-
tions to convex (respectively concave) functions.
The following statement is true.

## Theorem 1.10.4 Let M be a nonempty subset of Rn ; x0 be a point

from M; K be a convex cone in Rm and f : M ! Rm be a function.
66 1. CONVEX ANALYSIS

## 10 If the function f is convex at x0 with respect to K; then the

function f is quasiconvex at x0 with respect to K:
20 If the function f is concave at x0 with respect to K; then the
function f is quasiconcave at x0 with respect to K:
Proof. 10 Let x 2 M n fx0 g for which
f x0 f (x) 2 K

## and t 2]0; 1[ for which (1 t) x0 + tx 2 M: Then, by the convexity of

f at x0 with respect to K and by the fact that K is a convex cone, we
have
f x0 f (1 t) x0 + tx =

## = (1 t) f (x0 ) + tf (x) f (1 t) x0 + tx + t[f x0 f (x)] 2

2 K + tK K + K = K:
The concave case is established in a similar way.

## Remark 1.10.5 The converse of Theorem 4 is not true; there are

quasiconvex functions at x0 with respect to K which are not convex at
x0 with respect to K:
Indeed, the function f : R ! R dened by
f (x) = x + x3 ; for all x 2 R

## is quasiconvex at x0 = 0 with respect to R+ ; but it is not convex at

x0 = 0 with respect to K = R+ :
From Theorem 4 follows immediately the next statement:

## Theorem 1.10.6 Let M be a nonempty convex subset of Rn ; K be a

convex cone in Rm and f : M ! Rm be a function.
10 If the function f is convex on M with respect to K; then the
function f is quasiconvex on M with respect to K:
20 If the function f is concave on M with respect to K; then the
function f is quasiconcave on M with respect to K:

## Theorem 1.10.7 Let M be a nonempty subset of Rn ; x0 be an interior

point of M; K be a closed convex cone in Rm and f : M ! Rm be a
dierentiable function at x0 :
10 If the function f is quasiconvex at x0 with respect to K; then,
for each x 2 M n fx0 g with the property that
f x0 f (x) 2 K;
10. QUASICONVEX FUNCTIONS 67

we have
rf (x0 ) x0 x 2 K:
20 If the function f is quasiconcave at x0 with respect to K; then,
for each z 2 M with the property that
f (x) f x0 2 K;
we have
rf (x0 ) x x0 2 K:

## Proof. 10 Let x 2 M n fx0 g with property that

f x0 f (x) 2 K:
Since x0 is an interior point of M; there exists a real number r 2 ]0; +1[
such that
B x0 ; r := fx 2 Rn : x x0 < rg M:

## Then, for each t 2]0; 1[ for which

t < r= x x0 ;
we have
(1.10.1) w = (1 t) x0 + tx = x0 + t x x0 2 B x0 ; r M:

## The function f is quasiconvex at x0 with respect to K; then, for each

t 2]0; 1[; t < r= kx x0 k ; we have
f (x0 ) f (1 t) x0 + tx 2 K:

## From this, it follows that

1 1
(1.10.2) f x0 + t(x x0 ) f x0 2 K = K;
t t
for all t 2]0; 1[; t < r= kx x0 k ; the last equality in (1.10.2) holds
because K is a convex cone.
Furthermore, the function f is dierentiable at x0 ; then there exists
1
lim ff x0 + t(x x0 ) f x0 g = rf (x0 ) (x x0 ):
t!0 t
Then, taking the limit of expression (1.10.2) and taking into account
the fact that K is a closed set, we deduce that
(1.10.3) rf (x0 ) (x x0 ) 2 K:

## Statement 20 is established in a similar way.

68 1. CONVEX ANALYSIS

## Theorem 1.10.8 Let M be a nonempty open convex subset of Rn ;

K be a closed convex cone in Rm and f : M ! Rm be a dierentiable
function on M:
10 If the function f is quasiconvex on M with respect to K; then,
for each x; x0 2 M with the property that
f x0 f (x) 2 K;
we have
rf (x0 ) x0 x 2 K:
20 If the function f is quasiconcave on M with respect to K; then,
for each x; x0 2 M with the property that
f (x) f x0 2 K;
we have
rf (x0 ) x x0 2 K:

## Remark 1.10.9 If K Rm has the property that 0 2 K; then ob-

viously the implication from the denitions quasiconvex and quasicon-
cave functions are true for x = x0 and t = 0 and t = 1: Consequently, in
this case, it is not necessary to require that x 6= x0 ; t 6= 0 and t 6= 1:

## Denition 1.10.10 Let M be a nonempty subset of Rn ; x0 be a point

from M; K be a subset of Rm with nonempty interior and f : M ! Rm
be a function.
We say that the function f is strictly quasiconvex (respectively
strictly quasiconcave) at x0 with respect to K; if the function f is
quasiconvex (respectively quasiconcave) at x0 with respect to intK:

## Denition 1.10.11 Let M be a nonempty convex subset of Rn ; K be

a subset of Rm with nonempty interior and f : M ! Rm be a function.
We say that the function f is strictly quasiconvex (respectively
strictly quasiconcave) on M with respect to K; if the function f is
strictly quasiconvex (respectively strictly quasiconcave) at each point
x0 2 M with respect to K:

## Theorem 1.10.12 Let M be a nonempty subset of Rn ; x0 be a point

from M; K be a convex cone in Rm with nonempty interior and f :
M ! Rm be a function.
10 If the function f is convex at x0 with respect to K; then the
function f is strictly quasiconvex at x0 with respect to K:
10. QUASICONVEX FUNCTIONS 69

## 20 If the function f is concave at x0 with respect to K; then the

function f is strictly quasiconcave at x0 with respect to K:
Proof. 10 Let x 2 M n fx0 g with property that
f x0 f (x) 2 intK

## and t 2]0; 1[ such that (1 t) x0 + tx 2 M: By the convexity of the

function f at x0 with respect to K; we have
(1 t)f (x0 ) + tf (x) f ((1 t)x0 + tx) 2 K:
Then
f (x0 ) f ((1 t)x0 + tx) =
= [(1 t)f (x0 ) + tf (x) f ((1 t)x0 + tx)] + t [f (x0 ) f (x)] 2
2 K + tintK = intK:

## Consequently, the function f is strictly quasiconvex at x0 with respect

to K:
Statement 20 is established in a similar way.
From Theorem 12 follows.
Theorem 1.10.13 Let M be a nonempty convex subset of Rn ; K be
a convex cone in Rm with nonempty interior and f : M ! Rm be a
function.
10 If the function f is convex on M with respect to K; then the
function f is strictly quasiconvex on M with respect to K:
20 If the function f is concave on M with respect to K; then the
function f is strictly quasiconcave on M with respect to K:
CHAPTER 2

Optimality Conditions

1. Optimization Problems
By a minimization problem we understand a problem of the follow-
ing type:
Being given:
a) a nonempty subset M of the space Rn ;
b) a nonempty subset K of the space Rm ;
c) the functions f : M ! R and g : M ! Rm ;
nd all (locally) minimum points of the function f on the set
X = fx 2 M : g(x) 2 Kg:

## Formally, the minimization problem in complex space is written in

the literature as follows
(2.1.1) min f (x)
subject to
x 2 X;
or explicitly
min f (x)
subject to (
x 2 M;
g(x) 2 K:
The function f is called the criteria function, the set X is called
the set of the feasible solutions (or the feasible set) of Problem (2.1.1) ;
the relation g (x) 2 K is called the constraint of the problem and the
function g is called the constraint function.
Minimization problem (2.1.1) is said to be consistent if X 6= ; and
inconsistent if X = ;: All points x from the feasible set X are called
feasible points, or feasible solution of the minimization problem (2.1.1).
We say that the point x0 2 X is a solution of minimization problem
(2.1.1) if
f x0 = minff (x) : x 2 Xg.

71
72 2. OPTIMALITY CONDITIONS

## We say that the point x0 2 X is a local solution of minimization

problem (2.1.1) if there exists a neigborhood V of the point x0 such
that
f x0 = minff (x) : x 2 X \ Vg:
We say that the minimization problem (2.1.1) does not have solu-
tions if
inf ff (x) : x 2 Xg > 1
0
and an x 2 X such that
inf ff (x) : x 2 Xg = f x0
does not exist.
Problem (2.1.1) does not have nite solutions if there exists a se-
quence xk k2N with xk 2 X; for all k 2 N, such that
lim f xk = 1:
k!1

## Similarly, we can dene a maximization problemx. It is written in

the literature as follows
max f (x)
subject to
x 2 X;
or explicitly
max f (x)
subject to (
x 2 M;
g(x) 2 K:
Minimization problems and maximization problems are called op-
timization problems.
Since
max ff (x) : x 2 Xg = min f( f (x)) : x 2 Xg ;

## we shall study from now on only problems of the form:

min f (x)
subject to
x 2 X:
According to the properties of the sets M and K and of the func-
tions f and g which are used in order to dene Problem (2.1.1) one
may distinguish between dierent classes of optimization problems.
2. DOREL I. DUCA 73

Two types of questions can appear when working with the opti-
mization problem (2.1.1) :
(i) determine necessary, respectively su cient conditions for a
point x0 2 Rn to be a solution or a local solution of Problem (2.1.1) ;
and
(ii) discover some numerical methods for determining a solution or
a local solution of Problem (2.1.1) :
Even though for question (i) above one can give both necessary and
su cient conditions for a point x0 2 Rn to be a solution of Problem
(2.1.1) ; question (ii) is far from being answered.

## 2. Some General Results of Optimization

During this section we shall consider the optimization problem:

## (2.2.1) min f (x)

subject to
x 2 X;
where X is a nonempty subset of Rn and f : X ! R is a function.
We establish now a basic result concerning the set of solutions of
the optimization problem (2.2.1).

## Theorem 2.2.1 Let X be a nonempty convex subset of Rn ; and f :

X ! R be a quasiconvex function on X with respect to R+ : Then the
set of the solutions of Problem (2.2.1) is convex.
Proof. Let x0 and x1 be two solutions of Problem (2.2.1) : It is
obvious that x0 ; x1 2 X and
f x0 = f x1 = min ff (x) : x 2 Xg

and therefore
(2.2.2) f x0 f x1 0:

## Let now t 2 ]0; 1[ : Since X is a convex set, it follows that

xt = (1 t) x0 + tx1 2 X:

## Because the function f is quasiconvex on X with respect to R+ ; from

(2.2.2) it follows that
f x0 f x t 2 R+ ;

or equivalently,
f x0 = f xt :
74 2. OPTIMALITY CONDITIONS

## Hence, xt 2 X is also a solution of Problem (2.2.1) and thus the set

of the solutions of the same problem is convex.

## Theorem 2.2.2 Let X be a nonempty convex subset of Rn , f : X ! R

be a function and x0 be a solution of Problem (2.2.1) : If f is strictly
convex at x0 with respect to R+ , then x0 is the unique solution of Prob-
lem (2.2.1) :
Proof. Let x1 6= x0 be another solution of Problem (2.2.1) : Then
x ; x1 2 X and
0

f z0 f z 1 = 0:

## Let now t 2 ]0; 1[ : Since X is a convex set, it follows that

xt = (1 t) x0 + tx1 2 X:

## Moreover, since f is strictly convex at z 0 with respect to R+ ; it follows

that
(1 t) f x0 + tf x1 f xt > 0

and therefore
f z t < (1 t) f x0 + tf x1 =
= f x0 t f x0 f x1 =
= f x0 ;
which is a contradiction to the fact that x0 is a solution of Problem
(2.2.1) : Thus, x1 cannot be dierent from x0 :

## Theorem 2.2.3 If x0 is a solution of Problem (2.2.1); then x0 is a

local solution of Problem (2.2.1):
Proof. It is immediate.
The converse of Theorem 3 does not generally hold. The follow-
ing theorem gives a su cient condition for the converse statement of
Theorem 3 to hold.

## Theorem 2.2.4 Let X be a nonempty convex subset of Rn and f :

X ! R be a convex function on X with respect to R+ : If x0 is a local
solution of Problem (2.2.1), then x0 is a solution of Problem (2.2.1) :
2. DOREL I. DUCA 75

## Proof. Let x0 be a local solution of Problem (2.2.1) : Then there

exists r 2 R; r > 0 such that
f x0 5 f (x) ; for all x 2 X \ B x0 ; r :

Let us suppose that there exists a point x1 with the property that
(2.2.3) f x1 < f x0 :

## It is obvious that x1 6= x0 : Since X is a convex set, it follows that

xt = (1 t) x0 + tx1 2 X; for all t 2 ]0; 1[ :

## Let us choose t 2 ]0; 1[ such that

r
0<t< :
kx0 x1 k

Then
xt x0 = (1 t) x0 + tx1 x0 = t x0 x1 < r

and thus
xt 2 X \ B x0 ; r :

Therefore
(2.2.4) f xt = f x0 :

## Moreover, since the function f is convex on X with respect to R+ ;

taking into account (2.2.3) ; it follows that
f xt 5 (1 t) f x0 + tf x1 <
< (1 t) f x0 + tf x0 =
= f x0 ;
which is a contradiction to (2.2.4) :

## Theorem 2.2.5 Let X be a nonempty convex subset of Rn and f :

X ! R be a concave function on X with respect to R+ ; under the
assumption that it is not a constant. If x0 is a solution of Problem
(2.2.1) ; then x0 belongs to the boundary of X:
76 2. OPTIMALITY CONDITIONS

## Proof. If intX = ;; then x0 is a point of the boundary of the set

X:
Let us consider the case when intX 6= ;: Because f is not a constant
it follows that there exists x1 2 X such that
f x0 < f x1 :

Let now z 2 intX; then there exists a real number r > 0 such that
B (x; r) X:

Let us denote
kx x1 k + 1
t= :
kx x1 k + 1 + r
It is obvious that x 2 ]0; 1[ :
Let
1 1
(2.2.5) w = x+ 1 x1 :
t t
Since
1 r
kx wk = 1 x x1 = x x1 < r;
t kx x1 k + 1
it follows that w 2 B (x; r) and thus w 2 X: We deduce then that
f (w) = f x0 :

## From (2.2.5) it follows that

x = (1 t) x1 + tw

## and because X is a convex set, x1 2 X; w 2 B (x; r) X and t 2 ]0; 1[

it follows that x 2 X:
Moreover, since the function f is concave on X with respect to R+ ;
we obtain that
f (x) = f (1 t) x1 + tw =
= (1 t) f x1 + tf (w) >
> (1 t) f x0 + tf x0 =
= f x0 :
Hence, if x 2 intX; then
f (x) > f x0 :

## Thus, f cannot attain its minimum at an interior point of X:

2. DOREL I. DUCA 77

## Theorem 2.2.6 Let X be a nonempty convex subset of Rn and f :

X ! R be a concave function on X with respect to R+ : Then the set of
the points x 2 X which are not solutions of Problem (2.2.1) is convex.
Proof. Let x1 ; u1 2 X be two points which are not solutions of
Problem (2.2.1) ; then there exists x0 2 X such that
f x0 < f x1 and f x0 < f u1 :

## Let t 2 ]0; 1[ : Since X is a convex set, it holds

xt = (1 t) x1 + tu1 2 X:
Because the function f is concave on X with respect to R+ ; it follows
that
f (x ) = (1 t) f x1 + tf u1 >
> (1 t) f x0 + tf x0 = f x0 :
Consequently, the set of the points form X which are not solutions of
Problem (2.2.1) is convex.

## Theorem 2.2.7 Let M be a subset of Rn ; X be a convex subset of

M; x0 2 intX and f : M ! R be a dierentiable function at x0 : A
necessary condition, for x0 to be a solution of Problem (2.2.1) which
is also su cient if the function f is pseudoconvex at x0 with respect to
R+ ; is
(2.2.6) hrf (x0 ); x x0 i = 0; for all x 2 X:

Proof.

## Necessity. Let x 2 X: Let us consider the function g : [0; 1] ! R

dened by
g (t) = f x0 + t x x0 ; for all t 2 [0; 1] :
It is obvious that t = 0 is a minimum point of the function g; then
d
g (t) jt=0 = 0:
dt
Because
d
g (t) jt=0 = rf z 0 z z 0 + rz f z 0 z z0 ;
dt
78 2. OPTIMALITY CONDITIONS

## it follows that (2.2.6) holds.

Proof. Su ciency. Inequality (2.2.6) may be written as
T
(2.2.7) rz f z 0 z z 0 + rz f z 0 z z 0 2 CR+ ;

## for all z 2 X, where

CR+ = fv 2 C : Re v = 0g :

## Let now z 2 X: Because f has pseudoconvex real part at z 0 with

respect to R+ ; from (2.2.7) ; it follows that
f (z) f z 0 2 CR+ ;

or equivalently,
Re f (z) = Re f z 0 :

## Corollary 2.2.8 Let X be a nonempty subset of Rn ; x0 be an interior

point of X and f : M ! R be a dierentiable function at xz 0 : A
necessary condition for x0 to be a solution of Problem (2.2.1) is that:
rf (x0 ) = 0:

r > 0 such that
B(x0 ; r) X:

## Moreover, by Theorem 7, we have that inequality (2.2.6) holds; then

hrf (x0 ); vi = 0; for all v 2 B(0; r):

Therefore
rf (x0 ) 2 (B(0; r)) :

and since
(B(0; r)) = f0g;

## the corollary is proved.

3. NECESSARY AND SUFFICIENT OPTIMALITY CRITERIA 79

## Corollary 2.2.9 Let M be nonempty open subset of Rn , X be a non-

empty convex subset of M; x0 2 X and f : M ! R be a dierentiable
function at x0 and pseudoconvex at z 0 with respect to R+ : If
rf (x0 ) = 0;

## then x0 is a solution of Problem (2.2.1) :

Proof. It uses Theorem 7.

## 3. Necessary Optimality Criteria and Su cient Optimality

Criteria
The purpose of this section is to derive optimality criteria of the
saddle point type for optimization problem. We shall develop the opti-
mality criteria of this section without any dierentiability assumptions
imposed on the functions involved. In the next section will establish
optimality criteria that involve dierentiable functions.
Let M be a nonempty subset of Rn ; K be a nonempty subset of Rm
and f : M ! R and g : M ! Rm be two functions. We shall consider
the following optimization problem:
(2.3.1) min f (x)

subject to (
x2M
g(x) 2 K:
For each u 2 R we attach to Problem (2.3.1) the function Lu :
M Rm ! R dened by
Lu (x; v) := uf (x) hg (x) ; vi ; for all (x; v) 2 M Rm :

holds.

## Theorem 2.3.1 Let M be a nonempty convex subset of Rn ; K be a

polyhedral cone in Rm with nonempty interior, f : M ! R be a convex
function on M with respect to R+ and g : M ! Rm be a concave
function on M with respect to K.
If x0 is a solution of Problem (2.3.1), then there exist u0 2 R and
v 2 Rm such that:
0

## (2.3.2) u0 2 R+ ; v 0 2 K ; (u0 ; v 0 ) 6= (0; 0);

(2.3.3) g(x0 ); v 0 = 0
80 2. OPTIMALITY CONDITIONS

and
(2.3.4) Lu0 (x0 ; v) 5 Lu0 (x0 ; v 0 ) 5 Lu0 (x; v 0 );

## for all (x; v) 2 M K :

Proof. Since x0 is a solution of Problem (2.3.1) it follows that the
system
8 0
< f (x) f (x ) < 0
>
(2.3.5) g(x) 2 K
>
:
x2M

is inconsistent.
System (2.3.5) can be written as:
8 0
< f (x ) f (x) 2 intR+
>
(2.3.6) g(x) 2 K
>
:
x 2 M:

Taking into account Remark ?? and Corollary ??, it follows that there
exist u0 2 R and v 0 2 Rm such that
(2.3.7) u 0 2 R+ = R + ; v 0 2 K ; u0 ; v 0 6= (0; 0)

and
(2.3.8) f (x0 ) f (x); u0 + g(x); v 0 5 0; for all x 2 M:

g(x0 ); v 0 5 0:

## Because v 0 2 K and g (x0 ) 2 K we obtain that

g(x0 ); v 0 = 0

and therefore
g(x0 ); v 0 = 0;
i.e. equality (2.3.3) :
Now, from (2.3.8), (2.3.3) and the fact that x0 is a solution of
Problem (2.3.1) ; it follows that
u0 f (x0 ) g(x0 ); v 0 5 u0 f (x) g(x); v 0 ; for all x 2 M;

## which is exactly the second inequality in (2.3.4) :

3. NECESSARY AND SUFFICIENT OPTIMALITY CRITERIA 81

## In order to show that the rst inequality in (2.3.4) is satised, let

us remark that from
g x0 2 K;
we obtain that
g(x0 ); v = 0, for all v 2 K
and therefore
u0 f (x0 ) g(x0 ); v 5 u0 f (x0 ) g(x0 ); v 0 ; for all v 2 K ;

because
g(x0 ); v 0 = 0:

## Example 2.3.2 Let us consider the following optimization problem

in complex space:
(2.3.9) min f (x1 ; x2 ) := x21 + x22

subject to
(
(x1 ; x2 ) 2 M := f(x1 ; x2 ) 2 R2 : x1 = 0g
g (x1 ; x2 ) := x1 x2 1 2 K := R+ :

One can easily notice that the hypotheses of Theorem 1 are fullled
and it can easily be shown that x0 = (1; 1) is a solution of Problem
(2.3.9) ;
u 0 = 1 2 R+
and
v 0 = 0 2 K = R+
verify (2.3.2) (2.3.4) :

## Example 2.3.3 Let us consider the following optimization problem:

(2.3.10) min x

subject to
x 2 M := R
g(x) := x2 2 K := R+ :
It can be easily seen that the solution of Problem (2.3.10) is x0 =
0 2 R and the hypotheses of Theorem 1 are satised. Then, there exist
u0 2 R and
v 0 2 K = R+ ;
82 2. OPTIMALITY CONDITIONS

## such that (2.3.2) (2.3.4) hold.

Let us suppose that u0 is dierent from 0: Then, from the second
inequality in (2.3.4) it follows that, for each x 2 M; it holds
(2.3.11) 0 5 u0 x + x 2 v 0 :

## If v 0 were 0; then inequality (2.3.11) would become

0 5 u0 x; for all x 2 R;

## which obviously, holds if and only if u0 = 0, which is a contradiction to

the fact that (u0 ; v 0 ) 6= (0; 0) : If v 0 were dierent from 0, then taking
in (2.3.11)
x = u0 = 2v 0 2 M = R
we would obtain
2
(u0 )
u0 x + v 0 x 2 = < 0;
(4v 0 )

## which is a contradiction to (2.3.11) : Therefore, for Problem (2.3.10) ;

u0 cannot be dierent from 0.

## Denition 2.3.4 Let M be a nonempty subset of Rn ; K be a subset

of Rm , g : M ! Rm be a function and
X = fx 2 M : g (x) 2 Kg :

## We say that the function g satises :

10 Slaterss constraint qualication with respect to X if intK 6= ;
and there exists a point x1 2 M with the property that
g x1 2 intK:
20 The strict constraint qualication with respect to X if intK 6= ;;
and there exist two dierent points x0 ; x1 2 X and a real number
t 2 ]0; 1[ such that
(1 t) x0 + tx1 2 M

and
g (1 t) x0 + tx1 (1 t) g x0 tg x1 2 intK:

## 30 Karlins constraint qualication with respect to X if it does not

exist v 2 K n f0g such that
hg (x) ; vi 5 0; for all x 2 M:
3. NECESSARY AND SUFFICIENT OPTIMALITY CRITERIA 83

## Example 2.3.5 Let M be the entire space R; K = R+ ; g : R ! R be

the function dened by
g (x) = x, for all x 2 R
and
X = fx 2 M : g (x) 2 Kg = R+ :
Since x1 = 1 2 M = R satises the condition
g x1 = 1 2 intK = fx 2 R : x > 0g;
it follows that the function g satises Slaters constraint qualication
with respect to X:
The function g also satises Karlins complex constraint qualica-
tion with respect to X: Indeed, since
K = R+ ;
we obtain that, for each v 2 K n f0g = fx 2 R : x > 0g; the point
x = v 2 M satises the equality
hg (x) ; vi = v 2 > 0;
and consequently it does not exist v 2 K n f0gsuch that
hg (x) ; vi 5 0; for all x 2 M:
Let now x0 ; x1 2 X and t 2 ]0; 1[ : Obviously,
(1 t) x0 + tx1 2 M = R;
meanwhile
g (1 t) x0 + tx1 (1 t) g x0 tg t1 = 0 62 intK
and therefore the function g satises the strict constraint qualication
with respect to X:
Example 2.3.6 Let M be the entire space Rn ; K = f0g, g : M ! Rn
be the function dened by
g(x) = x, for all x 2 M
and
X = fx 2 M : g(x) 2 Kg:
On one hand, since intK = ;; it follows that the function g does
not satises Slaters constraint qualication with respect to X: Also,
the function g does not satisfy the strict constraint qualication with
respect to X:
84 2. OPTIMALITY CONDITIONS

## On the other hand, for each v 2 K = Rn ; v 6= 0; the point x =

v 2 M has the property that
hg(x); vi = kvk2 > 0;

## hence there exists no v 2 K n f0g such that

hg(x); vi 5 0; for all x 2 M:

## Therefore, the function g satises the Karlins constraint qualica-

tion with respect to X:

## Example 2.3.7 Let M be the entire space R, K := R+ ; g : M ! R

be the function dened by
g(x) := x2 ; for all x 2 M

and
X = fx 2 M : g(x) 2 Kg:
Obviously, intK 6= ;: We have
g(x) = x2 5 0; for all x 2 R;

## hence Slaters constraint qualication with respect to X is not satised.

Since X = f0g; if follows that there exist no points x0 ; x1 2 X;
x0 6= x1 and no real number t 2]0; 1[ such that
(1 t)x0 + tx1 2 M

and
g((1 t)x0 + tx1 ) (1 t)g(x0 ) tg(x1 ) 2 intK:
Therefore, the function g does not satisfy the strict constraint quali-
cation with respect to X:
Obviously, for each x 2 R and each v 2 K = R+ ; we have
hg(x); vi = x2 v 5 0:

## Therefore, the function g does not satises Karlins constraint quali-

cation with respect to X:

## Example 2.3.8 Let M be the entire space Rn ; K = R+ ; g : M 2 R

be the function dened by
g(x) = kxk , for all x 2 M

and
X = fx 2 M : g(x) 2 Kg:
3. NECESSARY AND SUFFICIENT OPTIMALITY CRITERIA 85

## The function g satises Slaters, Karlins and the strict constraint

qualications with respect to X:
Relations between the above constraint qualications are estab-
lished in the following theorem.

## Theorem 2.3.9 Let M be a nonempty subset of Rn ; K be a closed

convex cone in Rm , g : M ! Rm be a function and
X = fx 2 M : g(x) 2 Kg:

## 10 If the function g satises Slaters constraint qualication with re-

spect to X; then the function g satises Karlins constraint qualication
with respect to X:
20 If the function g satises the strict constraint qualication with
respect to X; then the function g satises Slaters constraint qualica-
tion with respect to X:
30 If, in addition, the set M is convex, K is a polyhedral cone with
nonempty interior and the function g is concave on M with respect to
K; then the function g satises Karlins constraint qualication with
respect to X if and only if the function g satises Slaters constraint
qualication with respect to X:
Proof. 10 Assume that the function g satises Slaters constraint
qualication with respect to X: Then there exists a point x1 2 M such
that
g(x1 ) 2 intK:
Now, if v 2 K n f0g; we have
g(x1 ); v > 0;

## and hence there exists no v 2 K n f0g such that

hg(x); vi 5 0, for all x 2 M:

## Therefore, the function g satises Karlins constraint qualication with

respect to X:
20 Assume that the function g satises the strict constraint qual-
ication with respect to X: Then there exist two points x0 ; x1 2 X;
x0 6= x1 and a real number t 2]0; 1[ such that
(1 t)x0 + tx1 2 M

and
(2.3.12) g((1 t)x0 + tx1 ) (1 t)g(x0 ) tg(x1 ) 2 int K:
86 2. OPTIMALITY CONDITIONS

## Since g(x0 ); g(x1 ) 2 K; t 2]0; 1[ and K is a cone, it follows that

(1 t)g(x0 ) 2 K and tg(x1 ) 2 K:

From (2.3.12) and from the fact that K is a convex cone, we have
g((1 t)x0 + tx1 ) 2 intK + K + K = intK + K intK:

## Therefore, the function g satises Slaters constraint qualication with

respect to X:
30 Assume that the function g satises Karlins constraint quali-
cation with respect to X: Then there exists no v 2 K n f0g such
that
hg(x); vi 5 0, for all x 2 M:
From this, we deduce that the system
(
g(x) 2 intK
x2M

## is consistent, hence the function g satises Slaters constraint quali-

cation with respect to X:
From this and from statement 10 above, it follows that the function
g satises Karlins constraint qualication with respect to X if and only
if the function g satises Slaters constraint qualication with respect
to X.

## Remark 2.3.10 Example 7 proves the existence of a function g :

M ! Rm ; where M Rn and of a set K Rm for which none of the
strict, Slaters, or Karlins constraint qualications with respect to
X = fx 2 M : g (x) 2 Kg
is satised.
We derive now one of the most important saddlepoint theorems of
Kuhn-Tucker type.

## Theorem 2.3.11 Let M be a nonempty convex subset of Rn ; K Rm

be a polyhedral cone with nonempty interior, f : M ! R be a convex
function on M with respect to R+ ; g : M ! Rm be a concave function
on M with respect to K; and
X = fx 2 M : g(x) 2 Kg:

## In addition, assume that one of the following conditions is satised:

i) The function g satises Slaters constraint qualication with re-
spect to X; or
3. NECESSARY AND SUFFICIENT OPTIMALITY CRITERIA 87

## ii) The function g satises the strict constraint qualication with

respect to X; or
iii) The function g satises Karlins constraint qualication with
respect to X:
If x0 is a solution of Problem (2.3.1), then there exists a point
v 2 Cm such that
0

(2.3.13) v0 2 K ;

(2.3.14) g(x0 ); v 0 = 0
and
(2.3.15) L1 (x0 ; v) 5 L1 (x0 ; v 0 ) 5 L1 (x; v 0 ); for all (x; v) 2 M K :

## Proof. In view of Theorem 9 above we only need to establish the

theorem under assumption i).
u0 ; vb0 ) 2 R Rm
By Theorem 1, it results that there exists a point (b
such that

(2.3.16) b0 2 R+ ; vb0 2 K ; (b
u u0 ; vb0 ) 6= (0; 0);

## (2.3.17) g(x0 ); vb0 = 0

and
(2.3.18)
Lub0 (x0 ; v) 5 Lub0 (x0 ; vb0 ) 5 Lub0 (x; vb0 ); for all (x; v) 2 M K :
We will show now by contradiction that u b0 6= 0: Suppose that u
b0 =
0; then from (2.3.16) ; if follows that vb0 6= 0: From (2.3.17) and the
second inequality in (2.3.18) we obtain
(2.3.19) 05 g(x); vb0 ; for all x 2 M:
Since the function g satises Slaters constraint qualication with re-
spect to X; there exists a point x1 2 M such that
g(x1 ) 2 intK;
then
g(x1 ); v > 0; for all v 2 K n f0g:
Since vb0 2 K n f0g; it results that
g(x1 ); vb0 > 0;

## which contradicts (2.3.19) for x = x1 2 M: Hence u

b0 6= 0:
88 2. OPTIMALITY CONDITIONS

## b0 > 0 and making the notation

Dividing (2.3.17) and (2.3.18) by u
v 0 = (1=b
u0 )b
v0

## Example 2.3.12 Let us consider the following optimization problem:

(2.3.20) min f (x) := x21 + x22 + ::: + x2n

subject to
(
x := (x1 ; x2 ; :::; xn ) 2 M = Rn
g(x) := hx; ei 1 2 K = R+ ;

## where e = (1; :::; 1) 2 Rn : Obviously, the set M is convex. Moreover,

the function f is convex on Rn with respect to R+ and the function
g is concave on Rn with respect to K and satises Slaters constraint
qualication with respect to
X = fx 2 M : g(x) 2 Kg:

## Hence the hypotheses of Theorem 11 are satised. It is easy to verify

that the above problem, (2.3.20) ; has the solution
x0 = (1=n)e 2 Rn

and that p
v0 = 1=n 2 R
satises (2.3.13) (2.3.15) :
The converse of Theorem 11 holds under wider assumptions.

## Theorem 2.3.13 Let M be a nonempty subset of Rn ; K be a closed

convex cone in Rm ; f : M ! R and g : M ! Rm be two functions. If
there exist x0 2 Rn ; u0 2 R; v 0 2 Rm such that
(2.3.21) x0 2 M; u0 > 0; v 0 2 K
and
(2.3.22)
Lu0 (x0 ; v) 5 Lu0 (x0 ; v 0 ) 5 Lu0 (x; v 0 ); for all (x; v) 2 M K ;

## then x0 is a solution of Problem (2.3.1) :

3. NECESSARY AND SUFFICIENT OPTIMALITY CRITERIA 89

## Proof. From the rst inequality of (2.3.22) ; it follows that

(2.3.23) g x0 ; v v 0 = 0; for all v 2 K ;

or
g x0 ; w = 0; for all w 2 K ;
from which we deduce that
g x0 2 (K ) = K;

## because K is a closed convex cone. Consequently x0 2 M is a feasible

solution of Problem (2.3.1) :
Moreover, since g (x0 ) 2 K and v 0 2 K it follows that
g x0 ; v 0 = 0

and hence
(2.3.24) g x0 ; v 0 = 0;

## because, (2.3.23) for v = 0 2 K ; becomes

g x0 ; v 0 5 0:
Let now x 2 M be a feasible solution of Problem (2.3.1) : Then
(2.3.25) g (x) ; v 0 = 0:

## From (2.3.25), the second inequality of (2.3.22) and (2.3.24) ; it

follows that
u0 f (x) = u0 f (x) hg(x); v 0 i =
= u0 f (x0 ) hg(x0 ); v 0 i =
= u0 f (x0 ):
Since u0 > 0 we deduce that
f (x) = f x0

## Example 2.3.14 Let us consider the following optimization problem:

(2.3.26) min f (x1 ; x2 ) := x1

subject to
(x1 ; x2 ) 2 M := R2
g (x1 ; x2 ) := x1 ; (x1 )3 + x2 2 K;
90 2. OPTIMALITY CONDITIONS

where
K := f(v1 ; v2 ) 2 R2 : v1 = 0; v2 = 0g:

## Obviously, K is a closed convex cone and K = K: The points x0 =

(0; 0) 2 R2 ; u0 = 1 2 R and v 0 = (1; 0) 2 R2 verify (2.3.21) and
(2.3.22) : Consequently, x0 is a solution of Problem (2.3.1) :

## Remark 2.3.15 The condition for the set K to be a closed convex

cone in Theorem 13 is essential. If K is not a closed convex cone we
cannot guarantee that x0 is a feasible solution of Problem (2.3.1) :
Indeed, let us consider the following optimization problem in com-
plex space:
(2.3.27) min f (x1 ; x2 ) := x1

subject to
(
(x1 ; x2 ) 2 M := R2
g(x1 ; x2 ) := (x1 ; x2 ) 2 K := f(x1 ; x2 ) 2 R2 : x2 < x1 ; x2 < x1 g:

## the conditions (2.3.21) (2.3.22) are satised. Still, x0 is not a solution

of Problem (2.3.27); Problem (2.3.27) does not have solutions. Let us
remark that K is a convex cone but it is not a closed set.

## Remark 2.3.16 In Theorem 13 the condition u0 > 0 is essential. If

u0 < 0 and the other hypotheses are satised, then it can be proved
that x0 is a solution of Problem
max f (x)

subject to
(
x2M
g (x) 2 K;

## and if u0 = 0; then the criteria function does not appear anywhere.

4. NECESSARY OPTIMALITY CRITERIA 91

## 4. Necessary Conditions for the Optimal Solutions of the

Optimization Problems with Dierentiable Hypothesis
The purpose of this section is to derive necessary optimality criteria
for optimization problems. In the necessary optimality conditions de-
rived here, convexity does not play a crucial role. The dierentiability
property of the functions is used to linearize the optimization problem
and then theorems of the alternative are used to obtain the necessary
optimality conditions. In order to derive the most important necessary
optimality conditions, constraint qualications are needed.
Let us consider the optimization problem:
(2.4.1) min f (x)
subject to
x2M
g (x) 2 K;
where M Rn and K Rm are two nonempty sets and f : M ! R
and g : M ! Rm are two functions.
In what follows we use the following notations: If M is a nonempty
subset of Rn , x0 is a point from M; q is a positive integer number,
a1 ; :::; aq are points from Rm n f0g,
S := \fH= (ak ) : k 2 f1; :::; qgg;
where
H= (ak ) := fv 2 Rm : hv; ak i = 0g; k 2 f1; :::; qg

## and g : M ! Rm is a function, then

(2.4.2) E := fk 2 f1; :::; qg : hg(x0 ); ak i = 0g;

## P := fk 2 E : g is pseudoconvex at x0 with respect to H= (ak )g;

Q := E n P; N := f1; :::; qg n E
and
K(g(x0 )) := \fH= (ak ) : k 2 Eg:

## We begin with the following linearization theorem, which estab-

lishes the nonexistence of a solution of a system of linear inequalities
whenever the optimization problem has a solution.
92 2. OPTIMALITY CONDITIONS

## Theorem 2.4.1 Let M be a nonempty subset of Rn ; x0 be an interior

point of M; f : M ! R and g : M ! Rm be two dierentiable functions
at x0 and
K := \fH= (ak ) : k 2 f1; :::; qgg
be a polyhedral cone in Rm :
If x0 2 Rn is a local solution of the optimization problem (2.4.1),
then the system:
8
>
> [rf (x0 )] (x) < 0
>
>
< [rg(x0 )](z); ak > 0, k 2 Q
(2.4.3)
>
>
> [rg(x0 )](z); ak = 0, k 2 P
>
:
x 2 Rn
has no solution x 2 Rn .
Proof. Since x0 2 M is a local solution of Problem (2.4.1); there
exists a real number re > 0 such that if
(2.4.4) B(x0 ; re) := fx 2 Rn : x x0 < reg
and
X := fx 2 M : g(x) 2 Kg;
then the system
(
f (x0 ) < f (x)
(2.4.5)
x 2 X \ B(x0 ; re);

is inconsistent.
We shall show that if x 2 Rn satises System (2.4.3), then a con-
Let x 2 Rn be a solution of System (2.4.5). Then, since x0 is an
interior point of M , there exists a real number rb 2]0; re] such that

x0 + tx 2 M
for all t 2 [0; rb[:,
Since f and g are dierentiable at x0 ; it follows that, for all t 2 [0; rb[;
we have:

## f x0 + tx = f x0 + t rf x0 (x) + t x x0 "f x0 + tx; x0 ;

and
g(x0 + tx) = g(x0 ) + t rg(x0 ) (x) + t x x0 "g (x0 + tx; x0 );

## where "f (:; x0 ) : M ! R and "g (:; x0 ) : M ! Rm have the properties:

4. NECESSARY OPTIMALITY CRITERIA 93

## (2.4.6) lim "f (x0 + tx; x0 ) = "f (x0 ; x0 ) = 0

t!0
and
(2.4.7) lim "g (x0 + tx; x0 ) = "g (x0 ; x0 ) = 0:
t!0

## From (??) and (2.4.7), we have

hg(x0 + tx); ak i = hg(x0 ); ak i + t [rz g(x0 )] (x); ak +
(2.4.8)
+ t kx x0 k h"g (x0 + tx; x0 ); ak i;

## (2.4.9) limh"g (x0 + tx; x0 ); ak i = 0; for all k 2 f1; :::; qg:

t!0

i) From (2.4.3), (??); (2.4.6) and (??); we deduce that there exists
a real number r0 2]0; rb] such that

## (2.4.10) f (x0 + tx) < f (x0 ); for all t 2]0; r0 [:

ii) From (2.4.8)-(2.4.9); (??) and (2.4.3) it follows that, for each
k 2 Q; there exists a real number rk 2]0; rb] such that

## hg(x0 + tx); ak i > hg(x0 ); ak i = 0; for all t 2]0; rk [:

Consequently,
(2.4.11) g(x0 + tx) 2 H= (ak ); for all t 2 [0; rk [ and all k 2 Q:

## iii) Let k 2 P: Since g is pseudoconvex at x0 with respect to H= (ak ),

from (2.4.3), we obtain that

## hg(x0 + tx) g(x0 ); ak i = 0; for all t 2 [0; rb[:

Consequently,
(2.4.12) g(x0 + tx) 2 H= (ak ); for all t 2 [0; rb[ and all k 2 P;

because
hg(x0 ); ak i = 0; for all k 2 P E:
iv) From (2.4.8) and (2.4.9), it follows that, for each k 2 N =
f1; :::; qg n E; there exists a real number rk 2]0; rb] such that

## hg(x0 + tx); ak i = 0; for all t 2 [0; rk [:

Consequently,
94 2. OPTIMALITY CONDITIONS

## Let r be the minimum of all positive numbers

r0 ; rk (k 2 N [ Q)

## dened above: Obviously, r > 0: Then, from (2.4.4), (??), (2.4.10)-

(2.4.13), it follows that
r
x1 = x0 + x 2 X \ B(x0 ; re)
2
and
f (x1 ) < f (x0 ).
Consequently, system (2.4.5) has the solution x = x1 which is a con-
We establish now a necessary optimality criteria based on the above
linearization theorem.

## Theorem 2.4.2 Let M be a nonempty subset of Rn and x0 be an inte-

rior point of M: Let f : M ! R and g : M ! Rm be two dierentiable
functions at x0 ;

## be a polyhedral cone in Rm with nonempty interior and E; P; Q; N;

K(g(x0 )) be the sets dened by (2.4.2 ):
If x0 is a local solution of the optimization problem (2.4.1), then
there exist the points u0 2 R, v 0 2 Rm and 2 Rq such that:

## (2.4.14) u0 = 0; v 0 2 (K g(x0 ) ) K , u0 ; v 0 6= (0; 0);

(2.4.15) = ( k ) 2 Rq+ ;

## (2.4.16) k = 0; for all k 2 N;

q
X
0 k
(2.4.17) v = ka ;
k=1

(2.4.18) if Q = ;; then u0 6= 0;

## (2.4.19) if Q 6= ; and Q = ( k )k2Q then u0 ; Q 6= (0; 0) ;

4. NECESSARY OPTIMALITY CRITERIA 95

(2.4.20) g(x0 ); v 0 = 0;

## Proof. By Theorem 1, system (2.4.3) has no solution x 2 Rn .

A) If P = Q = ;; then system (2.4.3) becomes:
(
[rf (x0 )] (x) < 0
x 2 Rn

## and, in view of Corollary ??, system (2.4.3) is inconsistent if and only

if the system
(
[rf (x0 ) (w) = 0
w 2 R+ n f0g

## has a solution w: Then the theorem holds with u0 = w; v 0 = 0 and

= 0 2 Rq :
B) If Q = ; and P 6= ;; then E = P and system (2.4.3) becomes
8 0
< [rf (x )] (x) < 0
>
[rg(x0 )](ak )(x) = 0, k 2 P
>
:
x 2 Rn :

If
rf x0 = 0;
then the theorem holds with u0 = 1 2 R, v 0 = 0 2 Rm and = 0 2 Rq :
If
rf x0 6= 0;
then by Corollary ??, the system
T
Xh T
iT
(2.4.22) rf x0 (u) rg x0 (ak ) (yk ) = 0
k2E

## has a solution (u0 ; yE ) (0; 0) with

(2.4.23) u0 > 0 and yE = (yk )k2E = 0:

Let us dene
q
X
0 k
(2.4.24) v = ka and = ( 1 ; :::; q) 2 Rq ;
k=1
96 2. OPTIMALITY CONDITIONS

where
(
yk ; if k 2 E
(2.4.25) k =
0; if k 2 N = f1; :::; qg n E:

Since
q
X X
0 k
(2.4.26) v = ka = y k ak ;
k=1 k2E

we have that
(2.4.27) v 0 2 K(g(x0 ) ) K :

## From (2.4.23) (2.4.27), it follows that (2.4.14)-(2.4.18) are true. From

(2.4.24), (2.4.25) (2.4.26) and (2.4.2) it follows that equality (2.4.20)
holds and from (2.4.22), (2.4.24); (2.4.25) and (2.4.26) follows equality
(2.4.21).
C ) Let Q 6= ;: If
rf x0 = 0;
then the theorem holds with u0 = 1 2 R; v 0 = 0 2 Rm and = 0 2 Rq :
If
rf x0 6= 0;
then by Corollary ??, the system
T
X T
(2.4.28) rf x0 (u) rg x0 (ak )(yk ) = 0
k2E

has a solution
(2.4.29) u0 ; yE > (0; 0)
with
(2.4.30) u0 ; yQ > (0; 0);
where
yE = (yk )k2E and yJ = (yk )k2J :
Let us dene
q
X
0 k
(2.4.31) v = ka and = ( 1 ; :::; q) 2 Rq ;
k=1

where
(
yk ; if k 2 E
(2.4.32) k =
0; if k 2 N = f1; :::; qg n E:
4. NECESSARY OPTIMALITY CRITERIA 97

Obviously,
q
X X
0 k
(2.4.33) v = ka = yk ak 2 (K(g(x0 ))) :
k=1 k2E

## Statements (2.4.14)-(2.4.17) are true. From (2.4.31), (2.4.32) (2.4.33)

and (2.4.2) it follows that equality (2.4.20) holds and from (2.4.28),
(2.4.31) and (2.4.32) follows equality (2.4.21). It remains to show that
(u0 ; v 0 ) 6= (0; 0) and (u0 ; Q ) 6= (0; 0): If u0 = 0; then from (2.4.30) it
follows that Q 0; hence (u0 ; Q ) 6= (0; 0):
If u0 = 0; since Q 0; from (2.4.31)-(2.4.32) we have that the
system
8 P q
< k
ka = 0
k=1
:
= ( k) > 0
has a solution = 2 Rq : By Corollary ??, the system
(
w; ak > 0; k 2 f1; :::; qg
w 2 Cm

## has no solution w 2 Rm ; which contradicts intK 6= ;: Consequently,

v 0 6= 0 and hence (u0 ; v 0 ) 6= (0; 0): This completes the proof.
We derive now the Fritz John type saddle point theorem.

## Theorem 2.4.3 Let M be a nonempty subset of Rn , x0 be an interior

point of M and K Rm be a polyhedral cone with nonempty interior.
Let also f : M ! R be a dierentiable function at x0 and convex at x0
with respect to R+ and g : M ! Rm be a dierentiable function at x0
and concave at x0 with respect to K:
If x0 is a local solution of Problem (??), then there exist u0 2 R
and v 0 2 Rm such that

(2.4.34) u0 2 R+ , v 0 2 K g x 0 K ; u0 ; v 0 6= (0; 0) ;

(2.4.35) g(x0 ); v 0 = 0;

## for all (x; v) 2 M K ; where Lu0 : M Rm ! R is the Lagrangian

of Problem (??) dened by
98 2. OPTIMALITY CONDITIONS

## Proof. If x0 is a local solution of Problem (??) then, in view of

Theorem 2, there exist u0 2 R and v 0 2 Rm such that (2.4.14)-(2.4.21)
are true. Hence (2.4.34) and (2.4.35) hold. Moreover, since g(x0 ) 2 K;
we have
hg(x0 ); vi = 0; for all v 2 K

and then

## Lu0 (x0 ; v 0 ) = [u0 f (x0 ) hg(x0 ); v 0 i] = [u0 f (x0 )] =

= [u0 f (x0 ) hg (x0 ) ; vi] = Lu0 (x0 ; v); for all v 2 K :

to R+ ; then

## f (x) f (x0 ) = rf (x0 ) x x0 ;

for all x 2 M:
The function g is dierentiable at x0 and concave at x0 with respect
to K; then

## hg(x) g(x0 ); v 0 i 5 h[rg(x0 )] (x x0 ) ; v 0 i =

(2.4.37) D E
T
= x x0 ; [rg(x0 )] (v 0 ) ;

for all x 2 M:
From (??), (2.4.37) and (2.4.21), it follows that

Lu0 (x; v 0 ) Lu0 (x0 ; v 0 ) = [u0 (f (x) f (x0 ))] [hg(x) g(x0 ); v 0 i] =
= hx
T T
x0 ; [rf (x0 )] (u0 ) [rg(x0 )] (v 0 )i = 0 for all x 2 M:
4. NECESSARY OPTIMALITY CRITERIA 99

## Again, just as in the case of the saddle point necessary optimality

criteria, there is no guarantee that u0 > 0 in Theorem 2 above. In the
case when u0 = 0; the function f disappears from (2.4.14)-(2.4.21) and
hence from (??), therefore we have a degenerate case. As it was done
in section 3, it is possible to exclude such cases by introducing restric-
tions on the constraints. These restrictions are again called constraint
qualications. In addition to Slaters constraint qualication, Kar-
lins constraint qualication and the strict constraint qualication, we
shall introduce the Arrow-Hurwicz-Uzawa constraint qualication, the
reverse-concave constraint qualication, the Kuhn-Tucker constraint
qualication and the weak constraint qualication.

point of M and

## be a polyhedral cone in Rm : The function g : M ! Rm which denes

the set
X := fx 2 M : g(x) 2 Kg
is said to satisfy:
10 The Arrow-Hurwicz-Uzawa constraint qualication at x0 with
respect to X if g is dierentiable at x0 and the system
8 0 k
< [rg(x )] (x) ; a > 0; k 2 Q
>
(2.4.38) [rg(x0 )] (x) ; ak = 0; k 2 P
>
:
x 2 Rn ;

## where E; Q; P and K(g(x0 )) are given by (2.4.2); is consistent.

20 The reverse-concave constraint qualication at x0 with respect
to X if g is dierentiable at x0 and pseudoconvex at x0 with respect to
K(g(x0 )):
30 The Kuhn-Tucker constraint qualication at x0 with respect to
X if g is dierentiable at x0 and for each x 2 Rn nf0g with the property
that

## (2.4.39) rg(x0 ) (x) 2 K(g(x0 ));

there exists a real number " > 0 and a function : [0; "[! Rn dieren-
tiable at 0 such that
d
(0) = x0 ; (0) = x
dt
100 2. OPTIMALITY CONDITIONS

and
(t) 2 M; g( (t)) 2 K; for all t 2 [0; "[:
40 The weak constraint qualication at x0 with respect to X if g is
dierentiable at x0 and the system
8 0 T
>
< [rg(x )] (v) = 0
(2.4.40) hg(x0 ); vi = 0
>
:
v2K

## has the unique solution v = 0:

Some relations between the constraint qualications are established
in what follows.

## Theorem 2.4.5 Let M be a nonempty subset of Rn ; x0 be an interior

point of M;
K := \fH= (ak ) : k 2 f1; :::; qgg
be a polyhedral cone in Rn , g : M ! Rm be a function and

x0 2 X =: fx 2 M : g(x) 2 Kg:

## 10 If the function g satises the reverse-concave constraint quali-

cation at x0 with respect to X; then the function g satises the Arrow-
Hurwicz-Uzawa constraint qualication at x0 with respect to X:
20 If the function g satises the reverse-concave constraint quali-
cation at x0 with respect to X; then the function g satises the Kuhn-
Tucker constraint qualication at x0 with respect to X:
30 Let g be dierentiable at x0 and concave at x0 with respect to
K(g(x0 )): If the function g satises Slaters constraint qualication
with respect to X; then the function g satises the Arrow-Hurwicz-
Uzawa constraint qualication at x0 with respect to X:
40 Let g be dierentiable at x0 and concave at x0 with respect to K:
If the function g satises Slaters constraint qualication with respect
to X; then the function g satises the weak constraint qualication with
respect to X:
50 Let K be with nonempty interior. If the function g satises the
weak constraint qualication at x0 with respect to X; then the function
g satises the Arrow-Hurwicz-Uzawa constraint qualication at x0 with
respect to X:
60 Let g be dierentiable at x0 and concave at x0 with respect to
K(g(x0 )): If the function g satises the strict constraint qualication
4. NECESSARY OPTIMALITY CRITERIA 101

## with respect to X; then the function g satises the Arrow-Hurwicz-

Uzawa constraint qualication at x0 with respect to X:
70 Let g be dierentiable at x0 and concave at x0 with respect to K:
If the function g satises the strict constraint qualication with respect
to X; then the function g satises the weak constraint qualication at
x0 with respect to X:
80 Let M be convex, g be concave on M with respect to K; intK 6= ;
and g dierentiable at x0 : If the function g satises Karlins constraint
qualication with respect to X; then the function g satises the Arrow-
Hurwicz-Uzawa constraint qualication at x0 with respect to X:
Proof. 10 Assume that the function g satises the reverse-concave
constraint qualication at x0 with respect to X: Since g is pseudoconvex
at x0 with respect to K(g(x0 )); we have Q = ;: Then system (2.4.38)
becomes
(
[rg(x0 )] (x) ; ak = 0; k 2 P
x 2 Rn ;
which, obviously, is consistent (x = 0 2 Rn is a solution). Hence the
function g satises the Arrow-Hurwicz-Uzawa constraint qualication
at x0 with respect to X:
20 Assume that the function g satises the reverse-concave con-
straint qualication at x0 with respect to X and let x 2 Rn n f0g
satises (2.4.39) : Let : R ! Rn be the function dened by

## Obviously, the function is dierentiable at the point t = 0 and

d
(2.4.41) (0) = x0 and (0) = x:
dt
We will show that there exists a real number " > 0 such that

such that:

## (2.4.43) rg(x0 ) (t) x0 2 K(g(x0 ));

102 2. OPTIMALITY CONDITIONS

## for all t 2 [0; r[:

Since g is pseudoconvex at x0 with respect to K(g(x0 )); from (2.4.43)
we obtain

and hence

## (2.4.44) g( (t)) 2 K(g(x0 )); for all t 2 [0; r[;

because
g(x0 ) 2 K(g(x0 ))
and K(g(x0 )) is a convex cone.
If E = f1; :::; qg; the assertion is trivial. Assume that E 6= f1; :::; qg:
The function g being dierentiable at x0 ; we have

## g( (t)) = g(x0 + tx) = g(x0 ) + tf rg(x0 ) (x) +

+ t kxk "(x0 + tx; x0 );
for all t 2 [0; r[; where

## lim "(x0 + tx; x0 ) = "(x0 ; x0 ) = 0:

t!0

From this it follows that, for each k 2 f1; :::; qg; we have

## g( (t)); ak = g(x0 ); ak + t [rg(x0 )] (x) ; ak +

(2.4.45)
+ t kxk "(x0 + tx; x0 ); ak ; for all t 2 [0; r[
and
(2.4.46) lim "(x0 + tx; x0 ); ak = "(x0 ; x0 ); ak = 0:
t!0

If
k 2 N = fk 2 f1; :::; qg : g(x0 ); ak > 0g;
then, from (2.4.45) and (2.4.46) ; it follows that there exists a real
number rk 2]0; r0 [ such that

(2.4.47) g( (t)); ak = 0;

## for all t 2 [0; rk [; and all k 2 N:

If we denote by
" = minfrk : k 2 N g;
then " > 0 and, by (2.4.47) ; it follows that
4. NECESSARY OPTIMALITY CRITERIA 103

## (2.4.48) g( (t)) 2 \fH= (ak ) : k 2 N g; for all t 2 [0; "[:

Since
K = K(g(x0 )) \ \fH= (ak ) : k 2 P g ;

## and hence the Kuhn-Tucker constraint qualication at x0 with respect

to X is satised.
30 Assume that the function g satises Slaters constraint quali-
cation with respect to X; then there exists x1 M such that g (x1 ) 2
intS. Since the function g is dierentiable at x0 and concave at x0 with
respect to K(g(x0 )); we have

## for all k 2 E; because

g x0 ; ak = 0; for all k 2 E:

## Therefore, system (2.4.38) is consistent (x = x1 x0 is a solution) and

hence the Arrow-Hurwicz-Uzawa constraint qualication at x0 with
respect to X is satised.
40 Assume that the function g satises Slaters constraint quali-
cation with respect to X; then there exists x1 2 M such that g(x1 ) 2
intS: It follows that

## If the weak constraint qualication at x0 with respect to X is not

satised, then there exists a point v 0 2 K n f0g such that
( T
[rg(x0 )] (v 0 ) = 0
(2.4.50)
hg(x0 ); v 0 i = 0:

## The function g being dierentiable at x0 and concave at x0 with respect

to K; we have
104 2. OPTIMALITY CONDITIONS

## hg(x1 ); vi 5 hg (x0 ) ; vi + h[rg(x0 )] (x1 x0 ); vi =

(2.4.51) D E
0 1 0 0 T
= hg(x ); vi + x x ; [rg(x )] (v) ;

## for all v 2 K : If we choose v = v 0 2 K in (2.4.51) and use (2.4.50) ;

the inequality (2.4.51) becomes

g(x1 ); v 0 5 0;

## which contradicts (2.4.49) if we set v = v 0 2 K n f0g:

50 Assume, by contradiction, that the function g does not satisfy
the Arrow-Hurwicz-Uzawa constraint qualication at x0 with respect
to X: Then system (2.4.38) is inconsistent. Obviously, in this case
Q 6= ;; otherwise z = 0 2 Rn is a solution of system (2.4.38), hence g
satises the Arrow-Hurwicz-Uzawa constraint qualication at x0 with
respect to X:
We distingwish two cases:
i) On one hand, for each k 2 Q; we have
T
(2.4.52) rg(x0 ) ak = 0

Let us denote X
v= ak 2 R m :
k2Q

## On the other hand, adding equalities (2.4.52) gives

T
(2.4.53) rg(x0 ) (v) = 0:

Obviously,
(2.4.54) v2K

and hence
X
(2.4.55) g(x0 ); v = g(x0 ); ak :
k2Q

## We will now show by contradiction that v 6= 0: Assume that v = 0;

then the system
8 P k
> ka = 0
>
< k2Q
> ( k )k2Q = 0
>
:
k 2 R, k 2 Q;
4. NECESSARY OPTIMALITY CRITERIA 105

## is consistent ( k = 1; (k 2 Q) is a solution): Consequently, in view of

Corollary ??, the system
(
w; ak > 0; k 2 Q
w 2 Rm

## is inconsistent. It follows that the system

(
w; ak > 0; k 2 f1; :::; qg
w 2 Rm

## is inconsistent, too. But this contradicts intS 6= ;: Hence

(2.4.56) v 6= 0:
From (2.4.53) (2.4.56) it follows that system (2.4.40) has nonzero
solutions. Consequently, the function g does not satysfy the weak con-
straint qualication at x0 with respect to X; which contradicts the
hypothesis.
ii) There exists k 2 Q such that
T
rg(x0 ) ak 6= 0:

## Then, because system (2.4.38) is inconsistent, in view of Corollary ??,

there exist the real numbers k , (k 2 E) such that
(2.4.57) ( k )k2E > 0;
and
T
(2.4.58) rg(x0 ) (v) 6= 0:

where
q
(
X k, if k 2 E
k
v= ka and k =
k=1
0; if k 2 f1; :::; qg n E:

Obviously
(2.4.59) v2K
and
X
(2.4.60) g(x0 ); v = k g(x0 ); ak = 0:
k2E
106 2. OPTIMALITY CONDITIONS

the system 8 P k
>
> ka = 0
< k2E
> ( k )k2E > 0
>
:
k 2 R, k 2 E

(
w; ak > 0; k 2 E
w 2 Rm ;

## is inconsistent, too. Consequently, the system

(
w; ak > 0; k 2 f1; :::; qg
w 2 Rm ;

## is inconsistent, too. But this contradicts intS 6= ;: It follows that v 6= 0;

but this, together with (2.4.58) (2.4.60) contradicts the fact that the
function g satises the weak constraint qualication at x0 with respect
to X:
60 Assertion 60 follows immediately from assertion 20 of Theorem
9 and the above assertion 30 :
70 Apply assertion 20 of Theorem 9 and the above assertion 40 .
80 Apply assertion 30 of Theorem 9 and the above assertions 30 and
0
4:

## Theorem 2.4.6 Let M be a nonempty subset of Rn , x0 be an interior

point of M; and K be a polyhedral cone in Rm . Let also A 2 Rm n ;
b 2 Rm and g : M ! Rm be the function dened by
g(x) := A (x) + b; for all x 2 M:
If
x0 2 X := fx 2 M : g(x) 2 Kg;
then the function g satises the reverse-concave constraint qualication
at x0 with respect to X:
Proof. Obviously the function g is dierentiable at x0 and
rg(x0 ) = A:

## The function g being pseudoconvex with respect to K(g(x0 )) at the

point x0 ; it follows that g satises the reverse-concave constraint qual-
ication at x0 with respect to X:
4. NECESSARY OPTIMALITY CRITERIA 107

## Theorem 2.4.7 Let M be a nonempty subset of Rn , x0 be an interior

point of M and K be a polyhedral cone in Rm . Let also A 2 Rm n ;
b 2 Rm and g : M ! Rm be the function dened by
g(x) =: A (x) + b for all x 2 M:
If
X =: fx 2 M : g(x) 2 Kg;
and x0 2 X, then the function g satises the Kuhn-Tucker constraint
qualication at x0 with respect to X:
Proof. Apply Theorem 6 and Theorem 5.
We derive now the fundamental necessary optimality criterion of
the optimization problem.

## Theorem 2.4.8 Let M be a nonempty subset of Rn ; x0 be an interior

point of M and K be a polyhedral cone in Rm . Let also f : M ! R
and g : M ! Rm be two dierentiable functions at x0 : Moreover, let us
assume that one of the following conditions fullled:
10 The function g satises the Arrow-Hurwicz-Uzawa constraint
qualication at x0 with respect to X and intK 6= ;; or
20 The function g satises the Kuhn-Tucker constraint qualication
at x0 with respect to X; or
30 The function g satises the reverse-concave constraint qualica-
tion at x0 with respect to X; or
40 The function g satises the weak constraint qualication at x0
with respect to X and intS 6= ;; or
50 The function g is concave at x0 with respect to K(g(x0 )); and
satises the Slater constraint qualication with respect to X; or
60 The function g is concave at x0 with respect to K(g(x0 )); and
satises the strict constraint qualication with respect to X; or
70 The set M is convex, intS 6= ;; g is concave on M with respect
to K; and satises Karlins constraint qualication with respect to X:
If x0 is a local solution of Problem (2.4.1); then there exists a point
v 2 Rm such that:
0

## (2.4.61) v 0 2 (K(g(x0 ))) K ;

(2.4.62) g(x0 ); v 0 = 0;
and
T
(2.4.63) rf (x0 ) rg(x0 ) v 0 = 0:
108 2. OPTIMALITY CONDITIONS

## Proof. The cone K Rm being polyhedral, there exists an integer

number q = 1 and q points a1 ; :::; aq 2 Rm n f0g such that
K = \fH= (ak ) : k 2 f1; :::; qgg;
where
H= (ak ) = fv 2 Rm : v; ak = 0g; for all k 2 f1; :::; qg:

## In view of Theorem 5 above we only need to establish the proof under

the assumptions 10 or 20 :
10 Since the hypotheses of Theorem 2 are satised there exist u1 2
R, v 1 2 Rm and = ( k ) 2 Rq such that

## (2.4.64) u1 = 0; v 1 2 (K g(x0 ) ) K , u1 ; v 1 6= (0; 0);

(2.4.65) = ( k ) 2 Rq+ ;

## (2.4.66) k = 0; for all k 2 N;

q
X
1 k
(2.4.67) v = ka ;
k=1

(2.4.68) if Q = ;; then u1 6= 0;

## (2.4.69) if Q 6= ; and Q = ( k )k2Q then u1 ; Q 6= (0; 0) ;

(2.4.70) g(x0 ); v 1 = 0;

## We will show that u1 6= 0: Indeed

i) If Q = ;; then (2.4.68) shows that u1 6= 0:
ii) If Q 6= ;; since g satises the Arrow-Hurwicz-Uzawa constraint
qualication at x0 with respect to X, system (2.4.38) is consistent. Let
x1 be a solution of system (2.4.38) : Assume that u1 = 0; then from
(2.4.69) it follows that
(2.4.72) Q = ( k )k2Q > 0;

## and by (2.4.71) we obtain that

(2.4.73) [rg x0 ]T (v 1 ) = 0:
4. NECESSARY OPTIMALITY CRITERIA 109

## Now, from (2.4.73) ; (2.4.66) ; (2.4.67) ; (2.4.72) and (2.4.38) we de-

duce that
0 = hx1 ; [rg (x0 )]T (v 1 ))i = h[rg(x0 )] (x1 ) ; v 1 i =
P q P
= [rg(x0 )] (x1 ) ; ka
k
= [rg(x0 )] (x1 ) ; ka
k
=
k=1 k2E
P P
= [rg(x0 )] (x1 ) ; ak = k [rg(x0 )] (x1 ) ; ak > 0;
k2E k2Q

## which is a contradiction. Hence u1 > 0:

Since v 1 2 (K(g(x0 ))) K and since (K(g(x0 ))) and K are
1
cones and u > 0; it follows that
(1=u1 )v 1 2 (K(g(x0 ))) K :

v 0 = (1=u1 )v 1

## it results that there exists v 1 2 Rm such that (2.4.61) (2.4.63) hold.

20 Let x 2 Rn n f0g with the property that
(2.4.74) [rg(x0 )] (x) 2 K(g(x0 )):

## Then, since g satises the Kuhn-Tucker constraint qualication at x0

with respect to X; there exists a real number " > 0 and a function
: [0; "[! Rm dierentiable at the point t = 0 such that
d
(2.4.75) (0) = x0 ; (0) = x
dt
and
(t) 2 M; g( (t)) 2 K, for all t 2 [0; "[:
Since x0 is a local solution of Problem (2.4.1) ; we have
d
f ( (t)) jt=0 = 0;
dt
hence
(2.4.76) [rf (x0 )] (x) = 0:

## Therefore, each x 2 Rn which satises (2.4.74) satises (2.4.76) ; too.

Hence the system
8
< [rf (x0 )] (z) < 0
(2.4.77) h i
: [rg(x0 )]T ak (x) = 0; k 2 E
110 2. OPTIMALITY CONDITIONS

is inconsistent.
If
rf (x0 ) = 0;
then the theorem holds with v 0 = 0 2 Rm: :
If
rf (x0 ) 6= 0;
then system (2.4.77) being inconsistent, it follows that the system

T
Xh T
iT
(2.4.78) rf (x0 ) (u) rg(x0 ) ak (yk ) = 0;
k2E

has a solution (u1 ; yE ) > 0 with u1 > 0; where yE = (yk )k2E : Now from
(2.4.78) ; we obtain
T
rf (x0 ) rg(x0 ) v 0 = 0;

where X yk
v0 = ak :
k2E
u1
By dening
(
(yk )= (u1 ) , if k 2 E
k =
0; if k 2 f1; :::; qg n f0g

we have that
q
X
0 k
v = ka and v 0 2 (K(g(x0 ))) K :
k=1

Moreover X
g(x0 ); v 0 = k g(x0 ); ak = 0
k2E

## and the theorem is proved.

We give the following Kuhn-Tucker type saddlepoint theorem as a
consequence of Theorem 8.

## Theorem 2.4.9 Let M be a nonempty subset of Rn ; x0 be an interior

point of M and K be a polyhedral cone in Rm . Let also f : M ! R
be a dierentiable function at x0 convex at x0 with respect to R+ and
let g : M ! Rm be a dierentiable function at x0 concave at x0 with
respect to K: In addition, let one of the following conditions hold:
5. SUFFICIENT CONDITIONS 111

## 10 The function g satises the Arrow-Hurwicz-Uzawa complex con-

straint qualication at x0 with respect to X and intK 6= ;; or
20 The function g satises the Kuhn-Tucker complex constraint
qualication at x0 with respect to X; or
30 The function g satises the reverse-concave complex constraint
qualication at x0 with respect to X; or
40 The function g satises the weak complex constraint qualication
at x0 with respect to X and intK 6= ;; or
50 The function g is concave at x0 with respect to K(g(z 0 )) and
satises the Slater complex constraint qualication with respect to X;
or
60 The function g is concave at x0 with respect to K(g(z 0 )) and
satises the strict complex constraint qualication with respect to X; or
70 The set M is convex, intK 6= ;; g is concave on M with respect to
K and satises the Karlin complex constraint qualication with respect
to X:
If x0 is a local solution of Problem (2.4.1) ; then there exists a point
v 2 Rm such that:
0

v 0 2 (K(g(z 0 ))) K ;

g(x0 ); v 0 = 0;

and
L1 (x0 ; v) 5 L1 (x0 ; v 0 ) 5 L1 (x; v 0 );

## for all (x; v) 2 M K ; where L1 : M K ! R is the Lagrangian of

Problem (2.4.1) given by the formula
L1 (x; v) = f (x) hg(x); vi for all (x; v) 2 M K :

## 5. Su cient Conditions for the Optimal Solutions of the

Optimization Problems with Dierentiable Hypothesis
The purpose of this paragraph is to give some su cient conditions
for the solution of an optimization problem
Consider the optimization problem:
(2.5.1) min f (x)
112 2. OPTIMALITY CONDITIONS

subject to (
x2M
g(x) 2 K;
where M is a nonempty subset of Rn ; K is a nonempty subset of Rm
and f : M ! R and g : M ! Rm are two functions.
Theorem 2.5.1 Let M be a nonempty subset of Rn and x0 be an
interior point of M: Let also f : M ! R and g : M ! Rm be two
dierentiable functions at x0 and K be a closed convex cone in Rm :
A su cient condition for
x0 2 X := fx 2 M : g(x) 2 Kg;
to be a solution of Problem (2.5.1) is that there exists a point v 0 2 Rm
such that the function L : M ! R dened by
(2.5.2) L(x) := f (x) hg(x); v 0 i; for all x 2 M;
is pseudoconvex at x0 with respect to R+ and
(2.5.3) v0 2 K ;

(2.5.4) hg(x0 ); v 0 i = 0;
and
(2.5.5) hx x0 ; rf (x0 ) [rg(x0 )]H v 0 i = 0;
for all x 2 X:
Proof. Obviously, the function L is dierentiable at x0 and
rL(x0 ) = rf (x0 ) [rg(x0 )]T v 0 ;

## Then (2.5.5) can be written as

(2.5.6) x x0 ; rL(x0 ) = 0; for all x 2 X:

## Hence x0 2 X; L is a dierentiable function at x0 and x pseudoconvex

at x0 with respect to R+ : Moreover, (2.5.6) is true. Therefore, the
hypotheses of Theorem 7 are satised. Then, it follows that
(2.5.7) L(x) = L(x0 ); for all x 2 X:
Let now x 2 X: Since v 0 2 K and g(x) 2 K we have
hg(x); v 0 i = 0:
Taking into account that (2.5.7) and (2.5.4) hold, we obtain
5. SUFFICIENT CONDITIONS 113

## f (x) = f (x) hg(x); v 0 i = L(x) = L(x0 ) = f (x0 ) hg(x0 ); v 0 i =

= f (x0 ); for all x 2 X:
Therefore x0 is a solution of Problem (2.5.1).

## Example 2.5.2 Consider the optimization problem:

(2.5.8) min f (x1 ; x2 ) := x21 + x22
subject to
(
(x1 ; x2 ) 2 R2
g (x1 ; x2 ) := z 2 + 2zz + z 2 2iz + 2iz 2 S;

## where K = f0g R. Obviously,

f (x1 ; x2 ) = x21 + x22 ; for all (x1 ; x2 ) 2 R2
and
g(x1 ; x2 ) = (x1 )2 + (x2 )2 ; for all (x1 ; x2 ) 2 R2 :
For v 0 = 0 2 R, the function L : R2 ! R dened by (2.5.2) becomes
L (x1 ; x2 ) = x21 + x22 ; for all (x1 ; x2 ) 2 R2

## and is convex on R2 with respect to R+ . At the point x0 = (0; 0) the

hypotheses of Theorem 1 are satised, then x0 = (0; 0) is a solution of
Problem (2.5.8).

Corollary ??.

## Theorem 2.5.4 Let M be a nonempty subset of Rn ; x0 be an interior

point of M and K be a polyhedral cone in Rm . Let also f : M ! R be a
dierentiable function at x0 which is pseudoconvex at x0 with respect to
R+ and g : M ! Rm be a dierentiable function at x0 and quasiconcave
at x0 with respect to K(g(x0 )):
A su cient condition for
x0 2 X := fx 2 M : g(x) 2 Kg

## to be a solution of Problem (2.5.1) is that there exists a point v 0 2 Rm

such that
(2.5.9) v 0 2 K g x0

and
(2.5.10) hx x0 ; rf (x0 ) [rg(x0 )]H v 0 i = 0; for all x 2 X:
114 2. OPTIMALITY CONDITIONS

## Proof. Let x 2 X; then

(2.5.11) g(x) 2 K K(g(x0 ))

and
(2.5.12) g(x0 ) 2 K(g(x0 )):

## Since K(g(x0 )) is a convex cone, from (2.5.11) and (2.5.12) ; we obtain

(2.5.13) g (x) g(x0 ) 2 K(g(x0 )):

## The function g being qusiconcave at x0 with respect to K(g(x0 )), from

(2.5.13) ; it results that
(2.5.14) [rg(x0 )] x x0 2 K(g(x0 )):

## Now from (2.5.9) and (2.5.14) ; it follows that

[rg(x0 )] x x0 ; v 0 = 0;

or, if we take into account the properties of the inner product we have
(2.5.15) x x0 ; [rg(x0 )]H v 0 = 0:
Adding (2.5.10) and (2.5.15) we obtain
x x0 ; rf (x0 ) = 0;
f (x) = f (x0 )
if we take into account the fact that the function f is pseudoconvex at
z 0 with respect to R+ : Therefore x0 is a solution of Problem (2.5.1) :

## Theorem 2.5.5 Let M be a nonempty subset of Rn ; x0 be an interior

point of M , f : M ! R and g : M ! Rm be twice dierentiable
functions at x0 and
K =: \fH= (ak ) : k 2 f1; :::; qgg

be a polyhedral cone in Rm :
Let
(2.5.16) ( k ) 2 Rq+
such that
q
X
0 k
(2.5.17) v = ka 2K
k 1
5. SUFFICIENT CONDITIONS 115

satises
(2.5.18) g x0 ; v 0 = 0

and
(2.5.19) rf (x0 ) [rg(x0 )]H v 0 = 0:

## If, for each solution d 2 Rn n f0g of the system

(
h[rg(x0 )] (d) ; ak i = 0; k 2 fk 2 E : k > 0g
(2.5.20)
[rg(x0 )] (d) 2 K(g(x0 ));

we have
(2.5.21) hd; [r2 f (x0 ) r2 g(x0 )u0 ]di > 0;

## then x0 is a local solution of Problem (2.5.1):

Proof. Assume that x0 is not a local solution of Problem (2.5.1):
Then there exists a sequence (xj )j2N from X n fx0 g which converges to
x0 such that
(2.5.22) f (xj ) < f (x0 ); for all j 2 N.

## For each j 2 N we denote by

1 j
tj = xj x0 and dj = (x x0 ):
tj
Then we have
tj > 0 and xj = x0 + tj dj ; for all j 2 N

and
lim tj = 0:
j!1

Since
dj = 1; for all j 2 N;
it follows that the sequence (dj )j2N contains a convergent subsequence.
Without loss of generality we can assume that the sequence itself is
convergent. Let
d = lim dj :
j!1
Then
kdk = 1:
From
xj 2 X; for all j 2 N;
116 2. OPTIMALITY CONDITIONS

we have
g(xj ) 2 K; for all j 2 N
and hence
g(xj ) g(x0 ) 2 K(g(x0 )); for all j 2 N.
Moreover
1
d x0 + tj dj g x0 2 K:
tj
From this, by passing to limit, we obtain that
(2.5.23) [rg(x0 )] (d) 2 K(g(x0 ));

or, equivalently
h[rg(x0 )] (d) ; ak i = 0, for all k 2 E:

## From (2.5.22) we deduce that

1
[f (x0 + tj dj ) f (x0 )] < 0; for all j 2 N.
tj

## From this, by passing to limit, we obtain that

(2.5.24) [rf (x0 )] (d) 5 0:

## We will show, by contradiction, that

(2.5.25) h[rg(x0 )] (d) ; ak i = 0 for all k 2 fk 2 E : k > 0g :

## Assume that there exists k0 2 E with k0 > 0 such that

(2.5.26) h[rg(x0 )] (d) ; ak0 i > 0:

## Then, from (2.5.16) ; (2.5.17) ; (2.5.18) ; (2.5.26) ; we obtain

P
q
h[rg(x0 )] (d) ; v 0 i = h[rg(x0 )] (d) ; ka
k
i=
k=1
P
q P
= k h[rg(x
0
)] (d) ; ak i = k h[rg(x
0
)] (d) ; ak i =
k=1 k2E

= k0 h[rg(x
0
)] (d) ; ak0 i > 0:

Hence
(2.5.27) h[rg(x0 )] (d) ; v 0 i > 0:
5. SUFFICIENT CONDITIONS 117

## On the other hand, from (2.5.19) and (2.5.24), we have

h[rg(x0 )] (d) ; v 0 i = hd; [rg(x0 )]H (v 0 )i =
= hd; [rf (x0 )]i = [rf (x0 )] (d) 5 0;

From (2.5.25) and (2.5.23); it follows that d is a solution of system
(??): Since d 2 Rn n f0g; we have
(2.5.28) hd; [r2 f (x0 ) r2 g(x0 )u0 ]di > 0:

## If we denote by F : M ! R the function

F (x) = f (x) hg(x); u0 i for all x 2 M;
then, from (2.5.28) we obtain
(2.5.29) hd; [r2 F (x0 )]d > 0:
We have
F (xj ) = f (xj ) hg(xj ); v 0 i 5 f (x0 ) =
5 f (x0 ) hg(x0 ); v 0 i = F (x0 ) for all j 2 N,
because
g(xj ) 2 K for all j 2 N,
v 0 2 K and hg(x0 ); v 0 i = 0:
Since
X
m
0 0
rF (x ) = rf (x ) vk0 rg(x0 );
k=1
it follows that
F (x0 + tj dj ) F (x0 ) tj hdj ; rF (x0 )i =
= f (x0 + tj dj ) hg(x0 + tj dj ); v 0 i f (x0 ) + hg(x0 ); v 0 i
P m
tj hdj ; rf (x0 ) vk0 rgk (x0 ) i =
k=1
0 j 0
= f (x + tj d ) f (x ) hg(x0 + tj dj ); v 0 i
f (x0 + tj dj ) f (x0 ) for all j 2 N.

Hence
1
[F (x0 + tj dj ) F (x0 ) tj hdj ; rF (x0 )i] 5 0 for all j 2 N.
t2j

## Then, by Theorem , we have

hr2 F (x0 )i 5 0
118 2. OPTIMALITY CONDITIONS

Therefore x0 is a local solution of Problem (2.5.1):
Bibliography

##  J.-P. Aubin: Optima and Equilibria: an Introduction to Nonlinear Analysis,

Springer-Verlag, Berlin, 1993
 V. Barbu and Th. Precupanu: Convexity and Optimization in Banach
Spaces, Publishing House of Roumanian Academy and Reidel Publishing
Comp., 1986
 A. Ben-Israel: Linear Equations and Inequalities on Finite Dimensional, real
or complex, vector space: a Unied Theory, J. Math. Anal. Appl., 27(1969),
no.2, 367-389
 J.M. Borwein and A.S. Lewis: Convex Analysis and Nonlinear Optimiza-
tion, Springer-Verlag, New York, Berlin, Heidelberg, 2000
 W.W. Breckner: Introducere n teoria problemelor de optimizare convexa cu
restrictii, partea I, Editura Dacia, Cluj, 1974
 W.W. Breckner: Cercetare operationala, Universitatea Babes-Bolyai, Fa-
cultatea de Matematic a, Cluj-Napoca, 1981
 W.W. Breckner si D.I. Duca: Culegere de probleme de cercetare operatio-
nala, Universitatea din Cluj-Napoca, Facultatea de Matematic a, Cluj-Napoca,
1983
 F.H. Clarke: Optimization and Nonsmooth Analysis, John Wiley&Sons, New
York, 1983
 M. Dragomirescu si M. Mali ta: Programare neliniara, Editura stiintic
a,
Bucuresti, 1972
 D.I. Duca: Constraint Qualications in Nonlinear Programming in Complex
Space, Studia Univ. Babes-Bolyai, Math., 23(1978), no.1, 61-65
 Duca D.I.: The Vectorial Programming Problem in Complex Space, Proceed-
ing of the Third Colloquim on Operations Research, Cluj-Napoca, october 20-
21, 1978, Babes-Bolyai University of Cluj-Napoca, Cluj-Napoca, 1979, pp.82-
89
 D.I. Duca: On Vectorial Programming Problem in Complex Space, Studia
Univ. Babes-Bolyai, Math., 24(1979), no.1, 51-56
 Duca D.I.: Proper E ciency in the Complex Vectorial Programing, Studia
Univ. Babes-Bolyai, Math., 25(1980), no.1, 73-80
 Duca D.I.: O teorem a de punct sa n programarea matematic a n domeniul
complex, Lucr arile Seminarului itinerant de ecuatii functionale, aproximare si
convexitate, Cluj-Napoca, mai, 1980, Univ. Babes-Bolyai, Cluj-Napoca, 1980,
pp. 35-39
 Duca D.I.: Necessary Optimality Criteria in NonlinearProgramming in Com-
plex Space with Dierentiability, Mathematica-LAnalyse numerique et la the-
orie de lapproximation, 9(1980), no.2, 163-179

119
120 BIBLIOGRAPHY

##  Duca D.I.: Saddlepoint Optimality Criteria of Nonlinear Programming in

Complex Space without Dierentiability, Studia Univ. Babes-Bolyai, Math.,
25(1980), no.4, 39-46
 Duca D.I.: On Su cient Optimality Conditions in Nonlinear Programming
in Complex Space, Mathematica, 22(45)(1980), no.2, 263-267
 Duca D.I.: On some Types of Optimization Problems in Complex Space,
LAnalyse numerique et la theorie de lapproximation, 10(1981), no.1, 11-16
 Duca D.I.: Linear Optimality Criteria in Nonlinear Programming in Complex
Space, Studia Univ. Babes-Bolyai, Math., 26(1981), no.3, 73-79
 Duca D.I.: Asupra unui rezultat din teoria dualit atii in programarea ma-
tematica n domeniul complex, Lucr arile Seminarului itinerant de ecuatii
functionale, aproximare si convexitate (Cluj-Napoca, decembrie 1981), 119-
126, Univ. Babes-Bolyai, Cluj-Napoca, 1981
 Duca D.I.: Mathematical Programming in Complex Space, Doctoral Thesis,
University of Cluj-Napoca, 1981
 Duca D.I.: (2,1)-Order Duality in Nonlinear Programming in Complex Space,
Lucrarile Seminarului Th. Angheluta (Cluj-Napoca, iunie 1983), 99-104,
Preprint, Institutul Politehnic din Cluj-Napoca, Cluj-Napoca, 1983
 Duca D.I.: E ciency Criteria in Vectorial Programming in Complex Space,
Itinerant Seminar on Functional Equations, approximation and Convexity
(Cluj-Napoca, 1983), 51-54, Preprint 83-2,Univ. Babes-Bolyai,Cluj-Napoca,
1983
 Duca D.I.: On Duality in Nonlinear Programming in Complex Space, Itin-
erant Seminar on Functional, Approximation and Convexity (Cluj-Napoca,
1984), 45-48, Preprint 84-6, Univ. Babes-Bolyai, Cluj-Napoca, 1984
 Duca D.I.: E ciency Criteria in Vectorial Programming in Complex Space
without Convexity, Cahiers du Centre Etudes Rech. Oper., 26(1984), no.3-4,
217-226
 Duca D.I.: Duality in Mathematical Programming in Complex Space.
Converse Theorems, LAnalyse numerique et la theorie de lapproximation,
13(1984), no.1, 15-22
 Duca D.I.: Second-Order Duality in Nonlinear Programming in Complex
Space, Proceedings of the Colloquium on Approximation and Optimization
(Cluj-Napoca, October, 1984), 213-222, Univ. Cluj-Napoca, Cluj-Napoca, 1985
 Duca D.I.: On the Higher-Order in Nonlinear Programming in Complex
Space, Seminar on Optimization Theory (Cluj-Napoca, 1985), 39-50, Preprint
85-5, Univ. Babes-Bolyai, Cluj-Napoca, 1985
 Duca D.I.: Vectorial Programming in Complex Space, Seminar on Opti-
mization Theory (Cluj-Napoca, 1986), 3-82, Preprint 86-8, Univ Babes-Bolyai,
Cluj-Napoca, 1986
 Duca D.I.: On the Farkas Type Theorem for Complex Linear Equations and
Inequalities, Itinerant Seminar on Functional Equations, Approximation and
Convexity (Cluj-Napoca, 1987), 143-148, Preprint 87-6, Univ. Babes-Bolyai,
Cluj-Napoca, 1987
 Duca D.I.: On Theorems of the Alternative for Nonhomogeneous Complex
Linear Equations and Inequalities, Seminar on Optimization Theory (Cluj-
Napoca, 1987), 29-40, Preprint 87-8, Babes-Bolyai University, Cluj-Napoca,
1987
BIBLIOGRAPHY 121

##  Duca D.I.: Saddlepoint Necessary Criteria for Nonlinear Programming

in Complex Space, LAnalyse numerique et la theorie de lapproximation,
16(1987), no.1, 29-33
 Duca D.I.: Theorem of Motzkins Alternative for Nonhomogeneous Com-
plex Linear Equations and Inequalities, LAnalyse numerique et la theorie de
lapproximation, 16(1987), no.2, 117-126
 Duca D.I.: On Theorems of the Alternative for Complex Linear Equations
and Inequalities, Seminar on Optimization Theory (Cluj-Napoca, 1989), 35-48,
Preprint 89-8, Babes-Bolyai University, Cluj-Napoca, 1989
 Duca D.I.: Theorems of the Alternative for Nonhomogeneous Complex Linear
Equations and Inequalities, Mathematica, 31(54)(1989), no.2, 127-142
 Duca D.I.: The Dual of the Dual in Mathematical Programming in Complex
Space, LAnalyse numerique et la theorie de lapproximation, 19(1990), no.1,
35-43
 Duca D.I., Duca E. and Lup sa L.: On the Structure of the Set of Points
Dominated and Nondominated in a Vectorial Optimization Problem, Seminar
on Functional Equations, Approximation and Convexity (Cluj-Napoca, 1993),
20-26, Preprint 93-6, Babes-Bolyai University, Cluj-Napoca, 1993
 D.I. Duca and L. Lup sa: On the E-Epigraph of an E-Convex Function,
Journal of Optimization Theory and Applications, 129(2006), 2, 341-348
 Duca D.I. and Stancu-Minasian I.M.: Mathematical Programming in
Complex Space: A Comprehensive Bibliography, LAnalyse numerique et la
theorie de lapproximation, 17(1988), no.2, 133-140
 Duca E. si Duca D.I.: Asupra structurii multimii punctelor eciente ntr-o
problema de programare vectorial a n domeniul complex, Lucrarile Seminarului
itinerant de ecuatii functionale aproximare si convexitate, Cluj-Napoca, mai
1979, Univ. Babes-Bolyai, Cluj-Napoca, 1979, pp. 41-47
 Farkas J.: Theorie der einfachen Ungleichungen, Journal fur die reine und
angewandte Mathematik, 124(1902), 1-27
 Gale D.: A Geometric Duality Theorem with Economic Applications, Review
of Economic Studies, 34(1967), 19-24
 Gordan P.: Uber die Auosung linearer Gleichungen mit reelen Coe cienten,
Mathematische Annalen, 6(1873), 23-28
 Karlin S.: Mathematical Methods and Theory in Games, Programming and
Economics, McGraw-Hill, New York, 1960
 Karush W.: Minima of Functions of Several Variables with Inequalities as
Side Conditions, Masters thesis, University of Chicago, 1939
 Kuhn H.W. and Tucker A.W.: Nonlinear Programming, In: Proceedings of
the Second Berkeley Symposium on Mathematical Statistics and Probability,
University of California Press, Berkeley,1951
 P.J. Laurent: Approximation et optimisation, Herman, Paris, 1972
 D.T. Luc: Theory of Vector Optimization, Lecture Notes in Econ. and Math.
Systems, vol. 319, Springer-Verlag, Berlin, 1989
 Lup sa L., Duca E. and Duca D.I.: On the Structure of the Set of Points
Dominated and Nondominated in an Optimization Problem, Revue dAnalyse
Numerique et de Theorie de lApproximation, 22(1993), no.2, 193-199
 O.L. Mangasarian: Nonlinear Programming, McGrow-Hill Book Company,
New York, 1969
122 BIBLIOGRAPHY

 T.S. Motzkin: Beitrage zur Theorie der linearen Ungleichungen, Inaugural
Dissertation, Basel 1933; Jerusalem Azriel 1936
 A. Ruszczynski: Nonlinear Optimization, Priceton University Press, Prince-
ton and Oxford, 2006
 I.M. Stancu-Minasian: Fractional Programming. Theory, Methods and Ap-
plications, Kluwer Academic Publishers Group, Dordrecht, 1997
 Stancu-Minasian I.M., Duca D.I. and Nishida T.: Multiple Objective
Linear Fractional Optimization in Complex Space, Math. Japonica, 35(1990),
no.1, 195-203
 J. Werner: Optimization Theory and Applications, Friendr. Vieweg&Sohn,
Braunschweig, 1984
 Zalinescu C.: A Generalization of the Farkas Lemma and Applications to
Convex Programming, J. Math. Anal. Appl., 66(1978), 651-678
 C. Zalinescu: Programare matematica n spatii normate innit dimensionale,