Pre-sessional Mathematics and Statistics EC961

Multivariate Calculus and Constrained Optimisation

Pre-requisites: For one variable function: simple derivatives (of log, exponential and
power functions etc.) and the chain rule, geometric interpretation of first and second
order derivatives of a function, concave and convex functions, stationary points,
Taylor series expansion. Partial derivatives.
1. Convex set and multivariate function
Definition: A set S is convex if for any two points
x
, ' x in S the point
' ) 1 ( x x +
is also in S when 1 0 < < . (Note that
x
is the abbreviation of
) , , , (
2 1 n
x x x x
.)
Example 1: The set { 0 , 0 ; 0 , 0 : ) , (
2 1 2 1
> > b a b x a x x x S is convex.
Definition: Let
n
S f : , then
f
is a function of
n
variables.
Here
f
maps a point in S to a single value in . In this course, we usually deal
with functions defined on convex set S .
Definition: The graph of
f
consists of the set of points
)) , , , ( , , , , (
2 1 2 1 n n
x x x f x x x
in
1 +

n
for
) , , , (
2 1 n
x x x
in S .
Example 2: Let S be the one given in example 1, and
2
2
2
1 2 1
) , ( x x x x f + . The
graph is as below
O
b
a
X
X
' ) 1 ( X X +
1
x
2
x
O
) , (
2 1
x x f
1
x
2
x
1
Definition: Let
f
be a function of two variables and
c
be a constant. The set of all
) , , , (
2 1 n
x x x
such that
c x x x f
n
) , , , (
2 1

is called a level curve of
f
(with
value
c
).
If
f
is a two variable function, the level curve is on a two-dimensional plane. (If
f

is a single variable function, what is its level curve?) For functions of three
variables, one would have a level surface.
Note that indifference curves are just level curves for utility functions.
Example 3: The level curve (heavy curve) of the function in example 2 with a value
0 > c is drawn below:
Definition: An upper level set of a function
f
with given
c
is a set
) (c U
such
that
{ c x f S x x c U ) ( and : ) (
.
A lower level set,
) (c L
, is defined as
{ c x f S x x c L ) ( and : ) (
.
So the upper level set is a collection of all level curves above value
c
, and the lower
level set is a collection of all level curves below value
c
.
Example 4: the upper and lower level sets for function in example 3 with respect to
c
are drawn below:
Definition: If
f
is a real-valued function, its gradient, denoted
f
, is defined as
O
) , (
2 1
x x f
1
x
2
x
c
O
) , (
2 1
x x f
1
x
2
x
c
Upper level set
O
) , (
2 1
x x f
1
x
2
x
c
Lower level set
2

,
`

.
|




n
n
x f
x f
x f
x x x f
/
/
/
) , , , (
2
1
2 1

.
Example 5: For the function in example 2 (let
2
S ), the gradient of the function is

,
`

.
|

,
`

.
|



2
1
2
1
2 1
2
2
/
/
) , (
x
x
x f
x f
x x f
.
Definition: A collection of second order derivatives arranged in the following way is
called the Hessian matrix

,
`

.
|

2 2
2
2
1
2
2
2 2
2
2
2 1
2
1
2
1 2
2 2
1
2
2 1
/ / /
/ / /
/ / /
) , , , (
n n n
n
n
n
x f x x f x x f
x x f x f x x f
x x f x x f x f
x x x H

Example 6: For the function in example 2, its Hessian matrix is

,
`

.
|

,
`

.
|

2 0
0 2
/ /
/ /
) , (
2
2
2
2 1
2
1 2
2 2
1
2
2 1
x f x x f
x x f x f
x x H
(Youngs Theorem: If a two variable function is
2
C , i.e. twice continuously
differentiable, then
2 1
2
1 2
2
/ / x x f x x f .
In this course, we only consider these functions. In this case, the Hessian matrix is
symmetric, i.e.
H H
T

.)
Theorem 1 (Taylor's Formula): Let
x
be a given point in
n
where the function
f
is defined, and h is a small vector, i.e.
h
close to zero. If
f
is
2
C , then
) ( ) (
2
1
) ( ) ( ) (
3
h O h x H h x f h x f h x f
T T
+ + + +
,
where
) (
3
h O
means of the same order as
3
h
.
If we know the property of a function at a given point
x
, then Taylors formula
allows us to approximate the value of that function in the neighbourhood of
x
(say,
h x + ) as long as the function is relatively smooth (
2
C ).
2. Stationary points and their classifications
2.1 Stationary points
Definition: A stationary point of a function is a place where
0 f
, that is a place
where the function's value does not change.
Example:
3

x *
f ( x )
x *
f ( x )
The definition of a stationary point also tells us how to find one: find the set of points
where
0 f
, that is to solve
n
equations in
n
unknowns.
Example 7a: The stationary point for the function in example 5 is

'

,
`

.
|

,
`

.
|

0
0
0
0
2
2
) , (
2
1
2
1
2 1
x
x
x
x
x x f
.
Example 7b: Find stationary points of the function ) /( ) , (
2 2
y x xy y x f + for
) 0 , 0 ( ) , ( y x
.

'

,
`

.
|

,
`

.
|
+
+

) 2 ( 0 ) (
) 1 ( 0 ) (
0
0
) /( ) (
) /( ) (
) , (
2 2
2 2
2 2 2 2 2
2 2 2 2 2
y x x
x y y
y x y x x
y x x y y
y x f
For
) 0 , 0 ( ) , ( y x
, (1) is satisfied iff
y x
or
y x
. These are also the solutions
to (2). So there are infinite stationary points given by
) , ( x x
or
) , ( x x
, 0 x .
2.2. Classifying Stationary Points
Definition: A local maxima is a stationary point where for a small neighbourhood
about the stationary point the function is no greater than the stationary point. A local
minima is a stationary point where the function attains a minimum in a small
neighbourhood of the point.
E.g.
local maximum local minimum
Theorem 2: Let
f
be a
2
C function. A stationary point ) , , , ( *
* *
2
*
1 n
a a a a is a
local maximum for
f
iff the Hessian matrix at
* a
,
*) (a H
, is negative semi-
definite (or negative definite); a stationary point
* a
is a local minimum for
f
iff
*) (a H
is positive semi-definite (or positive definite).
4
(Note that if
*) (a H
is indefinite, the stationary point
* a
is neither a local
maximum nor a local minimum.)
Proof: Here, we only look at the condition for local maximum, the condition for local
minimum can be obtained similarly. To show that
* a
is a local maximum, one has
to show that values of function in the neighbourhood of
* a
,
) * ( h a f +
where h
is small, are smaller or equal to the value of the function at
* a
,
*) (a f
.
Since
f
is a
2
C function, we can use the Taylors formula to expand
) * ( h a f +
at
point
* a
to obtain
) ( *) (
2
1
*) ( *) ( ) * (
3
h O h a H h a f h a f h a f
T T
+ + + +
Because
* a
is a stationary point,
0 *) ( a f
. The above equation is simplified
to
) 0 as ( *) (
2
1
) ( *) (
2
1
*) ( ) * (
3

+ +
h h a H h
h O h a H h a f h a f
T
T
So
*) ( ) * ( a f h a f +
iff 0 *) ( h a H h
T
, i.e.
*) (a H
is negative semi-definite.
QED
Example 8: Find and classify stationary points of 4 / 4 / ) , (
4 4
y xy x y x f + .
The stationary points require

'

,
`

.
|
+

) 2 ( 0
) 1 ( 0
0 ) , (
3
3
3
3
y x
y x
y x
y x
y x f
From (2), we have
3
y x (3)
Substitution of (3) into (1) yields
0 ) 1 (
5 6
y y y y (4)
Equation (4) has two real solutions 0
*
1
y and 1
*
2
y . Substituting 0
*
1
y into (3),
one obtains 0
*
1
x ; and substituting 1
*
2
y into (3) to obtain 1
*
2
x .
The two stationary points are therefore ) 0 , 0 ( ) , (
*
1
*
1
y x and ) 1 , 1 ( ) , (
*
2
*
2
y x .
The Hessian matrix for 4 / 4 / ) , (
4 4
y xy x y x f + is

,
`

.
|

2
2
3 1
1 3
) , (
y
x
y x H
.
Evaluating the Hessian at ) 0 , 0 ( ) , (
*
1
*
1
y x gives

,
`

.
|

0 1
1 0
) 0 , 0 ( H
which is indefinite. So ) 0 , 0 ( ) , (
*
1
*
1
y x is neither a local maximum nor a local
minimum.
5
Evaluating the Hessian at ) 1 , 1 ( ) , (
*
2
*
2
y x gives

,
`

.
|

3 1
1 3
) 1 , 1 ( H
which is positive definite. So ) 1 , 1 ( ) , (
*
2
*
2
y x is a local minimum.
3. Convexity and Concavity
3.1 Concave and convex functions
Concave and convex functions with single variable normally have the following
shapes:
concave function convex function
Notice from the figure above that the set of points below a concave function is convex
and the set of points above a convex function is convex. This will give us the
definitions.
Definition: A function
) (x f
is concave iff for any
x
and ' x in S , and for
1 0 < < :
) ' ( ) 1 ( ) ( ) ' ) 1 ( ( x f x f x x f + +
.
It is strictly concave iff
) ' ( ) 1 ( ) ( ) ' ) 1 ( ( x f x f x x f + > +
.
A function
) (x f
is convex iff for any
x
and ' x in S , and for 1 0 < < :
) ' ( ) 1 ( ) ( ) ' ) 1 ( ( x f x f x x f + +
.
It is strictly convex iff
) ' ( ) 1 ( ) ( ) ' ) 1 ( ( x f x f x x f + < +
.
Note that for functions to be concave or convex, the above conditions have to be
satisfied for all
x
and ' x in S on which the function is defined.
These definitions above are hard to apply in practical situations. If the second
derivatives of the function exist there is another way of determining whether a
function is concave or convex. Consider the picture below.
6
a
f(.) f(.)
f(a)
f(a)+x f
f(a)+z f
z+a
f(z+a)

The tangent to the function
f
at the point
a
has an equation
) ( ) ( a f x a f y
T
+ . The distance between the tangent to
f
at a point a z + is
indicated by the heavy line. The length of the heavy line is simply
) ( ) ( ) ( a z f a f z a f
T
+ + . Now we will use Taylor's formula for the value
) ( a z f +
z a H z
z a H z a f z a f a f z a f a z f a f z a f
T
T T T T
) (
2
1
] ) (
2
1
) ( ) ( [ ) ( ) ( ) ( ) ( ) (

+ + + + + +
If z is small the distance is given by the expression 2 / ) ( z a H z
T
. If
0 ) ( z a H z
T
the tangent always lies above the function. If 0 ) ( z a H z
T
then the
tangent always lies below the function. Notice that concave functions have the
property that the tangent always lies above the function for any starting point
a
,
whilst convex functions have the property that the tangent always lies below the
function for any starting point
a
. So we have the following theorem.
Theorem 3: A twice continuously differentiable function
f
is
(1) concave if the Hessian
) (x H
is negative semi-definite at all points
x
,
(2) strictly concave if the Hessian
) (x H
is negative definite at all points
x
,
(3) convex if the Hessian
) (x H
is positive semi-definite at all points
x
,
(4) strictly convex if the Hessian
) (x H
is positive definite at all points
x
.
In economics, we sometimes use concave function to represent utility, and we focus
explicitly on its indifference curves. In what follows, we provide a theorem outlining
properties of level curves associated with concave and convex functions.
Theorem 4: Let d be any real number that a function
f
can attain.
(1) If
f
is concave, then its upper level set
) (d U
is a convex set.
(2) If
f
is convex, then its lower level set
) (d L
is a convex set.
Proof: Here, we only prove the first part of the theorem, the second part can be done
similarly. Notice that the upper level set is defined as
{ d x f x d U ) ( : ) (
, i.e., all
x
at which the value of function is at least d . Lets choose two arbitrary points,
y

and
' y
, in
) (d U
, i.e.
d y f ) (
and
d y f ) ' (
. We have to show that the convex
combination
' ) 1 ( y y z +
(for 1 0 < < ) is also in
) (d U
, i.e.
d z f ) (
.
Using the first definition of concave function, we have
d d d y f y f y y f z f + + + ) 1 ( ) ' ( ) 1 ( ) ( ) ' ) 1 ( ( ) (
7
or
d z f ) (
and z is in
) (d U
. So
) (d U
is a convex set.
QED
3.2 Quasi-concave and quasi-convex functions
A standard consumer problem --- maximising utility subject to budget constraints
---would have a solution if the utility function has the property that its upper level set
is convex. Such utility function may arise from a concave function as Theorem 4
suggests. It may also arise from a function which is not concave. Pictures below
provide one example:
The 3-D surface depicted on the left panel shows a concave function. The upper level
set it generates is convex. The function shown on the right panel generates the same
upper level set, but its 3-D shape is very different from that on the left. It is obvious
that the function on the right panel is not concave. A function which generates upper
level sets the same as some concave function is called quasi-concave, and a function
which generates lower level sets the same as some convex function is called quasi-
convex. Formal definition is given below.
Definition: Let d be any real number that a function
f
can attain.
(1) If the upper level set
) (d U
is a convex set, then
f
is quasi-concave.
(2) If the lower level set
) (d L
is a convex set, then
f
is quasi-convex.
Theorem 4 and the above definition immediately imply that a concave function must
be a quasi-concave and a convex function must be a quasi-convex. (The reverse of
this statement is usually not true. The pictures above are an example.) From the
definition above it is clear that if a function
f
is quasi-concave then
f
must be
quasi-convex.
The definition introduced above can certainly be used to identify whether a function is
quasi-concave or quasi-convex. But it may sometimes involve knowing the properties
of complex level curves. A more straightforward method is to use the bordered
Hessian.
Definition: Let
) , , , (
2 1 n
x x x x
, the bordered Hessian determinants
) (x B
r
are
given by
8

,
`

.
|

) ( ) ( ) (
) ( ) ( ) (
) ( ) ( 0
det ) (
' ' ' '
1
'
' '
1
' '
11
'
1
' '
1
x f x f x f
x f x f x f
x f x f
x B
rr r r
r
r
r

,
. , , 2 , 1 n r
Theorem 5: Let
f
be a twice continuously differentiable function defined on a
convex S in
n
.
(1) If
f
is quasi-concave, then 0 ) ( ) 1 ( x B
r
r
for all
x
in S and for all
. , , 2 , 1 n r
(2) If 0 ) ( ) 1 ( > x B
r
r
for all
x
in S and for all
n r , , 2 , 1
, then
f
is strictly
quasi-concave.
For quasi-convexity, use Theorem 5 to
f
.
Example 9: Let 0 > x and
0 > y
. Show that the function
xy y x f ) , (
is not
concave but quasi-concave.
The Hessian of this function is

,
`

.
|

0 1
1 0
) , ( y x H
which is indefinite. So the function is neither concave nor convex.
The largest bordered Hessian is given by

,
`

.
|
0 1
1 0
0
x
y
x y
with 0 ) , ( ) 1 (
2
1
1
> y y x B and 0 2 ) , ( ) 1 (
2
2
> xy y x B . So it is strictly quasi-
concave.
9