You are on page 1of 12

# Optimization using Calculus

Optimization of
Functions of Multiple
Variables: Unconstrained
Optimization

Objectives

## To study functions of multiple variables, which are more difficult to

analyze owing to the difficulty in graphical representation and
tedious calculations involved in mathematical analysis for
unconstrained optimization.

To study the above with the aid of the gradient vector and the
Hessian matrix.

## Optimization Methods: M2L3

Unconstrained optimization

## If a convex function is to be minimized, the stationary point is the

global minimum and analysis is relatively straightforward as
discussed earlier.

## The necessary and sufficient conditions for the optimization of

unconstrained function of several variables are discussed.

## Optimization Methods: M2L3

Necessary condition

## In case of multivariable functions a necessary condition for a

stationary point of the function f(X) is that each partial derivative is
equal to zero. In other words, each element of the gradient vector x f
defined below must be equal to zero. i.e. the gradient vector of f(X),
at X=X*, defined as follows, must be equal to zero:
f
*
(

)
x
1

f
*
(

)
x
2
0
x f

*
(

dx
n

## Optimization Methods: M2L3

Sufficient condition

## For a stationary point X* to be an extreme point, the matrix of second

partial derivatives (Hessian matrix) of f(X) evaluated at X* must be:

## negative definite when X* is a relative maximum point.

When all eigen values are negative for all possible values of X, then
X* is a global maximum, and when all eigen values are positive for
all possible values of X, then X* is a global minimum.
If some of the eigen values of the Hessian at X* are positive and
some negative, or if some are zero, the stationary point, X*, is neither
a local maximum nor a local minimum.

## Optimization Methods: M2L3

Example
Analyze the function f ( x) x12 x22 x32 2 x1 x2 2 x1 x3 4 x1 5 x3 2
and classify the stationary points as maxima, minima and points of
inflection
Solution
f
*
(

x
1

2 x1 2 x2 2 x3 4
f
*

x f
( )
2 x2 2 x1

x2

2 x3 2 x1 5
f
*
( )

x
3

0
0

Example contd.

Example contd.

a11
0

a1n
a2 n
A
M M
M

0
0
L
a
nn

a12 L
a22 L

a11
0
det

0
or

a12

a22 L
M
0
L

a1n
a2 n
0
M

ann

Hence 1 a11' ,

2 a22' ,

n ann ' .

Example:
1 1
A

2
4

## The characteristic equation of A is

1
det(A- I) = det
2
(1 )(4 ) 2 0

2 5 6 0
The eigenvalues are therefore
1 2,
2 3

Example contd.

11

Thank you

12