OPTIMISATION IN ENGINEERING DESIGN

FORMULA SHEET
Section 2. Classical Optimisation Techniques
Taylors Theorem with a remainder after n terms:
( ) ( ) ( ) ( )
( )
( ) 1 0 where *
!
......... *
! 2
* * *
2
< < + +

+

+ + h x f
n
h
x f
h
x f h x f h x f
n
n
applied to multivariable functions:
( ) ( ) ( ) ( ) ( )
( ) ( )

+

+

+
+

+ +
n
i
h X X
T
i
i
n
i
n
j j i
j i
n
i i
n
i i
i
Jh h X
x
f
h X f
X
x x
f
h h X
x
f
X
x
f
h X f h X f
1
*
1 1
2
1 1
! 2
1
* *
*
! 2
1
* * * *

where the matrix of the second partial derivatives, J, is the Hessian matrix.
Useful concepts
32 23 33 22 11
1 1
11
32 23 33 22
33 32
23 22
11
13 13 12 12 11 11
33 32 31
23 22 21
13 12 11
) 1 ( a a a a M A
a a a a
a a
a a
M
A a A a A a
a a a
a a a
a a a


+ +
+
If the system of equations X X has solutions other than the trivial solution X = 0,
the values of are named the eigenvalues of A.
Rewriting the equation as (A I) X = 0, the condition for a non-trivial solution to
exist is det(A I) = 0.
Lagrangian Multipliers
( ) ( ) ( ) X g X f X L
j
m
j
j

+
1
,
( )
( )
( )
0
0 0 0
0 0 0
0 0 0

.........
....
....
.........
....
....
.... L
.... L
2 23 22 21
1 13 12 11
2 23 22 21
1 13 12 11
2 23 22 21
1 13 12 11

mn
m
m
mn
m
m
nn
n
n
g
g g g g
g g g g
g
g g g g
g g g g
z L
L L z L
L L L z


Section 3: Univariate numerical methods
Fibonacci method
2 1 1 0
; 1

+
n n n
F F F F F
n 2 3 4 5 6 7 8 9 10 11
0
L
L
n
0.5 0.333 0.2 0.125 0.076 0.047 0.029 0.018 0.011 0.006
0
2 *
2
L
F
F
L
n
n

Golden Section method

( )
0
1
0
1
618 . 0
1
L L L
k
k
k

,
_

which leads to
2
0 *
2

L
L
Quadratic Interpolation
( )
2
c b a h + +
Section 4: Unconstrained multi variable nonlinear optimisation
Pattern search
Pattern search direction,
m i i i
X X S


after m exploratory steps.
Find optimal step length
*
from
( ) S X f
m
+
1
Steepest Descent
i i i i i i i
f X S X X +
+
* *
1

Section 5: General Constrained Nonlinear Optimisation
Penalty Functions

) (
1
) (
x g
r x f P
k
+
Sequential Linear Programming
Approximation of a typical constraint
0 ) ( X g
about a point X
0
,
0
0
0
2
0
2 2
0
1
0
1 1 0
) ( ) ( ) ( ) ( ) (

,
_

+ +

,
_

,
_

+
n
n n
x
g
x x
x
g
x x
x
g
x x X g X g
or

,
_

,
_

+
n
j
n
j
j
j
j
j
x
x
g
x
x
g
X g X g
1 1
0
0 0
0
) ( ) (
where X is a neighbouring point of X
0
.
First order terms:

,
_

,
_

+
n
j
n
j
j
j
j
j
x
x
f
x
x
f
X f X f
1 1
0
0 0
0
) ( ) (

,
_

,
_

,
_

n
j
j
j
i
i
j
i
ij
j
j
x
x
g
X g b
i i
x
g
a
x
f
c
1
0
0
0
0
0
) (
constraint for the is index the note
Second order terms
The gradient is given by
) (
) (
0
0
j j
o
j
x x
X g
x
g

,
_

g(X
0
) is known and
0
j j
x x

can be obtained by solving the quadratic equation

( )
0
2
2
0
0
2
2
2
0 0 0
0
0
0
2
0
0
2
2
) ( 2
Hence
0 ) ( ) (
2
1

,
_

,
_

,
_

,
_

,
_

,
_

j
j j j
j j
j j
j
j j
j
x
g
X g
x
g
x
g
x
g
x x
X g x x
x
g
x x
x
g
integer penalty function
( )

'

,
_

,
_

D i
X X
i i
i i
i i
i i
D
y z
y x
y z
y x
X Q 1 4
log penalty function
( ) ( ) ( ) ( ) ( )
( ) ( ) ( )
( ) x g
x
g
x g l
x
where
x g x g l r x f r X P
n
n k k

1 ,