Engineering Optimization

Concepts and Applications

Fred van Keulen

Matthijs Langelaar
CLA H21.1
A.vanKeulen@tudelft.nl
 Optimization problem
● Design variables: variables with which the design
problem is parameterized: x   x1 , x2 , , xn 
● Objective: quantity that is to be minimized (maximized)
Usually denoted by: f ( x)
( “cost function”)
● Constraint: condition that has to be satisfied

– Inequality constraint: g ( x)  0
– Equality constraint: h( x)  0
 Optimization problem (cont.)
● General form of optimization problem:

min f (x)
x

subject to : g ( x)  0
h ( x)  0
n
x X  
x  x  x
 Classification
● Problems:

– Constrained vs. unconstrained

– Single level vs. multilevel

– Single objective vs. multi-objective

– Deterministic vs. stochastic

● Responses:

– Linear vs. nonlinear

– Convex vs. nonconvex

– Smooth vs. nonsmooth

● Variables:

– Continuous vs. discrete (integer, ordered, non-ordered)

 Solving optimization problems
● Optimization problems are typically solved using an
iterative algorithm:

Constants Responses
Model f , g, h

Design Derivatives of
variables responses
x (design sensi-
Optimizer tivities)
f g h
, ,
xi xi xi
 Optimization pitfalls!
● Proper problem formulation critical!

● Choosing the right algorithm

for a given problem
● Many algorithms contain lots
of control parameters
● Optimization tends to exploit
weaknesses in models
● Optimization can result in very sensitive designs

● Some problems are simply too hard / large / expensive

 Exercises
● Exercise 1: Introduction to the
valve spring design problem
– Study analysis model

– Formulation of spring optimization model

● Exercise 2: Model behavior / optimization formulation

– Study model properties (monotonicity, convexity,

nonlinearity)
– Optimization problem formulation
 Course overview
● General introduction, problem formulation, design
space / optimization terminology
● Modeling, model simplification

● Optimization of unconstrained / constrained problems

● Single-variable, zeroth-order and gradient-based

optimization algorithms
● Design sensitivity analysis (FEM)

● Topology optimization
 Defining a design model and
optimization problem
1. What can be changed and how can the design be
described?
– Dimensions
– Stacking sequence of laminates
– Ply orientation of laminates
– Thicknesses Bridgestone aircraft tire

For structures: distinguish sizing, material and shape

variables
Defining the optimization problem
2. What is “best”? Define an objective function:
– Weight

– Production cost

– Life-time cost

– Profits

3. What are the restrictions? Define the constraints:

– Stresses

– Buckling load

– Eigenfrequency
Defining the optimization problem (cont.)

4. Optimization: find a suitable algorithm to solve the

optimization problem. Choice depends on problem
characteristics:
– Number of design variables, constraints

– Computational cost of function evaluation

– Sensitivities available?

– Continuous / discrete design variables?

– Smooth responses?

– Numerical noise?

– Many local optima? (nonconvex)

 Summary

Defining an optimization problem:

1. Choose design variables and their bounds

2. Formulate objective (best?)

3. Formulate constraints (restrictions?)

4. Choose suitable optimization algorithm

 Standard forms
● Several standard forms exist:

Negative null form: Positive null form: g (x)  0

h ( x)  0
min f (x)
x

subject to : g ( x)  0 Neg. unity form: g ( x)  1

h( x)  0 h ( x)  1
x  X  n
Pos. unity form: g ( x)  1
x  x  x h ( x)  1
 Structural optimization examples
● Typical objective function: weight

W ( x)
f  Note the scaling!
W (x 0 )

● Typical constraint: maximum stress, maximum

displacement
 max (x) g   max (x)   allowed  0
g 1  0
 allowed
Scaled vs. Unscaled
 Example: minimum weight
tubular column design
● Length l given

● Load P given

● Design variables:
P
– Radius R  [Rmin, Rmax]
R
– Wall thickness t  [tmin, tmax]

● Objective: minimum mass l t

● Constraints: buckling, stress
 Tubular column design
P  2 EI
A  2Rt I  R 3t  Pcrit 
A 4l 2
Design problem:
min lA
 l  2Rt min   l  2Rt
R ,t R ,t

PP P
s.t.  max max s.t. 1  0
2ARt 2Rt   max
 32 ER
EI 3t P  4l 2
P 1  0
44l 2l 2 3
 ER t 3

Rmin  R  Rmax Rmin  R  Rmax

t min  t  t max t min  t  t max
R 10t
t  1  0
10 R
 Tubular column design (2)
● Alternative formulation:

 2
A   Ro  Ri
2
 

4
I  Ro  Ri
4
4

P
 min
Ro , Ri
 2
  l   Ro  Ri
2

P
s.t.   max
Ro
 2
 Ro  Ri
2

l Ri P
16l 2

 3E 4
Ro  Ri
4

Ri  Ro  
Ro min  Ro  Ro max
Ri min  Ri  Ri max
 Multi-objective problems
● Minimize c(x) Vector!
s.t. g(x)  0, h(x) = 0
● Input from designer required! Popular approach:
replace by weighted sum:
f (x)   wi ci (x)
i

● Optimum, clearly, depends on choice of weights

● Pareto optimal point: “no other feasible point exists that

has a smaller ci without having a larger cj”
 Multi-objective problems (cont.)
● Examples of multi-objective problems:

– Design of a structure for

 Minimal weight and

 Minimal stresses
– Design of reduction gear unit for

 Minimal volume
 Maximal fatigue life
– Design of a truck for

 Minimal fuel consumption @ 80 km/h

 Minimal acceleration time for 0 – 40 km/h
 Minimal acceleration time for 40 – 90 km/h
 Pareto set
● Pareto point: “Cannot improve an objective without
worsening another”

c2

Attainable set

Pareto set
Pareto point

c1
 Pareto set (cont.)
● Alternative view:

c1
c2

Pareto set
x
 Pareto set (cont.)
● Pareto set can be disjoint:

Attainable set
c2

Pareto set

c1
 Hierarchical systems
● Large system can be decomposed into subsystems /
components:

● Optimization requires specialized techniques,

multilevel optimization
 Structural hierarchical systems
● Example: wing box

● Too many design

variables to treat at once
● Global level: global loads,
global dimensions
● Local (rib / stiffner)
level: plate thickness,
fiber orientation
 Contents
● Defining an optimization problem

● The design space & problem characteristics

● Model simplification
 The design space
● Design space = set of all possible designs

● Example:
kmax
Feasible domain
F
f  k1  k2
k2
F  Fcr
k1  k max , k 2  k max
k2

k1
Optimum k1 kmax
 Isolines
● Isolines (level sets) connect points with equal function
values:
The design space (cont.)

Problem overconstrained:
no solution exists.

No feasible domain

Dominated constraint
(redundant)
 Design space (cont.)
A A and B inactive
A and B active
B active, A inactive
Objective
function
isolines

Interior B
optimum

Optimum
Optimum
 Active constraint optimization
● Idea of constraint activity at boundary optimum
sometimes used in intuitive design optimization:
– Fully stressed design (sizing / topology optimization)

– Simultaneous failure mode theory

F F

● Careful: does not always give the optimal solution!

 Problem characteristics
● Study of objective and constraint functions:

– simplify problem

– discover incorrect problem formulation

– choose suitable optimization algorithms

● Properties:

– Boundedness

– Linearity

– Convexity

– Monotonicity
 Boundedness
● Proper bounds are necessary to avoid unrealistic
solutions:
– Example: aspirin pill design
Objective: minimize dissolving time
= maximize surface area
(fixed volume)
h
r  max 2r 2  2rh
r ,h

s.t. r 2 h  1
 Boundedness (cont.)
● Volume equality constraint can be substituted, yielding:

1 22
h 2  max 2r 
r r r
500

450

400

f 350

300

250

200

150
100

50

0
0 1 2 3 4 5 6 7 8 9 10

r
 Linearity
“A function f is linear if it satisfies
f(x1+ x2) = f(x1)+ f(x2)
and
f( x1) =  f(x1)
for every two points x1, x2 in the domain, and all ”
 Linearity (2)
● Nonlinear objective functions can have multiple
local optima:
x2 f
f

x2 x1
x x1
● Challenge: finding the global optimum.
 Problem characteristics
● Study of objective and constraint functions:

– simplify problem

– discover incorrect problem formulation

– choose suitable optimization algorithms

● Properties:

– Boundedness

– Linearity

– Convexity

– Monotonicity
 Boundedness
● Surface maximization of aspirin pill not well bounded:

1 22
h 2  max 2r 
r r r
500

450

400

f 350

300

250

200

150
100

50

0
0 1 2 3 4 5 6 7 8 9 10

r
 Linearity
● Nonlinear objective functions can have multiple
local optima:
x2 f
f

x2 x1
x x1
● Challenge: finding the global optimum.
 Convexity
● Convex function: any line connecting any 2 points on
the graph lies above it (or on it):

● Linearity implies convexity (but not strict convexity)

 Convexity (cont.)
● Convex set [Papalambros 4.27]:

“A set S is convex if for every two points x1, x2 in S, the

connecting line also lies completely inside S”
 Convexity (cont.)
● Nonlinear constraint functions can result in nonconvex
feasible domains:
x2

x1
● Nonconvex feasible domains can have multiple local
boundary optima, even with linear objective functions!
 Monotonicity
● Papalambros p. 99:

– Function f is strictly monotonically increasing if:

f(x2) > f(x1) for x2 > x1 f2

– weakly monotonically increasing if: f1

f(x2)  f(x1) for x2 > x1
x1 x2
– Similar for mon. decreasing
df
● Similar: 0
dx
● Note: monotonicity  convexity!

● Linearity implies monotonicity

 Optimization problem
characteristics
● Responses:

– Boundedness

– Linearity

– Convexity

– Monotonicity

● Feasible domain:

– Convexity
Example: tubular column design
P
R

l t R g3
g1
min f  2lRt
R ,t
f
P
s.t. g1  1  0
2Rt max
4l 2 P
g2  3 3 1  0 g2
 ER t
10t
g3  1  0
R
Rmin  R  Rmax
t min  t  t max t
 Optimization problem analysis
● Motivation:

– Simplification

– Identify formulation errors early

– Identify under- / overconstrained problems

– Insight

● Necessary conditions for existence of optimal solution

● Basis: boundedness and constraint activity

 Well-bounded functions –
some definitions
● Lower bound:

l  f ( x) x  
● Greatest lower bound (glb):
f
g l l  f (x)
g

● Minimum: f ( x*)  g x* x

● Minimizer: x*
 Boundedness checking
● Assumption: in engineering optimization problems,
design variables are positive and finite
● Define N  x : 0  x   P  x : 0  x  
● Boundedness check:

– Determine g+ for x N
– Determine minimizers X  x : f ( x )  g 


– Well bounded if  X P
 Examples:
f ( x)  x g  0 X  0 P Bounded
at zero

1 g  0 X    P Asymptotically
f ( x)  bounded
x
f ( x)  ( x  1) 2 
g 0 X  
1 P
2 2
f ( x)  ( x  1) ( x  2)  3 g 
 3 X  1,2 P
2
f ( x)  ( x  1)  2 2
g  2 X   1,1  P

f ( x1 , x2 )  ( x2  1) 2  1 g 1 X  ( N ,1)  P 2
 Air tank design

● Objective: minimize mass

f (x)   (r  t ) 2  r 2 l  2 (r  t ) 2 h
l
 
  2rt  t 2 l  2(r  t ) 2 h 
r
 g  0 X  0  P
h

t
● Not well bounded: constraints needed
 Air tank constraints
● Minimum volume: r 21l.48
g1  1  2.12
107 2 7
10
r l0

● Min. head/radius ratio h h

g 2  1 7.07.13 0
(ASME code): r r
t t
● Min. thickness/radius ratio g 3  1 104
0.009590
r r
(ASME code):
● Room for nozzles: g 4  1l  010
.1l  0
min. length
r t
● Space limitations: gg5 5r  t 150
1 0
max. outside radius
150
 Partial minimization &
bounding constraints
● Partial minimization: keep all variables constant but
one. Example: air tank wall thickness t:
h ,l , r ,t
 
min f   2rt  t 2 l  2(r  t ) 2 h   
min f   2 Rt  t 2 L  2( R  t ) 2 H 
t
s.t. g1  1  1.48  107 r 2l  0 t
s.t. g 3  1  104 0
h R
g 2  1  7.7  0
r Rt
g5  1  0
t 150
g 3  1  104  0
r
Conclusion:
g 4  1  0.1l  0
• f not well bounded from below
r t
g5  1  0 • g3 bounds t from below
150
 Constraint activity
● Removing constraint = relaxing problem

● Solution set of relaxed problem without gi is Xi

A
1. X i  X  g i inactive A and B active

2. X i  X    g i active
3. X i  X  g i semiactive B

● Activity information can

simplify problem: ● Active: eliminate variable
● Inactive: remove constraint
 Constraint activity checking
● Example:
min f  x12  ( x2  1) 2 ( x2  3) 2 ( x2  4) 2  ( x3  5) 2
x1 , x2 , x3

s.t. g1  1  x1  0
g 2  2  x2  0 f(1,x2,5)

g3  x2  5  0
g2
g 4  1  x3  0
g3
Conclusion:
• g1 active
• g2 semiactive
x2
• g3 and g4 inactive
Activity and Monotonicity Theorem
● “Constraint gi is active if and f(x) f
g(x)
only if the minimum of the
relaxed problem is lower than
x
that of the original problem”
g2 g1

● “If f(x) and gi(x) all increase or f(x) f

g(x)
decrease (weakly) w.r.t. x, the
domain is not well constrained”
x

g
 First Monotonicity Principle
● “In a well-constrained minimization problem every
variable that increases f is bounded below by at least
one non-increasing active constraint”

● This principle can be f(x) f

g(x)
used to find active
constraints.
x
● Exactly one bounding
g
constraint: critical constraint
 Air tank design
● Monotonicity analysis:

 
min f   2rt  t 2 l  2(r  t ) 2 h  
f h , l  , r  , t  
g l , r 
h ,l , r ,t
 
7 2 1
Critical w.r.t. r
s.t. g1  1  1.48  10 r l  0
g h , r 
 
Critical w.r.t. h
h 2
g 2  1  7.7
r
0 g r , t 
3
 
Critical w.r.t. t
t g l 
g 3  1  104  0 4
r g r , t 
5
 

g 4  1  0.1l  0
r t
g5  1  0 What about l? Unclear.
150
 Optimizing variables out
● Critical constraints must be active:

min f  
h ,l , r ,t
 2rt  t  l  2(r  t ) h 
2 2

s.t. g1  1  1.48 107 r 2l 

0  r  1 (l )
h
g 2  1  7.7  0  h  2 (r )
r
t
g3  1  104  0  t  3 (r )
r
g 4  1  0.1l  0
r t
g5  1  0
150
 Optimizing variables out
● Critical constraints must be active:

min f  
h ,l , r ,t
 2rt  t  l  2(r  t ) h 
2 2

s.t. g1  1  1.48  107 r 2l  0

h h
g 2  1  7.7  0 g 2  1  7.7 0  h  0.13r
r r
t
g3  1  104  0
r
g 4  1  0.1l  0
r t
g5  1  0
150
 Optimizing variables out
● Critical constraints must be active:

 
min f   2rt  t 2 l  0.26(r  t ) 2 r
l , r ,t

s.t. g1  1  1.48  107 r 2l  0

t t r
g 3  1  104  0 g 3  1  104  0  t
r r 104
g 4  1  0.1l  0
r t
g5  1  0
150
 Optimizing variables out
● Critical constraints must be active:
 209 2  105  3 
2

min f    r l  0.26  r 
l ,r  104 2  104  

s.t. g1  1  1.48  10 7 r 2l  0 g1  1  1.48  10 7 r 2l  0
2600
 r
l
g 4  1  0.1l  0
105
g5  r 1  0
104 *150
 Optimizing variables out
● Critical constraints must be active:
 209 * 26002 2600 3
 105 
2

min f     0.26 *   
l  104 2
l l  104  

s.t.

 4 4.65  109 
g 4  1  0.1l  0 min f   13  10  
l
 l l 
105 * 2600 1 s.t. g 4  1  0.1l  0
g5  1  0
104 *150 l
l
g5  1  0
306
 Problem!
● Length not well bounded:

 4.65  109   4.65  109 

4
min f   13  10    
 4
min f l   13  10  
l
 l l  l
 l l 
s.t. g 4  1  0.1l  0 s.t. g 4  l  10
l g 5  l  306
g5  1  0
306

● Additional constraint from above is needed:

● Maximum plate width: l  610

 Air tank solution
● Length constraint is critical: must be active!

● Solution: l
l  610
r  105 r
t 1
h
h  13.6
t
● Result of Monotonicity Analysis:

● Problem found, and fixed

● Solution found without numerical optimization

 Recognizing monotonicity
● Some useful properties:
  
– Sums: f 3  f1  f 2  f3

Sums of similarly monotonic functions have the same

monotonicity

– Products: f 3  f1 * f 2  f 3 '  f1 ' f 2  f1 f 2 '

Products of similarly monotonic functions have:

– same monotonicity if f1  0, f 2  0
– opposite monotonicity if f1  0, f 2  0
 Recognizing monotonicity
● More properties:
  0 : f 3 
– Powers: f 3  f1   
 
  0 : f 3 
Positive powers of monotonic functions have the same
monotonicity, negative powers have opposite
monotonicity
– Composites: f 3  f1  f 2   f 3 '  f1 ' f 2 '
 f1 , f 2  
 f1 , f 2  
   f3    f3
 f1 , f 2  f1 , f 2
 Recognizing monotonicity
f1
● Integrals:
b
– w.r.t. limits: f 3 (a, b)   f1 ( x)dx
a 0 a b x
f1 (a  x  b)  0  f 3 (a  , b  )

f1 , f1 (a)  0, f1 (b)  0  f 3 (a  , b  )
b
– w.r.t. integrand: f 3 (a, b, y )   f1 ( x, y )dx f1
y
a

f1 ( y  )  f3 ( y  ) a
b x
 Criticality
Refined definitions:
# of variables # of constraints
critically bounded possibly critically
by constraint i Uncritical bounding variable j
constraint
0
Uniquely critical
1 1
constraint
>1 Multiple critical >1
constraint

Conditionally
critical constraint
 Air tank example
 
min f   2rt  t 2 l  2(r  t ) 2 h 
h ,l , r ,t
7 2

f h , l  , r  , t  
s.t. g1  1  1.48  10 r l  0
g l , r 
1
 
Critical w.r.t. r
h
g 2  1  7.7  0 g h , r 
 
Critical w.r.t. h
r 2

t g r , t 
3
 
Critical w.r.t. t
g 3  1  104  0
r g l 
4

Conditionally
g 4  1  0.1l  0 g r , t 
  critical w.r.t. l
5
r t
g5  1  0 Multiple critical!
150

Multiple critical constraint can obscure boundedness!

Eliminate if possible
 Air tank example
● Starting with eliminating r:

 
min f   2rt  t 2 l  2(r  t ) 2 h
h ,l , r ,t

s.t. g1  1  1.48  107 r 2l  0 g1  1  1.45  10 7 r 2l  0
h 2600
g 2  1  7.7  0  r
r l
t
g 3  1  104  0
r
g 4  1  0.1l  0
r t
g5  1  0
150
 Air tank example
● New problem:

?
 
    26002
min f    t 5200 l  tl  2h
t 2
 5200  t   h , l ? , t 
h ,l ,t
  l l 
s.t.
h l
g 2  1  7.7 0 Critical for h h , l 
2600
t l Critical for t l,t
g 3  1  104 0
2600
g 4  1  0.1l  0 l
52 t
g5   1  0 l,t
3 l 150
 Air tank example
● Finally, after also eliminating h and t:

 4.65  109 
min f   130625  3/ 2

l
l
 l 
s.t. g 4  1  0.1l  0 l Not well bounded!
35 l
g5  1  0
2 l

● Conclusion: multiple critical constraint

obscured ill-boundedness in l
 Summary
● Optimization problem checking:

– Boundedness check of objective

 Identify underconstrained problems

– Monotonicity analysis

 Identify not properly bounded problems

 Identify critical constraints

 Eliminate variables

 Remove inactive constraints

 But what about …
● Equality constraints:

– Active if all constraint variables in objective

– Otherwise semi-active

● Example: min f  3 x1
x1 , x2
x2 f
s.t. g1  1  x1  0
h1  x2  3  0 3

Relaxed problem: min f  3 x1

x1 , x2
1 x1
s.t. g1  1  x1  0
 More on nonobjective variables
● Monotonicity Principle for nonobjective variables:
“In a well-constrained minimization problem every
nonobjective variable is bounded below by at least one
non-increasing semiactive constraint and above by at
least one non-decreasing semiactive constraint”
g(x)
gi gj
0

x
 Nonobjective variables (2)
● Other options:

– Equality constraint

– Single nonmonotonic constraint

h(x) g(x)
hi gi
0 0

x x
● See example in book (Papalambros p. 114)
 Nonmonotonic functions
● Monotonicity analysis difficult!

– Sometimes regional monotonicity can be used

– Concave constraints can split feasible domain:

g(x)
gj gi
0

x
Model preparation procedure (3.9)
● Remove dominated constraints

● Check boundedness for each design variable:

– Objective monotonic? Constraints monotonic?

– Critical constraints?
Uniquely / conditionally / multiply?
● If possible, eliminate active constraints,
and repeat steps

Spending time on model checking usually pays off!