Introduction To Optimization: 01-Oct-13 Dr. Walid Al-Awad 1

Introduction to Optimization
01-Oct-13 Dr. Walid Al-Awad 1

 A structure in mechanics is defined as “any

assemblage of materials which is intended to
sustain loads.”
 Optimization means making things the best.
 Optimization can be defined as the process of
finding the conditions that give the maximum or
minimum of a function.

 Many times in life we are asked to do an optimization
problem - that is, find the largest or smallest value of some
quantity that will fulfill a need. Typical situations are:
 find the route which will minimize the time it takes me to
get to university.
 build a structure using the least amount of material.
 build a structure costing the least amount of money.
 build a yard enclosing the most amount of space.
 Design a beam using the smallest dimension

 they are all trying to maximize or minimize some
quantity
 In every optimization problem, you are always looking
for a quantity to be maximized or minimized, like
- minimize area ,
- smallest volume ,
- least amount of time ,
- shortest distance ,
- cheapest price.

 A discipline, optimization is often called mathematical

programming.
 (mathematical optimization (alternatively, optimization or mathematical
programming) is the selection of a best element (with regard to some
criteria) from some set of available alternatives)

Determining the root
 Recall, when determining the root, we were seeking x
where f(x) = 0
 With optimization, however we are seeking f '(x) = 0

Maximum, Minimum
 The Maximum occurs when f "(x)<0
 The Minimum occurs when f "(x)>0

Optimization
 In some techniques, we determine the
optima by solving the root problem:
f '(x) =0
 If f '(x) is not available analytically, we may
use a finite difference approximation to
estimate the derivative

Example max, min
 Determine the maximum and minimum values of
the function: f ( x)  12 x 5  45 x 4  40 x 3  5
 Solution: Since f `(x)=60(x4-3x3+2x2)=60x2(x-1)(x-2),
f `(x)=0 at x=0,x=1, and x=2.
The second derivative is: f ( x)  60(4 x 3  9 x 2  4 x)
At x=1, f’’(x)=-60 and hence x=1 is a relative maximum.
Therefore, fmax= f (x=1) = 12
At x=2, f’’(x)=240 and hence x=2 is a relative minimum.
Therefore, fmin= f (x=2) = -11

Example max, min
 Solution cont’d:
 At x=0, f’’(x)=0 and hence we must investigate the
next derivative.
f ( x )  60(12 x 2  18 x  4)  240 at x  0
 Since f ( x)  0 at x=0, x=0 is neither a maximum

nor a minimum, and it is an inflection point.

f(x)= x2 -2x -5
f’(x) = 2x – 2
f’(x) = 0 at x=1
f’(x) exists everywhere in [0,5]
So the critical point are x=0, x=1, x=5
f(0) = (0)2 – 2(0) -5 = -5
f(1) (1)2 – 2(1) -5 =-6
f(5) = (5)2 – 2(5) -5 = 10
The minimum value of f(x) occurs at x=1 and the
maximum value at x=5.
max, min

max, min - Search for Two Variables
 Find the values of x and z (both > 0) that maximize
 Solution:
Formulation as an optimization problem:
Design variables: x, z positive
Solving the two
equations for x and z gives:
x = 9.333 and z = 8.667
With these values, U = 83.33

GUIDELINES FOR SOLVING
MAX./MIN. PROBLEMS
 GUIDELINES OR (STEPS )FOR SOLVING MAX./MIN.
PROBLEMS
1. Read each problem slowly and carefully. Read the problem at
least three times before trying to solve it.
2. If appropriate, draw a sketch or diagram of the problem to be
solved. Pictures are a great help in organizing and sorting out
your thoughts.
3. Define variables to be used and carefully label your picture or
diagram with these variables. This step is very important
because it leads directly or indirectly to the creation of
mathematical equations.

MAX./MIN. PROBLEMS
4. Write down all equations which are related to your
problem or diagram. Clearly denote that equation which
you are asked to maximize or minimize.
5. Before differentiating, make sure that the optimization
equation is a function of only one variable. Then
differentiate using the well-known rules of differentiation.
6. Verify that your result is a maximum or minimum
value using the first or second derivative test for extrema.

MAX./MIN. PROBLEMS
 PROBLEM 1 :
Find two nonnegative numbers whose sum is 9 and so that the product of
one number and the square of the other number is a maximum.
 SOLUTION 1:
Let variables x and y represent two nonnegative numbers.
The sum of the two numbers is given to be 9 = x + y ,
so that y = 9 - x . We wish to MAXIMIZE the PRODUCT P = x y2 .
However, before we differentiate the right-hand side, we will write it as a
function of x only.
Substitute for y getting P = x y2
= x ( 9-x)2 .

MAX./MIN. PROBLEMS
 Now differentiate this equation using the product rule and
chain rule, getting
 P = x ( 9-x)2 = x(81 -18x +x2)
 = x3 -18*x2 +81x
 P ́ = 3*x -18*2*x +81 =0
P ́ = 0 for x=9 or x=3 .
Note that since both x and y are nonnegative numbers and
their sum is 9, it follows that 0≤ x≤ 9 . See the adjoining
sign chart for P' . If x=3 and y=6 , P=3*62
P=108
is the largest possible product.

MAX./MIN. PROBLEMS
 PROBLEM 2 :
 Build a rectangular pen with parallel partitions using 500 feet of
fencing. What dimensions will maximize the total area of the pen ?
 SOLUTION 2 :
Let variable x be the width of the pen and
variable y the length of the pen
The total amount of fencing is given to be
500 = 5 (width) + 2 (length) = 5x + 2y ,
2y = 500 - 5x
y = 250 - (5/2)x

MAX./MIN. PROBLEMS
 We wish to MAXIMIZE the total AREA of the pen
A = (width) (length) = x y .
before we differentiate the right-hand side, we will write it as a
function of x only. Substitute for y getting
A =xy
= x ( 250 - (5/2)x)
= 250x - (5/2)x2 .
Now differentiate this equation, getting A' = 250 - (5/2) 2x
= 250 - 5x
= 5 (50 - x ) = 0 x=50 .

MAX./MIN. PROBLEMS
 Note that since there are 5 lengths of x in this construction

and 500 feet of fencing, it follows that 0≤ x ≤100.
See the adjoining sign chart for A' .
x=50 ft. and y=125 ft. ,

A = 6250 ft.2
is the largest possible area of the pen.

MAX./MIN. PROBLEMS
 PROBLEM 3 :
 An open rectangular box with square base is to be made from 48 ft.2 of
material. What dimensions will result in a box with the largest
possible volume ?
SOLUTION 3 :
Let variable x be the length of one edge of the square base and variable y
the height of the box.
The total surface area of the box is given to be
48 = (area of base) + 4 (area of one side)
= x2 + 4 (xy) ,

MAX./MIN. PROBLEMS
 4xy = 48 - x2
We wish to MAXIMIZE the total VOLUME of the box
V = (length) (width) (height) = (x) (x) (y) = x2 y .
before we differentiate the right-hand side, we will write it as a function of x only. Substitute
for y getting V = x2 y
= 12x - (1/4)x3 .
Now differentiate this equation, getting
V' = 12 - (1/4)3x2
= 12 - (3/4)x2
= (3/4)(16 - x2 )
= (3/4)(4 - x)(4 + x)
=0

MAX./MIN. PROBLEMS
 x=4 or x=-4 .
But x ≠ -4 since variable x measures a distance and x > 0 .
Since the base of the box is square and there are 48 ft.2 of
material, it follows that . See the adjoining sign
chart for V' .
x=4 ft. and y=2 ft. ,
V = 32 ft.3
is the largest possible volume of the box.

Example max, min
 Problem 4: A rectangle has a perimeter of 71 feet. What
length and width should it have so that its area is a
maximum? What is this maximum area?
Area = x * y 2x + 2y =71 y= 35.5 – x
f(x)= Area = x * (35.5 –x)
= 35.5 x –x2 f `(x)=0
35.5 -2x =0 x= 17.25 ft y=17.25ft
Maximum area = 315.063 ft2

max, min
 Example

max, min

max, min

max, min

ENGINEERING APPLICATIONS OF OPTIMIZATION
 ENGINEERING APPLICATIONS OF OPTIMIZATION

 Optimization can be applied to solve any engineering
problem.
 Some typical applications:
 Design of aircraft and aerospace structures for minimum weight.
 Design of civil engineering structures such as frames, foundations,
bridges, towers, and dams for minimum cost.
 Minimum-weight design of structures for earthquake, wind, and other
types of random loading.
 Design of water resources systems for maximum benefit

ENGINEERING APPLICATIONS OF OPTIMIZATION
 Design of pumps, turbines, and heat transfer equipment for maximum

efficiency
 Optimum design of electrical machinery such as motors, generators,
and transformers
 Optimal production planning, controlling, and scheduling
 Design of optimum pipeline networks for process industries
 Optimum design of control systems
 Optimum design of chemical processing
 Analysis of statistical data and building empirical models from
experimental results to obtain the most accurate representation of the
physical phenomenon

Optimization
 STATEMENT OF AN OPTIMIZATION PROBLEM

 An optimization problem can be stated as follows.
A real function of n variables
 (1)
 Without these, we have unconstrained optimization

 With them, we have constrained optimization

Optimization
 where X is an n-dimensional vector called the design

vector, f (X) is termed the objective function, and gj (X) and
lj (X) are known as inequality and equality constraints,
respectively.
 The number of variables n and the number of constraints
m and/or p need not be related in any way.
 The problem stated in Eq. ( 1) is called a constrained
optimization problem. Some optimization problems do not
involve any constraints and can be stated as Eq. ( 2)

Optimization
 Such problems are called unconstrained optimization

problems.
(2)
 Design Vector :Any engineering system or component is

defined as variables during the design process.
 Variables: called design or decision variables x i , i = 1,
2, . . . , n.

Optimization constraints, Unconstrained
 Optimization with constraints
min f ( x, y )  x 2  2 y 2
x0
or
min f ( x, y )  x 2  2 y 2
 2  x  5, y  1
or
min f ( x, y )  x 2  2 y 2
x y 2
Unconstrained optimization min f ( x, y )  x  2 y
2 2

Optimization Procedure

Optimization
 Basic Ideas
 One-dimensional Unconstrained Optimization
- Golden-Section Search
- Newton’s Method
 Multidimensional Unconstrained Optimization
- Direct Methods
- Gradient Methods

Golden-Section Search
 A unimodal function has a single maximum or a minimum in

the a given interval. For a unimodal function:
 First pick two points that will bracket your extremism [x l, xu].
 Pick an additional third point within this interval to
determine whether a maximum occurred.
 Then pick a fourth point to determine whether the maximum
has occurred within the first three or last three points
 The key is making this approach efficient by choosing
intermediate points wisely thus minimizing the function
evaluations by replacing the old values with new values.

l0  l1  l2
l1 l2

l0 l1
l1 l l2
 2 R
l1  l2 l1 l1
1
1 R  R2  R 1  0
R
 1  1  4(1) 5 1
R   0.61803
2 2
The first condition specifies that the sum of the two sub lengths l1
and l2 must equal the original interval length.
The second say that the ratio of the length must be equal
 consider the Golden Section Search method which is based
on the Golden Ratio
The Parthenon (Ancient Greek: Παρθενών) is a temple in the Athenian

Acropolis, Greece, dedicated to of the Greek goddess Athena, whom
the people of Athens considered their protector. Its construction began
in 447 BC and was completed in 438 BC.
The Parthenon 5th century BC
The Golden Rectangle
A rectangle is called a golden rectangle
if the ratio of the sides of the rectangle
is equal to 0.61803, like the one shown . 0.61803
1

 Golden Ratio and Fibonacci Numbers
 0,1,1,2,3,5,8,13,21,34…..
 0/1=0
 1/1=1
 1/2=0.5
 2/3=0.667
 3/5==0.6
 5/8=0.625
 8/13=0.615
 Continue and the ratio approaches
the golden ratio!

Start by defining an interval contains single answer, which is
called unimodal interval.
Find the two interior points x1,x2
d=0.618 *( xU –xL)
x1 = xL + d x2 = xU – d
If f(x1)≥ f(x2) xopt =x1, xL =x2

If (x2) > f(x1) xopt=x2, xU =x1

Golden-section Search
 Initial Step of the Golden-section Search
 1) Guess initial bracket xL and xU

 2) Choose two interior points x1 and x2 according to golden ratio,
 3) If f (x1) > f (x2), eliminate [ xL , x2 ] and set x2 = xL for next round

Golden-section Search
 4) Only new x1 need to be determined,

Example: Golden-Section Search
to find maximum
 EX:
 Solution: (1) Create two interior points
 (2) Evaluate function at interior points,

to find maximum
 (3) Because f(x2) > f(x1), eliminate upper part,
 (4) Compute new x2
 (5) Evaluate function at x2

to find maximum

 Example
 Consider Figure below. The cross-sectional area A of a
gutter with equal base and edge length of 2 is given by

A= 4 sinθ(1+cosθ)
Find the angle θ which maximizes the cross-sectional area of
the gutter. Using an initial interval of [0,π/2], find the
solution after 2 iterations. Use an initial ε=0.05.
Cross section of the gutter

 Solution:
The function to be maximized is f( θ)=4sinθ (1+cosθ)
Iteration 1:
Given the values for the boundaries of x l=0 and xu=π /2, we can
calculate the initial intermediate points as follows:
x1=xl + (√5 -1)/2 *(xu – xl ) = (0 + √5 -1)/2 * 1.5708 = 0.9708
x2 = xu+ (√5 -1)/2 *(xu – xl )= (1.578 - √5 -1)/2*1.5708 = 0.60
f(0.6) 4.1227, f(0.9708)=5.165
f(x1) > f(x2) xl=x2
x1=xl + (√5 -1)/2 *(xu – xl )=0.6 + (√5 -1)/2(1.57-0.6)
= 1.2

To check the stopping criteria the difference between xu and
xl is calculated to be xu –xl = 1.57 -0.6 =0.97
which is greater than ε = 0.05. The process is repeated in
the second iteration.
Iteration 2:
The values for the boundary and intermediate points used in

this iteration were calculated in the previous iteration as
shown below.
xl= 0.6 xu=1.57 x1=1.2 x2=0.97


 Example
By using the golden section search method, find
the value of x that maximize
f (x) = -1.5x6 - 2x4 + 12x
Over the interval xl=0,xu=2, perform three iterations only

 Use the golden section search method to find the value of x that
maximize x0=-2, xu=4, εs=1%
f (x) = 4x -1.8x2 + 1.2x3 - 0.3x4

Newton- Raphson method
A similar approach to Newton- Raphson method can
be used to find an optimum of f(x) by defining a new
function g(x)=f’(x). Thus because the same optimal
value x* satisfies both f‘(x*)=g(x*)=0. We can use the
following as a technique to the extremism of f(x).

Newton’s Method
 Newton’s Method

Newton’s Method

Newton’s Method

Newton’s Method
 Use Newton”s method to minimize f(x). x0 = 1
f (x) = 0.5 – x e-x2
f(x)’ = (2x2 -1) * e-x2
f’’(x) = 2 x (3 – 2 x2) * e-x2
xi+1 = xi – f’(x) / f’’(x) = xi – ( 2xi2 -1)/(2xi(3-2xi2 ))
I xi f(xi)
0 1 0.132
1 0.5 0.111
2 0.7 0.071
3 0.707 0.071

Optimization
The End

Introduction To Optimization: 01-Oct-13 Dr. Walid Al-Awad 1

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Introduction To Optimization: 01-Oct-13 Dr. Walid Al-Awad 1

Uploaded by

Copyright:

Available Formats

Introduction to Optimization

01-Oct-13 Dr. Walid Al-Awad 1

 A structure in mechanics is defined as “any

01-Oct-13 Dr. Walid Al-Awad 2

01-Oct-13 Dr. Walid Al-Awad 3

01-Oct-13 Dr. Walid Al-Awad 4

 A discipline, optimization is often called mathematical

01-Oct-13 Dr. Walid Al-Awad 5

 With optimization, however we are seeking f '(x) = 0

01-Oct-13 Dr. Walid Al-Awad 6

 The Minimum occurs when f "(x)>0

01-Oct-13 Dr. Walid Al-Awad 7

01-Oct-13 Dr. Walid Al-Awad 8

01-Oct-13 Dr. Walid Al-Awad 9

 Since f ( x)  0 at x=0, x=0 is neither a maximum

01-Oct-13 Dr. Walid Al-Awad 10

01-Oct-13 Dr. Walid Al-Awad 12

01-Oct-13 Dr. Walid Al-Awad 13

01-Oct-13 Dr. Walid Al-Awad 14

01-Oct-13 Dr. Walid Al-Awad 15

01-Oct-13 Dr. Walid Al-Awad 16

01-Oct-13 Dr. Walid Al-Awad 17

01-Oct-13 Dr. Walid Al-Awad 18

01-Oct-13 Dr. Walid Al-Awad 19

 Note that since there are 5 lengths of x in this construction

x=50 ft. and y=125 ft. ,

is the largest possible area of the pen.

01-Oct-13 Dr. Walid Al-Awad 20

01-Oct-13 Dr. Walid Al-Awad 21

01-Oct-13 Dr. Walid Al-Awad 22

01-Oct-13 Dr. Walid Al-Awad 23

Maximum area = 315.063 ft2

01-Oct-13 Dr. Walid Al-Awad 24

01-Oct-13 Dr. Walid Al-Awad 25

01-Oct-13 Dr. Walid Al-Awad 26

01-Oct-13 Dr. Walid Al-Awad 27

01-Oct-13 Dr. Walid Al-Awad 28

 ENGINEERING APPLICATIONS OF OPTIMIZATION

01-Oct-13 Dr. Walid Al-Awad 29

 Design of pumps, turbines, and heat transfer equipment for maximum

01-Oct-13 Dr. Walid Al-Awad 30

 STATEMENT OF AN OPTIMIZATION PROBLEM

A real function of n variables

 Without these, we have unconstrained optimization

01-Oct-13 Dr. Walid Al-Awad 31

 where X is an n-dimensional vector called the design

01-Oct-13 Dr. Walid Al-Awad 32

 Such problems are called unconstrained optimization

 Design Vector :Any engineering system or component is

01-Oct-13 Dr. Walid Al-Awad 33

01-Oct-13 Dr. Walid Al-Awad 34

01-Oct-13 Dr. Walid Al-Awad 35

01-Oct-13 Dr. Walid Al-Awad 36

 A unimodal function has a single maximum or a minimum in

01-Oct-13 Dr. Walid Al-Awad 37

The Parthenon (Ancient Greek: Παρθενών) is a temple in the Athenian

01-Oct-13 Dr. Walid Al-Awad 39

01-Oct-13 Dr. Walid Al-Awad 40

If f(x1)≥ f(x2) xopt =x1, xL =x2

01-Oct-13 Dr. Walid Al-Awad 41

 Initial Step of the Golden-section Search

 1) Guess initial bracket xL and xU

 3) If f (x1) > f (x2), eliminate [ xL , x2 ] and set x2 = xL for next round