Optimization Techniques

Introduction to Optimization

1
 Books
Recommended Books:
1. Kalyanmoy Deb, optimization for engineering design,
algorithms and examples, Prentice Hall (2005).
2. M. Asghar Bhatti Practical Optimization Methods With
Mathematics Applications, Springer-Verlag N.Y, Inc., 2000.

nasir m mirza Slide 2

 Optimization
• Optimization is derived from the Latin word
“optimus”, the best and characterizes the
activities involved to find “the best”.
• The purpose of optimization is to achieve
the “best” design relative to a set of prioritized
criteria or constraints. These include
maximizing factors such as productivity,
strength, reliability, longevity, efficiency, and
utilization.

nasir m mirza Slide 3

 Need for Optimization
• Optimization problems arise naturally in many different
disciplines. In engineering design problems often goal is to
maximize or minimize certain parameter or a variable.
• For example optimization algorithms are often used in
aerospace design activity to minimize overall weight.
• For chemical engineers design and operation of a process
plant should have optimal rate of production.
• A mechanical engineer designs a component to minimize the
cost or to maximize the component life.
• An electrical engineer designs a network to achieve
minimum time for communications.

nasir m mirza Slide 4

 Need for Optimization
• A structural engineer designing a multi-story building
must choose materials in the building with a safe
structure and an economical as well.
• A portfolio manager for a large mutual fund company
must choose investments that generate the largest
possible rate of return for its investors
• A plant manager in a manufacturing facility must
schedule the plant operations such that the plant
produces products that maximize company's revenues.
• A scientist in a research laboratory may be interested in
finding a mathematical function that best describes an
observed physical phenomenon.
nasir m mirza Slide 5
 Need for Optimization
• All these situations have the following three things
in common.
• There is an overall goal, or objective, for the activity.
• For the structural engineer, the goal may be to
minimize the cost of the building,
• for the portfolio manager it is to maximize the rate of
return,
• for the plant manager it is to maximize the revenue,
and
• for the scientist, the goal is to minimize the
difference between the prediction from the
mathematical model and the physical observation.

nasir m mirza Slide 6

 Need for Optimization
• In addition to the overall goal, there are
constraints, that must be met.
• The structural engineer must meet safety
requirements dictated by applicable building
standards.
• The portfolio manager must keep the risk of major
losses below levels determined by the company's
management.
• The plant manager must meet customer demands
and work within available work force and raw
material limitations.
• For the laboratory scientist, there are no other
significant requirements.

nasir m mirza Slide 7

 Need for Optimization
• In all situations, choices available to meet the goals and
requirements.
• The choices are known as optimization variables.
• For example, from safety point of view, it does not matter
whether a building is painted purple or pink, and therefore the
color of a building would not represent a good optimization
variable.
• On the other hand, the height of one story could be a possible
design variable because it will determine overall height of the
building, which is an important factor in structural safety.
• The variables that do not affect the goals are clearly not
important.

nasir m mirza Slide 8

 Optimization
• All engineering tasks involve either minimization or
maximization of an objective function.
• It will be very difficult to discuss formulation of each
type of engineering optimization problem in one course.
• However a designer can learn different types of
optimization techniques and latter choose optimal
algorithm for his or her problem.
• First we shall learn optimal problem formulation.

nasir m mirza Slide 9

 Optimization Problem
• Some define the formulation of problem as
• taking statements,
• defining general goals and requirements of a given
activity, and
• converting them into a series of well-defined
mathematical statements.
• Others say the formulation of Opt. problem is:
1. Selecting one or more optimization variables,
2. Choosing an objective function, and
3. Identifying a set of constraints.

nasir m mirza Slide 10

 • The objective function and the constraints must
all be functions of one or more optimization
variables.
• Let us have examples on this.

nasir m mirza Slide 11

 Maximizing Area
EXAMPLE 1: Find the dimensions of the rectangular
garden of greatest area that can be fenced off (all four sides)
with 300 meters of fencing.
SOLUTION
The garden will be in the shape of a rectangle. The perimeter of it
is to be 300 meters. Let’s make a picture of the garden, labeling
the sides. y
x x
y
Since we know the perimeter is 300 meters, we can now construct
an equation based on the variables contained within the picture.
x + x + y + y = 2x + 2y = 300 (Constraint Equation)

nasir m mirza Slide 12

 • A=x y

nasir m mirza Slide 13

 Maximizing Area - CONTINUED

Now, we wish to maximize is area. Therefore, we will need

an equation that contains a variable representing area.

A = xy (Objective Equation)

Now we will rewrite the objective equation in terms of A (the

variable we wish to optimize) and either x or y. Since it
doesn’t make a difference which one we select, we will
select x.
2x + 2y = 300; This is the constraint equation.

2y = 300 – 2x ; y = 150 – x
Now we substitute 150 – x for y in the objective equation so
that the objective equation will have only one independent
variable.
nasir m mirza Slide 14
 Maximizing Area -CONTINUED

The objective equation is given as A = xy

Replace y with 150 – x. A = x(150 – x)
Finally we get A = 150x – x2

Now we will graph the resultant function;

6000

5000

4000
Area (A)

3000

2000

1000

0
0 50 100 150
x

nasir m mirza Slide 15

 Maximizing Area -CONTINUED

Since the graph of the function is obviously a parabola, then the

maximum value of A (along the vertical axis) would be found at the only
value of x for which the first derivative is equal to zero.

The area function is: A = 150x – x2

Differentiating we get: A΄ = 150 – 2x
Let the derivative equal to zero. 150 – 2x = 0; x = 75
Therefore, the slope of the function equals zero when x = 75.
This is the x-value for where the function is maximized. Then,

2x + 2y = 300; 2(75) + 2y = 300; y = 75

So, the dimensions of the garden will be 75 m x 75 m.
nasir m mirza Slide 16
 Minimizing Cost
EXAMPLE 2:
A rectangular garden of area 75 square feet is to be surrounded on three sides
by a brick wall costing $10 per foot and on one side by a fence costing $5 per
foot. Find the dimensions of the garden such that the cost of materials is
minimized.
SOLUTION
Below is a picture of the garden. The red side represents the side that is
fenced.
y
x x
y

The quantity that we will be minimizing is ‘cost’. Therefore, our objective

equation will contain a variable representing cost, C.

nasir m mirza Slide 17

 Minimizing Cost
EXAMPLE 2: -CONTINUED

C = (2x + y)(10) + y(5)

C = 20x + 10y + 5y
C = 20x + 15y (Objective Equation)

Now we will determine the constraint equation. The only piece of information
we have not yet used in some way is that the area is 75 square feet. Using this,
we create a constraint equation as follows.

75 = xy (Constraint Equation)
Now we rewrite the constraint equation, isolating one of the variables
therein.
75 = xy

75/y = x

nasir m mirza Slide 18

 Minimizing Cost EXAMPLE 2: -CONTINUED

Now we rewrite the objective equation using the substitution we just acquired
from the constraint equation.
The objective equation is: C = 20x + 15y

Replace x with 75/y: C = 20(75/y) + 15y;

2000
Simplify, 1800
1600
C = 1500/y + 15y 1400
Cost (C)

1200
1000
Now we use this 800
equation to 600
sketch a graph of 400
the function. 200
0
0 50 100 150
y
nasir m mirza Slide 19
 Minimizing Cost EXAMPLE 2: -CONTINUED

It appears from the graph that there is exactly one relative

extremum, a relative minimum around x = 10 or x = 15.
To know exactly where this relative minimum is, we need
to set the first derivative equal to zero and solve
The object function is: C = 1500/y + 15y

Differentiating we get: C΄ = -1500/y2 + 15

Set the function equal to 0.

-1500/y2 + 15 = 0;
15y2 = 1500
y2 = 100; y = 10.

nasir m mirza Slide 20

 Minimizing Cost EXAMPLE 2: -CONTINUED

Therefore, we know that cost will be minimized when y =

10. Now we will use the constraint equation to determine
the corresponding value for x.

The constraint equation is: 75 = xy

Replace y with 10: 75 = x(10)

Let us solve for x: 7.5 = x

So the dimensions that will minimize cost,

are x = 7.5 ft and y = 10 ft.

nasir m mirza Slide 21

 Minimizing Surface Area EXAMPLE 3:

(Volume) A canvas wind shelter for the beach has a back, two
square sides, and a top. Find the dimensions for which the
volume will be 250 cubic feet and that requires the least possible
amount of canvas.

SOLUTIO Below is a picture of the wind shelter.

N

x
y
x
The quantity that we will be minimizing the ‘surface area’.
Therefore, our objective equation will contain a variable
representing surface area, A.
nasir m mirza Slide 22
 Minimizing Surface Area
EXAMPLE 3: -CONTINUED

Sum of the areas of the sides: A = xx + xx + xy + xy

A = 2x2 + 2xy (Objective Equation)

Now we will determine the constraint equation. The only piece of
information we have not yet used in some way is that the volume
is 250 ft3. Using this, we create a constraint equation as follows.

250 = x2y (Constraint Equation)

Now we rewrite the constraint equation, isolating one of the

variables therein.
250 = x2y; 250/x2 = y
nasir m mirza Slide 23
 Minimizing Surface Area
EXAMPLE 3: -CONTINUED
Now we rewrite the objective equation using the substitution we just acquired
from the constraint equation.
The objective equation is: A = 2x2 + 2xy
Replace y with 250/x2 : A = 2x2 + 2x(250/x2)

Simplify. 4000
3500
A = 2x + 500/x
2 3000
2500
Area (A)

Now we use this 2000

1500
equation to sketch a
1000
graph of the function.
500
0
-5 5 15 25 35 45
x

nasir m mirza Slide 24

 Minimizing Surface Area
EXAMPLE 3: -CONTINUED
It appears from the graph that there is exactly one relative extremum, a
relative minimum around x = 5. To know exactly where this relative
minimum is, we need to set the first derivative equal to zero and solve (since
at this point, the function will have a slope of zero).
A = 2x2 + 500/x
Differentiating we get A΄ = 4x – 500/x2
Then set the function equal to 0. 4x - 500/x2 = 0

4x = 500/x2
4x3 = 500
x3 = 125
x=5

nasir m mirza Slide 25

 Minimizing Surface Area
EXAMPLE 3: -CONTINUED

Therefore, we know that surface area will be minimized when

x = 5. Now we will use the constraint equation to determine
the corresponding value for y.

The constraint equation: 250 = x2y

Replace x with 5.
250 = (5)2y; y = 10

So the dimensions that will minimize surface area,

are x = 5 ft and y = 10 ft.

nasir m mirza Slide 26

 Optimizing the garden area.
• You are building a right-angled triangular flower
garden along a stream as shown in the figure.
The fencing of the left border costs $10 per foot,
while the fencing of the lower border costs $2
per foot. (No fencing is required along the river.)
You want to spend $600 and enclose as much
area as possible. What are the dimensions of
your garden, and what area does it enclose?
[The area of a right-triangle is given by A =
xy/2.]

nasir m mirza Slide 27

 Minimizing the material used
• A box with a square base and open top must
have a volume of 32, 000cm3. Find the
dimensions of the box that minimize the amount
of material used.

nasir m mirza Slide 28

 Maximizing the Area.
• A rancher wants to create two rectangular pens,
as shown in the figure, using an existing fence
line as one side. If there are 510 feet of fence
available, what dimensions should be used to
maximize the total area of the pens

nasir m mirza Slide 29