1

OPTIMIZATION PROBLEMS
 WHAT IS 2

OPTIMIZATION?
Optimization problem: Maximizing or minimizing
some function relative to some set,
often representing a range of choices available in a
certain situation. The function
allows comparison of the different choices for
determining which might be “best.”
Common applications: Minimal cost, maximal profit,
minimal error, optimal design,
optimal management, variational principles.
 3

1. Objective function – mathematical function

which is optimized by changing the
values of the design variables.

2. Design Variables – Those variables which

we, as designers, can change.

3. Constraints – Functions of the design

variables which establish limits in
individual variables or
combinations of design
variables.
 4

3 basic ingredients…
– an objective function,
– a set of decision variables,
– a set of equality/inequality constraints.

The problem is
to search for the values of the decision variables that
minimize the objective function while satisfying the
constraints…
 5

Obective Decision Bounds constrains

vector

min  y  f  x   : x L  x  xU , h  x   0, g  x   0
x

– Design Variables: decision and objective vector

– Constraints: equality and inequality
– Bounds: feasible ranges for variables

– Objective Function: maximization can be converted to

minimization due to the duality principle

max f  x   min   f  x  
 6

1. Identify the quantity or function, f, to be

optimized.
2. Identify the design variables: x1, x2, x3, …,xn.

3. Identify the constraints if any exist

a. Equalities
b. Inequalities
4. Adjust the design variables (x’s) until f is
optimized and all of the constraints are satisfied.
 7

1. Objective functions may be unimodal or

multimodal.
a. Unimodal – only one optimum
b. Multimodal – more than one optimum

2. Most search schemes are based on the

assumption of a unimodal surface. The
optimum determined in such cases is called a
local optimum design.
3. The global optimum is the best of all local
optimum designs.
 8

• Existence of global minimum

• If f(x) is continuous on the feasible set S which is
closed and bounded, then f(x) has a global
minimum in S
– A set S is closed if it contains all its boundary pts.

– A set S is bounded if it is contained in the interior of

T
some circle ( x x  c , c : finite number )

compact = closed and bounded

 9

• Derivative-based optimization (gradient based)

– Capable of determining “search directions” according
to an objective function’s derivative information
• steepest descent method;
• Newton’s method; Newton-Raphson method;
• Conjugate gradient, etc.

• Derivative-free optimization
• random search method;
• genetic algorithm;
• simulated annealing; etc.
 10
PROBLEM SOLVING STRATEGY: Optimization
The strategy consists of two Big Stages. The first does not involve Calculus at all; the second is
identical to what you did for max/min problems.
Stage I: Develop the function.
Your first job is to develop a function that represents the quantity you want to optimize. It can
depend on only one variable. The steps:
Draw a picture of the physical situation.
Also note any physical restrictions determined by the physical situation.
Write an equation that relates the quantity you want to optimize in terms of the relevant variables.
If necessary, use other given information to rewrite your equation in terms of a single variable.
Stage II: Maximize or minimize the function.
You now have a standard max/min problem to solve.

Take the derivative of your equation with respect to your single variable. Then find the critical
points.
Determine the maxima and minima as necessary.
Remember to check the endpoints if there are any.
Justify your maxima or minima either by reasoning about the physical situation, or with the first
derivative test, or with the second derivative test.
Finally, check to make sure you have answered the question as asked: Re-read the problem and
verify that you are providing the value(s) requested: an x or y value; or coordinates; or a maximum
area; or a shortest time; whatever was asked.
 11

Quick portrait of an
Optimazation problem
• An optimization problem is a word problem in
which:
• Two qualities are related one of
them(dependent)being a function of the other
(independent)
• The goal is to identify the value of the
independent quatity that will make the dependent
quantity largest or smallest within a certain
acceptable range
SAMPLE PROBLEM OF DIFFERENTATION IN 12
OPTIMIZATION
A trough for holding water is formed by taking a piece
of sheet metal 60 cm wide and folding the 20 cm on
either end up as shown below. Determine the
angle θθ that will maximize the amount of water that
the trough can hold.
 
 13

• Now, in this case we are being asked to maximize the volume

that a trough can hold, but if you think about it the volume of a
trough in this shape is nothing more than the cross ‑sectional
area times the length of the trough. So, for a given length in order
to maximize the volume all you really need to do is maximize the
cross‑sectional area.
• To get a formula for the cross-sectional area let’s redo the sketch
above a little.
14
16