Mansoura University

Faculty of Engineering

Optimality conditions
Part one

Prepared by
Eng. /Adel Osama Abd Elmaksoud Ahmed
Student in master level 600

Supervision of
Dr. / Mohamed Taher Eleraky
Contents
1. Introduction
2. Definitions of local and global minima
3. Review of some basic calculus
4. Concepts of necessary and sufficient conditions
5. Unconstrained problem
6. Equality constrained problem
7. Necessary conditions for general constrained
problem
8. Physical meaning of Lagrange multipliers
9.Global optimality
10. Engineering design examples

PAGE 1
1.introduction :-
In this chapter, we discuss basic ideas, concepts, and theories used for design optimization (the minimization
problem).. It is assumed that variables are continuous and all problem functions are continuous and at least
twice continuously differentiable. Methods for discrete variable problems that may or may not need
derivatives of the problem functions are presented in later chapters.
Fig. 4.1 shows a broad classification of the optimization approaches for continuous variable constrained and
unconstrained optimization problems. The following two philosophically different viewpoints are shown:
optimality criteria methods and search methods.
Optimality criteria methods: Optimality criteria are the conditions a function must satisfy at its minimum
point. Optimization methods seeking solutions (perhaps using numerical methods) to the optimality conditions
are often called optimality criteria methods. In this chapter and the next one, we describe methods based on
this approach.

Search methods: Search methods are based on a different philosophy of numerically searching the design
space for optimum designs. Here we start with an estimate of the optimum design for the problem. Usually
this starting design will not satisfy the optimality criteria; therefore, it is improved iteratively until they are
satisfied. Thus, in this direct approach we search the design space for optimum points. Methods based on this
approach are described in later chapters.

PAGE 2
2.Definitions of local and global minima:-
In this section, concepts of local and global minima are defined and illustrated using the standard
mathematical model for design optimization . The design optimization problem is always converted to
minimization of a cost function subject to equality and inequality constraints. The problem is restated as
follows:
Find design variable vector x to minimize a cost function f (x) subject to the equality
constraints hj (x) = 0, j = 1 to p and the inequality constraints gi (x) ≤ 0, i = 1 to m.
In Section 2.11, we defined the feasible set S (also called the constraint set, feasible region, or feasible
design space) for a design problem as a collection of feasible designs:
S {x _ hj (x) 0, j 1 to p; gi (x) 0; i 1 to m} (4.1)
Since there are no constraints in unconstrained problems, the entire design space is feasible for them. The
optimization problem is to find a point in the feasible design space that gives a minimum value to the cost
function. Methods to find these points for a function are discussed throughout the text. There are two
types of minimum points for a function: local minimum point and global minimum point. To understand
the difference between global and local minima of a function, consider graphs of a function f (x) of one
variable, as shown in Fig. 4.2. In Fig. 4.2a, where x is between −∞and ∞(−∞≤x ≤∞), points xB and xD
are local minima since the function has its smallest value in their neighborhood. If we move to the left or
right of these points, the function value increases. These are called local minimum points because we
examine only a small neighborhood of the points. Similarly, both x A and xC are points of local maxima for
the function. For a global minimum (maximum) point, we need to examine the entire domain of the
function and determine if it has the minimum (maximum) value over its entire domain. According to
these definitions, there is no global minimum or maximum for the function of Fig. 4.2a since the domain
and the function f (x) are unbounded; that is, x and f (x) are allowed to have any value between −∞ and ∞.
If we restrict x to lie between −a and b, as in Fig. 4.2b, then point xE gives the global minimum and xF the
global maximum for the function. Both of these points have active constraints, while points xA, xB, xC,
and xD are unconstrained.
• A global minimum point (also called the absolute minimum), is the one where there are no
other feasible points with better-cost function values.
• A local minimum point (also called a relative minimum) is the one where there are no other
feasible points “in its vicinity” with better cost function values.
Now we are ready to precisely define global and local minima. In the following paragraphs,
x* is used to designate a particular point of the feasible set.
GLOBAL (ABSOLUTE) MINIMUM
A function f (x) of n variables has a global (absolute) minimum at x* if the value of the function at x* is
less than or equal to the value of the function at any other point x in the feasible set S. That is, f (x*) ≤ f
(x) (4.2)
for all x in the feasible set S. If strict inequality holds for all x other than x* in E q. (4.2), then x* is called
a strong (strict) global minimum; otherwise, it is called a weak global minimum.
LOCAL (RELATIVE) MINIMUM
A function f (x) of n variables has a local (relative) minimum at x* if inequality E q. (4.2) holds for
all x in a small neighborhood N (vicinity) of x* in the feasible set S. If strict inequality holds, then x* is
called a strong (strict) local minimum; otherwise, it is called a weak local minimum.

PAGE 3
The neighborhood N of point x* is defined as a set of points in its vicinity that is, N = {x lx єS with ll x −
x* ll<δ }
Note also that global and local maxima are defined in a similar manner by simply reversing
the inequality in E q. (4.2) that is, f (x*) ≥ f (x). We also note here that these definitions do not provide a
method for locating minimum points. Based on them, however, we can develop analyses and
computational procedures to locate them.

Weierstrass Theorem—Existence of a Global Minimum

If f (x) is continuous on a nonempty feasible set S that is closed and bounded, then f (x) has a
global minimum in S.
Note that when conditions of the Weierstrass theorem are satisfied, the existence of a global
optimum is guaranteed. It is important, however, to realize that when they are not satisfied, a
global solution may still exist; that is, it is not an “if and only if” theorem. The theorem does
not rule out the possibility of a global minimum if its conditions are not met. The difference
is that existence of global minimum is not guaranteed. For example, consider the problem of
minimizing f (x) = x2 subject to the constraints −1 < x < 1. Since the feasible set is not closed,
conditions of the Weierstrass theorem are not met. Therefore, existence of a global minimum
cannot be guaranteed. However, the function has a global minimum at the point x = 0.
Note also that the theorem does not provide a method for finding a global minimum point
even if its conditions are satisfied; it is only an existence theorem.

PAGE 4