CH 5 1

ENGINEERING OPTIMIZATION
Methods and Applications

A. Ravindran, K. M. Ragsdell, G. V. Reklaitis
Book Review
Page 1
Chapter 5: Constrained Optimality

Criteria
Part 1: Ferhat Dikbiyik
Part 2:Yi Zhang
Review Session
July 2, 2010
Page 2
Constraints:
Good guys or bad guys?
Page 3
Constraints:
reduces the region in
which we search for
optimum.
Page 4
Constraints:
makes optimization
process very
complicated
Page 5
( x 2)
x4
Page 6
Outline of Part 1
Equality-Constrained Problems
Lagrange Multipliers
Economic Interpretation of Lagrange Multipliers
Kuhn-Tucker Conditions
Kuhn-Tucker Theorem
Page 7
Outline of Part 1
Kuhn-Tucker Theorem
Page 8
GOAL
solving the problem as an unconstrained
problem by explicitly eliminating K
independent variables using the equality
constraints
Page 9
Example 5.1
Page 10
What if?
Page 11
Outline of Part 1
Kuhn-Tucker Theorem
Page 12
Converting constrained problem to an

unconstrained problem with help of
certain unspecified parameters known
as
Page 13
Lagrange
function
Page 14
Lagrange
multiplier
Page 15
Example 5.2
Page 16
Page 17
Test whether the stationary point

corresponds to a minimum
positive definite
Page 18
Page 19
Example 5.3
Page 20
Page 21
Page 22
max
positive
definite
negative
definite
Page 23
Outline of Part 1
Kuhn-Tucker Theorem
Page 24
The Lagrange multipliers have an

important economic interpretation as
shadow prices of the constraints, and
their optimal values are very useful in
sensitivity analysis.
Page 25
Outline of Part 1
Kuhn-Tucker Theorem
Page 26
Page 27
NLP problem
Page 28
Kuhn-Tucker conditions
(aka Kuhn-Tucker Problem)
Page 29
Example 5.4
Page 30
Example 5.4
Page 31
Example 5.4
Page 32
Outline of Part 1
Kuhn-Tucker Theorem
Page 33
Kuhn-Tucker Theorems
1. Kuhn Tucker Necessity Theorem

2. Kuhn Tucker Sufficient Theorem
Page 34
Kuhn-Tucker Necessity Theorem
Let
f, g, and h be differentiable functions
x* be a feasible solution to the NLP problem.
and
for k=1,.,K are
linearly independent
Page 35

Let
f, g, andConstraint
h be differentiable
functions
x*
be
a
qualification
feasible solution to the NLP problem.
and
for k=1,.,K are
linearly independent at the optimum
If x*
is an optimal
solution
NLP problem,
! Hard
to verify,
sincetoitthe
requires
that
then there
exists a (u*,
v*) suchbe
that
(x*,u*, v*)
the optimum
solution
known
solves the KTP given
by KTC.!
beforehand
Page 36
For certain special NLP problems, the

constraint qualification is satisfied:
1.When all the inequality and equality
constraints are linear
2.When all the inequality constraints are
concave functions and equality constraints
are linear
! When the constraint qualification is
not met at the optimum, there may not
exist a solution to the KTP
Page 37
Example 5.5
x* = (1, 0)
and
for k=1,.,K are
linearly independent at the optimum
Page 38
Example 5.5
x* = (1, 0)
No Kuhn-Tucker
point at the
optimum
Page 39

Given a feasible point
that satisfies the
constraint qualification
optimal
not
optimal
If it does not satisfy the
KTCs
If it does satisfy the

KTCs
Page 40
Example 5.6
Page 41
Kuhn-Tucker Sufficiency Theorem
Let
f(x) be convex
the inequality constraints gj(x) for j=1,,J be
all concave function
the equality constraints hk(x) for k=1,,K be
linear
If there exists a solution (x*,u*,v*) that

satisfies KTCs, then x* is an optimal solution
Page 42
Example 5.4
f(x) be convex
the inequality constraints gj(x) for j=1,
,J be all concave function
the equality constraints hk(x) for k=1,,K
be linear
Page 43
Example 5.4
f(x) be convex
semi-definite
Page 44
Example 5.4
f(x) be convex v
g1(x) linear, hence both convex and
concave
negative definite
Page 45
Example 5.4
f(x) be convex v
the equality constraints hk(x) for k=1,
,K be linear
Page 46
Remarks
For practical problems, the constraint qualification
will generally hold. If the functions are
differentiable, a KuhnTucker point is a possible
candidate for the optimum. Hence, many of the
NLP methods attempt to converge to a Kuhn
Tucker point.
Page 47
Remarks
When the sufficiency conditions of Theorem 5.2
hold, a KuhnTucker point automatically becomes
the global minimum. Unfortunately, the
sufficiency conditions are difficult to verify, and
often practical problems may not possess these
nice properties. Note that the presence of one
nonlinear equality constraint is enough to violate
the assumptions of Theorem 5.2
Page 48
Remarks
The sufficiency conditions of Theorem 5.2 have
been generalized further to nonconvex inequality
constraints, nonconvex objectives, and nonlinear
equality constraints. These use generalizations of
convex functions such as quasi-convex and
pseudoconvex functions
Page 49

CH 5 1

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

CH 5 1

Uploaded by

Copyright:

Available Formats

ENGINEERING OPTIMIZATION

Methods and Applications

Chapter 5: Constrained Optimality

Converting constrained problem to an

Test whether the stationary point

Economic Interpretation of Lagrange Multipliers

The Lagrange multipliers have an

1. Kuhn Tucker Necessity Theorem

Kuhn-Tucker Necessity Theorem

Kuhn-Tucker Necessity Theorem

Kuhn-Tucker Necessity Theorem

For certain special NLP problems, the

Kuhn-Tucker Necessity Theorem

If it does satisfy the

Kuhn-Tucker Sufficiency Theorem

If there exists a solution (x,u,v*) that

You might also like