Constraint Optimization

The prototype constraint optimization problem is

max 𝑓(𝑥1 , . . , 𝑥𝑛 )
where (𝑥1 , … , 𝑥𝑛 ) ∈ ℛ 𝑛 must satisfy
𝑔1 (𝑥1 , … , 𝑥𝑛 ) ≤ 𝑏1 , … , 𝑔𝑘 (𝑥1 , … , 𝑥𝑛 ) ≤ 𝑏𝑘
ℎ1 (𝑥1 , … , 𝑥𝑛 ) = 𝑐1 , … , ℎ𝑚 (𝑥1 , … , 𝑥𝑛 ) = 𝑐𝑚
The function 𝑓 is called the objective function, while 𝑔1 , … , 𝑔𝑘 and ℎ1 , … , ℎ𝑚 are
called constraint functions. The 𝑔𝑗 ′𝑠 define inequality constraints and the ℎ𝑖 ′𝑠

define equality constraints. In applications, the most common inequality constraints

are nonnegative constraints: 𝑥1 ≥ 0, … , 𝑥𝑛 ≥ 0.
Equality Constraints
Maximize a function of two variables with one equality constraint.
The optimization problem is
maximize 𝑓(𝑥1 , 𝑥2 )
subject to ℎ(𝑥1 , 𝑥2 ) = 𝑐
where 𝑓(𝑥1 , 𝑥2 ) is the objective function, ℎ(𝑥1 , 𝑥2 ) = 𝑐 is the equality constraint.
Solve this optimization problem geometrically: ℎ(𝑥1 , 𝑥2 ) = 𝑐 is a curve in the
𝑥1 𝑥2 −plane which represents the points 𝑓 constraints to. The level sets of the
function 𝑓 is defined as {(𝑥1 , 𝑥2 ): 𝑓(𝑥1 , 𝑥2 ) = 𝑓0 }. Actually, 𝑓(𝑥1 , 𝑥2 ) = 𝑓0 is a
curve in the 𝑥1 𝑥2 −plane which represents the points on which the function 𝑓 takes
the same value 𝑓0 . At the maximizer, the level curve of 𝑓 is tangent to the constraint
curve of ℎ which means their slopes are the same.
Applying the implicit function theorem, ℎ(𝑥1 , 𝑥2 ) = 𝑐
Then the slope of the constraint curve of ℎ:
𝑑𝑥2 𝜕ℎ ∗ 𝜕ℎ ∗
=− (𝑥 )⁄ (𝑥 )
𝑑𝑥1 𝜕𝑥1 𝜕𝑥2
The slope of the level curve of 𝑓:
𝑑𝑥2 𝜕𝑓 ∗ 𝜕𝑓 ∗
=− (𝑥 )⁄ (𝑥 )
𝑑𝑥1 𝜕𝑥1 𝜕𝑥2

1
Then we could obtain
𝜕ℎ ∗ 𝜕𝑓 ∗
(𝑥 ) (𝑥 )
𝜕𝑥1 𝜕𝑥1
=
𝜕ℎ ∗ 𝜕𝑓 ∗
(𝑥 ) (𝑥 )
𝜕𝑥2 𝜕𝑥2
This equation can also be written as
𝜕𝑓 ∗ 𝜕𝑓 ∗
(𝑥 ) (𝑥 )
𝜕𝑥1 𝜕𝑥2
=
𝜕ℎ ∗ 𝜕ℎ ∗
(𝑥 ) (𝑥 )
𝜕𝑥1 𝜕𝑥2
To avoid working with zero denominators, let 𝜇 denote the common value of the two
ratios. That is
𝜕𝑓 ∗ 𝜕𝑓 ∗
(𝑥 ) (𝑥 )
𝜕𝑥1 𝜕𝑥2
= =𝜇
𝜕ℎ ∗ 𝜕ℎ ∗
(𝑥 ) (𝑥 )
𝜕𝑥1 𝜕𝑥2
We could have two equations
𝜕𝑓 ∗ 𝜕ℎ ∗
(𝑥 ) − 𝜇 (𝑥 ) = 0
𝜕𝑥1 𝜕𝑥1
𝜕𝑓 ∗ 𝜕ℎ ∗
(𝑥 ) − 𝜇 (𝑥 ) = 0
𝜕𝑥2 𝜕𝑥2
In order to solve for three unknows (𝑥1 , 𝑥2 , 𝜇), at least three equations are needed.
The constraint ℎ(𝑥1 , 𝑥2 ) = 𝑐 contributes as the third equation. Then the system is
𝜕𝑓 ∗ 𝜕ℎ ∗
(𝑥 ) − 𝜇 (𝑥 ) = 0
𝜕𝑥1 𝜕𝑥1
𝜕𝑓 ∗ 𝜕ℎ ∗
(𝑥 ) − 𝜇 (𝑥 ) = 0
𝜕𝑥2 𝜕𝑥2
ℎ(𝑥1 , 𝑥2 ) = 𝑐
There is a convenient way to obtain this system. Form the Lagrangian function
𝐿(𝑥1 , 𝑥2 , 𝜇) = 𝑓(𝑥1 , 𝑥2 ) − 𝜇[ℎ(𝑥1 , 𝑥2 ) − 𝑐]
Take partial derivatives with respect to 𝑥1 , 𝑥2 , 𝜇 and then set these derivatives equal
to zero. Actually, we have transformed the constraint optimization problem to the
unconstraint optimization problem at the cost of introducing an additional variable 𝜇
which is called Lagrangian multiplier.
𝜕ℎ 𝜕ℎ
Constraint Qualification: (𝜕𝑥 𝜕𝑥2
) ≠ (0 0) . This is a restriction on the
1

constraint set.

2
Theorem Let 𝑓 and ℎ be 𝐶 1 functions of two variables. Suppose that 𝒙∗ =
(𝑥1∗ , 𝑥2∗ ) is a solution of the problem
maximize 𝑓(𝑥1 , 𝑥2 )
subject to ℎ(𝑥1 , 𝑥2 ) = 𝑐
Suppose further that (𝑥1∗ , 𝑥2∗ ) is not a critical point of ℎ. Then, there is a real number
𝜇 ∗ such that (𝑥1∗ , 𝑥2∗ , 𝜇 ∗ ) is a critical point of the Lagrangian function
𝐿(𝑥1 , 𝑥2 , 𝜇) = 𝑓(𝑥1 , 𝑥2 ) − 𝜇[ℎ(𝑥1 , 𝑥2 ) − 𝑐]
In other words, at (𝑥1∗ , 𝑥2∗ , 𝜇 ∗ )
𝜕𝐿 𝜕𝐿 𝜕𝐿
= 0, = 0, =0
𝜕𝑥1 𝜕𝑥2 𝜕𝜇
Examples:
(1) Consider the maximization problem
𝑚𝑎𝑥𝑖𝑚𝑖𝑧𝑒 𝑓(𝑥1 , 𝑥2 ) = 𝑥1 𝑥2
𝑠𝑢𝑏𝑗𝑒𝑐𝑡 𝑡𝑜 ℎ(𝑥1 , 𝑥2 ) = 𝑥1 + 4𝑥2 = 16
Since the gradient of ℎ is (1,4), ℎ has no critical point and the constraint
qualification is satisfied.
Form the Lagrangian:
𝐿(𝑥1 , 𝑥2 , 𝜇) = 𝑥1 𝑥2 − 𝜇(𝑥1 + 4𝑥2 − 16)
Set its partial derivatives equal to zero:
𝜕𝐿
= 𝑥2 − 𝜇 = 0
𝜕𝑥1
𝜕𝐿
= 𝑥1 − 4𝜇 = 0
𝜕𝑥2
𝜕𝐿
= −(𝑥1 + 4𝑥2 − 16) = 0
𝜕𝜇
Actually, this is a linear system of three equations in three unknowns.
Let’s solve this system. From the first two equations,
𝑥1 = 4𝜇, 𝑥2 = 𝜇
By the third equation,
4𝜇 + 4𝜇 − 16 = 0
Then 𝜇 = 2, and correspondingly 𝑥1 = 8, 𝑥2 = 2.
Now the solution of the system is
𝑥1 = 8, 𝑥2 = 2, 𝜇 = 2

3
(2) Consider the optimization problem
𝑚𝑎𝑥𝑖𝑚𝑖𝑧𝑒 𝑓(𝑥1 , 𝑥2 ) = 𝑥12 𝑥2
𝑠𝑢𝑏𝑗𝑒𝑐𝑡 𝑡𝑜 ℎ(𝑥1 , 𝑥2 ) = 2𝑥12 + 𝑥22 = 3
Check the constraint qualification. Compute the critical points of ℎ(𝑥1 , 𝑥2 ) =
𝜕ℎ 𝜕ℎ
2𝑥12 + 𝑥22 . Set its partial derivatives equal to zero: (𝜕𝑥 𝜕𝑥2
) = (4𝑥1 2𝑥2 ) =
1

(0 0). There is only one critical point (0 0) which is not in the constraint set.
Therefore, the constraint qualification is satisfied.
Form the Lagrangian
𝐿(𝑥1 , 𝑥2 , 𝜇 ) = 𝑥12 𝑥2 − 𝜇 (2𝑥12 + 𝑥22 − 3)
Set its partial derivatives equal to zero:
𝜕𝐿
= 2𝑥1 𝑥2 − 4𝜇𝑥1 = 2𝑥1 (𝑥2 − 2𝜇) = 0
𝜕𝑥1
𝜕𝐿
= 𝑥12 − 2𝜇𝑥2 = 0
𝜕𝑥2
𝜕𝐿
= −(2𝑥12 + 𝑥22 − 3) = 0
𝜕𝜇
The first equation yields 𝑥1 = 0 or 𝑥2 = 2𝜇. We work with each of these two

cases separately. If 𝑥1 = 0, then 𝑥2 = ±√3 from the third equation and 𝜇 = 0

from the second equation. Therefore,

(0, √3, 0) 𝑜𝑟 (0, −√3, 0)

are two solutions of the system.

If 𝑥1 ≠ 0, then 𝑥2 = 2𝜇 or 𝜇 = 𝑥2 ⁄2 from the first equation. Plug 𝜇 = 𝑥2 ⁄2
into the second equation to find 𝑥12 = 𝑥22 . Plug this expression into the third
equation to obtain 3𝑥12 = 3 or 𝑥1 = ±1 . Since 𝑥12 = 𝑥22 , 𝑥1 = ±1 implies
𝑥2 = ±1, and via the first equation, that 𝜇 = 0.5 when 𝑥2 = 1, and 𝜇 = −0.5
when 𝑥2 = −1. Consequently, we obtain four more solutions of the system
(1,1,0.5), (−1, −1, −0.5), (1, −1, −0.5), (−1,1,0.5),
Since the constraint set is a compact set, there exist a constraint maximum for this
problem. And we know that the constraint maximizer is one of these six
candidates. Plug each of these six candidates into 𝑓(𝑥1 , 𝑥2 ) to see which gives
the highest value.

4
 Since
𝑓(1,1) = 𝑓(−1,1) = 1
𝑓(1, −1) = 𝑓(−1, −1) = −1

𝑓(0, √3) = 𝑓(0, −√3) = 0

The maximum occurs at (1,1) and (−1,1).

Several Equality Constraints
Consider the maximization problem with 𝑛 variables and 𝑚 equality constraints.
maximize 𝑓(𝑥1 , … , 𝑥𝑛 )
subject to ℎ1 (𝑥1 , … , 𝑥𝑛 ) = 𝑎1
⋮
ℎ𝑚 (𝑥1 , … , 𝑥𝑛 ) = 𝑎𝑚
Constraint qualification
If we have only one constraint, ℎ(𝑥1 , … , 𝑥𝑛 ) = 𝑎, the constraint qualification requires

𝜕ℎ ∗ 𝜕ℎ ∗
( (𝒙 ), … , (𝒙 )) ≠ (0, … ,0)
𝜕𝑥1 𝜕𝑥𝑛

at the optimal 𝒙∗ .
If we are dealing with 𝑚 constraints, 𝑚 > 1, the constraint qualification requires the
rank of the Jacobian matrix
𝜕ℎ1 ∗ 𝜕ℎ1 ∗
(𝒙 ) ⋯ (𝒙 )
𝜕𝑥1 𝜕𝑥𝑛
𝐷ℎ(𝒙∗ ) = ⋮ ⋱ ⋮
𝜕ℎ𝑚 ∗ 𝜕ℎ𝑚 ∗
(𝒙 ) ⋯ (𝒙 )
( 𝜕𝑥1 𝜕𝑥𝑛 )

is 𝑚-as large as it can be, that is rank (𝐷ℎ(𝒙∗ )) = 𝑚. This is also the so called

nondegenerate constraint qualification (NDCQ) at 𝒙∗ .

Theorem Let 𝑓, ℎ1 , … , ℎ𝑚 be 𝐶 1 functions of 𝑛 variables. Consider the problem of
maximizing 𝑓(𝒙) on the constraint set
𝐶ℎ = {𝑥 = (𝑥1 , … , 𝑥𝑛 ): ℎ1 (𝑥1 , … , 𝑥𝑛 ) = 𝑎1 , … , ℎ𝑚 (𝑥1 , … , 𝑥𝑛 ) = 𝑎𝑚 }
Suppose that 𝒙∗ ∈ 𝐶ℎ and that 𝒙∗ is a local maximizer or minimizer of 𝑓 on 𝐶ℎ .
Suppose further that 𝒙∗ satisfies condition NDCQ. Then there exist 𝜇1∗ , … , 𝜇𝑚
∗
such
that (𝑥1∗ , … , 𝑥𝑛∗ , 𝜇1∗ , … , 𝜇𝑚
∗ )
is a critical point of the Lagrangian
𝐿(𝒙, 𝝁) = 𝑓(𝒙) − 𝜇1 [ℎ1 (𝑥1 , … , 𝑥𝑛 ) − 𝑎1 ] − 𝜇𝑚 [ℎ𝑚 (𝑥1 , … , 𝑥𝑛 ) − 𝑎𝑚 ]

5
In other words,
𝜕𝐿 ∗ 𝜕𝐿 ∗
(𝒙 ) = 0, ⋯ , (𝒙 ) = 0
𝜕𝑥1 𝜕𝑥𝑛
𝜕𝐿 ∗ 𝜕𝐿 ∗
(𝒙 ) = 0, ⋯ , (𝒙 ) = 0
𝜕𝜇1 𝜕𝜇𝑚
Example:
Consider the problem of maximizing 𝑓(𝑥, 𝑦, 𝑧) = 𝑥𝑦𝑧 on the constraint set defined
by
ℎ1 (𝑥, 𝑦, 𝑥) = 𝑥 2 + 𝑦 2 = 1, ℎ2 (𝑥, 𝑦, 𝑧) = 𝑥 + 𝑧 = 1
First, compute the Jacobian matrix of the constraint functions
2𝑥 2𝑦 0
𝐷ℎ(𝑥, 𝑦, 𝑧) = ( )
1 0 1
Its rank is less than 2 if and only if 𝑥 = 𝑦 = 0. Since any point with 𝑥 = 𝑦 = 0
would violate the first constraint, all points in the constraint set satisfy NDCQ.
Next, form the Lagrangian
𝐿(𝑥, 𝑦, 𝑧, 𝜇1 , 𝜇2 ) = 𝑥𝑦𝑧 − 𝜇1 (𝑥 2 + 𝑦 2 − 1) − 𝜇2 (𝑥 + 𝑧 − 1)
and set the its first partial derivatives equal to zero:
𝜕𝐿
= 𝑦𝑧 − 2𝜇1 𝑥 − 𝜇2 = 0
𝜕𝑥
𝜕𝐿
= 𝑥𝑧 − 2𝜇1 𝑦 = 0
𝜕𝑦
𝜕𝐿
= 𝑥𝑦 − 𝜇2 = 0
𝜕𝑧
𝜕𝐿
= −(𝑥 2 + 𝑦 2 − 1) = 0
𝜕𝜇1
𝜕𝐿
= −(𝑥 + 𝑧 − 1) = 0
𝜕𝜇1
Solve the second and third equations for 𝜇1 and 𝜇2 in terms of 𝑥, 𝑦, 𝑧 and plug
these into the first equation to obtain
𝑥𝑧
𝑦𝑧 − 2 ( ) 𝑥 − 𝑥𝑦 = 0
2𝑦
or
𝑦 2 𝑧 − 𝑥 2 𝑧 − 𝑥𝑦 2 = 0
Then solve the fourth equation for 𝑦 2 in terms of 𝑥 2 and the last equation for 𝑧 in
terms of 𝑥 and plug these into the above equation to obtain
(1 − 𝑥 2 )(1 − 𝑥) − 𝑥 2 (1 − 𝑥) − 𝑥(1 − 𝑥 2 ) = 0

6
 1
Then 𝑥 = 1, 𝑥 = 6 (−1 ± √13), approximately -0.7676 and 0.4343. Plugging this

number into the constraints, we obtain the four solution candidates

𝑥 = 0.4343, 𝑦 = ±0.9008, 𝑧 = 0.5657
𝑥 = −0.7676, 𝑦 = ±0.6409, 𝑧 = 1.7676
Evaluating the objective function at these four points, we found that the maximizer is
𝑥 = −0.7676, 𝑦 = −0.6409, 𝑧 = 1.7676

Inequality Constraints
Maximize a function of two variables with one inequality constraint.
The maximization problem:
𝑚𝑎𝑥𝑖𝑚𝑖𝑧𝑒 𝑓(𝑥1 , 𝑥2 )
𝑠𝑢𝑏𝑗𝑒𝑐𝑡 𝑡𝑜 𝑔(𝑥1 , 𝑥2 ) ≤ 𝑏
If the maximizer (𝑥1∗ , 𝑥2∗ ) is on the boundary of the constraint set where 𝑔(𝑥1 , 𝑥2 ) =
𝑏, we say that the constraint is binding. The gradients of 𝑓 and 𝑔 line up
∇𝑓(𝑥1∗ , 𝑥2∗ ) = 𝜆∇𝑔(𝑥1∗ , 𝑥2∗ )
and 𝜆 ≥ 0. Then we can use the Lagrangian
𝐿(𝑥1 , 𝑥2 , 𝜆) = 𝑓(𝑥1 , 𝑥2 ) − 𝜆[𝑔(𝑥1 , 𝑥2 ) − 𝑏]
If the maximizer (𝑥1∗ , 𝑥2∗ ) is in the interior of the constraint set, we say that the
constraint is not binding. (𝑥1∗ , 𝑥2∗ ) is a local unconstraint maximizer of 𝑓.
First order condition is
𝜕𝑓 ∗ ∗ 𝜕𝑓 ∗ ∗
(𝑥1 , 𝑥2 ) = 0 and (𝑥 , 𝑥 ) = 0
𝜕𝑥 𝜕𝑦 1 2
The derivatives of 𝑔 do not enter the criterion at (𝑥1∗ , 𝑥2∗ ). We can still use the
Lagrangian
𝐿(𝑥1 , 𝑥2 , 𝜆) = 𝑓(𝑥1 , 𝑥2 ) − 𝜆[𝑔(𝑥1 , 𝑥2 ) − 𝑏]
while set 𝜆 = 0.
In summary, either the constraint is binding, that is 𝑔(𝑥1 , 𝑥2 ) − 𝑏 = 0 in which case
the multiplier 𝜆 ≥ 0, or the constraint is not binding in which case 𝜆 = 0. Such a
condition in which one of the two inequalities must be binding is called a
complementary slackness condition. The way to summarize such conditions is
𝜆[𝑔(𝑥1 , 𝑥2 ) − 𝑏] = 0

7
Theorem Suppose that 𝑓 and 𝑔 are 𝐶 1 functions on ℛ 2 and that (𝑥1∗ , 𝑥2∗ )
maximizes 𝑓 on the constraint set 𝑔(𝑥1 , 𝑥2 ) ≤ 𝑏. If 𝑔(𝑥1∗ , 𝑥2∗ ) = 𝑏, suppose that
𝜕𝑔 ∗ ∗ 𝜕𝑔 ∗ ∗
(𝑥1 , 𝑥2 ) ≠ 0 𝑜𝑟 (𝑥 , 𝑥 ) ≠ 0
𝜕𝑥 𝜕𝑦 1 2
In any case, form the Lagrangian function
𝐿(𝑥1 , 𝑥2 , 𝜆) = 𝑓(𝑥1 , 𝑥2 ) − 𝜆[𝑔(𝑥1 , 𝑥2 ) − 𝑏]
Then, there is a multiplier 𝜆∗ such that:
𝜕𝐿 ∗ ∗ ∗ 𝜕𝐿 ∗ ∗ ∗
(𝑥 , 𝑥 , 𝜆 ) = 0, (𝑥 , 𝑥 , 𝜆 ) = 0
𝜕𝑥 1 2 𝜕𝑦 1 2
𝜆[𝑔(𝑥1∗ , 𝑥2∗ ) − 𝑏] = 0
𝜆∗ ≥ 0
𝑔(𝑥1∗ , 𝑥2∗ ) ≤ 𝑏

Example
Consider the maximization problem
𝑚𝑎𝑥𝑖𝑚𝑖𝑧𝑒 𝑓(𝑥, 𝑦) = 𝑥𝑦
𝑠𝑢𝑏𝑗𝑒𝑐𝑡 𝑡𝑜 𝑔(𝑥, 𝑦) = 𝑥 2 + 𝑦 2 ≤ 1
The only critical point of 𝑔 occurs at the origin-far away from the boundary of the
constraint set 𝑥 2 + 𝑦 2 = 1. So the constraint qualification will be satisfied at any
candidate for a solution. Form the Lagrangian
𝐿(𝑥, 𝑦, 𝜆) = 𝑥𝑦 − 𝜆(𝑥 2 + 𝑦 2 − 1)
And write out the first order conditions
𝜕𝐿 𝜕𝐿
= 𝑦 − 2𝜆𝑥 = 0, = 𝑥 − 2𝜆𝑦 = 0
𝜕𝑥 𝜕𝑦
𝜆(𝑥 2 + 𝑦 2 − 1) = 0, 𝑥 2 + 𝑦 2 ≤ 1, 𝜆≥0
If 𝜆 = 0, then 𝑥 = 𝑦 = 0. This combination satisfies all the first order conditions, so
it is a candidate for a solution.
If 𝜆 ≠ 0 , then 𝑥 2 + 𝑦 2 = 1 , from the first two equations 𝑥 2 = 𝑦 2 , then 𝑥 =

± 1⁄√2 and 𝑦 = ± 1⁄√2. Then we can obtain four candidates:

1 1 1
𝑥= , 𝑦= , 𝜆=
√2 √2 2
1 1 1
𝑥=− , 𝑦=− , 𝜆=
√2 √2 2

8
 1 1 1
𝑥= , 𝑦=− , 𝜆=−
√2 √2 2
1 1 1
𝑥=− , 𝑦= , 𝜆=−
√2 √2 2
We disregard the last two candidates since they involve a negative multiplier. So there
are totally three candidates which satisfy all five first order conditions. Plugging these
three into the objective function, we find that
1 1 1 1
𝑥= ,𝑦 = 𝑎𝑛𝑑 𝑥 = − ,𝑦 = −
√2 √2 √2 √2
are the solutions of our original problem.
Several Inequality Constraints
Consider the maximization problem:
maximize 𝑓(𝑥1 , … , 𝑥𝑛 )
subject to 𝑔1 (𝑥1 , … , 𝑥𝑛 ) ≤ 𝑏1
⋮
𝑔𝑘 (𝑥1 , … , 𝑥𝑛 ) ≤ 𝑏𝑘

Theorem Suppose that 𝑓, 𝑔1 , … , 𝑔𝑘 are 𝐶 1 functions of 𝑛 variables. Suppose that

𝒙∗ ∈ ℛ 𝑛 is a local maximizer of 𝑓 on the constraint set defined by the 𝑘
inequalities
𝑔1 (𝑥1 , … , 𝑥𝑛 ) ≤ 𝑏1 , … , 𝑔𝑘 (𝑥1 , … , 𝑥𝑛 ) ≤ 𝑏𝑘
For ease of notation, assume that the first 𝑘0 constraints are binding at 𝒙∗ and that
the last 𝑘 − 𝑘0 constraints are not binding. Suppose that the following nondegenerate
constraint qualification is satisfied at 𝒙∗ .
The rank at 𝒙∗ of the Jacobian matrix of the binding constraints
𝜕𝑔1 ∗ 𝜕𝑔1 ∗
(𝒙 ) ⋯ (𝒙 )
𝜕𝑥1 𝜕𝑥𝑛
⋮ ⋱ ⋮
𝜕𝑔𝑘0 ∗ 𝜕𝑔𝑘0 ∗
(𝒙 ) ⋯ (𝒙 )
( 𝜕𝑥1 𝜕𝑥𝑛 )
is 𝑘0 − as large as it can be.
Form the Lagrangian
𝐿(𝑥1 , … , 𝑥𝑛 , 𝜆1 , … , 𝜆𝑘 ) = 𝑓(𝒙) − 𝜆1 [𝑔1 (𝒙) − 𝑏1 ] − ⋯ − 𝜆𝑘 [𝑔𝑘 (𝒙) − 𝑏𝑘 ]
Then there exist multipliers 𝜆1∗ , … , 𝜆∗𝑘 such that

9
 𝜕𝐿 𝜕𝐿
(a) (𝒙∗ , 𝝀∗ ) = 0, … , (𝒙∗ , 𝝀∗ ) = 0
𝜕𝑥1 𝜕𝑥𝑛

(b) 𝜆1∗ [𝑔1 (𝒙∗ ) − 𝑏1 ] = 0, … , 𝜆∗𝑘 [𝑔𝑘 (𝒙∗ ) − 𝑏𝑘 ] = 0

(c) 𝜆1∗ ≥ 0, … , 𝜆∗𝑘 ≥ 0
(d) 𝑔1 (𝒙∗ ) ≤ 𝑏1 , … , 𝑔𝑘 (𝒙∗ ) ≤ 𝑏𝑘

Example
Consider the problem of maximizing 𝑓(𝑥, 𝑦, 𝑧) = 𝑥𝑦𝑧 on the constraint set defined
by the inequalities
𝑥 + 𝑦 + 𝑧 ≤ 1, 𝑥 ≥ 0, 𝑦 ≥ 0, 𝑧 ≥ 0
Rewrite the three nonnegative constraints as
−𝑥 ≤ 0, −𝑦 ≤ 0, −𝑧 ≤ 0
The Jacobian of the constraint function is
1 1 1
−1 0 0
( )
0 −1 0
0 0 −1
Its rank is 3. Since at most three of the four constraints can be binding at any one time,
the NDCQ hold at any solution candidate.
Form the Lagrangian
𝐿(𝑥, 𝑦, 𝑧, 𝜆1 , 𝜆2 , 𝜆3 , 𝜆4 ) = 𝑥𝑦𝑧 − 𝜆1 (𝑥 + 𝑦 + 𝑧 − 1) − 𝜆2 (−𝑥) − 𝜆3 (−𝑦) − 𝜆4 (−𝑧)
Rewrite the Lagrangian as
𝐿(𝑥, 𝑦, 𝑧, 𝜆1 , 𝜆2 , 𝜆3 , 𝜆4 ) = 𝑥𝑦𝑧 − 𝜆1 (𝑥 + 𝑦 + 𝑧 − 1) + 𝜆2 𝑥 + 𝜆3 𝑦 + 𝜆4 𝑧
The first order conditions:
𝜕𝐿
= 𝑦𝑧 − 𝜆1 + 𝜆2 = 0
𝜕𝑥
𝜕𝐿
= 𝑥𝑧 − 𝜆1 + 𝜆3 = 0
𝜕𝑦
𝜕𝐿
= 𝑥𝑦 − 𝜆1 + 𝜆4 = 0
𝜕𝑧
𝜆1 (𝑥 + 𝑦 + 𝑧 − 1) = 0, 𝜆2 𝑥 = 0, 𝜆3 𝑦 = 0, 𝜆4 𝑧 = 0
𝜆1 ≥ 0, 𝜆2 ≥ 0, 𝜆3 ≥ 0, 𝜆4 ≥ 0
𝑥 + 𝑦 + 𝑧 ≤ 1, 𝑥 ≥ 0, 𝑦 ≥ 0, 𝑧 ≥ 0

10
From the first three equations,
𝜆1 = 𝑦𝑧 + 𝜆2 = 𝑥𝑧 + 𝜆3 = 𝑥𝑦 + 𝜆4
We will look at two cases: 𝜆1 = 0 and 𝜆1 > 0,
If 𝜆1 = 0, then
𝑦𝑧 = 𝑥𝑧 = 𝑥𝑦 = 0, 𝑎𝑛𝑑 𝜆1 = 𝜆2 = 𝜆3 = 𝜆4 = 0
The set of solution candidates is that two of the variables equals zero and the third is
any number in the interval [0,1]. The objective function equals zero at these
candidates.
Next, look at the case 𝜆1 > 0, then 𝑥 + 𝑦 + 𝑧 = 1, at least one of 𝑥, 𝑦, 𝑧 must be
nonzero. Suppose for a moment that 𝑥 = 0, then 𝜆1 = 𝜆3 = 𝜆4 > 0 but then 𝑦 =
𝑧 = 0 a contradiction to 𝑥 + 𝑦 + 𝑧 = 1. Therefore, 𝑥 > 0. Similar arguments show
that 𝑦 > 0, 𝑧 > 0, then 𝜆2 = 𝜆3 = 𝜆4 = 0, then
𝑦𝑧 = 𝑥𝑧 = 𝑥𝑦
It follows that
1
𝑥=𝑦=𝑧=
3
Evaluating the objective function, we find
1 1 1 1
𝑓( , , ) = >0
3 3 3 27
Thus, this is the solution for the constraint maximization problem.
Mixed Constraints
Theorem Suppose that 𝑓, 𝑔1 , … , 𝑔𝑘 , ℎ1 , … , ℎ𝑚 are 𝐶 1 functions of 𝑛 variables.
Suppose that 𝒙∗ ∈ ℛ 𝑛 is a local maximizer of 𝑓 on the constraint set defined by the
𝑘 inequalities and 𝑚 equalities
𝑔1 (𝑥1 , … , 𝑥𝑛 ) ≤ 𝑏1 , … , 𝑔𝑘 (𝑥1 , … , 𝑥𝑛 ) ≤ 𝑏𝑘
ℎ1 (𝑥1 , … , 𝑥𝑛 ) = 𝑐1 , … , ℎ𝑚 (𝑥1 , … , 𝑥𝑛 ) = 𝑐𝑚
Without loss of generality, we can assume that the first 𝑘0 inequality constraints are
binding at 𝒙∗ and the other 𝑘 − 𝑘0 inequality constraints are not binding. Suppose
that the following nondegererate constraint qualification is satisfied, that is the rank at
𝒙∗ of the Jacobian matrix of the equality constraints and the binding inequality
constraints

11
 𝜕𝑔1 ∗ 𝜕𝑔1 ∗
(𝒙 ) ⋯ (𝒙 )
𝜕𝑥1 𝜕𝑥𝑛
⋮ ⋱ ⋮
𝜕𝑔𝑘0 ∗ 𝜕𝑔𝑘0 ∗
(𝒙 ) ⋯ (𝒙 )
𝜕𝑥1 𝜕𝑥𝑛
𝜕ℎ1 ∗ 𝜕ℎ1 ∗
(𝒙 ) ⋯ (𝒙 )
𝜕𝑥1 𝜕𝑥𝑛
⋮ ⋱ ⋮
𝜕ℎ𝑚 ∗ 𝜕ℎ𝑚 ∗
(𝒙 ) ⋯ (𝒙 )
( 𝜕𝑥1 𝜕𝑥𝑛 )
is 𝑘0 + 𝑚 − as large as it can be.
Form the Lagrangian
𝐿(𝑥1 , … , 𝑥𝑛 , 𝜆1 , … 𝜆𝑘 , 𝜇1 , … , 𝜇𝑚 )
= 𝑓(𝒙) − 𝜆1 [𝑔1 (𝒙) − 𝑏1 ] − ⋯ − 𝜆𝑘 [𝑔𝑘 (𝒙) − 𝑏𝑘 ] − 𝜇1 [ℎ1 (𝒙) − 𝑐1 ]
− ⋯ − 𝜇𝑚 [ℎ𝑚 (𝒙) − 𝑐𝑚 ]
Then there exist multipliers 𝜆1∗ , … 𝜆∗𝑘 , 𝜇1∗ , … , 𝜇𝑚
∗
such that
𝜕𝐿 ∗ ∗ ∗ 𝜕𝐿 ∗ ∗ ∗
(𝒙 , 𝝀 , 𝝁 ) = 0, … , (𝒙 , 𝝀 , 𝝁 ) = 0
𝜕𝑥1 𝜕𝑥𝑛
𝜆1∗ [𝑔1 (𝒙∗ ) − 𝑏1 ] = 0, … , 𝜆∗𝑘 [𝑔𝑘 (𝒙∗ ) − 𝑏𝑘 ] = 0
ℎ1 (𝒙∗ ) = 𝑐1 , … , ℎ𝑚 (𝒙∗ ) = 𝑐𝑚
𝜆1∗ ≥ 0, … , 𝜆∗𝑘 ≥ 0
𝑔1 (𝒙∗ ) ≤ 𝑏1 , … , 𝑔𝑘 (𝒙∗ ) ≤ 𝑏𝑘
Example
Consider the problem of maximizing 𝑥 − 𝑦 2 on the constraint set 𝑥 2 + 𝑦 2 = 4, 𝑥 ≥
0, 𝑦 ≥ 0. Check the NDCQ, the Jacobian matrix
2𝑥 2𝑥
(1 0)
0 1
At most one nonnegative constraint can be binding and the rank of each submatrix
formed by deleting one row is 2. The NDCQ will automatically be satisfied. Form the
Lagrangian
𝐿(𝑥, 𝑦, 𝜆1 , 𝜆2 , 𝜇) = 𝑥 − 𝑦 2 − 𝜇(𝑥 2 + 𝑦 2 − 4) + 𝜆1 𝑥 + 𝜆2 𝑦
The first order conditions:
𝜕𝐿
= 1 − 2𝜇𝑥 + 𝜆1 = 0
𝜕𝑥

12
 𝜕𝐿
= −2𝑦 − 2𝜇𝑦 + 𝜆2 = 0
𝜕𝑦
𝑥2 + 𝑦2 − 4 = 0
𝜆1 𝑥 = 0, 𝜆2 𝑦 = 0
𝜆1 ≥ 0, 𝜆2 ≥ 0
𝑥 ≥ 0, 𝑦 ≥ 0
1 + 𝜆1 = 2𝜇𝑥 ⇒ 𝜇 > 0, 𝑥 > 0 and 2𝑦(1 + 𝜇) = 𝜆2 ⇒ 𝑦 = 𝜆2 = 0 or 𝑦 > 0, 𝜆2 >
1
0. 𝜆2 𝑦 = 0 ⇒ 𝑦 = 𝜆2 = 0. Then 𝑥 = 2, 𝜆1 = 0, 𝜇 = 4.

Kuhn-Tucker Formulation
The most common constraint maximization problem in economics involve only
inequality constraint and a complete set of nonnegative constraints
maximize 𝑓(𝑥1 , … , 𝑥𝑛 )
subject to 𝑔1 (𝑥1 , … , 𝑥𝑛 ) ≤ 𝑏1
⋮
𝑔𝑘 (𝑥1 , … , 𝑥𝑛 ) ≤ 𝑏𝑘
𝑥1 ≥ 0, … , 𝑥𝑛 ≥ 0
Assume the NDCQ holds at the solution 𝒙∗ . Form the usual Lagrangian
𝐿(𝒙, 𝜆1 , … 𝜆𝑘 , 𝜈1 , … , 𝜈𝑛 )
= 𝑓(𝒙) − 𝜆1 [𝑔1 (𝒙) − 𝑏1 ] − ⋯ − 𝜆𝑘 [𝑔𝑘 (𝒙) − 𝑏𝑘 ] + 𝜈1 𝑥1 + ⋯ + 𝜈𝑛 𝑥𝑛
The first order conditions:
𝜕𝐿 𝜕𝑓 𝜕𝑔1 𝜕𝑔𝑘
= − 𝜆1 − ⋯ − 𝜆𝑘 + 𝜈1 = 0
𝜕𝑥1 𝜕𝑥1 𝜕𝑥1 𝜕𝑥1
⋮
𝜕𝐿 𝜕𝑓 𝜕𝑔1 𝜕𝑔𝑘
= − 𝜆1 − ⋯ − 𝜆𝑘 + 𝜈𝑛 = 0
𝜕𝑥𝑛 𝜕𝑥𝑛 𝜕𝑥𝑛 𝜕𝑥𝑛
𝜆1 [𝑔1 (𝒙) − 𝑏1 ] = 0
⋮
𝜆𝑘 [𝑔𝑘 (𝒙) − 𝑏𝑘 ] = 0
𝜈1 𝑥1 = 0
⋮
𝜈𝑛 𝑥𝑛 = 0
𝜆1 , … 𝜆𝑘 , 𝜈1 , … , 𝜈𝑘 ≥ 0

13
 𝑔1 (𝑥1 , … , 𝑥𝑛 ) ≤ 𝑏1
⋮
𝑔𝑘 (𝑥1 , … , 𝑥𝑛 ) ≤ 𝑏𝑘
𝑥1 ≥ 0, … , 𝑥𝑛 ≥ 0
Kuhn and Tucker worked with a Lagrangian 𝐿̃ which did not include the nonnegative
constraints
𝐿̃(𝒙, 𝜆1 , … 𝜆𝑘 ) = 𝑓(𝒙) − 𝜆1 [𝑔1 (𝒙) − 𝑏1 ] − ⋯ − 𝜆𝑘 [𝑔𝑘 (𝒙) − 𝑏𝑘 ]
We will call 𝐿̃ the Kuhn-Tucker Lagrandian. Note that
𝐿(𝒙, 𝜆1 , … 𝜆𝑘 , 𝜈1 , … , 𝜈𝑘 ) = 𝐿̃(𝒙, 𝜆1 , … 𝜆𝑘 ) + 𝜈1 𝑥1 + ⋯ + 𝜈𝑛 𝑥𝑛
The for each 𝑗 = 1, … , 𝑛

𝜕𝐿 𝜕𝐿̃ 𝜕𝐿̃
= + 𝜈𝑗 = 0 or = −𝜈𝑗
𝜕𝑥𝑗 𝜕𝑥𝑗 𝜕𝑥𝑗

Then

𝜕𝐿̃ 𝜕𝐿̃
≤ 0 and 𝑥𝑗 =0
𝜕𝑥𝑗 𝜕𝑥𝑗

On the other hand,

𝜕𝐿̃
= 𝑏𝑗 − 𝑔𝑗 (𝒙) ≥ 0
𝜕𝜆𝑗

Combine these conditions, we find the first order conditions in term of the
Kuhn-Tucker Lagrangian are

𝜕𝐿̃ 𝜕𝐿̃
≤ 0, … , ≤0
𝜕𝑥1 𝜕𝑥𝑛

𝜕𝐿̃ 𝜕𝐿̃
≥ 0, … , ≥0
𝜕𝜆1 𝜕𝜆𝑘

𝜕𝐿̃ 𝜕𝐿̃
𝑥1 = 0, … , 𝑥𝑛 =0
𝜕𝑥1 𝜕𝑥𝑛

𝜕𝐿̃ 𝜕𝐿̃
𝜆1 = 0, … , 𝜆𝑘 =0
𝜕𝜆1 𝜕𝜆𝑘

14