Constrained Optimization 1

Optimization in Economic Theory
PD Dr. Johannes Paha
University of Hohenheim
winter term 2021/22
johannes.paha@uni-hohenheim.de Optimization in Economic Theory (winter term 2021/22) 1 / 29

Chapter 5 – Constrained Optimization I
Chapter 5
Constrained Optimization I (One Constraint)
Nov 24, 2021
Optimization in Economic Theory

PD Dr. Johannes Paha
winter term 2021/22

Chapter 5 – Constrained Optimization I
Overview – Constrained Optimization
Local and global extrema with unconstrained optimization

The protoype constrained optimization problem
Lagrangian method with two variables and one constraint
Concluding remarks
Background reading: Simon and Blume (1994, Ch. 18)

Chapter 5 – Constrained Optimization I Double click here for audio contents
Learning goals
Students
Understand the general structure of constrained optimization problems
Can apply the Lagrangian method to solve constrained optimization problems with
two variables and one constraint
Can provide an economic interpretation of the Lagrange multiplier

Preliminaries – Critical points
Function of a real variable: A critical point is a point in the domain of the function
where the function is either not differentiable or the derivative is equal to zero.
Function of several real variables: A critical point is a value in its domain where
the gradient is undefined or is equal to zero.

Preliminaries – Local and global minima and maxima
Consider a multivariate function f (x1 , . . . , xn ).

If x∗ = (x1∗ , . . . , xn∗ ) is a (local or global) minimum or maximum, then
df ∗
(x ) = 0 for all i = 1, . . . , n
dxi
A sufficient condition for
A local minimum at x∗ : D 2 f (x∗ ) is positive definite
A local maximum at x∗ : D 2 f (x∗ ) is negative definite
A global minimum at x∗ : D 2 f (x) is positive definite for all x
A global maximum at x∗ : D 2 f (x) is negative definite for all x

The constrained optimization problem – Motivation
Recall: Economics can be defined as the study of optimal allocation of scarce

resources.
“Optimal” implies that an “optimization” problem is involved.
“Scarce” implies that the objects in the optimization problem
are not free to take any value but are constrained.
A household’s consumption is constrained by its available income.
A firm’s production is constrained by the cost and availability of inputs.
Hence, we have to develop tools for constrained optimization.

The constrained optimization problem
The prototype problem is

maximize F (x1 , . . . , xn ) (1)
n
where (x1 , . . . , xn ) ∈ R must satisfy
G1 (x1 , . . . , xn ) ≤ b1 , . . . , Gk (x1 , . . . , xn ) ≤ bk (2)
H1 (x1 , . . . , xn ) = c1 , . . . , Hm (x1 , . . . , xn ) = cm (3)
The function F is called the objective function
The fn’ns G1 , . . . , Gk and H1 , . . . , Hm are called constraint functions
The functions G1 , . . . , Gk are inequality constraints
The functions H1 , . . . , Hm are equality constraints
The most common inequality constraints are nonnegativity constraints
x1 ≥ 0, . . . , xn ≥ 0, or, in line with eq. (2), −xi ≤ 0

Example: Profit maximization of a competitive firm
A firm in a competitive industry uses n inputs to manufacture its product

Let y denote the amount of this output, x1 , . . . , xn the amounts of the
inputs, and y = f (x1 , . . . , xn ) the firm’s production function
Let p denote the price of the output and wi the cost of input i
Let g1 (x) ≤ b1 , . . . , gk (x) ≤ bk represent constraints on input availability
The profit maximization problem is n
X
maximize Π(x1 , . . . , xn ) = p · f (x1 , . . . , xn ) − wi xi
i=1
n
X
subject to p · f (x1 , . . . , xn ) − wi xi ≥ 0,
i=1
g1 (x) ≤ b1 , . . . , gk (x) ≤ bk
x1 ≥ 0, x2 ≥ 0, . . . , xn ≥ 0.

Utility Maximization – The maximization problem
xi represents the amount of commodity i

The objective function F (x1 , . . . , xn ) is the utility function,
typically written as U(x1 , . . . , xn ), which measures level of utility when consuming
xi units of good i.
The (equality) constraint H(x) is the budget constraint:
Let p1 , . . . , pn denote the prices of the commodities
and I the individual income level.
The utility maximization problem is
maximize U(x1 , . . . , xn )
subject to p1 x1 + p2 x2 + · · · + pn xn ≤ I,
x1 ≥ 0, x2 ≥ 0, . . . , xn ≥ 0.

Utility Maximization – The maximization problem
A simplified model
Consider the utility max’n problem with just two goods (variables)
maximize U(x1 , x2 )
subject to H(x1 , x2 ) = p1 x1 + p2 x2 = I
We ignore for a moment that consumption should be nonnegative
and the possibility that not all income I may be spent, i.e., we consider
one equality constraint.
Geometric solution to the problem: Tangency condition
Draw the constraint set (budget line) in the x1 x2 -plane.
Find the highest-valued level curve of the objective function ( =
indifference curve) which meets the constraint.

Utility Maximization – Slope of the (budget) constraint
Along the constraint H(x1 , x2 ), we have (the total differential)

dx2 H1 (x1 , x2 )
dH(x1 , x2 ) = H1 (x1 , x2 )dx1 + H2 (x1 , x2 )dx2 = 0 ⇔ =−
dx1 H2 (x1 , x2 )
where Hi (x) denotes the partial derivative with respect to xi
Example: Linear budget constraint
dx2 p1
dH(x1 , x2 ) = p1 dx1 + p
2 dx2 = dI = 0 ⇔ =−
|{z} |{z} dx1 p2
H1 (x1 ,x2 ) H2 (x1 ,x2 )
where the partial derivatives H1 (x1 , x2 ) = p1 and H2 (x1 , x2 ) = p2
do not depend on (x1 , x2 )

Utility Maximization – Slope of the objective / utility fn
Along the objective function F (x1 , x2 ) = ν, we have (total differential)

dF (x1 , x2 ) = F1 (x1 , x2 )dx1 + F2 (x1 , x2 )dx2 = 0
dx2 F1 (x1 , x2 )
⇔ =−
dx1 F2 (x1 , x2 )
Example: Utility function
dU(x1 , x2 ) = MU1 dx1 + MU2 dx2 = 0

|{z} |{z}
F1 (x1 ,x2 ) F2 (x1 ,x2 )
dx2 MU1
⇔ =− (= MRS)
dx1 MU2

Utility Maximization – Tangency condition
Optimality condition
F1 (x1 , x2 ) H1 (x1 , x2 ) F1 (x1 , x2 ) F2 (x1 , x2 )
= or =
F2 (x1 , x2 ) H2 (x1 , x2 ) H1 (x1 , x2 ) H2 (x1 , x2 )
Example: Utility maximization
MU1 MU2
=
p1 p2
which means that in optimum, the marginal utility of income is equalized
(Gossen’s Second Law).

Utility maximization – Extension
In addition to the definitions of the utility max’n problem, we let

w denote the wage rate,
I ′ the consumer’s non-wage income,
ℓ0 hours of labor, and
ℓ1 hours of leisure
The utility maximization problem with labor/leisure choice is
maximize U(x1 , . . . , xn , ℓ1 )
subject to p1 x1 + p2 x2 + · · · + pn xn ≤ I ′ + w ℓ0 ,
ℓ0 + ℓ1 = 24,
x1 ≥ 0, x2 ≥ 0, . . . , xn ≥ 0, ℓ0 ≥ 0, ℓ1 ≥ 0.

System of equations – Maximization
Suppose that the common value of both sides in the optimality condition from
slide 14 is λ: F1 (x1 , x2 ) F2 (x1 , x2 )
= =λ
H1 (x1 , x2 ) H2 (x1 , x2 )
Rewriting this as two equations, we have
F1 (x1 , x2 ) − λH1 (x1 , x2 ) = 0
F2 (x1 , x2 ) − λH2 (x1 , x2 ) = 0
How to solve a system of 2 equations for 3 unknowns (x1 , x2 , λ)?
Including the constraint equation, we obtain
F1 (x1 , x2 ) − λH1 (x1 , x2 ) = 0
F2 (x1 , x2 ) − λH2 (x1 , x2 ) = 0
H(x1 , x2 ) − c = 0

System of equations – Utility Maximization
Application to utility maximization
MU1 − λp1 = 0
MU2 − λp2 = 0
I − p1 x1 − p2 x2 = 0

The Lagrangian function
There is a convenient way of writing the system of 3 equations:

Form the Lagrangian function
L(x1 , x2 , λ) = F (x1 , x2 ) + λ [c − H(x1 , x2 )]
Find the critical points of L by computing the partial derivatives with respect to
x1 , x2 , and λ and setting each equal to zero (FOCs)
∂L !
= F1 (x1 , x2 ) − λH1 (x1 , x2 ) = 0
∂x1
∂L !
= F2 (x1 , x2 ) − λH2 (x1 , x2 ) = 0
∂x2
∂L !
= c − H(x1 , x2 ) = 0
∂λ
The result is precisely the system we derived above:
We have reduced the constrained optimization problem in 2 variables
to an unconstrained one in 3 variables

The variable λ is called the Lagrange multiplier.

The Lagrange multiplier λ measures the sensitivity of the optimal value of the
objective function to changes on the right hand side of the constraint (see
slide 24).
The Lagrange multipliers, thus, can be interpreted as a measure of the scarce
resources in the problem under consideration.

Lagrange’s theorem with 2 variables and 1 constraint:

Let F and H be C 1 functions of two variables. Suppose that x = (x 1 , x 2 ) is a solution of
the problem maximize F (x1 , x2 )
subject to H(x1 , x2 ) = c.
Suppose further that (x 1 , x 2 ) is not a critical point of H. Then, there is a real number λ
such that (x 1 , x 2 , λ) is a critical point of the Lagrangian function
L(x1 , x2 , λ) ≡ F (x1 , x2 ) − λ[H(x1 , x2 ) − c].

In other words, at (x 1 , x 2 , λ)
∂L ∂L ∂L
= 0, = 0, and = 0.
∂x1 ∂x2 ∂λ

Lagrange’s theorem in other words:

If (x̄1 , x̄2 ) maximizes F (x1 , x2 ) and satisfies the constraint H(x1 , x2 ) = c with no
other constraints and with Hj (x̄1 , x̄2 ) ̸= 0 for at least one j ∈ {1, 2}, then
Lj (x̄1 , x̄2 , λ) = 0 for j = 1, 2 and Lλ (x̄1 , x̄2 , λ) = 0.
Remarks
The method does not work if both H1 (x1 , x2 ) and H2 (x1 , x2 ) were zero
at the maximizer (x̄1 , x̄2 ). We need to check that H1 (x̄1 , x̄2 ) ̸= 0 or
H2 (x̄1 , x̄2 ) ̸= 0 or both (constraint qualification).
If the conditions yield no solutions, then either the problem has no
solution or the constraint qualification fails (see Ch. 6).
Second order conditions are needed to distinguish between local
minima and maxima.

Recall the tangency condition for the optimum:

The level curves of F and H are tangent at (x 1 , x 2 ).
Hence, the gradient vectors
∂F ∂H
∂x1 ∂x1
∇F (x) = ∂F and ∇H(x) = ∂H ,
∂x2 ∂x2
which are perpendicular to the level sets of F and H,
must line up at (x 1 , x 2 ).
They are scalar multiples of each other, so that ∇F (x) = λ∇H(x), which
translates into
F1 (x1 , x2 ) − λH1 (x1 , x2 ) = 0
F2 (x1 , x2 ) − λH2 (x1 , x2 ) = 0

Simon and Blume (1994, Figure 18.2)

Interpretation of the Lagrange multiplier λ
The Lagrange multiplier λ measures the sensitivity of the optimal value of the
objective function to changes on the right hand side of the constraint
Utility maximization example: λ measures the effect of a small increase in income
I (relaxing the budget constraint) on the utility (the value of the objective
function): dU/dI = λ
To see this, let (x̄1 , x̄2 ) be the optimal choice and ν = F (x̄1 , x̄2 ) the highest
attainable value of the objective function, and compute the effect of a change in c
on ν.

Interpretation of the Lagrange multiplier λ
Formal derivation
Since we have a small change, we consider the first order Taylor series
approximation to the changes in the objective function F
dν = F (x̄ + dx̄) − F (x̄) = F1 (x̄)d x̄1 + F2 (x̄)d x̄2
Using the opti’ty cond’ns Fj (x̄) = λHj (x̄), we can rewrite this eq’n as
dν = λ [H1 (x̄)d x̄1 + H2 (x̄)d x̄2 ]
Using the definition of a Taylor series approximation for the constraint H(x) = c,
we can rewrite this expression as
dν = λ [H(x̄ + dx̄) − H(x̄)] = λdc
dν
Hence, we have dc
=λ

Example: Logarithmic utility
Consider a consumer who chooses between two goods x and y that cost p and q,
respectively. The consumer has income I.
The budget constraint is px + qy = I.
Suppose the utility function is logarithmic:
U(x , y ) = α ln(x ) + β ln(y ).

Step 1: Set up the Lagrangian
L(x , y , λ) = α ln(x ) + β ln(y ) + λ [I − px − qy ] .

Step 2: Calculate the necessary FOCs

∂L α
= − λp = 0, (4)
∂x x
∂L β
= − λq = 0, (5)
∂y y
∂L
= I − px − qy = 0. (6)
∂λ
Step 3: Solve for the unknowns. For example: Isolating x and y from (4) and (5)
and plugging the result into (6) yields
α+β
λ= .
I
Step 4: Plugging this into x and y yields the demand functions
αI βI
x= , y= .
(α + β)p (α + β)q

Consider the case where α = β = 0.5, p = q = 1, and I = 2:

x = y = 1, λ = 0.5, and U(x , y ) = 0
Redo the exercise for I = 2.1:
x = y = 1.05, and U(x , y ) ≈ 0.049
dν
Applying the formula = λ predicts a change in utility of roughly
dc
0.1 · λ = 0.1 · 0.5 = 0.05,
which is close to the true value of ≈ 0.049

Concluding remarks
We have argued that constrained optimization problems are at the core of

economic analysis.
We have developed a tool (Lagrange method) to solve constrained optimization
problems with 2 variables and 1 constraint.
We have transformed the constrained problem
into an unconstrained problem with 3 variables.
We have explored the meaning of the additional variable
(the Lagrange multiplier).
We will see that the method can be easily generalized to the case of
n choice variables and m < n constraints and that it is also suited to deal with
inequality constraints.

Constrained Optimization 1

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Constrained Optimization 1

Uploaded by

Copyright:

Available Formats

Optimization in Economic Theory

PD Dr. Johannes Paha

winter term 2021/22

johannes.paha@uni-hohenheim.de Optimization in Economic Theory (winter term 2021/22) 1 / 29

Nov 24, 2021

Optimization in Economic Theory

johannes.paha@uni-hohenheim.de Optimization in Economic Theory (winter term 2021/22) 2 / 29

Overview – Constrained Optimization

Local and global extrema with unconstrained optimization

johannes.paha@uni-hohenheim.de Optimization in Economic Theory (winter term 2021/22) 3 / 29

johannes.paha@uni-hohenheim.de Optimization in Economic Theory (winter term 2021/22) 4 / 29

Preliminaries – Critical points

johannes.paha@uni-hohenheim.de Optimization in Economic Theory (winter term 2021/22) 5 / 29

Preliminaries – Local and global minima and maxima

Consider a multivariate function f (x1 , . . . , xn ).

johannes.paha@uni-hohenheim.de Optimization in Economic Theory (winter term 2021/22) 6 / 29

The constrained optimization problem – Motivation

Recall: Economics can be defined as the study of optimal allocation of scarce

johannes.paha@uni-hohenheim.de Optimization in Economic Theory (winter term 2021/22) 7 / 29

The constrained optimization problem

The prototype problem is

johannes.paha@uni-hohenheim.de Optimization in Economic Theory (winter term 2021/22) 8 / 29

Example: Profit maximization of a competitive firm

A firm in a competitive industry uses n inputs to manufacture its product

johannes.paha@uni-hohenheim.de Optimization in Economic Theory (winter term 2021/22) 9 / 29

Utility Maximization – The maximization problem

xi represents the amount of commodity i

johannes.paha@uni-hohenheim.de Optimization in Economic Theory (winter term 2021/22) 10 / 29

Utility Maximization – The maximization problem

johannes.paha@uni-hohenheim.de Optimization in Economic Theory (winter term 2021/22) 11 / 29

Utility Maximization – Slope of the (budget) constraint

Along the constraint H(x1 , x2 ), we have (the total differential)

johannes.paha@uni-hohenheim.de Optimization in Economic Theory (winter term 2021/22) 12 / 29

Utility Maximization – Slope of the objective / utility fn

Along the objective function F (x1 , x2 ) = ν, we have (total differential)

dU(x1 , x2 ) = MU1 dx1 + MU2 dx2 = 0

johannes.paha@uni-hohenheim.de Optimization in Economic Theory (winter term 2021/22) 13 / 29

Utility Maximization – Tangency condition

johannes.paha@uni-hohenheim.de Optimization in Economic Theory (winter term 2021/22) 14 / 29

Utility maximization – Extension

In addition to the definitions of the utility max’n problem, we let

johannes.paha@uni-hohenheim.de Optimization in Economic Theory (winter term 2021/22) 15 / 29

System of equations – Maximization

johannes.paha@uni-hohenheim.de Optimization in Economic Theory (winter term 2021/22) 16 / 29

System of equations – Utility Maximization

Application to utility maximization

johannes.paha@uni-hohenheim.de Optimization in Economic Theory (winter term 2021/22) 17 / 29

The Lagrangian function

There is a convenient way of writing the system of 3 equations:

johannes.paha@uni-hohenheim.de Optimization in Economic Theory (winter term 2021/22) 18 / 29

The Lagrangian function

The variable λ is called the Lagrange multiplier.

johannes.paha@uni-hohenheim.de Optimization in Economic Theory (winter term 2021/22) 19 / 29

The Lagrangian function

Lagrange’s theorem with 2 variables and 1 constraint:

L(x1 , x2 , λ) ≡ F (x1 , x2 ) − λ[H(x1 , x2 ) − c].

johannes.paha@uni-hohenheim.de Optimization in Economic Theory (winter term 2021/22) 20 / 29

The Lagrangian function

Lagrange’s theorem in other words:

johannes.paha@uni-hohenheim.de Optimization in Economic Theory (winter term 2021/22) 21 / 29

The Lagrangian function

Recall the tangency condition for the optimum:

johannes.paha@uni-hohenheim.de Optimization in Economic Theory (winter term 2021/22) 22 / 29