Lagrange Multipliers

Joseph-Louis Lagrange is usually considered to be a French mathematician, but the Italian

Encyclopedia refers to him as an Italian mathematician. 1736 - 1813
In Paris a street on the left bank is named after him. Rue Lagrange

Lagrange did lots of things in his life, but for us he showed how calculus could be used to find
optimal solutions of constrained models. But, since it is calculus based, all the limitations of
calculus apply.
It is limited to continuous functions.
It produces a maximum for concave functions, and a minimum for convex functions.

Consider the general optimization problem involving a vector of decision variables X and ‘n’
constraints. The objective is to optimize the function F(X) while satisfying the n constraints
gi(X) = bi. i = 1, 2, 3, …, n

Lagrange developed an approach for finding not only the maximum or minimum value of some
objective function F(X) and the values of all the decision variables X = {x1, x2, x3, .., xj, … xJ}
that result in that maximum or minimum, but also what economists call ‘shadow prices,’ and
operations researchers call ‘dual variables’ or ‘dual prices’. Associated with each constraint i is
a shadow price whose value is the change in the objective function F(X) with respect to a unit
change in the bi values for that constraint. These shadow prices are also called the Lagrangian
multipliers i.
The original objective F(X) is modified to become what we will call L(X, ):
Setting each constraint to equal 0 so that (g(X) = bi) becomes (g(X) – bi) for each constraint i, we
can add them, or subtract them, from F(X). We can also multiply each of these 0 values by a
constant i.

The result is:

L = F(X) - i i(gi(X) – bi)

Equating to 0 each of the partial derivatives of this function with respect to each of the unknown
variables, xj and i and solving the resulting set of equations will result in the ‘optimal’ values
of the variables X that maximize or minimize F(X) if they exist.

L = F(X) - ii(gi(X) – bi)

L/xj = 0 = F/xj - ii(gi(X))/xj for all j.
L/i = 0 = (gi(X) – bi) for all i.
If any constraint is an inequality such that
g(X) ≤ or ≥ bi
Then a slack or surplus variable, call it S, must be added or subtracted from the left hand side of
the inequality. To insure that it is a non-negative variable, it is convenient to square it.
(g(X) ≤ bi) becomes (gi(X) + Si2 –bi)
Now this constraint equals 0, and the values of a set of new variables Si must be solved for.
L = F(X) - ii(gi(X) + Si2 – bi)
L/xj = 0 = F/xj - ii(gi(X))/xj
L/i = 0 = (gi(X) + Si2 –bi)
L/Si = 0 = -2 Sii
Note from the last set of equations either or both Si or i will equal 0. If the constraint is binding
there will be no inequality, and the value of the slack or surplus variable Si will be 0. If the
constraint is not binding (doesn’t affect the values of the variables) then there is no change in
F(X) given a change in bi.
Derivation of i:
L/xj = 0 = F/xj - ii(gi(X))/xj  F = ii(gi(X))
F/bk = ii(gi(X))/bk = k for a constraint k since (gi(X))/bk = 0 except for i
= k. If i = k, (gi(X))/bk = 1 since bi = gi(X) and (gi(X)) = bi. Similar proof if
slack or surplus variable is included.
[ (gi(X) + Si2)/bi = 1 since gi(X) + Si2 = bi ]

Example Problems:
Find minimum perimeter of fence to enclose a rectangular area of at least A.
Find maximum rectangular area that can be enclosed by a fence of length L.
perimeter = 2(length) + 2(width). Solution: length = width = A
Show graphs of perimeter length as a function of A, or area as fn. of L.

Determine how best to store 10 m3 of NY State maple syrup.

One objective: Minimize total cost of material used to build tank. Let
Average cost per base area = Cb
Average cost per unit side area = Cs
Average cost per unit top area = Ct
Shape? Let the cost determine this. Sphere is best with respect to area but cost of
footprint area (more or less r2) and construction may be excessive. So let’s consider a
rectangular and cylindrical tank. Anything in between would not be optimal. Also an
inverted cone with or without a top area might also be acceptable.
Minimize Totalcost
Subject to:
Totalcost = basecost + sidecost + topcost
[cylindrical tank: unknown variables r and h ]:
basecost = Cb(r2)
sidecost = Cs(2rh)
topcost = Ct(r2)
r2h  10
[rectangular tank: unknown variables l, w, h ]:
basecost = Cb(lw)
 sidecost = Cs 2(lh + wh)
topcost = Ct(lw)
lwh  10

Solution: Using Lagrangian multipliers

For a rectangular tank:
L = 2 Cs h (l + w) + (Cb + Ct) l w - (l w h - 10 )
L/l = 0 = 2 Cs h + (Cb + Ct) w - w h
L/w = 0 = 2 Cs h + (Cb + Ct) l -  l h
L/h = 0 = 2 Cs (l + w) -  l w
L/ = 0 = l w h - 10.

From these first three equations you can prove that

Width w = length l, and that both = 2 Cs h / (Cb + Ct)
hence w/h = l/h = 2 Cs/(Cb+Ct) (compare with cylindrical tank)
since h = 10/ww, substituting that into {w = 2 Cs h / (Cb+Ct)} yields
w = l = [2 Cs Vol / (Cb + Ct)]1/3 and

h = Vol / [2 Cs Vol / (Cb + Ct)]2/3 or

h = Vol1/3 [(Cb + Ct) / 2 Cs ]2/3
Note: = dCost/dVolume at Volume = 10:
= 25.489 the objective worsens if 10 is increased.
Ratio of base + top cost to total cost = 1/3 no matter what costs are.
Ratio of side cost to total cost = 2/3 no matter what costs are.
For circular tank:
L = Cs 2 (22/7) r h + (Cb + Ct) (22/7) r r –  [(22/7)r r h – 10]
L/r = 0 = 2 Cs h (22/7) + 2 (Cb + Ct) (22/7) r - 2 (22/7) r h
L/h = 0 = 2 Cs r (22/7) - (22/7) r r
L/ = 0 = [(22/7)r r h – 10]
Use first two equations to find that r/h = Cs/(Cb+Ct)
r = [ Vol Cs /  (Cb+Ct) ]1/3
h = [ Vol (Cb+Ct)2 /  Cs2]1/3

Now look at cost ratios:

Sidecost/totalcost = Cs 2 (22/7) r h /[Cs 2 (22/7) r h + (Cb + Ct) (22/7) r r]

 = 2 Cs h / [ 2 Cs h + (Cb + Ct) r ]
= 2 Cs h / [ 2 Cs h + (Cb + Ct) h Cs /(Cb+Ct) ]
= 2/3 regardless of marginal (or unit) costs Cs, Ct, Cb!

Exercise: Maximize the total return from allocating 6 resources to three users having
benefit functions derived from the linear demand functions shown below and a third: 8-0.5Z.

$ Demand function: $ Demand function:

6-1X 7 – 1.5 Y

X Y

$ 6
Total Revenue:
(6 - 1 X ) X

X* 3 6 X
Marginal Revenue:
6–2X

Slope of 6 X – X2 = 6 – 2X.
Slope of 7Y – 1.5Y2 = 7 – 3Y.
Slope of 8Z – 0.5Z2 = 8 – Z.

When these slopes equal 0 the functions are at their maximum value. Thus the best for
each user is
X = 3; Y = 7/3; Z = 8.

If the maximum resources available are 6, the optimal solution will have equal slopes =
equal marginal revenues. Why?
Hence:
6 – 2X = MB; 7 – 3Y = MB; 8 – Z = MB. And X + Y + Z will equal 6.
Solving:
X = 1. Y = 1. Z = 4. MB = 4.

Using Lagrange multipliers:

General optimization model:

Max (6 X – X2 ) + (7Y – 1.5Y2 ) + (8Z – 0.5Z2)

Subject to:
X+Y+Z=6
since more resources are desired therefore all 6 will be allocated.

The function for Lagrange multipliers is

L = (6 X – X2 ) + (7Y – 1.5Y2 ) + (8Z – 0.5Z2) - (X+Y+Z-6)

Differentiating with respect to each unknown (X,Y,Z, and setting the result to 0
and solving results in:

L/X = 0 = 6 – 2X - .
L/Y = 0 = 7 – 3Y - .
L/Z = 0 = 8 – Z - .
L/X+Y+Z-6

X = 1, Y = 1, Z = 4, and 

dF/dbi = dF/d6 = 4 = called a ‘dual variable’ or ‘shadow price’