Lagrange Multipliers

Optimization in Engineering Design

Georgia Institute of Technology Systems Realization Laboratory

Lagrange Multipliers
The method of Lagrange multipliers gives a set of necessary conditions to identify optimal points of equality constrained optimization problems. This is done by converting a constrained problem to an equivalent unconstrained problem with the help of certain unspecified parameters known as Lagrange multipliers. The classical problem formulation minimize f(x1, x2, ..., xn) Subject to h1(x1, x2, ..., xn) = 0 can be converted to minimize L(x, l) = f(x) - l h1(x) where L(x, v) is the Lagrangian function l is an unspecified positive or negative constant called the Lagrangian Multiplier
Optimization in Engineering Design

Georgia Institute of Technology Systems Realization Laboratory

Finding an Optimum using Lagrange Multipliers

New problem is: minimize L(x, l) = f(x) - l h1(x)

Suppose that we fix l = l* and the unconstrained minimum of L(x; l) occurs at x = x* and x* satisfies h1(x*) = 0, then x* minimizes f(x) subject to h1(x) = 0. Trick is to find appropriate value for Lagrangian multiplier l. This can be done by treating l as a variable, finding the unconstrained minimum of L(x, l) and adjusting l so that h1(x) = 0 is satisfied.

Optimization in Engineering Design

Georgia Institute of Technology Systems Realization Laboratory

Method
1. 2. Original problem is rewritten as: minimize L(x, l) = f(x) - l h1(x) Take derivatives of L(x, l) with respect to xi and set them equal to zero.
If there are n variables (i.e., x1, ..., xn) then you will get n equations with n + 1 unknowns (i.e., n variables xi and one Lagrangian multiplier l)

3. 4. 5.

Express all xi in terms of Langrangian multiplier l Plug x in terms of l in constraint h1(x) = 0 and solve l. Calculate x by using the just found value for l. Note that the n derivatives and one constraint equation result in n+1 equations for n+1 variables!

(See example 5.3)

Georgia Institute of Technology Systems Realization Laboratory

Optimization in Engineering Design

Multiple constraints The Lagrangian multiplier method can be used for any number of equality constraints.
Suppose we have a classical problem formulation with k equality constraints minimize f(x1, x2, ..., xn) Subject to h1(x1, x2, ..., xn) = 0 ...... hk(x1, x2, ..., xn) = 0 This can be converted in minimize where L(x, l) = f(x) - lT h(x)

lT is the transpose vector of Lagrangian multpliers and has length k

Georgia Institute of Technology Systems Realization Laboratory

Optimization in Engineering Design

In closing Lagrangian multipliers are very useful in sensitivity analyses (see Section 5.3) Setting the derivatives of L to zero may result in finding a saddle point. Additional checks are always useful. Lagrangian multipliers require equalities. So a conversion of inequalities is necessary. Kuhn and Tucker extended the Lagrangian theory to include the general classical single-objective nonlinear programming problem:
minimize Subject to f(x) gj(x) 0 for j = 1, 2, ..., J hk(x) = 0 for k = 1, 2, ..., K x = (x1, x2, ..., xN)
Georgia Institute of Technology Systems Realization Laboratory

Optimization in Engineering Design