Find the solution using KKT

conditions
Problem 1
2 2 2 2
Minimizing f(x,y) = Hx - 2L + Hy - 3L subject to Hx - 4L + Hy - 5L - 6 £ 0.

In[1]:= f = Hx - 2L ^ 2 + Hy - 3L ^ 2; g = Hx - 4L ^ 2 + Hy - 5L ^ 2 - 6;
LagrangianFunction = f + u * Hg + s ^ 2L

2 2 2 2
Out[2]= H-2 + xL + u I-6 + s2 + H-4 + xL + H-5 + yL M + H-3 + yL

Next, compute the gradient of the Lagrangian function:

In[3]:= GradLF = D@LagrangianFunction, 88x, y, u, s<<D;

GradLF  MatrixForm

Out[4]//MatrixForm=

i
j 2 u H-4 + xL + 2 H-2 + xL y
z
j
j z
z
j
j 2 u H-5 + yL + 2 H-3 + yL z
z
j
j z
z
j
j z
z
j -6 + s2 + H-4 + xL2 + H-5 + yL2
j
j
z
z
z
j
j z
z
k2su {

Find all stationary points by solving the system of equations: GradLF = 0

In[5]:= eqns = Thread@GradLF Š 80, 0, 0, 0<D

2 2
Out[5]= 92 u H-4 + xL + 2 H-2 + xL Š 0, 2 u H-5 + yL + 2 H-3 + yL Š 0, -6 + s2 + H-4 + xL + H-5 + yL Š 0, 2 s u Š 0=

In[6]:= FindRoot@eqns, 8x, 0<, 8y, 0<, 8u, 1<, 8s, 1<D

-23
Out[6]= 9x ® 2.26795, y ® 3.26795, u ® 0.154701, s ® -1.91617 ´ 10 =

(A few remarks about the finction FindRoot: (1) an initial guess for each variable must be given, for example, {x=0, y=0, u=1,
s=1} in the above example. You might want to try a few sets of initial values to see if there exists more than one roots. (2)
FindRoot is a numerical approach and its solution is affected by "finite precision" limit of the computer. In the above example, the
23
solution s=-1.91617 10 can be considerd as s = 0.M
2 KKTconditions.nb

Problem 2
2
Minimize f(x,y) = Hx - 2L^2 + Hy - 3L^2 subject to Hx - 4L^2 + Hy - 5L - 6 = 0.

For this problem, we follow the same procedure as shown in Problem 1 (this only difference is that Problem 2 has an equality
constraint while Problem 1 has a "less than" type constraint) . Pay attention to the different definition of the Lagrangian function.

In[7]:= f = Hx - 2L ^ 2 + Hy - 3L ^ 2; h = Hx - 4L ^ 2 + Hy - 5L ^ 2 - 6;
LagrangianFunction = f + v * h

2 2 2 2
Out[8]= H-2 + xL + v I-6 + H-4 + xL + H-5 + yL M + H-3 + yL

In[9]:= GradLF = D@LagrangianFunction, 88x, y, v<<D;

GradLF  MatrixForm

Out[10]//MatrixForm=

i
j 2 v H-4 + xL + 2 H-2 + xL y
z
j
j z
z
j
j 2 v H-5 + yL + 2 H-3 + yL z
z
j
j z
z
j
j 2 2
z
z
k -6 + H-4 + xL + H-5 + yL {

In[11]:= eqns = Thread@GradLF Š 80, 0, 0<D

2 2
Out[11]= 92 v H-4 + xL + 2 H-2 + xL Š 0, 2 v H-5 + yL + 2 H-3 + yL Š 0, -6 + H-4 + xL + H-5 + yL Š 0=

Finally we are ready to find the stationary points using FindRoot:

In[12]:= FindRoot@eqns, 8x, 0<, 8y, 0<, 8v, 0<D

Out[12]= 8x ® 2.26795, y ® 3.26795, v ® 0.154701<

In[13]:= FindRoot@eqns, 8x, 10<, 8y, 10<, 8v, 0<D

Out[13]= 8x ® 5.73205, y ® 6.73205, v ® -2.1547<

Notice that by setting different initial guess in FindRoot, we end up with two totally different stationary points. You can try more
initial guesses to see if there exists more stationary points (The answer is no).
We need to decide which one of the stationary point is the solution. One thing we can try is to use the second order condition: if
the Hessian of the Lagrangian function with respect to (x,y) is positive definite along directions of feasible changes, then the
stationary point is a local minimum.
 KKTconditions.nb 3

In[14]:= HessianLF = D@ D@ LagrangianFunction, 88x, y<<D, 88x, y<<D;

HessianLF  MatrixForm

Out[15]//MatrixForm=

i2+2v 0
j y
z
j z
k0 2+2v {

At the first stationary point, x = 2.26..., y = 3.26..., v = 0.154701, substitute them into the Hessian matrix (as following), the
matrix is clearly positive definite.

In[16]:= HessianLF . v ® 0.154701  MatrixForm

Out[16]//MatrixForm=

i 2.3094 0
j y
z
j z
k0 2.3094 {

At the second point, x = 5.73 ..., y = 6.73 ... , v = -2.1547, substitute them into the Hessian matrix (as following), the matrix is
clearly negative definite

In[17]:= HessianLF . v ® -2.1547  MatrixForm

Out[17]//MatrixForm=

i -2.3094 0
j y
z
j z
k0 -2.3094 {

This means (2.26..., 3.26...) is a local minimum and (5.73..., 6.73...) is a local maximum.

A Graphical view
You might have already noticed that Problem 1 and Problem 2 are very, very similar. Indeed, using the graphical optimization we
have learned earlier, the relation of the two given functions will be clear:
4 KKTconditions.nb

In[18]:= Show@8
ContourPlot@h, 8x, -2, 10<, 8y, -2, 10<, Contours ® 80<,
Axes -> True, Frame ® False, AxesLabel ® 8"x", "y"<,
ContourShading ® 8White, White<, ContourStyle ® 8Black, Thick<D,
ContourPlot@f, 8x, -2, 10<, 8y, -2, 10<, Contours ® 80.14, 3, 10, 27.8, 50<,
Axes -> True, Frame ® False, AxesLabel ® 8"x", "y"<,
ContourShading ® False, ContourStyle ® Directive@Blue, DashedD,
ContourLabels ® Function@8x, y, z<, Text@Framed@zD, 8x, y<, Background ® WhiteDDD<,
Epilog ® 8
Text@Style@"Hx-4L^2+Hy-5L^2-6=0", Medium, BoldD, 85.5, 2.2<D,
Disk@82.27, 3.27<, 8.2, .2<D,
Disk@85.73, 6.73<, 8.2, .2<D<
D

10

50

8
27.8

3
Out[18]=
4

0.14
2 Hx-4L^2+Hy-5L^2-6=0

10 x
-2 2 4 6 8 10
27.8

-2 50