Lagrange

.
.
,
(Barrier and interior point
methods),
Lagrange (Penalty methods and Augmented Lagrangian methods).

.

.

.

. (Barrier
and interior point methods)
Lagrange

(Penalty methods and Augmented Lagrangian methods)

.

.
.

.

.

"

"

min imizef ( x)

gi ( x) 0,

i 1,..., m,

()

.
I (scalar) x

:
1: I ( x)

R 0 {x / g i ( x) 0, i 1,..., m} .

2: {x k } R 0
x B g i ( xB ) 0 i, lim I ( x k ) .
k

 s(r ) r
. r1 r2 0 , s (r1 ) s (r2 ) 0 . {rk }
lim rk 0 , lim s(rk ) 0 .
k

:
:
1. U ( x, r1 ) f ( x) s (r1 ) I ( x) , r1
. x 0 R 0 .

.
2. x 0 x(r1 )
U R {x / gi ( x) 0, i 1,..., m} . x(r1 )
R 0 ,
U = + , x(r1 )
U R.
3. x(r1 ) U ( x, r2 ) , r1 r2 0 .
4. , U ( x, rk )

x(rk 1 )

{rk } .

minimize x1 x2
g1 x12 x2 0,
g 2 x1

s(r)=r

0.

I ( x) ln g i ( x )
i 1

s I o .

L( x, r ) f ( x) r ln g i ( x)
i 1

f ( x) x1 x2 , g1 ( x) x12 x2 g 2 ( x) x1 .
L( x, r ) f ( x) r ln g1 ( x) r ln g 2 ( x)

L( x, r ) x1 x2 r ln( x12 x2 ) r ln x1 .

L( x, r )

.
L( x, r ) 0

L
x
1
L ( x, r )
=0
L
x
2
L
1
1
L
1
r
1 r
( x12 x2 ) r
1 r
(2 x1 )
2
2
x1
x1 x2
x1
x1
x1 x2
x1

r (2 x1 )
L
r
1

2
x1
x1 x2 x1
L
1
L
1
1 r
( x12 x2 )
1 r

2
2
x2
x1 x2
x2
x1 x2

L
r
1
2
x2
x1 x2

r (2 x1 )
r

1 x 2 x x
1
2
1
L ( x , r )
=0

r
1 x 2 x

1
2

r (2 x1 )
r
0 (1)
2
x1 x2 x1

r
0.
x x2

(2)

2
1

(2)

x12 x2 r

(1)

( x12 x2 )(2 x1 ) r
0
x12 x2
x1

1 2 x1

r
0.
x1

x1 0

x1 x1 2 x1 x1

r
0
x1

x1 2 x12 r 0

2 x12 x1 r 0

 2 4 12 4 2 ( r )

1 8r

x1 (r )

x1 (r )

1 1 8r
4

x1 (r ) , x1 (r )

1 1 8r
.
4

x1 (r ) (2)
x12 x2 r

1 1 8r

4

x2 (r )

x2 r

(1 1 8r ) 2
r
16

x1 (r ) x2 ( r ) r.

x2 (1)

r=1

x1 (1)

1 1 8 1 3

0.5
4
4

(1 3) 2
(1 1 8) 2
1 1.25 .
1
16
16

x1 (0.5)

r=0.5

1 1 8 0.5 1 5

0.309
4
4

x2 (0.5)

(1 1 8 0.5) 2
(1 5) 2
0.5
0.5 0.595 .
16
16

r=0.25

x2 (0.25)

(1 1 8 0.25) 2
(1 3) 2
0.25
0.25 0.283 .
16
16

r=0.1

x2 (0.1)

x1 (0.25)

x1 (0.1)

1 1 8 0.25 1 3

0.183
4
4

1 1 8 0.1 1 1.8

0.085
4
4

(1 1 8 0.1) 2
(1 1.8) 2
0.1
0.1 0.107 .
16
16

x1 (r )

x2 ( r )

r1

1.000

0.500

1.250

r2

0.500

0.309

0.595

r3

0.250

0.183

0.283

r4

0.100

0.085

0.107

x1 (r ) x2 ( r ) 4 r.

r. r
( x1 (r ), x2 (r )) (0, 0) .

a: .
. :
min imizef ( x1 ,x2 )= x1 x2
g1 ( x1 ,x2 ) x12 x2 0,
g 2 ( x1 ,x2 ) x1

0.

MATHEMATICA

 Lagrange(Penalty and Augmented

Lagrangian methods)


.
c
,
. c
, .
.

, ,

1.2.5.

min imizef ( x)
h( x) 0 , x X

f: R n

R, h : R n

(2.1)

R m , R n .

x *
Lagrange * = R n .

Lc : Rn Rm

R ,

Lc ( x, ) f ( x) h( x)

c
2
h( x ) ,
2

(2.2)

c .

Lc (, ) x * :

10

() * . , c
, > 0 > 0

Lc ( x, * ) Lc ( x* , * )

x x*

, x

x x* ,

(2.3)

x *
Lc (, ) * .
* , x *
Lc (, ) .
() c . c
.
Lc (, ) . Lc ( x, ) f ( x) x,
Lc ( x, ) f ( x) x . ,

x *

Lc (, ) c .

1
min imizef ( x) ( x12 x22 )
2
x1 =1

(2.4)

x * = (1,0) Lagrange * = -1,

Lc ( x, ) f ( x) h( x)

c
2
h( x ) ,
2

11

1
f ( x) ( x1 x2 ) h( x) x1 1 ,
2

1
c
Lc ( x, ) ( x12 x2 2 ) ( x1 1) ( x1 1) 2 ,
2
2

(2.5)

x( , c)
L( x, r ) 0

L
x
1
L ( x, r )
=0
L
x
2
L 1
c
L
2 x1 2( x1 1)( x1 1)
x1 c( x1 1)
x1 2
2
x1

L
x1 cx1 c .
x1
L 1
2 x2
x2 2

L
x2
x2

x cx1 c
L( x, ) 1
0
x2

12

x1 cx1 c 0 x1 (1 c) c x1

c
1 c

x2 0 .

x1 ( , c)

c
x2 ( , c) 0 .
c 1

(2.6)

, c > 0,

lim* x1 ( , c) x1 ( * , c)

lim* x2 ( , c) x2 ( * , c) .

Lagrange * = -1

lim x1 ( , c) x1 ( * , c) x1 (1, c)

lim x2 ( , c) x2 ( * , c) x2 (1, c) .

(2.6) x1 ( , c)

c
x2 ( , c) 0
c 1

13

lim x1 ( , c) x1 ( * , c) x1 (1, c)

c (1) c 1

1 x1* ,
c 1
c 1

lim* x2 ( , c) x2 ( * , c) x2 (1, c) 0 x2*

* ,
Lc ( x, )
(. 1.2.1).

1.2.1:

1
c
Lc ( x, ) ( x12 x2 2 ) ( x1 1) ( x1 1) 2 ,
2
2

, c= 1 .

Lc ( x , )

x* (1, 0) * = -1.

14

 (2.6) x1 ( , c)

c
x2 ( , c) 0 ,
c 1

,
c

c c 1
1

lim x1 ( , c) lim
c

.
DE L HOSPITAL
c
(c )
1
lim
lim 1 x1*
c c 1
c (c 1)
c 1

lim x1 ( , c) lim
c

lim x2 ( , c) lim0 0 x2* ,

c

c ,
Lc ( x, ) (.
1.2.2).

15

 1.2.2:
1
c
Lc ( x, ) ( x12 x2 2 ) ( x1 1) ( x1 1) 2 ,
2
2

, = 0 c.
Lc ( x, )

x* (1, 0) c .

-
X Rn
,

min imizef ( x)
h( x) 0.

(2.25)

Lc (, ) , :
() Lagrange.
() c .

* Lagrange.
, c k
.
(ill conditioning). ,
c k ,

k .

16

k 1 k c k h( x k ) .

(2.26)

1.2.2,
{ x k } x * ,
{ k c k h( x k )} Lagrange * .

k .
.

1
min imizef ( x) ( x12 x22 )
2
x1 1,

x * = (1,0) Lagrange * = -1.

Lc ( x, ) f ( x) h( x)

c
2
h( x ) ,
2

1
f ( x) ( x12 x2 2 ) h( x) x1 1 ,
2

1
c
Lc ( x, ) ( x12 x2 2 ) ( x1 1) ( x1 1) 2 .
2
2

17

 x k
Lck (, k )

ck k
xk k
,0 .
c 1

(2.27)

(2.27) h( x) x1 1 ,
(2.26) k 1 k c k h( x k )

k
k
ck k

ck 1
k
k c

k
k
c 1

c 1 c 1

k 1 k c k

k
ck k ck 1
k
k 1
c
c k

ck 1

c 1
k

c k 1 k k 1 k c k k k c k c k k c k
c k
k
,

ck 1
ck 1
c 1
c 1

k ck

k 1

ck 1

Lagrange * ,

k 1 *

k ck
ck 1

* .

* = -1

k 1 *

k ck
ck 1

k ck
ck 1

ck 1 k ck ck 1 k 1

k
,
ck 1
ck 1
c 1

18

 k 1 *

k *
ck 1

() k * 1 x k x* (1, 0) { c k }
[

1
k * ,
c 1
k

].
() c k ,
{ k * } c k .

c k .
.

1
min imize ( x12 x22 )
2
x1 1,

x * = (1,0) Lagrange * 1 .

Lc ( x, ) f ( x) h( x)

c
2
h( x ) ,
2

1
f ( x) ( x12 x2 2 )
2

h( x) x1 1 ,

19

1
c
Lc ( x, ) ( x12 x2 2 ) ( x1 1) ( x1 1)2 .
2
2

x k Lck ( x, k )

ck k
xk k
,0 .
c 1

(2.28)

, , c k 1 . c k 1
,
c k 1 k 1 . (2.28)
h( x) x1 1 , (2.26) k 1 k c k h( x k )

k 1

k
k
ck k
ck 1
k
k c
c k
1 c k
k
c 1

c 1 c 1
k

k
ck k ck 1
k
k 1

ck 1

c 1

k ck

c k 1 k k 1 k c k k k c k c k k c k
k c k
k
,

c 1
ck 1
c 1
c 1
k

k 1

k c k
,
ck 1

Lagrange * ,

k 1

k c k
k
* .
c 1
*

* = 1
20

 k 1 *

k c k
k c k c k 1 k c k c k 1 k 1

k
,
ck 1
ck 1
c 1
ck 1
c 1

k 1 *

k *
,
ck 1

k 1 *

k *
ck 1

(2.29)

,
. ,
c k ,
. ,
,
{ c k }, ,
c k 1 . ,
(2.29) , c k
2 [

1
k
c 1
k

]. ,

Lagrange

.
.

21


1.Karmarkar, N., 1984. A New Polynomial-Time Algorithm for Linear Programming,
Combinatorica, Vol.4, pp.373-395.
2.Linear and Nonlinear Rrogramming, Stephen G.Nash / Ariela Sofer
3.Anthony V.Fiacco and Garth P.McCormick Nonlinear Programming Sequential
Unconstrained Minimization Techniques
4.Bertsekas, D. P., 1976. On Penalty and Multiplier Methods for Constrained
Optimization, SIAM J. on Control and Optimization, Vol . 14, pp. 216 235.
5.Bertsekas, D. P., 1976. Multiplier Methods: A Survey, Automatica, Vol. 12, pp.
133 145 .
6.Bertsekas, D. P., 1982 .Constrained Optimization and Lagrange Multiplier Methods,
Academic Press, N. Y.
7.Conn, A. R., Gould, N. I. ., and Toint, 1991. A Globally Convergent Augmented
Lagrangian Algorithm for Optimization with General Constraints and Simple Bounds,
SIAM J .Numer. Anal., Vol. 28, pp. 545 572.
8.Fiacco, A.V., and McCormick, G.P., 1968. Nonlinear Programming: Sequential
Unconstrained Minimization Techniques, Wiley, N. Y.
9.Wright, S. J., 1993. Interior Point Methods for Optimal Control of Discrete Time
Systems, J . Opt. Theory and Appl., Vol .77, pp. 161 187.
10.Wright, S. J., 1993. A Path Following Infeasible - Interior Point Algorithm for
Linear Complementarity Problems, Optimization Methods and Software, Vol. 2, pp.
79 106 .

22