Professional Documents
Culture Documents
1 Introduction
1
Since f (x) is a nonlinear function, its minimum may occur any-
where within the feasible region D instead of the extreme points.
Hence we cannot use any technique similar to the Simplex Method.
2
In general case, we transform the constrained NLP (1) to an un-
constrained version by means of a general solution to the system
of linear equations
Ax = b. (2)
Since the system of constraints (2) is the same as what we have
in LP, the similar skill can be utilised. We first divide the n-
dimensional decision-variable vector x into an m-dimensional ba-
sis xB and an (n − m)-dimensional nonbasis xN , and correspond-
ingly the m × n constraint matrix A into an m × m basic matrix
B and an m × (n − m) nonbasic matrix N. Assume without loss
of generality that the first m columns of A give the basic matrix
B. Then the vector equation (2) can be rewritten in the form
BxB + NxN = b.
x = x + ZxN , (4)
3
where x is a particular solution of (2) by setting xN = 0, i.e.
Ax = b.
The matrix Z is called a basis matrix for the null-space of A.1
By replacing (4) into the objective function f (x), the constrained
NLP (1) is transformed into the unconstrained NLP
min z = f (x + ZxN ) = φ(xN )
Function φ(xN ) of (n − m)-dimensional vector xN is called the
reduced function of the constrained NLP problem (1).
x3 0 1
4
! ! ! −4
−1 1 5 −4
B−1 b = = ⇒ x = 13 ,
3 −2 1 13
0
and
! ! ! 5
4 −5 −B−1 N
N= ⇒ B−1 N = ⇒ Z= = −14 .
−1 14 I
1
Hence, the feasible set of the problem is the set of vectors in the
form
−4 5
x = x + Zx3 = 13 + −14 x3 ,
0 1
which is indeed identical to the set obtained by solving the system
of equations directly.
Hence the reduced function of the problem is
φ(x3 ) = f ((−4, 13, 0)T +(5, −14, 1)T x3) = f (5x3 −4, −14x3 +13, x3)
5
where φ(xN ) = f (x + ZxN ), by setting up some optimality nec-
essary conditions and sufficient conditions.
Applying the chain rule for differentiation gives the gradient of
φ(xN )
∇φ(xN ) = ZT ∇f (x),
and the Hessian matrix of φ(xN )
∇2 φ(xN ) = ZT ∇2 f (x)Z.
6
then x∗ a local minimiser of f (x) over the set {x : Ax = b}.
Notice that given a point x for a considered linear equality con-
strained NLP problem we can apply directly the above two theo-
rems without deriving a reduced function.
We have
2x1 − 2 2 0 0
2
∇f (x) = 2x2 and ∇ f (x) = 0 2 0 .
−2x3 + 4 0 0 −2
To find a basis matrix Z for the null-space of A1×3 = (1, −1, 2),
we choose xN = (x2, x3 )T and xB = x1 . Thus, we have N =
(−1, 2) and B = 1. Hence, we have
! 1 −2
−B−1 N
Z= = 1 0 .
I
0 1
7
2 0 0
! 1 −2 !
1 1 0 4 −4
ZT ∇2 f (x∗ )Z = 0 2 0 1 0 = .
−2 0 1 −4 6
0 0 −2 0 1
Since the eigenvalues of the Hessian matrix ZT ∇2 f (x∗ )Z are µ =
√
5 ± 17 > 0, ZT ∇2 f (x∗ )Z is positive definite.2 The second-order
sufficient conditions are satisfied, so x∗ = (2.5, −1.5, −1)T is a
local minimiser of f (x) over the feasible region.
8
This shows that x∗ will satisfy the constraints in (5) and thus be
feasible. To show x∗ is the optimal solution of (5), we let x′ be
any feasible solution to (5). Since (x∗ , Λ∗ ) minimises L(x, Λ), for
any number Λ′ we have
Since x∗ and x′ are both feasible in (5), the terms in (6) involving
the λ’s are all zeros, and (8) becomes f (x∗ ) ≤ f (x′ ). Thus x∗
does solve (5). In short, if (x∗ , Λ∗ ) solves the unconstrained NLP
problem
min L(x, Λ), (9)
then x∗ solves the constrained NLP problem (5).
From Theorem 5 in Lecture Note – Part 8, we know that for
(x∗ , Λ∗ ) to solve (9), it is necessary that at (x∗ , Λ∗ ),
∇L(x, Λ) = 0. (10)
9
∇Λ L(x, Λ) = 0 is exactly (7) and ∇x L(x, Λ) = 0 indicates
m
X
∇f (x) = λi ∇gi (x),
i=1
Then we have
2x1 − 2 − λ = 0 (11)
2x2 + λ = 0 (12)
∇L(x1 , x2, x3 , λ) = 0 ⇒
−2x3 + 4 − 2λ = 0 (13)
x1 − x2 + 2x3 − 2 = 0 (14)
10
which we had identified as a local minimiser of f (x1 , x2 , x3 ) over
the feasible region in Example 3.
In fact, the Lagrangian is especially useful when we consider
the inequality-constrained NLP.
min z = f (x)
(18)
s.t. Ax ≥ b
11
(ii) Λ∗ ≥ 0,
(ii) Λ∗ ≥ 0,
12
the Lagrange multipliers right.5 So we look to solve
min f (x1 , x2) = −2x21 + x1 x2 + x22 − 5x1 − x2
s.t. −x1 − x2 ≥ −1
x2 ≥ 0
Then we have
! !
−1 −1 −1 0
A= ⇒ AT = .
0 1 −1 1
Let Λ = (λ1 , λ2 )T be the vector of Lagrange multipliers associated
with the two constraints. The Lagrangian of the problem is
L(x, Λ) = −2x21 + x1 x2 + x22 − 5x1 − x2 − λ1 (−x1 − x2 + 1) − λ2 x2 .
Then by virtue of the feasibility condition and some simple nec-
essary conditions for a local minimum as follows:
• −x1 − x2 ≥ −1, x2 ≥ 0,
! !
−4x1 + x2 − 5 −λ1
(i) ∇f (x) = = ,
x1 + 2x2 − 1 −λ1 + λ2
(ii) λ1 ≥ 0, λ2 ≥ 0, and
(iii) λ1 (−x1 − x2 + 1) = 0, λ2 x2 = 0,
we can analyse the following four possible cases to obtain the
solution.
13
Since λ1 = − 94 < 0, the necessary conditions do not hold.
14
These sorts of problems are often solved with similar ways. The
modified Newton’s method, which can efficiently solve the con-
strained NLP problem, will be introduced in the advanced course
“Nonlinear Methods in Quantitative Management”.
15
We can treat the primal LP as a general minimisation problem
and form the following Lagrangian
2x1 + 5x2 − 7
x + 4x − 2
1 2
L(x, Λ) = 3x1 + x2 − (λ1 , λ2 , λ3 , λ4 )
x1 − 0
x2 − 0
The first-order optimality conditions are
! !
3 − 2λ1 − λ2 − λ3 0
(i) = ,
1 − 5λ1 − 4λ2 − λ4 0
(ii) Λ ≥ 0, and
16
active, the corresponding optimal dual variable must be zero; if a
constraint is active, then the corresponding optimal dual variable
can be any nonnegative number.
Further reading: Section 11.8–11.9 in the reference book “Operations Research: Ap-
plications and Algorithms” (Winston, 2004)
17