Quadratic Programs

(i) Equality-constrained QP’s

1 T
min x Qx − c T x subject to
x∈Rn 2
aiT x = bi i = 1, . . . , m
or
1
minn x T Qx − c T x subject to
x∈R 2
Ax = b
 T
a1
 .. 
where Q is symmetric, m < n and A =  . . The Lagrangian is
T
am
m
1 X
L (x, λ) = x T Qx − c T x − λi (aiT x − bi )
2
i=1
1
= x T Qx − c T x − λT (Ax − b)
2
October 5, 2007 (1 : 5)
 Quadratic Programs

KKT-point ∇x L (x ∗ , λ∗ ) = 0 yields the linear system

Qx ∗ − AT λ∗ = c
Ax ∗ = b
or
Q −AT x∗
    
c
=
A 0 λ∗ b
where
Q −AT
 
K=
A 0
is called a KKT matrix. Write the constraint −Ax = −b, substitute
B = −A. Then, the KKT matrix becomes symmetric:
  ∗  
Q BT

x c
=
B 0 λ∗ −b

October 5, 2007 (2 : 5)
 Quadratic Programs

Assumptions:
I A has linearly independent rows.
I Q is positive definite in the null space of A. (That is, x T Qx > 0 for
all x 6= 0 such that Ax = 0).

Then holds:

Theorem
Matrix K is nonsingular.
Theorem
The solution x ∗ of the KKT system is the unique global solution of the
equality-constrained QP.

I The equality-constrained QP is a convex problem under the above

assumptions.
I Note: matrix K is indefinite.
I Thus, solving a convex equality-constraint QP is “easy”
I Equivalent to solving a linear system (the KKT system)
October 5, 2007 (3 : 5)
 Quadratic Programs

(ii) Inequality-constrained QP’s

1
minn x T Qx − c T x subject to
x∈R 2
Ax ≥ b
Lagrangian: L (x, λ) = 21 x T Qx − c T x − λT (Ax − b)
KKT conditions:
Qx ∗ − AT λ∗ = c
λ∗ ≥ 0
Ax ∗ ≥ b
λ∗i (aiT x ∗ − bi ) = 0 i = 1, . . . , m
Define the active set n o
A= i | aiT x ∗ = bi
Note that λ∗i = 0 for i 6∈ A

October 5, 2007 (4 : 5)
 Quadratic Programs

I We may delete all inactive inequality constraints and corresponding

zero Lagrange multipliers
I Let Ā be A with all rows for i 6∈ A deleted
I Let b̄ be b with all rows for i 6∈ A deleted
I Let λ̄∗ be λ∗ with all components for i 6∈ A deleted.
Then the KKT conditions simplify to
Qx ∗ − ĀT λ̄∗ = c
Āx ∗ = b̄
i. e. the KKT conditions for a QP with equality constraints
Note: The above form assume that A is known (which it generally not is!)

October 5, 2007 (5 : 5)