Contents

0.1 Convex Quadratic Programming Problem . . . . . . . . . . . . . . . . 1

0.1.1 Solution of (QP1 ) . . . . . . . . . . . . . . . . . . . . . . . . . 1
0.1.2 Solution of (QP2 ) . . . . . . . . . . . . . . . . . . . . . . . . . 3

0.1 Convex Quadratic Programming Problem

A general convex quadratic programming problem is in the following two forms. Q, is
a real symmetric positive definite matrix of order n × n; c, x ∈ Rn , b ∈ Rm , A is real
matrix of order m × n.

1
(QP1 ) min xT Qx + cT x
2
subject toAx = b

1
(QP2 ) min xT Qx + cT x
2
subject toAx ≤ b, x ≥ 0

Both the quadratic programming problems are convex programming problems since (i)
Q  0 ⇒ xT Qx > 0 and Ax ≤, = b is a linear system.

0.1.1 Solution of (QP1 )

Since (QP1 ) is a convex programming problem so KKT conditions are both necessary
and sufficient for the existence of solution.

1
2 Contents

The lagrange function for (QP1 ) is

1
L(x, µ) = f (x) + µT h(x) = xT Qx + cT x + µT (Ax − b)
2

KKT optimality conditions are

∇x L(x, µ) = 0

Ax = b, µ ∈ Rm

Here Q  0 so Q−1 exists and Q = QT since Q is symmetric.

∇x L(x, µ) = 0

≡ Qx + c + AT µ = 0

≡ x = −Q−1 (c + AT µ)

What is µ?

Ax = b ≡ − AQ−1 (c + AT µ) = b

≡AQ−1 AT µ = −(b + AQ−1 c)

⇒µ = −([AQ−1 AT ]−1 ).(b + (AQ−1 c))

For any nonzero vector x ∈ Rn , xT AQ−1 AT x = (AT x)T Q−1 (AT x) = z T Q−1 z > 0
since Q  0. Hence inverse of AQ−1 AT exists.
Therefore µ = −(AQ−1 AT )−1 (b + AQ−1 c). Hence the optimal solution is

x = −Q−1 [c − AT (AQ−1 AT )−1 (b + AQ−1 c))]

Example 0.1.1. (a) Using the above process find the solution of the following quadratic
programming problems.
(b)Write a program for optimal solution formula and verify your answer.
0.1. Convex Quadratic Programming Problem 3

(c)Also, verify your answer in Python in built code.

1. min 3x21 + x22 + x1 x2 + 2x1 − x2 , s.to x1 + 2x2 = 4

2. min 3x21 + x22 + 2x23 + x1 x2 + x1 x3 + 2x1 − x2 + x3 , s.to x1 + 2x2 + x3 =

4, 3x1 − x2 + 3x3 = 5

0.1.2 Solution of (QP2 )

1
(QP2 ) min xT Qx + cT x
2
subject to Ax ≤ b, x ≥ 0

This is also a convex programming problem. Hence KKT optimality conditions are both
necessary and sufficient for the existence of solution. Denote

1
L(x, λ, θ) = f (x) + λT g(x) + θ(−x) = xT Qx + cT x + λT (Ax − b) + θ(−x)
2
n n n m n n
1 XX X X X X
= qij xi xj + cj x j + λi ( (aij xj − bi )) + θj (−xj )
2 i=1 j=1 j=1 i=1 j=1 j=1

KKT optimality conditions are:

• Normal condition:∇x L(x, λ, θ) = 0. That is, Qx + c + AT λ − θ = 0. which is

same as ni=1 qij xi + cj + m
P P
i=1 aij λi − θj = 0, j = 1, 2, ..., n

Pn
• Feasibility condition: Ax ≤ b, x ≥ 0. This is same as j=1 aij xj ≤ bi , xj ≥ 0,
j = 1, 2, ..., n. Using slack variables sj ≥ 0 these can be written as

n
X
aij xj + si = bi
j=1

• Complementary slackness condition: λi (Ax − b)i = 0 and θj xj = 0 (compo-

nent wise). That is, λi si = 0 and θj xj = 0

• Dual restriction: λi , θj ≥ 0
4 Contents

Summarizing,

∇x L(x, λ) = 0

Ax + s = b,

λi si = 0, θj xj = 0, λj ≥ 0, xj ≥ 0, θj ≥ 0

which is same as

n
X m
X
qij xi + aij λi − θj = 0 = −cj , j = 1, 2, ..., n
i=1 i=1
n
X
aij xj + si = bi , i = 1, 2, ..., m
j=1

λi si = 0, λi , si , xj , θj ≥ 0

Coefficient of si is a unit vector whose ith component is 1 and other components are
zero, i = 1, 2, ..., m. Using n artificial variables zj , j = 1, 2, ..., n in first equation we
can have an identity matrix as basis. So this system can be expressed as a modified linear
system as follows.

min z1 + z2 + .... + zn
n
X m
X
subject to qij xi + aij λi − θj + zj = 0 = −cj , j = 1, 2, ..., n
i=1 i=1
n
X
aij xj + sj = bi , i = 1, 2, ..., m
j=1

λi , si , xj , θj , zj ≥ 0

where λi si = 0, θj xj = 0 are known as complementary slackness condition. λi si = 0

means both can not be nonzero. That is, if λi is in the basis then si should not be in the
basis and vice versa. Both may not be in the basis is also admissible. A variable which
is not in the basis takes the value 0 and once this enters into the basis then its value is
strictly greater than 0. This rule is applicable to θj xj = 0. This rule is known as re-
stricted basis entry rule. This LPP can be solved using some LPP technique, taking care
0.1. Convex Quadratic Programming Problem 5

these restricted basis entry rule in every iteration of simplex table. This method is also
known as WOLFES MODIFIED SIMPLEX METHOD. This modified simplex method
is discussed in one example in the class. For details please see Quadratic Programming
chapter of Engineering Optimization by S.S.Rao.
Exercise: Solve the following quadratic programming problems
1. min 2x21 + 2x22 + 2x1 x2 − 4x1 − 6x2 subject to x1 + 2x2 ≤ 2, x1 , x2 ≥ 0
2. min x21 − x2 − 2x1 subject to 2x1 + 3x2 ≤ 6, 2x1 + x2 ≤ 4, x1 , x2 ≥ 0
3. Convert the quadratic programming problem min x21 −x2 −2x1 subject to 2x1 +3x2 ≤
6, 2x1 + x2 = 4, x1 , x2 ≥ 0 to a modified linear programming problem using KKT Op-
timality condition.