You are on page 1of 16

# Convex Optimization Boyd & Vandenberghe

8. Geometric problems

centering

classification

## placement and facility location

81
Minimum volume ellipsoid around a set

## Lowner-John ellipsoid of a set C: minimum volume ellipsoid E s.t. C E

parametrize E as E = {v | kAv + bk2 1}; w.l.o.g. assume A Sn++
vol E is proportional to det A1; to compute minimum volume ellipsoid,

## minimize (over A, b) log det A1

subject to supvC kAv + bk2 1

## minimize (over A, b) log det A1

subject to kAxi + bk2 1, i = 1, . . . , m

## also gives Lowner-John ellipsoid for polyhedron conv{x1, . . . , xm}

Geometric problems 82
Maximum volume inscribed ellipsoid
maximum volume ellipsoid E inside a convex set C Rn

## parametrize E as E = {Bu + d | kuk2 1}; w.l.o.g. assume B Sn++

vol E is proportional to det B; can compute E by solving

## maximize log det B

subject to supkuk21 IC (Bu + d) 0

## (where IC (x) = 0 for x C and IC (x) = for x 6 C)

convex, but evaluating the constraint can be hard (for general C)

## maximize log det B

subject to kBaik2 + aTi d bi, i = 1, . . . , m

## (constraint follows from supkuk21 aTi (Bu + d) = kBaik2 + aTi d)

Geometric problems 83
Efficiency of ellipsoidal approximations

## Lowner-John ellipsoid, shrunk by a factor n, lies inside C

maximum volume inscribed ellipsoid, expanded by a factor n, covers C

## example (for two polyhedra in R2)

factor n can be improved to n if C is symmetric

Geometric problems 84
Centering

## center of largest inscribed ball (Chebyshev center)

for polyhedron, can be computed via linear programming (page 419)
center of maximum volume inscribed ellipsoid (page 83)

xcheb xmve

## MVE center is invariant under affine coordinate transformations

Geometric problems 85
Analytic center of a set of inequalities

## the analytic center of set of convex inequalities and linear equations

fi(x) 0, i = 1, . . . , m, Fx = g

## is defined as the optimal point of

Pm
minimize i=1 log(fi(x))
subject to F x = g

## more easily computed than MVE or Chebyshev center (see later)

not just a property of the feasible set: two sets of inequalities can
describe the same set, but have different analytic centers

Geometric problems 86
analytic center of linear inequalities aTi x bi, i = 1, . . . , m

xac is minimizer of

m
X xac
(x) = log(bi aTi x)
i=1

where

## Einner = {x | (x xac)T 2(xac)(x xac) 1}

Eouter = {x | (x xac)T 2(xac)(x xac) m(m 1)}

Geometric problems 87
Linear discrimination
separate two sets of points {x1, . . . , xN }, {y1, . . . , yM } by a hyperplane:

aT xi + b > 0, i = 1, . . . , N, aT yi + b < 0, i = 1, . . . , M

## homogeneous in a, b, hence equivalent to

aT xi + b 1, i = 1, . . . , N, aT yi + b 1, i = 1, . . . , M

## a set of linear inequalities in a, b

Geometric problems 88
Robust linear discrimination

## (Euclidean) distance between hyperplanes

H1 = {z | aT z + b = 1}
H2 = {z | aT z + b = 1}

## to separate two sets of points by maximum margin,

minimize (1/2)kak2
subject to aT xi + b 1, i = 1, . . . , N (1)
aT yi + b 1, i = 1, . . . , M

## (after squaring objective) a QP in a, b

Geometric problems 89
Lagrange dual of maximum margin separation problem (1)

maximize 1T + 1T
P N PM
subject to 2 i=1 ixi i=1 iyi 1

(2)
2
1T = 1T ,  0, 0

interpretation

## change variables to i = i/1T , i = i/1T , t = 1/(1T + 1T )

invert objective to minimize 1/(1T + 1T ) = t

minimize t
P N PM
subject to i=1 ixi i=1 iyi t

2
 0, 1 = 1,  0, 1T = 1
T

## Geometric problems 810

Approximate linear separation of non-separable sets

minimize 1T u + 1T v
subject to aT xi + b 1 ui, i = 1, . . . , N
aT yi + b 1 + vi, i = 1, . . . , M
u  0, v  0
an LP in a, b, u, v
at optimum, ui = max{0, 1 aT xi b}, vi = max{0, 1 + aT yi + b}
can be interpreted as a heuristic for minimizing #misclassified points

## Geometric problems 811

Support vector classifier

## minimize kak2 + (1T u + 1T v)

subject to aT xi + b 1 ui, i = 1, . . . , N
aT yi + b 1 + vi, i = 1, . . . , M
u  0, v  0
produces point on trade-off curve between inverse of margin 2/kak2 and
classification error, measured by total slack 1T u + 1T v

with = 0.1:

## Geometric problems 812

Nonlinear discrimination

f (z) = T F (z)

## solve a set of linear inequalities in :

T F (xi) 1, i = 1, . . . , N, T F (yi) 1, i = 1, . . . , M

## Geometric problems 813

quadratic discrimination: f (z) = z T P z + q T z + r

xTi P xi + q T xi + r 1, yiT P yi + q T yi + r 1

## can add additional constraints (e.g., P  I to separate by an ellipsoid)

polynomial discrimination: F (z) are all monomials up to a given degree

## Geometric problems 814

Placement and facility location

## N points with coordinates xi R2 (or R3)

some positions xi are given; the other xis are variables
for each pair of points, a cost function fij (xi, xj )

placement problem
P
minimize i6=j fij (xi, xj )

## variables are positions of free points

interpretations
points represent plants or warehouses; fij is transportation cost between
facilities i and j
points represent cells on an IC; fij represents wirelength

## Geometric problems 815

P
example: minimize (i,j)A h(kxi xj k2), with 6 free points, 27 links

## optimal placement for h(z) = z, h(z) = z 2, h(z) = z 4

1 1 1

0 0 0

1 1 1
1 0 1 1 0 1 1 0 1

## histograms of connection lengths kxi xj k2

4 4 6
5
3 3
4
2 2 3
2
1 1
1
00 0.5 1 1.5 2 00 0.5 1 1.5 00 0.5 1 1.5