# Linear Conic Programming

Yinyu Ye
December 2004

i

ii

Then came the primal-dual algorithm. I suggested that he use X −1/2 ∆X −1/2 to keep symmetry instead of X −1 ∆ which he was using. He returned to Minneapolis and moved to Berkeley shortly after.. complexity bound. Information. and computer programs related to this book may be found at the following address on the World-Wide Web: http://www.edu A little story in the development of semideﬁnite programming.Preface This monograph is developed for MS&E 314. and one afternoon showed up in my oﬃce in Iowa City. and SDP established its full popularity. . At the same time or even earlier. many of which were incredibly novel and elegant. I urged Farid to look at the LP potential functions and to develop an SDP primal potential reduction algorithm. the Max-Cut. and few weeks later sent me an e-mail message telling me that everything had worked out beautifully. Boyd and his group later presented a wide range of SDP applications and formulations. algorithm. Farid Alizadeh.edu/class/msande314 Please report any question. YINYU YE iii . One day in 1990. about 300 miles from Minneapolis. He.. He worked hard for several months. including potential. supporting materials. where SDP was a special case. introduced me “semideﬁnite optimization” or linear programming over the positive deﬁnite matrix cone. We went to a bar nearby on Clinton Street in Iowa City (I paid for him since I was a third-year professor then and eager to demonstrate that I could take care of students). and even a “dictionary” from LP to SDP. but was stuck on one problem which was how to keep the symmetry of the matrix. “Semideﬁnite Programming”. After chatting for a while. Nesterov and Nemirovskii developed a more general and powerful theory in extending interior-point algorithms for solving conic programs. We had a very extensive discussion that afternoon and concluded that interior-point linear programming algorithms could be applicable to solving SDP.stanford. He had everything worked out. where X is the current symmetric positive deﬁnite matrix and ∆ is the symmetric directional matrix. I visited the Computer Science Department of the University of Minnesota and met a young graduate student. working then on combinatorial optimization. which I am teaching at Stanford. comment and error to the address: yinyu-ye@stanford.

2002 PREFACE .iv Stanford.

Chapter 1

Introduction and
Preliminaries
1.1

Introduction

Semideﬁnite Programming, hereafter SDP, is a natural extension of Linear programming (LP) that is a central decision model in Management Science and
Operations Research. LP plays an extremely important role in the theory and
application of Optimization. In one sense it is a continuous optimization problem in minimizing a linear objective function over a convex polyhedron; but it
is also a combinatorial problem involving selecting an extreme point among a
ﬁnite set of possible vertices. Businesses, large and small, use linear programming models to optimize communication systems, to schedule transportation
networks, to control inventories, to adjust investments, and to maximize productivity.
In LP, the variables form a vector which is regiured to be nonnegative, where
in SDP they are components of a matrix and it is constrained to be positive
semideﬁnite. Both of them may have linear equality constraints as well. One
thing in common is that interior-point algorithms developed in past two decades
for LP are naturally applied to solving SDP.
Interior-point algorithms are continuous iterative algorithms. Computation
experience with sophisticated procedures suggests that the number of iterations
necessarily grows much more slowly than the dimension grows. Furthermore,
they have an established worst-case polynomial iteration bound, providing the
potential for dramatic improvement in computation eﬀectiveness.
The goal of the monograph is to provide a text book for teaching Semideﬁnite
Programming, a modern Linear Programming decision model and its applications in other scientiﬁc and engineering ﬁelds. One theme of the monograph is
the “mapping” between SDP and LP, so that the reader, with knowledge of LP,
can understand SDP with little eﬀort.
The monograph is organized as follows. In Chapter 1, we discuss some
1

2

CHAPTER 1. INTRODUCTION AND PRELIMINARIES

necessary mathematical preliminaries. We also present several decision and
optimization problems and several basic numerical procedures used throughout
the text.
Chapter 2 is devoted to studying the theories and geometries of linear
and matrix inequalities, convexity, and semideﬁnite programming. Almost all
interior-point methods exploit rich geometric properties of linear and matrix
inequalities, such as “center,” “volume,” “potential,” etc. These geometries are
also helpful for teaching, learning, and research.
Chapter 3 is focused on interior-point algorithms. Here, we select two types
algorithms: the path-following algorithm and the potential reduction algorithm.
Each algorithm has three forms, the primal, the dual and the primal-dual form.
We analyze the worst-case complexity bound for them, where we will use the
real number computation model in our analysis because of the continuous nature
of interior-point algorithms. We also compare the complexity theory with the
convergence rate used in numerical analysis.
Not only has the convergnece speed of SDP algorithms been signiﬁcantly
improved during the last decade, but also the problem domain applicable by
SDP has dramatically widened. Chapters 4, 5, and 6 would describe some
of SDP applications and new established results in Engineering, Combinatory
Optimization, Robust Optimization, Euclidean Geometry Computation, etc.
Finally, we discuss major computational issues in Chapter 7. We discuss several eﬀective implementation techniques frequently used in interior-point SDP
software, such as the sparse linear system, the predictor and corrector step, and
the homogeneous and self-dual formulation. We also present major diﬃculties
and challengies faced by SDP.

1.2

Mathematical Preliminaries

This section summarizes mathematical background material for linear algebra,
linear programming, and nonlinear optimization.

1.2.1

Basic notations

The notation described below will be followed in general. There may be some
deviation where appropriate.
By R we denote the set of real numbers. R+ denotes the set of nonnegative

real numbers, and R+ denotes the set of positive numbers. For a natural number

n, the symbol Rn (Rn , Rn ) denotes the set of vectors with n components in
+
+

R (R+ , R+ ). We call Rn the interior of Rn .
+
+
The vector inequality x ≥ y means xj ≥ yj for j = 1, 2, ..., n. Zero represents
a vector whose entries are all zeros and e represents a vector whose entries
are all ones, where their dimensions may vary according to other vectors in
expressions. A vector is always considered as a column vector, unless otherwise
stated. Upper-case letters will be used to represent matrices. Greek letters will

1.2. MATHEMATICAL PRELIMINARIES

3

typically be used to represent scalars. For convenience, we sometime write a
column vector x as
x = (x1 ; x2 ; . . . ; xn )
and a row vector as
x = (x1 , x2 , . . . , xn ).
Addition of matrices and multiplication of matrices with scalars are standard. The superscript “T” denotes transpose operation. The inner product in
Rn is deﬁned as follows:
n

x, y := xT y =

xj yj

for

x, y ∈ Rn .

j=1

The l2 norm of a vector x is given by
x

2

=

xT x,

and the l∞ norm is
x

= max{|x1 |, |x2 |, ..., |xn |}.

In general, the p norm is
1/p

n

x

p

=

|xj |

p

,

p = 1, 2, ...

1

The dual of the p norm, denoted by . ∗ , is the q norm, where
1 1
+ = 1.
p q
In this book, . generally represents the l2 norm.
For natural numbers m and n, Rm×n denotes the set of real matrices with m
rows and n columns. For A ∈ Rm×n , we assume that the row index set of A is
{1, 2, ..., m} and the column index set is {1, 2, ..., n}. The ith row of A is denoted
by ai. and the jth column of A is denoted by a.j ; the i and jth component of
A is denoted by aij . If I is a subset of the row index set and J is a subset of
the column index set, then AI denotes the submatrix of A whose rows belong
to I, AJ denotes the submatrix of A whose columns belong to J, AIJ denotes
the submatrix of A induced by those components of A whose indices belong to
I and J, respectively.
The identity matrix is denoted by I. The null space of A is denoted N (A)
and the range of A is R(A). The determinant of an n × n-matrix A is denoted
by det(A). The trace of A, denoted by tr(A), is the sum of the diagonal entries
in A. The operator norm of A, denoted by A , is
A

2

:= max n
0=x∈R

Ax 2
.
x 2

if Q 0. INTRODUCTION AND PRELIMINARIES For a vector x ∈ Rn . The inner product in Mn is deﬁned as follows: X..4 CHAPTER 1. denoted by Q 0. denoted xk → x. the notation g(x) = O(x) means that g(x) ≤ cx for some constant c. the notation ¯ ¯ g(x) = Ω(x) means that g(x) ≥ cx for some constant c. . If Q 0. if ¯ ¯ xk − x → 0.. furthermore. x . the notation g(x) = θ(x) means that cx ≤ g(x) ≤ cx. + + {xk }∞ is an ordered sequence x0 . If Q is symmetric. If g(x) ≥ 0 is a real valued function of a real nonnegative variable. ¯ A point x is a limit point of {xk }∞ if there is a subsequence of {xk } convergent 0 to x.e. Q is PSD if and only if all its eigenvalues are non-negative. Another notation is g(x) = o(x). A matrix Q ∈ Rn×n is said to be positive deﬁnite (PD). for all 0. then −Q is called negative deﬁnite (ND). Dx = diag(x).j i.. Mn denotes the space of symmetric matrices in Rn×n . The matrix norm associated with the inner product is called Frobenius norm: √ X f = trX T X . denoted by Q 0. 0. Xi. ◦ Mn denote the set of positive semi-deﬁnite matrices in Mn . Mn denotes the + + ◦ set of positive deﬁnite matrices in Mn . Y := X • Y = trX T Y = for X. for vector x as x Q = xT Qx .. i. denoted by Q if xT Qx > 0. if x. A sequence {xk }∞ is 0 0 convergent to x. We call Mn the interior of Mn .. then −Q is called negative semi-deﬁnite (NSD).j This is a generalization of the vector inner product to matrices.. . Y ∈ Mn . x2 . and positive semi-deﬁnite (PSD). which means that ¯ g(x) goes to zero faster than x does: lim x→0 g(x) = 0. . then its eigenvalues are all real numbers.j Yi. for all x = 0. Given a PD matrix Q we can deﬁne a Q-norm .. x1 . denoted by Q xT Qx ≥ 0. Dx represents a diagonal matrix in Rn×n whose diagonal entries are the entries of x. Q . and Q is PD if and only if all its eigenvalue are positive. xk .

0 < α < 1.1. called the positive semi-deﬁnite matrix cone.3 The set {(t. implies x ∈ Ω. A set is compact if it is both closed and bounded. + Example 1. for a positive deﬁnite matrix Q of dimension n. The dual of the cone is also Rn . we write x ∈ Ω. the intersection of them is denoted S ∩ T .2 5 Convex sets If x is a member of the set Ω. y ≥ 0 for all x ∈ C}. A set can be speciﬁed in the form Ω = {x : P (x)} as the set of all elements satisfying property P . is a convex cone.2. ) = {x : x − y ≤ } is the ball or sphere of radius with center y. A set is bounded if it is contained within a ball with ﬁnite radius. Mn .2 The set of all positive semi-deﬁnite matrices in Mn . A cone that is also convex is a convex cone. Example 1.2. and neither the positive semi-deﬁnite matrix cone nor the second-order cone is polyhedral. is the set of points in Ω which ˆ are the centers of some balls contained in Ω. is a + convex cone. Rn = {x ∈ Rn : x ≥ + 0}. Example 1. ) ⊂ Ω. The ◦ (topological) interior of any set Ω. E(y.1). the dual of C is the cone C ∗ := {y : x. The boundary of Ω is the part of Ω that ◦ is not in Ω. x2 ∈ C and every real number α. B(y. called the second-order cone. A set Ω is closed if xk → x. The closure of Ω. + Example 1.. if y is not a member of Ω. we write y ∈ Ω.4 The non-negative orthant is a polyhedral cone. The union of two sets S and T is denoted S ∪ T . In addition. The vector y is the center of E(y. . denoted Ω. denoted Ω. MATHEMATICAL PRELIMINARIES 1. A set C is said to be convex if for every x1 . For y ∈ Rn and > 0. i. · is an inner product operation for space E. where ·. A set Ω is open if around every point y ∈ Ω there is a ball that is contained in Ω. A cone C is (convex) polyhedral if C can be represented by C = {x : Ax ≤ 0} for some matrix A (Figure 1. A set C is a cone if x ∈ C implies αx ∈ C for all α > 0. it is self-dual. The dual of the cone is also Mn . For a cone C ⊂ E. it is self-dual. the point αx1 + (1 − α)x2 ∈ C. Q). Q) = {x : (x − y)T Q(x − y) ≤ 1} is called an ellipsoid. is ˆ the smallest closed set containing Ω. x) ∈ Rn+1 : t ≥ x } is a convex cone in Rn+1 . The dual of the cone is also the second-order cone in Rn+1 . there is an > 0 such that B(y. where xk ∈ Ω.e.1 The n-dimensional non-negative orthant. The convex hull of a set Ω is the intersection of all convex sets containing Ω. it is self-dual.

1: Polyhedral and nonpolyhedral cones. Hyperplanes dominate the entire theory of optimization. Relating to hyperplane. Let a be a nonzero n-dimensional vector. .2: A hyperplane and half-spaces. and let b be a real number.6 CHAPTER 1.2). INTRODUCTION AND PRELIMINARIES Polyhedral Cone Nonpolyhedral Cone Figure 1. The set H = {x ∈ Rn : aT x = b} is a hyperplane in Rn (Figure 1. + H H - a + H 0 Figure 1. A set which can be expressed as the intersection of a ﬁnite number of closed half spaces is said to be a convex polyhedron: P = {x : Ax ≤ b}. positive and negative closed half spaces are given by H+ = {x : aT x ≥ b} H− = {x : aT x ≤ b}. The most important type of convex set is a hyperplane.

A polyhedron has only ﬁnite many faces.5 Let C be a unit circle centered at the point (1. there is a hyperplane containing y that contains C in one of its open half spaces. given a convex set C and a point y outside of C. The real function f (x) is said to be continuous on set Ω ⊂ E. 1) is a separating hyperplane vector. . because all our decision and optimization problems take variables from one or both of these two vector spaces. where E is either Rn or Mn . MATHEMATICAL PRELIMINARIES 7 A bounded polyhedron is called polytope. −1). throughout this book. 1). Let P be a polyhedron in Rn . Theorem 1.3: Illustration of the separating hyperplane theorem. That is. If y = (0. a = (−1. a = (0. F is a face of P if and only if there is a vector c for which F is the set of points attaining max {cT x : x ∈ P } provided the this maximum is ﬁnite.1. 0). If y = (2.3 Real functions The real function f (x) is said to be continuous at x if xk → x implies f (xk ) → f (x). It is worth noting that these separating hyperplanes are not unique. an exterior point y is separated by a hyperplane from a convex set C. We use the notation E to represent either Rn or Mn .1 (Separating hyperplane theorem) Let C ⊂ E. Then there is a vector a ∈ E such that a. Example 1.2.2. The most important theorem about the convex set is the following separating theorem (Figure 1. 1) is a separating hyperplane vector. C = {x ∈ R2 : (x1 − 1)2 + (x2 − 1)2 ≤ 1}. if f (x) is continuous at x for every x ∈ Ω. be a closed convex set and let y be a point exterior to C. x∈C The geometric interpretation of the theorem is that. y < inf a. where recall that E is either Rn or Mn . each face is a nonempty polyhedron. x . C y a Figure 1. 1. depending on the context.3).

A group of results that are used frequently in analysis are under the heading of Taylor’s theorem or the mean-value theorem.. C • X + and det(X) are homogeneous of degree 1.. ◦ Example 1.. we say f ∈ C p .. Then cT x is homogeneous of + degree 1 and n T P(x) = n log(c x) − log xj j=1 is homogeneous of degree 0. f2 . f (αx + (1 − α)y) ≤ max[f (x). j = 1.8 CHAPTER 1.... . where log is the natural logarithmic function.. The level set of f is given by L(z) = {x : f (x) ≤ z}. k.. . the Jacobian matrix of f is   f1 (x) .. Let ◦ C ∈ Mn be given and X ∈Mn . The gradient vector or matrix of a real-valued function f ∈ C 1 is a vector or matrix f (x) = {∂f /∂xij }. f2 . If f has continuous partial derivatives of order p. we deﬁne the Hessian of f to be the n2 -dimensional symmetric matrix 2 f (x) = ∂2f ∂xij ∂xkl for i.... A set of real-valued function f1 . n. f is a quasi-convex function implies that the level set of f is convex for any given z (see Exercise 1.9). fm )T ∈ Rm . .. l = 1. f is a (continuous) quasi-convex function if and only if for 0 ≤ α ≤ 1. f (αx + (1 − α)y) ≤ αf (x) + (1 − α)f (y). Thus. . n. a convex function is a quasi-convex function. f (x) =  fm (x) f is a (continuous) convex function if and only if for 0 ≤ α ≤ 1.. j. . fm deﬁned on E can be written as a single vector function f = (f1 .. . .6 Let c ∈ Rn be given and x ∈Rn . fm )T ∈ Rm . f2 . The theorem establishes the linear and quadratic approximations of a function.. and P(X) = n log(C • X) − log det(X) is homogeneous of degree 0. Then xT Cx is homogeneous of degree 2. INTRODUCTION AND PRELIMINARIES A function f (x) is called homogeneous of degree k if f (αx) = αk f (x) for all α ≥ 0. If f = (f1 . If f ∈ C 2 . for i. f (y)].

◦ Harmonic: given x ∈Rn .4 Inequalities There are several important inequalities that are frequently used in algorithm design and complexity analysis. The following proposition states that the Hessian of a convex function is positive semi-deﬁnite. Hadamard: given A ∈ Rm×n with columns a1 .. if f ∈ C 2 then there is a α. 0 ≤ α ≤ 1. then xT y ≤ x y .2. y ∈ Ω. such that f (y) = f (x) + f (x)(y − x) + (1/2)(y − x)T 2 f (αx + (1 − α)y)(y − x). y ∈ Rn . 1. MATHEMATICAL PRELIMINARIES 9 Theorem 1. y]. The ﬁrst indicates that the linear approximation of a convex function is a under-estimate.3 Let f ∈ C 1 . Then f is convex over a convex set Ω if and only if the Hessian matrix of f is positive semi-deﬁnite throughout Ω. Then f is convex over a convex set Ω if and only if f (y) ≥ f (x) + f (x)(y − x) for all x. then det(AT A) ≤ aj . Arithmetic-geometric mean: given x ∈ Rn . . 0 ≤ α ≤ 1. Proposition 1. Then there is a α. .. Proposition 1. + xj ≥ n xj 1/n . We also have several propositions for real functions.2. a2 .4 Let f ∈ C 2 . an . such that f (y) = f (x) + f (αx + (1 − α)y)(y − x). Cauchy-Schwarz: given x..2 (Taylor expansion) Let f ∈ C 1 be in a region containing the line segment [x.1. + xj 1/xj ≥ n2 . Furthermore.

In general. we list several well-known results of linear algebra. −1) and b = (1.5 Weierstrass theorem A continuous function f deﬁned on a compact set (bounded and closed) Ω ⊂ E has a minimizer in Ω. . Sp in this case is an aﬃne set.7 Let A = (1. one can easily check to see if x is in Sp by a matrix-vector multiplication and a vector-vector comparison. i. for each linear subspace of L of Rn there are matrices A and C such that L = N (A) = R(C). where necessity is straightforward. that is. More problems will be listed later when we address them. We say that a solution of this problem is easy to recognize. there is an x∗ ∈ Ω such that for all x ∈ Ω. The ﬁrst theorem provides two basic representations. the problem is to solve m linear equations for n unknowns: Ax = b. Theorem 1.6 Each linear subspace of Rn is generated by ﬁnitely many vectors. is called an infeasibility certiﬁcate for the system {x : Ax = b}. The data and solution sets are Zp = {A ∈ Rm×n .3 CHAPTER 1. It gives an example of a characterization in terms of necessary and suﬃcient conditions. problem P may have no solution. In what follows. P. with AT y = 0 and bT y = 1. Sp can be implicitly deﬁned by the so-called optimality conditions. Example 1. Theorem 1. the null and row spaces. Given an x. To highlight the analogy with the theories of linear inequalities and linear programming. A vector y. y = (1/2. The solution set may be empty. The system {x : Ax = b} has a solution if and only if that AT y = 0 implies bT y = 0. Then. and is also the intersection of ﬁnitely many linear hyperplanes. It is sometimes called the fundamental theorem of linear algebra. INTRODUCTION AND PRELIMINARIES Some Basic Decision and Optimization Problems A decision or optimization problem has a form that is usually characterized by the decision variables and the constraints. and suﬃciency is the key of the characterization.e. we list several decision and optimization problems. The following theorem was observed by Gauss. b ∈ Rm } and Sp = {x ∈ Rn : Ax = b}. 1/2) is an infeasibility certiﬁcate for {x : Ax = b}. of a linear subspaces.1 System of linear equations Given A ∈ Rm×n and b ∈ Rm . consists of two sets. A problem.. that is. 1). data set Zp and solution set Sp . 1.10 1. Theorem 1. f (x) ≥ f (x∗ ).3.7 Let A ∈ Rm×n and b ∈ Rm .

The data and solution sets are Zp = {A ∈ Rm×n . The inequality .3. SOME BASIC DECISION AND OPTIMIZATION PROBLEMS 1. c ∈ Rn } and Sp = {y ∈ Rm : AT y − c 2 ≤ AT x − c 2 for every x ∈ Rm }. the standard inverse of full rank matrix AAT . y is called the decision variable. we say this (optimization) problem is unconstrained. the problem is to ﬁnd an y ∈ Rm or s ∈ R(AT ) such that AT y − c or s − c is minimized. Such a case usually occurs when the number of equations is greater than the number of variables. where AAT is called normal matrix and (AAT )+ is called pseudo-inverse. from a projection theorem in linear algebra. However. the system of equations AT y = c may be overdetermined or have no solution.15). the solution set can be characterized and represented as Sp = {y ∈ Rm : AAT y = Ac}. In the full rank case. In the former format. the problem is to ﬁnd a solution x ∈ Rn satisfying Ax ≤ b or prove that the solution set is empty. It is the solution of the following least-squares problem: (LS) minimize subject to x−c 2 x ∈ N (A). Then. 2 (LS) minimize subject to s−c 2 s ∈ R(AT ). Given a y.2 11 Linear least-squares problem Given A ∈ Rm×n and c ∈ Rn . The vector c − AT y = (I − AT (AAT )+ A)c is the projection of c onto the null space of A.3 System of linear inequalities Given A ∈ Rm×n and b ∈ Rm . If A has full row rank then (AAT )+ = (AAT )−1 . Since y can be any point in Rm .3. both matrices AT (AAT )−1 A and I − AT (AAT )−1 A are called projection matrices. the term AT y − c 2 is called the objective function. which becomes a system of linear equations and always has a solution. to see if y ∈ Sp is as the same as the original problem.1. The vector s = AT y = AT (AAT )+ Ac is the projection of c onto the range of AT . We can write the problem in the following format: (LS) or minimize subject to AT y − c y ∈ Rm . neither is AAT and (AAT )+ AAT x = x only for x ∈ R(AT ). These symmetric matrices have several desired properties (see Exercise 1. 1.3. If A is not of full rank.

In this case we can show that any certiﬁcate proving their infeasibility must have large norm. 1/ ) has a large norm. the system {x : Ax = b. . Theorem 1. x ≥ 0} has a feasible solution x if and only if that AT y ≤ 0 implies bT y ≤ 0. . 1) and b = −1. x ≥ 0}.. 1) is an infeasibility certiﬁcate for {y : AT y ≤ c}. Let A = (1. Again. Then. Then. Example 1. Then.8 (Farkas’ lemma) Let A ∈ Rm×n and b ∈ Rm . 0) is approximately feasible for {x : Ax = b. −1) and c = (1. x ≥ 0}. x ≥ 0} or {y : AT y ≤ c} is “approximately infeasible” in the sense that we have an approximate certiﬁcate in Farkas’ lemma. x ≥ 0} or {y : AT y ≤ c} is approximately feasible in the sense that we have an approximate solution to the equations and inequalities.. and the infeasibility certiﬁcate y = −1/ has a large norm. A vector y.1 .9 Let A = (1. b ∈ Rm } and Sp = {x ∈ Rn : Ax = b. Then.10 Given > 0 but small. then there is a hyperplane separating b from cone(a. y = −1 is an infeasibility certiﬁcate for {x : Ax = b.9 (Farkas’ lemma variant) Let A ∈ Rm×n and c ∈ Rn .. y = 1 is approximately feasible for {y : AT y ≤ c}. Then. if {x : Ax = b. Theorem 1. a vector x ≥ 0. The data and solution sets of the latter are Zp = {A ∈ Rm×n .. −1 − ). −2).1 . is called a (primal) infeasibility certiﬁcate for the system {x : Ax = b.8 Let A = (1. a. with Ax = 0 and cT x = −1.12 CHAPTER 1. Example 1. −1) and c = (1. is called a (dual) infeasibility certiﬁcate for the system {y : AT y ≤ c}. and a strictly positive point in Sp is called a strictly feasible or interior feasible solution. Farkas’ lemma means that if a vector b ∈ Rm does not belong to the cone generated by a.n . We say {x : Ax = b. Geometrically. . INTRODUCTION AND PRELIMINARIES problem includes other forms such as ﬁnding an x that satisﬁes the combination of linear equations Ax = b and inequalities x ≥ 0.. a point in Sp is called a feasible solution.. The following results are Farkas’ lemma and its variants. x = (0. Then.n ). then any feasible solution must have large norm. x ≥ 0}. x = (1. and the infeasibility certiﬁcate x = (1/ . with AT y ≤ 0 and bT y = 1. the system {y : AT y ≤ c} has a solution y if and only if that Ax = 0 and x ≥ 0 imply cT x ≥ 0. Example 1. 1) and b = − . x ≥ 0}. Let A = (1. a. Traditionally. Conversely.

is the following problem: (LD) maximize subject to bT y AT y + s = c. cT x ≥ bT y where x ∈ Fp . c ∈ Rn } and Sp = {x ∈ Fp : cT x ≤ cT y. x ≥ 0} is called feasible set or feasible region. SOME BASIC DECISION AND OPTIMIZATION PROBLEMS 1. another linear program. respectively. where y ∈ Rm and s ∈ Rn . The linear function cT x is called the objective function. (y. and a feasible point x∗ is called an optimal solution if cT x∗ ≤ cT x for all feasible points x. If a variable is unbounded either from below or above. The components of s are called dual slacks.4 13 Linear programming (LP) Given A ∈ Rm×n . s) ∈ Fd . we can simplify the representation of the solution set signiﬁcantly. The set Fp = {x : Ax = b. l. given an x. b ∈ Rm and c. A point x ∈ Fp is called a feasible point. u ∈ Rn . the linear programming (LP) problem is the following optimization problem: minimize subject to cT x Ax = b. and some elements in u may be ∞ meaning that the associated variables are unbounded from above. then (LP) is said to be unbounded. The data and solution sets for (LP). s) that are feasible for the dual. which we will use throughout this book: (LP ) minimize cT x subject to Ax = b. Denote by Fd the sets of all (y. s ≥ 0. Then. x ≥ 0. where some elements in l may be −∞ meaning that the associated variables are unbounded from below. If there is a sequence {xk } such that xk is feasible and cT xk → −∞.1. are Zp = {A ∈ Rm×n . The following theorems give us an important relation between the two problems. and x is called the decision variables. b ∈ Rm . for every y ∈ Fp }. We see that (LD) is also a linear programming problem where y is a “free” vector. due to the duality theorem. In this problem.3. Again. However. . With every (LP). then it is called a “free” variable The standard form linear programming problem is given below. called the dual (LD). to see if x ∈ Sp is as diﬃcult as the original problem.10 (Weak duality theorem) Let Fp and Fd be non-empty. Ax = b and x ≥ 0 enforce constraints on the selection of x.3. l ≤ x ≤ u. Theorem 1.

. If one of (LP) or (LD) has no feasible solution. . R . then these are both optimal. If xT s = 0. Now it is easy to verify whether or not a pair (x.11 (Strong duality theorem) Let Fp and Fd be non-empty. If one of (LP) or (LD) is unbounded then the other has no feasible solution. s). Then. R+ ) :  −AT y − s  = 0  = b . n. s∗ ) ∈ Fd . . The converse is also true. iii) cT x∗ = bT y ∗ . ii) there is (y ∗ . y. Theorem 1. y. From this we have important results. then we say x and s are complementary to each other. Thus. xT s = 0 implies that xj sj = 0 for all j = 1.1) which is a system of linear inequalities and equations.14 CHAPTER 1. . . x∗ is optimal for (LP) if and only if the following conditions hold: i) x∗ ∈ Fp . The above theorems show that if a pair of feasible solutions can be found to the primal and dual problems with equal objective values.  = −c (1. Equations in (1. For feasible x and (y. then the other is either unbounded or has no feasible solution.” From this condition. s) ∈ (R+ . s) is optimal. xT s = xT (c − AT y) = cT x − bT y is called the complementarity gap. We call cT x − bT y the duality gap. It give a unique partition of the LP variables in terms of complementarity.12 (LP duality theorem) If (LP) and (LD) both have feasible solutions then both problems have optimal solutions and the optimal objective values of the objective functions are equal. one equation plus nonnegativity are transformed into n equations. The following theorem plays an important role in analyzing LP interiorpoint algorithms. Since both x and s are nonnegative. there is no “gap. (1.1) become Dx s = Ax = −AT y − s = 0 b −c. the solution set for (LP) and (LD) is  cT x − bT y  n m n Ax Sp = (x.2) This system has total 2n + m unknowns and 2n + m equations including n nonlinear equations. INTRODUCTION AND PRELIMINARIES This theorem shows that a feasible solution to either problem yields a bound on the value of the other problem. Theorem 1.

An optimal basic solution is an optimal vertex of the feasible region. then x is called an optimal basic solution and AB an optimal basis. except for sB .1. then x is called a basic feasible solution. are non-zero. from A. Note that in a nondegenerate basic solution the basic variables and the basis can be immediately identiﬁed from the nonzero components of the basic solution. A basic feasible solution is a vertex on the boundary of the feasible region. the pair of P ∗ and Z ∗ is called the (strict) complementarity partition. Then matrix AB is nonsingular and we may uniquely solve AB xB = b for the m-vector xB .14 (LP fundamental theorem) Given (LP) and (LD) where A has full row rank m. that is s = c − AT y ≥ 0. If the dual solution is also feasible. that basic solution x is said to be (primal) degenerate. Select m linearly independent columns. . If one or more components in xB has value zero.3. we obtain a solution x such that Ax = b. Given (LP) or (LD). {x : AP ∗ xP ∗ = b. denoted by the index set B. SOME BASIC DECISION AND OPTIMIZATION PROBLEMS 15 Theorem 1. xP ∗ ≥ 0. Theorem 1. cP ∗ − AT ∗ y = 0} is called the dual Z P optimal face. xN . x is said to be a (primal) basic solution to (LP) with respect to the basis AB . By setting the variables. The components of xB are called basic variables.13 (Strict complementarity theorem) If (LP) and (LD) both have feasible solutions then both problems have a pair of strictly complementary solutions x∗ ≥ 0 and s∗ ≥ 0 meaning X ∗ s∗ = 0 and x∗ + s∗ > 0. and {y : cZ ∗ − AT ∗ y ≥ 0. in the corresponding dual slack vector s. Then. A dual vector y satisfying AT y = cB B is said to be the corresponding dual basic solution. If all components. Moreover. then y is said to be (dual) nondegenerate. the supports P ∗ = {j : x∗ > 0} j and Z ∗ = {j : s∗ > 0} j are invariant for all pairs of strictly complementary solutions. xZ ∗ = 0} is called the primal optimal face. AB is called a nondegenerate basis. If both primal and dual basic solutions are nondegenerate. of x corresponding to the remaining columns of A equal to zero. If a basic solution x ≥ 0. sN .

the quadratic programming (QP) problem is the following optimization problem: (QP ) minimize subject to q(x) := (1/2)xT Qx + cT x Ax = b. in such a way as to continuously decrease the value of the objective function until a minimizer is reached. c ∈ Rn } and Sp = {x ∈ Fp : q(x) ≤ q(y) for every y ∈ Fp }. x ≥ 0. (1. In contrast. s ≥ 0. ii) if there is an optimal solution. A feasible point x∗ is called a KKT point. A ∈ Rm×n . The simplex method is to proceed from one basic feasible solution (an extreme point of the feasible region) to an adjacent one. y ∗ . interior-point algorithms will move in the interior of the feasible region and reduce the value of the objective function.16 CHAPTER 1. We may denote the feasible set by Fp . A ∈ Rm×n .3. By expanding upon this result. y. Similar to LP. s) ∈ (Rn . y) := bT y − (1/2)xT Qx AT y + s − Qx = c. x. Rm . a ﬁnite search procedure. b ∈ Rm . is derived. The data and solution sets for (QP) are Zp = {Q ∈ Rn×n . if the following KKT conditions hold: there exists (y ∗ ∈ Rm . there is an optimal basic solution. b ∈ Rm and c ∈ Rn . there is a basic feasible solution. The above theorem reduces the task of solving a linear program to that searching over basic feasible solutions. s∗ ) is feasible for the following dual problem: (QD) maximize subject to d(x. 1. INTRODUCTION AND PRELIMINARIES i) if there is a feasible solution.5 Quadratic programming (QP) Given Q ∈ Rn×n . Rn ) + + and Dx s = 0 Ax = b −AT y + Qx − s = −c. we can write the KKT condition as: (x. the simplex method. . where KKT stands for KarushKuhn-Tucker. this system has total 2n + m unknowns and 2n + m equations including n nonlinear equations. and satisﬁes the complementarity condition (x∗ )T s∗ = (1/2)(x∗ )T Qx∗ + cT x∗ − (bT y ∗ − (1/2)(x∗ )T Qx∗ = 0. s∗ ∈ Rn ) such that (x∗ .3) Again. hoping to by-pass many extreme points on the boundary of the region.

ALGORITHMS AND COMPUTATIONS 17 The above condition is also called the ﬁrst-order necessary condition. If fA (m. the error for an approximate solution as a parameter. ) is independent of and polynomial in m and n. One may also view an approximate solution an exact solution to a problem -near to P with a well-deﬁned measure in the data space. Example 1. such as solving a system of linear equations. approximation algorithms are widely used and accepted in practice. i. Z∈Zp We call this complexity model error-based. each iteration can be performed eﬃciently both sequentially and in parallel. Z). In this case. Let c(A. n. 1. In the real number model. ) is a polynomial in m. /. −.1. Then. multiplying a matrix by a vector. For every instance of problem P. n. with a well-deﬁned measure. ) be the total number of operations of algorithm A for generating an -approximate solution. denoted by ci (A. ∗.e. n. Again. pivoting operation of a matrix. c(A. the solution set for (QP) is characterized by a system of linear inequalities and equations. if fA (m. ) is a polynomial in m. Let us use Ap to denote the collection of all possible algorithm for solving every instance in P. and log(1/ ). Sometimes it is convenient to deﬁne the iteration complexity. Then. For some optimization problems. ) ≤ fA (m. Z. etc. and (1/ ). to problem P. then x∗ is an optimal solution for (QP) if and only if x∗ is a KKT point for (QP). then algorithm A is a polynomial approximation scheme or pseudo-polynomial algorithm . but also that they do not have polynomial approximation schemes. the complexity theory can be applied to prove not only that they cannot be solved in polynomial-time. and comparison on real numbers.11 There is a strongly polynomial algorithm for sorting a vector in . If fA (m. Z). This is the so-called backward analysis model in numerical analysis. Denote it by co (A. then we say algorithm A is a strongly polynomial algorithm. where we assume that each iteration costs a polynomial number (in m and n) of arithmetic operations. an algorithm for solving P either determines that Sp is empty or generates an output x such that x ∈ Sp or x is close to Sp in certain measure. n. we introduce . rank-one updating the inversion of a matrix.. then algorithm A is a polynomial algorithm and problem P is polynomially solvable. The latter x is called an approximate solution. ) := sup c(A. n. the (operation) complexity of an algorithm A ∈ Ap for solving an instance Z ∈ Zp is deﬁned as the total arithmetic operations: +. for every given data Z ∈ Zp . In most iterative algorithms.4. Z. One can see (LP) is a special case of (QP). n. ) for any > 0.4 Algorithms and Computations An algorithm is a list of instructions to solve a problem. In practice. If Q is positive semi-deﬁnite.

. One theory of iterative algorithms. the speed of the convergence of the error reﬂects the speed of the algorithm. It has been widely used by the nonlinear optimization and numerical analysis community as an efﬁciency criterion. Therefore.. we throw away [0. Deﬁnition 1. referred to as local or asymptotic convergence analysis. |rk |p Deﬁnition 1. In many cases. for matrix-vector multiplication. if each iteration of competing algorithms requires the same amount of work. Many algorithms may not be polynomial but work ﬁne in practice.. where f (0) > 0 and f (1) < 0. Let the sequence {rk } converge to zero such that lim sup k→∞ |rk+1 | < ∞. Consider a sequence of real numbers {rk } converging to zero. we throw away (1/2. They generate a sequence of everimproving points x0 .1 . If f (1/2) > 0. |rk |2 . Obviously. in log(1/ ) steps. 1]. this criterion generally provides a qualitative statement: if a problem is polynomial solvable. This convergence rate. Furthermore. the sequence will never exactly reach the solution set. Example 1.4. this criterion does explain practical behavior of iterative algorithms. approaching the solution set. . 1. The method calls the oracle to evaluate f (1/2) (counted as one operation). Each step of the method halves the interval that contains the root. it is ideal to develop an algorithm with both polynomiality and practical eﬃciency. although it may hold locally or asymptotically. and for computing the norm of a vector. x1 . 1/2). the bisection method is a polynomial algorithm. We have to admit that the criterion of polynomiality is somewhat controversial. provides evaluation and comparison of diﬀerent algorithms. INTRODUCTION AND PRELIMINARIES descending or ascending order. we must have an approximate root whose distance to the root is less than . One can deﬁne several notions related to the speed of convergence of such a sequence. . Then we repeat this process on the remaining half interval.. This is because polynomiality is built upon the worst-case analysis. However.. Let the sequence {rk } converge to zero.2 . xk . is concerned with the rate at which the optimality error of the generated sequence converges to zero. For many optimization problems and/or algorithms. Thus. if f (1/2) < 0.12 Consider the bisection method to locate a root of a continuous function f (x) : R → R within interval [0.18 CHAPTER 1.1 Convergence rate Most algorithms are iterative in nature. The order of convergence of {rk } is deﬁned as the supermum of the nonnegative numbers p satisfying 0 ≤ lim sup k→∞ |rk+1 | < ∞. 1]. then the problem is indeed relatively easy to solve regardless of the algorithm used.

the sequence is said to converge linearly to zero with convergence ratio β. As a rule. with convergence ratio β. 1. dk 2 Q This algorithm is superlinearly convergent (in fact. In this sense we might say that the order of convergence is a measure of how good the tail of {rk } is. The ultimate case where β = 0 is referred to as superlinear convergence. Linear convergence is the most important type of convergence behavior. . dk 2 Q and dk+1 = Qxk+1 − θk dk where θk = (dk )T Q(Qxk+1 + c) . we also call linear convergence geometric convergence. Thus.3 .13 Consider the conjugate gradient algorithm for minimizing 2 xT Qx+ c. 1 Example 1. BASIC COMPUTATIONAL PROCEDURES 19 Then. the sequence is said to converge quadratically to zero.5 Basic Computational Procedures There are several basic numerical problems frequently solved by interior-point algorithms.1. the comparison is based on their corresponding convergence ratio—the smaller the ratio. when comparing the relative eﬀectiveness of two competing algorithms both of which produce linearly convergent sequences. It should be noted that the order of convergence is determined only by the properties of the sequence that holds as k → ∞. Let the sequence {rk } converge to zero such that lim sup k→∞ |rk+1 | = β < 1. can be said to have a tail that converges to zero at least as fast as the geometric sequence cβ k for a ﬁxed number c. the faster the algorithm.5. |rk | Then. Large values of p imply the faster convergence of the tail. the method uses iterative formula xk+1 = xk − αk dk where αk = (dk )T (Qxk + c) . it converges in ﬁnite number of steps). Deﬁnition 1. Starting from an x0 ∈ Rn and d0 = Qx0 +c. A linearly convergence sequence.

A x1 x b1 b = . the Gaussian elimination method transforms A into the form U 0 C 0 where U is a nonsingular.5. We may assume a11 = 0 after some row switching. This is called the LU -decomposition. upper-triangular matrix. In matrix form. This whole procedure uses about nm2 arithmetic operations. it is a strong polynomial-time algorithm. recursively. lower-triangular matrix. The last process is called back-substitution.5. Substituting the solution x found into the ﬁrst equation yields a value for x1 . So the problem is reduced to solving a system of linear equations with a symmetric semi-positive deﬁnite matrix. Sometimes. U 0 A=L C 0 . we solve the system of the last m − 1 equations for x . where aij is the component of A in row i and column j. and A is called a Schur complement. 1. INTRODUCTION AND PRELIMINARIES Gaussian elimination method Probably the best-known algorithm for solving a system of linear equations is the Gaussian elimination method. .1 CHAPTER 1. Suppose we want to solve Ax = b. where a11 is called a pivot. Thus. the matrix is transformed further to a form D 0 C 0 where D is a nonsingular. This is a pivot step.20 1.2 Choleski decomposition method Another useful method is to solve the least squares problem: (LS) minimize AT y − c . Then we can subtract appropriate multiples of the ﬁrst equation from the other equations so as to have an equivalent system: a11 0 A1. Now. The theory says that y ∗ minimizes AT y − c if and only if AAT y ∗ = Ac. and L is a nonsingular. diagonal matrix.

1. and y ∗ can be obtained by solving two triangle systems of linear equations. 1. Rarely.3 The Newton method The Newton method is used to solve a system of nonlinear equations: given f (x) : Rn → Rn . where scalar α ≥ 0 is called step-size. where M k is an n × n symmetric matrix. . the method is called the gradient method. is the Jacobian matrix f inverted. where L is a lower-triangular matrix and Λ is a diagonal matrix. which can be carried out in strongly polynomial time. where f is viewed as the gradient vector of a real function. and believed to be the key to their eﬀectiveness. The direction vector dx is called a Newton step. A modiﬁed or quasi Newton method is deﬁned by xk+1 = xk − αM k f (xk ). Thus. The Newton method has a superior asymptotic convergence order equal 2 for f (xk ) . the problem is to solve n equations for n unknowns such that f (x) = 0.5. In particular. In matrix form. The idea behind Newton’s method is to use the Taylor linear approximation at the current iterate xk and let the approximation be zero: f (x) f (xk ) + f (xk )(x − xk ) = 0. (Such a transformation can be done in about nm2 arithmetic operations as indicated in the preceding section. It is frequently used in interior-point algorithms. Generally the system of linear equations f (xk )dx = −f (xk ) is solved and xk+1 = xk + αdx is used. the method transforms AAT into the form AAT = LΛLT .) L is called the Choleski factor of AAT . BASIC COMPUTATIONAL PROCEDURES 21 One method is Choleski’s decomposition. if M k = I. the above linear system becomes LΛLT y ∗ = Ac.5. The Newton method is thus deﬁned by the following iterative formula: xk+1 = xk − α( f (xk ))−1 f (xk ). however.

0. and if c − AT y = 0 then x∗ = −(c − AT y)/ c − AT y . µ∗ (1 − y ∗ 2 ) = 0. These conditions are necessary and suﬃcient. and µ∗ ≥ 0. y ¯ otherwise any feasible y is a solution. So these two problems can be reduced to solving a system of linear equations. ¯ and if y = 0 then set ¯ y ∗ = −¯/ AT y .5. or simply This problem is used by the classical trust region method for nonlinear optimization. (BD) minimize (1/2)y T Qy + bT y subject to y 2 ≤ 1. x 2 ≤ 1. y∗ (Q + µ∗ I) 2 ≤ 1. We will discuss them later in the book.22 CHAPTER 1. . In practice.5 Solving ball-constrained quadratic problem The ball-constrained quadratic problem has the following form: (BP ) minimize (1/2)xT Qx + cT x subject to Ax = 0. respectively.5. INTRODUCTION AND PRELIMINARIES 1. The optimality conditions for the minimizer y ∗ of (BD) are (Q + µ∗ I)y ∗ = −b. minimize subject to bT y AT y x 2 2 ≤ 1.4 Solving ball-constrained linear problem The ball-constrained linear problem has the following form: (BP ) or minimize subject to (BD) cT x Ax = 0. The ball-constrained quadratic problem will be used an a sub-problem by several interior-point algorithms in solving complex optimization problems. several trust region procedures have been very eﬀective in solving this problem. The solution y ∗ for (BD) is given as follows: Solve AAT y = b. This problem can be solved in polynomial time log(1/ ) and log(log(1/ )) by the bisection method or a hybrid of the bisection and Newton methods. 1. ≤ 1. otherwise any feasible x is a solution. ∗ x minimizes (BP) if and only if there always exists a y such that they satisfy AAT y = Ac.

and its applications can be seen in Nesterov and Nemirovskii [241]. Kuhn and Tucker [195]. and Dennis and e Schnable [76]. o a u Rockafellar [264]. also see Bland. Ghaoui. Pang and Stone [72]. It was shown by Klee and Minty [184] that the simplex method is not a polynomial-time algorithm.7 Exercises 1. Also see Garey and Johnson [118] and Papadimitriou and Steiglitz [248]. For a brief LP history. Linear programming and the simplex method were introduced by Dantzig [73]. Other inequality problems and convexity theories can be seen in Gritzmann and Klee [141]. The real computation model was developed by Blum. The N P theory was due to Cook [70] and Karp [180]. Many basic numerical procedures listed in this chapter can be found in Golub and Van Loan [133]. The ball-constrained quadratic problem and its solution methods can be seen in Mor´ [229]. see the excellent article by Wright [323]. Other complexity issues in numerical optimization were discussed in Vavasis [318]. and a and b be given Rn vectors. Goemans and Williamson [125] obtained several breakthrough results on approximation algorithms using positive semi-deﬁnite programming. NOTES 1. Show (Q + abT )−1 = Q−1 − 1 1+ bT Q−1 a Q−1 abT Q−1 . the ﬁrst polynomial-time algorithm for linear programming with rational data. Goldfarb and Todd [52]. The positive semi-deﬁnite programming. was proven by Khachiyan [181]. Gr¨nbaum [143]. Murray. The importance of P was observed by Edmonds [88]. Shub and Smale [53] and Nemirovskii and Yudin [239]. and Schrijver [271]. was developed by Karmarkar.1.6. The KKT condition for nonlinear programming was given by Karush. another polynomial-time algorithm for linear programming. and Boyd. Sorenson [281]. The method was devised independently by Shor [277] and by Nemirovskii and Yudin [239]. Various complementarity problems can be found found in Cottle. see Gill. Recently. . Gr¨tschel. Tomlin and Wright [124].6 23 Notes The term “complexity” was introduced by Hartmanis and Stearns [155]. Saunders. The complexity result of the ball-constrained quadratic problem was proved by Vavasis [318] and Ye [332]. Feron and Balakrishnan [56]. It is related to the classical barrier-function method studied by Frisch [109] and Fiacco and McCormick [104]. and Anstreicher [21]. an optimization problem in nonpolyhedral cones.1 Let Q ∈ Rn×n be a given nonsingular matrix. Lov´sz and Schrijver [142]. 1. The interior-point method. The ellipsoid method. This formula is called the Sherman-Morrison-Woodbury formula. Alizadeh [8].

i = 1. 1..7 Using the idea of Exercise 1. (1.. show that Q is PSD if and only if all its eigenvalues are non-negative.2. consider the function n B(y) = − j=1 log sj where s = c − AT y > 0.24 CHAPTER 1. 0). and Q is PD if and only if all its eigenvalues are positive. 1.4 Show that the biggest coordinate-aligned ellipsoid that is entirely contained ◦ in Rn and has its center at xa ∈Rn can be written as: + + E(xa ) = {x ∈ Rn : (X a )−1 (x − xa ) ≤ 1}. 2). The 2-dimensional ellipsoid centered at (1. 1..5 Show that the non-negative orthant. . and b ∈ Rm . 2) that passes the points (0. . 1). Find a separating hyperplane vector as a function of y. 1.. .6.3 and 1. the positive semi-deﬁnite cone. The 2-dimensional ellipsoid centered at (1.3 Using the ellipsoid representation in Section 1. y2 . 2. 3)... 1.10 Prove Propositions 1.. • Given C ∈ Mn . . ﬁnd the matrix Q and vector y that describes the following ellipsoids: 1. prove the separating hyperplane theorem 1.2. 4). 1. 2). y2 . 1. The best way to do this is to use the deﬁnition of the partial derivative f (y)i = lim δ→0 f (y1 . (0. 2. (3.8 • Given an m × n matrix A and a vector c ∈ Rn .9 Prove that the level set of a quasi-convex function is convex.4 for convex functions in Section 1. and (2. Furthermore. and passing through (−1. m. consider the m function B(y) := − log det(S). INTRODUCTION AND PRELIMINARIES 1. ym ) − f (y1 .2. 4). . yi + δ. ym ) . yi . 1.3.1. Ai ∈ Mn . where S = C − and 2 yi Ai 0. Find B(y) and 2 B(y) in terms of s. 0). Assuming y ∈ C. (2.. The 3-dimensional sphere of radius 2 centered at the origin. Find B(y) i=1 B(y) in terms of S.2 Prove that the eigenvalues of all matrices Q ∈ Mn×n are real.. δ 1. 3. and the second-order cone are all self-dual.6 Consider the convex set C = {x ∈ R2 : (x1 − 1)2 + (x2 − 1)2 ≤ 1} and let y ∈ R2 . · · · . 2) with axes parallel to the line y = x and y = −x. Find the point in C that is closest to y... and (1..

1. 1.12 Prove the Harmonic inequality described in Section 1. EXERCISES 25 ¯ 1.23 Let both (LP) and (LD) for a given data set (A.. .11 Let f1 . 1. 2. Prove that Ω(z) is bounded and has an interior for any ﬁnite z < z ∗ .21 Prove X • S ≥ 0 if both X and S are positive semi-deﬁnite matrices. 1.4.1.13 is P ∗ = B.15 Let P = AT (AAT )−1 A or P = I − AT (AAT )−1 A. 1.. Then.. The eigenvalues of P are either 0 or 1. Then prove 1. .m • m fi (x) i=1 1. X • S = 0.22 Prove that two positive semi-deﬁnite matrices are complementary to each other. P = P 2 . prove that the strict complementarity partition in Theorem 1.2.8 and 1. prove Farkas’ lemmas 1.16 Using the separating theorem.9.7.7 for linear equations.14. 1. prove that x∗ is an optimal solution for (QP) if and only if x∗ is a KKT point for (QP). 1.19 If (LP) and (LD) have a nondegenerate optimal basis AB . Then consider the level set Ω(z) = {y : c − AT y ≥ 0. P is positive semi-deﬁnite. 3. fm be convex functions. . 1.18 Prove the LP fundamental theorem 1.17 If a system AT y ≤ c of linear inequalities in m variables has no solution. .13 Prove Farkas’ lemma 1. c) have interior feasible points. b.. 1. even Fd is unbounded. −z + bT y ≥ 0} where z < z ∗ and z ∗ designates the optimal objective value. 1. 1.20 If Q is positive semi-deﬁnite. if and only if XS = 0. the function f (x) deﬁned below is also convex: • max fi (x) i=1.14 Prove the linear least-squares problem always has a solution. show that AT y ≤ c has a subsystem (A )T y ≤ c of at most m + 1 inequalities having no solution..

y ∗ ≤ 1. write the systems of linear equations used to calculate the Newton steps for ﬁnding points that satisfy the optimality equations (1.26 CHAPTER 1. µ∗ ≥ 0.3). . 1. 1. ﬁnd the feasible direction dx (Adx = 0) that achieves the steepest decrease in the objective function. Rn ) + + for the primal and dual.25 Given an (LP) data set (A.2) and (1. respectively. b.24 Given an (LP) data set (A. and are necessary and suﬃcient.26 Show the optimality conditions for the minimizer y ∗ of (BD) in Section 1. (Q + µ∗ I) 0. µ∗ (1 − y ∗ ) = 0. b. s0 ) ∈ (Rn . c) and a feasible point (x0 . INTRODUCTION AND PRELIMINARIES 1. y 0 . Rm .5: (Q + µ∗ I)y ∗ = −b.5. and ignoring the nonnegativity condition. c) and an interior feasible point x0 .

The problem of minimizing the max-eigenvalue of P (y) can be cast as a (SDD) problem... . . i = 1. we minimize a linear function of a matrix in the positive semi-deﬁnite matrix cone subject to aﬃne constraints. m Example 2. Recall that the • operation is the matrix inner product A • B := trAT B. i = 1. Here y ∈ Rm and S ∈ Mn . S 0) satisﬁes all equations in (SDD). where C and Ai .. and b ∈ Rm . The dual problem to (SDP) can be written as: (SDD) sup bT y m subject to i yi Ai + S = C. which is analogous to the dual (LD) of LP. 2... Ai ∈ Mn . In semi-deﬁnite programming. In contrast to the positive orthant cone of linear programming. The notation X 0 means that X is a positive semi-deﬁnite matrix.Chapter 2 Semideﬁnite Programming 2. .. 2. .1 Semi-deﬁnite programming (SDP) Given C ∈ Mn . .1 Let P (y ∈ Rm ) = −C + i yi Ai . S 0. . X 0. m. . it is called a dual interior feasible solution.0. If a point X 0 and satisﬁes all equations in (SDP). the positive semi-deﬁnite 27 . i = 1. the semi-deﬁnite programming problem is to ﬁnd a matrix X ∈ Mn for the optimization problem: (SDP ) inf subject to C •X Ai • X = bi . are given symmetric matrices. m. If a point (y. it is called a (primal) strictly or interior feasible solution. and X 0 means that X is a positive deﬁnite matrix. m.

an X 0. i = 1.28 CHAPTER 2. m. We have several theorems analogous to Farkas’ lemma. there exists a symmetric matrix X 0 with Ai • X = bi . . m. Similary.2 Consider A1 = 1 0 0 0 ... A2 • X = 2.. and ﬁnds many applications in engineering. m i i = 1.. . SEMIDEFINITE PROGRAMMING matrix cone is non-polyhedral (or “non-linear”).. . m. i yi Ai C if and only if X 0. with AT y ≤ 0 and T b y = 1. X 0} is not a closed set. and combinatorial optimization. . a vector y. m. and convex quadratic minimization with convex quadratic constraints. But this does not hold for matrix equations in the positive semi-deﬁnite matrix cone. .. control.. X = 0. i = 1. Note the diﬀerence between the LP and SDP. quadratic programming. have rank m m m (i.. such as linear programming. 2. and C • X ≤ 0 proves that yi Ai C is impossible. have rank m m (i..e. Semi-deﬁnite programming uniﬁes several standard problems. x ≥ 0}. Then. i = 1. + The problem is that {y : yi = Ai • X. Example 2.. .. Ai • X = 0. i = 1. implies C • X > 0.. X = 0 and Ai • X = 0. always exists and is called a (primal) infeasibility certiﬁcate for the system {x : Ax = b..e. if and only if m i yi Ai m i 0 and yi Ai = 0 implies bT y < 0... So positive semideﬁnite programs are convex optimization problems.2 (Farkas’ lemma in SDP) Let Ai ∈ Mn .. i yi Ai = 0 implies y = 0) and b ∈ Rm . but convex. i = 1. Corollary 2... m. 0 1 1 0 C= and A1 = 1 0 0 0 makes C A1 y1 infeasible but it does not have an infeasible certiﬁcate..1 (Farkas’ lemma in SDP) Let Ai ∈ Mn . In other words. m. i yi Ai = 0 implies y = 0) and C ∈ Mn . and b= A2 = 0 1 1 0 0 2 where we have the following matrix system A1 • X = 0. Theorem 2. . Then. X ∈ M2 .. From Farkas’ lemma for linear programming. The weak duality theorem for SDP is identical to that of (LP) and (LD).

iii) C • X = bT y or X • S = 0.5 (SDP duality theorem) If one of (SDP) or (SDD) has a strictly or interior feasible solution and its optimal value is ﬁnite. . It has turned out that most interiorpoint methods for LP have been generalized to semi-deﬁnite programs. S) ∈ Fd : XS = 0}. the feasible sets for the primal and dual. Example 2. these algorithms possess polynomial worst-case complexity under certain computation models. ii) there is (y. A1 =  0 0 0 and b= a duality gap:   0 0 0 1 0  . X is optimal for (PS) if and only if the following conditions hold: i) X ∈ Fp . In general. S) ∈ Fd . which is a system of linear matrix inequalities and equations. C • X ≥ bT y where X ∈ Fp . then the other is feasible and has the same optimal value. (y. This is in contrast to (LP) and (LD) where no duality gap exists if both are feasible. they are not much harder to solve. Note that a duality gap may exists if neither (SDP) nor (SDD) has a strictly feasible point.22). A1 =  −1 0 0 0 0 10  −1 0 0 0  0 2 . (y. If one of (SDP) or (SDD) is unbounded then the other has no feasible solution. Again note the diﬀerence between the above theorem and the strong duality theorem for LP.3 The  0 1 C= 1 0 0 0 following SDP has   0 0 0  . As in LP. Although semi-deﬁnite programs are much more general than linear programs. we have Theorem 2. S) ∈ Fd : C • X − bT y = 0}.29 Corollary 2. Two positive semi-deﬁnite matrices are complementary to each other. We will describe such extensions later in this book. if and only if XS = 0 (Exercise 1.4 (Strong duality theorem in SDP) Let Fp and Fd be non-empty and at least one of them has an interior. S) ∈ Fd . They also perform well in practice. But we need more to make the strong duality theorem hold. Theorem 2. Then. X•S = 0. or Sp = {X ∈ Fp . Then. be non-empty. From the optimality conditions.3 (Weak duality theorem in SDP) Let Fp and Fd . (y. the solution set for certain (SDP) and (SDD) is Sp = {X ∈ Fp .

Let A = (1.. Then the set of Ω is also the interval [−1. 1]. n n log(cj − aT y) = . 1). the analytic center (y a . j=1 where a. Thus. (2. Ω) = max log d(y. Note that d(−1/2) = (3/2)(1/2) = 3/4 and B(−1/2) = log(3/4). −1. i. j d(y) = y ∈ Ω. Ω = {y ∈ Rm : c − AT y ≥ 0}. SEMIDEFINITE PROGRAMMING 2. (2. denoted by y a and sa = c − AT y a .4 Let A = (1.1 2. −1) and c = (1. in what follows we may write B(s) to replace B(y) where s is always equal to c − AT y. Ω). Example 2. Then the set of Ω is the interval [−1.30 CHAPTER 2. since the potential function is strictly concave in a ◦ bounded convex Ω. we let s := c − AT y and call it a slack vector. and d (−1/2) = (3/)(1/2)(1/2) = 3/8 and B (−1/2) = log(3/8). where A ∈ Rm×n and c ∈ Rn are given and A has rank m. y∈Ω (y a . . the function is the product of all slack variables. i. Denote the interior of Ω by ◦ m T Ω= {y ∈ R : c − A y > 0}.1 Analytic Center AC for polytope Let Ω be a bounded polytope in Rm represented by n (> m) linear inequalities. sa ) is uniquely deﬁned.2) Note that adding or deleting a redundant inequality changes the location of the analytic center.1. Ω) = 0 and letting xa = (S a )−1 e. 1. sa ) together with xa satisfy the following optimality conditions: Xs = Ax = −AT y − s = e 0 −c. Traditionally.. in Ω that maximizes the potential function is called the analytic center of Ω.1) j=1 and −B(y) is the classical logarithmic barrier function. Deﬁne n (cj − aT y). B(Ω) := B(y a .j B(y) := log d(y) = j=1 log sj .e. 1].j is the jth column of A. Its logarithm is called the (dual) potential function. The interior point.e. Setting B(y. For convenience. −1) and c = (1. 1).

−y ≤ 0. Thus. The analytic center is y a = 1/2 with xa = (2. eT x = n. the interior of Ω is not used in the sense that the topological interior for a set is used. which is called Karmarkar’s canonical set. The analytic center for this system is y a = n/(n + 1) with xa = ((n + 1)/n. m Ω = {y ∈ Rm : C − yi Ai 0. i. y ≤ 1}. Rather. y ≤ 1}. in Ω that maximizes the potential function is called the analytic center of Ω. i. such as Ω = {y ∈ Rm : c − AT y ≥ 0.2. Example 2. Consider n times Ω = {y ∈ R : −y ≤ 0. Thus. we still say Ω is not empty. + ◦ Again Rn := {s ∈ Rn : s > 0}. + When say Ω has an interior. x ≥ 0}.e.. · · · . which is interval [0.e. denoted by y a and S a = C − i yi Ai . which is. again.1.. 2)T . · · · . interval [0. (n + 1))T . x ≥ 0} and the hyperplane {x : Ax = 0} (Figure 2. Then the analytic center is chosen on the hyperplane {y : By = b} to maximize the product of the slack variables s = c − AT y. y∈Ω . (n + 1)/n. The analytic center can be deﬁned when the interior is empty or equalities are presented. i Let S = C − m i yi Ai and m B(y) := log det(S)) = log det(C − yi Ai ). By = b}. (2.3) i m a The interior point.6 Consider the system Ω = {x : Ax = 0.1).e.2 AC for SDP Let Ω be a bounded convex set in Rm represented by n (> m) a matrix inequality.5 Consider Ω = {y ∈ R : −y ≤ 0. 2. }. it refers to the interior of the positive orthant of slack variables: Rn := {s : s ≥ 0}. we mean that ◦ Rn ∩{s : s = c − AT y for some y where By = b} = ∅. ANALYTIC CENTER 31 Example 2. 1] but “−y ≤ 0” is copied n times. i. If x = e is in Ω then e is the analytic center of Ω. the intersection of the simplex {x : eT x = n. 1].. the interior of the orthant Rn . max B(y). if Ω + + + ◦ has only a single point y with s = c − AT y > 0.1.

4) Potential Functions for LP and SDP We show how potential functions can be deﬁned to solve linear programming problems and semi-deﬁnite programming. Denote the feasible sets of (LP) and (LD) by Fp and Fd . and the interior of F by F . c). SEMIDEFINITE PROGRAMMING x3 (0. ◦ respectively. Ω(z2 ) ⊂ Ω(z1 ) and the inequality −z + bT y is translated from z = z1 to z = z2 . Ω(z) is bounded and has an interior for any ﬁnite z. Ω(z) ⊂ Ω.3.1.23). both the primal and dual have interior feasible point. b.3) Ax=0 .2. Since both (LP) and (LD) have interior feasible point for given (A. even Ω := Fd is unbounded (Exercise 1.0. −z + bT y ≥ 0}.5) where z < z ∗ . 2. We assume that for a given LP data set (A.0) (0. c). the analytic center (y a .32 CHAPTER 2. (y a . . S a ) together with X a satisfy the following optimality conditions: XS AX −AT y − S 2. since the potential function is strictly concave in a ◦ bounded convex Ω. We also let z ∗ be the optimal value of the standard form (LP) and (LD). and if z2 ≥ z1 .0.1 Primal potential function for LP Consider the level set Ω(z) = {y ∈ Rm : c − AT y ≥ 0. Clearly. b. (2. (2. S a ) is uniquely deﬁned. Denote by F = Fp × Fd .1) (3. Ω) = 0 and letting X a = (S a )−1 .2 = I = 0 = −C.0) x2 Figure 2.  ¡ x1 (1.1: Illustration of the Karmarkar (simplex) polytope and its analytic center. Setting B(y.

Pn+1 (x. If the objective hyperplane is repeatedly translated to the analytic center. b T y = bT y a ya ya bT y = z The objective hyperplane The updated objective hyperplane Figure 2.. the primal potential function for Ω(z) (Figure 2. is n P(x .2.e. (x . ﬁnding a point in Ω(z) has a homogeneous primal problem minimize cT x − zx0 s.2. z). (x . For (x . Ω(z)) = (n + 1) log(cT x − zx0 ) − log xj j=0 n = (n + 1) log(cT x − z) − log xj =: Pn+1 (x.2: Intersections of a dual feasible region and the objective hyperplane. x0 ) ≥ 0. Then.2). i.2. j=1 The latter. x0 ) > 0. is the Karmarkar potential function in the standard LP form with a lower bound z for z ∗ . POTENTIAL FUNCTIONS FOR LP AND SDP 33 From the duality theorem again. as described in the preceding section. ◦ let x := x /x0 ∈F p . bT y ≥ z on the left and bT y ≥ bT y a on the right. Ax = b. x0 ) satisfying Ax − bx0 = 0.t. Ax − bx0 = 0. the sequence of new . x > 0. z). One algorithm for solving (LD) is suggested in Figure 2.

The function n Pn+ρ (x. but it changes the location of its analytic center. x0 ) be the primal variables. symmetric to the primal. · · · .8) . z) = (n + ρ) log(z − bT y) − log sj . As we illustrated before. Ω(z)) = (n + ρ) log(cT x − z(ρx0 )) − log xj − ρ log x0 j=1 n = (n + ρ) log(cT x − z) − log xj + ρ log ρ j=1 =: Pn+ρ (x.t. Then.34 CHAPTER 2. It represents the volume of a coordinate-aligned ellipsoid whose intersection with AΩ(z) contains SΩ(z) . Geometrically. the primal potential function for the new Ω(z) given by (2. −z + bT y ≥ 0}. Ax −bx0 − · · · − bx0 = 0.2 Dual potential function for LP We can also develop a dual potential function. x0 ) ≥ 0. (2. Then the primal problem can be written as ρ times minimize T c x −zx0 − · · · − zx0 ρ times s.7) j=1 is an extension of the Karmarkar potential function in the standard LP form with a lower bound z for z ∗ . “−z + bT y ≥ 0” is copied ρ times. Let (x . s.2. z) + ρ log ρ. for ◦ (y. −z + bT y ≥ 0. one can represent Ω(z) diﬀerently: ρ times Ω(z) = {y : c − AT y ≥ 0. ◦ Let x = x /(ρx0 ) ∈F p . 2.6) i.e. Since the last ρ inequalities in Ω(z) are identical. this representation does not change Ω(z). z) = (n + ρ) log(cT x − z) − log xj (2.6) is n P(x. j=1 (2. SEMIDEFINITE PROGRAMMING analytic centers will converge to an optimal solution and the potentials of the new polytopes will decrease to −∞. s) ∈F d n Bn+ρ (y. where “−z + bT y ≥ 0” is duplicated ρ times. (x .. they must share the same slack value and the same corresponding primal variable.

6 Deﬁne the level set ◦ Ψ(δ) := {(x.2. (2. where “cT x − z ≤ 0” is copied ρ times (Exercise 2. cT x) − log xj . s) ∈F d it is deﬁned by n ψn+ρ (x. 2. for ρ > 0. s) ∈F : ψn+ρ (x. s. bT y) − log sj j=1 n = Bn+ρ (s. we may write Bn+ρ (y. s) → −∞ implies that xT s → 0.9) j=1 where ρ ≥ 0. . s) ≤ δ}. s) ≥ ρ log(xT s) + n log n. we have ψn+ρ (x. POTENTIAL FUNCTIONS FOR LP AND SDP 35 where z is a upper bound of z ∗ . More precisely.3 Primal-dual potential function for LP A primal-dual potential function for linear programming will be used later. One can show that it represents the volume of a coordinate-aligned ellipsoid whose intersection with the aﬃne set {x : Ax = b} contains the primal level set ρ times T {x ∈ Fp : c x − z ≤ 0. ψn+ρ (x. then.2.1). y. z) simply by Bn+ρ (s. z). j=1 Since ψn+ρ (x.2. s) := (n + ρ) log(xT s) − log(xj sj ). For symmetry. We have the relation: n ψn+ρ (x. i) Ψ(δ 1 ) ⊂ Ψ(δ 2 ) if δ1 ≤ δ2 . For ◦ ◦ x ∈F p and (y. cT x − z ≤ 0}. ρ We have the following theorem: Theorem 2. s) = ρ log(xT s) + ψn (x. s) − n log n xT s ≤ exp( ). · · · . s) = (n + ρ) log(cT x − bT y) − n log xj − j=1 log sj j=1 n = Pn+ρ (x. since we can always recover y from s using equation AT y = c − s.

y k . we assume that both (SDP) and (SDD) have interior feasible ◦ ◦ points. S) ∈F d the primal-dual potential function for SDP is deﬁned by ψn+ρ (X. z) := (n + ρ) log(C • X − z) − log det(X). where ρ ≥ 0 and z ∗ designates the optimal objective value.. s) ∈ F : ψn+ρ (x. ◦ ◦ For X ∈F p and (y. these deﬁnitions reduce to those for LP. sk+1 ) ≤ ψn+√n (xk . S) = ρ log(X • S) + ψn (X. we have ψn+ρ (X.. Ψ(δ) is bounded and its closure Ψ(δ) has non-empty intersection with the solution set. S) ∈F d . This indicates that the level sets shrink at least a constant rate independently of m or n. y k+1 . S) → −∞ implies that X • S → 0. for any X ∈F p and (y. 2. ψn+ρ (X. s) < δ}. S) ≥ ρ log(X • S) + n log n. S) − n log n X • S ≤ exp( ). for ρ > 0. Note that if X and S are diagonal matrices. z) := (n + ρ) log(z − bT y) − log det(S). z ≥ z∗.2) ψn+ρ (X.4 Potential function for SDP The potential functions for SDP of Section 2. the primal potential function is deﬁned by Pn+ρ (X. Then. ρ We also have the following corollary: .36 CHAPTER 2. sk ) − . S. sk } ∈F such that ψn+√n (xk+1 . y k . C • X) − log det(X). ˆ iii) For every δ. Note that we still have (Exercise 2. the dual potential function is deﬁned by Bn+ρ (y.1 are analogous to those for LP. bT y) − log det(S) Bn+ρ (S. .05 for k = 0. 2.0. S) := = = = (n + ρ) log(X • S) − log(det(X) · det(S)) (n + ρ) log(C • X − bT y) − log det(X) − log det(S) Pn+ρ (X. For given data. Later we will show that a potential reduction algorithm generates sequences ◦ k {x . z ≤ z∗. Then. where ρ ≥ 0. SEMIDEFINITE PROGRAMMING ii) ◦ Ψ (δ) = {(x.. y. More precisely. 1.2.

for some y ∈ Rm it satisﬁes the optimality conditions Xs = Ax = −AT y − s = µe b −c. y.e.. For any µ > 0 one can derive the central path simply by minimizing the primal LP with a logarithmic barrier function: (P ) minimize s. Then. x ≥ 0. (2.7 Let (SDP) and (SDD) have non-empty interior and deﬁne the level set ◦ Ψ(δ) := {(X. i. The central path can be expressed as C= ◦ (x.3.t. Assume that ◦ ◦ ◦ F = ∅. S) ≤ δ}. Ψ(δ) is bounded and its closure Ψ(δ) has non-empty intersection with the solution set. S) < δ}. both F p = ∅ and F d = ∅. ii) ◦ Ψ (δ) = {(X. ˆ iii) For every δ. s) = n log n . 2. ◦ Let x(µ) ∈F p be the (unique) minimizer of (P). y. i) Ψ(δ 1 ) ⊂ Ψ(δ 2 ) if δ1 ≤ δ2 .2. s) ∈F : Xs = xT s e n in the primal-dual form.3 Central Paths of LP and SDP Many interior-point algorithms ﬁnd a sequence of feasible points along a “central” path that connects the analytic center and the solution set. We now present this one of the most important foundations for the development of interior-point algorithms. 2. n cT x − µ j=1 log xj Ax = b. S) ∈ F : ψn+ρ (X. y.10) .3. s) ∈F : ψn (x. and denote z ∗ the optimal objective value. We also see ◦ C = (x. CENTRAL PATHS OF LP AND SDP 37 Corollary 2. S) ∈F : ψn+ρ (X.1 Central path for LP Consider a linear program in the standard form (LP) and (LD). y.

s(z)) exists and is uniquely deﬁned. Another way to derive the central path is to consider again the dual level set Ω(z) of (2. c). and x0 (z)(bT y − z) = 1. b. s(z)) of Ω(z) and a unique point (x (z).5) for any z < z ∗ (Figure 2. the point (x(z).3). s ≥ 0. y(z). n bT y + µ j=1 log sj AT y + s = c. As we proved earlier in Section 2.8 Let both (LP) and (LD) have interior feasible points for the given data set (A. The following theorem further characterizes the central path and utilizes it to solving linear programs. . (x(z). s(µ)) ∈F d be the (unique) minimizer of (D). the central path point (x(µ).t.3: The central path of y(z) in a dual feasible region. x0 (z)) satisﬁes Ax (z) − bx0 (z) = 0. X (z)s = e. ya The objective hyperplanes Figure 2. Let x(z) = x (z)/x0 (z). y(z). which imply the following theorem: Theorem 2. Then. the analytic center (y(z). for some x ∈ Rn it satisﬁes the optimality conditions (2. then we have Ax(z) = b. s(z)) is on the central path with µ = bT y(z) − z and cT x(z) − bT y(z) = x(z)T s(z) = n(bT y(z) − z) = nµ. s(µ)) exists and is unique.10) as well.38 CHAPTER 2. X(z)s(z) = e/x0 (z) = (bT y(z) − z)e. Then. Then for any 0 < µ < ∞. SEMIDEFINITE PROGRAMMING Consider minimizing the dual LP with the barrier function: (D) maximize s. Thus. ◦ Let (y(µ).2. y(µ). both minimizers x(µ) and (y(µ). s(µ)) are on the central path with x(µ)T s(µ) = nµ. s = c − AT y. Thus.

s(µ)) be on the central path. which proves (i). j j j or j∈P ∗ x∗ j x(µ)j + j∈Z ∗ s∗ j s(µ)j Thus. Let x∗ ∗ (x∗ ∗ = 0) and s∗ ∗ (s∗ ∗ = 0)..3. and iii) (x(µ). and the limit point s(0)Z ∗ is the analytic center on the dual optimal face. i) The central path point (x(µ). ii) For 0 < µ < µ. x(µ) and s(µ) are bounded. We leave the proof of (ii) as an exercise. cP ∗ − AT ∗ y = 0}. 2. Moreover. CENTRAL PATHS OF LP AND SDP 39 Theorem 2. P Z Z P respectively. s(µ)) is bounded for 0 < µ ≤ µ0 and any given 0 < µ0 < ∞. n}. Thus. Z ∗ ) is the strict complementarity partition of the index set {1..2. we have P n s∗ x(µ)j + x∗ s(µ)j = nµ. Note that (x(µ0 ) − x(µ))T (s(µ0 ) − s(µ)) = 0. . Again. Since x(µ) and s(µ) are both bounded. we have x(µ)j ≥ x∗ /n > 0. Proof.. they have at least one limit point which we denote by x(0) and s(0). j = n. j or n j x(µ)j s(µ)j + x(µ0 )j s(µ0 )j ≤ 2n. y(µ). where (P ∗ . j ∈ P ∗ j and s(µ)j ≥ s∗ /n > 0. .9 Let (x(µ). This can be rewritten as n s(µ0 )j x(µ)j + x(µ0 )j s(µ)j = n(µ0 + µ) ≤ 2nµ0 . xP ∗ ≥ 0} and {sZ ∗ : sZ ∗ = cZ ∗ − AT ∗ y ≥ Z 0. cT x(µ ) < cT x(µ) bT y(µ ) > bT y(µ). be the unique analytic centers on the primal and dual optimal faces: {xP ∗ : AP ∗ xP ∗ = b. j ∈ Z ∗ . s(µ)) converges to an optimal solution pair for (LP) and (LD). the limit point x(0)P ∗ is the analytic center on the primal optimal face. since (x(µ0 ) −x(µ)) ∈ N (A) and (s(µ0 ) −s(µ)) ∈ R(AT ).

40 CHAPTER 2. j∈Z ∗ However. min(x))|. Furthermore. We have the following theorem: xT s n . we must have        j∈P ∗ Therefore. j∈Z ∗ and s(0)Z ∗ = s∗ ∗ . can be any norm. We usually deﬁne a neighborhood of the central path as N (η) = ◦ (x. Z since x∗ ∗ and s∗ ∗ are the unique maximizer pair of the potential function. ( j∈P ∗ x∗ )( j∈Z ∗ s∗ ) is the maximal value of the potential function j j over all interior point pairs on the optimal faces. and x(0)P ∗ and s(0)Z ∗ is one interior point pair on the optimal face. s∗  =  j x∗   j x(0)j   j∈P ∗ j∈Z ∗ x(0)P ∗ = x∗ ∗ P s(0)j  . This P Z also implies that the limit point of the central path is unique. or even a one-sided “norm” as x −∞ = | min(0. s) ∈F : Xs − µe ≤ ηµ and µ = where . j ∈ Z ∗ and s(µ)j → 0. y.   j∈P ∗ or  j∈Z ∗    x∗ j  x(µ)j x∗   j j∈P ∗    s∗  ≤  j j∈Z ∗  s∗ j  ≤1 s(µ)j  x(µ)j   j∈P ∗ s(µ)j  . j ∈ P ∗ . SEMIDEFINITE PROGRAMMING This implies that x(µ)j → 0. Thus. .

ii) For 0 < µ < µ. We also have the following corollary: Corollary 2.11 Let both (SDP) and (SDD) have interior feasible points. S(µ)) converges to an optimal solution pair for (SDP) and (SDD). Moreover. Moreover. C • X(µ ) < C • X(µ) and bT y(µ ) > bT y(µ). S) = n log n .2 Central path for SDP ◦ Consider a SDP problem in Section 2. S) ∈F : ψn (X. S(µ)) is bounded where 0 < µ ≤ µ0 for any given 0 < µ0 < ∞.5 = X. y.. y. X . + + We also see ◦ C = (X. S) ∈F : X . i. n where x∗ is any optimal primal solution. y. or a symmetric form ◦ C = (X. i) the central path point (X(µ). 2. i. this deﬁnition is identical to LP. iii) (X(µ). for any j ∈ P ∗ xj ≥ (1 − η)x∗ j . Then for any 0 < µ < ∞. and the rank of the limit of X(µ) is maximal among all optimal solutions of (SDP) and the rank of the limit S(µ) is maximal among all optimal solutions of (SDD).5 ∈ Mn is the “square root” matrix of X ∈ Mn .2.0. The central path can be expressed as ◦ C = (X. 0 < µ < ∞ . 0 < µ < ∞ .e. s) : xT s ≤ nµ0 } is bounded for any given µ0 < ∞. . n where s∗ is any optimal dual solution.3.10 Let (x. both ◦ ◦ F p = ∅ and F d = ∅. i) The N (η) ∩ {(x. s) ∈ N (η) for constant 0 < η < 1. where X . the central path point (X(µ). y. S(µ)) exists and is unique.e. When X and S are diagonal matrices. y. S) ∈F : XS = µI. for any j ∈ Z ∗ sj ≥ (1 − η)s∗ j ..5 = µI. CENTRAL PATHS OF LP AND SDP 41 Theorem 2. ii) Any limit point of N (η) as µ → 0 is an optimal solution pair for (LP) and (LD).5 SX . y(µ).5 X .3.1 and Assume that F = ∅.

SEMIDEFINITE PROGRAMMING Notes The SDP with a duality gap was constructed by Freund. y. Here we assume that X(µ) = X(µ ) and y(µ) = y(mu ). z). s. where z is a upper bound of z ∗ . s) ≤ δ}.42 2. Then prove ψn (X.9.4 CHAPTER 2.4 Prove (ii) of Theorem 2.1 Let (LP) and (LD) have interior. 2. S ∈ Mn be both positive deﬁnite. 2.11. represents the volume of a coordinate-aligned ellipsoid whose intersection with the aﬃne set {x : Ax = b} contains the primal level set {x ∈ Fp : cT x ≤ z}.5 Exercises 2. S) = n log(X • S) − log(det(X) · det(S)) ≥ n log n. Prove that Ψ(δ 1 ) ⊂ Ψ(δ 2 ) if δ1 ≤ δ2 .2 Let X.10. 2. Prove the dual potential function Bn+1 (y. ˆ and for every δ Ψ(δ) is bounded and its closure Ψ(δ) has non-empty intersection with the solution set. 2. . 2.5 Prove Theorem 2.6 Prove Corollary 2.3 Consider linear programming and the level set ◦ Ψ(δ) := {(X. S) ∈F : ψn+ρ (x. 2.

The dual-scaling algorithm uses only S to generate the new iterate: X k+1 S k+1 = Fd (S k ). where Fpd is the primal-dual algorithm iterative mapping. Recall that Mn denotes the set of symmetric matrices in Rn×n . Other scaling algorithms have been proposed in the past. In other words. and it uses only X to generate the iterate direction. All these algorithms generate primal and dual iterates simultaneously. The third is the primal-dual scaling algorithm which uses both X and S to generate the new iterate and references therein): X k+1 S k+1 = Fpd (X k . One is the primal-scaling algorithm. and √ possess O( n ln(1/ )) iteration complexity to yield the duality gap accuracy . For example. However this algorithm may not even converge. S k ). where Fd is the dual algorithm iterative mapping. an SDP equivalent of Dikin’s aﬃne-scaling algorithm could be very fast.1. X k+1 S k+1 = Fp (X k ). where Fp is the primal algorithm iterative mapping. Let Mn + ◦ denote the set of positive semi-deﬁnite matrices and Mn the set of positive + deﬁnite matrices in Mn . There are actually several polynomial algorithms.0. Another is the dual-scaling algorithm which is the analogue of the dual potential reduction algorithm for linear programming.Chapter 3 Interior-Point Algorithms This pair of semideﬁnite programs can be solved in “polynomial time”. 43 . The goal of this section is to extend interior-point algorithms to solving the positive semi-deﬁnite programming problem (SDP) and (SDD) presented in Section 2.

z) = (n + ρ) log(C • X − z) − log det X. by X := X f = √ n (λj (X))2 .5 = µ0 I and µ0 = X 0 • S 0 /n. We will use it as our initial point throughout this section. Then consider the primal potential function P(X.n} where λj (X) is the jth eigenvalue of X.. Then S • X = C • X − z. Like in Chapter 4. In this section.”. (LP) and (LD) can be expressed as a positive semi-deﬁnite program by deﬁning C = diag(c). b = b. However.. (y. and the “Euclidean” or l2 norm. which is the traditional Frobenius norm. In fact. S) ∈F d . INTERIOR-POINT ALGORITHMS (SDP) and (SDD) are analogues to linear programming (LP) and (LD)..44 CHAPTER 3. see Section 2. Many of the theorems and algorithms used in LP have analogues in SDP. the optimal solution sets for both (SDP ) and (SDD) are bounded and the central path exists. these functions will be used to solve SDP problems. Let z = bT y. we are interested in ﬁnding an approximate solution for the SDP problem: C • X − bT y = S • X ≤ .3 Let z ∗ denote the optimal value and F = Fp × Fd . We assume that ◦ ◦ both F p and F d are nonempty. interior-point methods for SDP outperform other methods on even small problems. is known. For simplicity. X •X = j=1 . where ai· is the ith row of matrix A. ◦ ◦ Let X ∈F p . and the primal-dual potential function ψ(X. and we have ψ(x. j∈{1. Ai = diag(ai· ). s) = P(x. as the notation suggest. S) = (n + ρ) log(S • X) − log det XS. while interior-point algorithms for LP are generally considered competitive with the simplex method in practice and outperform it as problems become large. and z ≤ z ∗ . Denote the primal feasible set by Fp and the dual by Fd . z) − log det S. Deﬁne the “∞-norm. Thus. which satisﬁes (X 0 ). y 0 . we assume that a central path pair (X 0 . which is the traditional l2 operator norm for matrices. S 0 ). where ρ = √ n. of Mn by X ∞ := max {|λj (X)|}..5 S 0 (X 0 ).

2 Let X ∈ Mn and X ∞ d 2 2(1 − d .1 ∞) X 2 2(1 − X ∞) . k )T xk (s c x − zk We directly solve the ball-constrained linear problem for direction dx : minimize s. and the descent gets steeper as ρ increases. j=1 where ρ ≥ 0. Let z = bT y. tr(X) ≥ log det(I + X) ≥ tr(X) − 3. j=1 Recall that when ρ = 0. n tr(X) = n λj (X) and det(I + X) = j=1 (1 + λj (X)). We ﬁrst prove two important lemmas. when ρ > 0. we have the following lemma which resembles Lemma 3. (X k )−1 dx ≤ α. y k . z) − log sj . then the gradient vector of the primal potential function at xk is Pn+ρ (xk . s) → −∞ means that x and s converge to the optimal face. Then. Fix z k = bT y k . z k ) = (n + ρ) (n + ρ) c − (X k )−1 e = T k c − (X k )−1 e. j=1 Then. As the potential function decreases.1.3. Pn+ρ (xk .t. s) is minimized along the central path. sk ) ∈F . . The process calculates steps for x and s. ψn+ρ (x. Lemma 3. z k )T dx Adx = 0. s) = (n + ρ) log(xT s) − log(xj sj ).1. s) = Pn+ρ (x. ◦ Consider a pair of (xk . Then consider the primal-dual potential function: n ψn+ρ (x. for X ∈ Mn .1 If d ∈ Rn such that d ∞ < 1 then n eT d ≥ log(1 + di ) ≥ eT d − i=1 Lemma 3. both x and s are forced to an optimal solution pair. y. POTENTIAL REDUCTION ALGORITHM FOR LP 45 We rename these norms because they are perfect analogues to the norms of vectors used in LP. note that. Furthermore. < 1. Potential Reduction Algorithm for LP ◦ Let (x. then sT x = cT x − z and we have n ψn+ρ (x. In this section we √ choose ρ = n. which guarantee a constant reduction in the primal-dual potential function. ψn+ρ (x. However. s) ∈F .

the primal-dual potential function is also reduced. which can be rewritten as pk = (I − X k AT (A(X k )2 AT )−1 AX k )( = (n + ρ) X k s(z k ) − e. sk ) − ψn+ρ (xk .3) .2) where s(z k ) = c − AT y(z k ) (3. z k ).1) and. That is. even if pk is small. 2(1 − α) Here. s) and Pn+ρ (x. Then dx = −α X k pk . We focus on the expression of pk . z k ) − Pn+ρ (xk . z k ) ≤ −δ for some positive constant δ. INTERIOR-POINT ALGORITHMS Let the minimizer be dx .46 CHAPTER 3. z k )T dx + 2(1 − (X k )−1 dx ∞ ) α2 = −α pk + 2(1 − α) ≤ ∞) Thus. ψn+ρ (xk+1 . z). we may choose an appropriate α such that Pn+ρ (xk+1 . z k ) − Pn+ρ (xk . Update xk+1 = xk + dx = xk − α X k pk . cT xk − z k (n + ρ) X k c − e) cT xk − z k (3. Pn+ρ (xk+1 . pk where pk = p(z k ) :== (I − X k AT (A(X k )2 AT )−1 AX k )X k Pn+ρ (xk . as long as pk ≥ η > 0. z k ) − Pn+ρ (xk . we have used the fact Pn+ρ (xk+1 . By the relation between ψn+ρ (x. z k ) ≤ −α pk + α2 . sk ) ≤ −δ. we will show that the primal-dual potential function can be reduced by a constant δ by increasing z k and updating (y k . sk ). However. pk (3. z k ) n+ρ (X k )−1 dx 2 cT dx − eT (X k )−1 dx + cT xk − z k 2(1 − (X k )−1 dx (X k )−1 dx 2 = P(xk .

y1 = (A(X k )2 AT )−1 b.6) Proof.6) is violated. n If n .1. n + η2 p(z k ) < min η (3. (3.1 − η . then p(z k ) 2 = = ≥ ≥ (n + ρ) k k (n + ρ)µ (n + ρ)µ X s(z ) − e+ e−e 2 nµk nµk nµk (n + ρ) 2 k k (n + ρ)µ ( ) X s(z ) − µe 2 + e−e 2 k nµ nµk (n + ρ)µ 2 2 (n + ρ)µ ( ) η +( − 1)2 n (3.5η ≥ (1 + √ )(1 − √ ) ≥ 1.3 Let µk = cT xk − z k (xk )T sk = n n and µ= (xk )T s(z k ) .3. y2 = (A(X k )2 AT )−1 A(X k )2 c. nµk n n . POTENTIAL REDUCTION ALGORITHM FOR LP and T k 47 k x −z y(z k ) = y2 − c (n+ρ) y1 .7) nµk nµk n η2 .6) is not true. and √ µ < (1 − . then (n + ρ)µ 1 . n + η2 where the last relation prevails since the quadratic term yields the minimum at n (n + ρ)µ = .6) does not hold. The proof is by contradiction. nµk n + η2 iii) If the third inequality of (3. then ∃ j such that sj (z k ) ≤ 0 and p(z k ) ≥ 1 − (n + ρ) xj sj (z k ) ≥ 1. we have the following lemma: Lemma 3. nµk ii) If the second inequality of (3. i) If the ﬁrst inequality of (3. X k s(z k ) − µe < ηµ.4) Regarding pk = p(z k ) .5) then the following three inequalities hold: s(z k ) > 0. (3.5η/ n)µk .

3). sk ) − ψn+ρ (xk . sk ) ∈F .1).1 − η . i. sk+1 ) ≤ ψn+ρ (xk .. in view of (3. Thus.1 − η + .4) and s = s(z k ) in (3.3 the inequalities of (3.48 CHAPTER 3. .5) does not hold. Proof. we can increase z k to bT y(z k ) to cut the dual level set Ω(z k ). sk ) − δ where δ > 1/20. y k . xk+1 be given k+1 k k+1 by (3. sk ) − δ or ψn+ρ (xk . Let ρ = n. from Lemma 3. sk ) ≤ ψn+ρ (xk . We have the following potential reduction theorem to evaluate the progress. ◦ √ Theorem 3. when p(z k ) is small. leads to p(z k ) 2 ≥ ≥ ≥ ≥ (n + ρ)µ − 1)2 n nµk .6) hold: ◦ i) The ﬁrst of (3. z k ) ≤ −α min η n α2 . s(z k )) is in the neighborhood of the central path and bT y(z k ) > z k . n + η2 then Pn+ρ (xk+1 . then (xk . n + η2 2(1 − α) Otherwise. either ψn+ρ (xk+1 .1 − η + . ψn+ρ (xk+1 . Then. z k = bT y k .7).6) indicates that y k+1 and sk+1 are in F d .5η 1 ((1 + √ )(1 − √ ) − 1)2 n n n η η 2 (1 − − √ ) 2 2 n ( (1 − η)2 . The lemma says that. n + η2 2(1 − α) hence from the relation between Pn+ρ and ψn+ρ . z k ) − Pn+ρ (xk .e.4 Given (xk . y(z k ). INTERIOR-POINT ALGORITHMS which. If (3. p(z k ) ≥ min η n . sk ) ≤ −α min η α2 n . and y = y(z ) in (3.

¯ z and if ψn+ρ (xk . µk 2 Adding the two inequalities in ii) and iii). In practice. sk+1 ) ≤ ψn+ρ (xk . Let z 0 = bT y 0 . ◦ Algorithm 3.1 Given a central path point (x0 . This results in the following primal-dual potential reduction algorithm. Theorem 3. sk+1 = s(¯) and z z z z k+1 = bT y k+1 . s0 ) ∈F .6) and applying Lemma 3. 2(1 − η) iii) According to the third of (3. sk+1 = sk and z k+1 = z k . we have n log(xk sk+1 ) j j n log(xk )T sk+1 − j=1 n log(xk sk+1 /µ) j j = n log n − j=1 X k sk+1 /µ − e 2 2(1 − X k sk+1 /µ − e η2 ≤ n log n + 2(1 − η) ≤ n log n + n ≤ n log(xk )T sk − log(xk sk ) + j j j=1 ∞) η2 . While (sk )T xk ≥ do 1. sk ) then y k+1 = y(¯).6). y k+1 = y k . .1 to vector X k sk+1 /µ. otherwise. POTENTIAL REDUCTION ALGORITHM FOR LP 49 ii) Using the second of (3.1. 2. s(¯)) < ψn+ρ (xk . Set k := 0.4 establishes an important fact: the primal-dual potential function can be reduced by a constant no matter where xk and y k are. y 0 .3.3 we have the desired result. we have √ n(log(xk )T sk+1 − log(xk )T sk ) = √ n log µ η ≤− . s(z)).4). by choosing η = .43 and α = . Compute y1 and y2 from (3. 2 2(1 − η) Thus. sk ) − η η2 + . If there exists z such that s(z) > 0. one can perform the line search to minimize the primal-dual potential function. compute z = arg min ψn+ρ (xk . we have ψn+ρ (xk .

if 0 < γ < 1. Let xk+1 = xk − αX k p(z k+1 ) with ¯ α = arg min ψn+ρ (xk − αX k p(z k+1 ). 3. sk ) − ψn+ρ (x0 . (3. s. s(µ)) characterized by the system of equations (2. √ Proof. it steps toward the optimal solution characterized by the system of equations (1.1 terminates in at most O( n log(cT x0 − cT xk − bT y k ≤ . cT xk − bT y k = (xk )T sk ≤ .. Algorithm 3. dy and ds from the system of linear equations: Sdx + Xds = γµe − Xs.2). In the algorithm presented in . sk+1 ). i. Note that dT ds = −dT AT dy = 0 x x here. y(µ). ¯ α≥0 4. In O( n log((x0 )T s0 / )) iterations √ − n log((x0 )T s0 / ) = ψn+ρ (xk . Let d := (dx . we write d = d(x.5 Let ρ = bT y 0 )/ ) iterations with √ √ n.e. see Section 2. Adx = 0. ds ). s+ ) by solving for dx . s) which helps to ﬁnd the point on the central path with duality gap γnµ. s) and the parameter γ. If γ = 0. INTERIOR-POINT ALGORITHMS 3. it steps toward the central path point (x(µ). if γ = 1. Thus. The system (3. To show the dependence of d on the current pair (x. dy . it steps toward a central path point with a smaller complementarity gap. The performance of the algorithm results from the following corollary: Corollary 3. Let k := k + 1 and return to Step 1. √ n log(cT xk − bT y k ) = √ n log(xk )T sk ≤ √ n log .2 Primal-Dual (Symmetric) Algorithm for LP Another technique for solving linear programs is the symmetric primal-dual ◦ algorithm.1. Once we have a pair (x. s0 ) √ = n log((xk )T sk /(x0 )T s0 ). γ).3.50 CHAPTER 3. we can generate a new iterate x+ and (y + .8) −AT dy − ds = 0. s0 ) √ ≥ n log(xk )T sk + n log n − ψn+ρ (x0 . s) ∈F with µ = xT s/n. y.8) is the Newton step starting from (x. Then.10).

xT s 2(1 − α) Let v = Xs. Each iterate reduces the primaldual potential function by at least a constant δ. s0 ) ∈F . PRIMAL-DUAL (SYMMETRIC) ALGORITHM FOR LP 51 this section. To analyze this algorithm. s+ ) ∈F and ψn+ρ (x+ . s) ≤ −α 3/4 + α2 = −δ 2(1 − α) for a constant δ. . Then. s+ ) − ψn+ρ (x.7 Let v ∈ Rn be a positive vector and ρ ≥ n. we have (x+ .6 Let the direction d = (dx . dy . we can prove the following lemma (Exercise 3. min(v) V −1/2 (e − (n + ρ) v) ≥ eT v 3/4 . s) ≤ −α min(Xs) (XS)−1/2 (e − (n + ρ) α2 Xs) + . y + . as does the previous potential reduction algorithm. but a faster algorithm may be again implemented by conducting a line search along direction d to achieve the greatest reduction in the primal-dual potential function.2. and let θ= α min(Xs) xT s (XS)−1/2 ( (n+ρ) e − Xs) . Then. s) = (xk . y + = y + θdy . Let x+ = x + θdx . This result will provide a competitive theoretical iteration bound. (3. sk ) and γ = n/(n + ρ) and compute (dx . whose proof is omitted. ◦ Then. we choose γ = n/(n + ρ) < 1.8).3): √ Lemma 3. we present the following lemma. Set (x.8) with γ = n/(n + ρ). dy .3. ds ) be generated by equation (3. Lemma 3. Combining these two lemmas we have ψn+ρ (x+ . y 0 . ds ) from (3. 1.2 Given (x0 . and s+ = s + θds . This leads to ◦ Algorithm 3.9) where α is a positive constant less than 1. Set ρ ≥ While (sk )T xk ≥ do √ n and k := 0. s+ ) − ψn+ρ (x.

Fix z k = bT y k .5 ≤ α < 1.5 (X − X k )(X k )−. Sk • X k The following corollary is an analog to LP.3 Potential Reduction Algorithm for SDP ◦ Consider a pair of (X k . AX =  2   . Then. z k ) − P(X k . < 1.52 CHAPTER 3. y k . INTERIOR-POINT ALGORITHMS 2.t. z k ) • (X − X k ) A(X − X k ) = 0.. z k ) = n+ρ C − (X k )−1 . Then. Let xk+1 = xk + αdx .5 (X − X k )(X k )−. (X k )−.. i=1 Then.   A1  A  A =  2 . Let k := k + 1 and return to Step 1. ◦ Corollary 3.5 Let ∞) . sk + αds ). we directly solve the following “ball-constrained” problem: minimize s. ¯ α≥0 3.. Am • X and m AT y = y i Ai .5 2 2(1 − (X k )−.  Am Then.2 terminates in at most √ O( n log((x0 )T s0 / )) iterations with cT xk − bT y k ≤ . z k ) • (X − X k ) (X k )−.5 (X − X k )(X k )−.5 + X ◦ ∈Mn + ∞ and P(X. √ Theorem 3. deﬁne   A1 • X  A •X   = b.9 Let X k ∈Mn and (X k )−. and sk+1 = sk + αds where ¯ ¯ ¯ α = arg min ψn+ρ (xk + αdx . .5 (X − X k )(X k )−. Algorithm 3. y k+1 = y k + αdy . then the gradient matrix of the primal potential function at X k is P(X k .. 3. z k ) ≤ + P(X k . S k ) ∈F . P(X k .8 Let ρ = O( n).  .

z k ) • (X k+1 − X k ) P(X k . z k )(X k ).5 P k (X k ).5 P(X k . (X k ).. Am • Am yk = Here.5 − A y k T n+ρ (X k ). z k ) • (X k ).. z k )(X k ).5 Am (X k ).. A1 • Am  A • A1 A2 • A2 . where (X k ). i.5 X(X k )−.5 .. .3.5 • (X − I) A (X − I) = 0. + Q • X . Note that for any symmetric matrices Q. Then we transform the above problem into minimize s.  :=  .. z k )(X k ).5 Pk (X k ). and   A1 • A1 A1 • A2 . Am (X k ).5 .5 • P k −α Pk Pk 2 −α = −α P k ...5 . X − I ≤ α. T ∈ Mn ◦ and X ∈Mn ..5 = X .5 = or = (X k ).5 P(X k . PA In view of Corollary 3.5 ..5 QX .5 T X . .5 P(X k ..5 − I.5    .5 P(X k .. Then X − I = −α X k+1 − X k = −α Pk Pk .5 P(X k ..5 P k (X k ).5 A1 (X k ). Pk = −α = = .5 • T and XQ · = QX · = X . . i = 1.9 and P(X k .3.5 QX .5 A2 (X k ).. n+ρ is the projection operator onto the null space of A . z k )(X k )..   (X k ).5 · . . A A :=  2   . z k )(X k ). Sk • X k Sk • X k T (A A )−1 A (X k ).. A2 • Am  T  ∈ Mm . Am • A1 Am • A2 ...10) where Pk Pk = and PA (X k ). POTENTIAL REDUCTION ALGORITHM FOR SDP 53 Let X = (X k )−. 2.5 A1  (X k )..5 (C − AT y k )(X k ).t..5 X (X k ).  Let the minimizer be X and let X k+1 = (X k )...5  A2    A =  . Pk (3.

.5 S(z k )(X k ). z k ) − P(X k .15) then the following three inequalities hold: S(z k ) 0.5 S(z k )(X k ). (3.5 has at least one eigenvalue less than or equal to zero. 2(1 − α) Thus. T = (A A )−1 A (X k ). z k ) − P(X k . as long as P k ≥ β > 0. The proof is by contradiction. we have the following potential reduction theorem.13) where y1 and y2 are given by y1 y2 T T = (A A )−1 A I = (A A )−1 b. n+ρ n+ρ (3. z k ) ≤ −δ for some positive constant δ. For example. (3.1 − β . n If P (z k ) < min β n .11) (3. and P (z k ) ≥ 1. (X k ). then (X k ). we may choose an appropriate α such that P(X k+1 .5 S(z k )(X k ).5 − µe < βµ.5 C(X k ). we focus on the expression of P k . Based on this lemma. Lemma 3.16) Proof.12) Sk • X k C • X k − zk y1 = y2 − y1 . The proof of the second and third inequalities are similar to that of Lemma 3. n + β2 (3.3.5 . and √ µ < (1 − . Now.5β/ n)µk .16) is not true. which can be rewritten as P (z k ) := P k = with n+ρ (X k ). z k ) ≤ −α P k + α2 .14) Regarding P k = P (z k ) .3.10 Let µk = Sk • X k C • X k − zk = n n and µ= S(z k ) • X k . INTERIOR-POINT ALGORITHMS we have P(X k+1 .54 CHAPTER 3.5 − I Sk • X k S(z k ) = C − AT y(z k ) and y(z k ) := y k = y2 − (3. we have the following lemma resembling Lemma 3. if the ﬁrst inequality of (3.

16) and applying Lemma 3. S k ) − ψ(X k . Proof..3. by choosing β = .11 Given X k ∈F p and (y k .5 S k+1 (X k ). ii) Using the second of (3. since ψ(X k+1 . S k ) = P(X k+1 . S k+1 ) ≤ ψ(X k . S k ) − δ or ψ(X k .3 we have the desired result.43 and α = .15) does not hold.5 /µ − I 2 ≤ n log n + 2(1 − (X k ).5 S k+1 (X k ). z k = bT y k . S k ) ≤ ψ(X k . POTENTIAL REDUCTION ALGORITHM FOR SDP 55 ◦ ◦ √ Theorem 3.5 S k+1 (X k ). Then. we have n log S k+1 • X k − log det S k+1 X k = n log S k+1 • X k /µ − log det(X k ). S k ) − δ. If (3.e.16). S k ) − ψ(X k . let ρ = n. S k ) ≤ −α min β n . where δ > 1/20.13) and S k+1 = S(z k ) in (3. S k+1 ) ≤ ψ(X k . from Lemma 3.5 /µ − I ∞ ) β2 ≤ n log n + 2(1 − β) β2 ≤ n log S k • X k − log det S k X k + .1 − β n + β2 + α2 . 2(1 − β) iii) According to the third of (3.5 S k+1 (X k ). S k ) ∈F d . ψ(X k+1 . .2 to matrix (X k ). 2(1 − α) Otherwise. z k ) − P(X k . X k+1 be given by (3.10). k µ 2 Adding the two inequalities in ii) and iii).5 /µ.5 S k+1 (X k ). n + β2 then.16) indicates that y k+1 and S k+1 are in F d . S k ) − β2 β + . we have √ n(log S k+1 • X k − log S k • X k ) = √ n log µ β ≤− .5 /µ (X k ).16) hold: ◦ i) The ﬁrst of (3. and y k+1 = y(z k ) in (3. P (z k ) ≥ min β n . either ψ(X k+1 . i. we have ψ(X k .3.1 − β .10 the inequalities of (3. z k ). 2 2(1 − β) Thus.12).5 /µ = n log n − log det(X k ).

3.56 CHAPTER 3. √ n log .14). Algorithm 3.5 P (z k+1 )(X k ). ¯ z≥z k If ψ(X k . C • X k − bT y k = S k • X k ≤ .12 Let ρ = n. Let X k+1 = X k − α(X k ).3 terminates in at most √ O( n log(C • X 0 − bT y 0 )/ ) iterations with C • X k − bT y k ≤ .11 establishes an important fact: the primal-dual potential function can be reduced by a constant no matter where X k and y k are. s0 ) ∈F d . Set y k+1 = y(¯). one can perform the line search to minimize the primal-dual potential function. z k+1 = z k .3 Given x0 ∈F p and (y 0 . S k ) − ψ(X 0 . 2. INTERIOR-POINT ALGORITHMS Theorem 3. z k+1 = bT y k+1 with z z z = arg min ψ(X k . S k ) then y k+1 = y k . ¯ α≥0 4. The performance of the algorithm results from the following corollary: √ Corollary 3. S(z)). Then. Let z 0 = bT y 0 .5 . S k+1 ) > ψ(X k . S k+1 ). S k+1 = S(¯). S 0 ) √ n log(S k • X k /S 0 • X 0 ). Let k := k + 1 and return to Step 1. √ = ψ(X k .5 with ¯ α = arg min ψ(X k − α(X k ). ◦ ◦ Algorithm 3. S k+1 = S k . S 0 ) √ ≥ n log S k • X k + n log n − ψ(X 0 . While S k • X k ≥ do 1. Set k := 0. In practice.. = n log(C • X k − bT y k ) = √ n log S k • X k ≤ i. Compute y1 and y2 from (3.e. √ Proof. This results in the following potential reduction algorithm.5 P (z k+1 )(X k ). . In O( n log(S 0 • X 0 / )) iterations √ − n log(S 0 • X 0 / ) Thus.

 Again. Let ◦ V 1/2 = D−..5  A2    A =  . dS = −A dy . and let θ= α V −1/2 I•V −1/2 ∞ n+ρ V − V 1/2 .5 dX D−.5 . We now present the following lemma. Then. dy and dS from the system of linear equations: D−1 dX D−1 + dS AdX −AT dy − dS = R := γµX −1 − S.5 Am D.6 and will be omitted.. we can verify that S • X = I • V .5 A1  D.5 = D. Note that dS • dX = 0. + Then. R = D. (3.5 X .4.5 XD−.5 (X .5 (γµX −1 − S)D. = 0.5 )−. Lemma 3. we have dS • dX = 0.18) where dX = D−.5 ∈Mn .. assign dS = AT dy and dX = D(R − dS )D.5 . PRIMAL-DUAL (SYMMETRIC) ALGORITHM FOR SDP 3.5 SD. and dS = D.5 A1 D. = 0. D. y. S) ∈F with µ = S • X/n.17) with γ = n/(n + ρ).. we can apply the primaldual Newton method to generate a new iterate X + and (y + .5 dS D. = 0. whose proof is very similar to that for LP and 3.5 SX .19) . = 0.17) where D = X .5 .13 Let the direction dX .5 .5 Am    . and T T dy = (A A )−1 A R .   D.  :=  . and dX = R − dS .3. S + ) as follows: Solve for dX .4 57 Primal-Dual (Symmetric) Algorithm for SDP ◦ Once we have a pair (X. (3. dy and dS be generated by equation (3.5 A2 D. (3. This system can be written as dX + dS A dX T −A dy − dS = R.

y k+1 = y k + αdy . S k ) and γ = n/(n + ρ) and compute (dX . ◦ Algorithm 3. Let X + = X + θdX . y 0 . 2. ¯ α≥0 3. S k + αdS ). dy . S) ≤ −α 3/4 + α2 = −δ 2(1 − α) for a constant δ. dS ) from (3.7 to v ∈ Rn as the vector of the n eigenvalues of V . S) ≤ −α V −1/2 − n+ρ V 1/2 α2 I•V . This leads to Algorithm 3. Set ρ = While S k • X k ≥ do √ n and k := 0. 1.4 terminates in at most √ O( n log(S 0 • X 0 / )) iterations with C • X k − bT y k ≤ . we can prove the following lemma: ◦ Lemma 3. Let X k+1 = X k + αdX . V −1/2 − n+ρ V 1/2 I•V ≥ V −1/2 ∞ 3/4. S) = (X k .14 Let V ∈Mn and ρ ≥ + √ n. we have (X + . Let k := k + 1 and return to Step 1. From these two lemmas we have ψ(X + .17).58 CHAPTER 3. S 0 ) ∈F . and S + = S + θdS . y + = y + θdy .4 Given (X 0 . and S k+1 = S k + αdS . where ¯ ¯ ¯ α = arg min ψ(X k + αdX . S + ) − ψ(X. ◦ Then. + 2(1 − α) V −1/2 ∞ Applying Lemma 3.4. Set (X. S + ) ∈F and ψ(X + . S + ) − ψ(X. y + .15 Let ρ = n. Algorithm 3. Then. Then. . √ Theorem 3. INTERIOR-POINT ALGORITHMS where α is a positive constant less than 1.

we have n ln(X • S) − ln det X − ln det S ≥ n ln n. may be much less sparse and have a structure unknown beforehand. For large scale problems. We show that many large-scale semideﬁnite programs arisen from combinatorial and quadratic optimization have features which make the dual-scaling interior-point algorithm the most suitable choice: 1. Solving (SDD) alone would be suﬃcient to provide such a lower bound. our dual-scaling algorithm can generate an optimal X at the termination of the algorithm with little additional cost. S tends to be very sparse and structured since it is the linear combination of C and the Ai ’s.3. This sparsity allows considerable savings in both memory and computation time. the superlinear convergence exhibited by primal-dual algorithms may not be utilized in our applications. the box-constrained quadratic optimization problem. The dual-scaling algorithm has been shown to perform equally well when only a lower precision answer is required. DUAL ALGORITHM FOR SDP 3. we may not need to generate an X at all. In this paper we try to respond to this question. Therefore. .5. Also note that. the inﬁmum of the potential function occurs at an optimal solution. When ρ > n. the 0 − 1 integer set covering problem. 3. Although primal-dual algorithms may possess superlinear convergence. In most combinatorial applications. Thus. The dual-scaling algorithm. which is a modiﬁcation of the dual-scaling linear programming algorithm. Consequently. etc. The ﬁrst term decreases the duality gap. Even if an optimal primal solution is necessary. The computational cost of each iteration in the dual algorithm is less that the cost the primal-dual iterations. primal and primal-dual algorithms may not fully exploit the sparseness and structure of the data. while the second and third terms keep X and S in the interior of the positive semideﬁnite matrix cone.5 59 Dual Algorithm for SDP An open question is how to exploit the sparsity structure by polynomial interiorpoint algorithms so that they can also solve large-scale problems in practice. reduces the Tanabe-Todd-Ye primal-dual potential function Ψ(X. X. These problems include the semideﬁnite relaxations of the graph-partition problem. we need only a lower bound for the optimal objective value of (SDP). 2. The primal matrix. S) = ρ ln(X • S) − ln det X − ln det S. the approximation problems under consideration require less accuracy than some other applications. using the arithmetic-geometric mean inequality.

60

CHAPTER 3. INTERIOR-POINT ALGORITHMS
Let operator A(X) : S n →

m

be deﬁned as

A1 • X
 A2 • X 

A(X) = 
.
.
.

.
Am • X

Since A(X)T y =
AT : m → S n is

m
i=1

m
i=1

yi (Ai • X) = (

yi Ai ) • X, the adjoint operator

m
T

A (y) =

yi Ai .
i=1

Let z = C • X for some feasible X and consider the dual potential function
¯
ψ(y, z ) = ρ ln(¯ − bT y) − ln det S.
¯
z
ψ(y, z ) = −
¯

ρ
b + A(S −1 ).
z − bT y
¯

For any given y and S = C − AT (y) such that S

(3.20)

0 and

(S k )−.5 AT (y − y k ) (S k )−.5 < 1,
using the above lemma, the concavity of the ﬁrst term in the potential function,
and the fact that
(S k )−.5 S(S k )−.5 − I = (S k )−.5 (S − S k )(S k )−.5 = (S k )−.5 AT (y − y k ) (S k )−.5 ,
we establish an overestimater for the potential reduction:
ψ(y, z k ) − ψ(y k , z k )
¯
¯
= ρ ln(¯k − bT y) − ρ ln(¯k − bT y k ) − ln det((S k )−.5 S(S k )−.5 )
z
z
≤ ρ ln(¯k − bT y) − ρ ln(¯k − bT y k ) + trace((S k )−.5 S(S k )−.5 − I)
z
z
(S k )−.5 (AT (y−y k ))(S k )−.5
+ 2(1− (S k )−.5 (AT (y−yk ))(S k )−.5 ∞ )
= ρ ln(¯k − bT y) − ρ ln(¯k − bT y k ) + A((S k )−1 )T (y − y k )
z
z
(S k )−.5 (AT (y−y k ))(S k )−.5
+ 2(1− (S k )−.5 (AT (y−yk ))(S k )−.5 ∞ )
(S k )−.5 (AT (y−y k ))(S k )−.5
≤ ψ(y k , z k )T (y − y k ) + 2(1− (S k )−.5 (AT (y−yk ))(S k )−.5 ∞ ) .
¯

(3.21)

Therefore, beginning with a strictly feasible dual point (y k , S k ) and upper
bound z k , each iteration solves this following problem.
¯
Minimize
Subject to

ψ T (y k , z k )(y − y k )
¯
(S k )−.5 AT (y − y k ) (S k )−.5 ≤ α,

(3.22)

where α is a positive constant less than 1. For simplicity, in what follows we let
∆k = z k − b T y k .
¯

3.5. DUAL ALGORITHM FOR SDP

61

The ﬁrst order Karusch-Kuhn-Tucker conditions state that the minimum
point, y k+1 , of this convex problem satisﬁes
ρ
M k (y k+1 −y k )+β ψ(y k , z k ) = M k (y k+1 −y k )+β(− k
¯
b+A((S k )−1 )) = 0
z − bT y k
¯
(3.23)
for a positive value of β, where

A1 (S k )−1 • (S k )−1 A1 · · · A1 (S k )−1 • (S k )−1 Am

.
.
..
.
.
Mk = 

.
.
.
Am (S k )−1 • (S k )−1 A1
and

···

Am (S k )−1 • (S k )−1 Am

A1 • (S k )−1

.
.
A((S k )−1 ) = 
.
.
Am • (S k )−1

The matrix M k is a Gram matrix and is positive deﬁnite when S k 0 and the
Ai ’s are linearly independent. In this paper, it will sometimes be referred to as
M.
Using the ellipsoidal constraint, the minimal solution, y k+1 , of (3.22) is given
by
α
y k+1 − y k =
d(¯k )y
z
(3.24)
T (y k , z k )(M k )−1 ψ(y k , z k )
ψ
¯
¯
where
d(¯k )y = −(M k )−1 ψ(y k , z k ).
z
¯
(3.25)
Unlike linear programming, semideﬁnite programs require a signiﬁcant amount
of the time to compute the system of equations used to to determine the step
direction. For arbitrary symmetric matrices Ai , the AHO direction can be
computed in 5nm3 + n2 m2 + O(max{m, n}3 ) operations, the HRVW/KSH/M
direction uses 2nm3 + n2 m2 + O(max{m, n}3 ) operations, and the NT direction
uses nm3 + n2 m2 /2 + O(max{m, n}3 ) operations. The complexity of computing
the matrix is a full order of magnitude higher than any other step of the algorithm. Fujisawa, Kojima and Nakata explored another technique for computing
primal-dual step directions that exploit the sparsity of the data matrices. However, it is our belief that only the dual-scaling algorithm can fully exploit the
structure and sparsity of many problems, as explained below.
k
Generally, Mij = Ai (S k )−1 • (S k )−1 Aj . When Ai = ai aT , the Gram matrix
i
can be rewritten in the form
 T k −1 2

(a1 (S ) a1 )
· · · (aT (S k )−1 am )2
1

.
.
..
.
.
Mk = 
(3.26)

.
.
.
(aT (S k )−1 a1 )2
m
and

···

(aT (S k )−1 am )2
m

aT (S k )−1 a1
1

.
.
A((S k )−1 ) = 
.
.
T
k −1
am (S ) am

62

CHAPTER 3. INTERIOR-POINT ALGORITHMS

This matrix can be computed very quickly without computing, or saving, (S k )−1 .
Instead, S k can be factored, and then we can use

wm and they are generally dense... they all need to handle dense X.28) ..3.. Algorithm M. factor S k = Lk (Lk )T and do the following: For i = 1 : m.5 − A(X) = b. To ﬁnd a feasible primal point X. T k A((S k )−1 )i = wi wi and Mii = (A((S k )−1 )i )2 . end. given explicitly by X(¯k ) = z ∆k k −1 T A (d(¯k )y ) + S k (S k )−1 . Solve S k wi = ai .5 X(S k ). an alternative algorithm is Algorithm M’: To compute M k and A((S k )−1 ). Larger values of z ρ generally give a lower objective value. ∆k ρ I (3. end. am .27) This problem looks for a matrix X(¯k ) near the central path. k T A((S k )−1 )i = wi ai and Mii = (A((S k )−1 )i )2 . Solve Lk wi = ai . but even the quickest direction to compute takes about twice as long as our dual-scaling direction. end. Since there are m(m + 1)/2 vector multiplications. . .5. z (S ) ρ (3. so working with S k is faster than working with its inverse. Using matrices of the form Ai = ai aT also reduces the complexity of primal-dual i algorithms by a factor of n. For j = i + 1 : m. Furthermore.. DUAL ALGORITHM FOR SDP 63 Algorithm M: To compute M k and A((S k )−1 ). In applications like the maximum cut problem. To save storage and exploit the sparsity of ai . T k Mij = (wi wj )2 . Algorithm M needs to store all vectors w1 . Algorithm M’ does not need to store wj but uses one more back-solver for wi .27) is a by-product of computing (3. Solving each of the m systems of equations uses n2 + O(n) ﬂoating point operations. factor S k = Lk (Lk )T and do the following: For i = 1 : m. Note that these operations can be signiﬁcantly reduced if S k is structured and sparse. but provide a solution matrix that is not positive deﬁnite more frequently. we solve the least squares problem Minimize Subject to (S k ). k T Mij = (wi aj )2 . For j = 1 : m − 1. the matrix S k is indeed very sparse while its inverse is usually dense.25). uses nm2 + n2 m + O(nm) operations after factoring S k .. end. The answer to (3.

29) ∆ we have the following lemma: Lemma 3. saving time and memory.5 − µI ≤ αµ.5α/ n)µk . However. n n . 1 − α).5 − I 2 z nµk ρ ρµ ρµ k .5 has at least one nonpositive eigenvalue. If the ﬁrst inequality is false. Deﬁning ρ P (¯k ) = k (S k ). n P (¯k ) < min(α z µ = X(¯k )•S k z n = C•X(¯k )−bT y k z . the evaluation of the primal objective value C • X(¯k ) requires drastically less work.5 − nµk I + nµk I − I 2 z nµk ρµ ρµ ρ k . and α < 1.5 − I. the cost of computing a primal objective value is the cost of a vector dot product! The matrix X(¯k ) never gets computed during z the iterative process.5 X(¯k )(S k ). then P (¯k ) z 2 = = = > ≥ ρ (S k ).5 X(¯k )(S k ). then (S k ). nµk . z z (3. z C • X(¯k ) z = bT y k + X(¯k ) • S k z = bT y k + trace = bT y k + = bT y k + ∆k k −1 ρ (S ) k −1 AT (d(¯k )y ) + S k (S k )−1 S k z ∆k AT (d(¯k )y ) z ρ trace (S ) ∆k k −1 k T d(¯ )y A((S ) ) + n z ρ +I Since the vectors A((S k )−1 ) and d(¯k )y were previously found in calculating z the dual step direction.5 X(¯k )(S k ). (S k ). z If the second does not hold. X(¯k ) z 0.16 Let µk = √ n + n. µ ≤ (1 − . 2.5 k k . z √ 3. primal-dual methods require far more resources to compute the primal variables X. If ∆k n = z k −bT y k ¯ . INTERIOR-POINT ALGORITHMS Creating the primal matrix may be costly. The proofs are by contradiction.64 CHAPTER 3. Proof.5 X(¯k )(S k ). On the other hand.5 2 (S ) X(¯ )(S ) − nµk I + nµk I − I 2 z nµk 2 2 ρµ ρµ α2 + nµk − 1 n nµk n α2 n+α2 where the last inequality is true because the quadratic term has a minimum at ρµ n = n+α2 . which by (3. n + α2 ρ ≥ (3.30) then the following three inequalities hold: 1.29) imz plies that P (¯k ) ≥ 1.5 (S ) X(¯k )(S k ).

α ∈ (0.5 T k k −. set k = 0. then ρµ > nµk 1 1+ √ n .5α 1− √ n ≥ 1. 2(1 − α) (3. z k )d(¯k )y = − P (¯k ) ¯ z z and 2 ψ T (y k . Using (3.5 ∆ (S k )−1 AT (d(¯k )y ) + S k (S k )−1 (S k ).3.33) assures the positive deﬁniteness of S k+1 when α < 1. which assures that they are feasible. z k ) − ψ(y k . Focusing on the expression P (¯k ).32) and (3. z k ) ≤ −α P (¯k ) + ¯ ¯ z α2 . ¯ z (3.22). and do the following: while z k − bT y k ≥ do ¯ begin .5 AT y k+1 −y k β (S k )−.5 which by (3.34) The theoretical algorithm can be stated as follows. makes ψ T (y k . which leads to P (¯k ) z 2 ρµ −1 nµk ≥ n 1 1+ √ n ≥ = ≥ 2 1− 2 α 1− √ 2 n α α − √ 2 2 n −1 n 2 (1 − α)2 . DUAL ALGORITHM FOR SDP 65 If the third inequality does not hold. S 0 ) ¯√ such that S 0 = C − AT y 0 0. the reduction in the potential function satisﬁes the inequality ψ(y k+1 .5 = (S ) A d(¯ )y (S ) z = (S k )−. DUAL ALGORITHM. z k )(y k+1 − y k ) = −α P (¯k ) .5.32) Updating the dual variables according to y k+1 = y k + α d(¯)y z P (¯k+1 ) z and S k+1 = C − AT (y k+1 ).21). (3. Given an upper bound z 0 and a dual point (y 0 .5 − I z z ρ k −.31) (3. it can be rewritten as z k ρ P (¯k ) = ∆k (S k ). ρ > n + n. 1).

S k ). Ψ(X k+1 . z 4. one of the diﬀerences is zero.5 X k+1 (S k ). INTERIOR-POINT ALGORITHMS 1.30) is true. k := k + 1. S k ) + Ψ(X k+1 . S k+1 ) − Ψ(X k+1 . S k ) .16 √ n ln(X k+1 • S k ) − ln(X k • S k ) = √ n ln µ α ≤− µk 2 . ¯ ¯ d(¯k+1 )y .5 ) X k+1 (S k ).5 /µ−I ≤ n ln n + 2(1− (S k ). z 3. S k+1 = C − AT (y k+1 ).26) using Algorithm M or M’. In each iteration. then using Lemma 3. S k+1 ) = z ¯ (y k . if the primal matrix gets updated. S k ) − Ψ(X k . S k+1 ) ≤ Ψ(X k . Solve (3. Proof.66 CHAPTER 3.31).25) for the dual step direction d(¯k )y . S k+1 )−Ψ(X k .4.34) shows suﬃcient improvement in the potential function when α = 0. 2. If (3. S k ) = Ψ(X k+1 . z endif 5. the dual variables get updated and (3.5 X k+1 (S k ).30).5 /µ−I ∞ ) (S α2 2(1−α) k k ≤ n ln n + ≤ n ln X • S − ln det X k − ln det S k + α2 2(1−α) Additionally.17 Ψ(X k+1 . X k+1 = X k . If P (¯k ) does not satisfy z (3. n ln X k+1 • S k − ln det X k+1 − ln det S k = n ln X k+1 • S k − ln det X k+1 S k = n ln X k+1 • S k /µ − ln det X k+1 S k /µ = n ln n − ln det (S k ). by the third part of Lemma 3. end We can derive the following potential reduction theorem based on the above lemma: Theorem 3. α P (¯k ) z else y k+1 = y k + and z k+1 = z k . S k ) − δ where δ > 1/50 for a suitable α. Calculate P (¯k ) using (3. z k+1 = C•X k+1 . On the other hand.5 /µ k . then X k+1 = X(¯k ).2 and the ﬁrst two parts of Lemma 3.16. Compute A((S k )−1 ) and the Gram matrix M k (3. and (y k+1 .

Also. This theorem leads to √ Corollary 3. Combining the two inequalities. Theorem 3.19 Each iteration of the dual algorithm uses O(m3 + nm2 + n2 m + n3 ) ﬂoating point iterations. we have the desired result.4 again. NOTES 67 Adding the two inequalities gives Ψ(X k+1 . S k ) ≤ Ψ(X k . it is not needed in generating the iterate direction. S k ). These give the desired result. and it is only explicitly used for proving convergence and complexity. factoring it uses O(n3 ) operations. in which Ye has “suggested studying the primal-dual potential function for this problem” and “looking at symmetric preserving scal−1/2 −1/2 ings of the form X0 XX0 . Again. C •X A • X = b. the algorithm terminates in at most O((ρ − n) ln(X 0 • S 0 / )) iterations. X ∈ K. Then.t. Dot products for z k+1 and P (¯k ) .” and to Nesterov and Nemirovskii [241]. S k ) − α α2 + 2 2(1 − α) By choosing α = 0. S k )−n ln n ≤ Ψ(X k . In general. and the calculation of ¯ z k+1 y use only O(m) operations.3. In O((ρ − n) ln(X 0 • S 0 / )) iterations. 243]. One can also develop a dual potential reduction algorithm. S 0 ) ≤ (ρ − n) ln(X 0 • S 0 ).6 Notes The primal potential reduction algorithm for positive semi-deﬁnite programming is due to Alizadeh [9. and the primal-dual algorithm described here is due to Nesterov and Todd [242. and solving the system of equations uses O(m3 ) operations. 3. 8]. consider (P SP ) inf s. However. uses matrix additions and O(mn2 ) opz erations. Proof. Ψ(X k .18 Let ρ ≥ n + n and Ψ(X 0 . from (3. or S + AT (d(¯k )). Proof.28) we see that the algorithm can generate an X k as a byproduct. Creating the Gram matrix uses nm2 + 2n2 m + O(nm) operations. Creating S. (ρ−n) ln(C •X k −bT y k ) = (ρ−n) ln(X k •S k ) ≤ Ψ(X k .6. S k ) ≤ (ρ − n) ln( )). . C • X k − bT y k = X k • S k < .

12]. dS ) is determined by the following equations: A • dX = 0. Vandenberghe and Boyd [204]. Sturm and Zhang [285]. The search direction (dX . All three generate an optimal solution (X. Vanderbei and Wolkowicz [156]. Vandenberghe and Boyd [313. dY . X •S n+ρ S − F (X) (primal scaling). Shindoh and Kojima [276].37) n+ρ S − F (X) (joint scaling). dS ) and a new strictly feasible primal-dual pair X + and (Y + . dY . Todd and Ye [244]. Shindoh and Hara [190]. Tseng [305]. Primal-dual adaptive path-following algorithms. .. Haeberly and Overton [10. These algorithms have established the best approximation complexity results for some combinatorial problems. S). Potra and Sheng [257]. i. and Xue and Ye [326].e. Feron and Balakrishnan [56].t. S + ) is generated from X + = X + αdX . Helmberg. Roos and Terlaky.39) where Z is chosen to satisfy The diﬀerences among the three algorithms are the computation of the search direction and their theoretical close-form step-sizes. de Klerk. INTERIOR-POINT ALGORITHMS and its dual (P SD) sup s. S ∈ K. and references therein. Ghaoui. Jarre [170]. Kojima. Lobo. X •S (3. Monteiro and Zhang [228].38) S = F (Z)X. Y. S + = S + βdS . Y + = Y + βdY .36) (3. X •S ≤ in a guaranteed polynomial time. Other primal-dual algorithms for positive semi-deﬁnite programming are in Alizadeh. Nesterov. the predictor-corrector algorithms and the wide-neighborhood algorithms can also be developed for solving (SDP). Shida. Boyd.68 CHAPTER 3. Interior-point algorithms compute a search direction (dX .35) (3. for some step-sizes α and β. where K is a convex homogeneous cone. see Andersen and Christiansen [17]. Rendl. bT y A∗ • Y + S = C.[185]. dS + F (Z)dX = − (3. Eﬃcient interior-point algorithms are also developed for optimization over the second-order cone. 314]. X •S or (3. dS = −A∗ • dY and dX + F (S)dS = − or dS + F (X)dX = − (feasibility) n+ρ X − F (S) (dual scaling).

3.3. ﬁnd the minimizer of the least-squares problem minimize s. 3.5 Prove Corollary 3.t. Using this relation to formulate a convex quadratic minimization problem with convex quadratic inequalities as an (SDD) problem.3 Let v ∈ Rn be a positive vector and ρ ≥ min(v) V −1/2 (e − √ n.t.13.6 Prove Lemma 3.7 Describe and analyze a dual potential algorithm for positive semi-deﬁnite programming in the standard form.4 Prove the following convex quadratic inequality (Ay + b)T (Ay + b) − cT y − d ≤ 0 is equivalent to a matrix inequality I (Ay + b)T Ay + b cT y + d 0.7 69 Exercises 3.9. 3.7. S = C − AT y. . Prove (n + ρ) v) ≥ eT v 3/4 . 3. 0.1: If D ∈ Mn such that 0 ≤ D ∞ < 1 then −Trace(D) ≥ log det(I − D) ≥ −Trace(D) − 3.1 Prove a slightly stronger variant of Lemma 3. 3.2 Given S ∞) . 0 ﬁnd the minimizer of the least-squares problem minimize X 1/2 SX 1/2 − I s. Given X D 2 2(1 − D S 1/2 XS 1/2 − I AX = 0. EXERCISES 3.

INTERIOR-POINT ALGORITHMS .70 CHAPTER 3.

A (randomized) algorithm for a minimization problem is called (randomized) r-approximation algorithm. a r-approximate minimizer x satisﬁes w(x) ≤ r · w∗ . where 1 ≤ r. More precisely. let w∗ (> 0) be the (global) minimal value of a given problem instance. More precisely.1 Approximation A (randomized) algorithm for a maximization problem is called (randomized) r-approximation algorithm. Then.Chapter 4 SDP for Combinatorial Optimization 4. let w∗ (> 0) be the (global) maximum value of a given problem instance. 71 . or E[w(x)] ≤ r · w∗ . or E[w(x)] ≥ r · w∗ . if it outputs a feasible solution with its (expected) value at least r times the optimum value for all instances of the problem. where 0 < r ≤ 1. if it outputs a feasible solution with its (expected) value at most r times the optimum value for all instances of the problem. Then. a r-approximate maximizer x satisﬁes w(x) ≥ r · w∗ .

72 CHAPTER 4. (SDP) X 0.1 Homogeneous Case: q = 0 Matrix-formulation: Let z∗ = X = xxT . Rank(X) = 1. is the Euclidean norm. S 0. The dual of (SDP) can be written as: z SDP = Maximize y Subject to yI + S = Q. . (DSDP) y ≤ 0. SDP FOR COMBINATORIAL OPTIMIZATION 4. Theorem 4. and .1 The SDP relaxation is exact. vector q ∈ Rn . (BQP) X 0. z SDP := Minimize Q•X Subject to I • X ≤ 1. Here. Observation: z SDP ≤ z ∗ . the set of n-dimensional symmetric matrices.1) ≤ 1. the given matrix Q ∈ Mn . 4.2 Ball-Constrained Quadratic Minimization z ∗ := Minimize xT Qx + 2q T x (BQP) Subject to x 2 (4. SDP-relaxation: Remove the rank-one constraint. Minimize Q•X Subject to I • X ≤ 1. meaning z SDP ≥ z ∗ .2. X ∗ is a minimal matrix solution to SDP if and only if there exist a feasible dual variable y ∗ ≤ 0 such that S ∗ = Q − y∗ I 0 y ∗ (1 − I • X ∗ ) = 0 S ∗ • X ∗ = 0.

X 0.2. x)(1. S 0. let a decomposition of X ∗ be r X∗ = xj xT . z∗ = Minimize Q •X Subject to I • X ≤ 1. x∗ = (I • X ∗ ) · xj / xj j is a solution to (BQP). The dual of (SDP) can be written as: z SDP = Maximize y1 + y2 Subject to y 1 I + y 2 I1 + S = Q . I1 = 1 0 0T 0 . SDP-relaxation: Remove the rank-one constraint. X 0. for any j. BALL-CONSTRAINED QUADRATIC MINIMIZATION Moreover. (BQP) I1 • X = 1. 73 . I = 0 0 0T I .2 Non-Homogeneous Case Matrix-formulation: Let X = (1. j j=1 where r is the rank of X ∗ .4. Q = 0 q qT Q . (DSDP) y1 ≤ 0. Rank(X) = 1.2. and z∗ = Minimize Q •X Subject to I • X ≤ 1. Then. x)T ∈ Mn+1 . 4. (SDP) I1 • X = 1.

74 CHAPTER 4. 1 Theorem 4. y2 ) such that ∗ ∗ S ∗ = Q − y 1 I − y 2 I1 0 ∗ y1 (1 − I • X ∗ ) = 0 S ∗ • X ∗ = 0. Assume that pT Ap1 < 0 1 and pT Ap2 > 0. meaning z SDP ≥ z ∗ . Then. Let r pj pT j X= j=1 be any decomposition of X. SDP FOR COMBINATORIAL OPTIMIZATION X ∗ is a minimal matrix solution to SDP if and only if there exist a feasible ∗ ∗ dual variables (y1 ≤ 0. 2 Now let x1 = (p1 + γp2 )/ 1 + γ2 x2 = (p2 − γp1 )/ 1 + γ2.2 Let X be a positive semideﬁnite matrix of rank r.3 The SDP relaxation is exact. j j Proof. Lemma 4. A be a given symmetric matrix. We have p1 pT + p2 pT = x1 xT + x2 xT 2 1 2 1 and A • (X − x1 xT ) = 0. Observation: z SDP ≤ z ∗ . and where γ makes (p1 + γp2 )T A(p1 + γp2 ) = 0. . there is a decomposition of X r xj xT . xT Axj = A • (xj xT ) = A • X/r. j X= j=1 such that for all j. 1 Note that X − x1 xT is still positive semideﬁnite and its rank is r − 1. We prove for the case A • X = 0.

HOMOGENEOUS QUADRATIC MINIMIZATION 75 Moreover. y2 ≤ 0. X ∗ is a minimal matrix solution to SDP if and only if there exist a feasible dual variable ∗ ∗ (y1 . Minimize Q•X Subject to A1 • X ≤ 1. j j=1 ∗ where r is the rank of X . Assume that there is no gap between the primal and dual. Then. X 0. X 0. S 0. 4. Rank(X) = 1. (DSDP) y1 . Matrix-formulation: Let z∗ = X = xxT . there is a decomposition of X ∗ be r X∗ = xj xT .3. (QP-2) (4.4. y2 ) ≤ 0 such that S ∗ = Q − y1 A1 − y2 A2 0 . (QP-2) A2 • X ≤ 1. such that for any j.3 Homogeneous Quadratic Minimization z ∗ := Minimize xT Qx Subject to xT A1 x ≤ 1. SDP-relaxation: Remove the rank-one constraint. The dual of (SDP) can be written as: z SDP = Maximize y1 + y2 Subject to y1 A1 + y2 A2 + S = Q.2) xT A2 x ≤ 1. (SDP) A2 • X ≤ 1. z∗ = Minimize Q•X Subject to A1 • X ≤ 1. x∗ = xj /(x0 ) j j is a solution to (BQP).

. there is a decomposition of X ∗ r ∗ xj xT .. 4. j X = j=1 where r is the rank of X ∗ . or xi be 1. E[w(x)] ≥ 0. Then. circuit design.4 The SDP relaxation is exact. and scheduling applications. This problem can be formulated by assigning each node a binary variable xj : 1 z ∗ = Maximize w(x) := wij (1 − xi xj ) 4 i. n.j (MC) Subject to x2 = 1.76 CHAPTER 4. independently with probability . which is the problem of partitioning the nodes of V into two sets S and V \ S so that w(S) := wij i∈S.5 · z ∗ . Observation: z SDP ≤ z ∗ .4 Max-Cut Problem Consider the Max Cut problem on an undirected graph G = (V.. Theorem 4. meaning z SDP ≥ z ∗ . swap nodes from the majority side to the minority side using the greedy method. x∗ = αxj j for some α is a solution to (QP-2). SDP FOR COMBINATORIAL OPTIMIZATION ∗ y1 (1 − A1 • X ∗ ) = 0 ∗ y2 (1 − A2 • X ∗ ) = 0 S ∗ • X ∗ = 0. j∈V \S is maximized.5. The Coin-Toss Method: Let each node be selected to one side. j) ∈ E). A problem of this type arises from many network planning. Moreover.. i i = 1. such that for any j. E) with nonnegative weights wij for each edge in E (and wij = 0 if (i. .

) V i /||Vi || =1 = -1 vT u vT u j i Figure 4. 2. n. .. z SDP = maximize s. such that X = V TV .4. . MAX-CUT PROBLEM 4. (4. Q•X Ij • X = 1. V j /||Vj || U = -1 =1 arccos(. vn ) ∈ Rn×n . . j = 1. Generate a random vector u ∈ N (0. I): ∗ x = sign(V T u). .1 77 Semi-deﬁnite relaxation z SDP := The dual is minimize s. . π i. As the unit vector u is uniformly generated along the circle.5) 1 if xj ≥ 0 −1 otherwise. . X 0.t.2 (4.t. . . eT y Q D(y). i.4. the product is either 1 or −1. ˆ sign(xj ) = 4. n. j = 1..3) (4.e. Approximation analysis Then...4. vj is the jth column of V .1: Illustration of the product σ( vi ) · σ( vj ) on the 2-dimensional unit circle. .4) Let V = (v1 . . one can prove from Sheppard [274] (see Goemans and Williamson [125] and Bertsimas and Ye [50]): E[ˆi xj ] = xˆ 2 ¯ arcsin(Xij ).4.

ii) If Q is positive semideﬁnite E(ˆT Qˆ) ≥ x x so that z∗ ≥ 4.. n2 ) that is. SDP FOR COMBINATORIAL OPTIMIZATION Lemma 4.. i = 1.. when Qi 0 for all i. The previous best approximation ratio.78 CHAPTER 4.7 We have i) If Q is a Laplacian matrix. we can compute a feasible solution x. min(m2 . π π 2 SDP z . then E(ˆT Qˆ) ≥ . m. . . π Multiple Ellipsoid-Constrained Quadratic Maximization (HQP ) z ∗ := maximize subject to xT Q0 x xT Qi x ≤ 1.6 Let X 0 and d(X) ≤ 1. i = 1.5 For x ∈ [−1.. x x so that z ∗ ≥ . n2 ) SDP relaxation (SDP ) z SDP := maximize Q0 • X subject to Qi • X ≤ 1. min(m2 . Then arcsin[X] X.5 2 SDP 2 z ≥ z∗.878z SDP .5. .878.1 1 · z∗. was 1 . 1) 1 − (2/π) · arcsin(x) ≥ .. Theorem 4. m X 0. 1−x Lemma 4. such that x T Q0 x ≥ 4.878z ∗ .878z SDP ≥ . in polynomial time..

4.. and have j max xT Q0 xj ≥ j j=1. . there exist column vectors xj . each xj is a feasible solution of (HQP)... r. j = 1. 4.5. we can select an xj such that xT Q0 xj is the largest. z SDP = z SDD . Then we can compute another maximizer of (SDP) whose rank is no more than m by solving a linear program. i = 1.2 Bound on Rank Lemma 4. Suppose the rank of initial X ∗ is r. Then. m.. Then... Then. r. yi ≥ 0. j j Then consider linear program maximize subject to r j=1 r j=1 a0j vj aij vj ≤ 1. ..r 1 r r xT Q0 xj = j j=1 . j X = j=1 Note that for each i ≥ 1 and j xT Qi xj = Qi • xj xT ≤ Qi • X ∗ ≤ 1. j = 1.. such that r X∗ = xj xT .8 Let X ∗ be an maximizer of (SDP). Then (SDP) and (SDD) have no duality gap. Note that v1 = . we should have at most m of vj are positive at an optimal basic solution. . We assume that (SDD) is feasible. such that r ∗ xj xT . = vr = 1 is an optimal solution for the problem. z ∗ = z SDP if and only if (SDP) has a feasible rank-one matrix solution and meet complementarity condition.. Let X ∗ be an SDP maximizer whose rank r ≤ min(m... j j Thus. Thus... n). MULTIPLE ELLIPSOID-CONSTRAINED QUADRATIC MAXIMIZATION79 and its dual (SDD) z SDD := m i=1 minimize subject to yi m Z = i=1 yi Qi − Q0 Z 0. that is. m vj ≥ 0. r.. i = 1. . But we should have at least r inequalities active at an optimal basic solution. j = 1... . there exist column vectors xj ... Moreover.5.... j j=1 Let aij = Qi • xj xT = xT Qi xj .

SDP FOR COMBINATORIAL OPTIMIZATION 1 1 1 1 · Q0 • X ∗ = · z SDP ≥ · z ∗ ≥ · z∗. satisﬁes the inequality. we assume r(r + 1)/2 > m. we have Improvements Lemma 4. m. Note that V T CV .6) .11 For solving HQP-m... where Qi approximation ratio 1 . min(m. min(m − 1. Barvinok [34]) Let X ∗ be a minimizer of the SDP in the standard form.. Then we can compute a minimizer of the problem whose rank r satisﬁes r(r + 1)/2 ≤ m in polinomial time. we have For large m.9 For solving HQP-m. i = 1.. n) 4. we have Theorem 4. consider SDP in the standard form: z ∗ := Minimize Subject to C •X Ai • X = bi . Thus. where Qi approximation ratio 1 . V ∈ Rn×r . V T Ai V s and U are r × r symmetric matrices and V T CV • I = C • V V T = C • X ∗ = z ∗ . Theorem 4. m U 0.. m. .10 Let X ∗ be an maximizer of (SDP). r. Then consider Minimize V T CV • U Subject to V T Ai V • U = bi .12 (Pataki [252].. n) 0 for i = 1. . . n) Thus. we have Theorem 4. i = 1..3 0 for i = 1. m X 0. Proof.. (4. If the rank of X ∗ ..80 CHAPTER 4. then we need do nothing. r r r min(m.5. and let V V T = X ∗.. . Then we can compute another maximizer of (SDP) whose rank is no more than m − 1 by solving a linear program... Thus.

5. X(U (α∗ )) = V U (α∗ )V T ..e. let W be either indeﬁnite or negative semideﬁnite (if it is positive semideﬁnite. we take −W as W ). Consider the dual of (4.6). m where W is a r × r symmetric matrices (does not need to be deﬁnite).8) Let y ∗ be a dual maximizer.. Consider the system of homogeneous linear equations V T Ai V • W = 0.6) m z ∗ := Maximize bT y = i=1 bi yi m i=1 V T CV Subject to yi V T Ai V T . m. for any feasible solution of (4. we must be able to ﬁnd a symmetric matrix W = 0 to satisfy all m equations... that is. Now we show that any feasible solution U to (4. .. Then.7) Thus. Without loss of generality.6) satisﬁes the strong duality condition so that it must be also optimal. Since U = I is an interior optimizer for the primal. and consider U (α) = I + αW. W has at least one negative eigenvalue. and V T Ai V • U (α∗ ) = V T Ai V • (I + α∗ W ) = V T Ai V • I = bi . ¯ ¯ Choosing α∗ = 1/|λ| where λ is the least eigenvalue of W . the strong duality condition holds. This system has r(r + 1)/2 real number variables and m equations.6) one can construct a feasible solution for the original SDP using X(U ) = V U V T and C • X(U ) = V T CV • U.6). i = 1. i=1 Then.4. MULTIPLE ELLIPSOID-CONSTRAINED QUADRATIC MAXIMIZATION81 Moreover. (4.. Thus. . we have U (α∗ ) 0 and it has at least one 0 eigenvalue or rank(U (α∗ )) < r. That is. the minimal value of (4.6) is also z ∗ . and U = I is a minimizer of (4. m T ∗ yi V T Ai V T ) = 0 I • (V CV − i=1 so that we have m V T CV − ∗ yi V T Ai V T = 0.7) is a minimizer for the original SDP. i = 1. (4. i. thereby X(U ) of (4.6) is a minimizer for (4. any feasible solution of (4.6). as long as r(r +1)/2 > m. U (α∗ ) is a feasible and so it is an optimal solution for (4..

m.4 Randomized Rounding ¯ ¯ ¯ Let X ∗ = V T V and V is r × n. Thus. and rank(X(U (α∗ ))) < r.. . SDP FOR COMBINATORIAL OPTIMIZATION is a new minimizer for the original SDP.. i = 1. and U ∗ = I is an optimal solution for the problem. i = 1. in (strongly) polynomial time given the least eigenvalue of W . i. Let u= ˆ Then. 4.82 CHAPTER 4. ˆ ¯ ˆ . ¯ ¯ Without loss of generality. The total number of such reduction steps is bounded by n − 1 and each step uses no more than O(m2 n) arithmetic operations. which is necessarily given by r(r + 1)/2 ≤ m.. we must be able to ﬁnd an SDP solution whose rank satisﬁes the inequality. the total number of arithmetic operations is a polynomial in m and n. This process can be repeated till the system of homogemeous linear equations has only all zero solution. Rewrite the problem ¯ and Ai = V z SDP := maximize D • U subject to Ai • U ≤ 1. maxi (Ai • uuT ) Using the linear transformation x = V T u. Note that U = I is an optimal solution for the problem. Ai • uuT ≤ 1. and consider z SDP := maximize subject to ¯ ¯ V Q0 V T • U ¯ ¯ V Qi V • U ≤ 1. uuT is a feasible solution.. ˆˆ That is.. independently. D • uuT = D • I = z SDP ... m U 0. . i = 1.. .. . for i = 1. m.... Generates a random r-dimension vector u where. 1 Ai • uuT maxi · u. uj = Note that and 1 with probability. and ˆˆ D • uuT = ˆˆ z SDP . Therefore. E[uuT ] = I. m U 0.5 −1 otherwise. assume that D := V Q0 V T is a diagonal matrix ¯ Qi V .e.5..

.14 The probability Pr(Ai • uuT > α) ≤ Pr(Ai • uuT > α(Ai • I)) ≤ 2r exp(−α/2). then Pr(max(Ai • uuT ) > 2 ln(4m2 )) ≤ i 1 r ≤ . 2 Theorem 4.13 For any given vector a. Lemma 4. Lemma 4. i Choose α = 2 ln(4m2 ).. E) with non-negative weights wij for each edge in E (and wij = 0 if (i. j∈V \S . 2 ln(4m2 ) 2 ln(4m2 ) Max-Bisection Problem Consider the Max-Bisection problem on an undirected graph G = (V. j) ∈ E).4..6 1 1 · z SDP ≥ · z∗. MAX-BISECTION PROBLEM 83 xxT is a feasible solution for (SDP).5 E(ˆT Q0 x) = x ˆ 4. 2m 2 or Pr 1 1 ≥ maxi (Ai • uuT ) 2 ln(4m2 ) ≥ 1 ..15 The probability Pr(max(Ai • uuT ) > α) ≤ 2mr exp(−α/2). m Ai • I ≤ 1.16 With probability at least .6. maxi (Ai • uuT ) where for i = 1. which is the problem of partitioning the even number of nodes in V into two sets S and V \ S of equal cardinality so that w(S) := wij i∈S. the probability Pr(aaT • uuT > α a 2 ) ≤ 2 exp(−α/2). and ˆˆ Q0 • xxT = ˆˆ z SDP . Lemma 4.

Therefore. Note that xj takes either 1 or −1. superscript T is the transpose operator. circuit design. since otherwise we have to either add or subtract nodes from S.. is that two objectives are actually sought—the objective value of w(S) and the size of S. . so that we can choose either S = {j : xj = 1} or S = {j : xj = −1}.84 CHAPTER 4.j n subject to xj = 0 or eT x = 0 j=1 x2 = 1. or xi be 1. plus an improved proof technique.5. n. The Coin-Toss Method: Let each node be selected to one side. resulting in a deterioration of w(S) at the end. the popular and widely used MinBisection problem sometimes can be solved by ﬁnding the Max-Bisection over the complementary graph of G. at the same time. comparing to Max-Cut. In particular. This problem can be formulated by assigning each node a binary variable xj : w∗ := Maximize 1 4 (MB) wij (1 − xi xj ) i. and scheduling applications. we need to balance the (expected) quality of w(S) and the (expected) size of S. Our method is built upon a careful balance of the two.. We want high w(S). j = 1. SDP FOR COMBINATORIAL OPTIMIZATION is maximized. E[w(x)] ≥ 0. independently with probability . in any (randomized) rounding method at the beginning.j Xii = 1. zero or small diﬀerence between |S| and n/2. swap nodes from the majority side to the minority side using the greedy method. X i. . The constraint eT x = 0 ensures that |S| = |V \ S|.6. j where e ∈ n (n even) is the column vector of all ones. . Then. . Xij = 0. . . n. 0..1 SDP relaxation Maximize Subject to 1 4 wij (1 − Xij ) i.5 · z ∗ .j i = 1. but. 4. What complicates matters in Max-Bisection. A problem of this type arises from many network planning.

Since X is positive semidefinite. . using (4. ˆ ˆ using a greedy method. and Bertsimas and Ye [50]): E[ˆi xj ] = xˆ and 1− 2 ¯ arcsin(Xij ). ˆ Then. Thus. one can prove Sheppard [274] (see Goemans and Williamson [125]. (4.j 2 ¯ arcsin(Xij ) π ¯ wij · α(1)(1 − Xij ) i.6. and then proceed with our improved method. Frieze and Jerrum [110]. 2.j = α(1) · since 2 ¯ arcsin(Xij ) π n2 . that is.11) ¯ ¯ Xij = eeT • X = 0. the randomization method of Goemans and Williamson [125] essentially generates a random vector u from a multivariate normal distribution with 0 ¯ mean and covariance matrix X. S may not be a bisection. .4. x may not satisfy eT x = 0.j ¯ α(1)(1 − Xij ) i. MAX-BISECTION PROBLEM 4. n.5) to generate S = {i : xi = 1} ˆ or S = {i : xi = −1}. .2 85 The .651-method of Frieze and Jerrum We ﬁrst review the Frieze and Jerrum method.j However.878567 1−y −1≤y<1 which is a special case of Deﬁnition (4. ¯ ¯ Let X be an optimal solution of Problem (SDP).9) 2 ¯ ¯ arcsin(Xij ) ≥ α(1)(1 − Xij ). . i. π where α(1) := min 1− 2 π arcsin(y) ≥ .e. (4.j = α(1) · wSD ≥ α(1) · w∗ . Frieze and Jerrum have adjusted S by swapping nodes .21) in Section 5. π i. E[w(S)] = 1 4 ≥ 1 4 wij 1 − i.6. Then. j = 1..10) and n2 (eT x)2 ˆ − 4 4 = 1 4 ≥ E 1 4 1− i. i. 4 (4.

5). Frieze and Jerrum make sure that the least weighted node gets swapped ﬁrst.. i..86 CHAPTER 4.11).11) assures that not too many nodes need to be swapped.. and record the largest method (4. i2 . and for each i ∈ S.14) implies that m ≥ 2α(1) − λ. they deﬁne two random variables: 1 w := w(S) = wij (1 − xi xj ) ˆ ˆ 4 i. they are almost sure to have one z to meet its expectation before too long. i2 . Since E[z] ≥ 2α(1) and z ≤ 3. let ζ(i) = j∈S wij and S = {i1 .14) . .14) holds and suppose w(S) = λw∗ . Clearly.12) ˜ In order to analyze the quality of bisection S. 4 Thus. (w. create x or S using the randomization ˆ ˜ using the swapping procedure.j (since (eT x)2 = (2|S| − n)2 ).. ≥ ζ(i|S| ).e. They ¯ repeatedly generate samples u ∈ N (0.13) then E[z] ≥ 2α(1). to have z ≥ 2α(1). Note that inequality (4. w∗ m (4.j and m := |S|(n − |S|) = n2 (eT x)2 ˆ 1 − = 4 4 4 (1 − xi xj ) ˆ ˆ i. assign S = {i1 . X). SDP FOR COMBINATORIAL OPTIMIZATION from the majority block into the minority block until they are equally sized. i|S| }. Then. Also.. m∗ (4. where ζ(i1 ) ≥ ζ(i2 ) ≥ ˜ .13) and (4. For simplicity. . (4. in/2 }. Moreover. the construction of ˜ guarantees that bisection S ˜ w(S) ≥ n · w(S) .g) let |S| ≥ n/2. in selecting swapping nodes... when (4. which from (4.o.10) and (4. and where m∗ = n2 .l. 2|S| n/2 ≤ |S| ≤ n. construct S ˜ value of w(S) among these samples.. ∗ and z≤3 ∗ since w/w ≤ 2|S|/n ≤ 2 and m/m ≤ 1. ˆ E[w] ≥ α(1) · w∗ E[m] ≥ α(1) · m∗ . Frieze and Jerrum’s analysis can be described as follows. if a new random variable z= w m + ∗. Then. from (4. More precisely..

x = sign(u). one can see that ˜ w(S) ≥ = ≥ ≥ w(S) 2δ λw∗ 2δ (2α(1) − 4δ(1 − δ))w∗ 2δ 2( 2α(1) − 1)w∗ .651. i. Note that for ˜ α(1) ≥ . θX + (1 − θ)P ).6. the overall performance of the rounding method is determined by two factors: the expected quality of w(S) and how much S need to be downsized. the more use of X in the combination results in higher expected w(S) but larger expected diﬀerence between |S| and n/2. which was also proposed in Nesterov [240]. . and P is a feasible solution for (SDP). and then S from the Frieze and Jerrum swapping procedure.15).651-approximation algorithm for Max-Bisection. then the above inequality implies λ ≥ 2α(1) − 4δ(1 − δ).6. n−1 n 1 where I is the n-dimensional identity matrix. and by Zwick [345] for approximating the Max-Cut problem when the graph is sparse. Since the largest w(S) in the process is at least as good as the one who meets z ≥ 2α(1). ¯ The convex combination parameter θ used in θX + (1 − θ)P provides a balance ¯ between these two factors. 2( 2α(1) − 1) > .3 A modiﬁed rounding and improved analyses ¯ Our improved rounding method is to use a convex combination of X and a positive semideﬁnite matrix P as the covariance matrix to generate u and x. Typically. The other choice is P = I. they proceed to prove a . (4.e. and the more use of P results in less expected w(S) and more accurate |S|.4. Note that I − n eeT is the projection matrix onto the null space of eT x = 0.878567. One choice of P is n 1 P = (I − eeT ).15) Applying (4. 4. MAX-BISECTION PROBLEM 87 Suppose that |S| = δn. Again. The last inequality follows from simple calculus that δ = 2α(1)/2 yields the minimal value for (2α(1) − 4δ(1 − δ))/(2δ) when 0 ≤ δ ≤ 1.12) and (4. ˆ S = {i : xi = 1} ˆ or S = {i : xi = −1} ˆ ˜ such that |S| ≥ n/2. ˆ ¯ u ∈ N (0.

Thus. Again.11): 1 4 E[w(S)] = wij (1 − E[ˆi xj ]) ≥ α · w∗ xˆ (4.j n2 .17) such that α would be slightly less than α(1) but β would be signiﬁcantly greater than α(1). i.. we have E[z(γ)] ≥ α + γβ and z ≤ 2 + γ. our hope is that we could provide two new inequalities in replacing (4. ˆ ˆ i.e.17) hold. from (4. then ˜ w(S) ≥ 2 γ(α + γβ) − γ · w∗ . In particular. z(γ) ≥ α + γβ.j Then.651 for MaxBisection. if γ= 1 α (√ − 1) 2β 1−β (which is greater than α/(4 − β) since β > 0). Now we prove the following lemma: Lemma 4. ∗ w m (4. where m∗ = n2 . Then.88 CHAPTER 4.16) i.17). let new random variable z(γ) = w m + γ ∗. if random variable z(γ) meets its expectation. Before proceed. then ˜ w(S) ≥ α √ · w∗ . 4 (4. Note that Frieze and Jerrum used γ = 1 in their analyses. for any given γ ≥ α/(4 − β).16) and (4. 4 Furthermore. for a given parameter γ ≥ 0.10) and (4.16) and (4. we prove a technical result. let two random variables w := w(S) = 1 4 wij (1 − xi xj ) and ˆ ˆ m := |S|(n − |S|) = i.j 1 4 (1 − xi xj ). 1+ 1−β .18) Then. SDP FOR COMBINATORIAL OPTIMIZATION More precisely. we could give a better overall bound than .j and E[|S|(n − |S|)] = E (eT x)2 ˆ n2 1 − = 4 4 4 (1 − E[ˆi xj ]) ≥ β · xˆ i.17 Assume (4. E[w] ≥ α · w∗ and E[m] ≥ β · m∗ .

6. So their choice γ = 1 is almost “optimal”. Applying (4. 1+ α √ > 0.878567 as in the case of Frieze and Jerrum. 1]. Suppose w(S) = λw∗ 89 and |S| = δn.12) we see that ˜ w(S) w(S) 2δ λw∗ = 2δ α + γβ − 4γδ(1 − δ) · w∗ ≥ 2δ ≥ 2( γ(α + γβ) − γ) · w∗ .5-approximation To see the impact of θ in the new rounding method. MAX-BISECTION PROBLEM Proof.6.6515. 1−β which is just slightly better than 2( 2α(1) − 1) ≥ 0. when both α = β = α(1) ≥ . we analyze the other extreme case where θ = 0 and P = 1 n (I − eeT ). In particular. we have the second desired result in the lemma.4 A simple . ≥ The last inequality follows from simple calculus that √ α + γβ δ= √ 2 γ yields the minimal value for (α + γβ − 4γδ(1 − δ))/(2δ) in the interval [0. n−1 n .18) and z(γ) ≥ α + γβ implies that λ ≥ α + γβ − 4γδ(1 − δ). substitute α 1 − 1) (√ 2β 1−β γ= into the ﬁrst inequality.4. if γ ≥ α/(4 − β). The motivation to select γ = α √1 2β ( 1−β − 1) is that it yields the maximal value for 2( γ(α + γβ) − γ). 4. In fact. and is not used in the rounding method.6511 proved by Frieze and Jerrum. which from (4. We emphasize that γ is only used in the analysis of the quality bound.

here the ﬁrst inequality on w(S) is worse. SDP FOR COMBINATORIAL OPTIMIZATION That is. Therefore. n Comparing (4. as the covariance matrix to generate u. we have in Lemma 4. we generate u ∈ N (0. P ).j 4 π n−1 wij ≥ . wij = i<j i=j In other words.19) and (4.e. Now. x here is a bisection with probability almost 1 when n is ˆ large.5 vs . i. but the second inequality is substantially better. we have ˆ   1 1 2 1 E[w(S)] = E  wij (1 − xi xj ) = ˆ ˆ 1 + arcsin( ) 4 i.20) 4 4 4 4 4 π n−1 n 4 E where from (4. (4. Using Lemma 4.21) α(θ) := min −1≤y<1 1−y . we discuss using the convex combination of θX + (1 − θ)I ˜ for a given 0 ≤ θ ≤ 1. x = sign(u).878567. The same ratio can be established for P = I. S and S ˆ ¯ u ∈ N (0. 4. .5 A . π n−1 i=j wij ≥ w∗ . For simplicity. ˆ S = {i : xi = 1} ˆ or S = {i : xi = −1} ˆ ˜ such that |S| ≥ n/2. then x and S.699-approximation We now prove our main result.19) and n2 (eT x)2 ˆ n2 n n(n − 1) 2 1 1 n2 − ≥ − + arcsin( ) ≥ (1 − ) · .11).6.878567. θX + (1 − θ)P ).5 and β =1− 1 .5 √ > 1+ 1−β 1 + 1/n approximation method. and then S from the Frieze and Jerrum swapping procedure.10) and (4. i.. 1 1 − n vs .. x. we see that the method is a α .5·w∗ i=j (4. Deﬁne 2 1 − π arcsin(θy) . (4.e.9) we have used the facts that E[ˆi xj ] = xˆ and 1 2 2 −1 arcsin( ).20) to (4. we will use P = I in our rounding ¯ method.90 CHAPTER 4.17 α = .17.

Noting that ¯ Xij = −n. b(. Since 1 − have 2 π and β = β(θ).22) we have E[|S|(n − |S|)] = = n2 (eT x)2 ˆ − 4 4 1 2 ¯ 1 − arcsin(θXij ) 4 π E i=j = 1 4 = 1 4 ≥ 1 4 1− i=j 2 2 2 ¯ arcsin(θ) + arcsin(θ) − arcsin(θXij ) π π π b(θ) + i=j 2 2 ¯ arcsin(θ) − arcsin(θXij ) π π ¯ b(θ) + c(θ)(1 − Xij ) i=j . We now prove another technical lemma: Lemma 4.660695. one can also verify that α(. 1 and α(0) = .4.22) where b(θ) = 1 − 2 arcsin(θ) π and c(θ) = min −1≤y<1 2 arcsin(θ) − arcsin(θy) . from deﬁnition (4.16) and (4.17) hold for α = α(θ) Proof.6.301408 and c(.5 and β(0) = 1 − n .j = α(θ) · wSD ≥ α(θ) · w∗ .878567 as shown in Goemans and Williamson [125].89) ≥ . n (4.21) we E[w(S)] = ≥ 1 4 1 4 wij 1 − i. Similarly.89) ≥ . inequalities (4. MAX-BISECTION PROBLEM 91 and β(θ) := (1 − 1 )b(θ) + c(θ).18 For any given 0 ≤ θ ≤ 1 in our rounding method.89) ≥ .835578. arcsin(θ) ≥ 0 for any 0 ≤ θ ≤ 1.j 2 ¯ arcsin(Xij ) π ¯ wij · α(θ)(1 − Xij ) i. i=j from the deﬁnition (4. π 1−y Note that α(1) = β(1) ≥ .

18) meets its expectation.. if the number of nodes in the graph is suﬃciently large.22). our rounding method will generate a α(θ) ˜ w(S) ≥ 1+ 1 − β(θ) · w∗ as soon as (bounded) z(γ) of (4. .92 CHAPTER 4.89) 1+ 1 − β(.21) and β(θ) of (4. In particular. let 1+ 1−β(θ) θ∗ = arg max θ∈[0.8355. if parameter θ = . of the maximal bisection cut w∗ over the total weight i<j wij . The reader may ask why we have used two diﬀerent formulations in deﬁning α(θ) of (4.89) + c(.89 is selected in the new rounding method. 4 Lemmas 4.17 and 4. if θ = . ρ.e.89 is used.18 together imply that for any given θ between 0 and 1. SDP FOR COMBINATORIAL OPTIMIZATION = = = 1 (n2 − n)b(θ) + (n2 − n)c(θ) + nc(θ) 4 1 n2 (1 − )b(θ) + c(θ) · n 4 2 n β(θ) · . that is.19 There is a polynomial-time approximation algorithm for MaxBisection whose expected cut is at least rM B times the maximal bisection cut.89) > . n which imply rM B = α(θ∗ ) 1+ 1 − β(θ∗ ) ≥ α(.89) = (1 − 1 )b(. In particular.89) > . The reason is that we have no control on the ratio. 1] such that √ is maximized.9620. and for n suﬃciently large (≥ 104 ) β(.1] and rM B = α(θ) 1+ 1 − β(θ) α(θ∗ ) 1+ 1 − β(θ∗ ) . i. Thus. our rounding method is a .699-approximation for Max-Bisection. This bound yields the ﬁnal result: Theorem 4.89) > . α(. ρ := w∗ i<j wij .69920. we can set θ α(θ) to a value θ∗ in [0.

−e ≤ x ≤ e. .00. . For Max-Bisection on these graphs. for θ = .98.99 and report the best rounding solution among the 100 tries..7 Quadratic Optimization In this section we consider approximating the global minimizer of the following QP problem with certain quadratic constraints: q(Q) := minimize s. . . xT Qx + t · cT x n 2 j=1 aij xj = bi . 2ρ Thus.8113. . In general. 4. in the worse case ρ = 1. our method is a .4.8355 established by using the ﬁrst derivation. QUADRATIC OPTIMIZATION 93 Note that ρ ranges from 1/2 to 1. . then x is also optimal for the . . i = 1. Normally. 2 Then. i = 1. m. again.01. we can only establish α≥ 1 b(θ) + c(θ). −e ≤ x ≤ e. q(x) := xT Qx n 2 j=1 aij xj = bi .17 α≥ 1 b(θ) + c(θ).t. However. then we have α ≥ . However. we have a close to 1 approximation 1 1 if θ = 0 is chosen.. −1 ≤ t ≤ 1 by adding a scalar variable t..89. . if ρ ≤ . .7. Indeed. We assume that the problem is feasible and denote by x(Q) its global minimizer. There always is an optimal solution (x. When ρ near 1/2. .8. where symmetric matrix Q ∈ Mn .89. we can also prove that in Lemma 4. t) for this problem in which t = 1 or t = −1. we can run our rounding method 100 times for parameter θ = .710 approximation for setting θ = . In any case. we have α ≥ . but without the need to know ρ. which is less than . This bound can be further improved by setting a smaller θ. If t = 1.8490. . α ≥ 2ρ and β ≥ 1 − n . A = {aij } ∈ Rm×n and b ∈ Rm are given and e ∈ Rn is. This will ensure us to produce a solution with a near best guarantee.t. there is a linear term in the objective function: q(x) = xT Qx + cT x. the problem can be homogenized as minimize s. the quality bound improves as ρ decreases. since b(0) = 1. m. c(0) = 0. . the vector of all ones.18. using the second derivation in Lemma 4.

23) Here. .23) and dual (4. X 0. For simplicity. ¯ ii) p = eT z + bT y and it is the maximal objective value of Q • X in the feasible ¯ ¯ ¯ m set of the relaxation. j = 1. . d(X) ≤ e.t. z(Q)) an optimal solution pair for the primal (4. −z(−Q)).24). Furthermore. . z ) = (−y(−Q). Denote by X(Q) and (y(Q). ¯ ¯ . ¯ ¯ (¯. . Thus. then −x is optimal for the original problem. z ≤ 0.1 Positive semi-deﬁnite relaxation The algorithm for approximating the QP minimizer is to solve a positive semideﬁnite programming relaxation problem: p(Q) := minimize s. SDP FOR COMBINATORIAL OPTIMIZATION original problem. that is. i = 1.94 CHAPTER 4. p := −p(−Q). ain ). Let q := −q(−Q) and q := q(Q) denote their maximal and minimal ¯ objective values.t. (4. The function q(x) has a global maximizer over the bounded feasible set as well. We now present a “fast” algorithm to compute a 4/7-approximate minimizer. q := −q(−Q). and unknown X ∈ Mn is a symmetric matrix. q−q ¯ 4. . ai = (ai1 . eT z + bT y Q D(z) + m i=1 yi Ai . m. We have the following relations between the QP problem and its relaxation: Proposition 4. if t = −1.20 Let q := q(Q). such that ˆ q(ˆ) − q x ≤ 4/7. Then. d(X) is a vector containing the diagonal components of X. in what follows we let x = x(Q) and X = X(Q). y ¯ i) q is the maximal objective value of xT Qx in the feasible set of the QP problem. p := p(Q). we can assume q(x) = xT Qx throughout this section. Q•X Ai • X = bi . The dual of the relaxation is p(Q) = maximize s. Note that the relaxation is feasible and its dual has an interior feasible point so that there is no duality gap between the primal and dual. Ai = diag(ai ). . (4. . without loss of generality. respectively. z ¯ iii) p ≤ q ≤ q ≤ p. .24) where D(z) is the diagonal matrix such that d(D(z)) = z ∈ Rn . . to compute a feasible x. . Note that d(X) ≤ e can be written as Ij • X ≤ 1.7. n. . and D(¯) + i=1 yi Ai − Q 0. . where Ij is the all-zero matrix except the jth diagonal component equal to 1.

vj ≤ 1. . . d(X) ≤ e. . such that X = V T V . QUADRATIC OPTIMIZATION 95 Proof. ˆ where (4. . . . there is a matrix factorization V = (v 1 .2 Approximation analysis First. .21 Let u be uniformly distributed on the unit sphere in Rn . σ(x) ∈ Rn is the vector whose jth component is sign(xj ): sign(xj ) = 1 if xj ≥ 0 −1 otherwise. . . . n Ai • X = xT Ai x = aij (xj )2 = bi . . The ﬁrst and second statements are straightforward to verify. we have q(Q) ≥ p(Q). Thus. Lemma 4. or p ≥ q . . ˆ 4. q(Q) = minimize Eu (σ(V T u)T DQDσ(V T u)) s. v n ) ∈ Rn×n . n. satisﬁes x Eu q(ˆ) − q x π 4 ≤ −1≤ . we generate a random vector u uniformly distributed on the unit sphere in Rn and assign x = Dσ(V T u). Thus. . q−q ¯ 2 7 That is. i = 1. and. Then X 0. . we can prove q(−Q) ≥ p(−Q). x is a 4/7-approximate minimizer for the QP problem expectantly.e.. xnn ). Eu q(ˆ). Then. . . It is easily see that x is a feasible point for the QP problem and we will show ˆ later that its expected objective value. ¯ ¯ Since X is positive semi-deﬁnite. . . for any x ∈ Rn . . . v j is the jth column of V . m. One ˆ can generate u repeatedly and choose the best x in the process. . we present a lemma which will be only used in our analysis. . Then. v n ) = diag( x11 .7. i. . where D = diag( v1 . . j=1 and Q • X = xT Qx = q(x) = q(Q). . m.t. Similarly. . . . after obtaining X and V .25) √ √ D = diag( v 1 .4. vn ). i = 1. Let X = xxT ∈ Mn .7. . we will ˆ almost surely generate a x that is a 4/7-approximate minimizer. Ai • (V T V ) = bi . j = 1. or p ≤ q.

Lemma 4. . 2 −1 XD−1 ]D) π Q • (D arcsin[D Ai • X = bi . X where 0. These two relations give the desired result. i = 1. For any X = V T V 0. T T vj Eu (σ(vi u)σ(vj u)) = xi xj vi which implies that for this particular feasible V q(Q) = q(x) = Eu (σ(V T u)T DQDσ(V T u)). n. .t. vi vj . Theorem 4. where we use “inﬁmum” to replace “minimum. For any function of one variable f (t) and X ∈ Rn×n . for any feasible V . On the other hand. let f [X] ∈ Rn×n be the matrix with the components f (xij ). Dσ(V T u) is a feasible point for the QP problem. xnn ). we have n n Eu (σ(V T u)T DQDσ(V T u)) = qij vi T T vj Eu (σ(vi u)σ(vj u)). √ √ D = diag( x11 . d(X) ≤ e. (4. Since. i = 1. Then xi x x. . . . we have q(Q) ≤ Eu (σ(V T u)T DQDσ(V T u)).22 Let X 0 and d(X) ≤ 1. m. Note that V is feasible for the above T T Eu (σ(vi u)σ(vj u)) = Thus. for any ﬁxed u with u = 1. . .23 q(Q) = inﬁmum s. we have T T Eu (σ(vi u)σ(vj u)) T T = 1 − 2Pr{σ(vi u) = σ(vj u)} = 1 − 2Pr σ( T T vj u vi u ) = σ( ) . We have the next technical lemma whose proof is an exercise. 1 if σ(xi ) = σ(xj ) −1 otherwise.” since for simplicity we require X to be positive deﬁnite in our subsequent analysis. Now we are ready to prove the following theorem. d(X) ≤ e. Then arcsin[X] X. . . .96 CHAPTER 4. Proof. .26) i=1 j=1 Let us choose vi = problem. . SDP FOR COMBINATORIAL OPTIMIZATION Proof.

Recall z = −z(−Q) ≥ 0. Then. as illustrated in Figure 4. p π q−p≥ ¯ 2 (¯ − p). . QUADRATIC OPTIMIZATION We have the following relation: Pr σ( 97 1 T T vj u vi u ) = σ( ) vi vj = T vi vj vi vj 1 arccos π . • D arcsin[D−1 XD−1 ]D . from Theorem 4. . Theorem 4. π Finally.2 of Goemans and Williamson [125]. We prove (i). for any X 0.4.26) we further have n Eu (σ(V T u)T DQDσ(V T u)) n = qij vi vj i=1 j=1 = 2 arcsin π T vi vj vi vj 2 Q • (D arcsin[D−1 XD−1 ]D).7. Thus. . p π ii) iii) p−p≥q−q ≥ ¯ ¯ 4−π (¯ − p). noting arcsin(t) + arccos(t) = T T Eu (σ(vi u)σ(vj u)) = 2 arcsin π T vi vj vi vj π 2 we have . p = −p(−Q) = ¯ ¯ ¯ m T T e z + b y .1. .22 lead to our main result: Theorem 4. xnn ). d(X) ≤ e and ¯ ¯ z ¯ √ √ D = diag( x11 .23 π π q = q(Q) 2 2 ≤ Q • (D arcsin[D−1 XD−1 ]D) m = Q − D(¯) − z i=1 1 m yi Ai + D(¯) + ¯ z yi Ai ¯ i=1 Lemma 1. and D(¯) + i=1 yi Ai − Q 0. y = −y(−Q).23 and Lemma 4.24 We have i) p−q ≥ ¯ 2 (¯ − p). Lemma 4. and using equality (4. p π Proof.21 gives us the desired result.

98 CHAPTER 4. then Q • X → p and we prove (i). SDP FOR COMBINATORIAL OPTIMIZATION m = Q − D(¯) − z • D arcsin[D−1 XD−1 ]D yi Ai ¯ i=1 m + D(¯) + z • D arcsin[D−1 XD−1 ]D yi Ai ¯ i=1 m ≤ Q − D(¯) − z • DD−1 XD−1 D yi Ai ¯ i=1 m + D(¯) + z • D arcsin[D−1 XD−1 ]D yi Ai ¯ i=1 m (since Q − D(¯) − z yi Ai ¯ 0 and arcsin[D−1 XD−1 ] D−1 XD−1 . .) i=1 m = Q − D(¯) − z yi Ai ¯ •X i=1 m + D(¯) + z • D arcsin[D−1 XD−1 ]D yi Ai ¯ i=1 m = Q•X − D(¯) + z yi Ai ¯ •X i=1 m + D(¯) + z • D arcsin[D−1 XD−1 ]D yi Ai ¯ i=1 m = Q • X − z T d(X) − ¯ yi aT d(X) ¯ i i=1 m T +¯ d(D arcsin[D z −1 XD −1 yi aT d(D arcsin[D−1 XD−1 ]D) ¯ i ]D) + i=1 = = ≤ = π π ¯ Q • X − z T d(X) − y T b + z T ( d(X)) + y T ( b) ¯ ¯ ¯ 2 2 π −1 −1 (since d(D arcsin[D XD ]D) = d(X) and aT d(X) = bi ) i 2 π π T T Q • X + ( − 1)¯ d(X) + ( − 1)¯ b z y 2 2 π Q • X + ( − 1)(¯T e + y T b) z ¯ 2 (since 0 ≤ d(X) ≤ e and z ≥ 0) ¯ π p Q • X + ( − 1)¯. 2 Let X 0 converge to X. Replacing Q with −Q proves (ii) in the theorem. Adding the ﬁrst two inequalities gives (iii) of the theorem.

let X 0 → X. . we have the following theorem: Theorem 4. and x = Dσ(V T u) where u with u = 1 is a random ˆ vector uniformly distributed on the unit sphere. Eu q(ˆ) − q x q−q ¯ ≤ ≤ ≤ = = 4.8. D = √ √ diag( x11 . d(X) ≤ e. q−q ¯ 2 Proof.26 Let x be randomly generated from X. In [207]. and then the problem is projected back to obtain tighter . p π π π Finally. Moreover. a “lifting” procedure is presented to obtain 2 a problem in n . ¯ p π π π π we have. . lim Eu (q(ˆ)) = lim x X→X X→X 2 2 2 Q • (D arcsin[D−1 XD−1 ]D) ≤ p + (1 − )¯. Notes Semideﬁnite relaxations have recently appeared in relation to relaxations for 0-1 optimization problems. m). Since p≥q≥ ¯ ¯ 2 2 2 2 p + (1 − )p ≥ (1 − )¯ + p ≥ q ≥ p.8 2 πp 2 p + (1 − π )¯ − q q−q ¯ 2 πp + 2 ¯ πp + 2 πp + 2 ¯ πp + 2 (1 − π )¯ − q p 2 (1 − π )p − q 2 (1 − π )¯ − p p 2 (1 − π )p − p 2 (1 − π )(¯ − p) p 1 2 p π (¯ − p) 2 −π π = − 2 2 π 1. from Corollary 4.25 Let X = V T V 0. NOTES 99 The result indicates that the positive semi-deﬁnite relaxation value p − p ¯ is a constant approximation of q − q. xnn ). . . . Ai •X = bi (i = 1. Similarly. . .25. the following corollary can be ¯ devised: Corollary 4. Then. . Then ˆ Eu q(ˆ) − q x π ≤ − 1 < 4/7.4.

Several of the operators that arise in our applications are similar to those that appear in [207]. for the case k = n/2. a greedy heuristic by Asahiro et. A key step in designing a good approximation algorithm for such a maximization problem is to establish a good upper bound on the maximal objective value. The other approximation algorithms have performance guarantees which are the function of k/n.586 performance guarantee from 0. Kortsarz and Peleg [193] devised an approximation algorithm which has a performance guarantee O(n−0. E) with |V | = n. However. The previously best performance guarantee. see [100] for DSP.. where the total edge weights of the subgraph induced by S is maximized. Ye and Zhang [151].al. both LP and SDP based . Since these MAX-GP problems are NP-hard (e. Max-Vertex-Cover with size k (MVC).100 CHAPTER 4. obtained an improved 0. Max-Not-Cut with size k (MNC). non-negative weights wij on edges (i. Given an undirected graph G = (V. the maximization graph partition (MAX-GP) problem is to determine a subset S ⊂ V of k nodes such that an objective function w(S) is maximized. where the total edge weights of the edges crossing between S and V \ S is maximized. and [253] for MVC). where the total edge weights of the edges covered by S is maximized. Such an algorithm is often called (randomized) r-approximation algorithm. Linear programming (LP) and semideﬁnite programming (SDP) have been frequently used to provide such upper bounds for many NP-hard problems. Rendl and Wolkowicz [254]. Moreover. Goemans and Williamson [125] proved an approximation result for the Maxcut problem where ≤ 1 − 0. Nesterov [240] extended their result to approximating a boolean QP problem where ≤ 4/7. Max-Cut with size k (MC). was obtained by Feige and Langberg [99]. See also [29]. Most of the results are related to graph partition. and SDP relaxation based algorithms developed by Feige and Langberg [99] and Feige and Seltser [100].g.g. Kortsarz and Peleg [101] improved it to O(n−1/3+ ). one should not expect to ﬁnd polynomial time algorithms for computing their optimal solutions. Feige. j) ∈ E. [2] for MC. using a new SDP relaxation. A discussion of several applications for semideﬁnite relaxation appears in [9]. [151] for MNC. and an integer k (1 < k < n). For approximating the MC problem with size k. A (randomized) polynomial time approximation algorithm for a maximization problem has a performance guarantee or worst case ratio 0 < r ≤ 1. Therefore.3885 ). [23]. There are several approximation algorithms for DSP. k/n for general k and k/n+ k for k ∼ n/2. Some examples of MAX-GP are: Dense-k-Subgraph (DSP). Also see recent papers by Fujie and Kojima [113] and Polijak. where the total edge weights of the non-crossing edges between S and V \ S is maximized. we are interested in how close to optimality one can approach in polynomial time. e.878. During the past ten years. if it outputs a feasible solution whose (expected) value is at least r times the maximal value for all instance of the problem. and Srivastav and Wolf [282].. there have been several remarkable results on approximating speciﬁc quadratic problems using positive semi-deﬁnite programming. SDP FOR COMBINATORIAL OPTIMIZATION inequalities. our motivation and approach is diﬀerent.517 of Feige and Langberg [99]. for some > 0.

and the SDP-based n(n−1) algorithm has a ratio .733 for k = 2n/5. for example. comparing to the MAXCUT problem. at the same time. the MaxBisection.699 by Ye [334]. 0. in any (randomized) rounding method.811 for k = n/2.5 + k for k ∼ n/2. and Nesterov[240] and Zwick [345] for MAX-CUT. Frieze and Jerrum [110] obtained a 0. j j . they have yielded improved approximation performance ratios for DSP. this ratio has been improved to 0.4. Again.541 of Feige and Langberg [99]. This kind of randomized algorithm can be derandomized by the technique of Mahajan and Ramesh[214]. since otherwise we have to either add or subtract nodes from S. Our improved rounding method is built upon this balance need.651-approximation algorithm (the same bound was also obtained by Andersson [16] in his paper for max-psection). comparing to . This rounding technique is related to the well-known rounding method introduced by Goemans and Williamson [125]. we need to balance the (expected) quality of w(S) and the (expected) size of S. and 0. j X= j=1 such that for all j.. As consequences of the improved rounding method. Ye and Zhang [151] obtained a 0. see Feige and Langberg [99].845 for k = 3n/5. For MC and MNC. Both of their approximation algorithms are based on SDP relaxations. Feige and Goemans [98] for MAX-DICUT and MAX 2-SAT. Then. What complicates matters in the MAX-GP problem. Han et al. xT Axj = A • (xj xT ) = A • X/r.486 for k = 2n/5 and 0. 0. see Ageev and Sviridenko [2] and Feige and Langberg [99].278 for k = n/4. MNC and MVC. One wants high w(S). i. 4. on a wide range of k. 0. Subsequently. but.586 for k = n/2. Therefore. our algorithm has guaranteed performance ratios .5 for a wide range of k. Zwick [344] for constraint satisfaction problems.9. On approximating MVC. On approximating DSP. our algorithm has guaranteed performance ratios .648 for k = 3n/5. For k = n/2. A be a given symmetric matrix. the performance guarantees are also much better than 0. the LP-based approximation algorithm has a performance ratio 1 − 2k(n−k) for all k. EXERCISES 101 approximation algorithms have a performance ratio 1/2 for all k.9 Exercises 4. [151] has presented an improved method to round an optimal solution of the SDP relaxation of the MAX-GP problem for general k. there is a decomposition of X r xj xT . is that two objectives are sought—the objective value of w(S) and the size of S.602-approximation algorithm for MNC when k = n/2. resulting in a deterioration of w(S) at the end.e. MC.1 Let X be a positive semideﬁnite matrix of rank r. zero or small diﬀerence between |S| and k. For approximating the MNC problem with size k.

4. 4. decision variable xi is either 1 or −1. In reality. we have shown that there is an optimal SDP solution whose rank r satisfying r(r + 1) ≤ m.2 Prove Theorem 4.27) y ∈ Rm . Prove that. 4.10. Show that this problem is equivalent to the dual of the SDP relaxation for Max-Cut. Then arcsin[X] X. one may never generate an exactly optimal SDP solution but an -optimal solution X. Show how the rank-reduction procedure works.6 Prove Lemma 4. 4.102 CHAPTER 4. 4. for any three variables we have |xi + xj + xk | ≥ 1 and |xi − xj | + |xj − xk | ≥ |xk − xi | which call triangle-type inequalities.3 The s − t Max-Cut Problem: This is a max-cut problem where we require that a pair of nodes s and t be separated by the cut. • Construct the SDP relaxation for the problem. meaning its minimal objective value is away from the optimal z ∗ : C • X ≤ z∗ + . 4. .878. ¯ where λmax (A) is the maximum eigenvalue of the matrix A. 2 and there is a strongly polynomial-time algorithm to ﬁnd such a solution from any exactly optimal SDP solution.4 Triangle Inequality: In the Max-Cut or other problems. SDP FOR COMBINATORIAL OPTIMIZATION 4.7 In solving the standard SDP problem with m equality constraints. from X. • The result holds for more pairs need to be separated. Thus. • Show that the problem can be approximated by the factor 0.5 Let X 0 and d(X) ≤ 1. one can ﬁnd an -optimal SDP solution whose rank r satisﬁes r(r + 1) ≤m+1 2 in strongly polynomial time.8 Consider the following eigenvalue optimization problem minimize nλmax (Q + Diag(¯)) y subject to eT y = 0 ¯ (4. Show how to incoporate these inequalities in the SDP relaxation of the problem. 4.

28). a family {S1 . EXERCISES 103 4.9 Quadratic 0-1 Programming: The following problem is referred as the Quadratic 0-1 programming maximize xT Bx subject to x ∈ {0. Here I can be the list of goods. For any ¯ ¯ cut (S.29) ¯ rank(X) = 1. Sm } of subsets of I and an integer 1 ≤ k ≤ n. E).28) Show that the following matrix representation is equivalent to the above quadratic 0-1 programming problem (4.4. Problem (4.11 The Sparsest Cut Problem ([22]): Given a graph G = (V. (4. (4.29) without the rank 1 constraint on X is an SDP relaxation of (4. The problem is to ﬁnd two subsets A1 . · · · . 4.9.28): maximize B•X ¯ subject to X = 1 diag(X)T diag(X) X 0. where diag(A) is the vector of diagnoal elements in A. Find an SDP relaxation to the problem. |Si ∩ A2 |}. The ¯ problem is to ﬁnd the S such that |E(S. 1}n . the edge expansion of the cut is |E(S. 4. S2 . S) with |S| ≤ |V |/2. S)|/|S|. Find an SDP relaxation to the problem.10 The 2-Catalog Segmentation Problem ([325]): Given a ground set I of n items. S)|/|S| is the smallest. . there are m customers where customer i is interested in the goods of Si . A2 ⊂ I such that |A1 | = |A2 | = k to maximize m i=1 max{|Si ∩ A1 |. Note 1. It suﬃces to show that both problems have the same feasible region. Note 2. A1 and A2 are the two catalogs one of which could be sent to each of the customers such that the (total) “satisfaction” is maximized.

SDP FOR COMBINATORIAL OPTIMIZATION .104 CHAPTER 4.

1 The basic SDP model The basic mathematical model of Euclidean distance geometry optimization can be described as a quadratically constrained optimization problem... xn ) (dij )2 ≤ xi − xj xj ∈ Ω.Chapter 5 SDP for Geometry Computation One of the most studied problems in distance geometry is the Graph Realization problem. we discuss SDP formulations of such Euclidean distance geometry problems.. we are given ¯ Euclidean distance upper bound dij and lower bound dij between the two points xi and xj . we require that each point xj ∈ Ω where Ω is a convex set...e. ∀j.1) Note that the ”bounding from below” constraint (xi − xj ) 2 105 ≥ (dij )2 (5. Then. This problem and its variants arise from applications in various areas. ..2) . . and the goal is to compute a realization of G in the Euclidean space Rd for a given dimension d. For each pair of (i. Euclidean ball packing. 2 ¯ ≤ (dij )2 . ∀i < j (5. Let n unknown points xj ∈ Rd . xn . such as molecular conformation. i. j). and an objective function of x1 . j) ∈ E} on its edges.. Moreover. . j = 1. and more recently. the basic model can be formulated as: minimize subject to f (x1 . in which one is given a graph G = (V.. dimensionality reduction. n. to place the vertices of G in Rd such that the Euclidean distance between every pair of adjacent vertices vi .. vj equals to the prescribed weight dij . E) and a set of non–negative weights {dij : (i. 5. wireless sensor network localization. In this chapter.

[56]) Z := I XT X Y 0..2 Wireless sensor network localization There has been an increase in the use of semideﬁnite programming (SDP) for solving wide range of Euclidean distance geometry problems. −1 at the jth position and zero everywhere else. light etc) across an entire physical space. Then. ∀i < j. 1 : d) = I.106 CHAPTER 5. d + n. Let X = [x1 x2 .5) is a (linear) semideﬁnite program. ij xj ∈ Ω. Typical networks of this type consist of a large number of densely deployed sensor .3) Our approach is to relax problem (5. ∀i < j. (5. ∀j.5) If f (Z) is a convex function of Z. eij )T Z(0. In particular. Z 0. Z(1 : d. Then xi − xj 2 = eT X T Xeij . eij ) ≤ (dij )2 .5) becomes a convex optimization problem. Let Y = X T X. Y X T X. ∀j. chain folding etc. ij where eij is the vector with 1 at the ith position.1) can be rewritten as: minimize subject to f (X. . the problem can be written as a standard SDP problem: minimize subject to f (Z) Z(1 : d. Y ) ¯ (dij )2 ≤ eT Y eij ≤ (dij )2 . ¯ (dij )2 ≤ (0.. Y = X T X. such as data compression. then (5. ball packing... metric-space embedding. sound levels. then (5. if f (Z) is a linear function of Z and Ω is a polyhedron. ij xj ∈ Ω. SDP FOR GEOMETRY COMPUTATION is not convex.4) The last matrix inequality is equivalent to (Boyd et al. ∀i < j. xn ] be the d × n matrix that needs to be determined. (5. 5.. Then problem (5.3) to a semideﬁnite program: minimize subject to f (X. (5. One application of the SDP Euclidean distance geometry model lies in ad hoc wireless sensor networks which are constructed for monitoring environmental information(temperature. Y ) ¯ (dij )2 ≤ eT Y eij ≤ (dij )2 . so that the eﬃcient convex optimization techniques cannot apply to solving this problem. j) ∈ Ω j = d + 1.

270. m.. Therefore. j) ∈ Na we know distance d sensor j. 161. 65. Observable traces are developed to measure the quality of the distance data. and for (k.. let the sensor points be placed on a plane. The distance information could be based on criterion like time of arrival. . which can be used to detect erroneous or outlier sensors. 79. the use of a GPS system is a very expensive solution to this requirement. 163. 269.5. j) ∈ Nx . For every pair of two points. it is assumed that we already know the positions of a few anchor nodes.. WIRELESS SENSOR NETWORK LOCALIZATION 107 nodes which gather local data and communicate with other nodes within a small range. and n unknown sensor points xj ∈ R2 . Indeed. 268. In particular. Furthermore.. we have a Euclidean distance measure if the two are within a communication distance range R. Also the estimation errors are minimal even when the anchor nodes are not suitably placed within the network. 5. for each sensor the model generates numerical data to measure the reliability and accuracy of the positions computed from the model. we are given Euclidean distance data dij between unknown ˆkj between anchor k and sensors i and j. Depending on the accuracy of these measurements and processor.2. Therefore knowledge of of each sensor position becomes imperative. Recall that we have m known anchor points ak ∈ R2 . The optimization problem is set up so as to minimize the error in the approximate distances between the sensors. More importantly. 245. power and memory constraints at each of the nodes. n. The sensor data from these nodes are relevant only if we know what location they refer to. Very few anchor nodes are required to accurately estimate the position of all the unknown sensors in a network. .. The performance of this technique is highly satisfactory compared to other techniques. 273]. 116.2. The problem of ﬁnding the positions of all the nodes given a few anchor nodes and partial distance information between the nodes is called the position estimation or localization problem. there is some degree of error in the distance information.1 An SDP relaxation model For simplicity. the paper [51] describes an SDP relaxation based model for the position estimation problem in sensor networks. Therefore. the localization problem can be formulated as . The basic idea behind the technique is to convert the non-convex quadratic distance constraints into linear constraints by introducing relaxations to remove the quadratic term in the formulation. other techniques to estimate node positions are being developed that rely just on the measurements of distances between neighboring nodes [51. Generally. k = 1.. j = 1. ˆ say for (i. Note that for the rest of pairs we have only a lower bound R for their pair-wise distances. angle of arrival and received signal strength.

0) Z(1. Let X ∗ be the true locations of the n points. 1. the minimal value of α = 0 in (5. 1.j∈Na |αkj | ˆ = (dij )2 + αij . ∀ (k. 2 ≥ R2 . Moreover. then Z has a unique solution. 0) = 2 ˆ (0. the SDP relaxation solves the original problem exactly. ∀ (i. / (ak . SDP FOR GEOMETRY COMPUTATION an error minimization problem with mixed equalities and inequalities: minimize subject to |αij | + k. j) ∈ Na . 2 ≥ R2 . we can show Proposition 5.6). j) ∈ Na . 0) = 1 (0. That is. Z= ¯T Y ¯ X ¯ ¯ ¯ ¯ then we must have Y = (X)T X and X equal true positions of the unknown sensors. We present a simple case to show what it means for the system has a unique solution. 0)T Z(0. i<j T (5. for the rest k. T ¯ (ak . Then we have at least 2n + n(n + 1)/2 linear equalities among the constraints. Moreover. and α = 0 for the minimal solution of (5. ej ) Z(ak .1 If there are 2n + n(n + 1)/2 distance pairs each of which has an accurate distance measure and other distance bounds are feasible.6) The matrix of Z has 2n + n(n + 1)/2 unknown variables. for (k. i < j. i. j) ∈ Na . for the rest i < j. j) ∈ Nx . j.j∈Nx . j) ∈ Nx . / Z 0. Proof. j) ∈ Nx .108 CHAPTER 5. 0. 0. eij ) = (dij )2 + αij . Therefore. Then Z ∗ and α = 0 is a feasible solution for (5. 2 ˆ = (dkj )2 + αkj . j}. i<j 2 xi − xj ak − xj xi − xj ak − xj Thus.6).j∈Na |αkj | (1. Then. Consider the case that among {k. eij ) Z(0. ∀ (k. ∀ (i < j. if (5. eij )T Z(0. ej ) = (dkj )2 + αkj . we must have Z = Z ∗ so that Y = (X ∗ )T X ∗ = X T X.6).6) has a unique minimal solution ¯ I X ¯ . The accurate distance measures from . ej )T Z(ak . 1. T 2 (0. we relax the problem to a semideﬁnite program: minimize subject to |αij | + k. since Z is the unique solution to satisfy the 2n+n(n+1)/2 ¯ ¯ ¯ ¯ equalities. 0)T Z(1. eij ) ≥ R . ej ) ≥ R2 . there are 2n + n(n + 1)/2 of the pairs where each of them has same distance upper and lower bounds. i. ∀ (i < j. if these equalities are linearly independent. ¯ On the other hand. i. Consider n = 1 and m = 3. and Z∗ = I (X ∗ )T X∗ (X ) X ∗ ∗ T . 1. 0) = 1 (1.j∈Nx .

. n ¯ ¯ ¯ Trace(Y − X T X) = ¯ (Yjj − xj ¯ 2 ). . .. . .. 5. n. each xj can be viewed a random point xj since the distance measures contain random ˜ errors.. Generally.. (5. we have E[˜j ] ∼ xj .1 is deterministic. Here. WIRELESS SENSOR NETWORK LOCALIZATION 109 unknown b1 to known a1 . the three linear equations are y − 2xT a1 y − 2xT a2 y − 2xT a3 = (d11 )2 − a1 = (d21 )2 − a2 = (d31 )2 − a3 2 2 2 This system has a unique solution if it has a solution and the matrix 1 a1 1 a2 1 a3 is nonsingular. the ¯ SDP method reduces to the so-called triangular method.2. a2 and a3 are d11 . Therefore.. Alternatively. measures the quality of distance sample data dij and dkj . Such an interpretation appears to be ˜ ﬁrst stated in Bertsimas and Ye [50]. n. n x ¯ and ¯ E[˜T xj ] ∼ Yij . j=1 the total trace of the co-variance matrix. ¯ ¯ ¯ Y − XT X represents the co-variance matrix of xj . Then the solution to the SDP problem provides the ﬁrst and second moment information on xj .. For example.. and then x = b1 can be uniquely determined. j = 1..7) . ˜ These quantities also constitute error management and analyses of the original problem data. ¯ X ¯ Y I ¯ XT is the optimal solution of the SDP problem. In particular. Thus. a2 and a3 are not on the same line. n. respectively. j = 1. individual trace ¯ Yjj − xj ¯ 2 .. xi ˜ where ¯ Z= i.. d21 and d31 .1 and the example show that the SDP relaxation method has the advantage of the triangular method in solving the original problem..2. j = 1. j = 1.2 Probabilistic or error analyses The case discussed in Proposition 5. Proposition 5. This essentially means that the three points a1 .5.

Therefore.110 CHAPTER 5. and are often diﬃcult to detect since the true location of sensors is unknown. Moreover. Consider n = 1 and m = 3. y That is. and outlier or defect sensors. we have E[¯ − x 2 ] ≤ E[ 2 ] y ¯ and E[ x 2 ] − b1 ¯ 2 ≤ E[ 2 ]. This quantity gives ¯ an interval estimation of b1 . so that the quantity of y − x 2 is a lower bound estimate for the error variance ¯ and the variance of x is also bounded by the error variance. x and E[¯] − E[¯]T E[¯] = E[ 2 ] y x x or E[¯] = E[ 2 ] + b1 2 . The inexact distance measures from unknown b1 to known a1 . d21 + and d31 + . the three linear equations are y − 2¯T a1 + a1 ¯ x y − 2¯T a2 + a2 ¯ x y − 2¯T a3 + a3 ¯ x 2 2 2 = (d11 )2 + 2 d11 + = (d21 )2 + 2 d21 + = (d31 )2 + 2 d31 + 2 2 2 . These errors often occur in real applications either due to the lack of data information or noisy measurement. we have E[¯] − 2E[¯]T a1 + a1 y x E[¯] − 2E[¯]T a2 + a2 y x E[¯] − 2E[¯]T a3 + a3 y x 2 = (d11 )2 + E[ 2 ] = (d21 )2 + E[ 2 ] = (d31 )2 + E[ 2 ] 2 2 or E[¯] − E[¯]T E[¯] + E[¯] − a1 y x x x E[¯] − E[¯]T E[¯] + E[¯] − a2 y x x x T E[¯] − E[¯] E[¯] + E[¯] − a3 y x x x 2 2 2 = (d11 )2 + E[ 2 ] = (d21 )2 + E[ 2 ] = (d31 )2 + E[ 2 ]. We again use the same simple case to illustrate our theory. Taking expect values on both sides. where is a random error with zero mean. More generally. from ¯ E[¯] − E[ x 2 ] + E[ x 2 ] − b1 y ¯ ¯ 2 = E[¯] − b1 y 2 = E[ 2 ]. x is a point estimate of b1 . we have . a2 and a3 are d11 + . respectively. SDP FOR GEOMETRY COMPUTATION which is also the variance of xj . The solution to the linear equation is E[¯] = b1 . helps us to detect possible distance measure ˜ errors.

let the minimal α = 0 in (5. j or ¯ ¯ ¯ E[Yii ] − 2E[Yij ] + E[Yjj ] − E[¯i ] − E[¯j ] 2 + E[¯j ] − E[¯i ] x x x x ¯ E[Yjj ] − E[¯j ]T E[¯j ] + E[¯j ] − ak x x x 2 = (dij )2 + E[ 2 ] + E[ 2 ] i j 2 = (dkj )2 + E[ 2 ]. ∀i=j and ˆ dkj = dkj + j..5. we have.. Proof. ∀k. ¯ ¯ ¯ Yii − 2Yij + Yjj T ¯ Yjj − 2¯j ak + ak 2 x = (dij + = (dkj + + j )2 2 j) .2 Let the noisy measurements ˆ dij = dij + i + j. j = 1. for all i. n. j Thus. we have ¯ ¯ ¯ E[Yii ] − 2E[Yij ] + E[Yjj ] ¯ E[Yjj ] − E[¯j ]T E[¯j ] + E[¯j ] − ak x x x 2 (dij )2 + E[ 2 ] + E[ 2 ] i j = = (dkj )2 + E[ 2 ]. . Then. where bj is the true position of xj .2.6). Moreover. i Taking expect values on both sides. E[¯j ] = bj x ¯ and E[Yjj ] = bj 2 + E[ 2 ] ∀ j j and ¯ E[Yij ] = (bi )T bj ∀ i = j is the solution satisfying these equations.6) and the anchor points are linear independent. j. j where dij are the true distances and j are independent random errors with zero mean. k. Since α = 0. . we have E[¯j ] = bj x and ¯ E[Yjj ] = bj 2 + E[ 2 ] ∀ j j and ¯ E[Yij ] = (bi )T bj ∀ i = j. and ¯ Z= ¯ X ¯ Y I ¯ XT is the minimizer of (5. WIRELESS SENSOR NETWORK LOCALIZATION 111 Proposition 5..

we ignore the lower bound constraints and study the Graph Localization problem that is deﬁned as follows: ﬁnd a realization of x1 . . Further¯ more. then the graph can be localized by solving an SDP problem in polynomial time. In real applications. . The proof employs SDP duality theory and properties of interior– point algorithms for SDP. . Again. j) ∈ Na d2 ij ∀ (i. Note that this formulation forces any possible feasible solution matrix to have rank at least d. 5. . Informally. j) ∈ Na .112 5. j) ∈ Nx . we are given partial accurate Euclidean distance values dkj between ak and xj for (k. a graph is strongly localizable if it is uniquely localizable and remains so under slight perturbations. Can these graph be localized or realized in polynomial time? In this section. and dij between xi and xj for some (i < j. To the best of our knowledge. The result is close to be the best possible in terms of identifying the largest family of eﬃciently realizable graphs.1:d is the d × d principal submatrix of Z. . . SDP FOR GEOMETRY COMPUTATION General SDP theory on graph realization In the previous section. Informally. . ej )(ak . . Moreover. j) ∈ Nx ¯ (ak .1:d = Id (0. . we show that if all accurate distances are given.3 CHAPTER 5. . we can write the relaxed problem as a standard SDP problem. We show that the SDP model will identify all the strongly localizable subgraphs in the input graph. j) ∈ Nx = (5. xn ∈ Rd whose locations we wish to determine. ﬁnd a symmetric matrix Z ∈ R(d+n)×(d+n) to: maximize 0 subject to Z1:d. Speciﬁcally. we given a positive answer to a family of uniquely localizable graphs. only partial distances(or edges) are known to the model.1 Preliminaries For the simpliﬁcation of our analyses. al. eij )(0. where h > d. this is the ﬁrst time such a theoretical guarantee is proven for a general localization algorithm. we present an SDP model that guarantees to ﬁnd the unique realization in polynomial time when the input graph is uniquely localizable. in view of the hardness result of Aspnes et. [24]. and (ii) it does not have any nontrivial realization whose aﬃne span is Rh . namely. We also introduce the concept of strong localizability. am ∈ Rd whose locations are known. .9) 0 where Z1:d. . eij )T • Z = d2 ij ∀ (i. . xn ∈ Rd such that ak − xj 2 xi − xj 2 ¯ = d2 kj ∀ (k.8) Recall that m anchor points a1 . ej ) • Z = d2 kj ∀ (k. a graph is uniquely localizable in dimension d if (i) it has a unique realization in Rd . j) ∈ Na T Z (5.3. and n sensor points x1 .

since the primal is feasible. as V = 0. i. when will the solution matrix Z have rank d. Then..10) (i. Suppose that problem (5.e. ej )T + (5.9) has rank d. ¯ ¯ ¯¯ 1. ej )T + (k. for instance. xn ]. ¯ This occurs when.j)∈Nx wkj (ak . .10). the matrix Z = (Id . e. then every solution of (5. Rank(Z) + Rank(U ) ≤ d + n. i. from the duality theorem for SDP (see.8) can be solved as an SDP in polynomial time. ej )(ak .j)∈Nx V 0 subject to ¯ wkj d2 kj yij d2 + ij Id • V + 0 0 (k.9) have an exact relaxation.j)∈Na Then. That is. ¯ ¯ 3.j)∈Nx 0 (k.3 Let Z be a feasible solution for (5. eij )(0. ¯ ¯ 2. ej )(ak . dkj and dij represent exact distance values for ¯ ¯ ¯ ¯ the positions X = [¯1 x2 . Let U be the (d + n)–dimensional dual slack matrix. Rank(Z) ≥ d and Rank(U ) ≤ n. 5.5. X)T (Id . eij )T + (i.8) and (5. the minimal value of the dual must be 0. i. there is no duality gap between the primal and dual.3.9).g. Another technical result is the following: .3.e.j)∈Na Note that the dual is always feasible.9) are equivalent and (5. eij )T + wkj (ak . GENERAL SDP THEORY ON GRAPH REALIZATION 113 The dual of the SDP relaxation is given by: minimize (i.9) is feasible.e. Now. j) ∈ Nx and wkj = 0 for all (k.4 If an optimal dual slack matrix has rank n. [10]).: U V 0 = 0 0 yij (0. yij = 0 for all (i. problems (5. j) ∈ Na is a feasible solution. complementarity condition holds: Z • U = 0 or Z U = 0. eij )(0.j)∈Na yij (0.9) and U be an optimal slack matrix of (5. An immediate result from the theorem is the following: Corollary 5. Then. we have: ¯ ¯ Theorem 5.2 Analysis of the SDP relaxation We now investigate when will the SDP (5. . X) is a x ¯ ¯ feasible solution for (5.

9) has two rank–d feasible solutions: Z1 = Id T X1 X1 T X1 X1 and Z2 = Id T X2 X2 T X2 X2 .9) has rank d. from 2. if xi is connected to xj and Yjj is bounded. The equivalence between 2. .8) is uniquely localizable if there is a unique local¯ ization X ∈ Rd×n and there is no xj ∈ Rh .. 210] that various path–following interior–point algorithms compute the max–rank solutions for both the primal and dual in polynomial time. .114 CHAPTER 5. . m. n. . (5. . 0) for j = 1. . It is known [147. The max–rank solution matrix of (5. . 2. . directly or indirectly. the following 1.9) is a solution to (5. such that: ¯ (ak . The solution matrix of (5. is straightforward. . n} The latter says that the problem cannot have a non–trivial localization in some higher dimensional space Rh (i. i. statements are equivalent: Then. to an anchor point in (5.6 Suppose that the network is connected.8).5 If every sensor point is connected. This motivates the following deﬁnition. and 3. . We now develop the following theorem: Theorem 5.8). n). SDP FOR GEOMETRY COMPUTATION Proposition 5. 0) ∈ Rh . 130. a localization diﬀerent from the one obtained by setting xj = (¯j .9) is bounded. that is. In general. Proof. n.1 Problem (5. then we have: xj 2 − 2aT xj + ak k 2 ≤ Yjj − 2aT xj + ak k 2 ¯ = d2 kj so that from the triangle inequality xj in (5.9) must be bounded. . 0) x for some j ∈ {1. then any solution to (5. to 1.8) is uniquely localizable. j) ∈ Na kj 2 2 xi − xj = dij ∀ (i. .9) satisﬁes Y = X T X. Hence. . 3. Proof. Now. we have: ¯ Yjj ≤ d2 + 2 ak kj xj − ak 2 Furthermore. a primal (dual) max–rank solution is a solution that has the highest rank among all solutions for primal (5. where anchor points are augmented to x (ak . . where h > d.9) (dual (5.9) has rank d then it is unique. j = 1.e. Yjj is bounded for all j = 1. Problem (5. . . since any rank d solution of (5.e. 0) − xj 2 = d2 ∀ (k. .10)). Suppose not. . for k = 1. j) ∈ Nx xj = (¯j . . we have: Yii − 2 Yii Yjj + Yjj ≤ Yii − 2Yij + Yjj = d2 ij so that from the triangle inequality Yii must be also bounded. If sensor point xj is connected to an anchor point ak . Deﬁnition 5. . we need to prove that if the max–rank solution matrix of (5.

Z1 = Z2 . Next. This motivates us to deﬁne another notion called strong localizability. Then.3.9) whose rank is greater than d. but may no longer be so after small perturbation of the sensor points. . (˜i )T xj = Yij x ˜ ∀ i. the rank of a max–rank solution of (5.6 are close to be the best possible. n = Yjj . 0) − xj ˜ xi − xj ˜ ˜ 2 2 ¯ = d2 kj = d2 ij ∀ (k.. we prove the direction from 1. that as long as problem (5. then X is the unique realization of problem (5. . and there exist at least one xj such that xj = xj or xj = 0. [24] shows. for the ﬁrst time. al.5. we have: = Id T T αX1 + βX2 = Z Id BT αX1 + βX2 T T αX1 X1 + βX2 X2 B BT B where B = αX1 +βX2 . which is a contra˜ diction. GENERAL SDP THEORY ON GRAPH REALIZATION 115 Then. j) ∈ Nx In other words. since all feasible solution of (5. we have: (ak . Moreover. which is a contradiction. that is. then the realization can be computed in polynomial time by solving the SDP relaxation. to 2.3 Strongly Localizable Problem Although unique localizability is an useful notion in determining the solvability of the Sensor Network Localization problem. ˜ ˜ since the network is connected. as the recent result of Aspnes et. we T T have the decomposition Y −X X = (X ) X . Thus.8) is uniquely localizable. . Suppose that there is a feasible solution Z of (5. i. . j. where X = [x1 . . Moreover.6 establishes. we must have Y X T X and Y = X T X. we have: xj ˜ 2 ∈ Rd+r for j = 1. there exist networks which are uniquely localizable. or X1 −X2 = 0.8). . xn ] ∈ Rr×n and r is the rank of Y − X T X.e. it is not stable under perturbation. xj is a localization of problem (5. . the matrix Z = αZ1 + βZ2 . It follows that (X1 −X2 )T (X1 −X2 ) = 0.8) in Rd+r . Conversely.5 that Yii and Yij are bounded for all i. Therefore. β > 0 is a feasible solution and its rank must be d. Hence.4. where α + β = 1 and α. if the relaxation solution computed by an interior–point algorithm (which generates max–rank feasible solutions) has rank d (and hence Y = X T X). even when the instance has a unique solution on the plane. .3. 5.3. since the problem of computing a realization of the sensors on the plane is NP–complete in general. As we shall see in Section 5. Theorem 5. Now. consider the point: xj xj xj = ˜ Then. j) ∈ Na ∀ (i. we conclude from Proposition 5. the results of Theorem 5. .9) is d.9) has rank at least d but the max–rank is assumed to be d. j.

this shows that the trilateration graphs deﬁned in [89] are strongly localizable. Then. Since A and D are positive semideﬁnite. whence xi = 0. we have xi = 0. i) ∈ Na . where X is the matrix containing the true locations of the sensors. It has the form: U= U11 T U12 U12 U22 where U22 is an n × n matrix. Then.116 CHAPTER 5. n we have xT Dx = i=1 Dii x2 = 0. SDP FOR GEOMETRY COMPUTATION Deﬁnition 5. j) ∈ Nx . First. we have the following: Proposition 5. First. and let E be the number of sensor–sensor edges. suppose that yij < 0 for all (i.2 We say problem (5. a suﬃcient condition for solvability is that the system of equations are linearly independent. it has rank n. In )T W (−X T . and D is a diagonal matrix where Dii = − (k. for Dii > 0. Moreover. the above system has E + i |Ai | variables. since D is diagonal. Moreover. (If there is no (k. then both A and D are positive semideﬁnite. we need to set U12 = −XU22 . It follows that x = 0. Note that if a network is strongly localizable. i) ∈ Na . there exists an i such that Dii > 0. Then. The number of equations is E + 3m. we have xT U22 x ≥ 0 for all x ∈ Rn . where m is the number of sensors that are connected to some anchors. Note that the above condition is simply a system of linear equations. then it can be written in the form U = (−X T .e. Proposition 5. where Aij = yij if i = j. note that if U is an optimal dual slack matrix of rank n. note that: n n xT Ax = (xi − xj )2 Aij xi xj Aij = − i=1 j=1 i<j Thus.3 and 5. since the rank of all feasible solution of the primal is d. In particular. j. and that i |Ai | ≥ 3m. Suppose to the contrary that we have such an x. Then. and that wki < 0 for some (k.10) has an optimal dual slack matrix with rank n. Since Na = ∅.7 gives us a recipe for putting U into the desired form.8) is strongly localizable if the dual of its SDP relaxation (5. Furthermore. Let Ai be the set of sensors connected to anchor i. We now investigate when this is possible. We now show that there is no x ∈ Rn \{0} such that xT Ax = xT Dx = 0. i. U22 is positive deﬁnite. then it is uniquely localizable from Theorems 5. xT Ax = 0 implies that xi = xj for all i. i Now. Hence. then Dii = 0.) Note that if we impose the constraints yij ≤ 0 and wki ≤ 0. We now develop the next theorem. In particular. We show how we can construct a rank–n optimal dual slack matrix. In ) for some positive deﬁnite matrix W of rank n. Now.7 Suppose that the network is connected.i)∈Na wki . consider the dual matrix U .6. it can be decomposed as U22 = A + D. . with Na = ∅. we set U22 to be a positive ¯ ¯ deﬁnite matrix. Aii = − j Aij . Proof.

and it is complementary to any primal feasible solution of the (whole–problem) SDP relaxation: Zs ∗ Id Xs Z= 0 where Zs = T ∗ Xs Ys However. . the slack matrix: U= Us 0 0 0 0 must be optimal for the dual of the (whole–problem) SDP relaxation.8 If a problem (graph) contains a subproblem (subgraph) that is strongly localizable. we have 0 = Z • U = Zs • Us and Us . Since it is strongly localizable. strong localizability and rigidity in R2 are all distinct. I). i. the converse is not true.5 Unique Localizability ⇒ Strong Localizability We have already remarked earlier that a strongly localizable graph is necessarily uniquely localizable. It is not hard to check that this network is uniquely localizable.4 A Comparison of Notions In this section. then the submatrix solution corresponding to the subproblem in the SDP solution has rank d. ns . Zs 0. Then in the dual problem of the SDP relaxation for the whole problem. The key feature of G1 is that the sensor x2 lies on the line joining anchors a1 and a3 .11) (¯2 − a1 )¯12 + (¯2 − a3 )¯32 = −(¯1 − x2 )y12 x y x y x ¯ (5. Now.5. It is easy to verify that: U12 = (¯21 a2 + y31 a3 . Proof.12) . The rank of Us is ns implies that the rank of Zs is exactly d. so Xs is the unique realization of the subproblem. the dual slack matrix U admits the decomposition ¯ ¯ U = (−X T . the SDP relaxation computes a solution that localizes all possibly localizable unknown points. y12 a1 + y32 a3 ) y ¯ ¯ ¯ U22 = −(¯21 + y31 ) − y12 y ¯ y12 y12 −(¯12 + y32 ) − y12 y ¯ ¯ and the form of U requires that U12 = −XU22 . GENERAL SDP THEORY ON GRAPH REALIZATION 117 Theorem 5. Then.3. 5. as we shall see.e.1(a). That is. This is equivalent to the following system of equations: (¯1 − a2 )¯21 + (¯1 − a3 )¯31 = (¯1 − x2 )y12 x y x y x ¯ (5.3. . an optimal dual slack matrix Us of the SDP relaxation for the subproblem has rank ns . Then. 5. suppose to the contrary that G1 is strongly localizable. we will show that the notions of unique localizability. Ys = (Xs )T Xs . Let the subproblem have ns unknown points and they are indexed as 1. . we set V and those w· associated with the subproblem to the optimal slack matrix Us and set all other w· equal 0. However. Let G1 be the network shown in Figure 5.3. . I)T W (−X T .

To see this. which is a contradiction. However. Indeed. Then.4) x1 x2 x2=(0. However. 23 . Thus. 185 . a uniquely localizable network is rigid in R2 . x2 ) back and obtain an 3–dimensional realization of the network.6 Rigid in R2 ⇒ Unique Localizability By deﬁnition. we conclude that G1 is not strongly localizable.5) a2=(−1.1(b). both having radius 5/2. Thus. x2 . we can put the edge (x1 . a1 a1=(0. but not strongly localizable network (b) A rigid network that is not uniquely localizable Figure 5. More precisely. 64 . x2 ). the above network has an 3–dimensional realization. the reﬂected version of x2 has co√ 495 112 ordinates x2 = 173 .3. reﬂect the subgraph induced by a1 . for the above realization. It is straightforward 370 64 to verify that: x1 − a2 2 = x1 − a2 2 = 5 4 x1 − a3 2 = x1 − a3 2 = 5 4 x1 − x2 2 = x1 − x2 2 = 2 5 .11) would then imply that x1 lies on the ¯ aﬃne space spanned by a2 and a3 .6. consider two spheres. It is easy to verify that G2 is rigid. The intersection of these spheres is a circle. equation (5.6.7) x1=(0. a2 across the dotted line.12) implies ¯ that y12 = 0. by Theorem 5.0.118 CHAPTER 5.0.0) a2 a3=(1. Now.1: A comparison of graph notions 5. The idea for constructing such a realization is as follows. let G2 be the network shown in Figure 5. one √ centered at a2 and the other centered at a3 . let x1 = 0. equation (5. and we can move x1 along this circle until the distance between x1 and x2 equals to the prespeciﬁed value. the converse is not true.1. Note that G2 can be viewed as a perturbed version of G1 . SDP FOR GEOMETRY COMPUTATION Since x2 lies on the aﬃne space spanned by a1 and a3 . Then.0) a3 (a) A uniquely localizable. 0 . Let us ﬁrst remove the edge (x1 . Now. it can fail to be uniquely localizable only if it has a realization in some higher dimension.

2: Strongly localizable networks 5.2. they are both strongly localizable): a1 a1 x2 x2 x1 a2 a2 a3 (a) a3 x1 (b) Figure 5. we conclude that G2 is not uniquely localizable. If the distance between two notes was less than a given radiorange between [0. where noisyf actor was a given number between [0.0 of Benson and Ye. the networks shown in Figure 5.5]. while having the same combinatorial property as the one shown in Figure 5.3.5 . .5 . Very few anchor nodes are required to accurately estimate the position of all the unknown nodes in a network. a random error was added to it ˆ dij = dij · (1 + randn(1) ∗ noisyf actor). Simulations were performed on a network of 50 sensors or nodes randomly generated in a square region of [−. Also the estimation errors are minimal even when the anchor nodes are not suitably placed within the network. It would be nice to have a characterization on those graphs which are rigid in the plane but have higher dimensional realizations. and then both upper and lower bound constraints were applied for that distance in the SDP model. The performance of this technique seems highly satisfactory compared to other techniques.3. are uniquely localizable (in fact. However. For instance.7 Preliminary computational and simulation results The computational results presented here were generated using the interiorpoint algorithm SDP solvers SeDuMi of Sturm and DSDP2. The distances between the nodes was then calculated. 1]. ﬁnding such a characterization remains a challenging task. 1].5. only the lower bound constraint. If the distance was beyond the given radio − range. and would depend heavily on the geometry of the network.1(b).5] × [−. as such characterization would necessarily be non–combinatorial. GENERAL SDP THEORY ON GRAPH REALIZATION 119 Hence.

3 −0.4 0.3 total (b) error:0. Even with noisy data measurement.1 −0. was applied. green circle nodes to the true locations of the unknown sensors and red star nodes to their estimated positions ¯ from X. trace:0.4 0.001 ∗ radiorange2 .2 −0.3 −0.3 (a) error:0.120 CHAPTER 5. SDP FOR GEOMETRY COMPUTATION ≥ 1. 0.2 −0.2 −0. noisy factor=0.4 0.2 0. As discussed earlier.4 shows the correlation between individual oﬀset errors (not observable) and individual traces (observable) of corresponding sensors when the radio range is set at 0.5 −0. on average.5 −0. connectivity:7.5 −0.3 −0.1 0 0.4 −0.1 0.4 −0.2 0.1 0 0.4 −0.2 −0. Several key questions have to be answered for the future research: What is the minimal radio range such that the network is localizable? How does the sensor geographical distribution change the minimal radio range? What is the sensitivity of the distance data? How to ﬁlter the data noisy? etc.002.3 0. The average estimation error is deﬁned by n 1 · xj − aj . we called the trace of Y − X T X the total trace.1 0.5 0.5 −0. The discrepancies in the positions can be estimated by the oﬀsets between the true and the estimated points as indicated by the solid lines. the position estimations for the sensors are fairly accurate. .20. radio range=0.3 −0.023. are within the radio range of a sensor. Figure 5. 0. see Figure 5.1 0 0 −0.25.3. The blue diamond nodes refer to the positions of the anchor nodes.5. and various radio ranges.3: Position estimations with 3 anchors.5 −0. Connectivity indicates how many of the nodes.5 total Figure 5. connectivity:10.30.1 0. 0.2 0.16.2 0.3 0. ¯ n j=1 where xj comes from the SDP solution and aj is the true position of the jth ¯ ¯ ¯ ¯ node.4 0. radio range=0.1 −0. trace:0.5 0.4 −0.8. Also the original and the estimated sensors were plotted.04.

5.4. OTHER DISTANCE GEOMETRY PROBLEMS
0.5

121

0.25

0.4

0.3

0.2

0.2

0.1

0.15

0

−0.1

0.1

−0.2

−0.3

0.05

−0.4

−0.5
−0.5

−0.4

−0.3

(a) number
range=0.25.

−0.2

−0.1

of

0

0.1

0.2

0.3

0.4

anchors=3,

0.5

0

0

5

10

15

range=0.30.

20

of

25

30

35

anchors=3,

40

45

50

Figure 5.4: Diamond: the oﬀset distance between estimated and true positions,
¯
Box: the square root of trace Yjj − xj 2 .
¯

5.4

Other distance geometry problems

We list few other variances of the distance geometry or graph localization problem.

5.4.1

Metric distance embedding

The basic mathematical model of the metric distance embedding problem can be
described as a quadratically constrained optimization problem. We are given
metric p-norm distances dij for all i, j = 1, ..., n. The problem is to ﬁnd n
unknown points xj ∈ Rn , j = 1, ..., n such that
xi − xj

2

= (dij )2 ∀i = j

or the error is minimized.
Thus, an optimization problem can be formulated as:
minimize
subject to

α
(dij )2 ≤ xi − xj 2 ≤ α · (dij )2 , ∀i < j
xj ∈ Rn , j = 1, ..., n.

(5.13)

Its SDP relaxation becomes

αp :=

minimize
subject to

α
(dij )2 ≤ (eij eT ) • Y ≤ α · (dij )2 , ∀i < j
ij
Y
0.

(5.14)

122

CHAPTER 5. SDP FOR GEOMETRY COMPUTATION

Again
eij = (0, ..., 1, ..., −1, ...)T .
The SDP dual is

αp :=

Maximize
Subject to

i<j

0

uij (dij )2

T
i,j (uij − vij )eij eij ,
2
i,j vij (dij ) = 1

uij , vij ≥ 0.
Result and questions:

• α2 = 1, and αp ≤ O(log n) for all p.

• Reduce Y ∗ ’s rank to O(log n) for α2 = 1 + .

• rank ≤ O(log n) for αp ≤ O(log n) for all p.

• Better bound for α1 ?

5.4.2

Molecular conﬁrmation

The basic mathematical model of the molecular conﬁrmation problem can be
described as a quadratically constrained optimization problem as well. For a
¯
pair of (i, j) ∈ N , we are given Euclidean distance upper bound dij and lower
bound dij between points i and j, the problem is to ﬁnd Y such that
¯
(dij )2 ≤ (eij eT ) • Y ≤ (dij )2 , ∀i < j ∈ N .
ij
Furthermore, it is desired to ﬁnd other missing dij such that the Euclidean
distance matrix can be completed.
The progem can be formulated as an SDP:
minimize I • Y
¯
subject to (dij )2 ≤ (eij eT ) • Y ≤ (dij )2 , ∀i < j ∈ N
ij
Y
0.

5.4.3

(5.15)

Euclidean ball parking

The Euclidean ball packing problem is an old mathematical geometry problem
with plenty modern applications in Bio-X and chemical structures. It is a special
case of the Euclidean distance geometry problem, say d = 3, xj ∈ R3 is the
¯
unknown position of the center of ball j, dij = ∞ and dij = ri + rj , where ri
and rj are the radii of balls i and j, respectively; and Ω represents a convex
container where the balls should be packed.

5.4. OTHER DISTANCE GEOMETRY PROBLEMS

123

Say that we want to pack n balls in a box with width and length equal 2R
and like to minimize the height of the box, we can formulate it as
minimize
subject to

α
xi − xj 2 ≥ (ri + rj )2 , ∀i = j,
−R + rj ≤ xj (1) ≤ R − rj , ∀j,
−R + rj ≤ xj (2) ≤ R − rj , ∀j,
rj ≤ xj (3) ≤ α − rj , ∀j,

(5.16)

The ﬁrst constraint tells that the distance between any two balls should be
greater than or equal to the sum of the two balls’ radii, the second says that
the ﬁrst coordinate of xj is within the width constraint and the third is for the
length requirement, and the fourth indicates that the ball center has to be above
rj from the bottom and its height is below α − rj where α is to be minimized.

5.4.4

Data dimensionality reduction

Given P , a data point set of p1 , ..., pn ∈ Rd , a fundamental question is how to
embed P into Q of q1 , ..., qn ∈ Rk , where k
d, such that qj s keep all essential
information of P , such as the norms, distances and angles between pj s. In other
words, ﬁnd a d − k-dimension subspace such that the projections of pj s onto the
subspace has a minimal “information loss.” We believe that this problem can
be posted as an Euclidean distance geometry problem
Lk :=

minimize
subject to

α
n−k
T
2
i=1 ((pl ) xi ) ≤ α,
n−k
T
2
i=1 ((pl − pm ) xi )
2
xi = 1, i = 1, ..., n

∀l,
≤ α, ∀l < m,
− k, (xh )T xi = 0, ∀h < i.
(5.17)
Here, x1 , ..., xn−k represent the n − k orthogonal bases of a n − k-dimension subspace. Thus, (pT xi ; ...; pT xn−k ) represents the projection of pl onto the n − k
l
l
subspace. After the problem being solved and the subspace is found, the projections of pl s onto the complement k-dimension subspace must have all remaining
information of pl s. In fact, the model without pair-wise distance constraints
is called the minimum radius problem, which is a fundamental computational
geometry and statistics problem.
Assume that x1 , x2 , · · · , xn−k ∈ Rd are the optimal solution of (5.17). Then
one can easily verify that the matrix Y = x1 xT + x2 xT + · · · + xn−k xT
1
2
n−k is a
feasible solution for the following SDP model:

αk :=

minimize
subject to

α
pT Y pl ≤ α, ∀l,
l
(pl − pm )T Y (pl − pm ) ≤ α, ∀l < m,
Trace(Y ) = n − k, I − Y
0, Y

(5.18)
0.

It follows that αk ≤ Rk (P )2 . Then, we can design a rounding procedure to
¯ , a solution to (5.18), into orthogonal n − k bases and prove their
round Y
quality.

vr such that r i=1 (i). Let λ1 . λi = k. λr and r orthogonal unit vectors v1 .19) This leads to the following SDP relaxation: ∗ αk := minimize subject to α Trace(pl pT X) ≤ α. Computing the minimal radius is a fundamental problem in computational convexity with applications in global optimization. r i=1 λi p. · · · .. data mining. ∗ It follows that αk ≤ Rk (P )2 . The square of Rk (P ) can be deﬁned by the optimal value of the following Euclidean distance minimization: Rk (P )2 := minimize α k T 2 subject to i=1 ((pl ) xi ) ≤ α. vi i=1 2 = Trace(ppT T ∗ λi vi vi ) = Trace(ppT X ∗ ) ≤ αk ≤ Rk (P )2 .20) 0. 5. is at least k. r r λi p. vr be the corresponding eigenvectors. (5. say r. We claim that the rank of X ∗ . (ii). Trace(X ∗ ) = k implies that the sum of the eigenvalues of X ∗ is equal to k. SDP FOR GEOMETRY COMPUTATION Theorem 5. · · · . its projected value of (5. which implies that the rank of X ∗ is at least k. In other words. v2 . Lemma 5.10 There exists an integer r ≥ k such that we can compute. λ2 . is the minimum of the Euclidean radius of P projected onto a k-dimensional subspace. .20).5 A special case: The k-radius of P The outer-radius. · · · . for any p ∈ P . λr be the r nonnegative eigenvalues and r v1 . vi 2 ≤ Rk (P )2 . X (5. statistics and clustering. Therefore. and I − X ∗ 0 implies that the all eigenvalues are less than or equal to 1. Rk (P ). This follows from the fact that Trace(X ∗ ) = k and I − X ∗ 0. r nonnegative reals λ1 . ∀pl ∈ P..20).124 CHAPTER 5.17) is at most 12 log(n) of the minimal one. I − X 0. 2 xi = 1. in polynomial time.. (iii). k. v2 . Furthermore. ∀i = j ≤ k. ∀pl ∈ P. (xi )T xj = 0. We solve the semideﬁnte program (5. and has received considerable attention in the computational geometry literature [60. a (n − k)-subspace such 2 that. · · · . 153]. Proof. i=1 . λ2 . max1≤i≤r λi ≤ 1. and let X ∗ be an optimal solution of (5. X ∗ has at least k non-zero eigenvalues. with probability at least 1 − n .9 We can computed in polynomial time. l Trace(X) = k. Then we have i=1 λi = k and max1≤i≤r λi ≤ 1. for any p ∈ P . i = 1.

r}. Lemma 5. Li := Li + λj and I := I − {j}. we would have not assigned λs to It in the algorithm. 2. j∈Ii λj ≤ 1. set Ii = ∅ and let Li = 0. Assume that there exists 1 some t such that j∈It λj < 2 . Ii ∩ Ij = ∅. However. This method is suggested by an anonymous referee. and for any i = j. j∈Ii λj ≤ 1 2 for all i. suppose j∈It λj > 1 for some t . Now we prove condition (2) by contradiction.1 125 Deterministic First Rounding In this section. · · · . This However. choose set i with the smallest value Li . Otherwise. Note that λj ≤ 1 for every j and thus there are at least two eigenvalues are assigned to It . we know that. An alternative way of partitioning the eigenvalues is the following: First. say l. While I = ∅ choose j from I with the smallest index. · · · . · · · . Recall that j=1 λj = k and 0 ≤ λj ≤ 1 for all j. λs > j∈It λj − 1 > 1 . Notice that the running time of the partitioning algorithm is bounded by O(r · k). 2. · · · . Denote the last eigenvalue by λs . For i = 1. otherwise. Therefore. is not less than k. r} can be partitioned into k sets I1 . by condition (i). ∪k Ii = {1. The proof of the lemma is elementary but it plays an important role for proving our main result. I2 . · · · . It follows that j∈It λj − λs = j∈It \{s } λj ≤ j∈It λj since. 2 2 2 This is impossible since λs is the last eigenvalue assigned to It . 2. r Proof. · · · . If the number of such eigenvalues. j∈Ii λj = results a contradiction. r}. It follows that k i=1 k i=1 j∈Ii λj r j=1 λj = k. 2. which implies λs ≤ λj for every j ∈ It . we 2 must have j∈It λj − λs = j∈It \{s } λj < 1 . Let Ii := Ii ∪ {j}. The algorithm works as follows: STEP 1. I2 . we can assume that λ1 ≥ λ2 ≥ · · · ≥ λr . the sets I1 . We now claim that. It is clear that when the algorithm stops. arbitrarily put the remaining eigenvalues into k − l subsets such that the sum of eigenvalues in each subset is greater than or equal to 1/2. A SPECIAL CASE: THE K-RADIUS OF P 5. Without loss of generality. k. Otherwise.5. For any i : 1 ≤ i ≤ k. Ik satisfy condition (i). and in particular j∈It λj < 1 2. i=1 (ii). . Thus. for all i. STPE 2. < k. since j∈It λj < 1 . Our partitioning algorithm is the same as the Longest-Processing-Time heuristic algorithm for parallel machine scheduling problem.5. such t does not exists and we have proved condition (ii). Ik such that (i). put the eigenvalues that are greater than or equal to 1/2 into distinct subsets. j∈Ii 1 λj ≥ 2 .11 The index set {1.5. and we have already proved that there must exist an l such that s = l ∈ It and λl ≤ j∈It \{s } λj < 1 . Therefore. then we are done. we prove a lemma concerning how to deterministically group the eigenvalues and their eigenvectors. Let I = {1.

xk form a feasible solution for the original problem. · · · . let j∈Ii xi = φj λj · vj j∈Ii λj . · · · . 2. · · · . By deﬁnition. xt λj vj . −1 or 1 with probability 1 each way. Then our next randomized rounding procedure works as follows. · · · . For i = 1. xs . 2 STEP 2. 2. xi = j∈Ii = λj λj φj λj λj v j λj λj vj . vj = 0 for any l = j and vj = 1. The following Lemmas show that x1 . φj φj λj v j 2 (φj )2 λj vj λj v j 2 j∈Ii j∈Ii 1 j∈Ii j∈Ii j∈Ii 1 j∈Ii λj .12. Generate a r dimensional random vector φ such that each entry of φ takes value. Recall that vl . In other words. STEP 1. Lemma 5. The proof is similar as that of Lemma 5. k. xt = 0.13 If s = t then xs .5. xi = 1. x2 . Ik . I2 . j∈Ii 1 j∈Ii = λj 1 j∈Ii = λj 1 j∈Ii = λj v j φj j∈Ii φj j∈Ii = λj j∈Ii = 1 Lemma 5. Proof.126 CHAPTER 5. they are k orthogonal unit vectors. xi 2 = xi . φj j∈Ii φj λj vj . SDP FOR GEOMETRY COMPUTATION 5. Proof.12 For i = 1. independently. k.2 Randomized Second Rounding Assume now that we have found I1 .

5. xi = 2 j∈Ii p. Now we establish a bound on the performance of our algorithm. The last equality holds since for any j ∈ Is and l ∈ It . vj )2 = j∈Ii λj · p. we have prob{ p. j ∈ Ii . 2. λj v j φ j j∈Ii 2 ≤ 2 p. Then. 2 λj p. j∈Ii = 2 λj vj φj λj 2 1 j∈Ii λj p. A SPECIAL CASE: THE K-RADIUS OF P φj λj v j j∈Is j∈Is = λj λj v j j∈It λj 1 = j∈Is = φj j∈It . vj . j∈Ii On the other hand. e. n3 Proof. λj v j φ j j∈Ii (since λj ≥ j∈Ii 1 ) 2 . [238]). First. vl = 0. which is a form of the Chernoﬀ Bound. Lemma 5. xi 2 > 12 log(n) · Cip } < 2 . First notice that e 2 = ( λj p. Furthermore. · · · . for any vector e and β > 0. respectively. k and each p ∈ P .5 each way. let us introduce Bernstein’s Theorem (see. deﬁne a |Ii | dimensional vector e such that its entries are λj p.5. Given i and p. λj · j∈It λj 127 φj φj λj v j . we deﬁne the vector φ|Ii whose entries are those of φ with indices in Ii . λj v j j∈It j∈Is 0.g. vj . Then we have Lemma 5.14 Let φ be a random vector whose entries are independent and either 1 or −1 with probability . e Let Cip = j∈Ii β > β e 2 } < 2 · exp(− ). p. 2 prob{ φ. vj 2 ..15 For each i = 1. vj 2 = Cip .

6 Distributed SDP computing The SDP problems presented in this chapter can be solved in a distributed fashion which has not been studied before. 5.14 and by letting β = 6 log(n). e 2 > 6 log(n) e 2 }.10. SDP FOR GEOMETRY COMPUTATION  2 = 2 λj φj p. a (d − k)-ﬂat such that.p Bip .16 We can computed in polynomial time. We ﬁrst partition the anchors into many clusters according to their physical positions. p. 2 with probability at least 1 − n . xi 2 > 12 log(n) · Cip } < i. Proof. e 2 Thus prob{ p. then for any i and p. · · · . The probability that the event B happens is bounded by prob{ p. Therefore. xi i=1 2 Cip ≤ 12 log(n) · Rk (P )2 . 2. Theorem 5. xk . the distance between any point p ∈ P and F is at most 12 log(n) · Rk (P ). 3 n n If B does not happen. for each p ∈ P . ≤ 12 log(n) i=1 The last inequality follows from Lemma 5.128 CHAPTER 5. xi 2 ≤ 12 log(n) · Cip . For given i = 1. Here we describe an iterative distributed SDP computation scheme to for sensor network localization. xi 2 > 12 log(n) · Cip } and B = ∪i.p 2kn 2 ≤ . k k p. the conclusion of the lemma follows by using Lemma 5. x2 . xi 2 > 12 log(n)Cip } ≤ prob{ φ|Ii . vj  j∈Ii = 2 φ|Ii . and assign some sensors into these clusters if a sensor has a direct connection to one of the . k and p ∈ P . consider the event Bip = {φ| p. This completes the proof by taking F as the ﬂat which is orthogonal to the vectors x1 . Therefore. · · · .

This is done so as to choose the best estimation of the particular sensor from the estimations provided by solving the diﬀerent clusters. i. the constraints between two remote points. and then resolve it with a “warm-start” solution. Partition the anchors into a number of clusters according to their geographical positions. If it is below a predetermined small tolerance.6. we adapt an iterative active constraint generation method. many of those ”bounding away” constraints. For each cluster of anchors and unknown sensors. are inactive or redundant at the optimal solution.. 5. After solving each SDP model. If it does. If a sensor is assigned in multiple clusters.7) for each unknown sensor in the model. and solve the resulting SDP model if the number of anchors is more than 2. we choose the xj ¯ that has the smallest individual trace. First. 2. The only information that needs to be passed among the neighboring clusters is which of the unknown sensors become positioned after a round of SDP solutions. . We can repeat this process until all of the constraints are satisﬁed. In solving the SDP model for each cluster. DISTRIBUTED SDP COMPUTING 129 anchors. even if the number of sensors is below 100. it becomes an unknown sensor point in the cluster to which the anchor belongs. check the individual trace (5. Note that a sensor may be assigned into multiple clusters and some sensors are not assigned into any cluster. The distributed scheme then repeats. the total number of constraints could be in the range of thousands. Note that the solution of the SDP problem in each cluster can be carried out at the cluster level so that the computation is highly distributive. A round of the distributed computation method is straightforward and intuitive: 1. Secondly we verify the equality and inequality constraints and add those violated at the current solution into the model. Typically. label the sensor as positioned and its estimation xj becomes an ¯ “ anchor”.e. We then solve semideﬁnite programs independently at each cluster. and ﬁx those sensors’ positions which have high accuracy measures according the SDP computation. Each (unpositioned) sensor sees if it has a direct connection to an anchor (within the communication range to an anchor). Therefore. These positioned sensors become “ghost anchors” and are used to decide the remaining un-positioned sensors. Consider positioned sensors as anchors and return to Step 1 to start the next round of estimation. However. we partition the entire sensor area into a number of equal-sized squares and those anchors in a same square form a regional cluster. In our implementation.5. formulate the error minimization problem for that cluster. each cluster has less than 100 sensors and the model can be solved eﬃciently. 4. 3. we solve the problem including only partial equality constraints and completely ignoring the bounding-away inequality constraints to obtain a solution.

radio-range=0. It is usually the outlying sensors at the boundary or the sensors which do not have many anchors within the radio range that are not estimated in the initial . 5. where the entire sensor region is partitioned into 10 × 10 equal-sized squares.5 0. noisy-factor=0.5 Figure 5.2 −0. that is.0.2 GHz and 500 MB PC.1 0 0. 000 to 4.1 −0. One simulation solves a network localization with 4.2 0.3 0.5 .4 0. anchors are also uniformly distributed in the same random manner. and the radio range is set at .2 0.6. The tolerance for labeling a sensor as positioned is set at 0. We generally select the ﬁrst 5−10% of the points as anchors. is about four minutes using DSDP2. The total solution time for the ﬁve round computation on ˆ the single Pentium 1.1 Preliminary computational and simulation results Simulations were performed on networks of 2.5 .5: Third and ﬁnal round position estimations in the 4.5] × [−.035. 000 sensor network. only about O(n + m) constraints are active at the ﬁnal solution so that the total number of constraints in the model can be controlled at O(n+m).045.3 0.01 · (1 + noisyf actor) · radiorange. 0.4 0. that is.5].3 −0. The ﬁnal solution is ploted in Figure 3.1 0 −0. excluding computing dij . 000 sensor points which are randomly generated in a square region of [−. and the number of clusters=100. 100 clusters.4 −0. 000 sensors.1 0.130 CHAPTER 5. SDP FOR GEOMETRY COMPUTATION Typically.

there has been attempts to characterize families of graphs that admit polynomial time algorithms for computing a realization in the required dimension. and more recently. 270. 159. . .) However. . Under certain non– degeneracy assumptions. wireless sensor network localization [5. . the vertices of G correspond to sensors. Furthermore. Thus. For instance. whose positions are unknown. the question of whether a given instance has a unique realization on the plane can be decided eﬃciently [167]. the edges of G correspond to communication links. . In many sensor networks applications. It is interesting to note that from an algorithmic viewpoint. n such that (i) the ﬁrst d + 1 vertices form a complete graph. We shall refer to this problem as the Sensor Network Localization problem. more and more of these points are positioned. 273]. 269. d + 1. NOTES 131 stages of the method. Note that we can view the Sensor Network Localization problem as a variant of the Graph Realization problem in which a subset of the vertices are constrained to be in certain positions. 268. dimensionality reduction. This clearly indicates that the clustering approach prevents the propagation of errors to other clusters. 79. In the sensor networks setting. 159. d + 2. Eren et. the question of whether there exist larger families of graphs with such property is open. The goal is to determine the positions of all the sensors. 273]. Euclidean ball packing. 137]. . the two problems above have very diﬀerent characteristics. in R2 ). the vertices are partitioned into two sets – one is the anchors. 269. even if the given instance has a unique realization on the plane [24]. On another front. We have also estimated the same 4.5. It is noted that the erroneous points are isolated within particular regions. 5. 89. Indeed. We feel that our distributed computation techlogy is promising for solving very very large scale distance geometry related problems. such as molecular conformation. 79. Gradually. it is not surprising that all the aforementioned heuristics for computing a realization of a given instance do not guarantee to ﬁnd it in the required dimension. 000 network using noisy data. 51.7 Notes The distance geometry problem and its variants arise from applications in various areas. (A graph is a trilateration graph in dimension d if there exists an ordering of the vertices 1. for example). [89] have shown that the family of trilateration graphs has such property. . another related question is whether the given instance has a unique realization in the required dimension (say. while the problem of computing a realization on the plane is NP–complete in general. . and the other is the sensors. whose exact positions are known (via GPS. and the other deals with determining whether a given instance has a unique realization in Rd using graph rigidity [89. al. and the weights correspond to distances. sensors collect data that are location– dependent. most of the previous works on the Sensor Network Localization problem fall into two categories – one deals with computing a realization of a given instance [51. . Thus.7. as the number of well estimated sensors or ’ghost’ anchors grows. and (ii) each vertex j > d + 1 has at least d + 1 edges to vertices earlier in the sequence.

5. the width of point set P . 203]. i<j (αij ) + 2 2 ˆ xi − xj = (dij ) + αij . However. xi − xj 2 ≥ R2 . For some related work in this direction. ak − xj 2 ≥ R2 . The SDP models were ﬁrst used by [63. these papers mostly addressed the question of realizability of the input graph. . ˆ = dkj + αkj . these models are not quite suitable for our application. This ratio also matches the previously best known ratio for approximating the special case R1 (P ) of [238]. for the rest i < j.j∈Nx . For instance.j∈Nx . which√ improves the ratio √ of [317] by a factor of O( log d) that could be as large as O( log n). 34] has studied this problem in the context of quadratic maps and used SDP theory to analyze the possible dimensions of the realization. j.2 Find the SDP relaxation to the following minimization problem minimize subject to 2 i. for the rest i < j.j∈Na (αkj ) i. i < ˆ ak − xj 2 = (dkj )2 + αkj .. i<j (αij ) xi − xj ak − xj xi − xj ak − xj + k.j∈Na (αkj )2 ˆ = dij + αij . j. for the rest k. it would be interesting to extend our method to compute low–distortion realizations in a given dimension. e. 2 ≥ R2 . j. 2 ≥ R2 . Thus. j) ∈ Na .132 CHAPTER 5. j) ∈ Nx . Alfakih and Wolkowicz [6.g.1 Find the SDP relaxation to the following minimization problem minimize subject to 2 2 k. Thus. SDP FOR GEOMETRY COMPUTATION We should mention here that various researchers have used SDP to study the distance geometry problem (or its variants) before. i < j. In contrast. Moreover. 5. Barvinok [33. these methods can only guarantee to ﬁnd a realization that lies in a range of dimensions. j) ∈ Na . our analysis takes advantage of the presence of anchors and gives a condition which guarantees that our SDP model will ﬁnd a realization in the required dimension. for (k. We remark that SDP has also been used to compute and analyze distance geometry problems where the realization is allowed to have a certain amount of distortion in the distances [35. for the rest k. see. The result on the radii of point sets is due to [339]. and the results described here were based on their work. and the analyses of their SDP models only guarantee that they will ﬁnd a realization whose dimension lies within a certain range. ∀ (i. for (k. j) ∈ Nx . 7] have related this problem to the Euclidean Distance Matrix Completion problem and obtained an SDP formulation for the former. Alfakih has obtained a characterization of rigid graphs in [3] using Euclidean distance matrices and has studied some of the computational aspects of such characterization in [4] using SDP. Again.8 Exercises 5. and interpret the objective function. ∀ (i. [26]. 64] for the ad hoc wireless sensor network localization.

EXERCISES 133 and interpret the objective function.3 The Un-Folding Problem: with the same notions. Show that the graph is stronly localizable if and only if the sensor’s position lies in the liner space of the three anchors’ positions. 5. and let the three distances from the sensor to three anchors are given. 5. j) ∈ Nx (5.4 Given one sensor and three anchors in R2 . consider the problem maximize subject to i ak xi 2 − xj xi − xj 2 ¯ = d2 kj ∀ (k. How to incoporate this information in the SDP relaxation of the distance geometry problem? . 5.6 Suppose for the angle of two edges joint at a point is known.2(b) are strongly localizable. and explain what they are. 5.5.8. j) ∈ Na 2 = d2 ij ∀ (i.5 Prove that the graphs depicted in Figures 5.21) Find the SDP relaxation and its dual.2(a) and 5.

SDP FOR GEOMETRY COMPUTATION .134 CHAPTER 5.

and K is a convex cone. ξ) ∈ K ⊂ Rm . ξ may not be certain. • Knowledge of ξ belonging to a given uncertain set U .2) . we assume ξ is known and ﬁxed. • The constraints must be satisﬁed for every ξ in the uncertain set U . ξ) Subject to F (x.1) where ξ is the data of the problem and x ∈ Rn is the decision vector. ξ) ∈ K (ROPT) for all Other methods dealing with data uncertainty include: 135 ξ ∈ U.1 Robust Optimization Consider an optimization problem Minimize f (x. ξ) Subject to F (x. For deterministic optimization. (OPT) (6. (6. ξ). • Optimal solution must give the best guaranteed value of supξ∈U f (x. In reality. This leads to the so-called robust counterpart: Minimize supξ∈U f (x.Chapter 6 SDP for Robust Optimization 6.

This problem may be a semin-inﬁnite programming problem.1. is the Euclidean norm. A1 x. ≤1 . ξ k ) ≤ z for large sxamples ξ k ∈ U. 6.2 Robust Quadratic Optimization qT x Minimize (EQP) Subject to Ax 2 (6. vector q ∈ Rn and A ∈ Rm×n .1 Stochastic Method Minimize Eξ [f (x. ξ)] Subject to Eξ [F (x.2 Sampling Method Minimize z Subject to F (x.3) ≤ 1.1 Ellipsoid uncertainty Let A be uncertain and k A = A0 + uj Aj | uT u ≤ 1. SDP FOR ROBUST OPTIMIZATION 6. ξ k ) ∈ K (SOPT) f (x. 6. (EOPT) 6. j=1 Minimize qT x (REQP) Subject to (A0 + k j=1 uj Aj )x ≤ 1 ∀uT u ≤ 1. and . · · · Ak x).136 CHAPTER 6. uj Aj )x ∀uT u ≤ 1. Here.2. ξ)] ∈ K. The problem can be rewritten as Minimize qT x (REQP) Subject to Let (A0 + k j=1 2 F (x) = (A0 x.1.

2 Let P a symmetric matrix and A be a rectangle matrix.  0 I T F (x) F (x)  0 I which is a SDP constraint since F (x) is linear in x. .2. Here we have used another technical lemma Lemma 6.2 1 0 1 0 − F (x)T F (x) 0 −I 1 u 1 u ≥0 ≥ 0. we immediately see T 1 u 1 0 1 u ∀ 0 0 1 u − F (x)T F (x) T 1 0 1 u 0 −I ≥0 ≥0 if and only if there is a λ ≥ 0 such that 1 0 This is equivalent to   0 0 1 0 − F (x)T F (x) 0 0 +λ −1 0 λ 1 0 0 −I .6. From the lemma. Then P − AT A if and only if P A AT I 0 0. ROBUST QUADRATIC OPTIMIZATION Then k (A0 + uj Aj )x 2 T 1 u = j=1 137 1 u F (x)T F (x) The problem becomes qT x Minimize (REQP) T 1 u Subject to 0 0 T 1 u ∀ 6.1 Let P and Q be two symmetric matrices such that there exists u0 satisfying (u0 )T P u0 > 0. Then the implication that uT P u ≥ 0 ⇒ uT Qu ≥ 0 holds true if and only if there exists λ ≥ 0 such that Q λP. S-Lemma Lemma 6.2.

j=1 Minimize qT x Subject to −xT AT Ax + 2bT x + γ ≥ 0 (REQP) ∀uT u ≤ 1. .2. γ) = (A0 .4) Here. 6. · · · Ak x). Let again F (x) = (A0 x.3. A1 x. b. bj . b0 . I General Robust Quadratic Optimization Minimize qT x Subject to −xT AT Ax + 2bT x + γ ≥ 0. γ 0 ) + uj (Aj . b ∈ Rn and and A ∈ Rm×n .138 CHAPTER 6. (EQP) (6. REQP ﬁnally becomes a SDP problem: qT x Minimize (REQP)   Subject to 6. Then −xT AT Ax + 2bT x + γ =  1 u T γ 0 + 2xT b0  γ 1 /2 + xT b1   ··· γ k /2 + xT bk γ 1 /2 + xT b1 0 ··· 0 ··· ··· ··· ···   γ k /2 + xT bk   0  − F (x)T F (x)   ··· 0 1 u .3 1−λ 0 0 λI F (x)  T F (x)  0.1 Ellipsoid uncertainty Let A be uncertain and k (A.3 SDP for Robust QP Consider λ and x as variables. vector q. γ j )| uT u ≤ 1. SDP FOR ROBUST OPTIMIZATION 6.

6. .. MORE ROBUST QUADRATIC OPTIMIZATION 6.   F (x)T     I 0. i i (REQP) ∀uT u ≤ 1. bj . . i i 0 (Ai . t. we immediately see if and only if there is λ ≥ 0 such that  0  γ + 2xT b0 γ 1 /2 + xT b1 · · · γ k /2 + xT bk  γ 1 /2 + xT b1  0 ··· 0  −F (x)T F (x)) λ 1 0   0 −I ··· ··· ··· ··· γ k /2 + xT bk 0 ··· 0 which is is equivalent to   0 γ + 2xT b0 γ 1 /2 + xT b1   γ 1 /2 + xT b1 0     ··· ···   γ k /2 + xT bk 0  · · · γ k /2 + xT bk  ··· 0 +λ  ··· ··· ··· 0 F (x) . b0 ..2 139 SDP formulation From the S-lemma... REQP ﬁnally becomes a SDP problem: Minimize (REQP)   Subject to 6.3. . Consider λ and x as variables. t.4 qT x γ 0 + 2xT b0 − λ γ 1 /2 + xT b1   γ 1 /2 + xT b1 λ     ··· ···   γ k /2 + xT bk 0 F (x) ··· ··· ··· ···  γ k /2 + xT bk  0   ··· λ More Robust Quadratic Optimization Minimize qT x Subject to −xT AT Ai x + 2bT x + γi ≥ 0 i i (EQP) i = 1.4. vector q. which is a SDP constraint since the matrix is linear in x. bi ..1 Ellipsoid uncertainty Let A be uncertain and k j uj (Aj . γi ) + i i j=1 Minimize qT x Subject to −xT AT Ai x + 2bT x + γi ≥ 0.5) Here. (6.  −1 0 0 I  F (x)T     I 0.. bi ∈ Rn and and Ai ∈ Rm×n . γi ) = (A0 . γi )| uT u ≤ 1. i = 1.4. 6.

ξ) ≤ 0 for all ξ ∈ U (ROPT) dual constraints on λ0 .. t.t. . x) Subject to F (x. (ROPT) (6. .. x) where λ0 is the dual variables. SDP FOR ROBUST OPTIMIZATION 6. ξ(u)) ξ∈U ˆ u∈U In convex cases. i = 1. t.5 Tool for Robust Optimization Recall: Minimize supξ∈U f (x.140 CHAPTER 6. 6. ξ) = sup f (x. The dual objective function is an upper bound on f (x.2 SDP formulation Let again Fi (x) = (A0 x..7)       0. ξ) Subject to F (x. we represent sup f (x.. Then. and x as variables. . ξ(u)) ˆ u∈U by its conic dual problem and let the dual objective function be φ(λ0 .6) Express ξ = ξ(u) where ˆ u ∈ U. ξ(u)) so we could represent the problem by Minimize φ(λ0 . 0 γi + 2xT b0 − λi i   γ 1 /2 + xT b1 i i     ···  k  γi /2 + xT bk i 1 γi /2 + xT b1 i λi ··· 0 Fi (x) ··· ··· ··· ···  k γi /2 + xT bk i  0  Fi (x)T  ··· λi I i = 1. A1 x. sup f (x.4. REQP ﬁnally becomes a SDP problem: Min q T x   s. ξ) ≤ 0 for all ξ ∈ U. · · · Ak x).. i i i Consider λi . (6..

x) (ROPT) (6. i=1 Case of u ∞ ≤ 1: k a0 x + b0 + max u: u ∞ ≤1 ui (ai x + bi ) i=1 The dual is Minimize λ Subject to λ − (a0 x + b0 ) ≥ k i=1 |(ai x + bi )|. and the problem becomes Minimize φ(λ0 . the robust constraint becomes k 0 ≥ λ ≥ (a0 x + b0 ) + (ai x + bi )2 . . b0 ) + ˆ u ∈ U. TOOL FOR ROBUST OPTIMIZATION 141 We handle the constraints in the exact same manner. ui (ai . component-wise.8) Subject to Φ(λ.5. where Φ(λ.1 Examples ax + b ≤ 0 where k (a. 6. bi )| i=1 Case of u ≤ 1: k a0 x + b0 + max u: u ≤1 ui (ai x + bi ) i=1 The dual is Minimize λ Subject to λ − (a0 x + b0 ) ≥ k i i=1 (a x + bi )2 . b) = (a0 . ˆ u∈U and C is a suitable cone. the dual objective function of sup F (x. x) is. λ.6. Thus. x) ∈ C dual constraints on λ0 . ξ(u)).5.

i 6. i=1 Case of u 1 ≤ 1: k a0 x + b0 + max u: u 1 ≤1 ui (ai x + bi ) i=1 The dual is Minimize λ Subject to λ − (a0 x + b0 ) ≥ maxi {|(ai x + bi )|}. the robust constraint becomes k 0 ≥ λ ≥ (a0 x + b0 ) + |(ai x + bi )|.142 CHAPTER 6. c) are from the set k (A. These voltages induce currents in the circuit. 2 If the data (A. SDP FOR ROBUST OPTIMIZATION Thus. i=1 the robust counter part can be drawn from the earlier disscusion. each of which possesses a given conductance sij . 6. the circuit dissipates heat. c0 ) + ui (Ai . Thus. ci )| u ≤ 1. the heat is given by H = min (Qv + P y)T S(Qv + P y). m v∈R . since the original problem is a convex QP problem. the robust constraint becomes 0 ≥ λ ≥ (a0 x + b0 ) + max{|(ai x + bi )|}.6 6.1 Applications Robust Linear Least Squares Minimize Ay − c y ∈ Y. Assuming that the arcs of the circuit are charged with external voltages y and internal voltages v. According to Kirchoﬀ’s laws. as a result.6. c) = (A0 .6.2 Heat dissipation problem Consider an electric circuit represented as a connected oriented graph Γ with N arcs.

6. k 0 0 ui (V i .6. APPLICATIONS 143 where Q and P are some matrices given by the topology of the circuit. ˆ u∈U Portfolio optimization Minimize xT V x Subject to µT x ≥ α eT x = 1. (V. Now V and µ may be uncertain. Since (Qu + P y)T S(u)(Qu + P y) is convex in (y.6. m y∈Y v∈R where Y is a convex set. µ ) + i=1 The robust counterpart of the portfolio problem is Minimize maxV ∈V [xT V x] Subject to minµ∈S [µT x] ≥ α eT x = 1 . this problem is equivalent to min y∈Y. and S is N × N diagonal matrix with sij as its diagonal entries. Now if k S = S(u) = S 0 + ui S i | u ≤ 1. The problem is to choose y such that min min (Qv + P y)T S(Qv + P y) .v∈Rm 6. µi )| u ≤ 1. v) and linear in u.3 max(Qv + P y)T S(u)(Qv + P y) . µ) = (V . i=1 we have a robust optimization counterpart: min max min (Qv + P y)T S(u)(Qv + P y) m y∈Y v∈R ˆ u∈U . m is the number of nodes of the circuit. for example.

. .2 In the linear least squares problem.4 In the portafolio optimization problem.. assume that k (A. µi )| u ≤ 1. i=1 and V i 0 for all i = 0. Construct the robust counterparts of the problem for three norms on u: u 2 ≤ 1. c) = (A0 . c0 ) + ui (Ai . Construct the robust counterparts of the problem for three norms on u: u 2 ≤ 1. k. SDP FOR ROBUST OPTIMIZATION Exercises 6.3 In the Heat dissipation problem. 6. and u ∞ ≤ 1..7 CHAPTER 6. assume that k (V. µ) = (V 0 .144 6. i=1 and si 0 for all i = 0. assume that k S = S(u) = S 0 + ui S i | u ≤ 1. i=1 Construct the robust counterparts of the problem for three norms on u: u 2 ≤ 1. u 1 ≤ 1. 6. k.. 6. and u ∞ ≤ 1.1 Prove the S-lemma. . and u ∞ ≤ 1.. u 1 ≤ 1. µ0 ) + ui (V i . u 1 ≤ 1. .. ci )| u ≤ 1.

Chapter 7 SDP for Quantum Computation 7. V = i=1 where λi ≥ 0 for all i and i λi = 1. a quantum state. Nowadays. we restrict ourselves to ﬁnite-dimensional real vector spaces. The ﬁeld is closely linked to Quantum Information Theory and Statistics. In classical quantum physics. another formalism is widely accepted. and T r(V ) = 1.1 Quantum Computation Quantum Computation uses Quantum Mechanics to enhance computation power. A density matrix of rank 1 is called a pure sate. First concrete result was developed for Integer Factorization by Shor in 1994. in which a state is a postive semideﬁnite symmetric matrix V with trace 1 that belongs to RN ×N . or a state. and then for Quantum search algorithm and space reduction by Grover in 1996. Quantum Channels and Optimization. and a density matrix of rank higher than 1 is called a mixed state. In this lecture. A matrix of such property is called a density matrix. is N a √ column vector v in the N -dimensional vector space R with norm ||v||2 = v · v = 1. For example. Hilbert sapces and linear operators are generally used. a density matrix V with rank one can be represented as V = vv T . The state V is interpreted as a probalistic T mixture of pure states vi vi with mixing probabilities λi . 145 . A density matrix of rank higher than 1 can be written as N T λi v i v i . To describe quantum mechanics.

For a measurement M = {M1 . M2 . .e. Mj 0. . Given {A1 . we obtain a random varible X with Pr(X = l) = V • Ml . 1) represent the classical bits. i. ∀i = 1. k. i Let Wj = i ξi Vi cij . . Mk } whose sum is an identity matrix.. . · · · . Then the average cost is C(M ) = ξi P (j|i)cij = i. · · · . Mk } performed on the system. A measurement is described as a ﬁnite set of symmetric positive semideﬁnite matrices {M1 .2 CHAPTER 7.146 7. j = 1. k. such that k Ai AT = I. . · · · .j Tr j ξi Vi cij Mj . Given the system in state V and a measurement {M1 . k. When n = 1. · · · . . · · · . A2 . a unit orthogonal (1. the pure state n is in R2 or N = 2n . where Ai ∈ RM ×N . ξ2 . In quantum computing. ξm . M2 . . . · · · . · · · . called n-qubit. SDP FOR QUANTUM COMPUTATION Completely Positive Map The state of a system may change as time goes on. . and Mi = I. 0) and (0. . l = 1. . j = 1. i T (V ) = i=1 Note that T (V ) ∈ RM ×M . i=1 called Positive Operator-Valued Measures (POVM). k Mi 0. . Vm with corresponding prior probabilities ξ1 . k. Ak }. we have P (j|i) = T r(Vi Mj ). The problem of minimizing the average cost can be formulated as the following SDP: Minimize k j=1 T r(Wj Mj ) Subject to k j=1 Mj = I. i i=1 trace-preserving complete positive (CP) map of V is k Ai V A T . Let cij (≥ 0) be the cost (penlaty) of taking a true state i to be j. V2 . . so CP is a linear mapping that conveys quantum states in RN ×N to the states in RM ×M . Mk }. Suppose we are given m states V1 . A fundamental problem is quantum detection theory is to ﬁnd an optimal measurement for given mixed states.

i = 1. . k. V ) Subject to m i=1 pi = 1. with p = {pi } pi ≥ 0. Then. V ) = pi D(Vi· ||pV ). j=1 Therefore. bn } and a set of conn ditional probabilities Vij = P (bj |ai ) with Vij = 1. j = 1. . .. rj is a measure of distance between two probability distributions q = (qj ) and r = (rj ). where Vij is the probability that bj is received when ai is sent through the channel. m. V ) is concave in p = (pi ). . The capacity C(V ) of a channel V . the optimality Condition becomes k Wi k Mj Wj = j=1 7. V ) = pi Vij Vij log i pk Vkj j k The Kullback -Leibler divergence. . . . j=1 Channel Capacity Problem A classical discrete memoryless channel consists of a set of input symbols A = {a1 . . am }. is given by following theorem due to Shannon. the computatution of C(V ) is a concave maximization problem with linear constraints. .. CHANNEL CAPACITY PROBLEM 147 What is the dual? k j=1 Maximize Subject to T r(U ) Wj U. . The classical channel coding theorem is modeled as: C(V ) Maximize I(p. . . for i = 1. . With a channgel V ﬁexed. deﬁned as the maximum of rates (speeds) at which information can be transmitted reliably. b2 . Hence. k.7. deﬁned as follows D(q||r) = qj log j qj . where I(p. . . . . and plays an important role in information theory. m. .3 Wj Mj . I(p. . V ) can be written in terms of the above KullbackLeibler divergence I(p. i = 1.3. . i where r = pV is a distribution deﬁned by rj = i pi Vij .. . a2 . a set of output symbols B = {b1 . V = {Vij } is a stochastic matrix. . Thus. the function I(p.

in general. I({λi }. Γ) is a concave function in λ = {λi }. A2 . where d = n2 . . and ¯ λi T r(Γ(Xi )) log T r(Γ(Xi )) − log T r(Γ(X)) . {Xi }. Bi ∈ RM ×N . Γ) can be written in terms of the quantum counterpart of the Kullback-Leibler divergence (often called relative entropy) D(X||Y ) = T r(X)[log T r(X) − log T r(Y )]. The capacity C(Γ) of a quantum channel Γ. But. Γ) Subject to X i ∈ S1 .4 Quantum Interactive Proof System Interation Between Prover and Veriﬁer. {Xi }. T2 (V ) = i=1 .148 CHAPTER 7. . . {Xi }. where Ai . and a CP (map) Γ : S1 → S2 . Consider two CP mappings: given {A1 . {Xi }. S2 of output states (or density matrices) in Rn×n . This is again a convex programming problem. . Ak } and {B1 . Bl }. d λi = 1 λi ≥ 0. SDP FOR QUANTUM COMPUTATION A quantum memoryless channel consists of a set S1 of input states (or density matrices) in Rm×m . B2 . Γ) = i ¯ with X = i λi Xi . . such that l k Ai AT = I i T Bi Bi = I. {Xi }. i = 1. Γ) is neither convex nor concave in (λ. . . S. Observe that I({λi }. X). deﬁned to be the maximum of rates (speeds) at which reliable transmission of classical information through the channel Γ is possible. and i=1 i=1 Deﬁne k Ai V AT i T1 (V ) = i=1 and l T Bi V Bi . 7. · · · . Once the ensemble X = {Xi } is ﬁxed. The Quantum channel coding theorem due to A. . d i=1 i = 1. I({λi }. Holevo is modeled by C(V ) Maximize I({λi }. I({λi }. d. · · · .

7. are there two states V1 and V2 such that T1 (V1 ) = T2 (V2 ).5. An SDP Formulation for this problem is Minimize t Subject to I • V1 = I • V2 = 1. NOTES 149 The question is.5 Notes 7. or for all V1 and V2 T1 (V1 ) − T2 (V2 ) ≥ .6 Exercises 0 0. V 2 0 T1 (V1 ) − T2 (V2 ) + t · I t · I − T1 (V1 ) + T2 (V2 ) 7. V1 . .

150 CHAPTER 7. SDP FOR QUANTUM COMPUTATION .

University of Waterloo. A. Pardalos. Y. Advances in Optimization and Parallel Computing. (2000) 32-41. IEEE Signal Processing Letters. Linear Algebra and Its Applications 325:57–70. [8] F. H. M. Combinatorial optimization with interior point methods and semi–deﬁnite matrices. 2002. Graph Rigidity via Euclidean Distance Matrices. 2001. Dept. Ageev and M. H. [10] F. Ph. Wolkowicz. Alfakih. 2000. SIAM J. [2] A. I. In P. and E. Opt. 151 . 1998. and Appl. Alizadeh.” Proceedings of ESA’2000. [4] A. On Rigidity and Realizability of Weighted Graphs. [6] A. 5:13– 51. 1999. [5] A. The Netherlands. Moulines. Euclidean Distance Matrices and the Molecular Conformation Problem. Alizadeh. Alfakih. Alfakih. H. H. [9] F. Optimization over the positive semi-deﬁnite cone: Interior– point methods and combinatorial applications. Khandani. Alfakih. Optimization. Wolkowicz. 1995. Research Report CORR 2002–17. [3] A. University of Minnesota. Y. University of Waterloo. Research Report CORR 98–12. Y. El Nahas. pages 1–25. June 2002. A. Y. North Holland. On the Embeddability of Weighted Graphs in Euclidean Spaces. Lecture Notes in Computer Science 1879. Sviridenko. Alizadeh. Alfakih.D. editor. MN. 1992. Abdi. of Combinatorics and Optimization. of Combinatorics and Optimization. 12:13–30.Bibliography [1] M. Y. Solving Euclidean Distance Matrix Completion Problems Via Semideﬁnite Programming. Interior point methods in semideﬁnite programming with applications to combinatorial optimization. Dept. Thesis. Comput. Linear Algebra and Its Applications 310:149–165. [7] A. Jard. “An Approximation Algorithm for Hypergraph Max k-Cut with Given Sizes of Parts. Minneapolis. Wolkowicz. Semideﬁnite positive relaxation of the maximum-likelihood criterion applied to multiuser detection in a CDMA context. A. 9(6):165– 167. Amsterdam. 1991.

Iowa City. Andersen and K. Alizadeh. Expander ﬂows. SIAM J. D. Department of Management Science. Vazirani. Vandenberghe. J. A newton barrier method for minimizing a sum of Euclidean norms subject to linear equality constraints. [22] S. Optimization. (1998) 305-311. den Hertog. and M. Complementarity and nondegeneracy in semideﬁnite programming. C. 1997. [14] E. Department of Management. Fampa. [18] K.-P.” Technical Report 659. 1993. M. Odense University. 1995. and T. Ye. Programming. 6:33-46. A. [17] K. Mathematical Programming. 1994. [13] B. Andersen and Y. On long step path following and SUMT for linear and quadratic programming. [15] E. “Primal-dual interior point methods for semideﬁnite programming. Christiansen. Convex optimization problems involving ﬁnite autocorrelation sequences. J. M. D. To appear in Math. On a homogeneous algorithm for the monotone complementarity problem. 1995. Denmark. M. [19] K. Anstreicher and M. Haeberly. Overton. To appear. Arora. [21] K. A.152 BIBLIOGRAPHY [11] F. The University of Iowa. 1996. Andersson and L. Andersen. The University of Iowa. 77:111–128. [16] G. 1997. Terlaky. D. Working Paper. Anstreicher. Alkire and L. 1990. Odense University. Department of Management Sciences. The 35th STOC’04 2004.” Information Processing Letters 65. “Better Approximation Algorithms for Set Splitting and Not-All-Equal SAT. Roos. Preprint 95-07. New York University. Algorithmica. Anstreicher. L. [20] K. Department of Mathematics and Computer Science. L. Computer Science Department. . A long step barrier method for convex quadratic programming. 1996. Alizadeh. S. Math. Overton. Programming. Rao and U. IA. M. and M. Programming. Working Paper. Courant Institute of Mathematical Sciences. geometric embeddings and graph partitioning. “A long-step path following algorithm for semideﬁnite programming problems. D. Andersen and E. [12] F. Engebrestsen. 10:365–382.” Working Paper. D. Anstreicher. 47:337–351. P. Math. Haeberly. The APOS linear programming solver: an implementation of the homogeneous algorithm. Denmark. A standard form variant and safeguarded linesearch for the modiﬁed Karmarkar algorithm.

Barvinok.. 1097. Hajiaghayi. [34] A. 2001. Programming. Proc. “Greedily ﬁnding a dense subgraph. A cutting plane method from analytic centers for stochastic programming. [29] Balas. Low– a Dimensional Embedding with Extra Information. Springer-Verlag. Badoiu. David Goldenberg. Geom.” J. A lift-and-project cutting plane algorithm for mixed 0-1 programs. 1986. and J. Iwama. [30] R. Demaine. K. Feb. 2001. The nonlinear geometry of linear programming. E. [27] M. 1989. E. A. 1995. “Approximate Clustering via Core-sets. Barvinok. B´r´ny and F¨redi. Barvinok. Computing the volume is diﬃcult. ACM Symp. B˘doiu. ACM. page 442447. 49(2):354–359. Comp.” in Proceedings of the 5th Scandinavian Workshop on Algorithm Theory (SWAT). Banavar and A. H. . Kalele. ALGOSENSORS 2004. Bahn. Asahiro. Tokuyama. J. O. [36] D. C. 38:91-109. 1993. AMS.” In Proc. 69:45–74. Geom. T. Part I: Aﬃne and projective scaling trajectories. Tamaki. Du Merle. In Proceedings aa u of the 18th Annual ACM Symposium on Theory of Computing. Proc. S. P. Har-Peled and P. 1995. 1996. [35] A. [28] O. Math. Transactions of the American Mathematical Society. [24] James Aspnes. [33] A. Ceria. 314(2):499–526. in LNCS 3121:32–44. On the Computational Complexity of Sensor Network Localization. 2004. 2002. S. 13:189–202. Har-Peled. D. 2003. Barequet and S. “Eﬃciently Approximating the MinimumVolume Bounding Box of a Point Set in Three Dimensions. Comp. [31] I. Indyk. 2002. Lecture Notes in Computers Science. Mathematical Programming. [25] M. 14th SODA 434-443.BIBLIOGRAPHY 153 [23] Y. Problems of Distance Geometry and Convex Properties of Quadratic Maps. Vial.. Theory of Computing. 20th SoCG 320– 329. Goﬃn. 2004. and Cornuejols. Indyk. A Course in Convexity. [32] G. Approximation Algorithm for Embedding Metrics into a a Two–Dimensional Space. 58:295– 324. Algorithms. A mixed norm performance measure for the design of multirate ﬁlterbanks. B˘doiu. 136-148. Yang Richard Yang. and T. IEEE Transactions on Signal Processing. A Remark on the Rank of Positive Semideﬁnite Matrices Subject to Aﬃne Constraints. Disc. [26] M. L. 2001. P. Disc. G. Bayer and J. M. New York. 25(1):23–31. Lagarias.

Free material optimization via semideﬁnite programming: the multiload case with contact conditions. 24(2). Boston. SIAM Journal of Optimization. Athena Scientiﬁc. May 1999. 1991.154 BIBLIOGRAPHY [37] D. Belmont. SIAM Review. 4:596–612. Ben-Tal and A. 1995. The complexity of approximating a nonlinear program. [42] S. The nonlinear geometry of linear programming. and order statistics. Bellare and P.mcs. Nemirovski. J. MA. Y. Bayer and J. Robust truss topology design via semidefinite programming. 2000. the multi-station case. Pardalos (Eds. Ye. 2000.gov/∼ benson. unix. Optimization Methods and Software. Bertsimas and J. DSDP User Guide. Nemirovski. Rogaway. C. November 2000. and X. Karmarkar’s linear programming algorithm and Newton’s method. Bertsekas and J. 2000.. Ye. [47] D.-Z. 1997. Bayer and J. SIAM J. Zhang. and X. Mathematics of Operations Research.M. Mixed linear and semideﬁnite programming for combinatorial optimization. multivariate normal distributions. P. Kluwer. Zowe. M. Belmont. Handbook of Combinatorial Optimization (Vol. Semideﬁnite relaxations. 1989. S. [49] D. Programming. Ben–Tal and A. pages 443–467. Introduction to Linear Optimization. 3). 10:443–461. 42(4):695–715. 50:291–330. 314(2):527–581. Part II: Legendre transform coordinates. Optimization. 1-19. C. Du and P. Ye. Lagarias. Nonlinear Programming. 1994. 1999. Ben-Tal and A. J. D. [39] M. [50] D. . Lagarias. In Handbook on Semideﬁnite Programming.anl. [43] A. SIAM J. Benson. [45] A. Nemirovskii. 1997. Ye. Benson and Y. [38] D. Solving large scale sparse semideﬁnite programs for combinatorial optimization. A. MA. Optim. A. Nemirovski. Ben-Tal. Benson. 7(4):991–1016. N. Bertsimas and Y. Tsitsiklis. Structural design via semideﬁnite programming. [44] A. Kocvara. J. [46] A. Y. Programming.) pp. Math. Zhang. http://www- [41] S. (1998 Kluwer Academic Publishers). and J. [48] D. A. Interior point polynomial time method for truss topology design. Nino-Mora. [40] S. Bertsekas. Athena Scientiﬁc. P. 69:429-442. Transactions of the American Mathematical Society. Math. 10:515–544. Optimization of multiclass queuing networks with changeover times via the achievable region approach: part ii. 1995.

Borchers. [54] H. V. The ellipsoidal method: a survey. [56] S. L. Bland. CA 94305. Proc. “Geometric Optimization Problems Likely Not Contained in APX. Philadelphia.. The Simplex Method: A Probabilistic Analysis. “Approximation of Diameters: Randomization Doesn’t Help. NM. 29th Symp.0 : A collection of semideﬁnite programming test problems. Ye. Management Science and Engineering. Balakrishnan. Brieden. Goldfarb and M. Semideﬁnite Programming for Ad Hoc Wireless Sensor Network Localization. P. “Deterministic and Randomized Polynomial-time Approximation of Radii. J. R. Lovasz and M. H. 1981. 1999. Blum. Faculty of Mathematics. Y. Foundations of Computer Science. 244-251. Management Science and Engineering. Feron. 1988. Presentation at the Seventh DIMACS Implementation Challenge Piscataway. A distributed method for solving semideinite programs arising from Ad Hoc Wireless Sensor Network Localization. L. Geom. Operations Research. [64] P. a C library for semideﬁnite programming. Stanford. Biswas and Y. V. October 2003. “The Computational Complexity of Norm Maximization”. On a theory of computations over the real numbers: NP-completeness. July 1998. Klee. CSDP. 2000. Biswas. [59] B. Borgwardt. Optimization Software and Methods. recursive functions and universal machines. V. 1990. New Mexico Tech. 11:613–623. Socorro. IEEE Symp. Ghaoui. Gritzmann. Brieden. Brieden. [57] B. NJ. Proc. [55] K. USA. Gritzmann. 2002. Institute of Mining and Technology. Klee. [60] A. Van Leeuwen. R. Klee and J. P. [58] B.BIBLIOGRAPHY 155 [51] P. September 2003. SDPLIB 1. Lovasz and M. Boyd.” To appear in Mathematika. Berlin. Todd. 29:1039–1091.” Discrete Comput. Biswas and Y. E. 1998. Smale. G. [61] A. Stanford. 10: 203-225. Springer. Shub and S. Sci. Ye. [62] A. L. CA 94305. Foundations of Comp.. . Kannan. Combinatorica. November 2. SIAM Publications. M. Technical report. [63] P. Ye. SIAM. Bodlaender. and V. Borchers. Simonovits.L. 387–397. Borchers. 2004. [52] R. 3rd IPSN 46–54. E. P. Kannan. 28:201-209.” In Proc. 1994. Simonovits. Linear Matrix Inequalities in System and Control Science. Semideﬁnite Programming for Ad Hoc Wireless Sensor Network Localization. [53] L. 1987. D. Gritzmann.

[69] G. and K. Numerical Methods for Unconstrained Optimization and Nonlinear Equations. Proc. Complementary pivot theory in mathematical programming. Providence. September 1999. School of Industrial and Systems Engineering. part I: Transformations and derivatives. and Y.156 BIBLIOGRAPHY [65] N. 1983. Convexity. 1971. Indri. Princeton University Press. May 2000. GPS-less low cost outdoor localization for very small devices. 1968. Prentice–Hall. [76] J. Luo. E. GA USA. [73] G. pages 2003–2007. Cottle. Helly’s theorem and its relatives. April. 49(3):569–575. Solving semideﬁnite programs via nonlinear programming. Palhares. D. Design of orthogonal pulse shapes for communications via semideﬁnite programming. An eﬃcient algorithm for solving the MAXCUT SDP relaxation. and P. Englewood Cliﬀs. TX. Dantzig. C. Davidson. The Linear Complementarity Problem. Danzer. 151–158. October 1999. and R. . Monteiro. J. C. Pang. 2000. Estrin. Burer. Technical Report TR99-23. 3rd ACM Symposium on the Theory of Computing. Gr¨nbaum and V. [67] S. Dennis and R. TR 00-729. J. Academic Press. March 2001. E. [66] S. Stone. D. Z. E. Solving semideﬁnite programs via nonlinear programming. R. pages 461–475. Bulusu. Burer and R. [75] T. B. chapter 5. Robust calibration and control of robotic manipulators. New Jersey. A. Klee. 48(5):1433–1445. University of Southern California. Georgia Institute of Technology. IEEE Transactions on Signal Processing. W. Robust H∞ ﬁlter design for uncertain linear systems with multiple time-varying state delays. Princeton. The complexity of theorem-proving procedures. page 101–180. Manuscript. Department of Computational and Applied Mathematics. Peres. 1992. Burer. and Y. [70] S. Linear Algebra and its Applications. Rice University. Technical Report TR99-17. D. Atlanta. AMS. Linear Programming and Extensions. In American Control Conference. Schnabel. 1:103-125. Zhang. 2000. Cottle and G. [77] C. B. R. Boston. Dantzig. [74] L. Wong. Monteiro. [72] R. Zhang. de Soua. Rice University. editor. RI.9 : Interior–point methods. [68] S. December 1998. New Jersey. Cook. Heidemann. C. S. Calaﬁore and M. B. D. TX. IEEE Transactions on Communications. Klee. part II: Interior point methods for a subclass of SDPs. u in V. Department of Computational and Applied Mathematics. R. 1963. Monteiro. [71] R. 1963. Computer Science.

Pister. Discr.. June 2001. [90] G.. V. June 2000.” Mathematical Programming. R. Physical Review Letters. A central cutting plane algorithm for the convex programming problem. IEEE Transactions on Circuits and Systems I: Fundamental Theory and Applications. IEEE Transactions on Signal Processing. Soh. E. 17:967–974. Upravlyaemye Sistemi. Ghaoui. Proc. E. and F. N. Kern. B. S. pages 378–380. Distinguishing separable and entangled states. The complexity of computing the volume of a polyhedron. Elzinga and T. Streng. Springer-Verlag. Dikin. SIAM J. Eren. J. 1991. Xie. A random polynomial time algorithm for estimating volumes of convex bodies. L. H∞ ﬁltering of 2-D discrete systems. [89] T. Comput. 71B. The 2-catalog segmentation problem. and Y. Du. 1996. Moore. (In Russian). Doids. [86] M. 20th INFOCOM Volume 3:1655–1663. Geom. [82] C. Convex Position Estimation in Wireless Sensor Networks. Proc. Dyer and A. P. Du. [79] L. [85] B. [88] J. Elekes. . I. Doherty. W. Moore. Frieze. O. H∞ reduced-order approximation of 2-D digital ﬁlters. and Y. Dullerud and F. D. 49(11):2630–2639. A geometric inequality and the complexity of computing volume. S. Faigle. 73:15. 2000. Anderson. [92] U. Kannan. 1999. A. volume 36 of Texts in Applied Mathematics. 1975. L. Res. Dumitrescu. Bur. Yang. IEEE Transactions on Signal Processing. Mach. 1967. Guruswami. On the convergence of an iterative process. 1974. [83] H. Dyer. [80] A. Computation. 8:134–145. G. Belhumeur. A. Edmonds. Y. Rigidity. D. I.. “Note on the Computaional Complexity of j-Radii of Polytopes in Rn . Tabus.BIBLIOGRAPHY 157 [78] I. Assoc. M. In Proceedings of SODA. [84] G. pages 241–245. W. [81] Y. 1:289–292. and S. 88(18). M. Systems of distinct representatives and linear algebra. Comp. Nat. and P Stoica. 2004. Math. K. 12:54–60. 2002. Frieze and R. Parrilo. Comput. 2001. [87] M. Whiteley. Paganini. and M. On the parameterization of positive real sequences and MA parameter estimation. Spedalieri. 23rd INFOCOM. and Randomization in Network Localization. Programming. L. November 2001. J. Khanna. Doherty. J. 1986. Goldenberg. Xie. P. A Course in Robust Control Theory. 38:1–17. E. Soh. E. Standards. 48(6):688–698. [91] J. 1988. 48(6):1760–1768.

[100] U.” in Proceedings of the Third Israel Symposium on Theory of Computing and Systems. “A computational study of graph partitioning. 1995. Fiacco and G. Notre Dame. Feige and M. 2001. 1994. Approximation algorithms for maximization problems arising in graph partitioning. F. P. SIAM Publications. . Math. [96] L. [105] R. Peleg.158 BIBLIOGRAPHY [93] J. PA. [103] U. [97] U. Langberg. 1990. “On the densest k-subgraph problem. Kortsarz and D. Wolkowicz. Feige and M. Philadelphia. with applications to max 2sat and max dicut. Fang and S. Langberg. 41:174– 211. Prentice–Hall. [99] U. John Wiley & Sons. Faybusovich. Feige and M. In ICALP.” Mathematical Programming. Approximating the value of two prover proof systems. Feige.” Technical report. C. with applications to MAX 2SAT and MAX DICUT. and H. SIAM J. [95] L. Goemans. 1995. Optimization. Faybusovich. Goemans. Freund and F. Algorithms. NJ. [98] U. [94] S. X. “Approximation algorithms for maximization problems arising in graph partitioning.” to appear in J. [101] U. In Proceedings of the 3nd Israel Symposium on Theory and Computing Systems. [104] A. Feige and M. Semideﬁnite programming: a path-following algorithm for a linear-quadratic functional. Reprint : Volume 4 of SIAM Classics in Applied Mathematics. “The dense k-subgraph problem. 66(2):211-240. The Weizmann Institute. 1994. Seltser. 1994. [102] U. 1996. 2001. Nonlinear Programming : Sequential Unconstrained Minimization Techniques. Falkner. Rehovot. X. Springer. New York. “Approximating the value of two prover proof systems. Englewood Cliﬀs. University of Notre Dame. (1995) 182-189. Jordan algebras. Department of Applied Mathematics and Computer Science. Feige and M. An interior–point method for fractional programs with convex constraints. pages 182–189.” to appear in Algorithmica. Programming. V. Research Report. The rpr2 rounding technique for semideﬁnte programs. G. 6:1007–1024. Symmetric cones and Interior-point methods. Berlin. Rendl. Feige and M. Puthenpura. Lecture Notes in Computer Science. September 1997. Journal of Algorithms. Jarre. Langberg. McCormick. 1968. Linear Optimization and Extensions. 67:407–440.

Math. Luo. Tokyo 152. [109] K. and Z. [113] T. Freund. The logarithmic potential method for convex programming. Frieze and M. 18:67–81. M. 1955. IEEE Transactions on Signal Processing. [116] D. CANADA L8S 4K1. [110] A. A new H∞ ﬁlter design for linear time delay systems. Fridman and U. B. University of Oslo. pp.BIBLIOGRAPHY 159 [106] R. M. C. July 2001. Tokyo Institute of Technology. 1997. McMaster University. Algorithmica. Ganesan. “Semideﬁnite programming relaxation for nonconvex quadratic programs. Programming. Jerrum. 1960. Oslo. UCLA. Programming.-Q. The Theory of Linear Economic Models. Estrin. Nakata. Improved approximation algorithms for max k-cut and max bisection. (1997). Luo and Y. 1987. Fujie and M. 38:47–67. Fu. New York. An empirical study of epidemic algorithms in large scale multihop wireless networks. M. 2002. [117] D. Institute of Economics. Fujisawa. manuscript. R.” Journal of Global Optimization 10:367-380. [108] E. Culler. Kojima and K. de Souza. Fu. Krishnamachari. “Exploiting Sparsity in PrimalDual Interior Point Methods for Semideﬁnite Programming” Research Report B-324. with applications to quadratic programming and the minimum-norm problem. Unpublished manuscript. Gale. Kojima. Finite-horizon robust Kalman ﬁlter design. Norway. Hamilton. [112] M. 49(11):2839–2843. Math. 1991. Fukuda. 267–301. Meguro-ku. Z. Kluwer Academic Press. and K. Frisch. Fujisawa. [111] M. 1996. Sept. Nakata. . A. M. Freund. Woo. September 1997. Numerical Evaluation of SDPA (SemiDeﬁnite Programming Algorithm). D. Ye. McGraw-Hill. IEEE Transactions on Signal Processing. In High Performance Optimization. 51:203–222. 2001. Kojima. UCLA/CSD-TR-02-0013. [107] R. Wicker. M. Approximation algorithms for quadratic programming. D. [115] K. Department of Mathematical and Computing Sciences. Polynomial–time algorithms for linear programming based only on primal scaling and projected gradients of a potential function. 49(9):2103–2112. Computer Science. 1999. Shaked. and S. Ontario. Dual gauge programs. [114] K. Department of Electrical and Computer Engineering. Oh-Okayama.

SIAM J. Geromel. Murray. Murray. 1996. 46(1):100–107. Gill. 2001. Jan. P. M. 42:1115–1145. Journal of Assoc. 2001. Jan. On projected Newton barrier methods for linear programming and an equivalence to Karmarkar’s projective method. Math. J. J. Freeman. 1990. Goemans and D. and Y. Robust portfolio selection problems. Improved approximation algorithms for maximum cut and satisﬁability problems using semideﬁnite programming.. Ye. Interior Point Trajectories in Semideﬁnite Programming. X. H. Practical Optimization. 443-452. 26:103-124. Cutting planes and column generation techniques with the projective algorithm. Q. Optim. [120] J. W. [129] D. Errata in Math. Z. Goldfarb and G. 1986.. Vial. P. 1998. London. 1999. L. IEEE Transactions on Signal Processing. [128] J. 6(3):638–652. Tomlin. [130] D. Opt. M. W. Mach. X. 1996. Gay. K. Programming. Mathematics of Operations Research. Williamson. 1987. 65:409– 429. [123] P. Optimization. H. Complexity analysis of an interiorpoint cutting plane method for convex feasibility problem. Math. [124] P. [119] D. . Scheinberg. 8(4):871–886. IEEE Transactions on Automatic Control. Geromel and M. M. 1995. De Oliveira. Theory Appl. SIAM J. A continuous based heuristic for the maximum clique problem. S. [126] M. DIMACS Series in Discrete Mathematics and Theoretical Computer Science. Submitted. and M. E. “ Approximation algorithms for MAX-3-CUT and other problems via complex semideﬁnite programming. Garey and D. 1988. Hearn and P. Goldfarb. 40:111.” in 33rd STOC. W. Goemans and D. 36:183–209. 1979. [127] J. A variant of Karmarkar’s linear programming algorithm for problems in standard form. Luo. Saunders. Gibbons. Academic Press. pp. Wright. Programming.160 BIBLIOGRAPHY [118] M. San Francisco. [121] J. L. M. Comput. Wright. and M. R. Gill. E. H2 /H∞ robust ﬁltering for convex bounded uncertain systems. [125] M. P. Programming. [122] L. Pardalos. A. Goﬃn. Optimal linear ﬁltering under parameter uncertainty. W. E. Computers and Intractability: A Guide to the Theory of NP-Completeness. D. Johnson. Iyengar. 47(1):168–175. H. 37:81–90. 1981. A. Goﬃn and J. Williamson.

BIBLIOGRAPHY

161

[131] D. Goldfarb and M. J. Todd. Linear Programming. In G. L. Nemhauser,
A. H. G. Rinnooy Kan, and M. J. Todd, editors, Optimization, volume 1
of Handbooks in Operations Research and Management Science, pages
141–170. North Holland, Amsterdam, The Netherlands, 1989.
[132] A. J. Goldman and A. W. Tucker. Polyhedral convex cones. In H. W.
Kuhn and A. W. Tucker, Editors, Linear Inequalities and Related Systems, page 19–40, Princeton University Press, Princeton, NJ, 1956.
[133] G. H. Golub and C. F. Van Loan. Matrix Computations, 2nd Edition.
Johns Hopkins University Press, Baltimore, MD, 1989.
[134] C. C. Gonzaga. Polynomial aﬃne algorithms for linear programming.
Math. Programming, 49:7–21, 1990.
[135] C. C. Gonzaga. An algorithm for solving linear programming problems
in O(n3 L) operations. In N. Megiddo, editor, Progress in Mathematical
Programming : Interior Point and Related Methods, Springer Verlag,
New York, 1989, pages 1–28.
[136] C. C. Gonzaga. Path following methods for linear programming. SIAM
Review, 34(2):167–227, 1992.
[137] J. Graver, B. Servatius, H. Servatius. Combinatorial Rigidity. AMS,
1993.
[138] P. Gritzmann and V. Klee, “Inner and Outer j-Radii of Convex Bodies
in Finite-Dimensional Normed Spaces,” Discrete Comput. Geom., 7:255280, 1992.
[139] P. Gritzmann and V. Klee, “Computational Complexity of Inner and
Outer j-Radii of Polytopes in Finite-Dimenional Normed Spaces,” Math.
Program., 59:162-213, 1993.
[140] P. Gritzmann and V. Klee, “On the Complexity of Some basic Problems
in Computational Convexity: I. Containment Problems,” Discrete Math.,
136:129-174, 1994.
[141] P. Gritzmann and V. Klee. Mathematical programming and convex geometry. In P. Gruber and J. Wills, editors, Hand Book of Convex Geometry, North Holland, Amsterdam, 1993, page 627–674.
[142] M. Gr¨tschel, L. Lov´sz and A. Schrijver. Geometric Algorithms and
o
a
Combinatorial Optimization. Springer, Berlin, 1988.
[143] B. Gr¨nbaum. Convex Polytopes. John Wiley & Sons, New York, NY,
u
1967.
[144] O. G¨ler. Barrier functions in interior-point methods. Mathematics of
u
Operations Research, 21(4):860–885, 1996.

162

BIBLIOGRAPHY

[145] O. G¨ler. Existence of interior points and interior paths in nonlinear
u
monotone complementarity problems. Mathematics of Operations Research, 18(1):128–147, 1993.
[146] O. G¨ler and L. Tuncel. Characterizations of the barrier parameter of
u
homogeneous convex cones. Research Report CORR 95-14, Department
of Combinatorics and Optimization, University of Waterloo, Waterloo,
Ontario, Canada, 1995. To appear in Math. Programming.
[147] O. G¨ler and Y. Ye. Convergence behavior of interior point algorithms.
u
Math. Programming, 60:215–228, 1993.
[148] D. Guo, L. Rasmussen, S. Sun, and T. Lim. A matrix-algebraic approach
to linear parallel interference cancellation in CDMA. IEEE Transactions
on Communications, 48(1):152–161, Jan. 2000.
[149] E. Halperin and U. Zwick, “Approximation algorithms for MAX 4-SAT
and rounding procedures for semideﬁnite programs,” Proceedings of 7th
IPCO (1999), 202-217.
[150] E. Halperin and U. Zwick. A uniﬁed framework for obtaining improved
approximation algorithms for maximum graph bisection problems. In
IPCO, Lecture Notes in Computer Science. Springer, Berlin, 2001.
[151] Q. Han, Y. Ye, and J. Zhang. An improved rounding method and semidefinite programming relaxation for graph partition. Math. Programming,
92:509–535, 2002.
[152] S. Har-Peled and K.R. Varadarajan, “Approximate Shape Fitting via
Linearization,” In Proc. 42nd Annu. IEEE Sympos. Found. Comput. Sci.,
66-73, 2001.
[153] S. Har-Peled and K. Varadarajan, “Projective Clustering in High Dimensions Using Core-sets,” In Proc. ACM Symp. Comput. Geom., 2002.
[154] S. Har-Peled and K. Varadarajan, “High-Dimensional Shap Fitting in
Linear Time,” In Proc. ACM Symp. Comput. Geom., 2003.
[155] J. Hartmanis and R. E. Stearns, On the computational complexity of
algorithms. Trans. A.M.S, 117:285–306, 1965.
[156] C. Helmberg and F. Rendl. A spectral bundle method for semideﬁnite
programming. SIAM Journal of Optimization, 10:673–696, 2000.
[157] C. Helmberg, F. Rendl, R. J. Vanderbei and H. Wolkowicz. An interior point method for semideﬁnite programming. SIAM J. Optimization,
6:342–361, 1996.
[158] B. Hendrickson. Conditions for Unique Graph Realizations. SIAM J.
Comput. 21(1):65–84, 1992.

BIBLIOGRAPHY

163

[159] B. Hendrickson. The Molecule Problem: Exploiting Structure in Global
Optimization. SIAM J. Opt. 5(4):835–857, 1995.
[160] D. den Hertog. Interior Point Approach to Linear, Quadratic and Convex Programming, Algorithms and Complexity. Ph.D. Thesis, Faculty of
Mathematics and Informatics, TU Delft, NL–2628 BL Delft, The Netherlands, 1992.
[161] J. Hightower and G. Boriello. Location systems for ubiquitous computing.
IEEE Computer, 34(8) (2001) 57-66.
[162] R. Horst, P. M. Pardalos, and N. V. Thoai. Introduction to Global Optimization. Kluwer Academic Publishers, Boston, 1995.
[163] A. Howard, M. J. Mataric, and G. S. Sukhatme. Relaxation on a mesh:
a formalism for generalized localization. In Proc. IEEE/RSJ Int’l Conf.
on Intelligent Robots and Systems (IROS01) (2001) 1055-1060.
[164] L. Huaizhong and M. Fu. A linear matrix inequality approach to robust
H∞ ﬁltering. IEEE Transactions on Signal Processing, 45(9):2338–2350,
Sept. 1997.
[165] P. Huard. Resolution of mathematical programming with nonlinear constraints by the method of centers. In J. Abadie, editor, Nonlinear Programming, pages 207–219. North Holland, Amsterdam, The Netherlands,
1967.
[166] P. Huard and B. T. Liˆu. La m´thode des centres dans un espace
e
e
topologique. Numeriche Methematik, 8:56-67, 1966.
[167] B. Jackson, T. Jord´n. Connected Rigidity Matroids and Unique Reala
izations of Graphs. Preprint, 2003.
[168] R. Jagannathan and S. Schaible. Duality in generalized fractional programming via Farkas’ lemma. Journal of Optimization Theory and Applications, 41:417–424, 1983.
[169] B. Jansen. Interior Point Techniques in Optimization. Ph.D. Thesis,
Faculty of Mathematics and Informatics, TU Delft, NL–2628 BL Delft,
The Netherlands, 1995.
[170] F. Jarre. An interior–point method for minimizing the largest eigenvalue
of a linear combination of matrices. SIAM J. Control and Optimization,
31:1360–1377, 1993.
[171] F. Jarre. Interior–point methods for convex programming. Applied Mathematics & Optimization, 26:287–311, 1992.
[172] F. Jarre, M. Kocvara, and J. Zowe. Optimal truss design by interior
point methods. SIAM J. Optim., 8(4):1084–1107, 1998.

RI. E. Papadimitriou. [180] R. editor. Delft. Report 96–10. R. John. A new polynomial–time algorithm for linear programming. Initialization in semideﬁnite programming via a self–dual skew–symmetric embedding. 1984. pages 473–482. page 187-204. B. Faculty of Technical Mathematics and Computer Science. J. M. W. 1972. Programming. On the complexity of approximating the maximal inscribed ellipsoid for a polytope.” in 35th FOCS (1994). Johnson and S. F. 1948. [182] L. Lindenstrauss. 1997. Miller and J. [175] W. Karger. Clausen. [178] S. Karisch. Minty. 61:137–160. 26:189-206. [179] N. “Solving graph bisection problems with semideﬁnite programming. Raghavan.” Technical Report DIKU-TR-97/9. Soc. Trafalis. Khachiyan. A polynomial algorithm for linear programming. Roos and T. Distributions in Statistics: Continuous Multivariate Distributions. Karp. G. eds. 1998. In O. Translated in Soviet Math. 1993. Motwani and M. pages 85–103. In P. 2-13.B. 1979. Doklady. 20:191–194. J Kaiser. 1979. [176] M. University of Copenhagen. Reducibility among combinatorial problems. Comput. To appear in Operations Research Letters. Doklady Akad. 4:373–395. Karmarkar. 244:1093–1096. Terlaky. L. [183] J. “Extensions of Lipshitz Mapping into Hilbert Space. Sudan. 1972. C. [184] V. Klee and G. Morin and T. Combinatorica. 1991. “Approximate graph coloring by semideﬁnite programming. John Wiley & Sons. How good is the simplex method. de Klerk.. . Math. Math. 1972. Inequalities III. K. Shisha. In Proceedings of the 30th Symposium on Theory of Computation. E. January 8. Kotz. Courant on his 60th Birthday.” Comtemporary Mathematics. Mach. presented to R. Nauk USSR. Interscience. New York. [181] L. Todd. Kleinberg. editors. C. Centers and invariant points of convex bodies. T. New York. Johnson and J.. and Ass. 1996. New York.164 BIBLIOGRAPHY [173] F. Extremum problems with inequalities as subsidiary conditions. In Sduties and Essays. Plenum Press. Khachiyan and M. NY. and J. July 1. [174] N. and P. Gritzmann and B. Department of Computer Science. 1948. Applied Geometry and Discrete Mathematics: The Victor Klee Festschrift. Sturmfels. Thatcher. R. Complexity of Computer Computations. [185] E. G. Delft University of Technology. Providence. Rendl. J. Amer. Academic Press. 1984. Segmentation problems. [177] D.

pages 481–492. Kojima. Lagarias. A collinear scaling interpretation of Karmarkar’s linear programming algorithm. 1989. Megiddo. 44:1–26. 1992. Mizuno. Ye. and A. 7:86–125.BIBLIOGRAPHY 165 [186] M. 1991. Proceedings of the Second Berkeley Symposium on Mathematical Statistics and Probability. F. Programming. SIAG/OPT Views– and–News. Mizuno. Math. J. Math. Kojima. Noma and A. A polynomial–time algorithm for a class of linear complementarity problems. An interior point potential reduction algorithm for the linear complementarity problem. 1993. Kocvara. Nemirovski. SIAM J. University of California Press. 3:630–636. Yoshise. [187] M. S. Kojima. [193] G.” in 34th FOCS. Kojima. Tucker. and S. 1997. 1:11. 2000. [198] J. Hara Interior-point methods for the monotone semideﬁnite linear complementarity problem in symmetric matrices. [191] M. W. New York. The nonlinear geometry of linear programming. N. Lagarias. [190] M. and A. 320:193–225. Optimization. Part III : Projective Legendre transform coordinates and Hilbert geometry. [197] J. S. and Y. Potra. Kojima. Megiddo. Berkeley and Los Angeles. (1993) 692-701. [194] E. Neyman. and Y. Linear Algebra and Its Applications. A uniﬁed approach to interior point algorithms for linear complementarity problems. O. 152:169–189. SIAM J. Nonlinear programming. W. 1991. 1961. C. N. [195] H. C. 54:267–279. Programming. Global optimization with polynomials and the problem of moments. “On choosing a dense subgraph. Lasserre. 2001. Lecture Notes in Computer Science 538. √ [189] M. A Forum for the SIAM Activity Group on Optimization. Ye. Kortanek. [188] M. Shindoh. Computers and Structures. [196] J. Springer Verlag. Optimization. Peleg. Kuhn and A. [192] K. CA. Yoshise. On some eﬃcient interior point methods for nonlinear convex programming. Programming. In J. 1991. 76:431–442. Math. SIAM J. 1990. . and A. editor. An O( nL) iteration potential reduction algorithm for linear complementarity problems. Transactions of the American Mathematical Society. Zowe. S. T. Interior–point methods bibliography. 1992. 50:331–342. Cascading−an approach to robust material optimization. Yoshise. Kranich. Kortsarz and D. Optimization. 11:796–817.

Luo. Programming. 1984. Erasmus University Rotterdam. 1994.” IEEE Transactions a of Information Theory IT-25(1):1-7. . [206] L. 1990. [205] L. Stanford University. Lin and R. R. [202] C. Math. [211] Z. Stanford. Linial. 1998. IEEE Transactions on Circuits and Systems I: Fundamental Theory and Applications. Semideﬁnite programming vs. Superlinear convergence of a symmetric primal-dual path following algorithm for semideﬁnite programming. Sturm and S.. [210] Z. [203] N. [204] M. Second-order cone programming. The Geometry of Graphs and Some of Its Algorithmic Applications. Report 9620/A. Lobo. Shanno. 1997. Q. 1996. Zhang. G. Yu. Matrix Completion Problems. Mathematics of Operations Research. Lovasz. Rabinovich. F. 66:123–135. of Ind. Ann Arbor. Marsten. Luo. Lasserre. Levin. J. 1965. [208] W. J. Computational experience with a globally convergent primal–dual predictor–corrector algorithm for linear programming. Eng. The Encyclopedia of Optimization 3:221–229. Dept. CA. and S. Manuscript. Linear and Nonlinear Programming. Lu. May 2002. Menlo Park. and Oper. An Algorithmic Theory of Numbers. J.166 BIBLIOGRAPHY [199] J. 1986. Lustig. A uniﬁed approach for the design of 2-D digital ﬁlters via semidefinite programming. Optimization 8(1):59-81. MI. SIAM J. 6:286-290. Q. 2nd Edition. June 2002. SIAM. Econometric Institute. On an algorithm for the minimization of convex functions. [207] L.a case study of quadratic assignment problem. Optimization. of Electrical Engineering. Philadelphia. [200] M. Soviet Math. and 0 − 1 a optimization. University of Michigan. 2001. Graphs and Convexity. 1:166-190. Combinatorica 15(2):215–245. Lov´sz. Information Systems Laboratory. “On the Shannon Capacity of a graph. Vandenberghe. 49(6):814–826. Cones of matrices and setfunctions. volume 50 of CBMS-NSF Regional Conference Series in Applied Mathematics. Laurent. E. Addison–Wesley. London. lp relaxation for polynomial programming. [201] A. SIAM J. 27(2):347–360. 1995. 1997. E. Boyd. Saigal. Doklady. Dept. CA. Zhang. “On solving large scale semideﬁnite programming problems . Jan. Shrijver. Lov´sz and A. Sturm and S. [212] I. [209] D. Duality and self-duality for conic convex programming. and D. Luenberger. S. L. 1979.” Technical Report.

44:27– 41. Sept. Interior point methods : Computational state of the art. 1992. 1986.asu. Ye. 1995. Quadratic convergence in a primal–dual method. [220] G. Interior path following primal–dual algorithms : Part I : Linear programming. ORSA Journal on Computing. On the implementation of a primal–dual interior point method. Interior path following primal–dual algorithms : Part II : Convex quadratic programming. and Y. pp. Adler. Mahajan and H. [224] R. 2(4):575–601. R. [217] N. Megiddo. [215] M. [219] S. pages 131–158. Daafouz. The analogue of Moreau’s proximation theorem. New York. Ramesh. Independent http://plato. Boujarwah. Megiddo. Identical version in : Proceedings of the 6th Mathematical Programming Symposium of Japan. Mizuno. M. Pathways to the optimal set in linear programming. Robust H∞ ﬁltering for a class of linear parameter-varying systems. Mehrotra. Adler. [225] R. IEEE Transactions on Circuits and Systems I: Fundamental Theory and Applications. 1995. pages 1–35. 44:43–66. On adaptive step primal–dual interior–point algorithms for linear programming. with applications to the nonlinear complementarity problem. M. 1993. . 1980. Mathematics of Operations Research. Springer Verlag. 48(10):1170–1176. 6:1– 14. Global chaos synchronization and robust ﬁltering in noisy context. November 2000. “Derandomizing semideﬁnite programming based approximation algorithms. Monteiro and I. 1993. Todd.” in 36th FOCS. 18:741-751. Mehrotra. E. D. [214] S. C. [218] S. J. Mittelmann. editor. 1989.edu/. 2001. Oct. Japan. Mathematics of Operations Research. Shanno. Progress in Mathematical Programming : Interior Point and Related Methods.la. Ye. Mathematics of Operations Research. Paciﬁc Journal of Mathematics. Programming. and Y. 2001. Optimization. and D. A surface of analytic centers and primal–dual infeasible–interior–point algorithms for linear programming. [223] S. Millerioux and J. C. IEEE Transactions on Circuits and Systems I: Fundamental Theory and Applications. SIAM J. 20:135–162. Mahmoud and A. Monteiro and I. 1994. Lustig. Benchmark Results [222] S. 1989. 1989. [216] L. Todd. Math. J. Nagoya. In N. D. Math. [221] Hans D. 88:101–161. 162-169. J. Marsten. McLinden. 48(9):1131–1138. F. 18:964–981.BIBLIOGRAPHY 167 [213] I. Mizuno. Programming.

A. 1995. [235] M. Opt. Muramatsu and R. Japan. Berlin. C. 1997. [230] J. [234] M. Z. 86:463-473. R. 62:325–356. Research report No. Linear and Nonlinear Programming. D. Terlaky. [228] R. G. SOR. Linear Complementarity. Atlanta. 1996. [231] K. The Levenberg-Marquardt algorithm: implementation and e theory. Sofer.. Global Continuation for Distance Geometry Problems. Wu. C. Nash and A. Roos and T. [238] A. Aﬃne scaling algorithm fails for semideﬁnite programming. School of ISyE. Programming. Tsuchiya. “Implementation of primal-dual a methods for semideﬁnite programming based on Monteiro and Tsuchiya directions and their variants.. Zhang. editor: Numerical Analysis Springer-Verlag. . 39:117-129. J. C. [237] D. Monteiro and P. Program. 7:814–836. “On Maximization of Quadratic Forms Over Intersection of Ellipsoids with Common Center. An aﬃne scaling method with an infeasible starting point: convergence analysis under nondegeneracy assumption. Working Paper. Some NP-complete problems in quadratic and nonlinear programming.168 BIBLIOGRAPHY [226] R. Assoc. Nemirovski. 1987. New York. 1993. [233] M. Kabadi. Location of the maximum on unimodal surfaces. 1996. Mor´.” Technical Report. 1965. Mor´. Comput. In G. Math. 12:395-398. 1997. J. N. “Primal-dual aﬃne scaling algorithms fail for semideﬁnite programming. John Hopkins University. Newman. Verlag. Vanderbei. Sophia University. 1988. [229] J. ORSA Journal on Computing. Pang.” Math. D. Zanj´como. D. 5:40–53. Murty and S. Princeton University. A uniﬁed analysis for a class of pathfollowing primal-dual interior-point algorithms for semideﬁnite programming. Georgia Tech. J. G. GA 30332. A barrier method for large–scale constrained optimization. Heldermann. Math. On two interior-point mappings for nonlinear semideﬁnite complementarity problems. Muramatsu and T. C. Princeton. [227] R. School of Ind. NJ. Annals of Operations Research. 1977. e SIAM J.16. 1999. Monteiro and Y. [236] S.” Technical Report. July 1996. Muramatsu. and Systems Engineering. [232] K. G. Department of Mechanical Engineering. Watson. 1997. Murty. Monteiro and J-S. Georgia Institute of Technology.

Park. [242] Yu. Optimization Methods and Software 9 (1998) 141-160. J. Robust H∞ ﬁltering for uncertain discrete-time state-delayed systems. Mathematics of Operations Research. Peres. [241] Yu. Palhares. School of Operations Research and Industrial Engineering. Aug. S. Papadimitriou and K. 1995. Nesterov. 2001. E. 1983. NY. and P. H. 1987. to appear in SIAM J. Glavnaya redaktsiya ﬁziko-matematicheskoi literatury. In IEEE GlobeCom. Design of GBSB neural associative memories using semideﬁnite programming. Todd.BIBLIOGRAPHY 169 [239] A. Jan. NY. IEEE Transactions on Aerospace and Electronic Systems. Lecture Notes in Computer Sciences 268. [246] R. and D. E. Steiglitz. M. 1993. Technical Report No. Niculescu and B. J. NJ. Dias Peres. S. IEEE Transactions on Neural Networks. Englewood Cliﬀs. Rosen. Pardalos and J. Nemirovskii. Problem Complexity and Method Eﬃciency in Optimization. Ithaca. 2001. Interior Point Polynomial Methods in Convex Programming: Theory and Algorithms. July 1999 . M. Nov. Nesterov and A. [243] Yu. Nesterov and M. Philadelphia. [245] D. Self-scaled barriers and interior-point methods for convex programming. [250] J. R. Constrained Global Optimization: Algorithms and Applications. Springer-Verlag. Nesterov. 37(1):292–296. Infeasible-start primal-dual methods and infeasibility detectors for nonlinear programming problems. 22(1):1–42. Optimization. Semideﬁnite relaxation and nonconvex quadratic optimization. LMI approach to the mixed H2 /H∞ ﬁltering design for discrete-time uncertain systems. 1982. Park. Cornell University. Translated by E. B. B. SIAM Publications. E. 10(4):946–950. Ye. [244] Yu. 1125. 2001. Technical Report No. [240] Y. Todd and Y. Nath. 1156. E. Ithaca. Chichester. 1997. Primal-dual interior-point methods for self-scaled cones. Nemirovsky and D. de Souza. [247] R. Combinatorial Optimization: Algorithms and Complexity. IEEE Transactions on Signal Processing. School of Operations Research and Industrial Engineering. John Wiley and Sons. Ad-hoc positioning system. 1996. C. 1979. H. SIAM. Izdatelstva “Nauka”. Palhares and P. Dawson from Slozhnost’ Zadach i Eﬀektivnost’ Metodov Optimizatsii. J. Prentice–Hall. [248] C. Todd. Cho. Nesterov and M. Yudin. Cornell University. 48(8):1696–1703. [249] P.

Programming. [258] E. Polyak. [262] F. [256] F.” Computational Complexity. Polijak. New Jersey. 1992. Ramana. Programming. Convex Analysis. 82/1995. Ramana. Wolkowicz. Optimization. 1996. Journal of Global Optimization. Mathematical Programming. Semideﬁnite programming relaxations for semialgebraic problems. 54:177–222. Department of Combinatorics and Optimization. [263] J. Renegar. T. Ontario. November 2001. Lim. To appear. Strong duality for semidefinite programming. [259] M. 7:51-73. 77:273–300. 40:59–93. [254] S. IEEE Transactions on Communications. 1995. 2000. Pataki. 1995. 133-157. “The hardness of approximation: gap location. Princeton. based on Newton’s method. Jan. Rasmussen. [257] F. Potra and Y. Wolkowicz. 1998. Princeton University Press. Johansson. T. A semideﬁnite program for distillable entanglement. Potra and R. [252] G. Reports On Computational Mathematics. Modiﬁed barrier functions (theory and methods). IEEE Transactions on Information Theory. 1988. Rendl and H. Programming. 23(2):339-358. Petrank. 1997. University of Waterloo. 1970. Programming. An exact duality theory for semideﬁnite programming and its complexity implications. To appear in SIAM J. [253] E. Rockafellar. Sheng. No. [255] R. Waterloo. 77:129–162. 48(1):145–151. CORR 95-12. 47(7):2921–2933. Interior point methods for nonlinear complementarity problems. Tuncel and H. A recipe for semideﬁnite relaxation for 0-1 quadratic programming. Rendl and H. Canada.170 BIBLIOGRAPHY [251] P. L. A semideﬁnite framework for trust region subproblems with applications to large scale minimization. Math. Parrilo. Rains. [260] M. Mathematics of Operations Research. 1997. [261] L. 88:617–642. A polynomial–time algorithm. Homogeneous interior–point algorithms for semideﬁnite programming. Journal of Optimization Theory and Application. Ye. A matrix-algebraic approach to successive interference cancellation in CDMA. 4 (1994). F. Math. and A. Wolkowicz. for linear programming. Department of Mathematics. A. Math. Math. 1995. On the Rank of Extreme Matrices in Semideﬁnite Programs and the Multiplicity of Optimal Eigenvalues. The University of Iowa. [264] R. .

[274] W. M. Dept. [270] A. Vial. [276] Masayuki Shida. B. The Bits and Flops of the n– hop Multilateration Primitive for Node Localization Problems. Research Report B-310. Shaked. P. IEEE Transactions on Signal Processing. H. P. [268] C. J. [266] M. Susumu Shindoh and Masakazu Kojima. Optimization. [269] A. Sherali and W. M. Dynamic Fine–Grained Localization in Ad–Hoc Networks of Sensors. Ann Arbor. Proc. Y. L. 2002. M.” Transactions of the Cambridge Philosophical Society 19 (1900) 23-66. J. F. November 2001. Ruml. 1986.-Ph. [271] A. University of Michigan. 1996. Langendoen. 1997. 4th MOBIHOC. K. Savvides. 1996. 2002. Srivastava. Localization from Mere Connectivity. 7th MOBICOM. IEEE Transactions on Circuits and Systems II: Analog and Digital Signal Processing. Proc. Shang. 2003. Saadat and T. Xie. Schrijver. [273] Y. New approaches to robust minimum variance ﬁlter design. 49:1-6.BIBLIOGRAPHY 171 [265] C. Han. [272] U. Fromherz. Dept.–C. Tech. Design of nonuniform multirate ﬁlter banks by semideﬁnite programming. Adams. Existence of search directions in interior-point algorithms for the SDP and the monotone SDLCP. Oh-Okayama. Chen. Meguro. Dec 1995. Lin. Zhang. T. [267] R. Terlaky and J. Report. 166–179. An infeasible start predictor corrector method for semi-deﬁnite linear programming. of Industrial and Operations Engineering. 2001. [275] H. To appear in SIAM J. Robust Positioning Algorithms for Distributed Ad–Hoc Wireless Sensor Networks. Chichester. J. W. MI. 49(11):2620– 2629. Optima. 47(11):1311–1314. 112–121. Soh. B. Theory of Linear and Integer Programming. Savarese. C. John Wiley & Sons. Proc. November 2000. Sheppard. New York. John Wiley & Sons. Savvides. Theory and Algorithms for Linear Optimization: An Interior Point Approach. . USENIX Annual Technical Conference. 201–212. of Mathematical and Computing Sciences. Srivastava. Tokyo Institute of Technology. Roos. “Computational advances using the reformulation-linearization technique (rlt) to solve discrete and continuous nonconvex problems. Park. D. Saigal and C. and Y. 1st WSNA. Rabay. Tokyo 152 Japan. “On the calculation of the double integral expressing normal correlation.

September 1985. [282] A. System Modelling and Optimization : Proceedings of the 12th IFIP–Conference held in Budapest. Proceedings of the 13th IFIP Conference. “Semideﬁnite relaxations for parallel machine scheduling.kub. On cones of nonnegative quadratic functions. “Quadratic optimization problems. [288] K. pp. F. Minatoku. Skutella. In K. McKelvey. Anal. SIAM J. 19:409-426. volume 84 of Lecture Notes in Control and Information Sciences. Iri and K. Sonnevend. F.html. Originally published in Tekhnicheskaya Kibernetika. Centered Newton method for mathematical programming. Ma estimation in polynomial time. convex) programming. July 2000. In M. The Institute of Statistical Mathematics. SEEM2001-01.” in Approximation Algorithms for Combinatorial Optimization. Prekopa. 25:1-11. IEEE Transactions on Signal Processing. 1982. Tanabe. Japan. J. Shor. editor.” Journal of the ACM. [284] J. Germany. March 2001. “Convex quadratic and semideﬁnite programming relaxations in scheduling. you. Skutella. System Modeling and Optimization. 472-481. Szelezsan. K. Jansen and J. Tokyo 106. [278] M. Econometric Institute. pages 118–144. Srivastav and K. Tone. Springer Verlag. Erasmus University. Tanabe.” in Proceedings of the 39th Annual IEEE Symposium on Foundations of Computer Science (FOCS’98). [285] J. 1987. Yajima.nl/sturm/software/sedumi. 48(2). Z. Sturm and S. Symmetric primal-dual path following algorithms for semideﬁnite programming. Rolim (Eds. pages 866–876. editors. [280] G. [279] M. Sorenson. Zhang.” Soviet Journal of Computer and Systems Sciences. 1988. Berlin. C. Zhang. 1987. Japan. [287] K. Mari. pp. 1995 [286] J. [281] D. No. editors. In A. [283] P. An “analytic center” for polyhedrons and new classes of global algorithms for linear (smooth. October 2001. Complementarity–enforced centered Newton method for mathematical programming. 4–6–7 Minami Azabu. 128-139. 1998.F. Let SeDuMi seduce http://fewcal. 1986. Rotterdam. 181-191.) (1998). 48(7):1999–2012. Wolf. and J. . 1.172 BIBLIOGRAPHY [277] N. 2001. Newton’s method with a model trust region modiﬁcation. Numer. Strazicky. 1987. Chinese University of Hong Kong. Sturm. pages 197–206. Stoica. “Finding dense subgraphs with semideﬁnite programming. 206-242. Hungary. Springer-Verlag. Sturm and S. Report 9554/A.. New Methods for Linear Programming. and B. Tokyo. T. SeDumi.

[295] M. Erlikh. 1986. Boston. March 1987. Todd and Y. Ye. 19(8):1442–1449.BIBLIOGRAPHY 173 [289] R. editor. . J. Improved bounds and containing ellipsoids in Karmarkar’s linear programming algorithm. IEEE Journal on Selected Areas in Communications. Soh. Terlaky. [296] M. The application of semideﬁnite programming for detection in CDMA. 37:226–230. Y. Potential-reduction methods in mathematical programming. P. Todd and B. [294] T. P. Interior-Point Methods in Mathematical Programming Kluwer Academic Publisher. Xie. August 2001. 1(4):409–424. IEEE Transactions on Circuits and Systems II: Analog and Digital Signal Processing. An extension of Karmarkar’s algorithm for linear programming using dual variables. [301] K. G. SDPT3 – A MATLAB uu u software package for semideﬁnite programming. 15:508–529. Mathematics of Operations Research. 1990. C. Todd. Envelope-constrained H∞ ﬁlter design: an LMI optimization approach. Rasmussen. and R. Xie. SIAM News. and I. 2000. A. Todd. 76:3–45. [298] M. Toh and M. [292] Z. A centered projective algorithm for linear programming. 2000. Khachiyan. Tapia. 1996. 1988. Toh.3 Optimization Software and Methods. 20:10–11. To appear Computational Optimization and Applications. 2001. Tan and L. [302] K. 1999. ´ [290] S. [297] M. Solving some large scale semideﬁnite programs via the conjugate residual method. J. Mathematics of Operations Research. 48(10):2960–2963. 47(1):79–82. 13:650-659. Burrell. August 2000. J. C. Soh. Algorithmica. [291] H. M. K. IEEE Transactions on Signal Processing. Tarasov. C. Toh. 1996. Soviet Mathematics Doklady. A note on the calculation of steplengths in interior point methods for semideﬁnite programming. J. Tan. I. Programming. version 1. Working paper. Math. Todd. 11:545-581. Current research in numerical optimization. Oct. [293] Z. Envelope-constrained H∞ FIR ﬁlter design. The method of inscribed ellipsoids. Jan. L. “A study of search directions in primal-dual interior-point methods for semideﬁnite programming.” Optimization Methods and Software 11&12 (1999) 1-46. 1988. and L. H. T¨t¨nc¨. [299] M. J. Todd. Y. Kojima. Tan. [300] K. and L. J.

1995. M. Atkinson. San Diego. Vaidya. S. July 2002. 69:205– 236. 73:291–341. 1956. Narikiyo. [313] L.P. InformationBased Complexity. Washington. Approximating QP. S. IEEE Transactions on Signal Processing. Boyd. Tuan. P. Princeton. Determinant maximization with linear matrix inequality constraints. Pardalos. and S. Math. in H. 49(12):2975–2984. Dual systems of homogeneous linear relations. 2001. SIAM Review. Apkarian. IEEE Transactions on Signal Processing. New York. Report. Linear Inequalities and Related Systems. M. 1996. Vandenberghe. Springer-Verlag. Math. Princeton University Press. Academic Press. [308] H. [307] C. Wozniakowski. 1990. 1993. In P. W. Nguyen. 50(7):1759–1771. [315] L. CA. 1988. 2003. Semideﬁnite programming. The Multivariate Normal Distribution. Traub. W. page 3–. Robust mixed H2 /H∞ ﬁltering of 2-D systems. Vandenberghe and S. Tuan. Tseng. Robust and reduced-order ﬁltering: new LMI-based characterizations and methods. T. Complexity in Numerical Optimization page 462–489. Seattle. Vandenberghe and S. Nguyen. Wasilkowski and H. Department of Mathematics. [311] P. Nov. Programming. Boyd. L. . Kuhn and A. Report. Chen. F. [314] L. editor. Tucker. Vaidya and D. Technical Report. Tong. Tseng and B. 1996. and T. New Jersey. Programming. Editors. [310] A. [305] P. H∞ fuzzy estimation for a class of nonlinear discrete-time dynamic systems. Information Systems Laboratory. University of Washington.174 BIBLIOGRAPHY [303] Y. A technique for bounding the number of iterations in path following algorithms. Stanford University. 49(11):2605–2619. P. 38(1):49–95. 1996. Apkarian. CA. Wu. and T. [306] P. [304] J. 2001. Washington 98195. Search directions and convergence analysis of some infeasible path-following methods for the monotone semi-deﬁnite LCP. G. Seattle. Department of Mathematics. IEEE Transactions on Signal Processing. A new algorithm for minimizing a convex function over convex sets. W. [312] P. 1996. NJ. Boyd. Tucker. A primal–dual potential reduction method for problems involving matrix inequalities. W. Tseng. [309] H. World Scientiﬁc. Dec. University of Washington.

Found. SIAM J. NY. November 1985. Approximate the 2-Catalog Segmentation Problem Using Semideﬁnite Programming Relaxations. 74(2):159–196. Computing the minimum ﬁll–in is NP–complete. Balakrishnan. Wolkowicz and Q. L. 1981. A. Yannakakis. Wright.-S. [322] H. Optimization. 48(9):2696–2700. [320] S. Oxford Science. Eﬃcient algorithms for minimizing a sum of Euclidean norms with applications. SIAM News. New York. Program in Statistics & Operations Research. Ye. “On Approximating the Radii of Point Sets in High Dimensions. 2002. Primal-Dual Interior-Point Methods SIAM. 18:(6):705 . IEEE Transactions on Signal Processing.719. [327] F. An interior point potential reduction method for constrained equations. Math. Wright. Zhao. Department of Management Sciences. [323] M. and C. Princeton University. Venkatesh and J. Programming. C. .” manuscript. Working Paper. Wang.. D. 2003 [326] G. 18(3):4. J.. Vavasis. [319] F. Wang. Yang and Y. Xie. [318] S. IEEE Sympos. IEEE Transactions on Signal Processing. J. 1991. To appear in SIAM J.” In Proc. Robust Kalman ﬁlters for linear timevarying systems with stochastic parametric uncertainties. 49(8):1236–1241. A brief history of linear programming. 1996. Zhang. IEEE Transactions on Circuits and Systems I: Fundamental Theory and Applications. R. Varadarajan. Zhang. [328] M. Sci. Vanderbei. 1996. 2002. Nonlinear Optimization: Complexity Issues. 1995.BIBLIOGRAPHY 175 [316] R. Xu. Pang. NJ. 1995. Robust mixed H2 /H∞ ﬁltering with regional pole assignment for uncertain discrete-time systems. Sept. Algebraic Discrete Methods. [325] C. [321] T. 50(4):803–813. H. Philadelphia. “Semideﬁnite programming for the graph partitioning problem. S. 2000. [317] K. Ye and J. The University of Iowa. 1996. 43rd Annu. Aug. to appear Discrete Applied Math. Y. LOQO: An Interior Point Code for Quadratic Programming. Monteiro. April 2002. Comput. pages 77–79. Zhang. [324] S. Optimization Methods and Software. Wang and V. H2 optimal inverse of periodic FIR digital ﬁlters. Hung. Xue and Y. and J.

50:239–258. Recovering optimal dual solutions in Karmarkar’s polynomial algorithm for linear programming. Ye and J. Zhang. G¨ler. Math. The Chinese University of Hong Kong (May. WileyInterscience Series in Discrete Mathematics and Optimization.1. Zhang. “Semideﬁnite Programming for Assignment and Partitioning Problems” PhD thesis. [341] Q. L. Xie. An O(n3 L) potential reduction algorithm for linear programming. Wolkowicz. Programming. July 2002. “Semideﬁnite programming relaxations for the quadratic assignment problems. [340] Q. J. Ye. Combining binary search and Newton’s method to compute real roots for a class of real functions. Ye. An improved algorithm for approximating the radii of point sets. Programming. 10:271–280. APPROX 2003. 6:177–182. Ye. [330] Y. 1987. Zhao. Karisch. [334] Y. Combinatorial Optimization. Rendl. 1998. 1997. [342] H. . A direct approach to H2 optimal deconvolution of periodic digital channels. [331] Y. 2001). IEEE Transactions on Signal Processing. Mathematical Programming 84 (1999) 219-226. 1994. Kojima. New York. A quadratically convergent Ye. u O( nL)–iteration algorithm for linear programming. 59:151–162. Ye and M. Zhao.699 approximation algorithm for Max-Bisection. [338] Y. Mizuno. CORR 95-27. Ye. 1993.” J. 50(7):1685–1698. Optimization. Working Paper. Ye. F. Tapia. √ O. √ [337] Y. S. Operations Research Letters. [339] Y. New results on quadratic minimization. Mathematics of Operations Research. Math. Programming. to appear in SIAM J. 1987. A . August 2003. 1994. and C. [333] Y. Zhang. 39:305–317. Karmarkar’s algorithm and the ellipsoid method. John Wiley & Sons.176 BIBLIOGRAPHY [329] Y. A. [336] Y. M. An O( nL)–iteration homogeneous and self–dual linear programming algorithm. Zhou. Interior Point Algorithms : Theory and Analysis. 19:53–67. Journal of Complexity. Todd and S. 2. Math. 1996. Ye. Ye and S. 1991. Ye. University of Waterloo. Mathematical Programming 90:1 (2001) 101-111 [335] Y. [332] Y. Department of SSEM. R. Zhang. and Y. and H. Approximating quadratic programming with bound and quadratic constraints.

[344] U.BIBLIOGRAPHY 177 [343] J. and M. M. Mathematical Programming. Outward rotationvs: a tool for rounding solutions of semidefinite programming relaxations. In Proceedings of the 31th Symposium on Theory of Computation. pages 679–687. 9:445–466.” In 9th SODA (1998) 201-210. Zwick. with applications to max cut and other problems. “Approximation algorithms for constraint satisfaction problems involving at most three variables per constraint. Kocvara. Zwick. 1999. [345] U. Bendsoe. Free material optimization via mathematical programming. Zowe. 1997. .