1 views

Uploaded by dmp130

- MPZ5230-TMA4_2015_2016
- Linear and Convex Optimization
- Linear Programming
- L01_MINI.pdf
- RECONDUCTORIZAR_PLEM (1)
- 54IJMPERDDEC201854
- Limiting Factors
- Refinery Planning Optimization
- Assembly Line Balancing
- calculation of portal frame deflection
- Report Evaporator Sculptor Behr 120405
- Exercises Nosol
- Design Methodology of a Permanent Magnet
- Una revisión sobre la planificación de generación distribuida.pdf
- Optimization
- 10 Optimised Blasting
- Application of Linear Programming-1.docx
- 86.pdf
- NEUTROSOPHIC MAGDM BASED ON ENTROPIES OF DEGREES α, β AND RNORM
- Svm

You are on page 1of 32

logarithmic barrier function and central path

barrier method

feasibility and phase I methods

complexity analysis via self-concordance

generalized inequalities

121

Inequality constrained minimization

minimize f0(x)

subject to fi(x) 0, i = 1, . . . , m (1)

Ax = b

A Rpn with rank A = p

we assume p is finite and attained

we assume problem is strictly feasible: there exists x with

Examples

Pn

minimize i=1 xi log xi

subject to F x g

Ax = b

piecewise-linear minimization or -norm approximation via LP

inequalities (see later)

Logarithmic barrier

reformulation of (1) via indicator function:

Pm

minimize f0(x) + i=1 I(fi(x))

subject to Ax = b

approximation via logarithmic barrier

Pm

minimize f0(x) (1/t) i=1 log(fi (x))

subject to Ax = b

10

an equality constrained problem

5

for t > 0, (1/t) log(u) is a

smooth approximation of I 0

approximation improves as t

5

3 2 1 0 1

u

Interior-point methods 124

logarithmic barrier function

m

X

(x) = log(fi(x)), dom = {x | f1(x) < 0, . . . , fm(x) < 0}

i=1

twice continuously differentiable, with derivatives

m

X 1

(x) = fi(x)

i=1

f i (x)

m m

X 1 X 1

2(x) = 2

f i (x)f i (x) T

+ 2

fi(x)

f (x)

i=1 i i=1

fi(x)

Central path

subject to Ax = b

(for now, assume x(t) exists and is unique for each t > 0)

central path is {x(t) | t > 0}

minimize cT x

subject to aTi x bi, i = 1, . . . , 6

x(10)

x

T T

hyperplane c x = c x (t) is tangent to

level curve of through x(t)

Dual points on central path

x = x(t) if there exists a w such that

m

X 1

tf0(x) + fi(x) + AT w = 0, Ax = b

i=1

fi(x)

m

X

L(x, (t), (t)) = f0(x) + i(t)fi(x) + (t)T (Ax b)

i=1

this confirms the intuitive idea that f0(x(t)) p if t :

p g((t), (t))

= L(x(t), (t), (t))

= f0(x(t)) m/t

Interpretation via KKT conditions

2. dual constraints: 0

3. approximate complementary slackness: ifi(x) = 1/t, i = 1, . . . , m

4. gradient of Lagrangian with respect to x vanishes:

m

X

f0(x) + ifi(x) + AT = 0

i=1

Force field interpretation

Pm

minimize tf0(x) i=1 log(fi (x))

log(fi(x)) is potential of force field Fi(x) = (1/fi(x))fi(x)

m

X

F0(x(t)) + Fi(x(t)) = 0

i=1

example

minimize cT x

subject to aTi x bi, i = 1, . . . , m

constraint force field decays as inverse distance to constraint hyperplane:

ai 1

Fi(x) = T

, kFi(x)k2 =

bi a i x dist(x, Hi)

3c

t=1 t=3

Barrier method

repeat

1. Centering step. Compute x(t) by minimizing tf0 + , subject to Ax = b.

2. Update. x := x(t).

3. Stopping criterion. quit if m/t < .

4. Increase t. t := t.

f0(x(t)) p m/t)

centering usually done using Newtons method, starting at current x

choice of involves a trade-off: large means fewer outer iterations,

more inner (Newton) iterations; typical values: = 1020

several heuristics for choice of t(0)

Convergence analysis

log(m/(t(0)))

log

centering problem

tf0 + must have closed sublevel sets for t t(0)

classical analysis requires strong convexity, Lipschitz condition

analysis via self-concordance requires self-concordance of tf0 +

Examples

102 140

Newton iterations

120

0

10

duality gap

100

102 80

60

4

10

40

106 = 50 = 150 =2 20

0

0 20 40 60 80 0 40 80 120 160 200

Newton iterations

terminates when t = 108 (gap 106)

centering uses Newtons method with backtracking

total number of Newton iterations not very sensitive for 10

geometric program (m = 100 inequalities and n = 50 variables)

P

5 T

minimize log k=1 exp(a0k x + b0k )

P

5 T

subject to log k=1 exp(a ik x + bik ) 0, i = 1, . . . , m

102

100

duality gap

102

104

106 = 150 = 50 =2

0 20 40 60 80 100 120

Newton iterations

family of standard LPs (A Rm2m)

minimize cT x

subject to Ax = b, x0

35

Newton iterations

30

25

20

15 1

10 102 103

m

Feasibility and phase I methods

fi(x) 0, i = 1, . . . , m, Ax = b (2)

basic phase I method

minimize (over x, s) s

subject to fi(x) s, i = 1, . . . , m (3)

Ax = b

if optimal value p of (3) is positive, then problem (2) is infeasible

if p = 0 and attained, then problem (2) is feasible (but not strictly);

if p = 0 and not attained, then problem (2) is infeasible

sum of infeasibilities phase I method

minimize 1T s

subject to s 0, fi(x) si, i = 1, . . . , m

Ax = b

inequalities than basic phase I method

60 60

40 40

number

number

20 20

0 0

1 0.5 0 0.5 1 1.5 1 0.5 0 0.5 1 1.5

bi aTi xmax bi aTi xsum

right: sum of infeasibilities phase I solution; satisfies 79 inequalities

example: family of linear inequalities Ax b + b

data chosen to be strictly feasible for > 0, infeasible for 0

use basic phase I, terminate when s < 0 or dual objective is positive

Newton iterations

100

80 Infeasible Feasible

60

40

20

0

1 0.5 0 0.5 1

Newton iterations

Newton iterations

100 100

80 80

60 60

40 40

20 20

0 0

100 102 104 106 106 104 2

100

10

number of iterations roughly proportional to log(1/||)

Complexity analysis via self-concordance

tf0 + is self-concordant with closed sublevel sets

second condition

may require reformulating the problem, e.g.,

Pn Pn

minimize i=1 xi log xi minimize i=1 xi log xi

subject to F x g subject to F x g, x 0

self-concordance assumption does not apply

Newton iterations per centering step: from self-concordance theory

#Newton iterations +c

, c are constants (depend only on Newton algorithm parameters)

from duality (with = (t), = (t)):

Xm

= tf0(x) tf0(x+) + log(tifi(x+)) m log

i=1

m

X

tf0(x) tf0(x+) t ifi(x+) m m log

i=1

tf0(x) tg(, ) m m log

= m( 1 log )

total number of Newton iterations (excluding first centering step)

(0)

log(m/(t )) m( 1 log )

#Newton iterations N = +c

log

5 104

4 104

figure shows N for typical values of , c,

3 104

m

N

4 m = 100, = 105

2 10 t(0)

1 104

0

1 1.1 1.2

in practice, #iterations is in the tens; not very sensitive for 10

polynomial-time complexity of barrier method

for = 1 + 1/ m:

m/t(0)

N =O m log

number of Newton iterations for fixed gap reduction is O( m)

problem dimensions), to get bound on number of flops

fixed ( = 10, . . . , 20)

Generalized inequalities

minimize f0(x)

subject to fi(x) Ki 0, i = 1, . . . , m

Ax = b

cones Ki Rki

fi twice continuously differentiable

A Rpn with rank A = p

we assume p is finite and attained

we assume problem is strictly feasible; hence strong duality holds and

dual optimum is attained

Generalized logarithm for proper cone

(sy) = (y) + log s for y K 0, s > 0 ( is the degree of )

examples

Pn

nonnegative orthant K = Rn+: (y) = i=1 log yi , with degree = n

positive semidefinite cone K = Sn+:

(Y ) = log det Y ( = n)

2

(y) = log(yn+1 y12 yn2 ) ( = 2)

properties (without proof): for y K 0,

(y) K 0, y T (y) =

Pn

nonnegative orthant Rn+: (y) = i=1 log yi

(Y ) = Y 1, tr(Y (Y )) = n

y1

2 .. T

(y) = 2 2

, y (y) = 2

2

yn+1 y1 yn y n

yn+1

Logarithmic barrier and central path

m

X

(x) = i(fi(x)), dom = {x | fi(x) Ki 0, i = 1, . . . , m}

i=1

is convex, twice continuously differentiable

subject to Ax = b

Dual points on central path

x = x(t) if there exists w Rp,

m

X

tf0(x) + Dfi(x)T i(fi(x)) + AT w = 0

i=1

1 w

i(t) = i(fi(x(t))),

(t) =

t t

m

X

f0(x(t)) g((t), (t)) = (1/t) i

i=1

example: semidefinite programming (with Fi Sp)

minimize cT x Pn

subject to F (x) = i=1 xiFi + G 0

central path: x(t) minimizes tcT x log det(F (x)); hence

maximize tr(GZ)

subject to tr(FiZ) + ci = 0, i = 1, . . . , n

Z0

Barrier method

repeat

1. Centering step. Compute x(t) by minimizing tf0 + , subject to Ax = b.

2. Update. x := x(t).

P

3. Stopping criterion. quit if ( i i)/t < .

4. Increase t. t := t.

P

only difference is duality gap m/t on central path is replaced by i i /t

& '

(0)

P

log(( i i)/(t ))

log

Examples

102

Newton iterations

120

duality gap

0

10

102 80

104 40

106 = 50 = 200 =2

0

0 20 40 60 80 20 60 100 140 180

Newton iterations

140

2

10

Newton iterations

duality gap

100 100

102

60

4

10

106 = 150 = 50 =2 20

0 20 40 60 80 100 0 20 40 60 80 100 120

Newton iterations

Interior-point methods 1230

family of SDPs (A Sn, x Rn)

minimize 1T x

subject to A + diag(x) 0

35

Newton iterations

30

25

20

15

101 102 103

n

Primal-dual interior-point methods

between inner and outer iterations

often exhibit superlinear asymptotic convergence

search directions can be interpreted as Newton directions for modified

KKT conditions

can start at infeasible points

cost per iteration same as barrier method

- MPZ5230-TMA4_2015_2016Uploaded bythar
- Linear and Convex OptimizationUploaded byMEME123
- Linear ProgrammingUploaded byDuma Dumai
- L01_MINI.pdfUploaded byAli Almisbah
- RECONDUCTORIZAR_PLEM (1)Uploaded byXergio Velarde Gomez
- 54IJMPERDDEC201854Uploaded byTJPRC Publications
- Limiting FactorsUploaded byERIC
- Refinery Planning OptimizationUploaded bynntool
- Assembly Line BalancingUploaded byHarshad_S
- calculation of portal frame deflectionUploaded byKonstantinos Kalemis
- Report Evaporator Sculptor Behr 120405Uploaded bygosculptor
- Exercises NosolUploaded byLokesh Gagnani
- Design Methodology of a Permanent MagnetUploaded byJian Zhang
- Una revisión sobre la planificación de generación distribuida.pdfUploaded byWILMAR RUIZ HUACCACHI HERRERA
- OptimizationUploaded byFco Javier Bernal Gongora
- 10 Optimised BlastingUploaded byiwanAni
- Application of Linear Programming-1.docxUploaded byAbdul Sattar
- 86.pdfUploaded byAbdulGhaffar
- NEUTROSOPHIC MAGDM BASED ON ENTROPIES OF DEGREES α, β AND RNORMUploaded byMia Amalia
- SvmUploaded bySinaAstani
- 11 OD Nonlinear Programming a-2008-2Uploaded bycarolinarvsocn
- linear programming project spring 2015Uploaded byapi-240786401
- PWOCalc Installation ManualUploaded byOsama Hassan
- Teoria Dos ContratosUploaded bythiagopoleto
- CSC-9_2Uploaded byDEEPAK
- SimplexMethod1_07312014Uploaded byAlexia Jane J. Afable
- 32 Ch1 QuestionsUploaded byChloe Tham
- KLUWERUploaded byEdward Quijada Orellana
- ResearchUploaded byTejas Gosavi
- 78c75b8cf081b2b3dd5e80848df32425951cUploaded byJunaid Arshad

- Proceedings Multigrid Methods IVUploaded bydmp130
- PanneerchelvamUploaded bydmp130
- qft-II-21-1pUploaded bydmp130
- qft-II-20-1pUploaded bydmp130
- gmsh ManualUploaded byMohamed Salah Sedek
- qft-II-3-1pUploaded bydmp130
- qft-II-2-1pUploaded bydmp130
- Sherry_LiUploaded bydmp130
- 10df0c544ae3e915aedb10fab0c3b0725ef0Uploaded bydmp130
- 2015_GEOPHYS_Frequencies of the Ricker WaveletUploaded bydmp130
- qft-II-17-1pUploaded bydmp130
- 02-syllabusUploaded byandrewkz
- 41 Section Handout 8Uploaded bydmp130
- Coding StyleUploaded byelmoreilly
- geom.pdfUploaded bydmp130
- StatUploaded bydmp130
- Convex Optimization — Boyd & VandenbergheUploaded byPremSuman8
- DualityUploaded byxcentricme
- Intro to PETScUploaded bydmp130
- Effective Parameter (Permittivity and User Guide_epsmuestUploaded bydmp130
- 2011-09_haefele_HDF5-XDMFUploaded bydmp130
- Home Maintenance Guide and TipsUploaded bydmp130
- Syllabus_13Uploaded bydmp130
- PartA7Uploaded bydmp130
- LaiChingMa PhD DissertationUploaded bydmp130
- FDTD TFSF 2DUploaded bydmp130
- MIT22_05F09_lec04Uploaded bydmp130
- Supersymmetry and Cosmology - J. FengUploaded bydmp130
- Standard Practices c++Uploaded bydmp130

- EE_Assignment_1-5.pdfUploaded byShubhekshaJalan
- Manual Adobe AnimateUploaded byMaycol Vivanco Mayta
- How to Fill the Application FormUploaded byNikhil Bhati
- Notecard 2Uploaded byRachel Ann Nelson
- InputDirector LOGUploaded byjharajnish30
- Epson WorkForce ES-300W.pdfUploaded byProduccion TAM
- SPD Brochure Masters in Transportation and Car DesignUploaded byArpitPatel
- Brochure of CinacUploaded byRahul Jain
- 1 Retail Turnaround VFUploaded byTakudzwa S Mupfurutsa
- Chapter 4Uploaded byFatimaIjaz
- Project Management Bo Tonnquist ENGUploaded byIgor Brodin
- 5060-2XX LEXMARK C752.pdfUploaded bygruptnt
- Why Interest Rates Are IncreasingUploaded bymindlesstenant469
- ALS Geochemistry Fee Schedule AUD.pdfUploaded byDanton Johannes
- Anderson Kathleen 1976 ThailandUploaded bythe missions network
- Combined Equity Oriented Scheme FOF Gold 21092016Uploaded byShailendra Sinha
- Beauty and Personal Care - SingaporeUploaded byEdric Lin Yunzhi
- ciancone tuning methodUploaded byNazif Jaafar
- 10.1.1.135Uploaded bytomana mansor
- Monitoring SOPUploaded byNita Purmalasari
- Online Module 2 Assignment-Trophic Levels and You(1).docxUploaded bysameer
- Manual Full Range Analyzer StandardUploaded byEnder Ibañez
- Rule on Declaration of Absolute Nullity of Void Marriages and Annulment of Voidable MarriagesUploaded byKaren Ylaya
- technology integration planUploaded byapi-161778850
- `The coolies revolt': exclusive excerpt from 'The Devil's Milk: A social history of rubber'Uploaded byTerry Townsend, Editor
- Panasonic Panafax DX600 800 Service ManualUploaded bychrisban35
- 6.Review and Example on Combustion TheoryUploaded byVan Nguyen Huu Van
- Make Cluster Computer Part 1Uploaded byGustavo Luis Condoy Pogo
- Big Bazar Company AnalysisUploaded byGurbachan Singh
- Determining MotorUploaded bydoeri96