Welcome to Scribd, the world's digital library. Read, publish, and share books and documents. See more
Download
Standard view
Full view
of .
Save to My Library
Look up keyword or section
Like this
1Activity

Table Of Contents

0 of .
Results for:
No results containing your search query
P. 1
Kelley MethIterForOPtimization

Kelley MethIterForOPtimization

Ratings: (0)|Views: 12 |Likes:

More info:

Published by: Julio Cesar Barraza Bernaola on Jul 31, 2010
Copyright:Attribution Non-commercial

Availability:

Read on Scribd mobile: iPhone, iPad and Android.
download as PDF, TXT or read online from Scribd
See more
See less

10/25/2012

pdf

text

original

 
Iterative Methods for OptimizationC.T.Kelley
North Carolina State UniversityRaleigh,North CarolinaSociety for Industrial and Applied MathematicsPhiladelphia
 
Contents
Preface xiiiHow to Get the Software xvI Optimization of Smooth Functions 1
1 Basic Concepts 3
1.1 The Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.2 Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41.3 Necessary Conditions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51.4 Sufcient Conditions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61.5 Quadratic Objective Functions . . . . . . . . . . . . . . . . . . . . . . . . . . 61.5.1 Positive Denite Hessian . . . . . . . . . . . . . . . . . . . . . . . . . 71.5.2 Indenite Hessian . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91.6 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91.6.1 Discrete Optimal Control . . . . . . . . . . . . . . . . . . . . . . . . . 91.6.2 Parameter Identication . . . . . . . . . . . . . . . . . . . . . . . . . 111.6.3 Convex Quadratics . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121.7 Exercises on Basic Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2 Local Convergence of Newtons Method 13
2.1 Types of Convergence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132.2 The StandardAssumptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142.3 Newtons Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142.3.1 Errors in Functions, Gradients, and Hessians . . . . . . . . . . . . . . 172.3.2 Termination of the Iteration . . . . . . . . . . . . . . . . . . . . . . . 212.4 Nonlinear Least Squares . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222.4.1 GaussNewton Iteration . . . . . . . . . . . . . . . . . . . . . . . . . 232.4.2 Overdetermined Problems . . . . . . . . . . . . . . . . . . . . . . . . 242.4.3 Underdetermined Problems . . . . . . . . . . . . . . . . . . . . . . . 252.5 Inexact Newton Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282.5.1 Convergence Rates . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292.5.2 Implementation of NewtonCG . . . . . . . . . . . . . . . . . . . . . 302.6 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 332.6.1 Parameter Identication . . . . . . . . . . . . . . . . . . . . . . . . . 332.6.2 Discrete Control Problem . . . . . . . . . . . . . . . . . . . . . . . . 342.7 Exercises on Local Convergence . . . . . . . . . . . . . . . . . . . . . . . . . 35ix
Copyright ©1999 by the Society for Industrial and Applied Mathematics. This electronic version is for personal use and may not be duplicated or distributed.
Buy this book from SIAM at http://www.ec-securehost.com/SIAM/FR18.html.
 
x CONTENTS
3 Global Convergence 39
3.1 The Method of Steepest Descent . . . . . . . . . . . . . . . . . . . . . . . . . 393.2 Line Search Methods and theArmijo Rule . . . . . . . . . . . . . . . . . . . . 403.2.1 Stepsize Control with Polynomial Models . . . . . . . . . . . . . . . . 433.2.2 Slow Convergence of Steepest Descent . . . . . . . . . . . . . . . . . 453.2.3 Damped GaussNewton Iteration . . . . . . . . . . . . . . . . . . . . 473.2.4 Nonlinear Conjugate Gradient Methods . . . . . . . . . . . . . . . . . 483.3 Trust Region Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 503.3.1 Changing the Trust Region and the Step . . . . . . . . . . . . . . . . . 513.3.2 Global Convergence of Trust RegionAlgorithms . . . . . . . . . . . . 523.3.3 A Unidirectional Trust RegionAlgorithm . . . . . . . . . . . . . . . . 543.3.4 The Exact Solution of the Trust Region Problem . . . . . . . . . . . . 553.3.5 The Levenberg–Marquardt Parameter . . . . . . . . . . . . . . . . . . 563.3.6 Superlinear Convergence: The Dogleg . . . . . . . . . . . . . . . . . . 583.3.7 A Trust Region Method for Newton–CG . . . . . . . . . . . . . . . . . 633.4 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 653.4.1 Parameter Identication . . . . . . . . . . . . . . . . . . . . . . . . . 673.4.2 Discrete Control Problem . . . . . . . . . . . . . . . . . . . . . . . . 683.5 Exercises on Global Convergence . . . . . . . . . . . . . . . . . . . . . . . . 68
4 The BFGS Method 71
4.1 Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 724.1.1 Local Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 724.1.2 Global Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 774.2 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 784.2.1 Storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 784.2.2 A BFGSArmijoAlgorithm . . . . . . . . . . . . . . . . . . . . . . . 804.3 Other Quasi-Newton Methods . . . . . . . . . . . . . . . . . . . . . . . . . . 814.4 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 834.4.1 Parameter ID Problem . . . . . . . . . . . . . . . . . . . . . . . . . . 834.4.2 Discrete Control Problem . . . . . . . . . . . . . . . . . . . . . . . . 834.5 Exercises on BFGS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
5 Simple Bound Constraints 87
5.1 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 875.2 Necessary Conditions for Optimality . . . . . . . . . . . . . . . . . . . . . . . 875.3 Sufcient Conditions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 895.4 The Gradient ProjectionAlgorithm . . . . . . . . . . . . . . . . . . . . . . . . 915.4.1 Termination of the Iteration . . . . . . . . . . . . . . . . . . . . . . . 915.4.2 ConvergenceAnalysis . . . . . . . . . . . . . . . . . . . . . . . . . . 935.4.3 Identication of theActive Set . . . . . . . . . . . . . . . . . . . . . . 955.4.4 A Proof of Theorem 5.2.4 . . . . . . . . . . . . . . . . . . . . . . . . 965.5 Superlinear Convergence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 965.5.1 The Scaled Gradient ProjectionAlgorithm . . . . . . . . . . . . . . . . 965.5.2 The Projected Newton Method . . . . . . . . . . . . . . . . . . . . . . 1005.5.3 A Projected BFGS–ArmijoAlgorithm . . . . . . . . . . . . . . . . . . 1025.6 OtherApproaches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1045.6.1 Infinite-Dimensional Problems . . . . . . . . . . . . . . . . . . . . . . 1065.7 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1065.7.1 Parameter ID Problem . . . . . . . . . . . . . . . . . . . . . . . . . . 1065.7.2 Discrete Control Problem . . . . . . . . . . . . . . . . . . . . . . . . 106
Copyright ©1999 by the Society for Industrial and Applied Mathematics. This electronic version is for personal use and may not be duplicated or distributed.
Buy this book from SIAM at http://www.ec-securehost.com/SIAM/FR18.html.

You're Reading a Free Preview

Download
/*********** DO NOT ALTER ANYTHING BELOW THIS LINE ! ************/ var s_code=s.t();if(s_code)document.write(s_code)//-->