15 views

Uploaded by gacongnghiep7786

save

- 5_AC654im
- 04b-Non Linear Programming
- 1-s2.0-S2212670814000384-main
- Fista
- Lecture Module 1 - Introduction New
- Sensitivity Analysis
- Nas107 Optimization Seminar
- seismic-tomography8.ppt
- SLOPE CONSTRAINED TOPOLOGY OPTIMIZATION.pdf
- assign3sols
- 6a. Construction Process Optimization
- Aguirre 2004
- Monte_Carlo.pdf
- Re-Allocation
- affisco2002
- 947
- Optimization MassMin04 Final
- 09 Locate
- 82 Optimization of a ComplexChemProcess Using an EquationOrientedModel
- doc-1.pdf
- Normalization PDF
- 03-Cost Concepts and Design Economics CHE40A
- metal
- MAIT Intro Slides
- Users Stable 5.4
- paper ID-28201438
- EDC_1
- MME3271_Lecture00_IntroductionToMME3271
- Compos
- Use of Particle Swarm Optimization Algorithm.pdf
- Motor Fund Am
- 53172487-on-ap-may-phat-AVR2-
- GW40N120KD
- lv25p
- Library File Description OrCAD PSpice
- lecture10v01_descent2
- Spartan 3e - Vietnamese
- MPLAB C30 User's Guide
- FPGA Fucntions
- C30 Users Guide
- Dspic30f6010a 30i Pt Microchip

You are on page 1of 18

• • • • •

Generic descent algorithm Generalization to multiple dimensions Problems of descent methods, possible improvements Fixes Local minima

**Gradient descent (reminder)
**

Minimum of a function is found by following the slope of the function

f

f(x) guess

f(m) m

x

1

Gradient descent (illustration) f f(x) guess next step f(m) m x Gradient descent (illustration) f f(x) guess next step new gradient f(m) m x 2 .

Gradient descent (illustration) f f(x) guess next step f(m) m x Gradient descent (illustration) f f(x) guess f(m) stop m x 3 .

Gradient descent: algorithm Start with a point (guess) Repeat Determine a descent direction Choose a step Update Until stopping criterion is satisfied guess Gradient descent: algorithm Start with a point (guess) Repeat Determine a descent direction Choose a step Update Until stopping criterion is satisfied Direction: downhill 4 .

Gradient descent: algorithm Start with a point (guess) Repeat Determine a descent direction Choose a step Update Until stopping criterion is satisfied step Gradient descent: algorithm Start with a point (guess) Repeat Determine a descent direction Choose a step Update Until stopping criterion is satisfied Now you are here 5 .

Gradient descent: algorithm Start with a point (guess) Repeat Determine a descent direction Choose a step Update Until stopping criterion is satisfied Stop when “close” from minimum Gradient descent: algorithm Start with a point (guess) Repeat Determine a descent direction Choose a step Update guess = x direction = -f’(x) step = h > 0 x:=x–hf’(x) f’(x)~0 Until stopping criterion is satisfied 6 .

Example of 2D gradient: pic of the MATLAB demo Illustration of the gradient in 2D Example of 2D gradient: pic of the MATLAB demo Illustration of the gradient in 2D 7 .

Example of 2D gradient: pic of the MATLAB demo Illustration of the gradient in 2D Example of 2D gradient: pic of the MATLAB demo Definition of the gradient in 2D This is just a genaralization of the derivative in two dimensions. 8 . This can be generalized to any dimension.

Example of 2D gradient: pic of the MATLAB demo Illustration of the gradient in 2D Example of 2D gradient: pic of the MATLAB demo Gradient descent works in 2D 9 .

Generalization to multiple dimensions Start with a point (guess) Repeat Determine a descent direction Choose a step Update Until stopping criterion is satisfied guess Generalization to multiple dimensions Start with a point (guess) Repeat Determine a descent direction Choose a step Update Until stopping criterion is satisfied Direction: downhill 10 .

Generalization to multiple dimensions Start with a point (guess) Repeat Determine a descent direction Choose a step Update Until stopping criterion is satisfied step Generalization to multiple dimensions Start with a point (guess) Repeat Determine a descent direction Choose a step Update Until stopping criterion is satisfied Now you are here 11 .

Generalization to multiple dimensions Start with a point (guess) Repeat Determine a descent direction Choose a step Update Until stopping criterion is satisfied Stop when “close” from minimum Generalization to multiple dimensions Start with a point (guess) Repeat Determine a descent direction Choose a step Update guess = x direction = -f(x) step = h > 0 x:=x–h Vf’(x) Until stopping criterion is satisfied Vf’(x)~0 12 .

you need to compute the gradient of its price: Stock 1 13 . For the descent method. Example of 2D gradient: MATLAB demo The cost to buy a portfolio is: Stock N Stock i Stock 2 If you want to minimize the price to buy your portfolio. and by in N dimensions. f’(x) can be replaced by In two dimensions.Multiple dimensions Everything that you have seen with derivatives can be generalized with the gradient.

large steps: potentially bad results f f(x) guess f(m) stop m Too many steps: takes too long to converge x Problem 1: choice of the step When updating the current computation: .Problem 1: choice of the step When updating the current computation: .large steps: potentially bad results f f(x) Next point (went too far) f(m) m Current point x 14 .small steps: inefficient .small steps: inefficient .

2004 ] 15 . Notes. Boyd.Problem 2: « ping pong effect » [S. Boyd. Notes. L. Stanford Univ. Convex Convex Optimization lect. L. Convex Convex Optimization lect. Stanford Univ. Vandenberghe. Vandenberghe. 2004 ] Problem 2: « ping pong effect » [S.

L. Convex Convex Optimization lect. Prof.Problem 2: (other norm dependent issues) [S. 2004 ] Problem 3: stopping criterion Intuitive criterion: In multiple dimensions: Or equivalently Rarely used in practice. More about this in EE227A (convex optimization. Stanford Univ. Notes. Vandenberghe. El Ghaoui). L. Boyd. 16 .

Newton steps Fundamental problem of the method: local minima Local minima: pic of the MATLAB demo The iterations of the algorithm converge to a local minimum 17 .Normalized steepest descent .Backtracking line search .Line search methods. in particular .Fixes Several methods exist to address this problem .Exact line search .

Local minima: pic of the MATLAB demo View of the algorithm is « myopic » 18 .

- 5_AC654imUploaded byMohammad Rashman
- 04b-Non Linear ProgrammingUploaded byAni
- 1-s2.0-S2212670814000384-mainUploaded byCristian Matamoros Paitan
- FistaUploaded byLelouch V. Britania
- Lecture Module 1 - Introduction NewUploaded byAhmad Sabree Abdul Basit
- Sensitivity AnalysisUploaded byPolu Vidya Sagar
- Nas107 Optimization SeminarUploaded bysuhas110
- seismic-tomography8.pptUploaded bysatyam mehta
- SLOPE CONSTRAINED TOPOLOGY OPTIMIZATION.pdfUploaded bybaba97k
- assign3solsUploaded byAvik Mukherjee
- 6a. Construction Process OptimizationUploaded byAbdelhadi Sharawneh
- Aguirre 2004Uploaded byJames Brown
- Monte_Carlo.pdfUploaded bythuyptt01
- Re-AllocationUploaded bynidee_nishok
- affisco2002Uploaded byBunty Evol
- 947Uploaded byAmin Gholami
- Optimization MassMin04 FinalUploaded byColleen Callahan
- 09 LocateUploaded bychienthan256buoi
- 82 Optimization of a ComplexChemProcess Using an EquationOrientedModelUploaded byEdith Pacheco Contreras
- doc-1.pdfUploaded byAnonymous EvrtA70
- Normalization PDFUploaded byvu3020
- 03-Cost Concepts and Design Economics CHE40AUploaded byFaye Blair Markova
- metalUploaded byMatiz Carmen
- MAIT Intro SlidesUploaded bySOHAM2002
- Users Stable 5.4Uploaded bySandro Ilegor
- paper ID-28201438Uploaded byIJRAT
- EDC_1Uploaded byMohamad Sannan
- MME3271_Lecture00_IntroductionToMME3271Uploaded byAmir Sollehin
- ComposUploaded byalexmechdesigner
- Use of Particle Swarm Optimization Algorithm.pdfUploaded bynguyenlon

- Motor Fund AmUploaded bygacongnghiep7786
- 53172487-on-ap-may-phat-AVR2-Uploaded bygacongnghiep7786
- GW40N120KDUploaded bygacongnghiep7786
- lv25pUploaded bygacongnghiep7786
- Library File Description OrCAD PSpiceUploaded bygacongnghiep7786
- lecture10v01_descent2Uploaded bygacongnghiep7786
- Spartan 3e - VietnameseUploaded bygacongnghiep7786
- MPLAB C30 User's GuideUploaded bygacongnghiep7786
- FPGA FucntionsUploaded bygacongnghiep7786
- C30 Users GuideUploaded bygacongnghiep7786
- Dspic30f6010a 30i Pt MicrochipUploaded bygacongnghiep7786