1 views

Uploaded by nezutron

- Lattice_Optimisation_Tutorial.pdf
- Bottou L. Stochastic Learning
- OR
- Optimization Non-Linear Optimization and Kuhn-Tucker
- Significance of the Study
- Tugas Poster
- E289Cd01
- CPD 95a HEC ResPRM QuickStartGuide.desbloqueado
- A Unified Approach to Optimizing Performance In
- Optimizatio journal research
- Optimization of Pressure Vessel Under Thermo-Elastic Condition.pdf
- AN_APPLICATION_OF_BRANCH_AND_CUT_TO_OPEN_PIT_MINE_SCHEDULING.pdf
- Fmin Search
- Artigo IEEE - PESCHON 1968 - Sensitivity in Power Systems
- Maximizing a Psychological Uplift in Love Dynamics
- HDMPCFlyer General
- mehran 0919-3g.pdf
- Laboratorio de Redes Neuronales
- A new computational measurement and optimization approach for D SmT
- Quantitative Techniques

You are on page 1of 37

Lecture 3

January 28, 2002

Algorithms

There are a many different types of optimization and even more

algorithms to perform it.

Why are there so many algorithms?

None of the traditional algorithms are robust.

solution method.

This can become a problem when a user only knows a limited

number of algorithms and tries to apply them to problems they

were not intended for.

Types of Optimization

Important Categories

Methods can be classified according to:

Convex vs. Multi-Modal

Continuous vs. Discrete

Constrained vs. Unconstrained

Linear, Quadratic and Nonlinear Programming

Analytical or Numerical

Important Categories

Methods can be classified according to:

Deterministic vs. Random

Derivative-based vs. Derivative free

Global vs. Local

1. Strictly Convex Problems have a single optima

F(x)

x*

1. Non Convex Problems may have multiple local optima

F(x)

1. In continuous problems x is defined over a range

xlxxu

F(x)

1. In discrete problems x is defined only for specific values

F(x)

1. Problems with Linear or Quadratic Objective Functions and

constraints are much easier to solve than those with general

nonlinear terms.

Examples:

Linear F=ax1 + bx2+ c

Quadratic F=F=ax1 + bx2+ cx22 +d

Nonlinear F=ax1 + bx22+ cx25 +x1x21/2

objective function and constraints

F(x)=ax1 + bx2+ c

g1(x)=x1+x20

objective function and constraints

ANALYSIS

F(x)

g(x)

Deterministic Methods:

follow the same path.

Xo

Conjugate Gradient, Powells

Random Methods:

different paths with each run

Xo

Annealing

Derivative Based Methods:

calculated either numerically of analytically

Examples: Steepest Descent, Newtons Method, Conjugate

Gradient, Varible Metric Method etc.

Derivative Free Methods:

Only F(X) information is used to guide the search

Examples: Powells Search, Random Search, SA, GA, etc.

Local Search Methods: Find best design within a portion of the

design space usually a convex space (neighborhood).

These methods include almost all of the traditional numerical

techniques zero-order and derivative-based.

Tend to get stuck in valleys of the design space

F(x)

Global

Opt.

Local Opt.

x*

Local Opt.

Global Search Methods: Find best design within entire design

space.

Tend to be robust but inefficient and expensive to run.

F(x)

Global

Opt.

Local Opt.

x*

Local Opt.

1. It is very important to match the optimization method to the

problem that you are solving.

2. Know the assumptions involved with each method.

problem will be very inefficient.

in finite time. Use methods that find good solutions fast.

Which algorithm is right for a problem is strongly dependent on

how the problem is mathematically described.

Three basic concepts common to every algorithmic approach to

problem solving

1. A representation of the problem

2. The objective

3. The evaluation function

The representation encodes alternative candidate solutions for

manipulation and defines the size of the search space.

For example in NLP typically the design variables are

represented as real numbers in n dimensions. Is this the only

possibility?

the natural numbers 1,2,3,.,n.

The size of the search space is not determined by the problem, it

is determined by your representation and the way that you handle

this encoding.

Your choice of representation will determine how you will be

able to manipulate the potential solutions

Alternative Representations for a NLP

{x1,x2}={5,6}

{x1,x2}={101,110}

10

X2

0

0

x1

10

The objective represents mathematically the task to be achieved

covered by the salesman subject to the constraint of visiting each

city once and only once and returning the starting city.

mindist (legal trip)

For NLP an example might be:

min F=x1+x2 +x3

The evaluation function is typically a mapping from the space

of possible candidate solutions to a set of numbers , where each

element from the space of possible solutions is assigned a

number to indicate its quality.

The EF allows you to compare solutions in terms of quality.

Current Candidate is the tenth best

Numerical evaluations allow you to rank the solutions and also

their degree of quality.

Current Candidate is 1.3 units better that the second best

candidate

Since it could be computationally expensive to determine a

numerical value for how good or bad a candidate it might be

good enough to know approximately how good a solution is.

Candidate 1

Eval. Function

Candidate 2

Superior

Candidate

For every real world problem the evaluation problem is chosen

by the designer.

It should of course indicate for instance that a solution that

meets the objective is better than one that does not.

It should also depend on factors such as the computational

complexity of the problem.

Often the objective function indicates a good evaluation

function.

Other times you cannot derive a useful Evaluation Function

from the objective:

In the SAT problem the objective is to find a set of boolean

(TRUE,FALSE) variables that satisfies a logical statement

(makes it TRUE).

All wrong candidate solutions return FALSE which does not tell

you how to improve the solution.

Problem

When you design an evaluation function you need to consider that

for many problems the only solutions of interest are the subset that are

Feasible (satisfy the constraints).

The feasible space can be defined as F where F S.

A search problem can then be defined as:

Given a search space S and its feasible part F S find x F such that

eval(x) eval(y)

for all y F

Note that the objective does not appear at all in the formulation!!

If your EF does not correspond with the objective you will

searching for the answer to the wrong problem.

Problem

A point x that satisfies the condition is called a global solution.

Finding a global solution can be difficult and impossible to prove in

some cases.

It would be easier if we could limit the search to a smaller area of S.

This fact underlies many search techniques.

If we concentrate on the area of S near to some point in the search

space we can more easily look in this neighborhood.

N(x)

N(x) of x is a set of all points in the search space that close to the

given point x.

N(x) ={y S: dist(x,y)}

For a continuous NLP the Euclidean distance can be used to define a

neighborhood.

n

dist ( x, y )

2

(

x

y

)

i i

i 1

candidates that would result from swapping two cities in a given

tour.

A solution x (a permutation of n=5 cities)

1-2-3-4-5 has n(n-1)/2 neighbors including

1-3-2-4-5 (swapping cities 2 and 3)

5-2-3-4-1 (swapping cities 1 and 5)

etc.

Example: Quadratic Objective with no Constraints

F=x2+3

xc

Min F=x2+3

xc

N(x): xc- x xc +

F=x2+3

x1

xc

evaluate it.

if F(x1)>F(xc) reject point and choose another.

F=x2+3

xc

x1

x1.

F=x2+3

xc

Most realistic problems are considerably more difficult than a

quadratic bowl problem.

The evaluation function defines a response surface that describes

the topography of the search space with many hills and valleys.

Finding the best peak or the lowest valley is like trying to navigate

a mountain range in the dark with only a small lamp.

Your decisions must be made using local information. You can

sample points in a local area and then decide where to walk next.

If you decide to always go uphill then you will reach a peak but not

necessarily the highest peak.

You may need to walk downhill in order to eventually reach the

highest peak in the space.

- Lattice_Optimisation_Tutorial.pdfUploaded byvovanpedenko
- Bottou L. Stochastic LearningUploaded byfrobenjamin
- ORUploaded byrockstarali
- Optimization Non-Linear Optimization and Kuhn-TuckerUploaded byAhmed Africa Ahmed
- Significance of the StudyUploaded byJonelou Cusipag
- Tugas PosterUploaded byPraba Fitra
- E289Cd01Uploaded byMuhammad Imran
- CPD 95a HEC ResPRM QuickStartGuide.desbloqueadoUploaded byJose Luis Cruzado Ruiz
- A Unified Approach to Optimizing Performance InUploaded byShabeer Vpk
- Optimizatio journal researchUploaded byPrathu 05 - Official
- Optimization of Pressure Vessel Under Thermo-Elastic Condition.pdfUploaded byHerdiana Kristi
- AN_APPLICATION_OF_BRANCH_AND_CUT_TO_OPEN_PIT_MINE_SCHEDULING.pdfUploaded byElmejri Hatem
- Fmin SearchUploaded byhshokri70
- Artigo IEEE - PESCHON 1968 - Sensitivity in Power SystemsUploaded byAndrey Lopes
- Maximizing a Psychological Uplift in Love DynamicsUploaded byAlexander Ainslie (AAinslie)
- HDMPCFlyer GeneralUploaded byTica
- mehran 0919-3g.pdfUploaded byMehran
- Laboratorio de Redes NeuronalesUploaded byOscarPintoRojas
- A new computational measurement and optimization approach for D SmTUploaded byMia Amalia
- Quantitative TechniquesUploaded byPrateek Dave
- 72Uploaded bysudhialamanda
- Multiobjective OptimizationUploaded byAmir Mosavi
- Midterm AUploaded byGlenn Candranegara
- 1-s2.0-S1474667016311983-mainUploaded byMyo Zin Aung
- CDC96Uploaded byDya Ionut
- BiochemicalUploaded byrioR
- Single Machine Power Network Load Frequency Control Using T-S Fuzzy Based Robust Control With Relax LMI ConditionUploaded bytheijes
- 1-s2.0-S0169023X12001000-mainUploaded byagrawal.dipesh
- Final MarioUploaded byAnonymous dUXvWL61
- Optimising Reservoir Operation Hoa Binh, VietnamUploaded byWilliam Tejada

- 815664621Uploaded bynezutron
- 654169233.pdfUploaded bynezutron
- TaxonomyOfLearning.pdfUploaded bynezutron
- Dynamic BasicUploaded byXiaolong Bruce Liu
- Technical Sketching WorksheetsUploaded bynezutron
- Multi Degree of FreedomUploaded bySrinath Gudur
- Font LogUploaded bynezutron
- SKMM 4533 system design marking scheme.pdfUploaded bynezutron
- UTM Thesis Manual 2007Uploaded byifahkhalid
- YSD_2011_FinalUploaded bynezutron
- BoschPowerCordlessDrill2011Uploaded bynezutron
- Qua Tern IonsUploaded bynezutron

- aai syllabusUploaded bysuresh_496
- HW7SolUploaded bymetmet476
- Solution to Least Squares ProblemUploaded byedwards2718
- Exp 2Uploaded bytjrocz
- D1_001_005_McKenna_OSDPT2014Uploaded byMohinuddin Ahmed
- 4-The Simplex MethodUploaded byKalai Vani
- Nm ProgrammingUploaded byrichshy dragon
- List of Chemical Engineering eBooksUploaded byJg Glez
- Mechanics of Composite Materials with MatLabUploaded byjohanncito92
- B.E Syllabus(4 Sem- 7 Sem)Uploaded byShivendra Mishra
- 101-Ways-Extract-Modal-Parameters-Which-is-One-Me-.pdfUploaded byValentino de George
- 2018 R Coısson Et Al Et Al Mathieu Functions and Numerical Solutions of the Mathieu EquationUploaded byTheodora
- Linear ProgrammingUploaded byDeepjyoti Roy
- rocket lab report 2017Uploaded byapi-325274340
- lab 04Uploaded byTenis Vijesti
- Course Structure Syllabus B.tech CSEUploaded bySrimanthula Srikanth
- Gsus ProceedingsUploaded byGlobal Research and Development Services
- ICS2009Abstracts- Octavia Nica - Math and Comput ScienceUploaded byoctavia_octavia
- Numerical Solution Algorithms for Compressible FlowsUploaded byosoreneg
- Updated Room NumbersUploaded bySri Mathi
- aeronasyll.pdfUploaded byNithin
- Factoring PolynomialsUploaded byKelly Cheng
- ASTM E1641 Decomposition Kinetics by ThermogravimetryUploaded byandresjyp
- Chapter 3 - Solution to Systems of Linear Equations.pptxUploaded byaben777
- V. Daru and C. Tenaud- High order one-step monotonicity-preserving schemes for unsteady compressible flow calculationsUploaded byWhiteLighte
- Syllabus de OklahomaUploaded byAlvaro Ramos
- master of computer application bookUploaded byAhamed Ibrahim
- flujo probabilistico de potencia optimoUploaded byWld Lal
- Karmarkar’s AlgorithmUploaded bySajal Agarwal
- 3DCV_svd_000Uploaded byawais04