The essence of metaheuristics

METAHEURISTICS
Achmad P. Rifai
• Optimization is the act of obtaining the
best result under given circumstances.
• The process of finding the conditions that
give the maximum or minimum of a
function.
• Most business decisions = optimization:
varying some decision parameters to
maximize profit (e.g. investment
portfolios, production management,
supply chains, etc.)
 
The Essence of an Optimization Algorithm
To move to a new, better point from an existing point

 
Mathematically, it can be written as

The behaviour of the algorithm can be controlled by α.

Optimization is like finding
a needle in a haystack
Where should we start?
How do we search?
What is the best strategy?
Almost everything is optimization ... or needs optimization ...
• Maximize efficiency, accuracy, profit, performance, sustainability, ...
• Minimize costs, wastage, energy consumption, travel distance/time, CO2 emission,
impact on environment, ...

 Mathematical Optimization
Objectives: maximize or minimize

subject to multiple equality and/or inequality design constraints:

In case of , it becomes a single-objective optimization problem.

Key Components for
Optimization
There are thousands of optimization method

Traditional techniques
• Linear programming (LP) and mixed integer programming.
• Convex optimization and quadratic programming.
• Nonlinear programming: Newton's method, trust-region method, interior point
method, ..., barrier Method, ... etc.
But most real-world problems are not linear or convex, thus traditional techniques often
struggle to cope, or simply do not work...

• Heuristic technique: any approach to problem solving, learning, or discovery that

employs a practical method not guaranteed to be optimal or perfect, but sufficient for
the immediate goals.
Some issues with traditional optimization algorithms
• Traditional algorithms are mostly local search, thus cannot
guarantee global optimality (except for linear programming and
convex optimization).
• Results often depend on the initial starting points (except linear
and convex problems).
• Methods tend to be problem-specific (e.g., k-opt, branch and
bound).
• Cannot deal with highly nonlinear optimization problems
efficiently.
• Struggle to cope problems with discontinuity.
A metaheuristic is formally defined as an iterative generation
process which guides a subordinate heuristic by combining
intelligently different concepts for exploring and exploiting the
search space, learning strategies are used to structure information
in order to find efficiently near optimal solutions [Osman and
Laporte, 1996].
Evolutionary algorithms Physics-based Swarm Based Other algorithms
algorithms algorithms
Genetic Algorithms (GA) Simulated Annealing Particle Swarm Variable Neighborhood
(Holland 1992) (SA) (Kirkpatrick et al. Optimization (PSO) Search (VNS)
1983; Cerný 1985) (Kennedy and Eberhart (Mladenovic and Hansen,
1995) 1997)
Evolution Strategy (ES) Gravitational Search Ant Colony Optimization Large Neighborhood
(Rechenberg, 1978) Algorithm (GSA) (ACO) (Dorigo and Search (LNS) (Pisinger
(Rashedi et al. 2009) Stützle, 2004) and Ropke, 2010)

Genetic Programming Charged System Search Grey Wolf Optimizer Stochastic Fractal Search
(GP) (Koza 1992) (CSS) (Kaveh and (GWO) (Mirjalili et al. (SFS) (Salimi 2015)
Talatahari 2010) 2014)

Biogeography Based Central Force Dragonfly Algorithm Sine Cosine Algorithm

Optimizer (BBO) (Simon Optimization (CFO) (DA) (Mirjalili 2016) (SCA) (Mirjalili2016)
2008) (Formato 2007)

.…. and many more methods

Type of optimization problem

Discrete SA, VNS, LNS GA, BBO, ACO

PSO, GWO, SCA,

Continuous SA, ILS, VNS ABC (artificial bee
colony)

Trajectory Population

Type of metaheuristics
 Trajectory vs. population
• The number of solutions to be carried in based metaheuristics
search process determines whether the
metaheuristic is a single-solution
(trajectory) or population-based
algorithm
• Basic single-solution based algorithms
are more exploitation oriented, whereas
basic population-based metaheuristics
are more explorative in nature (Boussad
et al.2013).
• Trajectory methods use one solution at a
time start from the initial to the
manipulation process during the
iteration. Meanwhile population based
methods use multiple solutions.
 Task 1

Review a journal paper that developed a metaheuristic method

(except GA, SA, PSO)
• Identify the type of problem (continuous, discrete,
combinatorial?)
• Identify the type of metaheuristics (trajectory, population?)
• Find its application for real-world problem

To be submitted by next week

 References
• Martí, R., Pardalos, P. M., & Resende, M. G. (2018). Handbook of heuristics.
Springer International Publishing.
• Gendreau, M., & Potvin, J. Y. (2010). Handbook of metaheuristics. Springer, Boston,
MA.
• Ezugwu, A. E., Adeleke, O. J., Akinyelu, A. A., & Viriri, S. (2020). A conceptual
comparison of several metaheuristic algorithms on continuous optimisation
problems. Neural Computing and Applications, 32(10), 6207-6251.
• Abdel-Basset, M., Abdel-Fatah, L., & Sangaiah, A. K. (2018). Metaheuristic
algorithms: A comprehensive review. In Computational intelligence for multimedia
big data on the cloud with engineering applications (pp. 185-231). Academic Press.