A Tour of Machine Learning Algorithms

A Tour of Machine Learning Algorithms https://machinelearningmastery.
com/a-tour-of-machine-learning-algorithms/
Never
 miss a tutorial:
Navigation
Picked for you:
A Tour of Machine Learning Algorithms

Click to Take the FREE Algorithms Crash-Course
Search... 
Supervised and Unsupervised Machine

Learning Algorithms
by Jason Brownlee on August 12, 2019 in Machine Learning Algorithms

Logistic Regression Tutorial for Machine
Tweet
Learning
Share Share
Last Updated on August 14, 2020

Simple Linear Regression Tutorial for
In this post, we will
Machine take a tour of the most popular machine learning algorithms.
Learning
It is useful to tour the main algorithms in the field to get a feeling of what methods are available.
Bagging and Random Forest Ensemble
There areAlgorithms
so manyforalgorithms that it can feel overwhelming when algorithm names are thrown around
Machine Learning
and you are expected to just know what they are and where they fit.
I want to give you two ways to think about and categorize the algorithms you may come across in the
Loving the Tutorials?
field.
The Machine Learning Algorithms EBook is
The firstyou'll
where is a find
grouping of algorithms
the Really Good stuff. by their learning style.
The second is a grouping of algorithms by their similarity in form or function (like grouping similar
animals>>together).
SEE WHAT'S INSIDE
Both approaches are useful, but we will focus in on the grouping of algorithms by similarity and go on a
tour of a variety of different algorithm types.
After reading this post, you will have a much better understanding of the most popular machine learning
algorithms for supervised learning and how they are related.
Kick-start your project with my new book Master Machine Learning Algorithms, including step-by-step
tutorials and the Excel Spreadsheet files for all examples.
Let’s get started.
1 of 68 2021-04-11, 9:17 p.m.

A Tour of Machine Learning Algorithms https://machinelearningmastery.com/a-tour-of-machine-learning-algorithms/
Never miss a tutorial:
Picked for you:

Learning Algorithms

Learning
Simple Linear Regression Tutorial

A cool example of anfor
ensemble of lines of best fit. Weak members are grey, the
Machine Learning combined prediction is red.
Plot from Wikipedia, licensed under public domain.

Algorithms Grouped by Learning Style
Algorithms for Machine Learning
There are different ways an algorithm can model a problem based on its interaction with the experience
or environment or whatever we want to call the input data.
It is popular in machine learning and artificial intelligence textbooks to first consider the learning styles
The
that anMachine Learning
algorithm Algorithms EBook is
can adopt.
where you'll find the Really Good stuff.
There are only a few main learning styles or learning models that an algorithm can have and we’ll go
>> SEE WHAT'S INSIDE
through them here with a few examples of algorithms and problem types that they suit.
This taxonomy or way of organizing machine learning algorithms is useful because it forces you to think
about the roles of the input data and the model preparation process and select one that is the most
appropriate for your problem in order to get the best result.
Let’s take a look at three different learning styles in machine learning

algorithms:
1. Supervised Learning
Input data is called training data and has a known label or result
such as spam/not-spam or a stock price at a time.
2 of 68 2021-04-11, 9:17 p.m.

A model is prepared through a training process in which it is

required to make predictions and is corrected when those
predictions are wrong. The training process continues until the
model achieves a desired level of accuracy on the training data.
Picked forproblems
Example you: are classification and regression.
A Tour
Example of Machine
algorithms Learning
include: Algorithms
Logistic Regression and the Back
Propagation Neural Network.
2. Unsupervised Learning
Input data is notAlgorithms
Learning labeled and does not have a known result.
A model is prepared by deducing structures present in the input

data. This mayRegression
Logistic be to extract general
Tutorial rules. It may be through a
for Machine
Learning
mathematical process to systematically reduce redundancy, or it
may be to organize data by similarity.
Simple
Example Linear are
problems Regression Tutorial
clustering, for
dimensionality reduction and
Machine Learning
association rule learning.
Example algorithms include: the Apriori algorithm and K-Means.

3. Semi-Supervised Learning
Input data is a mixture of
labeled and unlabelled
examples.
There is ayou'll
where desired
find the Really Good stuff.
prediction problem but the
model must >> learn
SEE WHAT'S
the INSIDE
structures to organize the

data as well as make
predictions.
Example problems are

classification and
regression.
Example algorithms are extensions to other flexible methods that make assumptions about how to
model the unlabeled data.
Overview of Machine Learning Algorithms
3 of 68 2021-04-11, 9:17 p.m.

When crunching data to model business decisions, you are most typically using supervised and
unsupervised learning methods.
A hot topic at the moment is semi-supervised learning methods in areas such as image classification
where there are large datasets with very few labeled examples.
Picked for you:
Algorithms Grouped By Similarity
Algorithms are often grouped by similarity in terms of their function (how they work). For example, tree-
based methods, and neural network inspired methods.
I think this is the most useful way to group algorithms and it is the approach we will use here.
Learning Algorithms
This is a useful grouping method, but it is not perfect. There are still algorithms that could just as easily
fit into multiple categoriesTutorial
Logistic Regression like Learning Vector Quantization that is both a neural network inspired
for Machine
methodLearning
and an instance-based method. There are also categories that have the same name that
describe the problem and the class of algorithm such as Regression and Clustering.
We could handle
Simple these
Linear cases Tutorial
Regression by listing
for algorithms twice or by selecting the group that subjectively is the
Machine Learning
“best” fit. I like this latter approach of not duplicating algorithms to keep things simple.
In this section, we list many of the popular machine learning algorithms grouped the way we think is the
Bagging The
most intuitive. and Random Forest
list is not Ensemble
exhaustive in either the groups or the algorithms, but I think it is
representative and will be useful to you to get an idea of the lay of the land.
Please Note: There is a strong bias towards algorithms used for classification and regression, the two
most prevalent
Loving supervised machine learning problems you will encounter.
the Tutorials?
The Machine
If you know of Learning Algorithms
an algorithm or a EBook
group is
of algorithms not listed, put it in the comments and share it with
wheredive
us. Let’s you'llin.
find the Really Good stuff.

Regression Algorithms
Regression is concerned with modeling the relationship between
variables that is iteratively refined using a measure of error in the
predictions made by the model.
Regression methods are a workhorse of statistics and have been co-

opted into statistical machine learning. This may be confusing
because we can use regression to refer to the class of problem and
the class of algorithm. Really, regression is a process.
The most popular regression algorithms are:
Ordinary Least Squares Regression (OLSR)
4 of 68 2021-04-11, 9:17 p.m.

Linear Regression
Logistic Regression
Stepwise Regression
Multivariate Adaptive Regression Splines (MARS)
Locally Estimated Scatterplot Smoothing (LOESS)
Picked for you:
Instance-based Algorithms
Instance-based learning model is a decision problem with instances
or examples of training data that are deemed important or required
to the model.
Learning Algorithms
Such methods typically build up a database of example data and
compare new data to the database using a similarity measure in
order to find the best match and make a prediction. For this reason,
instance-based
Learning methods are also called winner-take-all methods
and memory-based learning. Focus is put on the representation of
the stored instances and similarity measures used between
instances.
Machine Learning
The most popular instance-based algorithms are:
Bagging and
k-Nearest Random
Neighbor Forest Ensemble
(kNN)
Algorithms for Machine Learning(LVQ)
Learning Vector Quantization
Self-Organizing Map (SOM)
Locally Weighted Learning (LWL)
Support Vector
Loving theMachines (SVM)
Tutorials?

Regularization Algorithms
An extension made to another method (typically regression
methods) >> SEE
that WHAT'S models
penalizes INSIDE based on their complexity, favoring
simpler models that are also better at generalizing.
I have listed regularization algorithms separately here because they

are popular, powerful and generally simple modifications made to
other methods.
The most popular regularization algorithms are:
Ridge Regression
Least Absolute Shrinkage and Selection Operator (LASSO)
Elastic Net
Least-Angle Regression (LARS)
5 of 68 2021-04-11, 9:17 p.m.

Decision
Never miss a Tree Algorithms
tutorial:
Decision tree methods construct a model of decisions made based
on actual values of attributes in the data.
Decisions
Picked fork in tree structures until a prediction decision is made
for you:
for a given record. Decision trees are trained on data for
A Tour of
classification andMachine Learning
regression Algorithms
problems. Decision trees are often fast
and accurate and a big favorite in machine learning.
The most popular decision tree algorithms are:

Learning Algorithms
Classification and Regression Tree (CART)
Iterative Dichotomiser 3 (ID3)
C4.5 and C5.0 (different versions of a powerful approach)
Chi-squared
Learning Automatic Interaction Detection (CHAID)
Decision Stump
M5
Conditional Decision
Simple Linear Trees
Regression Tutorial for
Machine Learning
Bayesian Algorithms
Bayesian methods
Bagging are those
and Random thatEnsemble
Forest explicitly apply Bayes’ Theorem
for problems such
Algorithms foras classification
Machine Learningand regression.
The most popular Bayesian algorithms are:
Loving
Naive Bayes the Tutorials?
Gaussian
The Naive Bayes
Machine Learning Algorithms EBook is
Multinomial
where Naive
you'll find Bayes
the Really Good stuff.
Averaged One-Dependence Estimators (AODE)
>> SEE
Bayesian WHAT'S
Belief INSIDE
Network (BBN)
Bayesian Network (BN)
Clustering Algorithms
Clustering, like regression, describes the class of problem and the
class of methods.
Clustering methods are typically organized by the modeling

approaches such as centroid-based and hierarchal. All methods are
concerned with using the inherent structures in the data to best
organize the data into groups of maximum commonality.
The most popular clustering algorithms are:
6 of 68 2021-04-11, 9:17 p.m.

k-Means
k-Medians
Expectation Maximisation (EM)
Hierarchical Clustering
Association
Picked for you: Rule Learning Algorithms
Association
A Tourrule learning
of Machine methods
Learning extract rules that best explain
Algorithms
observed relationships between variables in data.
These rules can discover important and commercially useful

Supervised
associations and Unsupervised
in large Machine
multidimensional datasets that can be
Learning Algorithms
exploited by an organization.
The most popular association rule learning algorithms are:

Learning
Apriori algorithm
Eclat algorithm
Artificial Neural
Machine Learning Network Algorithms
Artificial Neural Networks are models that are inspired by the

structure and/or function of biological neural networks.
They are a class of pattern matching that are commonly used for
regression and classification problems but are really an enormous
subfield comprised of hundreds of algorithms and variations for all
Loving
manner of the
problem Tutorials?
types.

Note that I have separated out Deep Learning from neural networks
because of the massive growth and popularity in the field. Here we
are concerned
>> SEE with the more
WHAT'S classical methods.
INSIDE
The most popular artificial neural network algorithms are:
Perceptron
Multilayer Perceptrons (MLP)
Back-Propagation
Stochastic Gradient Descent
Hopfield Network
Radial Basis Function Network (RBFN)
Deep Learning Algorithms

Deep Learning methods are a modern update to Artificial Neural
7 of 68 2021-04-11, 9:17 p.m.

Networks that exploit abundant cheap computation.

They are concerned with building much larger and more complex
neural networks and, as commented on above, many methods are
concerned with very large datasets of labelled analog data, such as
Picked
image,for you:
text. audio, and video.
A Tour
The most of Machine
popular deepLearning
learningAlgorithms
algorithms are:
Convolutional Neural Network (CNN)

Recurrent Neural Networks (RNNs)
Long Short-Term Memory Networks (LSTMs)
Learning Algorithms
Stacked Auto-Encoders
Deep Boltzmann Machine (DBM)
Deep Belief
Logistic Networks
Regression (DBN)
Tutorial for Machine
Learning
Dimensionality Reduction Algorithms
Like clustering methods, dimensionality reduction seek and exploit
the inherent
Machinestructure
Learningin the data, but in this case in an unsupervised
manner or order to summarize or describe data using less
information.
Algorithms
This can be usefulforto
Machine Learning
visualize dimensional data or to simplify data
which can then be used in a supervised learning method. Many of
these methods can be adapted for use in classification and
regression.
Principal
The Machine Component Analysis
Learning Algorithms (PCA)
EBook is
where you'llComponent
Principal find the Really Good stuff. (PCR)
Regression
Partial Least Squares Regression (PLSR)
Sammon Mapping
Multidimensional Scaling (MDS)
Projection Pursuit
Linear Discriminant Analysis (LDA)
Mixture Discriminant Analysis (MDA)
Quadratic Discriminant Analysis (QDA)
Flexible Discriminant Analysis (FDA)
Ensemble Algorithms
Ensemble methods are models composed of multiple weaker models
that are independently trained and whose predictions are combined
in some way to make the overall prediction.
8 of 68 2021-04-11, 9:17 p.m.

Much effort is put into what types of weak learners to combine and
the ways in which to combine them. This is a very powerful class of
techniques and as such is very popular.
Boosting
PickedBootstrapped
for you: Aggregation (Bagging)
AdaBoost
Weighted Average (Blending)
Stacked Generalization (Stacking)
Gradient Boosting Machines (GBM)
Gradient Boosted
Supervised RegressionMachine
and Unsupervised Trees (GBRT)
Random
LearningForest
Algorithms
Other Machine Learning Algorithms

Many algorithms
Learning were not covered.
I did not cover algorithms from specialty tasks in the process of machine learning, such as:
Feature selection
Machine Learningalgorithms
Algorithm accuracy evaluation
Performance measures
Bagging andalgorithms
Optimization Random Forest Ensemble
I also did not cover algorithms from specialty subfields of machine learning, such as:
Computational intelligence (evolutionary algorithms, etc.)

Computer Vision (CV)
Natural
The Language
Machine Processing
Learning Algorithms (NLP)
EBook is
Recommender
where Systems
you'll find the Really Good stuff.
Reinforcement Learning
>> SEE
Graphical WHAT'S INSIDE
Models
And more…
These may feature in future posts.
Further Reading on Machine Learning Algorithms

This tour of machine learning algorithms was intended to give you an overview of what is out there and
some ideas on how to relate algorithms to each other.
I’ve collected together some resources for you to continue your reading on algorithms. If you have a
specific question, please leave a comment.
9 of 68 2021-04-11, 9:17 p.m.

Other
Never Lists
miss of Machine
a tutorial: Learning Algorithms
There are other great lists of algorithms out there if you’re interested. Below are few hand selected
examples.
PickedList
forofyou:
Machine Learning Algorithms: On Wikipedia. Although extensive, I do not find this list or the
organization of the algorithms particularly useful.
A Tour of
Machine Machine Algorithms
Learning Learning Algorithms
Category: Also on Wikipedia, slightly more useful than Wikipedias
great list above. It organizes algorithms alphabetically.
CRAN Task View: Machine Learning & Statistical Learning: A list of all the packages and all the
algorithms supported by each machine learning package in R. Gives you a grounded feeling of
what’s out there and what people are using for analysis day-to-day.
Learning Algorithms
Top 10 Algorithms in Data Mining: on the most popular algorithms for data mining. Another
grounded and less overwhelming take on methods that you could go off and learn deeply.
How Learning
to Study Machine Learning Algorithms
Algorithms are a big part of machine learning. It’s a topic I am passionate about and write about a lot on
this blog. Below
Simple areRegression
Linear few hand Tutorial
selectedforposts that might interest you for further reading.
Machine Learning
How to Learn Any Machine Learning Algorithm: A systematic approach that you can use to study
and understand any machine learning algorithm using “algorithm description templates” (I used this
Bagging to
approach andwrite
Random Forest
my first Ensemble
book).
How to Create Targeted Lists of Machine Learning Algorithms: How you can create your own
systematic lists of machine learning algorithms to jump start work on your next machine learning
problem.
HowLoving
to Research a Machine Learning Algorithm: A systematic approach that you can use
the Tutorials?
to research machine learning algorithms (works great in collaboration with the template approach
listed above).
How to Investigate Machine Learning Algorithm Behavior: A methodology you can use to
understand
>> SEE how machine
WHAT'S INSIDElearning algorithms work by creating and executing very small studies into
their behavior. Research is not just for academics!
How to Implement a Machine Learning Algorithm: A process and tips and tricks for implementing
machine learning algorithms from scratch.
How to Run Machine Learning Algorithms

Sometimes you just want to dive into code. Below are some links you can use to run machine learning
algorithms, code them up using standard libraries or implement them from scratch.
How To Get Started With Machine Learning Algorithms in R: Links to a large number of code
examples on this site demonstrating machine learning algorithms in R.
Machine Learning Algorithm Recipes in scikit-learn: A collection of Python code examples
demonstrating how to create predictive models using scikit-learn.
10 of 68 2021-04-11, 9:17 p.m.

How to Run Your First Classifier in Weka: A tutorial for running your very first classifier in Weka (no
code required!).
Final Word
Picked
I hopefor
youyou:
have found this tour useful.
Please,Aleave
Tour ofaMachine
comment Learning
if youAlgorithms
have any questions or ideas on how to improve the algorithm tour.
Update: Continue the discussion on HackerNews and reddit.

Learning Algorithms
Discover How Machine Learning Algorithms Work!

Learning See How Algorithms Work in Minutes
...with just arithmetic and simple examples
Simple Linear Regression Tutorial for Discover how in my new Ebook:

Machine Learning Master Machine Learning Algorithms
It covers explanations and examples of 10 top algorithms, like:

Linear Regression, k-Nearest Neighbors, Support Vector Machines and much
more...
Finally, Pull Back the Curtain on

Machine Learning Algorithms
Loving the Tutorials? Skip the Academics. Just Results.

SEE WHAT'S INSIDE
Tweet Share Share
About Jason Brownlee

Jason Brownlee, PhD is a machine learning specialist who teaches developers how to get results
with modern machine learning methods via hands-on tutorials.
View all posts by Jason Brownlee →
 A Gentle Introduction to the Progressive Growing GAN How to Implement Progressive Growing GAN Models in Keras 
11 of 68 2021-04-11, 9:17 p.m.

312 Responses to A Tour of Machine Learning Algorithms
REPLY 
Bruce December 20, 2013 at 5:10 pm #
Picked for you:
What
A Tour about reinforcement
of Machine learning algorithms in algorithm similarity classification?
Learning Algorithms
There is also one called Gibbs algorithm under Bayesian Learning

REPLY 
Learning jasonb
Algorithms
December 26, 2013 at 8:34 pm #
Good point bruce, I left out those methods. Would you like me to write a post about
reinforcement learning
Logistic Regression methods?
Tutorial for Machine
Learning
REPLY 
Simple LinearJason's fan
Regression Augustfor
Tutorial 22, 2015 at 6:39 am #
Machine Learning
Yes!!!!
P.S. Please :0
REPLY 
Bk vasan August 19, 2017 at 10:02 am #
Jason,
I enjoy your
The Machine Learning blog and
Algorithms youriswriting style of explaining a complex topic in simple terms.
EBook
where you'll Ifind
havetheone request.
Really GoodDo you have a cheat sheet in choosing the right algorithm. I would like to
stuff.
know when to use what ML algorithm as a starters guideline?
Bk
Jason Brownlee August 20, 2017 at 6:04 am #
Choosing the “right” algorithm for a problem is a process:

http://machinelearningmastery.com/a-data-driven-approach-to-machine-learning/
REPLY 
Rajat July 21, 2016 at 5:21 pm #
Hello Jason,hope u r fine!
12 of 68 2021-04-11, 9:17 p.m.

How can we make a recommender system with the help of Neural network
&
how to implement Collaborative filtering with Neural network
Picked for you:

REPLY 
samLearning
A Tour of Machine February 17, 2017 at 6:09 pm #
Algorithms
What is the difference between a classifier and algorithm? Both r same?

Learning Algorithms
REPLY 
Jason Brownlee February 18, 2017 at 8:36 am #
Hi Sam,
LearningAn algorithm is a procedure. Like learning a tree from data. The outcome of an algorithm is a
model or a classifier, like the tree used to make predictions.

Machine Learning
Beat Tödtli August 3, 2017 at 10:14 pm #
Bagging and Random Then

ForestI Ensemble
don’t quite understand the listing of both Back-Propagation and Hopfield
Algorithms forNetwork
Machineunder
Learning
the title of “Artificial Neural Network Algorithms”. Back-Propagation is
clearly a training algorithm, whereas a Hopfield Network is probably a classifier?
Nice post!


That is fair. Rather than backprop we should list MLP.
Jaideep Adhvaryu August 1, 2020 at 4:58 am #
Hi Jason, very interesting article … I am bit of newbie … i come across

“machine learning algorithms”, “machine learning methods” and “machine learning
models” phrases.
I think i understand algorithm and models. What is specific nuance of “method”?
13 of 68 2021-04-11, 9:17 p.m.

Good question, this will help:

https://machinelearningmastery.com/difference-between-algorithm-and-model-
in-machine-learning/
Picked for you:

REPLY 
Sameer July 28, 2017 at 6:21 am #
Yes Please
Release a ebook on reinforcement learning and Unsupervised Deep learning too . You are
awesome Jason. You made things very simple for us to understand this difficult concepts .
Learning Algorithms
REPLY 
Jason Brownlee July 28, 2017 at 8:36 am #
Learning Thanks for the suggestion Sameer.

REPLY 
Machine Learning Bob November 10, 2019 at 9:44 pm #
Absolutely. I keep turning to Jason books/blogs over and over again for various
Bagging tasks and it all
and Random works
Forest with little tweaking
Ensemble
Jason Brownlee November 11, 2019 at 6:07 am #

Thanks!

EMMANUEL January 12, 2021 at 1:36 am #
Jason’s materials helped me do my Ph.D. research with success. Many thanks

to him
Jason Brownlee January 12, 2021 at 7:52 am #
Thanks! Well done on your phd!
REPLY 
Bob November 10, 2019 at 9:42 pm #
14 of 68 2021-04-11, 9:17 p.m.

Never missI awould

tutorial:
really like that too
REPLY 
qnaguru February 17, 2014 at 5:46 pm #
Picked for you:
Where do newbies (with no analytics/stats background) start learning about this algorithms? And
moreA so
Tour of Machine
how does oneLearning Algorithms
use them with Big Data tools like Hadoop?
Supervised and Unsupervised

jasonb Machine
February 19, 2014 at 8:44 am #
REPLY 
Learning Algorithms
Hi qnaguru, I’d recommend starting small and experimenting with algorithms on small
datasets using a tool like Weka. It’s a GUI tool and provides a bunch of standard datasets and
Logistic Regression
algorithms Tutorial
out of the box. for Machine
Learning
I’d suggest you build up some skill on small datasets before moving onto big data tools like Hadoop
and Mahout.
Machine Learning
REPLY 
swainjo June 9, 2014 at 6:24 pm #
qnaguru,
I would recommend the Coursera courses.
I would also read a couple of books to give you some background into the possibilities and
Loving Nate
limitations. the Tutorials?
Silver; The Signal and The Noise & Danial Kahneman; Thinking Fast and Slow.

REPLY 
Ismo May 20, 2014 at 2:50 am #
The best written one I have found is: “The Elements of Statistical Learning: Data Mining,
Inference, and Prediction, Second Edition”. However you probably need to have some background on
maths/stats/computing before reading that (especially if you are planning to implement them too). For
general algorithms implementation I recommend reading also “Numerical Recipes 3rd Edition: The Art of
Scientific Computing”.
REPLY 
jasonb May 23, 2014 at 8:01 am #
I’m a huge fan of Numerical Recipes, thanks for the book refs.
15 of 68 2021-04-11, 9:17 p.m.

Never miss a tutorial: REPLY 

William May 23, 2014 at 1:37 am #
Not a single one for recommender systems?
Picked for you:

REPLY 
I would call recommender a higher-order system that internally is solving regression or

classification problems. Do you agree?

Learning Algorithms
REPLY 
Jesse February 4, 2020 at 11:00 pm #
exactly
Learning
Simple Linear Regression Tutorial for REPLY 

Jon May 23, 2014 at 2:47 am #
Machine Learning
genetic algorithms seem to be dying a slow death these days (discussed previously
https://news.ycombinator.com/item?id=7712824 )
REPLY 
It’s a good point. Computers are fast enough that you can enumerate the search space
faster than
The Machine a GA Algorithms
Learning can converge (atisleast with the classical toy problems).
EBook

REPLY 
Alex May 23, 2014 at 8:47 am #
I enjoyed this post but I think that this is a misinformed statement. Genetic Algorithms
are most useful in large search spaces (enumerating here would be impossible, were talking
about spaces that could be 10^100) and highly complex non-convex functions. Modern
algorithms are much more sophisticated than the simple techniques used in the 80s e.g.
(http://en.wikipedia.org/wiki/CMA-ES) and (http://en.wikipedia.org
/wiki/Estimation_of_distribution_algorithm). Here is a nice fun recent application:
http://www.cc.gatech.edu/~jtan34/project/learningBicycleStunts.html
REPLY 
Jason Brownlee August 22, 2015 at 4:39 pm #
16 of 68 2021-04-11, 9:17 p.m.

Thanks Alex, you can also check out my 2011 book of algorithm recipes titled Clever
Algorithms: Nature-Inspired Programming Recipes. In it I cover 5 different estimation of
distribution algorithms and 10 different evolutionary algorithms.
Picked for you:

REPLY 
Vinícius May 23, 2014 at 6:29 am #
Hi guys, this is great! What about recommendation systems? I’m fascinated about, how netflix,
amazon and others websites can recommend items based on my taste.

Learning Algorithms
REPLY 
Good point.
You can break a recommender down into a classification ore regression problem.
Learning

REPLY 
Rixi July 12, 2014 at 10:52 am #
Machine Learning
True, or even use rule induction like Apriori…

REPLY 
mycall May 26, 2014 at 3:50 pm #
Wherethe
Loving doesTutorials?
imagination lie? Would it be a Unsupervised Feedback Learning? Maybe its Neural
Deep Essemble Networks. I presume dreaming = imagination while sleeping, hence daydreaming is
The MachineofLearning
imagining Algorithms
new learning EBook�
algorithms is

REPLY 
This is too deep for me @mycall
REPLY 
vas May 27, 2014 at 5:28 am #
I lot of people swear by this chart for helping you narrow down which machine learning approach
to take: http://scikit-learn.org/stable/_static/ml_map.png. It doesn’t seem to cover all the types you list in
your article. Perhaps a more thorough chart would be useful.
17 of 68 2021-04-11, 9:17 p.m.


Thanks for the link vas!
Picked for you:

REPLY 
Rizwan Mian, PhD September 2, 2017 at 6:58 am #
Thanks for sharing and is posted on my wall. Useful but not exhaustive. missing some
topics: preprocessing including feature selection, NLP

Learning Algorithms
REPLY 
Nevil Nayak May 27, 2014 at 7:22 am #
Thid is great. I had always been looking for “all types” of ML algorithms available. I enjoyed
Learning
reading this and look forward to further reading

Machine Learning REPLY 
You’re very welcome @Nevil.

REPLY 
UD May 30, 2014 at 12:42 am #
This is nice and useful…I have been feeling heady with too much data and this kinda gives me a
menu
The from which
Machine to choose
Learning what
Algorithms all isison offer to help me make sense of stuff � Thanks
EBook

REPLY 
That is a great way to think about @UD, a menu of algorithms.
REPLY 
Tim Browning May 30, 2014 at 4:15 am #
You might want to include entropy-based methods in your summary. I use relative-entropy based
monitoring in my work to identify anomalies in time series data. This approach has a better recall rate and
lower false positive rates when tested with synthetic data using injected outliers. Just an idea, your
summary is excellent for such a high level conceptual overview.
18 of 68 2021-04-11, 9:17 p.m.


Bhaskar January 9, 2015 at 7:27 am #
HI Tim
Can you give me some reference from which I can learn about relative-entropy based monitoring ?
Picked for you:

REPLY 
Thanks @Tim, I’ll add a section on time series algorithms I think.

Learning Algorithms
REPLY 
Vincent June 9, 2014 at 7:50 pm #
Hi,
Learning
Thank’s for this tour, it is very useful ! But I disagree with you for the LDA method, which is in the Kernel
Methods. First of all, by LDA, do you mean Linear Discriminant Analysis ? Because if it’s not, the next
partsSimple
of myLinear
commentRegression Tutorial
are useless :p for
Machine Learning
If you are talking about this method, then you should put KLDA (which stand for Kernel LDA) and not
simply LDA. Because LDA is more a dimension reduction method than a kernel method (It finds the best
hyperplane
Baggingthat
and optimize the Fisher
Random Forest discriminant in order to project data on it).
Ensemble
Next, I don’t know if we can view the RBF as a real machine learning method, it’s more a mapping
function I think, but it is clearly used for mapping to a higher dimension.
Except these two points, the post is awesome ! Thank’s again.


where you'll find the Really Good stuff. REPLY 
Thanks @Vincent, I’ll look into moving the algorithms around a bit in their groupings.
REPLY 
@Vincent, I think he mentions Radial Based Network or RBN, which is artificial neural
network (ANN) that uses radial basis functions [1]. Jason is correct in placing it under ANN
classification.
[1] https://en.wikipedia.org/wiki/Radial_basis_function_network
REPLY 
Rémi June 10, 2014 at 8:50 pm #
19 of 68 2021-04-11, 9:17 p.m.

Great post, but I agree with Vincent. Kernel Methods are not machine learning methods by
themselve, but more an extension that allows to overcome some difficulties encountered when
input data are not linearly separable. SVM and LDA are not Kernel-based, but their definition can be
adapted to make use of the famous kernel-trick, giving birth to KSVM and KLDA, that are able to
separate data linearly in a higher-dimensional space. Kernel trick can be applied to a wide variety of
Machine
Picked learning methods:
for you:
– LDA
– SVM
– PCA
– KMeans
and the list goes on…
Moreover, I don’t
Learning think that RBF can be considered a machine learning method. It is a kernel function
Algorithms
used alongside the kernel trick to project the data in a high-dimensional space. So the listing in “Kernel
methods” seems to have a typing error :p
Last point, don’t you think LDA could be added to the “Dimensionality Reduction” category ? In fact, it’s
Learning
more an open question but, mixture methods (clustering) and factor analysis could be considered
“Dimensionality Reduction methods” since data can be labeled either by it’s cluster id, or its factors.
Thanks again
Simple for this
Linear post, giving
Regression an for
Tutorial overview of machin learning methods is a great thing.
Machine Learning
Bagging and Random Forest Ensemble REPLY 

Great comments @Rémi I’ll move things around a bit.

REPLY 
I had this confusion and had to look it up.
– Radial Based Function (RBF) can be used as a kernel
– Radial Based Network (RBN) is a artificial neural network that uses radial basis functions
REPLY 
Pranav Waila June 10, 2014 at 9:24 pm #
Hi qnaguru, I have collected some nice reference books to start digging Machine learning. I
would suggest you to start with “Introduction to statistical learning” and after that you can look into “The
Elements of Statistical Learning: Data Mining, Inference, and Prediction, Second Edition”, “Probabilistic
Machine Learning by David Barber”.
REPLY 
Dean Abbott July 3, 2014 at 9:48 am #
20 of 68 2021-04-11, 9:17 p.m.

Very nice taxonomy of methods. Two small quibbles, both in the Decision Tree section.
1) MARS isn’t a tree method, it’s a spline method. You list it already in the regression group, though could
even go in the regularization group. (not a natural fit in any, IMHO).
2) Random Forests is an ensemble method and sticks out a bit in the trees group. Yes, they are trees,
but so is the MART (TreeNet) and some flavors of Adaboost. Since you already have an ensembles and
RF for
Picked is already
you: there, I think you can safely remove it from the Trees.
Again, you’ve done a great job with this list. Congrats!
Dean

Learning Jason
Algorithms REPLY 
Brownlee August 22, 2015 at 4:49 pm #
Thanks Dean, I’ll take your comments on board.

Learning
REPLY 
sravan August 6, 2014 at 8:41 pm #
Greate
Machine article. my knowledge in Machne learning is improving in bredth not in depth.how should
Learning
i improve my learning.i have done some real time implementations with Regression analysis.and
Random forest.and also i am atteding coursera courses.how would i get real time experience on ML R
with Bagging
Hadoop.and Random Forest Ensemble
REPLY 
lale November 23, 2014 at 9:16 pm #
Thanks Mr.Brownly for your useful guide.Where can we find the implementations of all of these
algorithms?I’ve installed weka but it doesnot have some of these algorithms
REPLY 
You may have to make use of other platforms like R and scikit-learn.
Were you looking for an implementation of a specific algorithm?
REPLY 
SHI XUDONG November 25, 2014 at 2:42 pm #
Great Post!
I am currently learning Sparse Coding. And I have difficulty putting Sparse Coding into the categories you
created.
–What is your idea about Sparse Coding?
21 of 68 2021-04-11, 9:17 p.m.

–Which category should it belong to?

Can you provide some suggestions for learning sparse coding
— what mathematical foundations should I have?
— any good tutorial resources?
— can you suggest a learning roadmap
Picked for you:
I am now taking convex optimization course. Is it a correct roadmap?
REPLY 
Lee January 13, 2015 at 8:48 pm #
Where
Learning does ranking fit into the machine learning algorithms? Is it by any chance under some of
Algorithms
the categories mentioned in the article? The only time I find ranking mentioned in relation to machine
learn is when I specifically search for ranking, none of the machine learning articles discuss it.
Learning
REPLY 
Amelie February 3, 2015 at 10:41 am #
which
Machine algorithm is the more efficient of the similarity algorithm .?
Learning

REPLY 
Jason
Algorithms Brownlee
for Machine Learning
February 19, 2015 at 8:42 am #
Assess similarity algorithms using computational complexity and empirically test them and
see Amelie.

Gudi February 13, 2015 at 3:31 pm #

hat methods/algorithms are suitable for applying to trading patterns analysis. I mean looking at
the trading graphs of the last 6 months (e.g. SPY). Currently, I am looking at the graphs visually. Can an
algorithm come to my aid (I am currently enrolled in an online data mining course) ?
REPLY 
Sounds like a timeseries problem, consider stating out with an auto-regression.
REPLY 
saima May 25, 2015 at 4:23 pm #
22 of 68 2021-04-11, 9:17 p.m.

Hi Jason,
Its a great article. I wish if you could give a list of machine learning algorithms popular in medical
research domain.
regards,
Saima Safdar
Picked for you:

REPLY 
Vicc May 27, 2015 at 7:22 pm #
Great list. Definitely cleared things up for me, Jason! I do have a question concerning Batch
Gradient Descent and the Normal Equation. Are these considered Estimators?
Learning Algorithms
I would love to see a post that addresses the different types of estimators / optimizers that could be used
for each of these algorithms that is simple to understand. Also where does feature scaling (min max
scaling & standardization)
Logistic and for
Regression Tutorial other things fall into all of this? Are they also optimizers? So many things!
Machine
Learning
Thanks so much for spreading your knowledge!

Machine Learning
REPLY 
Henry Thornton June 6, 2015 at 10:49 pm #
Hi Jason
Intrigued by your
Algorithms for comments above about recommendation systems ie.
Machine Learning
“I would call recommender a higher-order system that internally is solving regression or classification
problems.” and,
“You can break a recommender down into a classification or a regression problem.”
The Machine
Could Learning
you please Algorithms
expand EBook
on your is process? In general, I find that people talk about building or
thought
wanting a “classifier” since it is the de-jeure buzzword (and related to deep learning) when in fact, a
recommender or something else will do the job. Anyway, great discussion.
REPLY 
Aharon Robinson June 11, 2015 at 8:53 am #
Great stuff here Jason! Regarding your comments on 12/26, I’ll vote yes to seeing a post on
reinforcement learning methods
REPLY 
Vijay Lingesh June 11, 2015 at 4:13 pm #
Hi Jason,
I’m trying to implement object detection through computer vision through Machine Learning but I’m hitting
23 of 68 2021-04-11, 9:17 p.m.

a wall when trying to find a suitable approach. Can you suggest which kind of algorithm will help me? I’d
like to research more on it.
REPLY 
Rajmohan July 16, 2015 at 3:44 pm #
Picked for you:
Hi.. i am working on finding the missing values by using machine learning approaches..
Any body can suggest new methods to be used..
I am a research scholar

Learning Algorithms
REPLY 
Oren August 5, 2015 at 7:04 pm #
Hi Jason,
Learning
just a small question: In my opinion k-NN, SVM, Naive Bayes, Decision Trees, MaxEnt (even if it’s not
mentioned here) are all considered to be instance-based, isn’t it right?

Machine Learning
REPLY 
Vaibhav Agarwal September 10, 2015 at 3:20 am #
Bagging and Random

Awesome Forest
post now Ensemble
I know where I stand.
REPLY 
shani September 10, 2015 at 10:07 pm #
i started reading and i feel i don’t succeed to understand it.
I where
don’t understand
you'll find thewhich algorithm
Really is good for which type of problem.
Good stuff.
I think that little example for each algorithm will be useful.
REPLY 
Gian September 22, 2015 at 11:30 pm #
Hi,
How can I classify the support vector machines and its extensions in your list?
REPLY 
Stephen Thompson October 7, 2015 at 1:25 am #
Jason: Nice addition of the simple graphic to each of the “families” of machine learning
algorithms. This is a change from what I recall was a previous version of this post. The diagram helps
24 of 68 2021-04-11, 9:17 p.m.

visualize the activity of the family and thus aid developing an internal model of how the members of the
family operate.
A simple but powerful effect.
Picked for you:

REPLY 
Kevin Keane October 28, 2015 at 5:45 am #
The Bayesian Algorithms graphic should be reworked. In particular,
1) the area under both density functions should integrate to one. While no scale is provided, the prior
appears to integrate to a much smaller number than the posterior.
2) inSupervised
general, a and Unsupervised Machine
posterior is narrower / more concentrated than a prior given an observation.
Learning Algorithms
3) (interpreting the baseline as zero density) a posterior typically concentrates the probability of the prior
in a smaller range; it never “moves” probability to a range where the prior density was zero.

Learning
REPLY 
Alvin November 11, 2015 at 8:11 pm #
Hi jason,
Machine Learning
Can you recommend any algorithm to my problem below please?
I need one that does time series analysis that does Bayesian analysis too.
For test set,
I’m given data for hourly price movements for half a day, and tasked to predict for the second half of day.
Clearly a time series (TS) problems.
But on top of that I’m also given information on 10 discrete factors for each day in the training and testing
set.
The
Do Machine
you know Learning Algorithms
of any algo EBook multiple
that creates is TS models conditional upon the values (or bands) of the
various discrete factors at the onset?
REPLY 
Farnaz January 6, 2016 at 5:41 pm #
Hi Jason
Thank you very much for your sharing,I would like to know about Machine Learning methods (algorithms)
which are useful in Prediction.
REPLY 
Lady January 11, 2016 at 9:18 pm #
Hi Jason
I would be very grateful if you could let me know which neural network is useful for multivariate time
25 of 68 2021-04-11, 9:17 p.m.

series classification.For example, classifying patient and healthy people when we have multiple time
series of each feature.
REPLY 
Picked for you:Fredrik February 22, 2016 at 11:55 pm #
Did you find any solution for this? I have the same problem. I was thinking about convolution
neural networks and use the feature space to create a heatmap image and use that as input. For
example, each row in the pixel image will be a RGB value mapped from a feature and each column
will be a specific time point. That way you have all multivariate time series in one image. You might
need to reduce
Supervised the dimensionality
and Unsupervised of the time interval though if they are very high, alternatively take
Machine
partitions of the time interval in sections of the image (first 500 time points in the upper half of the
Learning Algorithms
image for example). I have no idea if this would work, just some thoughts…

Learning
REPLY 
lrq3000 January 28, 2016 at 8:03 pm #
I have
Simple Lineara good background
Regression Tutorialin
forartificial intelligence and machine learning, and I must say this is a
really good list,
Machine I would not have done the categories any other way, it’s very close to perfection. Very
Learning
pertinent information.
However, it would be nice to include Learning Style categories for reinforcement learning, genetic
algorithms and probabilistic models, (but meanwhile you already mention them at the end so this gives a
good pointer for the readers).
The ending links are very good, particularly the “How to Study Machine Learning Algorithms”. It would be
also nice to put a list of machine learning online courses (coursera, udacity, etc. – there’s even a course
by Geoffrey Hinton!), and links to tutorials on how to check and verify that your ML algo works well on
yourMachine
The datasetLearning
(cross-validation,
Algorithms generalization
EBook is curve, ROC, confusion matrix, etc.).
Also, thank’s to previous commenters, your comments are also very pertinent and a good addition to the
article!
REPLY 
lrq3000 January 28, 2016 at 8:25 pm #
To develop my suggestion for adding Learning Style categories: I think these classes of learning
algorithms should be added, since they are used more and more (albeit being less popular than the
currently listed methods) and they cannot be replaced by other classes of learning, they bring their own
capabilities:
– Reinforcement learning models a reward/punishment way of learning. This allows to explore and
memorize the states of an environment or the actions with a way very similar on how the actual brain
learns using the pleasure circuit (TD-Learning). It also has a very useful ability: blocking, which naturally
allows a reinforcement learning model to only use the stimuli and information that is useful to predict the
26 of 68 2021-04-11, 9:17 p.m.

reward, the useless stimuli will be “blocked” (ie, filtered out). This is currently being used in combination
with deep learning to model more biologically plausible and powerful neural networks, that can for
example maybe solve the Go game problem (see Google’s DeepMind AlphaGo).
– Genetic algorithms, as a previous commenter said, are best used when facing a very high dimension
problem or multimodal optimizations (where you have multiple equally good solutions, aka multiple
Picked for you: Also, a big advantage is that genetic algorithms are derivative-free cost optimization
equilibriums).
methods, so they are VERY generic and can be applied to virtually any problem and find a good solution
(even if other algorithms may find better ones).
– Probabilistic models (eg, monte-carlo, markov chains, markovian processes, gaussian mixtures, etc.)
and probabilistic graphical models (eg, bayesian networks, credal networks, markov graphs, etc.) are
greatSupervised and situations
for uncertain Unsupervised
andMachine
for inference, since they can manipulate uncertain values and hidden
Learning Algorithms
variables. Graphical models are kinda close to deep learning, but they are more flexible (it’s easier to
define a PGM from a semantic of what you want to do than a deep learning network).
– Maybe mention
Logistic at theTutorial
Regression end Fuzzy Logic, which is not a machine learning algorithm per se but is close to
for Machine
probabilistic
Learning models, except that it can be seen as a superset that also allows to define a possibility value
(see possibilistic logic, and the works by Edwin Jaynes).

Machine Learning
REPLY 
vicky March 13, 2016 at 6:28 am #
hi, and Random Forest Ensemble

Bagging
you Algorithms
have nicelyforillustrated the algos,thanks.
Machine Learning
cheers,
vicky | techvicky.com

where you'll find the22,Really REPLY 
Ting April 2016 atGood stuff.
1:11 am #
I’m just getting started on learning about machine learning algorithms. I still need some time to digest
what I’ve read here. My background comes from finance/investing and therefore I’ve been trying to learn
more about how machine learning is used in investing. I come from a fundamental investing background
and therefore I’m curious if you have an insight. Given there are so many algorithms (and different
branches https://www.youtube.com/watch?v=B8J4uefCQMc which I thought this was an interesting
video) I wanted to ask how do you know which type of branch/algorithm in machine learning would be
more useful for investing?
Best,
Ting
REPLY 
27 of 68 2021-04-11, 9:17 p.m.

Never miss Marc

a tutorial:
June 30, 2016 at 9:07 pm #
Thank you for this great article.
It really helps untangling the variety of algorithm types and muddling through the complexity of this
interesting field.
Picked for you:
REPLY 
I’m glad to hear that Marc.

Learning Algorithms
REPLY 
Jitu
Logistic Rout JulyTutorial
Regression 1, 2016 at 3:14 pm #
for Machine
Learning
Very useful one.

Machine Learning
REPLY 
Thanks Jitu.
REPLY 
RZZ July 13, 2016 at 11:49 am #
not able to download anything. Just keeps cinfirming my subscriptions
REPLY 
>> SEE WHAT'S INSIDE July 13, 2016 at 11:55 am #
Sorry to hear that. After you confirm your subscription you will be emailed the mindmap.
Perhaps check one of your other email folders?
REPLY 
Francis Kim July 13, 2016 at 3:41 pm #
Great insight, thank you for the write up.
REPLY 
Jason Brownlee July 13, 2016 at 4:32 pm #
28 of 68 2021-04-11, 9:17 p.m.

You’re welcome Francis.

REPLY 
perumahan di semarang atas July 13, 2016 at 5:07 pm #
Picked for you:
Excellent post. I was checking constantly this blog and I’m impressed!
Very useful information particularly the last part :
) I care for such information a lot. I was looking for this
certain information for a long time. Thank you and good luck.

Learning Algorithms
REPLY 
I’m glad you found it useful.

Learning
REPLY 
beyond
Simple July 14, 2016 Tutorial
Linear Regression at 5:28 pm
for#
Machine Learning
Hi Jason,
I would like to know the class for SVM.
REPLY 
It does
Loving not fit neatly into this taxonomy.
the Tutorials?

REPLY 
diem du lich nha trang July 26, 2016 at 11:58 am #
Appreciating the dedication you put into your site and
detailed information you offer. It’s nice to come across
a blog every once in a while that isn’t the same old
rehashed material. Excellent read! I’ve bookmarked your site and I’m including your RSS feeds
to my Google account.
REPLY 
Jason Brownlee July 26, 2016 at 2:06 pm #
Thanks.
29 of 68 2021-04-11, 9:17 p.m.


Frank Ihle July 27, 2016 at 3:26 am #
You listed logistic regression as an regression algorithm. I always believed method is the base of
neuronal networks, and thus more a classifier than a regression algorithm.
Picked for you:

REPLY 
Ben Bothur June 23, 2017 at 7:19 pm #
I fully agree with your opinion. The outcome of a (simple) logistic regression is binary and
the algorithm should be part of a classification method, like the neural networks you mentioned.
Learning Algorithms
REPLY 
Abhishek August 6, 2016 at 2:50 am #
Learning
Hello sir. Thank you so much for your help. But as we know Machine Learning require a strong
‘Math’ background. I am very interested math but, i am little bit week in that. So,I want good
understandable resources for math required in Machine Learning. Thank you.
Machine Learning
REPLY 
I teach an approach to getting started without the theory or math understanding. By treating
ML as a tool you can use to solve problems and deliver value. The deep mathematical understanding
can come later if and when you need it in order to deliver better solutions.
See this post:
http://machinelearningmastery.com/how-do-i-get-started-in-machine-learning/

REPLY 
Vladimir August 14, 2016 at 12:00 am #
Jason, thanks for the write-up. When it comes it supervised learning using regression analysis
all examples I have found deal with simple scalar inputs and perhaps multiple features of one input. What
if an input data is more complicated, say two values where one is a quadratic curve and another is a real
number? I have data that consists of two pairs of values: univariate quadratic function (represented as
quadratic functions or an array of points) and a real value R. Each quadratic function F rather predictably
changes its skew/shape based on its real value pair R and becomes(changes) into F’. Given a new real
value R’, it becomes F” and so on. This is the training data and I have about thousand pairs of functions
and real values. Based on a current function F and a new real value R can we predict the shape of F’
using supervised learning and regression analysis? If so, what should I look out for? Any help would be
much appreciated!
30 of 68 2021-04-11, 9:17 p.m.


Irina Max August 14, 2016 at 12:07 pm #
What about Best-subset Selection, Stepwise selection, Backward Selection as Dimension

reduction?? This is Regularization methods but you also can use it as shinkage dimension.
Picked for you:

REPLY 
jalg August 15, 2016 at 11:37 pm #
Do any of the algorithms have a feedback loop?

Learning Algorithms
REPLY 
Bryan August 19, 2016 at 9:20 am #
Jason- would like to discuss in detail the ability to project outcomes of sporting events using
Learning
your algorithms. Have you ever researched?
Please email
utdad1@gmail.com
Machine Learning

REPLY 
Jason
Algorithms Brownlee
for Machine August 20, 2016 at 6:01 am #
Learning
I have only done a little work in that area Bryan.
You can contact me directly here:

http://machinelearningmastery.com/contact
REPLY 
>> SEE WHAT'S INSIDE September 11, 2016 at 7:00 am #
Thanks for this wonderful tour of the machine learning algoritem zoo — more fun than the real
one.
REPLY 
Jason Brownlee September 12, 2016 at 8:29 am #
I’m glad you found it useful Howard.
REPLY 
srinivas n September 15, 2016 at 7:28 am #
31 of 68 2021-04-11, 9:17 p.m.

This is useful, but it could be made more useful for someone new to the field, specifcally in the section
where algorithms are grouped by similarity, by clarifying exactly what is being learned. Eg Regression
algorithms learn the curve that best fits the data points, Bayesian learning algorithms learn the
parameters and structure of a Bayesian network, Decision Tree algorithms learn the structure of the
decision tree, etc.
Picked for you:some kind of task based classification would be helpful. Eg if you’re trying to classify then the
Additionally
following kinds of ML algorithms are best, if you’re trying to do inference then rule learning and bayesian
network learning are good, if you’re curve fitting then regression is good, etc.
Supervised and Unsupervised Machine REPLY 

srinivas n September 15, 2016 at 7:31 am #
Learning Algorithms
Just to clarify that first point I made: eg when you write Naive Bayes, its not the Naive Bayes
method itself that’s being learned, nor whether a given fruit is an apple or pear, but the structure and
parameters of that network that apply Bayes method and can then be used to classify a given fruit
Learning

REPLY 
Machine Learning
Great suggestion, thanks Srinivas.

REPLY 
Anuj Jain September 21, 2016 at 10:04 pm #
Hi Jason,
This article depicted almost all algorithms theoretically best at least for me (as a beginner)
But i am new to ML so i am not able to relate algorithms use cases in a real life problems/scenarios. Can
u please suggest me some links where i could be able to relate each algorithms with a different real
time/real life business problem?
Thanks in advance! �
REPLY 
Hi Anuj, it is generally helpful think of predictive modeling problems in terms of classification

(predict a class or category) and regression (predict a number).
You can then divide algorithms into classification and regression types.
This page has a nice list of modern and popular machine learning problems:
http://machinelearningmastery.com/tour-of-real-world-machine-learning-problems/
32 of 68 2021-04-11, 9:17 p.m.


Yadav Avdhesh October 24, 2016 at 10:23 pm #
Hi Jason.
I am beginner of Machine learning. Can you suggest that how to start learning and what are the basic
things to need for this.
Picked for you:
Best Regards,
Avdhesh
Supervised and Unsupervised

Jason Machine
Brownlee October 25, 2016 at 8:24 am #
REPLY 
Learning Algorithms
Great question Avdhesh,
ILogistic
teach aRegression
top-down Tutorial
approachfor to machine learning, you can learn all about it here:
Machine
http://machinelearningmastery.com/start-here/#getstarted
Learning

REPLY 
Steffen
Machine November 19, 2016 at 2:24 am #
Learning
Jason, this is an excellent list, thank you.
I amBagging and to
totally new Random Forest
the topic – soEnsemble
it is a good starting point. In other ‘domains’ of methods, patterns or
algorithm types I am more familiar with one could typically define generic weaknesses/pains,
strengths/gains and things to look at with care (e.g. how to set parameters). I wonder what those would
be for each of the algorithm groups you specified.
I haveLoving
seen thatthe
youTutorials?
described use cases, e.g. one could use Bayesian Algorithms and Decision Tree
Algorithms for classification. But when would I e.g. prefer the one over the other for classification? …just
an example…
REPLY 
It’s a great question Steffen, and very hard to answer.
The best practical approach to find the best/good algorithms for a given problem is trial and error.
Heuristics provide a good guide, but sometimes/often you can get best results by breaking some
rules or modeling assumptions.
I recommend empirical trial and error (or a bake-off of methods) on a given problem as the best
approach.
REPLY 
Dr. Khalid Raza November 20, 2016 at 6:04 pm #
33 of 68 2021-04-11, 9:17 p.m.

Dear Dr. Jason,

Very nice post. thanks
Picked for you:Jason Brownlee November 22, 2016 at 6:47 am # REPLY 
A Tour of I’m
Machine Learning
glad you foundAlgorithms
it useful Khalid.
Supervised
Tammi and Unsupervised
November 29, 2016 Machine
at 11:32 pm #
REPLY 
Learning Algorithms
Hi Jason,
Thank you so much for your article.

HowLearning
would you suggest NLP can be utilised to measure research performance?

Machine Learning
Jason Brownlee November 30, 2016 at 7:56 am # REPLY 
Sorry, I’m not an expert in NLP Tammi.

REPLY 
sbollav December 10, 2016 at 4:01 am #

Hi Jason,
The Machinepost
Wonderful Learning Algorithms
! Really helped EBook
me a lotis in understanding different algorithms.
My question is i have seen lot of algorithms apart from the above list.
>>please
Can you SEE WHAT'S INSIDE
post the algorithms, how they work with list of examples.
Example: Cart algorithm (Decision Trees) – How they split, entropy, info gain, gini index and impurity.
I hope you got my question. Likewise for all algorithms.
REPLY 
Jason Brownlee December 10, 2016 at 8:08 am #
Sure, I explain how algorithms work in this book:

https://machinelearningmastery.com/master-machine-learning-algorithms/
If you’re more of a coder, I explain how they work with Python code in this book:
https://machinelearningmastery.com/machine-learning-algorithms-from-scratch/
34 of 68 2021-04-11, 9:17 p.m.

I hope that helps.

REPLY 
Iman December 11, 2016 at 7:32 pm #
Picked for you:
Your page, no your website is gold. I have very poor knowledge in Machine learning, and you
helped me in few paragraphs to learn more when to use which algorithm.
I am yet to go through your book, but I decided a thank you is a must. Thanks a lot.

REPLY 
Learning Jason Brownlee December 12, 2016 at 6:47 am #
Algorithms
Thanks Iman.
Learning
REPLY 
sbollav December 15, 2016 at 9:57 pm #
Hi Jason,
Machine Learning
Thank you for kind reply.
Master Machine Learning Algorithms – With this book, Is it possible to understand how the algorithm
works and howfor
Algorithms to Machine
build theLearning
predictive models for different kinds training sets.
And by seeing the problem or train data, can we say that the machine learning (tree based, knn, Naive
base or optimisation ) and the algorithms (cart, c4.5) are best suitable.
Loving the
I can purchase Tutorials?
that above book that you have mentioned –
ButMachine
The I am more concerned
Learning with how
Algorithms theisalgorithm works (more illustration) and apply in machine learning.
EBook
Present i amfind
where you'll using
theR.
Really Good stuff.
35 of 68 2021-04-11, 9:17 p.m.


Hi sbollav, after reading Master Machaine Learning Algorithms you will know how 10 top
algorithms work.
Picked for
For you:
working through predictive modeling problems in R, I would suggest the book: Machine Learning
Mastery With R:
http://machinelearningmastery.com/machine-learning-with-r/
It does not teach how algorithms work, instead, after reading it you will be able to confidently work
through your own machine learning problems and get usable results with R.
I hope that helps.
Learning Algorithms
Logistic Regression Tutorial for Machine REPLY 

Jemmy January 16, 2017 at 10:41 am #
Learning
Thanks ever so much for your great post
Do you have
Simple any idea
Linear about PQSQ
Regression Tutorial algorithms?
for Could you dive in?
Machine Learning
REPLY 
Jason
Bagging and Brownlee
Random Forest Ensemble
January 16, 2017 at 11:02 am #
Sorry, I have not heard of this type of algorithm.

REPLY 
Cara February 7, 2017 at 1:17 am #
where you'll
Heyfind the Really Good stuff.
Jason.
So I’m writing my thesis on MLAs in Motion Analysis (focussing on instrumented insoles) and I was
wondering which type of MLA would be the most useful and if your ebook has the information I need (like
what kind of data needs what kind of MLA) or if I should keep scouring PubMed for answers. Mostly I
found C4.5, CART, Naïve Bayes, Multi-Layer Perceptrons, and Support Vector Machines (especially
SVM, it seems like the most popular in rehab technologies), but I want to be thorough. After all my
degree depends on it �
Your summary on this page was already very helpful, so thank you for that!
36 of 68 2021-04-11, 9:17 p.m.


Hi Cara, I do not cover the problem of motion analysis directly.
I would advise evaluating a suite of algorithms on the problem and see what works best. Use what
Picked for you:
others have tried in the literature as a heuristic or suggestions of things to try.
REPLY 
Vicky February 10, 2017 at 11:28 pm #
Supervised and, Unsupervised Machine

Hi Jason
Learning Algorithms
This is really a superb classification of algorithms.
Can you please help me with below.

I have few rules and I classified my target variable as 0 or 1 by these rules.
Learning
Now I want Machine to learn these rules and predict my target variable .
CanSimple
you please
Linearsuggest me which
Regression Tutorialalgorithm
for is good to do so.
Machine Learning
REPLY 
Jason
Random February 11, 2017 at 5:02 am #
Forest Ensemble
Why not just use your rules directly Vicky? Why is another algorithm required?

REPLY 
Vicky February 12, 2017 at 2:52 am #
where you'll find the Really ,Good stuff.
Hi Jason
>>hese rules areINSIDE

SEE WHAT'S not straight forward and requires SME judgement.
I wanted to know if there is any possibility to teach machine these rules.
REPLY 
Hi Vicky,
Yes, there will be a number of ways. Generally, machine learning is intended to learn the
mapping/rules automatically from examples of the data input and output.
Perhaps try this approach, try a few different methods. You may even come up with an
objectively better mapping.
37 of 68 2021-04-11, 9:17 p.m.


Dina February 21, 2017 at 6:38 am #
Hi Jason
First thank you for your explanation..

Picked for you:
I’m new in Machine learning and i have a question,, all the algorithm can i use it in the supervised
learning ??ofand
A Tour how to
Machine know what
Learning is the best model can i use it for the classification image?
Algorithms
Thank you

Learning Algorithms REPLY 
Great question Dina,

We cannot know which algorithm will be best for a given problem. We must design experiments to
Learning
discover it.
See this post on the topic:

Machine Learning

Abhisek February 22, 2017 at 12:59 am #
Let’s take a look at four different learning styles in machine learning algorithms:
Where is the fourth one ?


Thanks, fixed.
REPLY 
Olatunde Tijani March 14, 2017 at 2:41 pm #
Jason, am happy to find your site where machine learning and its algorithm are discussed. Its
comforting. Am working on Natural Language Processing and intend to add a machine learning algorithm
to it but alas you listed NLP under other type of machine learning algorithm. That’s startling! Because my
aim was to locate the best algorithm to use.
REPLY 
Jason Brownlee March 15, 2017 at 8:08 am #
38 of 68 2021-04-11, 9:17 p.m.

Thanks, it’s great to have you here.

I hope to cover NLP in detail later this year.
Picked for you:

Nipuna March 24, 2017 at 1:55 am # REPLY 
A Tour of Machine
Jason, how toLearning Algorithms
use machine learning, NLP or both to predict user next sentence based on
previously entered text

Learning Jason
Algorithms
Brownlee March 24, 2017 at 7:56 am # REPLY 
I don’t have many NLP examples yet, soon hopefully.

This might help as a start:
Learning
http://machinelearningmastery.com/predict-sentiment-movie-reviews-using-deep-learning/

Machine Learning
REPLY 
PatsWagh96 March 24, 2017 at 12:57 pm #
Sir need a formal introduction for “Grouping of algorithms by similarity in form or function”.
Everywhere onfor
Algorithms internet
Machineit comes under the supervised learning style classified a scluster classification so
Learning
is it a part of learning style??

REPLY 
Michael May 15, 2017 at 1:51 pm #
where you'll find the
I guess youReally
missedGood stuff. algorithms like ARIMA, TBATS, Prophet and so on.
forecasting
REPLY 
Jason Brownlee May 16, 2017 at 8:34 am #
I cover time series in detail here:

http://machinelearningmastery.com/start-here/#timeseries
REPLY 
Deepak May 17, 2017 at 11:03 am #
Hi Jason ..You had created a great site.
Keep sharing the good stuff.
39 of 68 2021-04-11, 9:17 p.m.

God Bless you.

REPLY 
Picked for you:
Thanks Deepak!
REPLY 
Azarm Nowzad May 18, 2017 at 12:00 am #
Hi Jason,
Learning Algorithms
thanks for sharing this great stuff. I need to choose an ML algorithm on a non-rigid object detection in an
image data base ( smoke, cloud,…). Do you have any suggestion on the proper algorithm or a way to
find Logistic Regression
it out. I come to theTutorial
point toforuse
Machine
CNN. Still not sure why should it be ?
Learning
tnx a lot

Yes, I would recommend a CNN.

REPLY 
Fakhre Alam May 29, 2017 at 8:12 am #
Loving the
How do weTutorials?
decide which machine learning algorithm to use for a specified problem?
REPLY 
>> SEE WHAT'S INSIDE June 2, 2017 at 12:18 pm #
A great question, see this post:

REPLY 
Kumar May 31, 2017 at 4:03 pm #
Hi Jason, its amazing material. Do you have any suggestion for online clustering techniques ?
REPLY 
Jason Brownlee June 2, 2017 at 12:44 pm #
40 of 68 2021-04-11, 9:17 p.m.

Thanks Kumar, sorry I don’t have material on clustering.

REPLY 
Girish Korekar June 14, 2017 at 2:55 am #
Picked for you:
I would like know about the How an ‘algorithms’ works on “Machines”?
Here, please consider “Machines” as a “Humans” or “biological VIRUS” or “any living cells”.
I would say biological individuals have a logical series of an algorithm, which is regulates their commands
and response. These algorithms we may call as a ‘Genetic Material’ as ‘DNA or RNA’; but I would like to
see them as an “ALGORITHMS” which is regulates there all activities like responses and commands.
Because, particular DNA or RNA sequences have special type of code, which can be used by different
Learning Algorithms
performers, here performers are Enzymes.
My query is that, can we able to form algorithms like DNA or RNA which can be able to run a Machine?
It can be possible
Logistic to form
Regression a Human
Tutorial made Biological VIRUS that can be cure our infected cells within a
for Machine
Learning
Human Body?

Machine Learning
Jason Brownlee June 14, 2017 at 8:49 am # REPLY 
Sure, take a look at biologically inspired computational methods.

I have a whole book on the topic:
http://cleveralgorithms.com/nature-inspired/index.html

REPLY 
Pavan GS July 14, 2017 at 1:27 am #
where you'll
Great findarticle
the Really Good stuff.
Jason…and a engaging comments section which is rarely the case.
Appreciate the effort and many thanks to all the others. The comments are as informative as the article
itself >> SEE WHAT'S INSIDE
REPLY 
Thanks Pavan.
41 of 68 2021-04-11, 9:17 p.m.


Pavan GS July 14, 2017 at 1:33 am #
#edit#
“a engaging comments section which is rarely the case on so many other sites”
Picked for you:

REPLY 
I work hard to respond to every comment I see. It’s getting harder and harder with 100s per
day now.
Learning Algorithms
REPLY 
Ahmed July 17, 2017 at 4:38 am #
Learning
Hi Dr. Jason;
I have problem with Fast Orthogonal Search (FOS) for dimensionality reduction. So do you have any
suggestion to build it on MATLAB.
thanks
Machine Learning

REPLY 
Jason
Algorithms Brownlee
for Machine July 17, 2017 at 8:47 am #
Learning
I do not, sorry.
The Machine Learning

diviya Algorithms
July 24, EBook
2017 at 6:37 pm # is
REPLY 

I have a doubt can we combine nature inspired algorithm with machine learning to improve
>> level
accuracy SEE WHAT'S INSIDE
of our data
REPLY 
Perhaps. For example, genetic algorithms can help turning hyperparameters or choosing
features.
I have a book on nature inspired algorithms I wrote right after completing my Ph.D., it’s free online
here:
http://cleveralgorithms.com/nature-inspired/index.html
REPLY 
42 of 68 2021-04-11, 9:17 p.m.

Never miss Joshua Reeve July 31, 2017 at 4:17 am #

a tutorial:
Hello Jason, could you label all the algorithms on this page as supervised, unsupervised, or
semi-supervised? It’s easy enough to understand what these three different types are, but which ones
are which? Thanks.
Picked for you:

REPLY 
Most of applied machine learning (e.g. predictive modeling) is concerned with supervised
learning algorithms.
Learning Algorithms
The majority of algorithms listed on this page are supervised.

Learning
REPLY 
David Nettleton August 1, 2017 at 2:33 am #
Hi Jason, thanks for your great article! I would propose an alternative classification of ml
algorithms into two groups: (i) those which always produce the same model when trained from the same
Machine Learning
dataset with the récords presented in the same order and (ii) those which produce a different model each
time. But I would be interested on your thoughts about this.
BestBagging
regards,and
David
REPLY 
Nice David. Really this is the axis of “model variance”.
Think
where of find
you'll it asthe
a continuum though,
Really Good stuff. not binary. Many (most!) ML algorithms suffer variance of some
degree.
REPLY 
David Nettleton August 4, 2017 at 7:25 pm #
Thanks for your reply, Jason. Yes, the continuos scale would be better. Some years ago
I worked with simulated annealing/gradient descent, genetic algs. and neural networks (which
performed random jumps to escape local minimums). However, on the other hand, the
information gain calculation inside a rule
induction algorithm such as M5Rules always follows the same path (?) Could be the basis for an
article ;-
REPLY 
43 of 68 2021-04-11, 9:17 p.m.

Thanks for the suggestion David.

REPLY 
Mirna Magdy August 9, 2017 at 3:14 am #
Picked for you:
Great job , but you didn’t include References for your topic!!!
REPLY 
Thanks Mirna, what references do you want to see?
Learning Algorithms

Jeven Dale M. Marfil August 10, 2017 at 10:31 am #
Learning
Is it possible to produce a function from the unsupervised machine learning?

Machine Learning
REPLY 
Sure, what do you mean exactly?
REPLY 
Jeven
Loving the Dale M. Marfil August 10, 2017 at 6:13 pm #
Tutorials?
The Machine Learning Thank you for EBook

Algorithms the reply
is sir. Say I collected a large amount of data e.g. temperature for a
where you'll find the Really Good stuff. how to apply machine learning in interpreting the data. That is
period of time. I was wondering
why my idea was to produce a function out from the graph, is this still relevant to machine
REPLY 
It sounds like you are describing a regression equation, like a line of best fit.
If a line of best fit is good enough for you, then I would recommend using it.
You can use a suite of machine learning algorithms to make predictions, this process will give
you an idea of what is involved:
http://machinelearningmastery.com/start-here/#process
44 of 68 2021-04-11, 9:17 p.m.

Never miss Massimo

a tutorial:August 29, 2017 at 9:42 pm #
Hi Jason
I have created several supervised models with some regression and classification estimators. I want to
know how to create a data driven application using these models? Could you explain what does it mean?
Picked for you:

REPLY 
You would need to treat it like any other software project and start by defining the
goals/requirements of the project.
Learning Algorithms

Learning
Thanks Jason. Another superb job. �
ForLinear
Simple sometime now, I Tutorial
Regression have beenfor looking for an authoritative paper on taxonomy, survey and
classification
Machine Learning of ML algorithms with examples. This article is absoulutely a step in that direction
— can be massaged into a taxonomy/survey paper?
Nonetheless as other readers noticed, it is missing some topics: preprocessing including

anomaly detection and feature selection, NLP, genetic algorithms, recommender systems etc.
Wonder if you know of any academic work on the topic. I did not find any in ACM CSUR.
Waiting anxiously! �


Thanks for the suggestion.
I will update the post soon and add more algorithms. I don’t have plans on turning it into a
survey sorry.
REPLY 
ammar September 6, 2017 at 10:18 pm #
Hi Jason
i am work on classification project and i have uncertain rules in the final classifier.
which algorithm you thing it will be more efficient in this case ?
REPLY 
45 of 68 2021-04-11, 9:17 p.m.

Jason Brownlee September 7, 2017 at 12:54 pm #

Try a suite of algorithms and see what works best on your problem?
Picked for you: REPLY 

shalini September 17, 2017 at 12:24 pm #
What algorithms can one use to retrain a Model every day using new data that has user
feedback as to whether the model classified the Response into correct Label, and if not the
correct label is provided by the user. I want to retrain the model using this feedback and going
forward give more weight to the signatures with correct labels.
Learning Algorithms
REPLY 
Jason
Logistic Regression Brownlee
Tutorial September 18, 2017 at 5:43 am #
for Machine
Learning
Many algorithms are updatable. I would recommend testing a suite of algorithms
and see what works on your problem, then pick one that is updatable.
Machine Learning
REPLY 
prasad October 13, 2017 at 4:43 pm #
Hi Jason, I’m a front end developer, but now i would like to learn machine learning. Could you
guide me to learn it in best way?

REPLY 
Jason Brownlee October 14, 2017 at 5:39 am #
where you'll find the Really
My best adviceGood stuff.
is here:
https://machinelearningmastery.com/start-here/#getstarted
I would recommend that you start with Weka:
https://machinelearningmastery.com/start-here/#weka
REPLY 
MBarnett October 19, 2017 at 1:24 pm #
Hmmm.. the download map link is broken….
REPLY 
MBarnett October 19, 2017 at 1:25 pm #
Sorry… spoke too soon.. link of the website went to an error, but just got the download email.
46 of 68 2021-04-11, 9:17 p.m.

Thank you !
REPLY 
Jason Brownlee October 19, 2017 at 3:59 pm #
Picked for you:
Sorry about that, the download works, but the redirect to my thank-you page is broken. I’m
working on it.
REPLY 
viswanath
Supervised yakkala November
and Unsupervised Machine13, 2017 at 8:37 pm #
Learning Algorithms
Support Vector Machines, a supervised ML Algorithm is not there explicitly in the ML Algorithm
mindmap.
Am iLogistic
overlooked?
Regression Tutorial for Machine
Learning
REPLY 
Jason
Simple Linear Brownlee
Regression November
Tutorial for 14, 2017 at 10:08 am #
Machine Learning
Correct. You could add it if you wish.

REPLY 
ravi November 17, 2017 at 5:29 pm #
Which Optimisation Algorithm is best? Genetic Algorithm (or) ABC Algorithm (or) Support Vector
Machine (or) Paricle
Loving swam Optimisation (or) Ant Colony Optimisation.And explain it
the Tutorials?

REPLY 
There is no best algorithm. Try a suite and see what works best for your specific sample of
data and requirements.
REPLY 
Shima December 1, 2017 at 3:07 am #
Hi Jason, very useful classification. Thank you. I wanted to know that HMM and FST are being
considered as machine learning algorithms or not?
REPLY 
47 of 68 2021-04-11, 9:17 p.m.


Yes, sure.
REPLY 
Gaurav Jain December 12, 2017 at 7:11 pm #
Picked for you:
Thanks for such an awesome blog entry!
REPLY 
Learning You’re welcome, I’m glad it helped!
Algorithms

REPLY 
Alex December 27, 2017 at 10:03 pm #
Learning
Awesome work and page Jason! Really a fantastic job you have made. Many many thanks for
yourSimple
effort. Linear Regression Tutorial for
Machine Learning
REPLY 
Jason
December 28, 2017 at 5:22 am #
Thanks Alex!

REPLY 
kawther March 6, 2018 at 10:26 pm #
Many thanks for this tour,
I am working on anomaly detection in networks, which kind of algorithms you may suggest,
Thank you
REPLY 
Try many algorithms and see what works best for your specific data.
48 of 68 2021-04-11, 9:17 p.m.


Divyansh Upman April 13, 2018 at 7:43 pm #
Hi Jason
May I please know if there are any pre-requisites for ML in python

Picked for you:

REPLY 
Jason Brownlee April 14, 2018 at 6:33 am #
Not really other than learning some python.

You can get started with ml in python here:
Learning Algorithms
https://machinelearningmastery.com/start-here/#python

Learning
REPLY 
Blair July 4, 2018 at 7:14 pm #
Hi Jason,
I amMachine Learning
a beginner in programming and I am planning to use Machine Learning algorithm to calibrate sensor
data against reference data (in Python). What are some algorithms that you would suggest?
Thanks!
REPLY 
Try a suite of methods to see what works for your specific prediction problem.
Alsoyou'll
where seefind
this:the Really Good stuff.
https://machinelearningmastery.com/faq/single-faq/what-algorithm-config-should-i-use
49 of 68 2021-04-11, 9:17 p.m.


Raj July 13, 2018 at 10:10 pm #
Thanks Jason for sharing very nice article.
Suppose consider a scenario where a patient took drug (X) and develop five possible side effect (X-a,
Picked
X-b,for
X-c,you:
X-d,X-e).
I need to find
A Tour out the signal
of Machine i.e.Algorithms
Learning causal relationship between drug and its side effect based on few
parameters (like seriousness, suspected etc..).
If the parameter is present, I give score as 1, if not present- score as 0 and -1 for not applicable.
Which algorithm should I use to find the best drug-event relation based on score or any alternative
approach do u prefer.
Learning Algorithms

Learning
This process will help you to work through your problem systematically:
https://machinelearningmastery.com/start-here/#process
Machine Learning
REPLY 
Raj July 14, 2018 at 12:30 pm #
Hi Jason,
I am currently learing Machine Learning and the link that you shown, is 50% done.
I am bit confused which algorithm is suitable to find best event based on few paramaters.
Is it something Hierarchical clustering or decision tree or any other algorithm which u recommend.
>> SEE WHAT'S INSIDE REPLY 

This is a common question that I answer here:

REPLY 
Olanrewaju Sanni September 5, 2018 at 9:07 pm #
Hi Jason,
Is it possible to incorporate machine learning into the heuristic or semi-heuristic algorithms in job
scheduling to improve optimisation? If yes, can you recommend some materials on this to me?
50 of 68 2021-04-11, 9:17 p.m.


Perhaps. I don’t have examples, sorry.
Picked for you:

REPLY 
Pranas September 14, 2018 at 1:20 pm #
Maybe SVM should be slotted into instance based algos class, but where to put RVM … ;-)?

Jason Brownlee September 14, 2018 at 2:36 pm #
Agreed.
Learning
REPLY 
Kazim Raza Talpur September 16, 2018 at 8:46 pm #
Machine Learning
In Machine Learning: what are State-Of-The-Art Machine Learning Algorithms?

Algorithms for Machine Learning REPLY 
Deep learning.

REPLY 
Kazim
where you'll find the Raza
Really Talpur
Good stuff. September 17, 2018 at 3:14 pm #
Dear @Jason
>> SEE WHAT'S INSIDE thanks for your prompt reply,
So which techniques used in Deep Learning ? can you tell me the list?
REPLY 
The most widely use methods are MLPs, CNNs and LSTMs.
REPLY 
JP October 5, 2018 at 10:15 pm #
Nice article. Under semi-supervised learning, there is a statement “the model must learn the
structures ..”. I believe you meant Algorithm here..Even you clarified this in your FAQ, algorithm is the
51 of 68 2021-04-11, 9:17 p.m.

one learns from data to create a prediction model. Agree ?

REPLY 
Picked for you:
Yes.
REPLY 
bilgi bank December 23, 2018 at 1:34 am #
better explained with more detailed sample codes
Learning Algorithms

REPLY 
Learning
Thanks for the suggestion, here are some code examples:

https://machinelearningmastery.com/start-here/#code_algorithms
Machine Learning
REPLY 
Brush December 24, 2018 at 6:43 pm #
I am a newcomer to machine learning. I majored in geosciences at the university and only
studied Python for two months.For students like me, how to prepare for machine learning, and there are
many directions for machine learning, such as computer vision. Natural language processing, deep
learning. Shouldthe
Loving we Tutorials?
study machine learning as a whole, or choose a direction to study.I will graduate in
April next year, how can I find a job related to machine learning. I am a student from China. My English
The
mayMachine Learning
not be very Algorithms
good. EBook
Please don’t is my mistakes.Thanks.
mind

REPLY 
Start small, right here:

https://machinelearningmastery.com/start-here/
52 of 68 2021-04-11, 9:17 p.m.


Cheng January 5, 2019 at 7:21 am #
Hi Jason,
Thanks for the post, how could you categorize ReLU, GANs and RBM in this mindmap? If you put
Picked for you:
activation functions such as logistic regression and linear regression under the Regression section,
should ReLU be there as well? And likewise, should GANs and RBM under Deep Learning?
REPLY 
Learning Relu
Algorithms
is an activation function, a part of a neural net, not an algorithm per se.
GAN is an unsupervised neural network algorithm.

Logistic
RBM is Regression Tutorial
a supervised neuralfornet
Machine
algorithm and also redundant, replaced by MLPs with Relu.
Learning
Logistic regression and linear regression are not activation functions, they are algorithms, solved via
least squares.
Machine Learning
REPLY 
Shabnam November 8, 2019 at 10:27 am #
Algorithms forHi,
Machine Learning
In follow up with this comment, I was wondering if you have any post about RBM. I just found a
reference to “http://videolectures.net/deeplearning2015_lee_boltzmann_machines/” in one of
your posts.
Thanks.

Jason Brownlee November 8, 2019 at 1:50 pm #
No, they are no longer useful:

https://machinelearningmastery.com/faq/single-faq/do-you-have-examples-of-the-restricted-
boltzmann-machine-rbm
Tal February 14, 2020 at 9:39 pm #
Perhaps it’s worth nothing that Deep Belief Networks, which appear in the
above list, are comprised of stacked RBMs.
53 of 68 2021-04-11, 9:17 p.m.


Thanks.
Picked for you:

REPLY 
Anitha Florence January 29, 2019 at 5:07 pm #
Its good to learn machine learning. Can the non- computer Science people learn this. Please tell
way to learn.

Learning Algorithms
REPLY 
Yes, you can start here:
Learning
https://machinelearningmastery.com/start-here/#getstarted
I recommend using Weka that does not need any code:

https://machinelearningmastery.com/start-here/#weka
Machine Learning
Bagging and Random

Nancy V April 11,Forest
2019 atEnsemble
9:40 pm #
REPLY 

Hello Jason your article is crystal clear so that it is useful for everyone and particularly me. I am
beginner to this field and I want to know which algorithm will be suitable for detecting and classifying an
object based on thermal image?? I have a set of data for example such as chair , table , phone etc. and I
want to classify them based on detected thermal image. Could you please give some suggestions ???
REPLY 
>> SEE WHAT'S INSIDE April 12, 2019 at 7:45 am #
I would guess a convolutional neural network (CNN) sound perform well.
REPLY 
Sagar Reddy June 14, 2019 at 3:25 am #
Please, provide the details whether you provide any online classes or institutions to learn in real
time as well as.
REPLY 
Jason Brownlee June 14, 2019 at 6:50 am #
54 of 68 2021-04-11, 9:17 p.m.

No, just ebooks:

https://machinelearningmastery.com/products/
REPLY 
Gyanendra June 20, 2019 at 9:38 am #
Picked for you:
You made it really very clear.
I believe many who work for several years in this field struggle to figure it out what they are doing and
how is it related to big picture of ML and AI.
People also very much confused and consider, NN or DNN is all about AI, where in my view ML
(statistical method,
Supervised NN based method,
and Unsupervised any mathematical algorithmic formulation), Bio/Brain-inspired
Machine
algorithm (genetic
Learning algorithm, SWARM etc), cryptography, cognitive etc. that all are subset of AI. Some
Algorithms
are partially overlapped and some majorly overlapped with each other.

Learning
REPLY 
Jason Brownlee June 20, 2019 at 1:58 pm #
Thanks.
Simple Linear Perhaps
Regression you’refor
Tutorial right.
Machine Learning
REPLY 
sandipan
Bagging sarkar
and Random Forest
July Ensemble
7, 2019 at 3:56 am #
hello jason,
This article was wonderful.But my question is how can I learn the algorithms in a systematic way in order
to clear the interview from the technical part.I hope you do understand.Thanks

Sorry, I don’t know about interviews.
REPLY 
guhan July 24, 2019 at 10:56 pm #
hello Jason,
Is there any possibility to use reinforcement learning on regression?
55 of 68 2021-04-11, 9:17 p.m.


Probably not, the method is more appropriate for an agent in an environment.
Picked for you:

REPLY 
Machine learning September 7, 2019 at 4:15 pm #
I am feeling motivated and now work harder to start the career in Machine-Learning, hope will
get similar success. Thanks for sharing your Machine-Learning experience.

Learning Algorithms
REPLY 
Thanks. It’s great to have you here as part of the ML Mastery community.
Learning
Simple Linear Regression Tutorial for REPLY 

NguWah September 14, 2019 at 4:15 pm #
Machine Learning
Hi Jason, I am ambiguous about is MLP is machine learning or deep learning?

REPLY 
This map help:

http://machinelearningmastery.com/neural-networks-crash-course/
REPLY 
>> SEE WHAT'S
OctoberINSIDE
1, 2019 at 8:51 pm #
Hi Jason,
Can you help me how many types of machine learning algorithms can be used in
1) Semi- Supervised learning.

2) Reinforcement learning.
In general i need to know the name of algorithms in both cases.
Regards
Ashish
REPLY 
56 of 68 2021-04-11, 9:17 p.m.


Perhaps start here:
https://en.wikipedia.org/wiki/Semi-supervised_learning
And here:
Picked for you:
https://en.wikipedia.org/wiki/Reinforcement_learning
REPLY 
J Ravindra October 4, 2019 at 2:51 am #
Supervised
Jason and Unsupervised Machine
Is it possible Algorithms
Learning to know the completely of deep and machine learning algorithms.

Learning Jason Brownlee October 4, 2019 at 5:46 am # REPLY 
No one knows any field of study completely. In fact, no one knows anything completely.
We all just
Machine learn enough to get good enough results, then move on.
Learning

REPLY 
Bob November
Algorithms 10, 2019
for Machine at 9:41 pm #
Learning
Please write on reinforcement learning! I’ll buy your book on day one I promise
The Machine Learning

Jason Algorithms
BrownleeEBook
Novemberis 11, 2019 at 6:07 am # REPLY 

Thanks for the suggestion and your support.
REPLY 
Emilia Jazz November 12, 2019 at 2:28 am #
Thanks for updating the algorithms. Very useful.
REPLY 
Thanks.
REPLY 
57 of 68 2021-04-11, 9:17 p.m.

Never miss Chen

a tutorial:
Mei November 16, 2019 at 6:51 pm #
you’re my spiritual supervisor !
Picked for you:

REPLY 
Thanks!

Mario November 20, 2019 at 6:46 am #
This is a very interesting, but more important amusing post.

It is provoking, extremely well written and easy to read, after reading it a sort of road map has formed
Learning
which is fantastic for a newbi just like yours truly.
Only one thing troubles me a little bit, a tendency that may lead many newcomers to think that math is
Simple Linear
unimporttant, Regression
machine Tutorial
learning is notforeasy at all, it requieres lots and lots of mathematics and
Machine Learning
deisregarding them buy telling the story of someone that came, understood and jumped to program lots
of code but hiding the fact that she was alrady an engineer, or mathematician, or statistican, etc .
Any Bagging
way, Jason, I have just
and Random been
Forest converted and will follow your blog very closely since I am convinced
Ensemble
yourAlgorithms
writtings will
for be very helpful
Machine for a chnage of carrer from theoretical physicist to data scientist
Learning

Jason Brownlee November 20, 2019 at 1:49 pm # REPLY 

Thanks Mario.

REPLY 
doiki November 29, 2019 at 10:44 am #
and generally speaking, compare these algorithms. I would add HDT, Jackknife regression,
density estimation, attribution modeling (to optimize marketing mix), linkage (in fraud detection),
indexation (to create taxonomies or for clustering large data sets consisting of text), bucketisation, and
time series algorithms.
REPLY 
Jason Brownlee November 29, 2019 at 1:42 pm #
Very nice, thanks!
58 of 68 2021-04-11, 9:17 p.m.


Sunardi Hafizhuddin December 10, 2019 at 4:26 pm #
Machine learning is currently one of the major methods of artificial intelligence. Logistic
regression, Random Forest and Deep Learning are three common machine learning methods. Please
explain how these three methods perform Supervised Learning.
Picked for you:
REPLY 
I have tutorials that explain all three, you can search for them on the blog – use the search
box.
Learning Algorithms

REPLY 
Rafael January 8, 2020 at 12:28 pm #
Learning
Which “learning style” are “Deep Learning Algorithms” in?
Simple Linear
For example, Regression
which learningTutorial
style isfor
CNN algorithm in?
Machine Learning

Jason Brownlee January 8, 2020 at 2:25 pm #
Supervised learning.

REPLY 
Jisna Antony January 28, 2020 at 12:19 am #
Can you suggest a deep learning algorithm that learns the relationships existing between
consecutive(adjacent) data points.
For example predicting a structure by identifying the relationships among 3D coordinates?
REPLY 
Perhaps manifold learning:

https://scikit-learn.org/stable/modules/manifold.html
59 of 68 2021-04-11, 9:17 p.m.


Oliver Smith February 5, 2020 at 6:17 pm #
Should XGBoost algorithm be a part of Ensemble Algorithms?
Picked for you:

REPLY 
Yes!

Hacke February 20, 2020 at 7:03 am #
Hi, Thanks for nice information.

Learning
REPLY 
Machine Learning
You’re welcome!

Algorithms for Machine Learning REPLY 
JG April 8, 2020 at 5:07 pm #
Great summary and broad perspective of artificial intelligence techniques to apply in order to
develop algorithms using machine learning! Thanks to Mr. Jason Brownlee !!

REPLY 
Thanks!
REPLY 
Skylar May 8, 2020 at 3:12 pm #
Hi Jason,
Very nice summary! One confusion for me: I can understand that knn belongs to instance-based
algorithms, but why SVM also belongs to this category? Any material you would suggest to explain this?
Thank you!
REPLY 
60 of 68 2021-04-11, 9:17 p.m.

Jason Brownlee May 8, 2020 at 3:57 pm #

SVM finds support vectors – points in the feature space – used to define the boundary between the
classes. E.g. it’s probably an instance-based method given this understanding.
Picked for you:

REPLY 
Skylar
A Tour of Machine May 9, 2020 at 3:56 am #
Learning Algorithms
I see, then it is much clear for me, thank you Jason!

Learning Algorithms
REPLY 
You’re welcome.
Learning
REPLY 
Skylar May 9, 2020 at 3:38 pm #
Machine Learning
Hi Jason,
I read your post again, because I really think it is a great classification way for different ML algorithms! I
haveBagging
one question about Forest
and Random unsupervised
Ensemble ML when I was reading again your post, maybe you can help?
Which algorithms that you listed in your post belong to unsupervised ML? Clustering? In principle, we
don’t need to split the data to training and test dataset for the model performance evaluation for
unsupervised ML algorithms, instead, we need to input our entire dataset for training, right? If so, how
can weLoving
know whether we should use this unsupervised method or not?
the Tutorials?
Many
The thanks!
Machine Learning Algorithms EBook is

REPLY 
Good question, this framework will help you determine if you need supervised learning:
http://machinelearningmastery.com/how-to-define-your-machine-learning-problem/
REPLY 
Skylar May 11, 2020 at 8:39 am #
Thank you Jason, this post helps a lot!
REPLY 
Jason Brownlee May 11, 2020 at 1:35 pm #
61 of 68 2021-04-11, 9:17 p.m.

You’re welcome.
REPLY 
Tushar June 19, 2020 at 4:17 pm #
Picked for you:
This was really awesome….
Kudos to you Jason….
What are the subsets of ML alongside of DL???
DL is subset of ML right???? Like that what are the other subsets??? Plz help…�
�

Learning Algorithms
REPLY 
Jason Brownlee June 20, 2020 at 6:07 am #
Yes, deep Tutorial

Logistic Regression learningforisMachine
a subset of machine learning. All of the algorithms listed above sit
along side deep learning.
Learning

REPLY 
Ropo
Machine August 3, 2020 at 7:32 pm #
Learning
This is so helpful, thank you!

Bagging
I have and Random Forest Ensemble
a question:
I want to estimate the elastic modulus of an aluminium metal plate with density, resonant frequencies and
plate dimensions as inputs. And the elastic modulus as the output.
which is the best algorithm to do so, and where can I get the algorithm to optimize?
thanks in advance!

Good question, I answer it here:

REPLY 
Alaaedi August 15, 2020 at 4:19 pm #
Hi sir, please which Reinforcement learning algorithms fit and perfect to solve continuous
computational complexity?
REPLY 
62 of 68 2021-04-11, 9:17 p.m.


Sorry, I don’t have tutorials on RL, I may write about the topic in the future:
https://machinelearningmastery.com/faq/single-faq/do-you-have-tutorials-on-deep-reinforcement-
learning
Picked for you:

REPLY 
dscrib1221 August 21, 2020 at 1:59 am #
Are real analysis and measure theory necessary to know to engage in machine/deep learning?
Learning Algorithms
REPLY 
No.
Learning
Is theory of computation, type theory, or compiler theory required to write code / be a developer?

Machine Learning
REPLY 
Ramya T August 26, 2020 at 10:12 pm #
Bagging and Random

Thanks a ton forForest Ensemble
this post!! Really helped!!
REPLY 
You’re welcome.

Aashish September 6, 2020 at 5:36 am #
Excellent post! To the point! No unnecessary theory. Great going
Regards,
Aashish
Munich, Germany
REPLY 
Thanks!
63 of 68 2021-04-11, 9:17 p.m.


java November 24, 2020 at 6:18 am #
This was really awesome….

Kudos to you Jason….
What are the subsets of ML alongside of DL???
Picked for you:
DL is subset of ML right???? Like that what are the other subsets??? Plz help
REPLY 
Thanks.
Learning Algorithms
DL is a subset of ML:
https://machinelearningmastery.com/faq/single-faq/how-are-ml-and-deep-learning-related
Learning
REPLY 
Elena February 15, 2021 at 6:43 pm #
Machine Learning
Awesome insight for Beginner like me in ML. I am starting with the Andrew’s ML course at
coursera. I have programming experience of more than 10 years, I was wondering what does a machine
learning expert exactly do after solving a complex ML problem? I mean the algorithm is optimized and
Baggingisand
hypothesis Randomthe
predicting Forest Ensemble
correct results, then what will the expert do? Gets fired �
REPLY 
A final model
The Machine Learning is fit EBook
Algorithms on all data
is ready for making predictions on new cases:
https://machinelearningmastery.com/train-final-machine-learning-model/
The model is deployed and monitored over time, like any software system.
REPLY 
S Mahesh Acharya March 15, 2021 at 12:11 am #
Tour of machine learning is best for beginner students like me.

Thank you
REPLY 
Thanks.
64 of 68 2021-04-11, 9:17 p.m.


Filippo March 18, 2021 at 11:45 pm #
A fantastic overwiew!
Compliments
Picked for you:

REPLY 
Thanks!

Learning Algorithms
REPLY 
Sarker Asish March 20, 2021 at 8:25 am #
Hi Jason,
Learning
Would you please provide some online machine learning classification tutorial for real time data?

Yes, perhaps start with the examples here:

https://machinelearningmastery.com/start-here/
REPLY 
Reesethe
Loving Evans March 22, 2021 at 5:02 pm #
Tutorials?
Shared
The Machine a nice
Learning post about
Algorithms a journey
EBook is of machine learning algorithms.

REPLY 
Thanks!
REPLY 
Divya March 25, 2021 at 4:39 am #
Thank you for sharing this valuable content.

I love it. it’s very valuable.
REPLY 
65 of 68 2021-04-11, 9:17 p.m.

You’re welcome!
REPLY 
Hannah Kuijpers March 26, 2021 at 6:00 pm #
Picked for you:
Hey Jason!
Thank A Tour
you of
soMachine
much for Learning Algorithms
these great tips.
I will definitely follow these instructions. Keep writing.
Thanks.
Hannah
Learning Algorithms
REPLY 
Jason Brownlee
Logistic Regression Tutorial forMarch 29, 2021 at 5:50 am #
Machine
Learning
You’re welocme.

Machine Learning
REPLY 
Alex Rahlis March 27, 2021 at 8:26 pm #
Hi Jason, thanks for a great summary,

I amAlgorithms
new to ML,fortrying to understand
Machine Learning groups of algorithms.
I have stumbled upon a similar summary here
https://www.r-bloggers.com/2019/07/101-machine-learning-algorithms-for-data-science-with-cheat-
sheets/
and this list has a separate group called
“Classification
The Algorithms”
Machine Learning that EBook
Algorithms you don’t
is have, and he put there algorithms from multiple your group, is
where
this you'll find
an error? the Really
it looks Good
like he stuff. “Algorithm Type” and “Problem Type” here, am I right?
confused
REPLY 
There’s no best way. Perhaps you can ask the author themselves about their approach to
grouping algorithms.
REPLY 
Anurag Singh April 8, 2021 at 6:38 pm #
Nice explained article
REPLY 
66 of 68 2021-04-11, 9:17 p.m.


Thanks!
Picked for you:
LeaveA Tour
a Reply
of Machine Learning Algorithms

Learning Algorithms

Learning
Name (required)
Machine Learning
Email (will not be published) (required)

Website
Loving
SUBMIT the Tutorials?
COMMENT

Welcome!
Jason Brownlee PhD
and I help developers get results with machine learning.
Read more
67 of 68 2021-04-11, 9:17 p.m.

Picked for you:

Learning Algorithms

Learning

Machine Learning


© 2021 Machine Learning Mastery Pty. Ltd. All Rights Reserved.
LinkedIn | Twitter | Facebook | Newsletter | RSS
Privacy | Disclaimer | Terms | Contact | Sitemap | Search
68 of 68 2021-04-11, 9:17 p.m.

A Tour of Machine Learning Algorithms

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

A Tour of Machine Learning Algorithms

Uploaded by

Copyright:

Available Formats

A Tour of Machine Learning Algorithms https://machinelearningmastery.

Picked for you: