Unit 1:

Supervised Learning (Regression/Classification)

Basic methods: Distance-based methods, Nearest-Neighbours, Decision

Trees, Naive Bayes

Linear models: Linear Regression, Logistic Regression, Generalized

Linear Models

Support Vector Machines, Nonlinearity and Kernel Methods

Beyond Binary Classification: Multi-class/Structured Outputs, Ranking

----------------------------------------------------------------------------------------------------------------------

Algorithm:

Data Program

Computer

Output

Machine Learning:

Data Output

Computer

Program
Tom Mitchell in his book Machine Learning provides a definition:

A computer program is said to learn from experience E with respect to some class of tasks T and
performance measure P, if its performance at tasks in T, as measured by P, improves with
experience E.

Experience E- Data

Task T- Prediction / Classification / Acting

Measure of improvement - P

Learning System:
Here we feed experience, some background knowledge to the learner.

Then it creates a model.

Reasoner gets the problem then solve it.

Steps for creating a learner

Choose training experience -> Features

Choose the target function (function need to be learned)

Choose how to represent the target function (appropriate class of functions for different features)

Choose a learning algorithm to insert the target function.

 If we choose a Rich Representation then we can solve a complex function but it may be more
difficult to learn.
 Class of functions are called Hypothesis Language
Supervised Learning

Discrete- Whether it will rain or not tomorrow.

According to the symptoms predict whether a patient has a particular disease.

Continuous- Price of a house based on various constrains.

Training Inputs Output

Instances X1 X2 X3 … Xn Y
I1 a1 a2 a3 … an Y1
I2 b1 b2 b3 … bn Y2
I3 c1 c2 c3 … cn Y3
Microsoft Corporation – name of a company

First word- Microsoft

Later word- Corporation

Acquired XYZ

Outside- Acquired (not a company name)

XYZ- Company Name

In supervised learning, the goal is to learn the mapping (the rules) between a set of inputs & outputs.

For example, the inputs could be the weather forecast, and the outputs would be the visitors to the
beach. The goal in supervised learning would be to learn the mapping that describes the
relationship between temperature and number of beach visitors.

Example labelled data is provided of past input and output pairs during the learning process to teach
the model how it should behave, hence, ‘supervised’ learning. For the beach example, new inputs can
then be fed in of forecast temperature and the Machine learning algorithm will then output a future
prediction for the number of visitors.

Being able to adapt to new inputs and make predictions is the crucial generalisation part of machine
learning. In training, we want to maximise generalisation, so the supervised model defines the real
‘general’ underlying relationship. If the model is over-trained, we cause over-fitting to the examples
used and the model would be unable to adapt to new, previously unseen inputs.

A side effect to be aware of in supervised learning that the supervision we provide introduces bias to
the learning. The model can only be imitating exactly what it was shown, so it is very important to
show it reliable, unbiased examples. Also, supervised learning usually requires a lot of data before it
learns. Obtaining enough reliably labelled data is often the hardest and most expensive part of using
supervised learning. (Hence why data has been called the new oil!)

The output from a supervised Machine Learning model could be a category from a finite set e.g [low,
medium, high] for the number of visitors to the beach:

Input [temperature=20] -> Model -> Output = [visitors=high]

When this is the case, it’s deciding how to classify the input, and so is known as classification.

Alternatively, the output could be a real-world scalar (output a number):

Input [temperature=20] -> Model -> Output = [visitors=300]

When this is the case, it is known as regression.

Classification
Classification is used to group the similar data points into different sections in order to classify them.
Machine Learning is used to find the rules that explain how to separate the different data points.

But how are the magical rules created? Well, there are multiple ways to discover the rules. They all
focus on using data and answers to discover rules that linearly separate data points.

Linear separability is a key concept in machine learning. All that linear separability means is ‘can the
different data points be separated by a line?’. So put simply, classification approaches try to find the
best way to separate data points with a line.

The lines drawn between classes are known as the decision boundaries. The entire area that is chosen
to define a class is known as the decision surface. The decision surface defines that if a data point falls
within its boundaries, it will be assigned a certain class.

Machine learning for classification

 Logistic Regression (LR)

 K-Nearest Neighbours (K-NN)
 Support Vector Machine (SVM)
 Kernel SVM
 Naïve Bayes
 Decision Tree Classification
 Random Forest Classification
It’s considered that methods like SVM and random forests work best. Keep in mind that there are no
one-size-fits-all rules, and they probably won’t operate properly for your task.

Use cases of Classification Algorithms

Classification algorithms can be used in different places. Below are some popular use cases of
Classification Algorithms:

1. Email Spam Detection

2. Speech Recognition
3. Identifications of Cancer tumor cells
4. Drugs Classification
5. Biometric Identification, etc.

Regression
Regression is another form of supervised learning. The difference between classification and regression
is that regression outputs a number rather than a class. Therefore, regression is useful when predicting
number based problems like stock market prices, the temperature for a given day, or the probability of
an event.

Examples

Regression is used in financial trading to find the patterns in stocks and other assets to decide when to
buy/sell and make a profit.

For classification, it is already being used to classify if an email you receive is spam.

Both the classification and regression supervised learning techniques can be extended to much more
complex tasks. For example, tasks involving speech and audio, image classification, object detection
and chat bots are some examples.

A recent example shown below uses a model trained with supervised learning to realistically fake
videos of people talking.

You might be wondering how this complex image based task relates to classification or regression.
Well, it comes back to everything in the world, even complex phenomenon, being fundamentally
described with math and numbers. In this example, a neural network is still only outputting numbers
like in regression. But in this example the numbers are the numerical 3d coordinate values of a facial
mesh.
Machine learning for regression

Below is a short list of machine learning methods (having their own advantages and disadvantages)
that can be used for regression

 Liner regression
 Polynomial regression
 Ridge regression
 Decision trees
 SVR (Support Vector Regression)
 Random forest

You can find out the detailed explanation of each method here.

Difference between Regression and Classification

Regression Algorithm Classification Algorithm

In Regression, the output variable must be of In Classification, the output variable must be a
continuous nature or real value. discrete value.

The task of the regression algorithm is to map The task of the classification algorithm is to
the input value (x) with the continuous output map the input value(x) with the discrete output
variable(y). variable(y).

Regression Algorithms are used with Classification Algorithms are used with discrete
continuous data. data.

In Regression, we try to find the best fit line, In Classification, we try to find the decision
which can predict the output more accurately. boundary, which can divide the dataset into
different classes.

Regression algorithms can be used to solve the Classification Algorithms can be used to solve
regression problems such as Weather classification problems such as Identification of
Prediction, House price prediction, etc. spam emails, Speech Recognition, Identification
of cancer cells, etc.

The regression Algorithm can be further The Classification algorithms can be divided
divided into Linear and Non-linear Regression. into Binary Classifier and Multi-class Classifier.