Lecture .......... Types of Learning in Machine Learning and Machine Learning Cycle

‫َّر‬ ‫َّر‬
‫ِبْس اللِه ال ْحٰمِن ال ِحْي‬

‫ِم‬ ‫ِم‬
Data Mining
Types of Learning & Machine
Learning Cycle
Types of Learning
1 Supervised Learning
Three main
types of 2 Unsupervised Learning
learning are
3 Semi-supervised Learning
Dr. Rao Muhammad Adeel Nawab 3

Supervised Learning
Definition
In Supervised Learning, a Machine Learning

Algorithm learns from Annotated Data
Annotated Data means that for all Training Examples,
Output is associated with Inputs

Types of Supervised Learning
Two main 1 Classification

types of
Supervised
Learning are 2 Regression

Types of Supervised Learning Cont...
Classification
Definition
In Classification, the Output is Categorical (or Discrete)

Example
Task: Gender Identification – Annotated Data
Instance No. Input Output
Height Weight Hair Length Beard Scarf Gender
d1 180.3 196 Bald Yes No Male
d2 170.0 120 Long No No Female
d3 178.5 200 Short No No Male
d4 163.4 110 Medium No Yes Female
d5 175.2 220 Short Yes No Male
d6 165.0 150 Medium No Yes Female
d7 179.1 185 Long Yes No Male
d8 160.5 130 Short No No Female
d9 177.8 160 Bald No No Male
d10 161.1 100 Medium No No Female
Regression
Definition
In Regression, the Output is Numeric (or Continuous)

Example
Task: House Price Prediction– Annotated Data
Instance Input Output
No. House Price
rm age dis rad Istat
d1 6.575 65.2 4.0900 1 4.98 24.0
d2 6.421 78.9 4.9671 2 9.14 21.6
d3 6.998 45.8 6.0622 3 2.94 33.4
d4 7.147 54.2 6.0622 3 5.33 36.2
d5 6.012 66.6 5.5605 5 12.43 22.9
Unsupervised Learning
Definition
In Unsupervised Learning, a Machine Learning

Algorithm learns from Unannotated Data
Unannotated Data means that for all Training Examples,
Output is not associated with Inputs

Unsupervised Learning Cont...
Strengths Weaknesses
You can easily and quickly Learning may not be

collect large amount of accurate, since the quality
Unannotated Data of Data is low because it is
unannotated

Semi-Supervised Learning
Definition
In Semi-supervised Learning, a Machine Learning

Algorithm learns from Semi-annotated Data
Semi-annotated Data means that only for some Training
Examples, Output is associated with Inputs

Semi-Supervised Learning Cont...
Strengths Weaknesses
You can quickly collect Learning may not be

large amount of Semi- accurate, since the quality
annotated Data of Data is low because all
Training Examples are not
annotated

Popular and Widely Used Machine Learning Algorithms for
Different Types of Learning
ML Algorithms for Supervised Learning
Naïve Bayes Gradient Boosting

Random Forest Multi-layer Perceptron
Support Vector Machine K-Nearest Neighbors
Logistic Regression

Different Types of Learning Cont...
ML Algorithms for Classification
Naïve Bayes Gradient Boosting

Random Forest Classifier Multi-layer Perceptron
Support Vector Machine K-Nearest Neighbors
Classifier
Logistic Regression

ML Algorithms for Regression
Random Forest Regressor

Support Vector Machine Regressor
Logistic Regression
Linear Regression

ML Algorithms for Unsupervised Learning

K-means Clustering Algorithm DBSCAN – Density-Based Spatial
Clustering of Applications with
Noise
Agglomerative Hierarchical EM using GMM – Expectation-
Clustering Algorithm Maximization (EM) Clustering
using Gaussian Mixture Models
(GMM)
Mean-Shift Clustering Algorithm

ML Algorithms for Semi-supervised Learning
Label Propagation

Machine Learning
Cycle
Machine Learning Cycle
Training /
Feedback
Learning
Phase
Phase
Testing /
Application
Phase Evaluation
Phase

Balanced Data vs Unbalanced Data
For more accurate learning, it is important to have

Balanced Data
Balanced Data
For each class, the Dataset must contain the same

number of instances

Example 01 – Balanced Data vs Unbalanced Data
Machine Learning Problem: Gender Identification
No. of Classes
Class 01 = Male Class 02 = Female
Dataset Size
300 Instances

Example 01 –
Balanced Data vs Unbalanced Data Cont...
Examples – Balance &Unbalanced Datasets
Unbalanced Dataset 01 Balanced Dataset 02
Male = 50, Female = 250 Male = 150, Female = 150
Note that this Dataset is Note that this Dataset is

highly unbalanced Balanced
Reason for Unbalanced Datasets: For Male and Female

classes, the number of instances in not same

Data Split
Problem A Possible Solution
For both Training Phase Split available Data into

and Testing Phase, we  Train Data (or Train set)
 Test Data (or Test set)
need
 Data Use Train Data in the
Training Phase and Test
Data in the Testing Phase

Data Split Cont...
Standard Approach for Data Split
Train Set Use 2 / 3 (67%) of Data

Test Set Use 1 / 3 (33%) of Data

Data Split Cont...
Two main approaches to Data split:

 Random Data Split
 Class Balanced Data Split
Random Data Split Class Balanced Data Split
In this approach, the Data In this approach, the Data

Distribution (for each class) in the Distribution (for each class) in the
original Dataset is not followed original Dataset is strictly followed
while splitting Data while splitting Data

Example 01 – Data Split
Machine Learning Problem: Gender Identification
No. of Classes
Class 01 = Male Class 02 = Female
Original Dataset Size

600 Instances

Example 01 – Data Split Cont...
Data Distribution
Male = 300 (50%) Female = 300 (50%)
Train-Test Split Ratio

67%-33%

Random Data Split Approach
Train Set Test Set
400 instances 200 instances

(67% of Original Dataset) (33% of Original Dataset)
Male = 250 Male = 150
Female = 150 Female = 50

Class Balanced Data Split Approach
Train Set Test Set
400 instances 200 instances

(67% of Original Dataset) (33% of Original Dataset)
Male = 200 Male = 100
Female = 200 Female = 100

Machine Learning Cycle
Recall
Training /
Feedback
Learning
Phase
Phase
Testing /
Application
Phase Evaluation
Phase

Training / Learning Phase
Definition
Use Training Data to build a Purpose

Model
Build a Model (or Intelligent
Program) from Training Data,
to make predictions on
unseen Data

Testing / Evaluation Phase
Definition
Use Test Data to evaluate the

performance of Model (build Purpose
in the Training Phase)
Judge how good Model has
Learned from the Training
Data using Evaluation
Measure

Standard Evaluation Measures
Classification - Standard Evaluation Measures
Baseline Accuracy F1 – Measure

Accuracy Area Under the Curve (AUC)
Precision
Recall

Standard Evaluation Measures Cont...
Regression - Standard Evaluation Measures

Mean Absolute Error (MAE)
Mean Squared Error (MSE)
Root Mean Squared Error (RMSE)
R² or Coefficient of Determination
Adjusted R²

Summary - Training and Testing Phases
Recall the Equation
𝑫𝒂𝒕𝒂=𝑴𝒐𝒅𝒆𝒍+𝑬𝒓𝒓𝒐𝒓
 In Training Phase, Model is build using the Training Data
 In Testing Phase, Test Data is used to check the Error in the
Model

Application Phase
Definition
Deploy the Model in the real

world Purpose
Use the Model to make

predictions on future unseen
Data for a range of Real-world
Applications

Feedback Phase
Definition
Take Feedback from Users and

Domain Experts on your Purpose
deployed Model
To further improve the

deployed Model

Machine Learning
– Training Regimes
Training Regime
Definition Purpose
A systematic way in which When we learn (or train)

Training Data is used by a systematically, the quality
Machine Learning of Training increases
Algorithm to learn from it

Types of Training Regimes
1 Batch Method
Some of the
main types of
Training 2 Incremental Method
Regimes are
3 On-line Method

Types of Training Regimes Cont...
Batch Method
In this method, all Training Examples are available

and used all at once to build the Model (or
Hypothesis h)

Incremental Method
In the method, one member (Training Example) of

the Training Data is selected at a time and used to
modify the current Hypothesis (h)

On-line Method
If Training Examples become available one at a time

and are used as they become available, the Training
Regime is called On-line Method
Example
 A robot which is learning a Hypothesis (h) from sensory
inputs which controls its actions (and hence determines
its future sensory inputs)

Lecture .......... Types of Learning in Machine Learning and Machine Learning Cycle

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Lecture .......... Types of Learning in Machine Learning and Machine Learning Cycle

Uploaded by

Copyright:

Available Formats

‫َّر‬ ‫َّر‬

‫ِبْس اللِه ال ْحٰمِن ال ِحْي‬

Dr. Rao Muhammad Adeel Nawab 3

In Supervised Learning, a Machine Learning

Dr. Rao Muhammad Adeel Nawab 4

Two main 1 Classification

Dr. Rao Muhammad Adeel Nawab 5

Dr. Rao Muhammad Adeel Nawab 6

Dr. Rao Muhammad Adeel Nawab 8

In Unsupervised Learning, a Machine Learning

Dr. Rao Muhammad Adeel Nawab 10

You can easily and quickly Learning may not be

Dr. Rao Muhammad Adeel Nawab 11

In Semi-supervised Learning, a Machine Learning

Dr. Rao Muhammad Adeel Nawab 12

You can quickly collect Learning may not be

Dr. Rao Muhammad Adeel Nawab 13

ML Algorithms for Supervised Learning

Naïve Bayes Gradient Boosting

Dr. Rao Muhammad Adeel Nawab 14

ML Algorithms for Classification

Naïve Bayes Gradient Boosting

Dr. Rao Muhammad Adeel Nawab 15

ML Algorithms for Regression

Random Forest Regressor

Dr. Rao Muhammad Adeel Nawab 16

ML Algorithms for Unsupervised Learning

Mean-Shift Clustering Algorithm

Dr. Rao Muhammad Adeel Nawab 17

ML Algorithms for Semi-supervised Learning

Dr. Rao Muhammad Adeel Nawab 18

Dr. Rao Muhammad Adeel Nawab 20

For more accurate learning, it is important to have

For each class, the Dataset must contain the same

Dr. Rao Muhammad Adeel Nawab 21

Machine Learning Problem: Gender Identification

Dr. Rao Muhammad Adeel Nawab 22

Examples – Balance &Unbalanced Datasets

Unbalanced Dataset 01 Balanced Dataset 02

Male = 50, Female = 250 Male = 150, Female = 150

Note that this Dataset is Note that this Dataset is

Reason for Unbalanced Datasets: For Male and Female

Dr. Rao Muhammad Adeel Nawab 23

Problem A Possible Solution

For both Training Phase Split available Data into

Dr. Rao Muhammad Adeel Nawab 24

Standard Approach for Data Split

Train Set Use 2 / 3 (67%) of Data

Dr. Rao Muhammad Adeel Nawab 25

Two main approaches to Data split:

Random Data Split Class Balanced Data Split

In this approach, the Data In this approach, the Data

Dr. Rao Muhammad Adeel Nawab 26

Machine Learning Problem: Gender Identification

Original Dataset Size

Dr. Rao Muhammad Adeel Nawab 27

Train-Test Split Ratio

Dr. Rao Muhammad Adeel Nawab 28

Random Data Split Approach

Train Set Test Set

400 instances 200 instances

Dr. Rao Muhammad Adeel Nawab 29