How To Calculate Precision, Recall, and F-Measure For Imbalanced Classification

6/17/2020 How to Calculate Precision, Recall, and F-Measure for Imbalanced Classification
 Navigation
Click to Take the FREE Imbalanced Classification Crash-Course
Search... 
How to Calculate Precision, Recall, and F-Measure

for Imbalanced Classification
by Jason Brownlee on January 3, 2020 in Imbalanced Classification
Tweet Share Share
Last Updated on February 6, 2020
Classification accuracy is the total number of correct predictions divided by the total number of predictions
made for a dataset.
As a performance measure, accuracy is inappropriate for imbalanced classification problems.
The main reason is that the overwhelming number of examples from the majority class (or classes) will
overwhelm the number of examples in the minority class, meaning that even unskillful models can achieve
accuracy scores of 90 percent, or 99 percent, depending on how severe the class imbalance happens to
be.
An alternative to using classification accuracy is to use precision and recall metrics.
In this tutorial, you will discover how to calculate and develop an intuition for precision and recall for
imbalanced classification.
After completing this tutorial, you will know:
Precision quantifies the number of positive class predictions that actually belong to the positive class.
Recall quantifies the number of positive class predictions made out of all positive examples in the
dataset.
F-Measure provides a single score that balances both the concerns of precision and recall in one
number.
Discover SMOTE, one-class classification, cost-sensitive learning, threshold moving, and much more in my
new book, with 30 step-by-step tutorials and full Python source code.
Start Machine Learning
https://machinelearningmastery.com/precision-recall-and-f-measure-for-imbalanced-classification/ 1/19
Let’s get started.
Update Jan/2020: Improved language about the objective of precision and recall. Fixed typos about
what precision and recall seek to minimize (thanks for the comments!).
Update Feb/2020: Fixed typo in variable name for recall and f1.
Start Machine Learning ×

You can master applied Machine Learning
without math or fancy degrees.
Find out how in this free and practical course.
Email Address
START MY EMAIL COURSE
How to Calculate Precision, Recall, and F-Measure for Imbalanced Classification

Photo by Waldemar Merger, some rights reserved.
Tutorial Overview
This tutorial is divided into five parts; they are:
1. Confusion Matrix for Imbalanced Classification

2. Precision for Imbalanced Classification
3. Recall for Imbalanced Classification
4. Precision vs. Recall for Imbalanced Classification
5. F-Measure for Imbalanced Classification
Confusion Matrix for Imbalanced Classification

Before we dive into precision and recall, it is important to review the confusion matrix.
For imbalanced classification problems, the majority class is typically referred to as the negative outcome
(e.g. such as “no change” or “negative test result“), and the minority class is typically referred to as the
positive outcome (e.g. “change” or “positive test result”).
The confusion matrix provides more insight into not only the performance of a predictive model, but also
which classes are being predicted correctly, which incorrectly, and what type of errors are being made.
The simplest confusion matrix is for a two-class classification problem, with negative (class 0) and positive
(class 1) classes.
In this type of confusion matrix, each cell in the table has a specific and well-understood name,
summarized as follows:
1 | Positive Prediction | Negative Prediction

2 Positive Class | True Positive (TP) | False Negative (FN) Start Machine Learning ×
3 Negative Class | False Positive (FP) | True Negative (TN)
The precision and recall metrics are defined in terms of the cells math
without in theorconfusion matrix, specifically terms
fancy degrees.
like true positives and false negatives. Find out how in this free and practical course.
Now that we have brushed up on the confusion matrix, let’s take a closer look at the precision metric.
Email Address
Precision for Imbalanced Classification

Precision is a metric that quantifies the number of correct positive predictions made.
Precision, therefore, calculates the accuracy for the minority class.
It is calculated as the ratio of correctly predicted positive examples divided by the total number of positive
examples that were predicted.
Precision evaluates the fraction of correct classified instances among the ones classified as
 positive …
— Page 52, Learning from Imbalanced Data Sets, 2018.
Precision for Binary Classification

In an imbalanced classification problem with two classes, precision is calculated as the number of true
positives divided by the total number of true positives and false positives.
Precision = TruePositives / (TruePositives + FalsePositives)
The result is a value between 0.0 for no precision and 1.0 for full or perfect precision.
Let’s make this calculation concrete with some examples.

Consider a dataset with a 1:100 minority to majority ratio, with 100 minority examples and 10,000 majority
class examples.
A model makes predictions and predicts 120 examples as belonging to the minority class, 90 of which are
correct, and 30 of which are incorrect.
The precision for this model is calculated as:

Precision = 90 / (90 + 30)
Precision = 90 / 120
Precision = 0.75
The result is a precision of 0.75, which is a reasonable value but not outstanding.
You can see that precision is simply the ratio of correct positive
You canpredictions outMachine
master applied of all positive
Learningpredictions
made, or the accuracy of minority class predictions. without math or fancy degrees.
Consider the same dataset, where a model predicts 50 examples belonging to the minority class, 45 of
which are true positives and five of which are false positives. We can calculate the precision for this model
Email Address
as follows:

Precision = 45 / (45 + 5)
Precision = 45 / 50
Precision = 0.90
In this case, although the model predicted far fewer examples as belonging to the minority class, the ratio
of correct positive examples is much better.
This highlights that although precision is useful, it does not tell the whole story. It does not comment on how
many real positive class examples were predicted as belonging to the negative class, so-called false
negatives.
Want to Get Started With Imbalance Classification?

Take my free 7-day email crash course now (with sample code).
Click to sign-up and also get a free PDF Ebook version of the course.
Download Your FREE Mini-Course
Precision for Multi-Class Classification

Precision is not limited to binary classification problems.
In an imbalanced classification problem with more than two classes, precision is calculated as the sum of
true positives across all classes divided by the sum of true positives and false positives across all classes.
Precision = Sum c in C TruePositives_c / Sum c in C (TruePositives_c + FalsePositives_c)
For example, we may have an imbalanced multiclass classification problem where the majority class is the
negative class, but there are two positive minority classes: class 1 and class 2. Precision can quantify the
ratio of correct predictions across both positive classes.
Consider a dataset with a 1:1:100 minority to majority class ratio, that is a 1:1 ratio for each positive class
and a 1:100 ratio for the minority classes to the majority class, and we have 100 examples in each minority
class, and 10,000 examples in the majority class. You can master applied Machine Learning
A model makes predictions and predicts 70 examples forFind the out
firsthow
minority
in this class,
free andwhere 50course.
practical are correct and
20 are incorrect. It predicts 150 for the second class with 99 correct and 51 incorrect. Precision can be
calculated for this model as follows:
Email Address
Precision = (TruePositives_1 + TruePositives_2) / ((TruePositives_1 + TruePositives_2) +

(FalsePositives_1 + FalsePositives_2) ) START MY EMAIL COURSE
Precision = (50 + 99) / ((50 + 99) + (20 + 51))
Precision = 149 / (149 + 71)
Precision = 149 / 220
Precision = 0.677
We can see that the precision metric calculation scales as we increase the number of minority classes.
Calculate Precision With Scikit-Learn

The precision score can be calculated using the precision_score() scikit-learn function.
For example, we can use this function to calculate precision for the scenarios in the previous section.
First, the case where there are 100 positive to 10,000 negative examples, and a model predicts 90 true
positives and 30 false positives. The complete example is listed below.
# calculates precision for 1:100 dataset with 90 tp and 30 fp

from sklearn.metrics import precision_score
# define actual
act_pos = [1 for _ in range(100)]
act_neg = [0 for _ in range(10000)]
y_true = act_pos + act_neg
# define predictions
pred_pos = [0 for _ in range(10)] + [1 for _ in range(90)]
pred_neg = [1 for _ in range(30)] + [0 for _ in range(9970)]
y_pred = pred_pos + pred_neg
# calculate prediction Start Machine Learning
precision = precision_score(y_true, y_pred, average='binary')
Running the example calculates the precision, matching our manual calculation.
1 Precision: 0.750
Next, we can use the same function to calculate precision for the multiclass problem with 1:1:100, with 100
examples in each minority class and 10,000 in the majority class. A model predicts 50 true positives and 20
false positives for class 1 and 99 true positives and 51 false positives for class 2.
When using the precision_score() function for multiclass classification, it is important to specify the minority
classes via the “labels” argument and to perform set the “average” argument to ‘micro‘ to ensure the
calculation is performed as we expect.
The complete example is listed below. Start Machine Learning ×

1 # calculates precision for 1:1:100 dataset with You can master
50tp,20fp, applied Machine Learning
99tp,51fp
2 from sklearn.metrics import precision_score without math or fancy degrees.
3 # define actual Find out how in this free and practical course.
4 act_pos1 = [1 for _ in range(100)]
6 act_neg = [0 for _ in range(10000)]
Email Address
7 y_true = act_pos1 + act_pos2 + act_neg
8 # define predictions
9 pred_pos1 = [0 for _ in range(50)] + [1 for _ in range(50)]
11 pred_neg = [1 for _ in range(20)] + [2 for _ in range(51)] + [0 for _ in range(9929)]
12 y_pred = pred_pos1 + pred_pos2 + pred_neg
13 # calculate prediction
14 precision = precision_score(y_true, y_pred, labels=[1,2], average='micro')
15 print('Precision: %.3f' % precision)
Again, running the example calculates the precision for the multiclass example matching our manual
calculation.
1 Precision: 0.677
Recall for Imbalanced Classification

Recall is a metric that quantifies the number of correct positive predictions made out of all positive
predictions that could have been made.
Unlike precision that only comments on the correct positive predictions out of all positive predictions, recall
provides an indication of missed positive predictions.
In this way, recall provides some notion of the coverage of the positive class.
For imbalanced learning, recall is typically used to measure the coverage of the minority class.

— Page 27, Imbalanced Learning: Foundations, Algorithms, and Applications, 2013.
Recall for Binary Classification

In an imbalanced classification problem with two classes, recall is calculated as the number of true
positives divided by the total number of true positives and false negatives.
Recall = TruePositives / (TruePositives + FalseNegatives)
The result is a value between 0.0 for no recall and 1.0 for full or perfect recall.
Let’s make this calculation concrete with some examples.
As in the previous section, consider a dataset with 1:100 minority to majority ratio, with 100 minority
examples and 10,000 majority class examples.
Start
A model makes predictions and predicts 90 of the positive Machine
class predictions Learning
correctly and 10 incorrectly. We
×
can calculate the recall for this model as follows:
Recall = 90 / (90 + 10)
Recall = 90 / 100
Email Address
Recall = 0.9
This model has a good recall.

Recall for Multi-Class Classification

Recall is not limited to binary classification problems.
In an imbalanced classification problem with more than two classes, recall is calculated as the sum of true
positives across all classes divided by the sum of true positives and false negatives across all classes.
Recall = Sum c in C TruePositives_c / Sum c in C (TruePositives_c + FalseNegatives_c)
As in the previous section, consider a dataset with a 1:1:100 minority to majority class ratio, that is a 1:1
ratio for each positive class and a 1:100 ratio for the minority classes to the majority class, and we have
100 examples in each minority class, and 10,000 examples in the majority class.
A model predicts 77 examples correctly and 23 incorrectly for class 1, and 95 correctly and five incorrectly
for class 2. We can calculate recall for this model as follows:
Recall = (TruePositives_1 + TruePositives_2) / ((TruePositives_1 + TruePositives_2) +

(FalseNegatives_1 + FalseNegatives_2))
Recall = (77 + 95) / ((77 + 95) + (23 + 5))
Recall = 172 / (172 + 28)
Recall = 172 / 200
Recall = 0.86
Calculate Recall With Scikit-Learn

The recall score can be calculated using the recall_score() scikit-learn function.
For example, we can use this function to calculate recall for the scenarios above.
First, we can consider the case of a 1:100 imbalance with 100 and 10,000 examples respectively, and a
model predicts 90 true positives and 10 false negatives.
The complete example is listed below.
1 # calculates recall for 1:100 dataset with 90 tp and 10 fn

2 from sklearn.metrics import recall_score
3 # define actual
4
5
act_neg = [0 for _ in range(10000)] Start Machine Learning ×
6 y_true = act_pos + act_neg
7 # define predictions You can master applied Machine Learning
8 pred_pos = [0 for _ in range(10)] + [1 for _ in range(90)]
9 pred_neg = [0 for _ in range(10000)]
10 y_pred = pred_pos + pred_neg Find out how in this free and practical course.
11 # calculate recall
12 recall = recall_score(y_true, y_pred, average='binary')
13 print('Recall: %.3f' % recall) Email Address
Running the example, we can see that the score matches the manual calculation above.
1 Recall: 0.900
We can also use the recall_score() for imbalanced multiclass classification problems.
In this case, the dataset has a 1:1:100 imbalance, with 100 in each minority class and 10,000 in the
majority class. A model predicts 77 true positives and 23 false negatives for class 1 and 95 true positives
and five false negatives for class 2.
1 # calculates recall for 1:1:100 dataset with 77tp,23fn and 95tp,5fn

2 from sklearn.metrics import recall_score
3 # define actual
6 act_neg = [0 for _ in range(10000)]
7 y_true = act_pos1 + act_pos2 + act_neg
8 # define predictions
11 pred_neg = [0 for _ in range(10000)]
12 y_pred = pred_pos1 + pred_pos2 + pred_neg
13 # calculate recall
14 recall = recall_score(y_true, y_pred, labels=[1,2], average='micro')
15 print('Recall: %.3f' % recall)
Again, running the example calculates the recall for the multiclass example matching our manual
calculation.
1 Recall: 0.860
Precision vs. Recall for Imbalanced Classification

You may decide to use precision or recall on your imbalanced classification problem.
Maximizing precision will minimize the number false positives, whereas maximizing the recall will minimize
the number of false negatives.
Precision: Appropriate when minimizing false positives is the focus.

Recall: Appropriate when minimizing false negatives is the focus.
Sometimes, we want excellent predictions of the positive class. We want high precision and high recall.
This can be challenging, as often increases in recall oftenStart Machine

come at the expenseLearning
of decreases in precision.
×
In imbalanced datasets, the goal is to improve recall without
without mathhurting
or fancy precision.
degrees. These goals,
 Find out the
however, are often conflicting, since in order to increase howTP
in this
for free
the and practical
minority course.
class, the
number of FP is also often increased, resulting in reduced precision.
Email Address
Nevertheless, instead of picking one measure or the other, we can choose a new metric that combines
both precision and recall into one score.
F-Measure for Imbalanced Classification

Classification accuracy is widely used because it is one single measure used to summarize model
performance.
F-Measure provides a way to combine both precision and recall into a single measure that captures both
properties.
Alone, neither precision or recall tells the whole story. We can have excellent precision with terrible recall,
or alternately, terrible precision with excellent recall. F-measure provides a way to express both concerns
with a single score.
Once precision and recall have been calculated for a binary or multiclass classification problem, the two
scores can be combined into the calculation of the F-Measure.
The traditional F measure is calculated as follows:
F-Measure = (2 * Precision * Recall) / (Precision + Recall)
This is the harmonic mean of the two fractions. This is sometimes called the F-Score or the F1-Score and
might be the most common metric used on imbalanced classification problems.
… the F1-measure, which weights precision and recall equally, is the variant most often used
 when learning from imbalanced data.
Like precision and recall, a poor F-Measure score is 0.0 and a best or perfect F-Measure score is 1.0
For example, a perfect precision and recall score would result in a perfect F-Measure score:

F-Measure = (2 * 1.0 * 1.0) / (1.0 + 1.0)
F-Measure = (2 * 1.0) / 2.0
F-Measure = 1.0
Let’s make this calculation concrete with a worked example.
Consider a binary classification dataset with 1:100 minority to majority ratio, with 100 minority examples
and 10,000 majority class examples.
Email Address
Consider a model that predicts 150 examples for the positive class, 95 are correct (true positives), meaning
five were missed (false negatives) and 55 are incorrect (false positives).
We can calculate the precision as follows:

Precision = 95 / (95 + 55)
Precision = 0.633
We can calculate the recall as follows:

Recall = 95 / (95 + 5)
Recall = 0.95
This shows that the model has poor precision, but excellent recall.
Finally, we can calculate the F-Measure as follows:

F-Measure = (2 * 0.633 * 0.95) / (0.633 + 0.95)
F-Measure = (2 * 0.601) / 1.583
F-Measure = 1.202 / 1.583
F-Measure = 0.759
We can see that the good recall levels-out the poor precision, giving an okay or reasonable F-measure
score.
Calculate F-Measure With Scikit-Learn

The F-measure score can be calculated using the f1_score() scikit-learn function.
For example, we use this function to calculate F-Measure for the scenario above.
This is the case of a 1:100 imbalance with 100 and 10,000 examples respectively, and a model predicts 95
true positives, five false negatives, and 55 false positives.
1 # calculates f1 for 1:100 dataset with 95tp, 5fn, 55fp

2 from sklearn.metrics import f1_score
3 # define actual
4
5
act_neg = [0 for _ in range(10000)] Start Machine Learning ×
6 y_true = act_pos + act_neg
7 # define predictions You can master applied Machine Learning
8 pred_pos = [0 for _ in range(5)] + [1 for _ in range(95)]
9 pred_neg = [1 for _ in range(55)] + [0 for _ in range(9945)]
10 y_pred = pred_pos + pred_neg Find out how in this free and practical course.
11 # calculate score
12 score = f1_score(y_true, y_pred, average='binary')
13 print('F-Measure: %.3f' % score) Email Address
Running the example computes the F-Measure, matching our manual calculation, within some minor
rounding errors. START MY EMAIL COURSE
1 F-Measure: 0.760
Further Reading
This section provides more resources on the topic if you are looking to go deeper.
Tutorials
How to Calculate Precision, Recall, F1, and More for Deep Learning Models
How to Use ROC Curves and Precision-Recall Curves for Classification in Python
Papers
A Systematic Analysis Of Performance Measures For Classification Tasks, 2009.
Books
Imbalanced Learning: Foundations, Algorithms, and Applications, 2013.
Learning from Imbalanced Data Sets, 2018.
API
sklearn.metrics.precision_score API.
sklearn.metrics.recall_score API. Start Machine Learning
sklearn.metrics.f1_score API.
Articles
Confusion matrix, Wikipedia.
Precision and recall, Wikipedia.
F1 score, Wikipedia.
Summary
In this tutorial, you discovered you discovered how to calculate and develop an intuition for precision and
recall for imbalanced classification.
Specifically, you learned:

Precision quantifies the number of positive class predictions that actually
You can master belong to
applied Machine the positive class.
Learning
Recall quantifies the number of positive class predictions made
without mathout of all positive
or fancy degrees. examples in the
dataset. Find out how in this free and practical course.
F-Measure provides a single score that balances both the concerns of precision and recall in one
number. Email Address
Do you have any questions?

Ask your questions in the comments below and I will do mySTART
best to
MYanswer.
EMAIL COURSE
Get a Handle on Imbalanced Classification!

Develop Imbalanced Learning Models in Minutes
...with just a few lines of python code
Discover how in my new Ebook:

Imbalanced Classification with Python
It provides self-study tutorials and end-to-end projects on:

Performance Metrics, Undersampling Methods, SMOTE, Threshold Moving, Probability
Calibration, Cost-Sensitive Algorithms
and much more...
Bring Imbalanced Classification Methods to Your Machine

Learning Projects
SEE WHAT'S INSIDE
Tweet Share Share
About Jason Brownlee

Jason Brownlee, PhD is a machine learning specialist who teaches developers how to get results with
modern machine learning methods via hands-on tutorials.
View all posts by Jason Brownlee →
 Failure of Classification Accuracy for Imbalanced Class Distributions

ROC Curves and Precision-Recall Curves for Imbalanced Classification 

25 Responses to How to Calculate Precision, Recall, and F-Measure for
Imbalanced Classification Find out how in this free and practical course.
Email Address REPLY 

Curtis January 3, 2020 at 8:18 am #
“Precision: Appropriate when false negatives are more costly.

Recall: Appropriate when false positives are more costly.”
This seems backwards. Generally, isn’t Precision improved by increasing the classification threshold (i.e., a
higher probability of the Positive class is needed for a True decision) which leads to fewer FalsePositives
and more FalseNegatives. And similarly, isn’t Recall generally improved by lowering the classification
threshold (i.e., a lower probability of the Positive class is needed for a True decision) which leads to more
FalsePositives and fewer FalseNegatives.
Thanks for maintaining an excellent blog.
REPLY 
Jason Brownlee January 4, 2020 at 8:12 am #
Thanks, updated!
REPLY 
Ashok Chilakapati January 3, 2020 at 11:59 am #
Very nice set of articles on DS & ML Jason. Thanks for taking the time to write up.
Curtis’ coment is right.
Precision = TP / (TP + FP)
So as FP => 0 we get Precision => 1 Start Machine Learning
Likewise
Recall = TP / (TP + FN)
So as FN => 0 we get Recall => 1
By the way, in the context of text classification I have found that working with those
so called “significant terms” enables one to pick the features that enable
better balance between precision and recall
REPLY 
Thanks, I have updated the tutorial.

REPLY 
MarkK January 3, 2020 at 1:03 pm # Find out how in this free and practical course.
yikes — great catch Curtis — seems rather basic and Im guessing the cause is too much Holiday
Email
Cheer — still a fantastic article Jason, thank you. I am a huge fan.Address
Mark K
REPLY 
Anirban January 3, 2020 at 1:20 pm #
There are 3 modes for calculating precision and recall in a multiclass problem, micro, macro and
weighted. Can you kindly discuss when to use which.
REPLY 
Great suggestion, thanks!
As a start, see the description here:

https://scikit-learn.org/stable/modules/generated/sklearn.metrics.precision_score.html
REPLY 
vaibhav khandelwal January 3, 2020 at 4:55 pm #
Please elaborate the following para :
The main reason is that the overwhelming number of examples from the majority class (or classes) will
overwhelm the number of examples in the minority class, meaning that even unskillful models can achieve
accuracy scores of 90 percent, or 99 percent, depending on how severe the class imbalance happens to be.
REPLY 
A model will perform “well” by ignoring the minority class and modeling the majority class.
Does that help?
REPLY 
Cleyton Farias January 3, 2020 at 8:54 pm #
Great post Jason. I had the same doubt as Curtis. If you have some time to explain the logic behind
the following statement,I would appreciate it.
Start
“Precision: Appropriate when false negatives are more costly. Machine Learning ×
Recall: Appropriate when false positives are more costly.”
REPLY 
Email Address
Thanks, I have updated the tutorial.
REPLY 
Kamal Pandey January 17, 2020 at 4:00 am #
Hi Jason Can i have f1_score(y_true, y_pred, average='weighted') for binary

classification.
REPLY 
Perhaps test it?
REPLY 
Kamal Pandey January 17, 2020 at 10:59 am #
Thanks I used it but precision recall and fscore seems to be almost similar just differ in
some digits after decimal is it valid?
REPLY 
Jason Brownlee January 17, 2020 at 1:50 pm #
Perhaps investigate the specific predictions made on the test set and understand what
was calculated in the score.
REPLY 
faezeh March 26, 2020 at 2:32 am #
Hello , I’m confused!

What is the difference in computing the methods Precision, Recall, and F-Measure for balanced and
unbalanced classes?
REPLY 
Jason Brownlee March 26, 2020 at 7:58 am #
The metrics are more useful for imbalanced dataset generally.

The main consideration is to ensure that the “positive class” is class label 1 or marked via the pos_label
argument. You can master applied Machine Learning
REPLY 
faezeh March 26, 2020 at 5:14 pm # Email Address
Thanks
it’s true START MY EMAIL COURSE
I want to know how to calculate Precision, Recall, and F-Measure for balanced data?
Does it differ from the unbalanced data method?
REPLY 
No different, as long as you clearly mark the “positive” class.
faezeh March 27, 2020 at 5:34 pm #
I just got acquainted with your website

You have some useful content
I got a lot of use
Thank you
Thanks.
REPLY 
Jean May 6, 2020 at 11:37 pm #
Great article Jason! The concepts of precision and recall can be useful to assess model
performance in cybersecurity. To illustrate your explanation, here is a blog post about accuracy vs precision
vs recall applied to secrets detection in cybersecurity :
https://blog.gitguardian.com/secrets-detection-accuracy-precision-recall-explained/
REPLY 
Jason Brownlee May 7, 2020 at 6:53 am #
Thanks for sharing Jean.

You can master applied Machine Learning REPLY 
miae jeong May 21, 2020 at 8:39 pm #
Findaout
Hello. Thank you for your tutorial. And I’d like to ask how in this
question. How free
canand practical
I set whichcourse.
is positive
class and which is negative class? I am using tensorflow 2. version offering metrics like precision and recall.
Then I don’t know this result is which class’s result. Could you give
Email me a clue?
Address

REPLY 
Jason Brownlee May 22, 2020 at 6:06 am #
You’re welcome.
You can set the “pos_label” argument to specify which is the positive class, for example:
https://scikit-learn.org/stable/modules/generated/sklearn.metrics.precision_score.html
Leave a Reply
Name (required)
Email (will not be published) (required)
Website
SUBMIT COMMENT
Welcome!
My name is Jason Brownlee PhD, and I help developers get results with machine learning.
Read more

Never miss a tutorial: without math or fancy degrees.
Email Address
Picked for you:

8 Tactics to Combat Imbalanced Classes in Your Machine Learning Dataset
SMOTE for Imbalanced Classification with Python
Imbalanced Classification With Python (7-Day Mini-Course)
A Gentle Introduction to Threshold-Moving for Imbalanced Classification
Tour of Evaluation Metrics for Imbalanced Classification
Loving the Tutorials?
The Imbalanced Classification with Python EBook is where I keep the Really Good stuff.
SEE WHAT'S INSIDE
© 2020 Machine Learning Mastery Pty. Ltd. All Rights Reserved.

Address: PO Box 206, Vermont Victoria 3133, Australia. | ACN: 626 223 336.
LinkedIn | Twitter | Facebook | Newsletter | RSS
Privacy | Disclaimer | Terms | Contact | Sitemap | Search

Email Address

How To Calculate Precision, Recall, and F-Measure For Imbalanced Classification

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

How To Calculate Precision, Recall, and F-Measure For Imbalanced Classification

Uploaded by

Copyright:

Available Formats

6/17/2020 How to Calculate Precision, Recall, and F-Measure for Imbalanced Classification

Click to Take the FREE Imbalanced Classification Crash-Course

How to Calculate Precision, Recall, and F-Measure

Tweet Share Share

Last Updated on February 6, 2020

As a performance measure, accuracy is inappropriate for imbalanced classification problems.

An alternative to using classification accuracy is to use precision and recall metrics.

After completing this tutorial, you will know:

Let’s get started.

Start Machine Learning ×

START MY EMAIL COURSE

How to Calculate Precision, Recall, and F-Measure for Imbalanced Classification

1. Confusion Matrix for Imbalanced Classification

Confusion Matrix for Imbalanced Classification

Start Machine Learning

1 | Positive Prediction | Negative Prediction

Precision for Imbalanced Classification

Precision, therefore, calculates the accuracy for the minority class.

— Page 52, Learning from Imbalanced Data Sets, 2018.

Precision for Binary Classification

Precision = TruePositives / (TruePositives + FalsePositives)

Let’s make this calculation concrete with some examples.

The precision for this model is calculated as:

Precision = TruePositives / (TruePositives + FalsePositives)

Precision = TruePositives / (TruePositives + FalsePositives)

Want to Get Started With Imbalance Classification?

Download Your FREE Mini-Course

Start Machine Learning

Precision for Multi-Class Classification

Precision = Sum c in C TruePositives_c / Sum c in C (TruePositives_c + FalsePositives_c)

Precision = (TruePositives_1 + TruePositives_2) / ((TruePositives_1 + TruePositives_2) +

Calculate Precision With Scikit-Learn

# calculates precision for 1:100 dataset with 90 tp and 30 fp

precision = precision_score(y_true, y_pred, average='binary')

The complete example is listed below. Start Machine Learning ×

Recall for Imbalanced Classification

Recall for Binary Classification

Recall = TruePositives / (TruePositives + FalseNegatives)

Let’s make this calculation concrete with some examples.

This model has a good recall.

Recall for Multi-Class Classification

Recall = Sum c in C TruePositives_c / Sum c in C (TruePositives_c + FalseNegatives_c)

Recall = (TruePositives_1 + TruePositives_2) / ((TruePositives_1 + TruePositives_2) +

Calculate Recall With Scikit-Learn

The complete example is listed below.

1 # calculates recall for 1:100 dataset with 90 tp and 10 fn

The complete example is listed below.

1 # calculates recall for 1:1:100 dataset with 77tp,23fn and 95tp,5fn

Precision vs. Recall for Imbalanced Classification

Precision: Appropriate when minimizing false positives is the focus.

This can be challenging, as often increases in recall oftenStart Machine

F-Measure for Imbalanced Classification

The traditional F measure is calculated as follows:

F-Measure = (2 * Precision * Recall) / (Precision + Recall)

— Page 27, Imbalanced Learning: Foundations, Algorithms, and Applications, 2013.

F-Measure = (2 * Precision * Recall) / (Precision + Recall)

Precision = TruePositives / (TruePositives + FalsePositives)

We can calculate the recall as follows:

Recall = TruePositives / (TruePositives + FalseNegatives)

Finally, we can calculate the F-Measure as follows:

F-Measure = (2 * Precision * Recall) / (Precision + Recall)

Calculate F-Measure With Scikit-Learn

The complete example is listed below.