ASSIGNMENT 02

Submitted To
Dr. Kashif sultan
Submitted by
Asma Farooq
01-241192-003
Date: 25-12-2020
Department of Software Engineering,
Bahria University Islamabad.
Q1: What is Bayes’ Theorem? How is it useful in a machine learning context?

Bayes’ theorem:

Bayes’ theorem is a formula to determine conditional probability. Conditional probability is that

how likely there are chances of an event to occur with some relationship to one or more other
events (based on history). Bayes' theorem gives a way to revise current estimates or theories
(update probabilities) given new or supplementary evidence.

Bayes’ theorem in machine learning:

Bayes’ theorem gives a new point of view about the relationship between data and model. A
machine learning algorithm or model is a specific way of thinking about data-based
relationships. In this way, a model acts as a hypothesis about data-based relationships such as
relationship between input and output. Testing and analysis of different models on given
dataset is machine learning. Bayes’ theorem defines the relationship between data and
hypothesis by providing probabilistic model e.g.
P(h|D) = P(D|h) * P(h) / P(D)
On the basis of prior probability Bayes’ theorem offers a way to calculate the probability of
hypothesis, the probabilities of observing various data given the hypothesis, and the observed
data itself. It is useful for classification and regression specially when training data is unreliable

Q2: What’s the difference between a generative and discriminative model?

Generative model Discriminative model
 Actual distribution of each class is  Decision boundary between the classes is
explicitly modeled by Generative model. modeled by discriminative model.

 It models the distribution of individual  It  learns the (hard or

classes soft) boundaries between classes.

 It takes care about how the data was  It does not care about how the data was
generated in order to categorize a signal. generated, it simply categorizes a given
  signal.
 A Generative Model ‌learns the joint
probability distribution p(x,y). It predicts  A Discriminative model l‌earns
the conditional probability with the help the conditional probability distribution
of Bayes Theorem. p(y|x).
Generative classifiers Discriminative classifiers
  It assumes functional form for P(Y), P(X|  It assumes functional form for P(Y|X)
Y)  Parameters of P(Y|X) are
 Parameters of P(X|Y), P(Y) are estimated estimated directly from training data.
directly from training data.
 It uses Bayes rule to calculate P(Y |X)
Examples Examples
 ‌ aïve Bayes
N  L‌ ogistic regression
 Bayesian networks  Scalar Vector Machine
 Markov random fields  ‌Traditional neural networks
 ‌Hidden Markov Models (HMM)  ‌Nearest neighbour
 Conditional Random Fields (CRF)s

Q3: Which is more important to you: model accuracy or model performance?

Explain with examples.
Answer:
Model accuracy is a metric for understanding performance. There are models that have good
accuracy but have poor performance when it comes to prediction. It is because model accuracy
Is only a subset of model performance. To decide, if model accuracy is essential or
performance, is highly context dependent.
For example, in medical imaging applications, accuracy is more important. Any false diagnosis
can lead to unnecessary treatment in case of false positive or loss of life in case of a false
negative.
On other hand, there are situations where accuracy alone cannot help in improving
performance. For example, if the goal is to predict a specific label, higher accuracy will not
ensure good performance on predicting a specific label because accuracy is about how well a
model predicts "all" of the labels. In such cases, we must involve other performance metrics.
Thus, we have situations where model performance is more important, and we can
compromise on accuracy for a while.

Q4: When should you use classification over regression? Give Example as well.
Answer:
As classification and regression both are associated with prediction so, it depends on the
problem statement and data that which should be used; classification or regression. When the
outcomes of the model need to return the belongingness of data points in a dataset to precise
obvious classes then classification technique is chosen over regression. As classification make
predictions about the belongingness to a class and regression make predictions about values in
a continuous set.

For example, we have names of fruits and vegetables. We are interested in finding out each
name belongs to the fruit’s category or to the vegetable category instead of finding out that
how correlated they are with fruits and vegetables names.