MEHTA FAMILY CENTRE FOR DATA SCIENCE AND ARTIFICILA

INTELLIGENCE, IIT ROORKEE

November 16, 2022
AID-505: MACHINE LEARNING
TOPIC: Bayesian Learning

1. Define Bayes Theorem with its basic assumptions. Explain some of its applications in
machine learning.
2. 2 percent of the email Ram receives is spam. The spam detector, available with Ram, is
really good and when an email is spam, it detects with an accuracy of 99 percent, and when
an email is not spam, it will mark it as spam with a very low rate of 0.1 percent. Consider
the case where Ram receives an email and the spam detector puts it in the spam folder;
what is the probability that the email is spam?
3. An univariate mixture of Gaussian models consists of 2 components. The probability of
any number belonging to first component (mean=0, standard deviation=1) is 0.7 and
probability of belonging to second component (mean=6, standard deviation=2) is 0.3.
Determine in which component do number 2 belongs to.
4. Find the predictive value for parameter C, using Markov Model in MATLAB/
Python, Given the data and Observations for parameters A, B and C are in file:
Tutorial Data -23092022.xls
5. In a day, Sita does either of these four activities : Painting, Cleaning the house, Biking,
Shopping for groceries. Now, in some particular four days, Sita did: painting, cleaning,
shopping, biking. Find whether the days were sunny or rainy. The HMM model for Sita
has been found to as follows:

Find whether the days were sunny or rainy.

 6. Find the predictive value for parameter F, using Hidden Markov Model in
MATLAB/ Python. Assume, the data in E depends on A, B, C & D for Observations
at F in file: Tutorial Data -16112022.xls

Answers
2. Using Bayes Theorem, we can write
P (Spam| Detected) = P(Detected| Spam) * P(Spam) / P(Detected)
From, given data, P(Detected| Spam) = 0.99
P(Spam) = * 0.02
And, P(Detected) = P(Detected| Spam) * P(Spam) + P(Detected| not Spam) * P(not Spam)
Given, P(Detected| not Spam) = 0.001
P(not Spam) = 1 – P(Spam) = 1 – 0.02 = 0.98
Therefore, P(Detected) = 0.99 * 0.02 + 0.001 * 0.98 = 0.0198 + 0.00098 = 0.02078
Thus, P (Spam| Detected) = 0.99 * 0.02 / 0.02078 = 0.0198 / 0.02078 = 0.95283926852743

That is, if an email is in the spam folder, there is a 95.2 percent probability that it is, in fact,
spam.

3. As per given data, let

Z ~ multi-monial (0.7, 0.3); X| Z=1 ~ Gaussian (0,1); X| Z=2 ~ Gaussian (6,2)
𝑛
P(X)= ∑𝑘=0 𝑃(𝑍). 𝑃(𝑋|𝑍)
𝐾
= ∑𝑘=1 𝑃(𝑍). 𝑃(𝑋|𝑍)

= 𝑃(𝑍 = 1). 𝑃(𝑋|𝑍 = 1) + 𝑃(𝑍 = 2). 𝑃(𝑋|𝑍 = 2)

𝑃(𝑍=1). 𝑃(𝑋 |𝑍 = 1)
P(Z=1|X) = 𝑃(𝑋)

𝑃(𝑍=1). 𝑃(𝑋 |𝑍 = 1)
= 𝑃(𝑍=1).𝑃(𝑋 |𝑍 = 1)+ 𝑃(𝑍=2).𝑃(𝑋|𝑍=2)
By Given, P(Z=1) =0.7; P(Z=2) =0.3;
𝑃(𝑋 = 2|𝑍 = 1) = Gaussian (x=2; 0,1) = 0.054
𝑃(𝑋 = 2|𝑍 = 2) = Gaussian (x=2; 6, 2) = 0.027
0.7𝑋0.054
P(Z=1|X) = 0.7𝑋0.054+0.3𝑋0.027 = 0.824
0.3𝑋0.027
P(Z=2|X) = 0.7𝑋0.054+0.3𝑋0.027 =
Number 2 belongs to Gaussian (0,1)
5. Hints:
O = (Painting, Cleaning, Shopping, Biking);
No. of hidden states, N = 2
No. of observations states, T = 4
No. of possible sequences of states = 𝑁 𝑇 = 24 = 16
Joint Probability (Painting, Sunny) = P (Painting| Sunny). P(Sunny).

HMM MODEL

P (P,C,SH,B – S, R, S, S) = [P (S). P(R|S). P(S|R). P(S|S)] . [P(P|S). P(C|R). P(SH|S). P(B|S)]

= [0.6X0.2X0.4X0.8][0.4X0.45X0.2X0.3]