NATIONAL UNIVERSITY OF

SCIENCES AND TECHNOLOGY

Probability: basic concepts

• probability of event = p
0 <= p <= 1
0 = certain non-occurrence
1 = certain occurrence
• .5 => even odds
• .1 => 1 chance out of
10
• Tossing a coin P(H)?,
P(H,H,T,H)?
• Let Xi is result of ith coin flip, then
P(X1=X2=X3)?
• P(A) = p => P(~A) = 1-p
NATIONAL UNIVERSITY OF
SCIENCES AND TECHNOLOGY
NATIONAL UNIVERSITY OF
SCIENCES AND TECHNOLOGY
NATIONAL UNIVERSITY OF
SCIENCES AND TECHNOLOGY
NATIONAL UNIVERSITY OF
SCIENCES AND TECHNOLOGY
NATIONAL UNIVERSITY OF
SCIENCES AND TECHNOLOGY

Bayes’ Theorem

P(B A)P(A)
P(A B) P(B) Simplest Bayes Net

Not A
observable

Observable’
/evidence B

7
 NATIONAL UNIVERSITY OF
SCIENCES AND TECHNOLOGY

2-test Cancer example

• P(c)=.01 C

• P(t1|c) = .9, P(t2|c) = .9

• P(t1|~c) = .2, P(t2|~c) = .2
t1 t2
• P(c|t1,t2) = ?

8
 NATIONAL UNIVERSITY OF
SCIENCES AND TECHNOLOGY

2-test Cancer example

• P(c)=.01 C
• P(t1|c) = .9, P(t2|c) = .9
• P(t1|~c) = .2, P(t2|~c) = .2

• P(c|t1,t2) =
𝑃 𝑡 1 , 𝑡2 𝑐 𝑃 𝑐 [by Bayes Rule] t1 t2
𝑃 𝑡 1 ,𝑡 2
𝑃 𝑡2 𝑡 1 , 𝑐 𝑃 𝑡1 𝑐 𝑃 𝑐
= 𝑃 𝑡 1 ,𝑡 2 [by product rule]

In order to calculate 𝑃 𝑡2 𝑡1, 𝑐 , we will make some simplifying assumptions

9
 NATIONAL UNIVERSITY OF

SCIENCES AND

C o n dLOGitY ional
TEC HN O

Conditional Independence Assumption

• Given A, B and C are independent
A
BC
P(B | A,C) = P(B|A)
B C
• Conditional independence does not imply
absolute independence and vice-versa
BC|A

• P(B | C)   P(B | C, Ai )P(Ai |

C)

i P(B | Ai )P(Ai | 10
 NATIONAL UNIVERSITY OF
SCIENCES AND TECHNOLOGY

2-test Cancer example

• P(c)=.01 C
• P(t1|c) = .9, P(t2|c) = .9
• P(t1|~c) = .2, P(t2|~c) = .2
• P(c|t1,t2)
t1 t2
𝑃 𝑡 1 , 𝑡2 𝑐 𝑃 𝑐
=
𝑃 𝑡 1 ,𝑡 2
𝑃 𝑡2 𝑡 1 , 𝑐 𝑃 𝑡1 𝑐 𝑃 𝑐
= 𝑃 𝑡 1 ,𝑡 2
𝑃 𝑡2 𝑐 𝑃 𝑡1 𝑐 𝑃 𝑐
= 𝑃 𝑡 1 ,𝑡 2

11
NATIONAL UNIVERSITY OF
SCIENCES AND TECHNOLOGY
NATIONAL UNIVERSITY OF
SCIENCES AND TECHNOLOGY
NATIONAL UNIVERSITY OF
SCIENCES AND TECHNOLOGY

Learning Bayesian Network

NATIONAL UNIVERSITY OF
SCIENCES AND TECHNOLOGY

Learning Bayesian Network

NATIONAL UNIVERSITY OF
SCIENCES AND TECHNOLOGY

Learning Bayesian Network

NATIONAL UNIVERSITY OF
SCIENCES AND TECHNOLOGY

Learning Bayesian Network

NATIONAL UNIVERSITY OF
SCIENCES AND TECHNOLOGY

Learning Bayesian Network

NATIONAL UNIVERSITY OF
SCIENCES AND TECHNOLOGY

Learning Bayesian Network

NATIONAL UNIVERSITY OF
SCIENCES AND TECHNOLOGY

Learning Bayesian Network

NATIONAL UNIVERSITY OF
SCIENCES AND TECHNOLOGY

Laplace Smoothing
NATIONAL UNIVERSITY OF
SCIENCES AND TECHNOLOGY

Laplace Smoothing
NATIONAL UNIVERSITY OF
SCIENCES AND TECHNOLOGY

Laplace Smoothing
NATIONAL UNIVERSITY OF
SCIENCES AND TECHNOLOGY

Laplace Smoothing
NATIONAL UNIVERSITY OF
SCIENCES AND TECHNOLOGY

Laplace Smoothing