Professional Documents
Culture Documents
Uncertainty
12/03/2024
Koustav Rudra
Probability Distributions
• A probability distribution is a description of how likely a random variable is to take on each of its
possible states
• Notation: 𝑃(𝑋) is the probability distribution over the random variable 𝑋
• Associate a probability with each value
• Non-negativity
• For any event A ∈ 𝒮, 𝑃(𝐴) ≥ 0
• Probabilistic models:
T W P
• Random variables with domains
• Assignments are called outcomes Hot Sun 0.4
• Joint distributions: say whether
Hot Run 0.1
• assignments (outcomes) are likely
• Normalized: sum to 1.0 Cold Sun 0.2
• Ideally: only certain variables directly interact Cold Run 0.3
Events
• An event is a set E of outcomes
• 𝑃 𝐸 = ∑𝑥1,𝑥2,…,𝑥𝑛∈" P(𝑥1,𝑥2,…,𝑥𝑛)
T W P
• From a joint distribution, we can calculate the probability of any event Hot Sun 0.4
• Probability that it’s hot AND sunny?
Hot Run 0.1
• Probability that it’s hot?
• Probability that it’s hot OR sunny? Cold Sun 0.2
Cold Run 0.3
Marginal Probability Distribution
• It is calculated based on the joint probability distribution 𝑃(𝑋,𝑌) using the sum rule:
• 𝑃 𝑋 = 𝑥 = ∑# 𝑃(𝑋 = 𝑥, 𝑌 = 𝑦)
Marginal Distributions
• Marginal distributions are sub-tables which eliminate variables
• Marginalization (summing out): Combine collapsed rows by adding
T P
T W P 𝑃 𝑡 = $ 𝑃(𝑡, 𝑠) = 𝑃(𝑇)
! Hot 0.5
Hot Sun 0.4 Cold 0.5
$(3∧5|7)
• 𝑃 𝐴 𝐵∧𝐶 =
$(5|7)
$(3|7)∧$(5|7)
• 𝑃 𝐴 𝐵∧𝐶 =
$(5|7)
• 𝑃 𝐴 𝐵 ∧ 𝐶 = 𝑃(𝐴|𝐶)
Conditional Independence and Chain Rule
Trivial decomposition:
P(Traffic, Rain, Umbrella)=
P(Rain)P(Traffic|Rain)P(Umbrella|Rain,Traffic)
• 𝑃𝑎𝑟𝑒𝑛𝑡𝑠 𝑥; : minimal set of predecessors of 𝑋𝑖 in the total ordering such that other predecessors
are conditionally independent of 𝑋𝑖 given 𝑃arent(𝑋𝑖)
Bayesian Network
• What is the issue with joint probability distribution?
• Become intractably large as the number of variables grows
• Specifying probabilities for atomic events is really difficult
Alarm
A P(J) A P(M)
t 0.90 B E P(A) t 0.70
f 0.05 t t 0.95 f 0.01
t f 0.94
f t 0.29
JohnCalls f f 0.001 MaryCalls
Joint probability distribution
• 𝑃 𝑥! , … , 𝑥" = ∏"#$! 𝑃(𝑥# |𝑃𝑎𝑟𝑒𝑛𝑡𝑠(𝑥# ))
• 𝑃(𝐽 ∧ 𝑀 ∧ 𝐴 ∧ ~𝐵 ∧ ~𝐸)
• 𝑃 𝐽𝐴 ∗
• 𝑃 𝑀𝐴 ∗
• 𝑃 𝐴 ~𝐵 ∧ ~𝐸 ∗
• 𝑃 ~𝐵 ∗
• P(~𝐸)
• 𝑃 𝐽 ∧ 𝑀 ∧ 𝐴 ∧ ~𝐵 ∧ ~𝐸 = 0.9×0.7×0.001×0.999×0.998 = 0.00062
• 𝑃 𝐽 =?
Conditional Independence
• 𝑃 𝐽, 𝑀, 𝐴, 𝐵, 𝐸 = 𝑃 𝐽 𝑀, 𝐴, 𝐵, 𝐸 𝑃(𝑀, 𝐴, 𝐵, 𝐸)
• 𝑃 𝐽, 𝑀, 𝐴, 𝐵, 𝐸 = 𝑃 𝐽 𝐴 𝑃 𝑀 𝐴, 𝐵, 𝐸 𝑃(𝐴, 𝐵, 𝐸)
• 𝑃 𝐽, 𝑀, 𝐴, 𝐵, 𝐸 = 𝑃 𝐽 𝐴 𝑃 𝑀 𝐴 𝑃 𝐴 𝐵, 𝐸 𝑃(𝐵, 𝐸)
• 𝑃 𝐽, 𝑀, 𝐴, 𝐵, 𝐸 = 𝑃 𝐽 𝐴 𝑃 𝑀 𝐴 𝑃 𝐴 𝐵, 𝐸 𝑃(𝐵)𝑃(𝐸)
Alarm
Burglary Earthquake
Conditional Independence
• Alarm, Burglary, Earthquake, JohnCalls, MaryCalls
MaryCalls JohnCalls
Earthquake
Burglary Alarm
Incremental Network Construction
• Choose the set of relevant variables 𝑋; , that describe the domain