Professional Documents
Culture Documents
[These slides were created by Dan Klein and Pieter Abbeel for CS188 Intro to AI at UC Berkeley. All CS188 materials are available at http://ai.berkeley.edu.]
Today
Probability
Random Variables
Joint and Marginal Distributions
Conditional Distribution
Product Rule, Chain Rule, Bayes’ Rule
Inference
Independence
Temperature: Weather:
W P
T P
sun 0.6
hot 0.5
rain 0.1
cold 0.5
fog 0.3
meteor 0.0
Probability Distributions
Unobserved random variables have distributions
Shorthand notation:
T P W P
hot 0.5 sun 0.6
cold 0.5 rain 0.1
fog 0.3
meteor 0.0
T W P
Must obey: hot sun 0.4
hot rain 0.1
cold sun 0.2
cold rain 0.3
X Y P
+x +y 0.2
P(+x) ?
+x -y 0.3
-x +y 0.4
-x -y 0.1
P(-y OR +x) ?
Marginal Distributions
Marginal distributions are sub-tables which eliminate variables
Marginalization (summing out): Combine collapsed rows by adding
T P
hot 0.5
T W P
cold 0.5
hot sun 0.4
hot rain 0.1
cold sun 0.2 W P
cold rain 0.3 sun 0.6
rain 0.4
Quiz: Marginal Distributions
X P
+x
X Y P
-x
+x +y 0.2
+x -y 0.3
-x +y 0.4 Y P
-x -y 0.1 +y
-y
Conditional Probabilities
A simple relation between joint and conditional probabilities
In fact, this is taken as the definition of a conditional probability
P(a,b)
P(a) P(b)
T W P
hot sun 0.4
hot rain 0.1
cold sun 0.2
cold rain 0.3
Quiz: Conditional Probabilities
P(+x | +y) ?
X Y P
+x +y 0.2 P(-x | +y) ?
+x -y 0.3
-x +y 0.4
-x -y 0.1
P(-y | +x) ?
Conditional Distributions
Conditional distributions are probability distributions over
some variables given fixed values of others
Conditional Distributions
Joint Distribution
W P
T W P
sun 0.8
hot sun 0.4
rain 0.2
hot rain 0.1
cold sun 0.2
W P cold rain 0.3
sun 0.4
rain 0.6
Normalization Trick
T W P
hot sun 0.4
W P
hot rain 0.1
sun 0.4
cold sun 0.2
rain 0.6
cold rain 0.3
Normalization Trick
Example 1 Example 2
W P W P T W P T W P
Normalize
hot sun 20 Normalize hot sun 0.4
sun 0.2 sun 0.4
hot rain 5 hot rain 0.1
rain 0.3 Z = 0.5 rain 0.6 Z = 50
cold sun 10 cold sun 0.2
cold rain 15 cold rain 0.3
Probabilistic Inference
Probabilistic inference: compute a desired
probability from other known probabilities (e.g.
conditional from joint)
Step 1: Select the Step 2: Sum out H to get joint Step 3: Normalize
entries consistent of Query and evidence
with the evidence
Inference by Enumeration
S T W P
P(W)?
summe hot sun 0.30
r
summe hot rain 0.05
P(W | winter)? r
summe cold sun 0.10
r
summe cold rain 0.05
r
P(W | winter, hot)? winter hot sun 0.10
winter hot rain 0.05
winter cold sun 0.15
winter cold rain 0.20
Inference by Enumeration
Obvious problems:
Worst-case time complexity O(dn)
Space complexity O(dn) to store the joint distribution
The Product Rule
Sometimes have conditional distributions but want the joint
The Product Rule
Example:
D W P D W P
wet sun 0.1 wet sun 0.08
R P
dry sun 0.9 dry sun 0.72
sun 0.8
wet rain 0.7 wet rain 0.14
rain 0.2
dry rain 0.3 dry rain 0.06
The Chain Rule
Dividing, we get:
Example:
M: meningitis, S: stiff neck
Example
givens