Class Notes 04apr2023

Lecture 22: 4 April, 2023
Madhavan Mukund
https://www.cmi.ac.in/~madhavan
Data Mining and Machine Learning

January–April 2023
Probabilistic graphical models
Underlying DAG, no cyclic

dependencies
Each node has a local (conditional)
probability table
P(A)B,E) -
Madhavan Mukund Lecture 22: 4 April, 2023 DMML Jan–Apr 2023 2/9
Conditional independence
x ? y — x and y are independent
P(x ^ y ) = P(x) · P(y )
P(x ^ y ) = P(x) · P(y )
x ?y |z
x and y are independent given z
P(x ^ y | z) = P(x | z) · P(y | z)
P(x(y,z) P(x(z) =
P(y(x,2) 0(y)2)
=
P(x ^ y ) = P(x) · P(y )
x ?y |z
P(x ^ y | z) = P(x | z) · P(y | z)
Is JohnCalls independent of MaryCalls

(j ? m)?
P(x ^ y ) = P(x) · P(y )
x ?y |z
P(x ^ y | z) = P(x | z) · P(y | z)

(j ? m)?
No — value of j tells us something about
value of m and vice versa
P(x ^ y ) = P(x) · P(y )
x ?y |z
⑧
P(x ^ y | z) = P(x | z) · P(y | z)

(j ? m)?

given Alarm (j ? m | a)?
P(x ^ y ) = P(x) · P(y )
x ?y |z
P(x ^ y | z) = P(x | z) · P(y | z)

(j ? m)?

given Alarm (j ? m | a)?
Yes — by semantics of network, local
independence
Probabilistic graphical models
Fundamental assumption
A node is conditionally independent
of non-descendants, given its parents
Student example
SAT ? Grade | Difficulty ?
Student example

No
Student example

No
Can we calculate
conditional independence
from the graph?
Student example

No
Can we calculate
conditional independence
from the graph?
In general, check if
X ? Y | Z for sets of
variables X , Y , Z
How does dependence “flow” through

a network?

a network?
For neighbouring nodes, dependence
flows both ways
If x ! y , knowing x tells us about
y and vice versa
How does dependence “flow” through ........

a network? x z
y
- -

flows both ways
If x ! y , knowing x tells us about x- y z
y and vice versa
- iguire directors ->
Examine trails between nodes
Not
Paths in the underlying undirected apal
graph
*x X
i
y

a network?
flows both ways
If x ! y , knowing x tells us about
y and vice versa
Examine trails between nodes

Paths in the underlying undirected
graph
Basic trails — (undirected) paths of

length 2
Four basic trails
Basic trails
Basic trails
(a), (b) and (c): Z blocks flow
between X and Y , by semantics of
:)
Bayesian networks
Basic trails
Bayesian networks
In (d), knowing Z allows influence to
flow
Basic trails
Bayesian networks
flow
Z : Car does not start
X : Low Battery, Y : No Fuel
Basic trails
Bayesian networks
flow
Z : Grass is wet
X : Overnight rain, Y : Sprinkler ran
Basic trails
-
Bayesian networks
flow
to
Z : Grass is wet
X : Overnight rain, Y : Sprinkler ran
Simplest form of V-structure
x
1 Y/u?
x
=+ y1z
D-Separation
Check if X ? Y | Z
D-Separation
Check if X ? Y | Z
Dependence should be blocked on
every trail from X to Y
**
xyx
D-Separation
Check if X ? Y | Z
Each undirected path from X to Y
is a sequence of basic trails
- /Y
D-Separation
Check if X ? Y | Z
For (a), (b), (c), need Z present
----. O -
->
x
v 2 y
A
D-Separation
Check if X ? Y | Z
For (d), need Z absent
D-Separation
Check if X ? Y | Z
&
In general, V-structure includes
descendants of the bottom node
D-Separation
Check if X ? Y | Z
x and y are D-separated given z if all trails are blocked
D-Separation
Check if X ? Y | Z
Variation of breadth first search (BFS) to check if y is reachable from x through
some trail
Best
Search
Reachability on a
graph
me?
Is
y
reachable from
reachable
neighbours of
is
-Mack all
as
wbrs uses
- Mark
of
-
No new marks
Is Y marked?
... -
W
N I
.....
fixe
vy.--
v
D-Separation
Check if X ? Y | Z
Variation of breadth first search (BFS) to check if y is reachable from x through
some trail
Extends to sets — each x 2 X is D-separated from each y 2 Y
Conditional independence, example
Is SAT independent of Difficulty

given Intelligence?
Yes, Difficulty – Grade – Intelligence
– SAT trail is blocked at Grade
(V-structure) and Intelligence
Conditional independence, example
Is SAT independent of Difficulty

given Intelligence? OPEN
Yes, Difficulty – Grade – Intelligence
– SAT trail is blocked at Grade V
(V-structure) and Intelligence
Is SAT independent of Difficulty Or E ↑

given Letter?
No, Difficulty – Grade – Intelligence
– SAT trail is open
Letter is known, hence something
about Grade is known (V-structure)
Intelligence is not known

Class Notes 04apr2023

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Class Notes 04apr2023

Uploaded by

Copyright:

Available Formats

Lecture 22: 4 April, 2023

Data Mining and Machine Learning

Underlying DAG, no cyclic

Is JohnCalls independent of MaryCalls

Is JohnCalls independent of MaryCalls

Is JohnCalls independent of MaryCalls

Is JohnCalls independent of MaryCalls

Is JohnCalls independent of MaryCalls

Is JohnCalls independent of MaryCalls

SAT ? Grade | Difficulty ?

SAT ? Grade | Difficulty ?

SAT ? Grade | Difficulty ?

SAT ? Grade | Difficulty ?

How does dependence “flow” through

How does dependence “flow” through

How does dependence “flow” through ........

For neighbouring nodes, dependence

How does dependence “flow” through

Examine trails between nodes

Basic trails — (undirected) paths of

Is SAT independent of Difficulty

Is SAT independent of Difficulty

Is SAT independent of Difficulty Or E ↑

You might also like