Bayesian Networks

Javier Larrosa

UPC Barcelona Tech

Javier Larrosa (UPC Barcelona Tech) BN . 1 / 23

Bayesian Networks

A Bayesian Network (BN) is:

A directed acyclic graph whose nodes represent random variables
X = {X1 , X2 , . . . , Xn }
There is an arrow from Xj to Xi if Xj has a direct influence on the
behaviour of Xi
Example:

Javier Larrosa (UPC Barcelona Tech) BN . 2 / 23

Bayesian Networks

For each node Xi , a conditional probability distribution (CPD)

P(Xi |parents(Xi ))

Example:

Javier Larrosa (UPC Barcelona Tech) BN . 3 / 23

Bayesian Networks
Semantics
A Bayesian Network represents a join distribution
Y
P(X1 , X2 , . . . , Xn ) = P(Xi |parents(Xi ))
i

Javier Larrosa (UPC Barcelona Tech) BN . 4 / 23

Space Considerations

A BN is a compact representation of a join distribution P(X )

Property: consider a BN over X = {X1 , X2 , . . . , Xn } random
variables with domains of size k.
The BN represents a join distribution of O(k n ) entries.
The CPD associated to variable Xi with pi = |parents(Xi )| has size
O(k pi +1 )
Let p = maxi {pi }. Then, the size of the BN is O(nk p+1 ).

Warning
Avoid nodes with many parents!!

Javier Larrosa (UPC Barcelona Tech) BN . 5 / 23

Knowledge Elicitation Considerations

It is easier to obtain probabilities about pieces of the problem than about

the whole thing,

Javier Larrosa (UPC Barcelona Tech) BN . 6 / 23

Queries

Once you have build a BN, you can query it,

For
instance:
P(L), P(L|I ), P(L|S), P(I |L), ...
MAP of L in terms of D, I

Javier Larrosa (UPC Barcelona Tech) BN . 7 / 23

Example: Traffic

Javier Larrosa (UPC Barcelona Tech) BN . 8 / 23

Example: Traffic

Javier Larrosa (UPC Barcelona Tech) BN . 9 / 23

A more realistic BN

Javier Larrosa (UPC Barcelona Tech) BN . 10 / 23

A real BN: Model for Diagnosis of Liver Disorders

Javier Larrosa (UPC Barcelona Tech) BN . 11 / 23

A real BN
A BN for diagnosis of the causes of B cell chronic lymphocytic leukemia
(B-CLL). It has well over 2,000 variables (each spot is one variable). And
it works.

Javier Larrosa (UPC Barcelona Tech) BN . 12 / 23

Flows of Probabilistic Influence

Is there dependence between a pair of variables?

Reformulated as: is there any open path between a pair of
variables?
path is defined disregarding the direction of arcs
One path is open if every triplet is active

Javier Larrosa (UPC Barcelona Tech) BN . 13 / 23

Independence Quiz

Javier Larrosa (UPC Barcelona Tech) BN . 14 / 23

Uniqueness

Let P(X ) be a probability distribution over X

Is there a unique BN representing it? NO
Some BN are more natural than others
Some BN are smaller than others

Javier Larrosa (UPC Barcelona Tech) BN . 15 / 23

Some BN are both more natural and smaller than
others: Example of spurios arcs

P(X , Y ) = P(X |Y )P(Y )

X Y P(X , Y ) X Y P(X |Y ) Y P(Y )

0 0 .07 0 0 .1 0 .7
0 1 .03 0 1 .1 1 .3
1 0 .63 1 0 .9
1 1 .27 1 1 .9

But, as you can see, X and Y are independent, so what makes sense is
P(X , Y ) = P(X )P(Y )

Javier Larrosa (UPC Barcelona Tech) BN . 16 / 23

Some BN are more natural than others: Example

Example
I have two coins, one is fair (head and tail) and the other is tricked
(two-headed). I am going to chose one of them at random, flip it and it it
comes out head I will buy a motorcycle with probability 0.9. If it comes
out tail I will buy a motorcycle with probability 0.1

There are three variables:

The chosen coin C = {f , t}
The flip outcome F = {h, t}
My decision M = {yes, no}

Javier Larrosa (UPC Barcelona Tech) BN . 17 / 23

Example

P(M, F , C ) = P(M|F )P(F |C )P(C )

F M P(M|F ) C F P(F |C ) C P(C )

h yes 0.9 f h 0.5 f 0.5
h no 0.1 f t 0.5 t 0.5
t yes 0.1 t h 1
t no 0.9 t t 0

Javier Larrosa (UPC Barcelona Tech) BN . 18 / 23

Example: unnatural factorization

P(M, F , C ) = P(C |F )P(F |M)P(M)

F C P(C |F ) M F P(F |M) M P(M)

h f yes h yes
h t yes t no
t f no h
t t no t

Javier Larrosa (UPC Barcelona Tech) BN . 19 / 23

Example: unnatural and larger factorization

P(M, F , C ) = P(F |M, C )P(M|C )P(C )

M C F P(F |M, C ) C M P(M|C )

yes f h f yes
yes f t f no
yes t h t yes
yes t t t no
no f h
no f t C P(C )
no t h f 0.5
no t t t 0.5

Javier Larrosa (UPC Barcelona Tech) BN . 20 / 23

Some BN are more natural than others: Example

It is known that people from Northern Europe are more likely to be

tall and blond than people from Southern Europe
Different factorizations:
P(N, B, T ) = P(N|B, T )P(B|T )P(T )
P(N, T , B) = P(N|T , B)P(T |B)P(B)
P(B, T , N) = P(B|T , N)P(T |N)P(N)
...
Think about what info you are more likely to estimate

Javier Larrosa (UPC Barcelona Tech) BN . 21 / 23

Exercise

.
Proof that the join distribution that a bayesian network represents,
Y
P(X1 , X2 , . . . , Xn ) = P(Xi |parents(Xi ))
i

is really a join distribution

That is, proof,

Q
For all x1 , x2 , ..., xn we have that 0 ≤ i P(xi |parents(xi )) ≤ 1
P Q
x1 ,x2 ,...,xn i P(xi |parents(xi )) = 1

Javier Larrosa (UPC Barcelona Tech) BN . 22 / 23

Exercise

We turn on the radio and they are broadcasting a basketball game.

Somebody in our favorite team is shooting 3 free shots, but we could not
hear who it was. What is our best guess and how do we update it
depending on how the shots go? (for simplicity, let suppose that the team
is made of three players and they are all on court)
The free throws numbers of the three players are: p1 : 90/100,
p2 : 100/200,p3 : 200/1000,

Javier Larrosa (UPC Barcelona Tech) BN . 23 / 23