Announcements

• Teaching assistants:
– Feray Tuncalp, Mert Gurel, Onur Demiray, Syed Adil Abbas
Kazmi, Ibrahim Pehlivan
• Quiz 1: next class on October 8th, Tuesday at 14:30-14:50.
– Make sure to be in class on time!
– All subjects covered in homework 1 and 2 + independence.
• Make-up lecture for last Thursday
– Next Thursday, October 10 at 8:30 in SOSB08
– The time conflicts with some of your programs. It will be
videotaped and uploaded to BB.
• Homework 1&2 are on BB. Homework 3 will be on by
tomorrow.
• Office hours: on BB
1
ENGR 200
Lecture 5: Agenda
1. Review:
a) Conditional probability: total probability
law and Bayes’ rule
b) Independence
2. Conditional independence
3. Independence of many events

2
 Definition of Conditional
Probability

A, B : two events
P(A|B) : probability that event A occurs
given event B has already occurred

P(A  B)
P( A | B)  A

P(B) B

3
 The law of total probability
Let A1, A2, … Ak be mutually exclusive
and exhaustive (i.e.,  Ai =S) and B be
an arbitrary event in S. P(Ai)’s and P(B|
Ai)’s are known. Then the total
probability of B is given by:
k
P ( B )   P ( B | Ai ) P ( Ai )
i 1
 Bayes theorem

Let A1, A2, … Ak be mutually exclusive and

exhaustive (i.e.,  Ai =S) and B be an
arbitrary event in S. Then for j=1,…,k:
PA j | B   ?

P( A j  B) P B | A j PA j 
PA j | B    k
j  1,2,..., k
P( B)
 PB | A PA 
i 1
i i
 Conditional Probability in
Pictures
N : event of no virus
P(N) = 0.99
V : event of virus
+: event that the test shows virus P(+|V) = 0.90
- : event that the test shows no virus P(+|N) = 0.10

Random person comes to doctor

Has virus Does not have virus

Test shows Test shows Test shows Test shows

virus no virus virus no virus

Interesting Questions:
• Does the person have the virus?
• Does his test result show that he has virus?
• Does the person have the virus when his test result shows that he has virus?
Conditional Probability in
Pictures
After the
test Probability that the person really
shows has the virus given that the test
virus shows virus?

Random person comes to doctor

Has virus Does not have virus

Test shows Test shows Test shows Test shows

virus no virus virus no virus

New Sample Space

 Conditional Probability in
Pictures
N : event of no virus
P(V | +) = ?
V : event of virus
+ : event that the test shows virus
- : event that the test shows no virus
After the
P(N) = 0.99 test
P(+|V) = 0.90 shows
P(+|N) = 0.10 virus

Random person comes to doctor

Has virus Does not have virus

Test shows Test shows Test shows Test shows

virus no virus virus no virus
 Class exercise
• A first step towards identifying spam is to create a list of
words that are more likely to appear in spam than in
normal messages. For instance, words like buy or the
brand name of an enhancement drug are more likely to
occur in spam messages than in normal messages.
• Suppose a specified list of words is available and that your
data base of 5000 messages contains 1700 that are spam.
Among the spam messages, 1343 contain words in the list.
Of the 3300 normal messages, only 297 contain words in
the list.
• Let:
– N be the event that the message is normal,
– S be the event that the message is spam,
– A be the event a message is identified as spam (i.e., the
message contains words in the list.
9
 Class exercise
• Find the probabilities of the above
events, using the above information,
i.e., find P(S), P(N) and P(A).

P(S)= P(N)=

10
 Class exercise
• Find the probability that a message
is spam given that it is identified as
spam (it contains words in the list),
i.e., find the conditional probability
P(S|A).

11
 Two Events are
Independent If…
• When knowing one event does not
alter the probability of the other.

• P(A|B) = P(A)
• P(B|A) = P(B)

• P(AB) = P(A) P(B)

Example

13
 Conditional Independence
• Two events do not say anything about each other
GIVEN a third:
• P(A∩B | C) = P(A|C) P(B|C)
• P(A | B∩C) = P(A|C)
• P(B | A∩C) = P(B|C)
• Note: everything is the same except the C.

In words, this states that if C is Note: Independence of two

known to have occurred, the
additional knowledge that B also
occurs does not change the
conditional probability of A, if A&B
are conditionally independent!
events does not imply
conditional independence,
and vice versa!
 Adana'da ''yılan bebek''
How do you
explain this
dünyaya geldi
news?
Adana’da, milyonda bir rastlandığı bildirilen ve "yılan
bebek" olarak bilinen bir kız çocuğu dünyaya geldi. 2 kilo
900 gram ağırlığındaki bebek kuvözde bakıma alındı.
…
Baba A. K. ise ilk çocuklarının da aynı şekilde, ancak ölü
doğduğunu ifade etti. Baba A. K.:
"İlk kız çocuk da son kız çocuk da böyle oldu. İki erkek
çocuğumuzda herhangi bir sorun yok.
Doktorlar, kız çocukta genetik olarak böyle bir hastalık
oluştuğunu söylüyorlar" dedi.

www.milliyet.com.tr, 27 Eylül 2007 / Perşembe - 11:00

 Are these conditionally
independent?
• Consider two coin tosses:
– A = { 1st toss is a head }
– B = { 2nd toss is a head }

given

– D = { the tosses have different results }

 Are these events
conditionally independent?
• Blue coin: P(H) = 0.99
• Red coin: P(H) = 0.01
• Pick a random coin, toss it twice.
• H1 = { 1st toss is heads }
• H2 = { 2nd toss is heads }
– given
• B = { the blue coin was selected }
Independence of Multiple
Events

A1 , A2 ,..., An are independen t if

P(  Ai )   P( Ai )
iG iG

for every subset G of {1,2,..., n}

 Example: Three Events

P( A1  A2 )  P( A1 ) P( A2 )
P( A1  A3 )  P( A1 ) P( A3 ) Pairwise
Independence

P( A2  A3 )  P( A2 ) P( A3 )
P( A1  A2  A3 )  P( A1 ) P( A2 ) P( A3 )
 Are these independent?
• Consider two fair coin tosses:
– H1 = { 1st toss is a head }
– H2 = { 2nd toss is a head }
– D = { the two tosses are different }
Pairwise Independence does
not Imply Independence
• P(D|H1) = P(H1∩D)/P(H1) = ½ = P(D)

• P(D|H2) = P(H2∩D)/P(H2) = ½ = P(D)

• P(H1∩H2) = P(H1) P(H2)

=> Pairwise independence is satisfied

• P(H1∩H2∩D) = 0

• P(H1)P(H2)P(D) = ½ x ½ x ½ = 1/8
=> They are not independent
Example

22
 Are these independent?
• Consider two rolls of a die.
– A = { 1st roll is 1, 2, or 3 }
– B = { 1st roll is 3, 4, or 5 }
– C = { the sum of the two rolls is 9 }
 What did you learn/remember
in this class?
1. Conditional probability
2. Multiplication rule
3. Total probability law
4. Bayes’ rule
5. Conditional independence
6. Independence of many events

24