Professional Documents
Culture Documents
of
Computa%onal
Journalism
Columbia
Journalism
School
Week
10:
Drawing
Conclusions
from
Data
November
19,
2012
How likely is it that the temperature won't increase over next decade?
It is conceivable that the 14 elderly people who are reported to have died soon aPer receiving the vaccina%on died of other causes. Government ocials in charge of the program claim that it is all a coincidence, and point out that old people drop dead every day. The American people have even become familiar with a new sta%s%c: Among every 100,000 people 65 to 75 years old, there will be nine or ten deaths in every 24-hour period under most normal circumstances. Even using the ocial sta%s%c, it is disconcer%ng that three elderly people in one clinic in PiYsburgh, all vaccinated within the same hour, should die within a few hours thereaPer. This tragedy could occur by chance, but the fact remains that it is extremely improbable that such a group of deaths should take place in such a peculiar cluster by pure coincidence. - New York Times editorial, 14 October 1976
Assuming that about 40 percent of elderly Americans were vaccinated within the rst 11 days of the program, then about 9 million people aged 65 and older would have received the vaccine in early October 1976. Assuming that there were 5,000 clinics na%onwide, this would have been 164 vaccina%ons per clinic per day. A person aged 65 or older has about a 1-in-7,000 chance of dying on any par%cular day; the odds of at least three such people dying on the same day from among a group of 164 pa%ents are indeed very long, about 480,000 to one against. However, under our assump%ons, there were 55,000 opportuni%es for this extremely improbable event to occur 5,000 clinics, mul%plied by 11 days. The odds of this coincidence occurring somewhere in America, therefore, were much shorter only about 8 to 1 - Nate Silver, The Signal and the Noise, Ch. 7 footnote 20
Given:
R
=
49%,
O=47%
MOE(R)
=
MOE(O)
=
5.5%
How likely is it that Obama is actually ahead? Let D = R-O = 2%. This is an observed value, and if we polled the whole
popula%on, we would see a true value D'. We want to know probability that Obama is actually ahead, i.e. P(D' < 0) Margin of error on D MOE(R) + MOE(D) = 11% because they are almost completely dependent, R+O 100. For beYer analysis, see hYp://abcnews.go.com/images/PollingUnit/MOEFranklin.pdf Gives MOE(D) = 10.8%
P(Obama ahead)
P(Romney ahead)
Std. dev of D MOE(D)/1.96 as MOE is quoted as 95% condence interval = 5.5%. Z-score of -D = -2%/5.5% = -0.36 P(z<-0.35) = 0.36, so 36% chance a Romney is not ahead, or about 1 in 3.
Random
Happens
"Unlikely
to
happen
by
chance"
is
only
a
good
argument
if
you've
es%mated
the
chance.
Also:
a
par9cular
coincidence
may
be
rare,
but
some
coincidence
somewhere
occurs
constantly.
Condi%onal
probabili%es
Pr(posi%ve|cancer)
=
75%
Pr(posi%ve|no
cancer)
=
10%
What
is
Pr(cancer|posi%ve)?
Bayes
Theorem
Tells
us
how
to
go
from
Pr(A|B)
to
Pr(B|A)
Pr(B|A)
=
Pr(A|B)Pr(B)
/
Pr(A)
Bayesian
Mammograms
Pr(cancer|posi%ve)
=
Pr(posi%ve|cancer)
Pr(cancer)
/
Pr(posi%ve)
Pr(posi%ve|cancer)
=
0.75
Pr(cancer)
=
0.014
Pr(posi%ve)
=
Pr(posi%ve|no
cancer)Pr(no
cancer)
+
Pr(posi%ve|cancer)Pr(cancer)
=
0.10*0.986
+
0.75*0.014
=
0.1091
Bayesian
Mammograms
Pr(cancer|posi%ve)
=
Pr(posi%ve|cancer)
Pr(cancer)
/
Pr(posi%ve)
=
(0.75
*
0.014)
/
(0.1091)
=
0.0962
=
9.6%
chance
she
has
cancer
if
mammogram
is
posi%ve
cancer no cancer
posi%ve
nega%ve
cancer
no
cancer
Pr(posi%ve|cancer)
=
0.75
=
N(posi%ve
&
cancer)
/
N(cancer)
N(cancer)
=
4
N(posi%ve
&
cancer)
=
3
posi%ve
nega%ve
cancer
no
cancer
Pr(posi%ve|no
cancer)
=
0.1
=
N(posi%ve
&
no
cancer)
/
N(posi%ve)
N(no
cancer)
=
1000
N(posi%ve
&
no
cancer)
=
100
posi%ve
nega%ve
cancer
no
cancer
Pr(cancer)
0.0014
=
N(cancer)
/
N
posi%ve
nega%ve
cancer no cancer
Pr(cancer|posi%ve) = 9.6%
posi%ve
nega%ve
Cogni%ve
biases
Availability
heuris%c:
we
use
examples
that
come
to
mind,
instead
of
sta%s%cs.
Preference
for
earlier
informa%on:
what
we
learn
rst
has
a
much
greater
eect
on
our
judgment.
Memory
forma%on:
whatever
seems
important
at
the
9me
is
what
gets
remembered.
Conrma%on
bias:
we
seek
out
and
give
greater
importance
to
informa%on
that
conrms
our
expecta%ons.
Conrma%on
bias
Comes
in
many
forms.
...unconsciously
ltering
informa%on
that
doesn't
t
expecta%ons.
...not
looking
for
contrary
informa%on.
...not
imagining
the
alterna%ves.
Go looking for informa%on that gives you the best ability to discriminate between hypotheses. Evidence which supports Hi is much less useful than evidence which supports Hi much more than Hj, if the goal is to choose a hypothesis.
A good model has a theory of the world. Bad models, bad inferences
For each new piece of evidence, use Bayes' rule to update probability on all hypotheses. Inference result is probabili%es of dierent hypotheses given all evidence { P(H0|E), P(H1|E), ... , P(HN|E) }
0 H0 H1 H2
0 H0 H1 H2
What
is
"causa%on"?
Y X thing
in
the
world
interac%on
observable
thing
Z X Y X Y
Z causes X and Y
random chance!
A
dicult
example
NYPD
performs
~600,000
street
stop
and
frisks
per
year.
What
sorts
of
conclusions
could
we
draw
from
this
data?
How?