Hypothesis testing

QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
 Hypothesis testing
• Want to know something about a
population
• Take a sample from that population
• Measure the sample
• What would you expect the sample to
look like under the null hypothesis?
• Compare the actual sample to this
expectation
population

sample
Y = 2675.4
 Hypothesis testing

• Hypotheses are about populations

• Tested with data from samples
• Usually assume that sampling is
random
 Types of hypotheses
• Null hypothesis - a specific statement
about a population parameter made for
the purposes of argument
• Alternate hypothesis - includes other
possible values for the population
parameter besides the value states in
the null hypothesis
The null hypothesis is
usually the simplest
statement, whereas the
alternative hypothesis
is usually the
statement of greatest
interest.
A good null hypothesis would
be interesting if proven wrong.
A null hypothesis is specific;
an alternate hypothesis is not.
 Hypothesis testing: example
Can sheep recognize each other?

QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
 The experiment and the
results
• Sheep were trained to get a reward
near a certain other sheep’s picture
• Then placed in a Y-shaped maze

QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.

You must choose…

Stating the hypotheses
H0: Sheep go to each face with
equal probability (p = 0.5).

HA: Sheep choose one face over

the other (p ≠ 0.5).
 Estimating the value
• 16 of 20 is a proportion of p = 0.8

• This is a discrepancy of 0.3 from the

proportion proposed by the null
hypothesis, p =0.5
 Null distribution

• The null distribution is the sampling

distribution of outcomes for a test
statistic under the assumption that the
null hypothesis is true
Proportion of correct Probability
choices
0 0.000001
1 0.00002
2 0.00018

3 0.0011

4 0.0046

5 0.015

6 0.037

7 0.074
8 0.12

9 0.16
10 0.18

11 0.16

12 0.12

13 0.074

14 0.037

15 0.015

16 0.0046

17 0.0011

18 0.00018

19 0.00002
20 0.000001
• Test statistic = a quantity calculated
from the data that is used to evaluate
how compatable the data are with the
expectation under the null hypothesis
 The null distribution of p
0.2

0.18

0.16

0.14

Test statistic = 16
0.12

0.1

Frequency
0.08

0.06

0.04

0.02

0
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Number of correct choices
 The null distribution of p
0.2

0.18

0.16
Values at least
0.14
as extreme as
0.12 the test statistic
0.1

Frequency
0.08

0.06

0.04

0.02

0
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Number of correct choices
• P-value - the probability of obtaining the
data* if the null hypothesis were true

*as great or greater difference from the null

hypothesis
 P-value
0.2

0.18

0.16
P=
0.012
0.14

0.12

0.1

Frequency
0.08

0.06

0.04

0.02

0
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Number of correct choices
 P-value calculation
P
=2*(Pr[16]+Pr[17]+Pr[18]+Pr[19]+Pr[20])
=2*(0.005+0.001+0.0002+0.00002+0.000001)
= 0.012
 How to find P-values
• Get test statistic
• Compare with null distribution from:
– Simulation
– Parametric tests
– Non-parametric tests
– Re-sampling
 Statistical significance

The sig nifica nce le ve l, α, is a probability used as a criterion

for re jecting t he nullhypothe sis. If the P-value for a test is

le ss than or eq ualto α, the n the nullhypothe sis is re jecte d.

 is often 0.05
 Significance for the sheep
example
• P = 0.012

• P < , so we can reject the null

hypothesis
 Larger samples give more
information
• A larger sample will tend to give and
estimate with a smaller confidence
interval

• A larger sample will give more power to

reject a false null hypothesis
Hypothesis testing: another example
The genetics of symmetry in flowers

QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.

Heteranthera - Mud plantain

Stigma and anthers are asymmetric
in different genotypes
Can the pattern of inheritance
be explained by a single locus
with simple dominance?
Model predicts a 3:1 ratio of right-handed
flowers

H0: Right- and left-handed offspring occur at a

3:1 ratio (the proportion of right-handed
individuals in the offspring population is p = 3/4)
HA: Right- and left-handed offspring do not occur
at a 3:1 ratio (p ≠ 3/4)
 Data

Of 27 offspring, 21 were “right-

handed” and 6 were “left-handed.”
 Estimating the proportion

21
pˆ = = 0.778
27

* The “hat” notation denotes an estimate for a

population parameter from a sample

€
 Sampling distribution of null
hypothesis
Number of right-handed flowers in a random sample
Probability
of 27
0-8 0
9 0.000005
10 0.000031
11 0.000124
12 0.000492
13 0.001752
14 0.005301
15 0.013830
16 0.031094
17 0.060673
18 0.100891
19 0.143051
20 0.172339
21 0.171782
22 0.141034
23 0.091477
24 0.045499
25 0.016409
26 0.003759
27 0.000457
The P-value:

P =
0.83.
Rock-paper-scissors battle

QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.

QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
Jargon
 Significance level
• A probability used as a criterion for
rejecting the null hypothesis
• Called 
• If p < , reject the null hypothesis
• For most purposes,  = 0.05 is
acceptable
 Type I error
• Rejecting a true null hypothesis

• Probability of Type I error is  (the

significance level)
 Type II error
• Not rejecting a false null hypothesis

• The probability of a Type II error is 

• The smaller , the more power a test

has
 Power
• The probability that a random sample of
a particular size will lead to rejection of
a false null hypothesis

• Power = 1- 
 Reality

Ho true Ho false

Result
Reject Ho Type I error correct

Do not reject Ho correct Type II error

 One- and two-tailed tests
• Most tests are two-tailed tests

• This means that a deviation in either

direction would reject the null
hypothesis

• Normally  is divided into /2 on one

side and /2 on the other
 

Test statistic
 One-sided tests
• Also called one-tailed tests
• Only used when one side of the null
distribution is nonsensical

• For example, comparing grades on a

multiple choice test to that expected by
random guessing
 Critical value
• The value of a test statistic beyond
which the null hypothesis can be
rejected
 “Statistically significant”
• P<

• We can “reject the null hypothesis”

We never “accept the null
hypothesis”