Professional Documents
Culture Documents
Hypothesis Testing
Fall
2013
Prof.
Yao
Xie,
yao.xie@isye.gatech.edu
H.
Milton
Stewart
School
of
Industrial
Systems
&
Engineering
Georgia
Tech
Midterm 1 Score
• 46
students
• Highest
score:
98
• Lowest
score:
34
• Mean:
70.2
• Median:
72
Outline
• General
concept
• Type
of
errors
and
power
• z-‐test
for
the
mean,
variance
known
• t-‐test
for
the
mean,
variance
unknown
• test
for
sample
proporTon
• Confidence
interval
and
hypothesis
test
• P-‐value
Hypotheses Testing
The hypothesis test has two possible but contradictory
truths, written in terms of Hypotheses:
H0 Null Hypothesis
• The prior belief with the data.
H1 Alternative Hypothesis
• The alternative belief we have with the data
• Varies from problem to problem
Goal of hypothesis testing is to use data to tell which
hypothesis is true.
Mathematical formulation
• For
example,
comparing
the
mean
and
variance
of
samples
from
normal
distribuTon
Null
Hypothesis
" H 0 : µ = µ0 "$ H 0 : σ 2 = σ 02
# #
AlternaTve
Hypothesis
$ H1 : µ ≠ µ 0 $% H1 : σ 2 ≠ σ 02
(2) Apple produces a batch of iPhones, with battery life (X) with
mean µ , variance = σ 2
Is the variability of battery under control?
Here the variability is under control when it is
smaller than a known value σ 02 ⎧⎪ H 0 : σ 2 ≤ σ 02
⎨
⎪⎩ H 1 : σ > σ 0
2 2
8
Two-sided versus one-sided
Two-‐sided
AlternaTve
Hypothesis
H 0 : µ = 10 H 0 : µ = 100
H1 : µ < 10 H1 : µ > 100
Composite Hypotheses
Testing a range H 0 : µ = 10
of values H1 : µ < 10
X: customers’ waiting Average diameter
time in a bank of screw
Outline
• General
concept
• Type
of
errors
and
power
• z-‐test
for
the
mean,
variance
known
• t-‐test
for
the
mean,
variance
unknown
• test
for
sample
proporTon
• Confidence
interval
and
hypothesis
test
• P-‐value
Errors in Hypothesis Testing
α = P(type I error) = P(Reject H0, when H0 is true)
This error rate can be set low enough to
ensure the test is “safe”.
β = P(type II error) = P(Accept H0, when H1 is true)
truth
H0 is True H0 is False
Correct
Accept H0 Type II Error
Decision
decision
Correct
Reject H0 Type I Error
Decision
12
Court room
Suppose you are the prosecutor in a
courtroom trial. The defendant is
either guilty or not. The jury will
either convict or not.
truth
Not Guilty Guilty
μ
2. Test H0: µ = 160 vs. H1: µ < 160. What is P(Reject H0 | µ = 160)?
A. Type I error B. Type II error C. Power
3. Test H0: µ = 160 vs. H1: µ < 160. What is P(Reject H0 | µ < 160)?
A. Type I error B. Type II error C. Power 16
Outline
• General
concept
• Type
of
errors
and
power
• z-‐test
for
the
mean,
variance
known
• t-‐test
for
the
mean,
variance
unknown
• test
for
sample
proporTon
• P-‐value
One-sided z-test
H 0 : µ = 160
H1 : µ < 160
The conventional way to test this hypothesis is to find the test for
which the type-I error is fixed at a particular value (e.g., α = 0.01,
0.05, 0.10).
b-160
α = P(X<b|µ = 160) = P(Z< )
σ n
b-160
This determines b because ≡ Φ −1 (α )
σ n
b= σ ( n Φ ) −1
(α ) + 160
19
Numerical values
b= σ ( )
n Φ −1 (α ) + 160
H 1 : µ ≠ 160
x ∉[160 − k,160 + k]
Reject H0 when
k
α = P( x -160 > k|µ = 160) = P( Z > )
σ n
k α = 0.05, z0.025 = 1.96
= zα /2
σ n σ = 10, n = 100, k = zα /2 σ n = 1.96
H 0 : µ = 160
H 1 : µ ≠ 160
27
Formulation: coke machine
H 0 : µ = 200
H 1 : µ ≠ 200
x − 200
Test
staTsTcs
s/ n
Threshold
k = t 0.025,8 = 2.306
x − 200
Reject
H0
when
> 2.306
s/3
x − 200 210 − 200
With
our
data,
= = 1 < 2.306
s/ n 30 / 3
So
machine
is
running
OK.
Outline
• General
concept
• Type
of
errors
and
power
• z-‐test
for
the
mean,
variance
known
• t-‐test
for
the
mean,
variance
unknown
• test
for
sample
proporTon
• Confidence
interval
and
hypothesis
test
• P-‐value
Test on population proportion
Sample proportion, X ~ BIN(n, p) ⎧ H 0 : p = p0
⎨
⎩ H 1 : p ≠ p0
Based on CLT approximation, n > 30
p̂ − p0 0.31− 0.2
= = 17.74>1.96
p0 (1− p0 ) / n 0.2 (1− 0.2 ) / 4115
Reject
the
null
hypothesis:
the
proporTon
is
not
0.2
(very
likely
to
be
greater
than
0.2)
Summary
of
Tests
Summary: test for mean
Null
Hypothesis
Test
StaTsTc
H 0 : µ = µ0 x
Significance
level:
α
H 0 : p = p0 p̂ − p0
p0 (1− p0 ) / n
Significance
level:
α
H 1 : p ≠ p0 p̂ − p0
> zα /2
p0 (1− p0 ) / n
Outline
• General
concept
• Type
of
errors
and
power
• z-‐test
for
the
mean,
variance
known
• t-‐test
for
the
mean,
variance
unknown
• test
for
sample
proporTon
• p-‐value
• Confidence
interval
and
hypothesis
test
Motivation for p-value
Think
about
the
GT
student
average
weight
example.
Assume
average
weight
is
160lb.
Based
on
the
test,
we
reject
H0
if
sample
average
is
less
than
158.35,
and
this
gives
significance
level
0.05
(i.e.
chance
of
making
type
I
error
=
0.05)
Consider
case1:
sample
average
=
150lb
case2:
sample
average
=
155lb
In
both
cases
we
reject
H0
However,
if
sample
sample
average
=
150lb,
we
should
reject
H0
with
more
confidence
than
if
sample
average
=
155lb.
37
In
other
words,
when
sample
average
=
150
lb,
the
chance
for
H0
to
happen
is
smaller.
p-value
• p-‐value
=
probability
of
observing
some
data
even
more
“extreme”
than
the
given
data
• The
smaller
the
p-‐value
is,
the
less
likely
H0
is
true
Reject
H0
when
p-‐value
is
small
Test
can
also
be
performed
as
rejecTng
H0
when
p-‐value
is
less
than
prescribed
significant
level
Convention: decisions based on P-value
• If P-value < 0.01
• H0 is not plausible/ H1 is supported
H 0 :µ = µ0
H1:µ > µ0
X-µ0
Test statistic: Z =
σ/ n
observed value of test statistic: x
x-µ0
p-value: P(X> x)=P(Z> )
σ/ n
⎛ x-µ0 ⎞
= 1− Φ ⎜
⎝ σ / n ⎟⎠
Example: p-value for one-sided Test
Claim:
mean
baoery
life
of
a
cellphone
is
9
hours.
Observed
mean
baoery
life:
8.5
hours,
from
44
observaTons,
σ
=
1
hour
Test:
the
mean
baoery
life
of
a
cellphone
exceeds
10
hours
X-µ0
H 0 :µ = 9 Test statistic: Z = σ / n
H1:µ < 9 observed value of test statistic: x=8.5
x-µ0
p-value: P(X< x)=P(Z< )
σ/ n
⎛ 8.5-9 ⎞
= Φ⎜ ⎟ = 0.0095
⎝ 1 44 ⎠
Example: p-value nicotine content
A cigarette manufacturer claims that the average nicotine
content of a brand of cigarettes is at most 1.5.
0.3
.121
= P( |Z| > ) 0.2
.325 13 0.1
= 2Φ(-1.34) = 0.18
Example: p-value for two-sided test
Test to see if the mean is significantly far from 90. The
sample mean is 87.9 with known standard deviation of
5.9 for a sample of size 44.
0.2
0.1
-3 -2 -1 1 2 48
3
Outline
• General
concept
• Type
of
errors
and
power
• z-‐test
for
the
mean,
variance
known
• t-‐test
for
the
mean,
variance
unknown
• test
for
sample
proporTon
• P-‐value
• Confidence
interval
and
hypothesis
test
Connection between confidence
Intervals and Hypothesis Test
CI :
X -k X X + k
If X not in [ µ0 − k, µ0 + k] , reject Ho
Under H0, making error with probability 1− α
More
Examples
Two types of approaches to hypothesis
testing
Fixed
significance
level
p-‐value
Decide
decision
staTsTcs
Calculate
p-‐value:
Reject
H0,
when
what
is
the
probability
to
Decision
staTsTc
><
observe
a
staTsTc
more
threshold
“extreme”
than
data
The
smaller
p-‐value
is,
the
more
likely
H1
is
true
à
reject
H0
Procedure of hypothesis test
(sec. 9.1.6)
1. Set
the
significance
level
(.01,
.05,
.1)
2. Set
null
and
alternaTve
hypothesis
3. Determine
other
parameters
4. Decide
type
of
the
test
-‐
test
for
mean
with
known
variance
(z-‐test)
-‐
test
for
mean
with
unknown
variance
(t-‐test)
-‐
test
for
sample
proporTon
parameter
6. Use
data
available:
-‐
perform
test
to
reach
a
decision
-‐
and
report
p-‐value
Example 1: change of average height?
z-test
The average height of females in the freshman class
at GT has been 162.5 cm with a a standard
deviation of 6.9 cm.
54
Solution to example 1
1. Significance
level:
α = 0.05
2. Set
null
and
the
alternaTve
H 0 : µ = 162.5
H 1 : µ ≠ 162.5
3. Determine
other
parameters:
4. Decide
type
of
test:
n = 50, σ = 6.9
known
variance,
test
for
mean,
use
z-‐test
Reject
H0
if
x − µ0 > zα 2σ / n
Plug
in
values:
z
0.025 = 1.96
Reject
H0
if
x − 162.5 > 1.96 × 6.9 / 50 = 1.9126
5.
Use
data
to
perform
test:
x = 165.2, x − 162.5 = 2.7 > 1.9126 Reject
H
0
6.
Report
p-‐value:
p-value
= P(| Z | > |165.2 − 162.5 | /(6.9 / 50 ))
= P( |Z| > 2.7669)
= 2(1− Φ(2.7669)) = 0.0056
Example 2: effective hours of drug
t-test
In a study for the effective hours of certain drug, the
sample average time for n = 9 individual was 6.32
hours and the sample standard deviation was 1.65
hours.
57
Solution to example 2
1. Significance
level:
α = 0.05
2. Set
null
and
the
alternaTve
H0 : µ = 7
H : µ ≠ 7
1
3. Determine
other
parameters:
n
= 9, s = 1.65
4. Decide
type
of
test:
unknown
variance,
test
for
mean,
use
t-‐test
Reject
H0
if
x − µ0 > tα 2,n−1s / n
Plug
in
values:
t
0.025,8 = 2.306
Reject
H0
if
x − 7 > 2.306 × 1.65 / 9 = 1.2683
5.
Use
data
to
perform
test:
x = 6.32, x − 7 = 0.68 < 1.2683 Accept
H
0
60
Summary: test for sample proportion
Null
Hypothesis
Test
StaTsTc
H 0 : p = p0 p̂ − p0
p0 (1− p0 ) / n
Significance
level:
α
H 1 : p ≠ p0 p̂ − p0
> zα /2
p0 (1− p0 ) / n
Solution to example 3
1. Significance
level:
α = 0.05
2. Set
null
and
the
alternaTve
H 0 : p = 0.33
H 1 : p ≠ 0.33
3. Determine
other
parameters:
n = 85
4. Decide
type
of
test:
sample
proporTon
test
Reject
H0
if
p̂ − p0
> zα /2
( )p0 1− p0 / n
Plug
in
values:
z0.025 = 1.96
Reject
H0
if
p̂ − 0.33
> 1.96
0.33(1− 0.33) / 85
Solution
to example 3
5.
Use
data
to
perform
test:
p̂ = 26 / 85 = 0.3059
0.3059 − 0.33
= 0.4725 < 1.96 Accept
H0
0.33(1− 0.33) / 85
• Examples:
– Test for normal mean, variance known, use
z-test
– Test for normal mean, variance unknown,
use t-test
– Test for sample proportion, using normal
approximation and z-test