Chapter 15

Tests of Significance:
The Basics
09/19/22 Basics of Significance Testing 1
 Significance Testing
• Also called “hypothesis testing”
• Objective: to test a claim about parameter
μ
• Procedure:
A.State hypotheses H0 and Ha
B.Calculate test statistic
C.Convert test statistic to P-value and interpret
D.Consider significance level (optional)
09/19/22 Basics of Significance Testing 2
 Hypotheses
• H0 (null hypothesis) claims “no difference”
• Ha (alternative hypothesis) contradicts the null
• Example: We test whether a population gained
weight on average…
H0: no average weight gain in population
Ha: H0 is wrong (i.e., “weight gain”)
• Next  collect data  quantify the extent to
which the data provides evidence against H0

Basic Biostat 9: Basics of Hypothesis Testing 3

 One-Sample Test of Mean
• To test a single mean, the null hypothesis is
H0: μ = μ0, where μ0 represents the “null value” (null
value comes from the research question, not from data!)
• The alternative hypothesis can take these forms:
Ha: μ > μ0 (one-sided to right) or
Ha: μ < μ0 (one-side to left) or
Ha: μ ≠ μ0 (two-sided)
• For the weight gain illustrative example:
H0: μ = 0
Ha: μ > 0 (one-sided) or Ha: μ ≠ μ0 (two-sided)
Note: μ0 = 0 in this example
09/19/22 Basics of Significance Testing 4
 Illustrative Example: Weight Gain
• Let X ≡ weight gain Figure: Two possible
• X ~N(μ, σ = 1), the xbars when H0 true
value of μ unknown
• Under H0, μ = 0
• Take SRS of n = 10
• σx-bar = 1 / √(10) =
0.316
• Thus, under H0
x-bar~N(0, 0.316)
09/19/22 Basics of Significance Testing 5
 One-Sample z Statistic
Take an SRS of size n from a Normal population.
Population σ is known. Test H0: μ = μ0 with:

x  μ0 x  μ0
z Equivalently z 
σ x
n
For “weight gain” data, x-bar = 1.02, n = 10, and σ = 1
x  μ0 1.02  0
z   3.23
σ 1
n 10

09/19/22 Basics of Significance Testing 6

 P-value
• P-value ≡ the probability the test statistic would
take a value as extreme or more extreme than
observed test statistic, when H0 is true
• Smaller-and-smaller P-values → stronger-and-
stronger evidence against H0
• Conventions for interpretation
P > .10  evidence against H0 not significant
.05 < P ≤ .10  evidence marginally significant
.01 < P ≤ .05  evidence against H0 significant
P ≤ .01  evidence against H0 very significant
09/19/22 Basics of Significance Testing 7
 P-Value
Convert z statistics to P-value :
• For Ha: μ> μ0
P = Pr(Z > zstat) = right-tail beyond zstat

• For Ha: μ< μ0

P = Pr(Z < zstat) = left tail beyond zstat
• For Ha: μμ0
P = 2 × one-tailed P-value

09/19/22 Basics of Significance Testing 8

 Illustrative Example
• z statistic =
3.23
• One-sided
P = P(Z > 3.23)
= 1−0.9994 =
0.0006
• Highly
significant
evidence
against H0

09/19/22 Basics of Significance Testing 9

 Significance Level
• α ≡ threshold for “significance”
• We set α
• For example, if we choose α = 0.05, we require
evidence so strong that it would occur no more
than 5% of the time when H0 is true
• Decision rule
P ≤ α  statistically significant evidence
P > α  nonsignificant evidence
• For example, if we set α = 0.01, a P-value of
0.0006 is considered significant

09/19/22 Basics of Significance Testing 10

Summary

09/19/22 Basics of Significance Testing 11

Illustrative Example: Two-sided test
1. Hypotheses: H0: μ = 0 against Ha: μ ≠ 0

x  μ0 1.02  0
2. Test Statistic: z    3.23
σ 1
n 10

3. P-value: P = 2 × Pr(Z > 3.23) = 2 × 0.0006 =

0.0012
Conclude  highly significant evidence against
H0
09/19/22 Basics of Significance Testing 12
 Relation Between Tests and CIs
• For two-sided tests, significant results at the α-
level  μ0 will fall outside (1–α)100% CI
• When α = .05  (1–α)100% = (1–.05)100% =
95% confidence
• When α = .01, (1–α)100% = (1–.01)100% =
99% confidence
• Recall that we tested H0: μ = 0 and found a two-
sided P = 0.0012. Since this is significant at α
= .05, we expect “0” to fall outside that 95%
confidence interval … continued …
09/19/22 Basics of Significance Testing 13
 Relation Between Tests and CIs
Recall: xbar = 1.02, n = 10, σ = 1. Therefore, a 95%
CI for μ =
 σ 1
x z  1.02  1.96
n 10
 1.02  0.62
 0.40 to 1.64
Since 0 falls outside this 95% CI  the test of H0: μ
= 0 is significant at α = .05
09/19/22 Basics of Significance Testing 14
 Example II: Job Satisfaction
Does the job satisfaction of assembly workers differ when
their work is machine-paced rather than self-paced? A
matched pairs study was performed on a sample of
workers. Workers’ satisfaction was assessed in each
setting. The response variable is the difference in
satisfaction scores, self-paced minus machine-paced.
The null hypothesis “no average difference” in the
population of workers. The alternative hypothesis is “there
is an average difference in scores” in the population.
H0:  = 0 Ha:  ≠ 0
This is a two-sided test because we are interested in
differences in either direction.
09/19/22 Basics of Significance Testing 15
 Illustrative Example II
 Job satisfaction scores follow a Normal
distribution with standard deviation  = 60.
 Data from 18 workers gives a sample mean
difference score of 17.
 Test H0: µ = 0 against Ha: µ ≠ 0 with

x  μ017  0
z   1.20
σ 60
n 18

09/19/22 Basics of Significance Testing 16

 Illustrative Example II
• Two-sided P-value
= Pr(Z < -1.20 or Z > 1.20)
= 2 × Pr (Z > 1.20)
= (2)(0.1151) = 0.2302
• Conclude: 0.2302
chance we would see
results this extreme
when H0 is true 
evidence against H0 not
strong (not significant)
09/19/22 Basics of Significance Testing 17
 Example II: Conf Interval Method
Studying Job Satisfaction
A 90% CI for μ is
 σ 60
xz  17  1.645  17  23.26
n 18
 6.26 to 40.26

This 90% CI includes 0. Therefore, it is plausible

that the true value of  is 0  H0: µ = 0 cannot be
rejected at α = 0.10.
09/19/22 Basics of Significance Testing 18