Making Sense of Statistical Prepared by:

Significance Jhunar John M. Tauy, RPm

(Decision Errors, Effect Size, and Statistical Power)
 Decision Errors

• incorrect conclusions in
hypothesis testing in relation to
the real (but unknown) situation,
such as deciding the null
hypothesis is false when it is
really true.
 Decision Errors
Type I Error
• rejecting the null hypothesis
when in fact it is true
• getting a statistically significant
result when in fact the research
hypothesis is not true
 Decision Errors
Type II Error
• failing to reject the null when in
fact it is false
• failing to get a statistically
significant result when in fact the
research hypothesis is true
Decision Errors
 Effect Size

• standardized measure of
difference (lack of overlap)
between populations
• it increases with greater
differences between means
Effect Size: M = 220
Effect Size: M = 210
 Figuring the
Effect Size

𝜇1 − 𝜇 2
• 

𝑑=
𝜎
 Effect Size
Conventions
• standard rules about what to
consider a small, medium, and
large effect size, based on what is
typical in psychology research;
also known as Cohen’s
conventions.
 Effect Size
Conventions
Effect Size Conventions
 Meta-Analysis

• statistical method for

combining effect sizes from
different studies
 Statistical
Power

• probability that the study

will give a significant result if
the research hypothesis is
true.
 Determining
Statistical Power
Power Tables
• table for a hypothesis-
testing procedure showing
the statistical power of a
study for various effect sizes
and sample sizes.
 Determining
Statistical Power
Power Tables
• table for a hypothesis-
testing procedure showing
the statistical power of a
study for various effect sizes
and sample sizes.
 What Determines
the Power of a
Study?
• 
1. Effect Size
Determining Power from The
Predicted Effect Sizes
Predicted µ1 = µ2 + (d)(
 What Determines
the Power of a
Study?
2. Sample Size
Figuring the Sample Size based on
Power
• Begin with the level of power
• Figure how many participants you need
to get that level of power using the
formula in determining the power.
(better use a power table)
 Other Influences on
Power

1. Significance level (alpha)

2. One- versus two-tailed
Tests
3. Type of hypothesis-testing
procedure
 Role of Power When
Planning a Study
1. Increase effect size by
increasing the predicted
difference between
population means.
2. Increase effect size by
decreasing the population
standard deviation
 Role of Power When
Planning a Study
3. Increase the sample size
4. Use a less extreme level of
significance
5. Use a one-tailed test.
6. Use a more sensitive
hypothesis-testing procedure
Practical Ways of Raising Power
 Role of Power When
Interpreting the
Results of a Study

• Is it statistically significant or
clinically/practically significant?
 Role of Power When a
Result is Not
Statistically Significant
• A nonsignificant result from a study
with low power is truly inconclusive.
• A nonsignificant result from a study
with high power does suggest either
that the research hypothesis is false
or that there is less of an effect than
was predicted when figuring power.
Role of Power When Evaluating Results of a
Study
 Advanced Topic:
Figuring the
Statistical Power
1. Gather the needed information: the
mean and standard deviation of
Population 2’s distribution of means
(the comparison distribution) and
the predicted mean of Population
1’s distribution of means (the
population that is given the
experimental procedure).
 Advanced Topic:
Figuring the
Statistical Power
2. Figure the raw-score cutoff point
on the comparison distribution to
reject the null hypothesis.
3. Figure the Z score for this same
point, but on the distribution of
means for the population that
receives the experimental
procedure (Population 1).
 Advanced Topic:
Figuring the
Statistical Power
4. Using the normal curve
table, figure the
probability of getting a
score more extreme than
that Z score.
 Example

In a planned study, the population is

known to have a mean of 500 and a
standard deviation of 100. The researchers
will give the experimental treatment to 60
people and predict that the mean for
those 60 will be 540. They will use the .05
significance level. (a) Figure the power of
this study and (b) sketch the distributions
involved. (c) What is beta in this study?