CHAPTER EIGHT Confidence Intervals, Effect Size, and Statistical Power

Essentials of Statistics for the Behavioral Sciences

3rd Edition Nolan
Full download at:
Solution Manual:
https://testbankpack.com/p/solution-manual-for-essentials-of-statistics-for-the-
behavioral-sciences-3rd-edition-by-nolan-heinzen-isbn-1464107777-
9781464107771/
Test bank:
https://testbankpack.com/p/test-bank-for-essentials-of-statistics-for-the-behavioral-
sciences-3rd-edition-by-nolan-heinzen-isbn-1464107777-9781464107771/

CHAPTER EIGHT

Confidence Intervals, Effect Size, and Statistical Power

NOTE TO INSTRUCTORS
Students have a tendency to think that if something is statistically
significant, the story is over and that’s all that a person needs to
know. In other words, they frequently confuse “statistically
significant” with “meaningful.” This chapter will help students
recognize that this is not always the case. Aside from using the
CHAPTER EIGHT Confidence Intervals, Effect Size, and Statistical Power

discussion questions and classroom exercises, present examples of

studies that demonstrate a significant difference between groups but
not a very meaningful one. It is also important to break students of
the habit of using phrases such as “very significant” by discussing
effect sizes. Although students might be tempted to describe an
effect as “very significant,” emphasize that they should use effect
sizes for this purpose instead.

OUTLINE OF RESOURCES

I. Confidence Intervals
 Discussion Question 8-1

 Classroom Activity 8-1: Understanding Confidence Intervals

 Discussion Question 8-2

II. Effect Size

 Discussion Question 8-3

 Discussion Question 8-4

 Discussion Question 8-5

III. Meta-Analysis
 Discussion Question 8-6

IV. Statistical Power

 Discussion Question 8-7
 LaunchPad Statistical Applets: Statistical Power
 Classroom Activity 8-1: Calculating Power for Proposed Research

 Discussion Question 8-8

 Classroom Activity 8-2: Working with Confidence Intervals and

Effect Size
 LaunchPad Video Resources

 Additional Readings

 Online Resources

V. Handouts
 Handout 8-1: Classroom Activity: Calculating Power for Proposed

Research
 Handout 8-2: Classroom Activity: Working with Confidence

Intervals and Effect Size

CHAPTER GUIDE
CHAPTER EIGHT Confidence Intervals, Effect Size, and Statistical Power

II. Confidence Intervals

1. A point estimate is a summary statistic from a sample that is
just one number used as an estimate of the population
parameter.
2. Instead of using a point estimate, it is wiser to use an interval
estimate, which is based on a sample statistic and provides a
range of plausible values for the population parameter.
> Discussion Question 8-1
What is the difference between a point estimate and an interval estimate?
Your students’ answers should include:
 A point estimate is a summary statistic from a sample that is just

one number used as an estimate of the population parameter.

Point estimates are useful for gauging the central tendency, but by
themselves can be misleading.
 An interval estimate is based on a sample statistic and provides a

range of plausible values for the population parameter. Interval

estimates are frequently used in media reports, particularly when
reporting political polls.
Classroom Activity 8-1
Understanding Confidence Intervals
The following Web site provides a nice applet to help your students
understand confidence intervals:
http://onlinestatbook.com/stat_sim/conf_interval/index.html
Note that you will need to change the security settings to Java and
add to the exception list the URL for this site, onlinestatbook.com.

The applet simulates a known population mean and standard

deviation and allows you to control the sample size, providing a
graphical display of the resulting confidence intervals.
3. A confidence interval is an interval estimate that includes the
mean we would expect a certain percentage of the time for the
sample statistic were we to sample from the same population
repeatedly.
4. With a confidence interval, we expect to find a mean in this
interval 95% of the time we conduct the same study (if our
confidence level is 95%).
5. To calculate a confidence interval with a z test, we first draw a
normal curve that has the sample mean in the center.
6. We then indicate the bounds of the confidence interval on either
end and write the percentages under each segment of the
curve.
7. Next, we look up the z statistics for the lower and upper ends of
the confidence interval in the z table.
8. We then convert the z statistic to raw means for the lower and
upper ends of the confidence interval. To do so, we first
CHAPTER EIGHT Confidence Intervals, Effect Size, and Statistical Power

calculate the standard error as our measure of spread using the

formula M = /. Then, with this standard error and the
sample mean, we can calculate the raw mean at the upper and
lower end of the confidence interval. For the lower end we use
the formula: MLower = –z(M) + MSample. For the upper end, we
use the formula: MUpper = –z(M) + MSample.
9. Lastly, we should check our answer to ensure that each end of
the confidence interval is exactly the same distance from the
sample mean.
> Discussion Question 8-2
How would you calculate a confidence interval with a z test?
Your students’ answers should include:
To calculate a confidence interval with a z test:
 Draw a normal curve with a sample mean in the center.

 Indicate the bounds of the confidence interval on either end and

write the percentages under each segment of the curve.

 Look up the z statistics for the lower and upper ends of the

confidence interval in the z table.

 Convert the z statistic to raw means for the lower and upper ends

of the confidence interval. For the lower end, use the formula:
MLower = –z(M) + MSample. For the upper end, use the formula:
MUpper = z(M) + MSample.
 Lastly, check the answer to ensure that each end of the
confidence interval is exactly the same distance from the sample
mean.
II. Effect Size
1. Increasing the sample size will lead to an increased test statistic
during hypothesis testing because as N increases the M
decreases. In other words, it becomes progressively easier to
reject H0 and achieve statistical significance as we increase the
sample size.

> Discussion Question 8-3

What happens to statistical significance when sample size increases?
Your students’ answers should include:
 As sample size increases, it becomes much easier to find a
statistical difference.
 As a result, with larger sample sizes, you may be rejecting the
null hypothesis when there really is no statistically significant
difference.

2. An effect size indicates the size of a difference within the

population and is unaffected by sample size.
CHAPTER EIGHT Confidence Intervals, Effect Size, and Statistical Power

3. Effect size tells us how much two populations do not overlap.

Two populations will overlap less if their means are farther apart
and/or the variation within each population is smaller.
4. Statistical significance tells us about where the mean falls in our
sampling distribution, while the effect size tells us where the
mean falls within the population.
> Discussion Question 8-4
What is an effect size, and why would reporting it be useful?
Your students’ answers should include:
 An effect size is a measure of the degree to which groups differ in

the population on the dependent variable.

 It is useful to report the effect size because it provides you with a

standardized value of the degree to which two populations do not

overlap and addresses the relative importance and generalizability
of your sample statistics.
5. Cohen’s d is a measure of effect size that assesses the
difference between two means in terms of standard deviation,
not standard error.
6. The formula for Cohen’s d for a z distribution is: d = (M – )/
7. The sign of the effect size does not matter.
8. A d of .2 is considered a small effect size, a d of .5 is considered
a medium effect size, and a d of .8 is considered a large effect
size.
> Discussion Question 8-5
Imagine you obtain an effect size of –0.3. How would you interpret this
number?
Your students’ answers should include:
 If you obtained an effect size of –0.3, you would interpret this as a

small or small-to-medium effect size.

III. Meta-Analysis
1. A meta-analysis is a study that involves the calculation of a
mean effect size from the individual effect sizes of many
studies.
2. A meta-analysis can provide added statistical power by
considering many studies at once. In addition, a meta-analysis
can help to resolve debates fueled by contradictory research
findings.
> Discussion Question 8-6
What is a meta-analysis and why is it useful?
Your students’ answers should include:
 A meta-analysis is a study that involves the calculation of a mean

effect size from the individual effect sizes of many studies.

 It is useful because it considers many studies at once and helps to

resolve debates fueled by contradictory research findings.

CHAPTER EIGHT Confidence Intervals, Effect Size, and Statistical Power

3. The first step in a meta-analysis is to choose the topic and make

a list of criteria for which studies will be included.
4. Our next step is to gather every study that can be found on a
given topic and calculate an effect size for every study that was
found.
5. Lastly, we calculate statistics—ideally, summary statistics, a
hypothesis test, a confidence interval, and a visual display of
the effect sizes.
IV. Statistical Power
1. Statistical power is a measure of our ability to reject the null
hypothesis, given that the null hypothesis is false. In other
words, it is the probability that we will not make a Type II error,
or the probability that we will reject the null hypothesis when we
should reject the null hypothesis.
2. Our calculation of statistical power ranges from a probability of
0.00 to 1.00. Historically, statisticians have used a power of .80
as the minimum for conducting a study. In other words, when
the null hypothesis is wrong, we want to have an 80%
probability of being able to reject that null hypothesis.
3. There are three steps to calculating statistical power. In the first
step, we determine the information needed to calculate
statistical power, including the population mean, the population
standard deviation, the hypothesized mean for the sample, the
sample size, and the standard error based on this sample size.
4. In step two, we calculate the critical value in terms of the z
distribution and the raw mean so that statistical power can be
calculated.
5. In step three we calculate the statistical power—the percentage
of the distribution of means for population 1 (the distribution
centered around the hypothesized sample mean) that falls
above the critical value.
> Discussion Question 8-7
What is statistical power, and how would you calculate it?
Your students’ answers should include:
 Statistical power is the probability of rejecting the null hypothesis

when it is false.
 You calculate statistical power in three steps. First, determine the

characteristics of the two populations. Next, calculate the raw

mean value that determines your cutoff values. Finally, determine
the percentage that falls above the raw mean and at the cutoff
value using population 1.
6. There are five ways that we can increase the power of a
statistical test. First, we can increase alpha; this increases the
percent of scores that will fall within the critical region. Second,
we could turn a two-tailed hypothesis into a one-tailed
CHAPTER EIGHT Confidence Intervals, Effect Size, and Statistical Power

hypothesis, increasing the size of the critical region in one tail of

the distribution. Third, we could increase N, which will reduce the
M and make it more likely that our sample statistic falls within
the critical region. Fourth, we could exaggerate the levels of the
independent variable; this will result in a larger test statistic and
increase the chances that we obtain a value that will fall within
the critical rejection region. Lastly, we could decrease the
standard deviation, which will also reduce the M and make
rejection of H0 more likely. It should be noted that statisticians
generally frown on increasing alpha because this increases the
likelihood of a Type I error.
LaunchPad Statistical Applets
Statistical Power
This applet illustrates the power of statistical tests. Finding the
power of a test assumes that you have set a fixed significance
level α for the test. The top curve shows the sampling distribution
of the sample mean M when your null hypothesis is true. The
yellow area under this curve is α, the probability of rejecting H0
when it is really true. The bottom curve shows the sampling
distribution of M when your chosen alternative is true. The red
area under this curve is the power, the probability of rejecting H0
when the alternative is really true.
Classroom Activity 8-2
Calculating Power for Proposed Research
Power calculations are typically computed prior to the collection of
any data. For instance, funding agencies will want the researchers to
demonstrate that a proposed study has a large enough sample size
to have sufficient power before providing funds. Likewise, they (and
the researchers) don’t want to collect more data than necessary, so
it is important to be sure that the research study is not overpowered.
In this practice problem, students are asked to evaluate the power of
a study evaluating a brain training program that is expected to
increase users’ IQ scores.
 Have students calculate the mean difference associated with an

effect size of d = 0.5.

 Have students identify the raw mean value associated with a

critical z score of 1.96.

 Have students identify the proportion of the distribution under H
1
that lies above that critical raw mean.
 Because the power for this anticipated study is less than .80,

students will come up with several recommendations for increasing

power, including using a one-tailed test, increasing the effect of the
training program, and increasing the sample size. This last option is
the most appropriate solution to this problem, and if you wish you
CHAPTER EIGHT Confidence Intervals, Effect Size, and Statistical Power

could have students redo their power calculations for a sample size
of N = 49.
Use Handout 8-1 to complete the activity.
> Discussion Question 8-8
What are ways that you could increase statistical power?
Your students’ answers should include:
The different ways that you could increase your statistical power
include:
 Adapt a more lenient alpha level.

 Use a one-tailed test in place of a two-tailed test.

 Increase the size of the sample.

 Exaggerate the levels of the independent variable, in other words

increase the effect size.

 Decrease the standard deviation.

Classroom Activity 8-3

Working with Confidence Intervals and Effect Size
For this activity, you will need to have the class take a sample
personality test assessing the Big 5 personality traits. You can find
many examples of abbreviated versions of the NEO-PI online
(http://www.outofservice.com/bigfive/ is one such site). Have
students anonymously submit their scores and compare the class
data on one or all five of these dimensions of personality to data for
the general population (population mean = 50, population standard
deviation = 29). Using these data:
 Have students calculate the confidence interval for the analysis.

 Have students calculate the effect size.

Use Handout 8-2 to complete the activity.

LaunchPad Video Resources

Snapshots: Comparing Two Distributions
Snapshots: Confidence Intervals
Snapshots: Interpreting Inference
StatClips: Confidence Interval Intro Parts I and II
StatClips Examples: Confidence Interval Intro Part II, Example A, B,
and C
StatClips: Confidence Intervals: Interpretation
StatClips Examples: Confidence Intervals: Interpretation Example A
StatClips: The Central Limit Theorem Part III
StatClips: Sampling Distributions: Overview and Motivation Part II,
Lake Full of Fish Example

Additional Readings
Cohen, J. (1988). Statistical Power Analysis for the Behavioral
Sciences. Hillsdale, NJ: Lawrence Erlbaum.
CHAPTER EIGHT Confidence Intervals, Effect Size, and Statistical Power

This is arguably the definitive source for power analysis. Many

of the procedural guidelines for determining power that are useful in
many types of research design are clearly laid out in this text.
Neyman, J. (1937). Outline of a theory of statistical estimation based
on the classical theory of probability. Philosophical Transactions of
the Royal Society of London. Series A, 236, 333–380.
This is considered the seminal paper for confidence intervals.
Rosenthal, R. (1994). Parametric measures of effect size. In Cooper,
H., and Hedges, L. V. (Eds.), The Handbook of Research Synthesis
(pp. 231–244). New York: Russell Sage Foundation.
A very readable account of many of the techniques for
calculating effect sizes. The chapter also includes a lot of
background information about these techniques and how to interpret
them.

Online Resources
This is an excellent Web site with numerous statistical
demonstrations that you can run in your classroom to help explain
the concepts concretely: http://onlinestatbook.com/. Here you will
find demonstrations of effect size, power, and other statistical
concepts.
As noted above, you will need to change the security settings to
Java and add to the exception list the URL for this site:
onlinestatbook.com.

Math World is an excellent and extensive resource site, providing

background information and succinct explanations for all of the
statistical concepts covered in the textbook and beyond.
http://mathworld.wolfram.com/topics/ProbabilityandStatistics.html