Power, Power Curves and

Sample Size

Planning Reliable and

Efficient Tests
For any job, you need a tool that offers the
right amount of power for the task at hand.
You wouldn't use a telescope to examine a
stamp collection, or a handheld magnifying
glass to search for new galaxies, because
neither would provide you with meaningful
observations. To complicate matters, if
detecting a galaxy really was your goal, the
cost of gaining the necessary power might
be more than you can afford.
2

Planning Reliable and

Efficient Tests
Anyone using statistical tests faces
the same issues. You must consider
the precision you need to meet your
goals (should your test detect subtle
effects or massive shifts?), and
balance it against the cost of
sampling your population (are you
testing toothpicks or jet engines?).
3

Planning Reliable and

Efficient Tests
You also want the confidence in your
results that's appropriate for your
situation (testing seat belts demands
a greater degree of certainty than
testing shampoo). We measure this
certainty with statistical power the
probability your test will detect an
effect that truly exists.
4

Planning Reliable and

Efficient Tests
Minitab's Power and Sample Size
tools, with Power Curves, help you
balance these issues that may
compete for your limited resources.
Here are three examples of how a
quick Power and Sample Size test can
help you save time and money getting
results you can trust.
5

Don't Leave Success to

Chance
A paper clip manufacturer wants to
detect significant changes in clip
length. They sample thousands of
clips because it is cheap and quick to
do. But this huge sample makes the
test too sensitive: the broken line
(next slide) shows it will sound the
alarm if the average length differs by
a trivial amount (0.05).
6

Don't Leave Success to

Chance
Power Curve for 1-Sample t Test
1.0

Sample
Size
100
2000

0.8

A ssumptions
Alpha
0.05
StDev
1
Alternative
>

Power

0.6

0.4

0.2

0.0
0.00

0.05

0.10

0.15
Difference

0.20

0.25

0.30

Don't Leave Success to

Chance
This Power Curve shows they are
wasting resources on excessive
precision. A sample size of just 100
will detect meaningful differences
(0.25) without crying wolf at every
negligible blip.

Don't Leave Success to

Chance

Don't Leave Success to

Chance
An aerospace company is designing an
experiment to test a new rocket.
Each rocket is very expensive, so it is
critical to test no more than
necessary.

10

Don't Leave Success to

Chance
Power Curve for 2-Level Factorial Design
1.0

Reps,
Ctr Pts Per Blk
6, 0

0.8

Assumptions
A lpha
0.05
StDev
1
# Factors
3
# Corner Pts
8
# Blocks
none
# Terms Omitted
0

Power

0.6

0.4

0.2

0.0

-1.0

-0.5

0.0
Effect

0.5

1.0

11

Don't Leave Success to

Chance
This Power Curve confirms an
experiment with 6 replicates will give
researchers the power they need
without spending more than they
must.

12

Don't Leave Success to

Chance

13

Don't Leave Success to

Chance
We've always done it this way.
That's why a lumber company would
sample 10 beams to test whether
their strength meets the target.

14

Don't Leave Success to

Chance
Power Curve for 1-Sample t Test
1.0

Sample
Size
10
34

0.8

Assumptions
Alpha
0.05
StDev
1
Alternative Not =

Power

0.6

0.4

0.2

0.0

-1.0

-0.5

0.0
Difference

0.5

1.0

15

Don't Leave Success to

Chance
According to the Power Curve, this
small sample size made their test
incapable of detecting important
effects. They must sample 34 beams
to detect meaningful differences
(0.50).

16

Don't Leave Success to

Chance

17

See the Big Picture

A power analysis helps you weigh your
resources against your demands, and
quantifies a test's ability to answer
your question. It can expose design
problems, like the lumber company's
insufficient sample size. It can also
reveal design solutions you hadn't
considered.
18

See the Big Picture

Take for instance the packaging plant
of a snack company. Customers
complain that the company's pretzel
bags are sealed with glue that's too
strong, so researchers use One-Way
ANOVA to compare their current
glue with three potential
replacements.
19

See the Big Picture

Differences in seal strength less than
10 are undetectable to most people,
so their test only needs to detect a
difference of 10. A power value of
80% is acceptable, but 90% is ideal.
What sample size meets their needs?
20

See the Big Picture

30 samples of each glue ensure the

test detects a difference of 10 with
90% power.
21

See the Big Picture

Or, they could detect the same difference with 23

samples and 80% power. If this represents
considerable savings, the researchers may consider
using the smaller sample.
22

See the Big Picture

The Power Curve illustrates this
information, but it also charts every
other combination of power and
difference for a given sample size.

23

See the Big Picture

Power Curve for One-way ANOVA
1.0

Sample
Size
23
30

0.8

A ssumptions
Alpha
0.05
StDev
10
# Levels
4

Power

0.6

0.4

0.2

0.0

10
Maximum Difference

15

20

24

See the Big Picture

The solid line indicates researchers
can attain 90% power with just 23
samples if they are willing to seek a
difference of 12 instead of 10. This
might just be the ideal choice.

25

See the Big Picture

26

How to Create Power

Curves in Minitab
Performing power analyses with Power
Curves couldn't be simpler. You supply
the factors you know, and Minitab
calculates the one you omit.

27

How to Create Power

Curves in Minitab
Suppose a trainer wants to compare
two training courses for forklift
operators.
She will use a two-sample t-test to
compare the average scores that
operators from each course earned
on the final exam.
28

How to Create Power

Curves in Minitab
She knows she must detect a
difference of 5 in either direction
between the two courses with 80%
power, and historical data suggest a
standard deviation of 5.
But how many participants must she
sample from each course?
29

How to Create Power

Curves in Minitab

Choose
Stat > Power and Sample Size
> 2-Sample t
In Differences, type -5, 5
In Power values, type 0.80
In Standard deviation, type 5
Click OK

30

How to Create Power

Curves in Minitab

31

How to Create Power

Curves in Minitab
Power Curve for 2-Sample t Test
1.0

Sample
Size
17

0.8

Assumptions
Alpha
0.05
StDev
5
Alternative Not =

Power

0.6

0.4

0.2

0.0

-6

-4

-2

0
Difference

32

How to Create Power

Curves in Minitab
According to this Power Curve,
sampling 17 participants from each
class enables her test to detect the
difference she seeks with 80%
power.

33

How to Create Power

Curves in Minitab

34

Putting Power Curves to

Use
Without knowing the power of your test,
it's hard to know if you can trust your
results: your test could be too weak to
answer your question, or too strong for
your needs. Minitab's Power Curves
(available for many common statistical
procedures) help you balance your
resources against your goals and design a
test you can trust that costs no more than
necessary.
35

Putting Power Curves to

Use
Power Curves graph the dynamic
relationships that define power,
revealing the big picture and ensuring
no option escapes your consideration.
And perhaps most importantly, they
make power analysis an easier and
more accessible part of every
project. Empower your test. Trust
your results.
36