Using and Understanding Mathematics: Statistical Reasoning

Using and Understanding Mathematics
Sixth Edition
Chapter 5
Statistical Reasoning
Copyright © 2015, 2011, 2008 Pearson Education, Inc. All Rights Reserved Slide - 1
5.A Should You Believe a Statistical
Study?
Evaluating a Statistical Study
Most statistical research is carried out with integrity and
care. Nevertheless, statistical research is sufficiently
complex that bias can arise in many different ways.
There are eight guidelines that can help you answer the
question “Should I believe a statistical study?”
Guideline 1
Get a Big Picture View of the Study.
Try to answer the following questions.
• What was the goal of the study?
• What was the population under study? Was the

population clearly and appropriately defined?
• What type of study was used? Was the type

appropriate for the goal?
Example 1
Researchers gave 100 participants their astrological
horoscopes and asked whether the horoscopes
appeared to be accurate. 85% of the participants
answered yes. The researchers concluded that
horoscopes are valid most of the time.
Solution:
The goal of the study was to determine the validity of
horoscopes. Based on the news report, it appears that
the study was observational. However, the accuracy
of the horoscopes were subjective. This study should
have been a controlled experiment and double-
blinded.Copyright © 2015, 2011, 2008 Pearson Education, Inc. All Rights Reserved
Slide - 5
Guideline 2
Consider the source.
• Statistical studies are supposed to be subjective, but
the people who carry them out and fund them may be
biased.
Example 2
Is Smoking Healthy?
• While the vast majority of studies show that smoking
is unhealthy, a few studies found no dangers from
smoking, and perhaps even healthy benefits. These
studies were generally carried out by the Tobacco
Research Institute funded by the tobacco companies.
• Solution:
• Tobacco companies had a financial interest in
minimizing the dangers of smoking. Therefore, there
may have been pressure on researchers to produce
the results to the company’s liking
Copyright © 2015, 2011, 2008 Pearson Education, Inc. All Rights Reserved
Slide - 7
Activity 1
A study by the conservative Heritage Foundation is
designed to assess a new Democratic spending
plan
Should you trust this study according to Guideline 2?
Guideline 3
Look for bias in the sample.
• Selection bias occurs whenever researchers select
their sample in a way that tends to make it
unrepresentative of the population.
• Participation bias occurs primarily with surveys and

polls; it arises whenever people choose whether to
participate. Because people who feel strongly about
an issue are more likely to participate, their opinions
may not represent the larger population.
Activity 2
What type of bias exist?
• A TV Talk show host asks the TV audience to text 1 if
they support and to text 2 if they oppose a law to
require background checks on all firearms sales.
• A state republican party polls 1600 of its members to

determine whether its candidate for the US Senate is
likely to win against the Democratic candidate
Guideline 4
Look for problems in defining or measuring the
variables of interest.
• A variable is any item or quantity that can vary or take

on different values.
• The variables of interest in a statistical study are the
items or quantities that the study seeks to measure.
Example 3 (1 of 2)
A commonly quoted statistic is that law enforcement
authorities succeed in stopping only about 10% to 20%
of the illegal drugs entering the United States. Should
you believe this statistic?
Solution
There are essentially two variables in the study:
quantity of illegal drugs intercepted and quantity of
illegal drugs NOT intercepted. It should be relatively
easy to measure the quantity of illegal drugs that law
enforcement officials intercept.
Example 3 (2 of 2)
However, because the drugs are illegal, it’s unlikely that
anyone is reporting the quantity of drugs that are not
intercepted. How, then, can anyone know that the
intercepted drugs are 10% to 20% of the total? In a
New York Times analysis, a police officer was quoted
as saying that his colleagues refer to this type of
statistic as “PFA,” for “pulled from the air.”
Guideline 5
Beware of confounding variables.
• Variables that are not intended to be part of the study

can sometimes make it difficult to interpret the results
properly. Such variables are often called confounding
variables, because they confound (confuse) a study’s
results.
Activity 3
In a study of obesity among children researchers
monitor the eating and exercise habits of the
participating children, carefully recording everything
they eat and all their activity.
Does the study include confounding variables?
Guideline 6
Consider the setting and wording in surveys.
• Even when a survey is conducted with proper sampling

and with clearly defined terms and questions, it is
important to watch out for problems in the setting or
wording that might produce inaccurate or dishonest
responses.
Example 4 (1 of 2)
The Republican National Committee commissioned a
poll to find out whether Americans supported their
proposed tax cuts. Asked “Do you favor a tax cut?” a
large majority answered yes. Should we conclude that
Americans supported the proposal?
Example 4 (2 of 2)
Solution
A question like “Do you favor a tax cut?” is biased
because it does not give other options (much like the
fallacy of limited choice discussed in Unit 1A). In fact,
other polls conducted at the same time showed a
similarly large majority expressing great concern about
federal deficits. Indeed, support for the tax cuts was far
lower when the question was asked by independent
organizations in the form “Would you favor a tax cut
even if it increased the federal deficit?
Guideline 7
Check that results are presented fairly.
• The study may be misrepresented in graphs or
concluding statements.
• Researchers may misinterpret the results or jump to
conclusions not supported by the results.
Example 5 (1 of 2)
The school board in Boulder, Colorado, created a
hubbub when it announced that 28% of Boulder school
children were reading “below grade level” and hence
concluded that methods of teaching reading needed to
be changed. The announcement was based on reading
tests on which 28% of Boulder school children scored
below the national average for their grade. Do these
data support the board’s conclusion?
Example 5 (2 of 2)
Solution
The fact that 28% of Boulder children scored below the
national average for their grade implies that 72% scored at
or above the national average. Therefore, the school board’s
ominous statement about students reading “below grade
level” makes sense only if “grade level” means the national
average score for a particular grade. This interpretation of
“grade level” is curious because it means that half the
students in the nation are always below grade level—no
matter how high the scores. The conclusion that teaching
methods needed to be changed was not justified by these
data.
Guideline 8
Stand back and consider the conclusions.
Ask yourself the following questions.
• Did the study achieve its goals?
• Do the conclusions make sense?
• Can you rule out alternative explanations for the
results?
• If the conclusions do make sense, do they have any
practical significance?
Example 6 (1 of 2)
An experiment is conducted in which the weight losses
of people who try a new “Fast Diet Supplement” are
compared to the weight losses of a control group of
people who try to lose weight in other ways.
Assuming that it has no dangerous side effects, does

this study suggest that the Fast Diet Supplement is a
good treatment for people wanting to lose weight?
Example 6 (2 of 2)
Solution
So even if the study is flawless, the results don’t seem

to have much practical significance.

Using and Understanding Mathematics: Statistical Reasoning

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Using and Understanding Mathematics: Statistical Reasoning

Uploaded by

Copyright:

Available Formats

Using and Understanding Mathematics

• What was the population under study? Was the

• What type of study was used? Was the type

• Participation bias occurs primarily with surveys and

• A state republican party polls 1600 of its members to

• A variable is any item or quantity that can vary or take

• Variables that are not intended to be part of the study

• Even when a survey is conducted with proper sampling

Assuming that it has no dangerous side effects, does

So even if the study is flawless, the results don’t seem

You might also like