66  Chapter 3
has little to do with a lack of interest by researchers; rather, poli-
ticians responding to advocacy groups have blocked funding or
permission to do research.3 In 1970, the United States Con-
gress “failed to follow its usual review process dictated by the
Controlled Substances Act that requires scientific evaluation
and testimony before legislative action. It declared cannabis ille-
gal in the absence of such evidence” (Bostwick, 2012:181).
Since then, researchers have been prohibited from studying it.
As Bostwick (2012: 182) noted, “the political climate at the fed-
eral level has essentially quashed the type of research that is
routine before commercial introduction of new drugs.” As a
result, the gradual introduction of “medical marijuana” and can-
nabis legalization by several states has proceeded based on
opposing political pressure, not based on neutral evidence from
scientific study. Likewise, in 1996 the U.S. Congress enacted
strict restrictions and cut Center for Disease Control research on
the issue of firearms and violence, and funding dropped by 96
percent. This effectively stopped government-funded research
on the issue until a Presidential order in 2014 in the wake of the
“Sandy Hook” mass shooting of elementary school children. As
a result, the amount of research-based knowledge has been
miniscule. In both situations, politicians and special interest
groups feared that objective, neutral research might uncover
findings that could contradict their ideological position. They
preferred a situation of ignorance, where their ideological posi-
tion would prevail in the absence of evidence, to a climate of
open inquiry and research-based truth seeking.
Pay attention to who paid for research and when doing
sponsored research, negotiate conditions for releasing find-
ings prior to beginning the study or signing a contract. It is best
to begin with an explicit guarantee that you will only conduct
ethical research. It is legitimate to delay the release of findings
to protect the identity of informants, to maintain access to a
research site, or to protect your personal safety. It is not legiti-
mate to censor findings because a sponsor does not want to
look bad or wants to protect its reputation. The researcher
directly involved and knowledgeable about a study shoulders a
responsibility for both conducting the research and making its
findings public.
Blowing the Whistle  Whistle-blowing occurs
when a researcher informs an external audience of a seri-
ous ethical problem that is being ignored. It is never a first
step; rather it occurs after the researcher has repeatedly
attempted to inform superiors and fix the problem inter-
nally. The whistle-blowing researcher must believe that the
situation is a serious breach of ethics and the organization
will not end it without public pressure. Whistle-blowing is
risky in several ways:
• Outsiders may not be interested in the ethical abuse
and simply ignore it.
3
See American Psychological Association (Crawford [2014], Gutting [2012],
Jamieson [2013], and Underwood [2013] on gun violence). See Bostwick (2012),
Crawford (2014), and Harris (2010) on medical marijuana.
• Outsiders might not care about ending unethical
behavior, but instead use the exposure of unethical
behavior to advance their own goals.
• Managers will try to protect the organization and dis-
credit the whistle-blower.
• The whistle-blower often experiences emotional dis-
tress and strained relations, and even lawsuits.
• Future employers may not trust the whistle-blower
and may avoid hiring him or her.
A whistle-blower needs to be prepared to make sacri-
fices—loss of a job or no promotions, lowered pay, an
undesirable transfer, abandonment by friends at work, or
legal costs. There is no guarantee that doing the ethical-
moral thing will end the unethical behavior or protect an
honest researcher from retaliation.4
3.4:  Politics and Social
Research
3.4 Recognize the impact that politics and large
corporations have on research
The ideals of a free, open, and democratic society include
advancing and sharing knowledge. People have a right to
study and inquire into any question and to share their find-
ings publicly. Ethical issues largely address moral concerns
and standards of professional conduct that are usually under
the researcher’s control. Political concerns can also influence
and interfere with the research process. Organized advocacy
groups, powerful interests, government officials, or politi-
cians may try to restrict or control the direction of research.
In the past, powerful political interests and groups
have tried to stop research or the spread of legitimate
research findings to advance their own political goals.
They have used their political power to threaten research-
ers or their employers, to cut off research funds, to harass
individual researchers and ruin their careers, and to censor
publication of findings that they disliked.
Politically powerful groups have directed research
funds away from studying questions that researchers saw
as important and toward studies that supported the policy
positions of their political views. Members of the U.S. Con-
gress have targeted and removed funds for social research
projects that panels of independent scientists evaluated as
being well designed and critical to advanced knowledge.
Often the reason has been that the politicians personally
disliked the study topics (e.g., sexual behavior of teens, ille-
gal drug users, voting behavior). Politicians are not the only
4
See Yong, Ledford, and Van Noorden (2013) on whistle-blowing and scientific
fraud.

ones who block the free flow of knowledge. Large corpora-
tions have threatened individual researchers with lawsuits
for delivering expert testimony in public about research
findings that revealed to the public the corporation’s bad
conduct, or have hired researchers to discredit studies with
findings unfavorable for their financial ­interests.5
Example Study
Political Influence on Crime
Research
Savelsberg, King, and Cleveland (2002) conducted a content
analysis study on shifts in U.S. criminal justice policy over the
last thirty years of the twentieth century. They looked at articles
in leading scholarly journals between 1951 and 1993 to see
whether politicians changed how the federal government
awards research funds due to increasing politicization of crimi-
nal justice research.
Read More
More specifically, they asked whether the government money
went to studies that supported specific crime policy ideas that
powerful politicians favored rather than to purely science-
based ideas. For example, they asked whether funds directed
research away from looking at some ways to fight crime (alter-
ing social conditions, using informal community controls, and
emphasizing rehabilitation) and toward others (imposing more
formal, coercive police actions and emphasizing punishment).
They found that politics significantly influenced the direction of
crime research. If the head of a government agency with
research funds was politically appointed or funds only were for
a crime-fighting policy that was closely tied to political ideol-
ogy, they called it “political funding.” They found that the pro-
portion of research articles listing sponsors classified as
“political funding” grew from 3 percent to 31 percent over the
time period they examined. They documented a large-scale
shift in research, away from examining the sociological condi-
tions of crime and the effects of rehabilitation and toward a
focus on control and punishment (Savelsberg, King, and
Cleveland [2002] and Savelsberg, Cleveland, and King [2004]).
They argued that politicians and political movements had
gained control of government research funding and then used
funding to redirect it toward certain study topics over others.
Powerful institutions and organized groups try to control
or censor research out of fear that free, unbiased inquiry might
damage their interests. They place a higher value on protecting
and advancing a political or economic agenda than on the
open pursuit of truth. This shows the tight connection between
unimpeded, open scientific inquiry and the ideals of open pub-
lic debate, democracy, and freedom of expression. Censoring
and controlling research has always been the practice in dicta-
torships and totalitarian regimes.
5
See Mooney 2005; Oreskes and Conway 2010.
Becoming an Ethical Researcher 67
WRITING PROMPT
Was Crime Research Censored?
Studies showed how politicians have redirected crime research
toward certain issues and away from others. Do you consider this
a negative form of government censorship, or is this an example of
good representative democracy in action? Why?
The response entered here will appear in the
performance dashboard and can be viewed by
your instructor.
Submit
3.5:  “Value-Free” and
“Objective” Research
3.5 Examine the complexity of “value-free” and
“objective” research
You have heard the phrase “value-free” research and the
importance of being “objective” in research. These ideas
are not as simple as they first appear for the following three
reasons:
• The terms value-free and objective have multiple
­meanings.
• Researchers have alternative ultimate goals for doing
research.
• Doing value-free, objective research does not mean
that individual researchers are devoid of all values.
Review Table 3.3 for multiple meanings of research
ideas.
Alternative Goals  Some professional researchers say
they reject value-free research. They mean that they maintain
personal values in certain parts of the research process, not
that they embrace sloppy and haphazard research or research
procedures that follow personal whims. They believe that a
researcher should be explicit about his or her values, not that
a study has foregone conclusions and automatically supports
a specific value position. You should reflect carefully on rea-
sons for doing a study and the procedures used. In this way,
other researchers see the values and judge for themselves
whether the values unfairly influenced a study’s findings.
Devoid of Values  Even researchers who strongly
advocate value-free and objective studies admit a place for
personal, moral values. Personal, moral views can enter
select parts of the process: when you select a topic to study
and decide where to publicize the findings. Although you
must follow standard procedures and contain personal
views and values in some parts of the research process,
you can still study the questions you believe to be impor-
tant and make extra efforts to publicize the findings among
specific interest groups.
68  Chapter 3
Table 3.3  Multiple Meanings of Research Ideas
Research Idea Meanings
1. Research without any prior assumptions
or theory.
Value-free 2. Research free of influence from an individual
researcher’s personal prejudices/beliefs.
1. Focus only on what is external or visible.
Objective
2. Follow clear and publicly accepted research
­procedures and not haphazard, invented
­personal ones.
Summary: What You Learned about Becoming
an Ethical Researcher?
We conduct research to gain knowledge about the social
world. The perspectives and techniques of social research
can be powerful tools for understanding the world. Never-
theless, with that power to discover comes the responsibil-
ity to be ethical. It is a responsibility to yourself, a
responsibility to your sponsors, a responsibility to the com-
munity of researchers, and a responsibility to the larger
society. The responsibilities of research can conflict with
each other. As Rik Scarce (1999:984–985) observed, “Ethics
are morality—fundamental rights and wrongs—in prac-
tice. They are not . . . legally acceptable statements . . . .
They are standards higher than any law.” Ultimately, you
personally must decide to conduct research in an ethical
manner. Research is not automatically moral. Individual
researchers must uphold ethical-moral research and
demand ethical conduct by others. The truthfulness and
use/misuse of the knowledge we gain from research
depends on individual researchers like you.
Quick Review
The Ethical Issues Involved in Using
People as Research Participants
1. Ethics requires a balance of competing values, but codes
of ethics, and sometimes the law, recognize four clear
Description
The first meaning is rarely possible. It means “just facts” without theory or
assumptions. Assumptions and theory are in virtually every research study.
The best thing is to acknowledge them and make them explicit. Having theo-
retical assumptions does not prevent study findings from reversing or over-
turning them.
The second meaning is standard practice. It means that an individual person
doing the research temporarily “locks up” his or her personal beliefs, values,
and prejudices during the research process (i.e., design, data collection, data
interpretation). Your personal beliefs should not distort using standard
research procedures but can still influence the choice of a study topic or
research question or how to publicize or use findings.
The first meaning is not accurate. We conduct empirical research based on
direct or indirect evidence. Although some evidence is not directly visible,
such as a person’s personality or opinion, you can create measures to make
it visible.
The second meaning is standard practice. You should always conduct
research in an open, public manner that fits with widely accepted proce-
dures.
­ rohibitions: (a) Never cause unnecessary or irreversible
p
harm to research participants; (b) Always get voluntary
consent from research participants before a study begins;
(c) Never unnecessarily humiliate or degrade research par-
ticipants; and (d) Never release information about specific
individuals collected for research purposes.
2. Ethics for social research developed out of concerns over
medical research on people and after serious past abuses.
A key turning point was the creation of the Nuremburg
Code created after the discovery of horrific experiments
on humans in Nazi Germany.
3. Researchers should be aware of and protect research par-
ticipants from several kinds of harm: physical harm or
bodily injury, emotional distress or psychological harm,
and legal harm and damage to a person’s career, reputa-
tion, or income.
4. The principle of voluntary consent is a core ethical princi-
ple. It says participants must voluntarily agree to become
involved in a study—their participation cannot be forced.
The agreement to participate or a person’s consent must
be informed in that they must know what they are
­agreeing to.
5. Using deception on research participants is a mild type of
harm, and researchers disagree over its use in social
research. If used, it might be ethical, but only if there is no
better alternative, if it is temporary and limited, and if it is
followed immediately by debriefing.
6. When conducting research, to be ethical, you can only
violate a participant’s privacy to the minimum degree

­ ecessary and collect private information only for a legiti-
n 3. Research fraud occurs when a researcher engages in seri-
mate research purpose. In addition, you must take several
steps to protect the information.
7. Researchers attempt to protect both anonymity and confi-
dentiality. These are related but different ideas. Anonymity
means to remain anonymous or nameless. No one can
trace information back to a specific individual. Confidenti-
ality allows you to attach the information to particular indi-
viduals, but keep it secret from public disclosure. You only
release data in ways that do not permit anyone else to link
specific individuals to information.
8. Special populations such as students, prison inmates,
employees, military personnel, the homeless, welfare recip-
ients, children, or the developmentally disabled may not be
fully capable of giving consent freely and require extra pro-
tections when they are participants in a research study.
9. Most professional organizations have codes of ethics that
specify principles of ethical conduct in research. In addi-
tion, several government agencies have rules or guidelines
regarding human subject research participants. Colleges,
hospitals, and research centers have an institutional review
board (IRB). It applies ethical guidelines by reviewing
research procedures at a proposal or preliminary stage.
What is the Ethical Imperative?
1. Researchers have a strong obligation to act ethically at all
times and in all situations, even if research participants are
unaware of this or are unconcerned. It may be difficult to
appreciate ethical issues fully until conducting a study, but
waiting until you are in the middle of doing research is too
late. Many pressures on researchers make acting ethically
difficult at times.
2. Two major types of scientific misconduct are research
fraud and plagiarism. Both are serious ethical violations for
which there is never an excuse.
Becoming an Ethical Researcher 69
ous deception and lies about the data or study procedure.
4. Plagiarism is “misrepresenting what someone else wrote
or thought as your own.” It is a form of “stealing” ideas or
writings that happens when using someone’s ideas with-
out properly citing the source.
5. Being ethical is not the same as acting legally. A research
action can be fully legal (i.e., not breaking any law) but
clearly unethical.
Shared Writing: Free Inquiry and
the Autonomy of Social Research:
A Core Principle or Outdated Ideas
You have read about how the sponsors who pay for research
sometimes attempt to control it, how powerful interests pres-
sure government do research on some issues but not others,
and how politicians try to suppress or redirect research based
on their own agendas. Do you think these are serious prob-
lems, or minor issues that can be ignored? Do you believe
whomever pays for a study should control how it is conducted,
and should decide whether findings are ever made public?
Defend your position on these two questions and comment on
at least two classmates’ responses whose selection differs
from yours.
A minimum number of characters is required to
post and earn points. After posting, your
response can be viewed by your class and
instructor, and you can participate in the
class discussion.
Post 0 characters | 140 minimum
Chapter 4
Sampling
Learning Objectives
4.1 Identify reasons why we use samples in
research
4.2 Describe four nonprobability sampling
techniques in research
4.3 List the specialized vocabulary used in the
process of random sampling
4.4 Evaluate the three main steps that are
followed in simple random sampling
Many teens in large urban areas of the United States, espe-
cially in areas with high concentrations of crime and pov-
erty, carry illegal guns or other weapons. They are no more
likely than teens in other countries to get into fights, but
they are far more likely to have deadly weapons, so con-
frontations are more likely to be deadly. When asked why
70
4.5 Identify the three main types and uses of
random sampling
4.6 Identify potential challenges presented in
sampling for research and how to effectively
manage those difficulties
4.7 Examine how sampling errors lead to
incorrect inferences
they carry weapons, the most common response is self-
defense. Yet, prior studies indicated that many teens hold
misperceptions and often overestimate the behavior of
peers regarding sexual activity, drug use, and weapons.
Hemenway and colleagues (2011) conducted a survey
of teens in the Boston area in 2008 to learn whether

­ verestimation and misperceptions about others contrib-
o yields highly ­representative samples; to simplify things
uted to a student carrying a gun. They drew a random
sample of students in 31 area public and private high
schools. In each school, they stratified the sample by
required humanities (e.g., English) classes and by grade,
and then randomly selected classrooms within each
grade. They selected about four classrooms per school,
with about 100 students per school in the study sample. If
a school had 100 or fewer students, everyone was selected.
There were 2,725 students enrolled in the selected class-
rooms at the time, of whom 69 percent completed sur-
veys. The remaining students were absent on the day of
the survey (724), declined to participate (99), or were not
permitted to participate by their parents (24). Students
were asked whether they had carried a gun to school in
the past 12 months. The researchers also asked a series of
questions about their experiences and perceptions about
whether they knew of others who had guns, had gotten in
fights, had friends who had gotten in fights, could easily
get a gun if they wanted one, and so forth. Of 1,878
respondents, 1,737 (92.5 percent) answered the question
on gun carrying.
The researchers found that over 5 percent of students
reported carrying a gun, 9 percent of boys and 2 percent of
girls. Students substantially overestimated the percentage
of their peers who carried guns. The chances that a student
had a gun were strongly predicted by the perception that
many peers had guns. Also, most believed it was easier for
other students to obtain guns than it was for them.
4.1:  Why Do We Use
Samples?
4.1 Identify reasons why we use samples in research
When we draw a sample, we examine the small subset, or
the sample. We do this because we lack the time and
resources or we do not need to look at every single unit or
case in the large collection, or population. If we sample
carefully, we can reach highly accurate conclusions about
the whole population from the sample alone. We have sev-
eral ways to sample. Two factors influence the type of sam-
pling design to use, whether the data are quantitative or
qualitative, and the purpose of the study.
In quantitative research, we put a lot of effort into
sampling design because the goal is to produce genu-
inely representative sample, which has all the features of
the population from which it came. A representative
sample enables us to make highly accurate generaliza-
tions about the entire population from using the sample
data alone. From probability theory in applied mathe-
matics we know that using a random selection process
Sampling 71
we call ­probability-based samples, random samples.
Random samples are highly efficient in terms of time
and cost. A properly conducted sample can yield results at
1/1,000th the cost and time of contacting and gathering
data on a population. Recall that the study conducted by
Hemenway and colleagues on Boston teens and guns had a
sample of 1,737 students who answered questions. Their
results would be very similar if they had asked all students
in all Boston schools.
Some people do not realize that a well-designed, care-
fully executed random sample yields results that are equally,
or even more accurate, than attempting to reach everyone in
the population. You probably heard of the census. It provides
information on things, such as sex, age, job, and so forth. In
the United States, there is an official count of the population
every 10 years. For the 2000 census, leading scientists advised
the government to use specialized statistical sampling to get
the most accurate measure of the population rather than try-
ing to count everyone, as was the case in the past. Unfortu-
nately, political considerations rather, than solid scientific
advice prevailed, and the government used the less accurate
method of trying to count everyone.
Conversely, the goals in qualitative research may not
require getting a representative sample of a population. We
may want to examine a small collection of cases, units, or
events, and have them illuminate certain key features of an
activity or the flow of social life. Random sampling is rare
in qualitative studies because our goal is less to represent
the entire population than to highlight certain informative
cases, events, or actions. We use the highlighted cases to
clarify and deepen our understanding of specified areas of
social life.
4.2:  Types and Applications
of Nonrandom Samples
4.2 Describe four nonprobability sampling techniques
in research
Random samples are best to use when we want to create an
accurate representation of a population, but they can be
difficult to conduct. Researchers unable to draw random
sample or who have goals that differ from creating a repre-
sentative sample often use nonprobability sampling tech-
niques. Four such techniques are: convenience, quota,
purposive, and snowball sampling.
4.2.1:  Convenience Sampling
Convenience sampling (also called accidental or haphaz-
ard sampling) is appealing because it is easy, cheap, and
72  Chapter 4
fast. Its biggest drawback is that it frequently produces
highly unrepresentative samples; plus, it lacks the depth
and context sensitivity desired for qualitative research. If
you haphazardly select convenient cases, your sample may
distort what is in the population (see Figure 4.1).
Figure 4.1  Representative and Nonrepresentative
Samples of 6 out of 18
Representative Samples Allow Accurate Generalization to
the Population
Representative
Nonrepresentative
In some cases, convenience samples may be acceptable.
They are used during the preliminary phase of an explor-
atory study, but otherwise they are of limited value. A per-
son-on-the-street interview, like those conducted by
television programs, is an example of convenience sampling.
Television interviewers go out on the street with a camera to
talk to a few people who are convenient to interview. The
people walking past a television studio in the middle of the
day do not represent everyone. Likewise, the interviewers
Figure 4.2  Quota Sampling
Quota Samples Produce an Approximation of the Population
Of 32 adults and children in the street scene, select 10 for the sample:
4 Adult Males
1 Male Child
tend to pick p ­ eople who look “normal” to them and avoid
unattractive, very busy, or inarticulate people. Their sample
is for entertainment purposes and not for serious research.
Have you watched a television show that asks you to call
in your opinion or visited a web page that asked you a
few survey questions?
These too are convenience samples. Only some people who
are watching television or visiting the web page respond.
Even if the number who do so is large (e.g., 500,000), we
cannot generalize accurately from sample to the popula-
tion. Like a person-on-the-street interview, such samples
can seriously distort what is in the population. It is impor-
tant to remember that the method of sample selection (not
the number responding) is the most important factor. A
large convenience sample should not be confused with a
true representative sample.
4.2.2:  Quota Sampling
Quota sampling can produce a representative sample but
one that is less accurate than a random sample (see Figure
4.2). It is much easier than random sampling and a big
improvement over convenience sampling. The procedure
for quota sampling is as follows:
• First, identify several relevant categories of people or
units. The categories should reflect aspects of diversity
in the population that you believe to be important.
4 Adult Females
1 Female Child

• Next, decide how many units to get for each category.
• After you fix the categories and number of units in
each category, select units by any method.
For example, you are interested in a sample of 80 shoppers
at a grocery store. You think gender and age are important
aspects of diversity. You select 10 males and 10 females
under age 30, 10 of each gender aged 30–39, 10 of each gen-
der aged 40–50, and 10 of each gender over age 60. You
might interview the first 12 males who walk into the store,
asking each his age. Once you have 10 who are under 30
years of age, you have to skip all other males in that age
group because you have filled your quota.
Quota sampling is an improvement over convenience
sampling because it ensures that some major differences
within the population will also appear in a sample. In con-
venience sampling, everyone selected might be of the same
age, gender, or racial category. However, quota sampling
has limitations and can yield a nonrepresentative sample.
A first limitation is due to the selection process for
placing cases into quota categories. Quota sampling relies
on a convenience selection process. This means, we might
select only people who “act friendly,” are easy to reach, or
who want to be part of the study into the sample. A second
limitation is little diversity in a set of quota categories.
Quota sampling categories only capture the diversity of a
few predetermined population characteristics. A popula-
tion might differ in 20 ways, but quota samples rarely
include more than three characteristics. Let us say a quota
sample of grocery shoppers includes combinations of gen-
der (male/female), age (over/under 50), race (white/non-
white), and shopping companion (none/with others). For
this quota sample, you must find enough people in each
combination. A third limitation is the sample size for each
quota category. Often, you set the number of cases to select
for each quota category without having knowledge of the
true population. For example, you set a quota of 10 percent
of the grocery shopper sample to be males under age 30.
Yet, perhaps they actually make up 18 percent of the gro-
cery shopping population.
Bad Samples Often Yield Inaccurate Results
Sampling 73
An interesting historical case illustrates the limitations of
quota sampling. George Gallup’s American Institute of Public
Opinion used quota sampling. It successfully predicted the
outcomes of the 1936, 1940, and 1944 U.S. presidential elec-
tions. However, in 1948, Gallup predicted wrong and said
Thomas Dewey would win over Harry Truman. The incorrect
prediction had several causes (e.g., many voters were unde-
cided, interviewing stopped early), but the main reason was
that quota sampling categories did not represent all geo-
graphical areas and all types of people who actually voted.
4.2.3:  Purposive Sampling
Purposive sampling is a widely accepted special sampling
technique. It is appropriate when the goal is other than get-
ting a representative sample of the whole population. Pur-
posive sampling requires having a very specific purpose in
mind and using judgment to select cases, and is sometimes
called judgmental sampling. In a way, it is convenience
sampling for a highly targeted, clearly defined population.
We use it in two situations:
• To select especially informative cases.
• To select cases from a specific, difficult-to-reach
population.
We may want to pick cases that have richer information. For
example, you want to examine magazine content for cultural
themes. You select two specific popular women’s magazines
to study because they are most trend setting rather than
select a representative sample of all women’s magazines.
To study a targeted group, we may use many diverse
methods to identify as many cases as possible. For exam-
ple, you want to study people under 30 years old who use
wheelchairs in the Seattle metropolitan area. Without a list
of wheelchair users, you cannot use a random sampling
method. To use purposive sampling, you use many diverse
forms of information to get a sizable collection (maybe of
60 names). To get the names, you may go to locations
where wheelchairs are sold and repaired or ask knowl-
edgeable local experts (e.g., health workers, other wheel-
chair users, or disability advocate groups).
Example Study
Purposive Sample
In Italy and Japan, unmarried adults in their 30s often live at
home with their parents. In fact, more than half of Italian men,
aged 25–35 still live at home with their mothers. In Japan, 60
percent of unmarried men and 70 percent of unmarried
women aged 30–34 remain at home with their parents. There
is even a term, “parasite single,” to describe the situation. Yet
in Scandinavia, teens as young as 16 live independently of
74  Chapter 4

their parents. Katherine Newman (2008) wanted to find out • Drug dealers and suppliers who work together to form
what such relationships mean to those involved, and adopted a distribution network
a comparative, qualitative approach. She wanted to uncover
• People on a college campus who have had sexual rela-
the subjective, cultural understandings of autonomy and inde-
tions with one another
pendence of “delayed departure” and “accelerated indepen-
dence.” She assembled a research team from three “delayed The crucial feature is that each person or case has a connec-
departure” countries (Spain, Italy, and Japan) and two “accel- tion with the others. The linkage can be either direct or
erated independence” countries (Denmark and Sweden). indirect. Members of the network may not directly know
She used judgment sampling to obtain 49–52 interviews or interact with all others in the network. Rather, taken as a
per country, a total of 250 interviews. Qualitative interviews were
whole, each is part of a larger linked web.
conducted with unmarried adults over 22, some co-residing
with parents, others living independently, and parents of an
unmarried adult. The interviews included people in both urban
and rural areas and from different regions of each country. The
sample included parents and adult children in the same families
when possible, and in different families when a parent and child Example Study
in the same family could not be interviewed. She says (p. 651)
“we cannot claim that these samples are representative in any
Snowball Sample
definitive way.” The interviews by native speakers took place in
homes and cafes. She found that in the “delayed departure”
countries the parents had almost uniformly experienced “a
before (childhood) and an after (adulthood) that was marked by
clear behavioral changes in their lives” (p. 652). The clear divid-
ing line included marriage, full-time employment, and childbear-
ing. For the young adults today, adult maturity was marked less
by outward behavioral changes than a slowly evolving inner feel-
ing of more responsibility and making independent decisions.
This occurs while still living in the natal home. The unmarried
adults and parents both recognized that staying at home was
often an economic necessity and required creating new parent–
Anju Mary Paul (2011) studied “stepwise international migra-
child relationships. In “accelerated independence” countries,
tion.” It is when people with limited income and skills from
attending high school or university away from home where gov-
low-income countries engage in a multistage process of inter-
ernment aid covers all expenses is common. Remove the eco-
national labor migration. They work in intermediate countries
nomic necessity and physical separation from the natal family
as a conscious strategy to reach a desired final destination
occurs early and brings slightly more internal, subjective auton-
country. As they work outside their home country, the
omy and emotional separation for the unmarried adults. Some
migrants increase savings and gain work experience and edu-
youth said that they felt somewhat less close to their family, and
cational certifications. They also build a network of overseas
believed family relations were stronger in other countries.
contacts with the goal of accumulating resources to gain
entry into a desirable destination country. Dr. Paul conducted
in-depth interviews with 95 prospective, current, and former
Filipino domestic workers in the Philippines, Hong Kong, and
4.2.4:  Snowball Sampling Singapore about their migration and destination decisions.
Snowball sampling is a special technique used when we She approached local nongovernmental organizations that
sought to improve the welfare of migrant workers and asked
want to capture an already-existing network. Its name
for help in finding potential interviewees. She also approached
comes from an analogy to the way a snowball increases in
Filipino domestic workers in Singapore and Hong Kong during
size: It begins small but gets larger as you roll it, and it
their off days when they gathered in downtown shopping cen-
picks up additional snow. In this multistage technique, you ters. In the Philippines and Singapore, she included clients of
start with one or a few cases, then spread out based on recruitment agencies. She used snowball sampling by ask-
direct or indirect linkages to the initial case. We often use ing the first contact interviewees to refer her to other Filipinos
snowball sampling if we want to sample a social network they knew. Her final sample had 27 women in the Philippines,
of people or linked organizations. Networks for which 26 women and 2 men in Hong Kong, and 40 women in Singa-
researchers used snowball sampling include the following: pore. She conducted semistructured interviews about deci-
sions to leave the Philippines and work overseas as domestic
• Scientists around the world who are investigating the workers. Dr. Paul learned that many migrants (40 percent) had
same issue worked two or more years in several other low or middle
• The elites of a medium-sized city who consult with countries in Asia or the Middle East as a conscious strategy to
one another reach their goal of entering a high-income Western country.

Each change in country was a movement upward, with better
wages and working conditions. Most of the intermediate
countries in Asia and the Middle East had policies to facilitate
the entry of foreign domestic workers on renewable short-
term contracts. However, none offered the option of perma-
nent residence.
4.3:  The Terminology Used
to Discuss Random
Sampling
4.3 List the specialized vocabulary used in the process
of random sampling
A random sampling method will produce a sample (i.e.,
small collection of cases) that most accurately represents a
far larger population. The process of random sampling has
a specialized vocabulary (see Figure 4.3).
You recall the three terms: population, sample, and
universe. The case or unit of analysis in the population is a
sampling element. It can be a person, a group, an organi-
zation, a written document or symbolic message, or even a
social action (e.g., an arrest, a divorce, or a kiss). Three
terms with similar meanings can be a source of confusion.
We select the sample from a population, but usually we tar-
get a more specific and concrete collection of sampling ele-
ments within the overall population.
For example:
• Universe: all people in Florida
• Population: all adults in the Miami metro area
• Target population: people aged 18–88 who had a
permanent address in Dade County, Florida, in
September 2016, and who spoke English, Spanish,
or Haitian Creole
The population is more an idea than it is something con-
crete. Except for small or specialized populations (e.g., all
Figure 4.3  A Model of the Logic of Sampling
The Logic of Sampling
What You Population
Would Like to
Talk About
Sampling Process
Sampling
Frame
Sampling 75
the students in a classroom, all employees currently work-
ing the second shift at factory number three of Tom’s shoe
company on March 30 of this year), you have to refine the
population to be very specific (i.e., the target population)
before you can draw a sample.
Once you designate a target population, you must
create a list of all its sampling elements, or the sampling
frame. There are many types of sampling frames: tele-
phone directories, tax records, driver’s license records,
and so on. In the opening study about teens carrying
guns, the researchers had a complete list of schools and
English classes for the sampling frames. The researchers
did not create a list of all individual students because
all students in the same English class were in the sam-
ple. Listing the elements is often difficult because no
good list of the elements in a population exists. A good
sampling frame is crucial to accurate sampling. If there
is a mismatch between the sampling frame and the pop-
ulation, it can create major errors and cause invalid
sampling.
Any statistical characteristic of an entire population
(e.g., the percentage of city residents who smoke cigarettes,
the average height of all women over the age of 21, the per-
cent of people who believe in UFOs) is a population
parameter. If you have all the elements in a population,
you accurately compute a parameter with absolute accu-
racy. For very large populations (e.g., an entire nation), you
never have all elements, so you use information in a sam-
ple to estimate the population parameter. If you end up
taking a statistics class and hear about “parameter estima-
tion,” this is where it comes from.
The sample will be smaller than a target population,
and a sampling ratio indicates the percentage of the target
population that is in a sample. To compute this, we simply
divide sample size by target population. For example, a
target population has 50,000 people and you draw a sam-
ple of 150 from it. Your sampling ratio is 150/50,000 =
0.003, or 0.3 percent. If the target population is 500 hospi-
tals and you sample 100 of them, your sampling ratio is
100/500 = 0.20, or 20 percent.
What You Actually
Observe in the Data
Sample
76  Chapter 4
Sampling with a Random Selection Process  As
you learned, random samples are most likely to represent
the population. However, sampling with a random selec-
tion processes requires a lot more work than nonrandom
ones. In statistics, the word random refers to a random
selection process, one that gives each element in a popula-
tion an equal (or known) probability of being selected. Two
critical features of true random processes are:
1. They are purely mechanical or mathematical without
human involvement.
2. They allow us to calculate the probability of outcomes
with great precision.
A random process enables us to estimate mathematically the
degree of match between the sample and the population, or
estimate the sampling error. Whenever we sample and do
not have the entire population, the sample might deviate
from the entire population. Sampling error indicates the size
of this deviation or mismatch. Later in this chapter, we will
consider ways to minimize the sampling error.
The several kinds of random samples have three key
features:
• You must begin with an accurate sampling frame or
list of elements in the target population.
• You must use a random selection process without
subjective human decisions (e.g., a computer program,
random number table).
• You must identify and pick a particular sampling ele-
ment, rarely using substitutions.
For example, if you use a telephone directory as your sam-
pling frame (actually is it inaccurate, as you will see later)
and sample names for a telephone survey, you must reach
the specific sampled household or person. This means call-
ing back many times before giving up and going to a sub-
stitute randomly selected alternative.
Learning from History
The Famous Literary Digest
Mistake
The Literary Digest was a major U.S. magazine that pre-
dicted presidential elections in the 1920s and 1930s. The
magazine sent postcards to people before the U.S. presi-
dential elections. The magazine staff created a sampling
frame by taking names from automobile registration and
telephone directories. People returned the postcards indi-
cating the candidate they supported. The magazine cor-
rectly predicted election outcomes in 1920, 1924, 1928, and
1932. The magazine’s success with predictions was well
known. In 1936, the magazine increased its sample size to
10 million. The magazine ­predicted a huge victory for Alf
Landon over Franklin D. Roosevelt. In this election, the Liter-
ary Digest was very wrong; Franklin D. Roosevelt won by a
landslide. The prediction was wrong because the sampling
frame did not accurately represent all voters. It excluded
people without telephones or automobiles, as much as 65
percent of the population in 1936, during the worst part of
the Great Depression. More importantly, this excluded seg-
ment of the voting population (lower income) tended to favor
Roosevelt. The magazine had been accurate in earlier elec-
tions because people with higher and lower incomes did not
differ in how they voted. In addition, before the Depression,
more lower-income people could afford telephones and
automobiles. We can learn two lessons from the Literary
Digest mistake. First, the sampling frame is crucial. Second,
the size of a sample is less important than whether or not it
accurately represents the population. An excellent sample of
3,000 can produce more accurate predictions about the 300
million in the U.S. population than a nonrepresentative sam-
ple of 30 million people.
WRITING PROMPT
Sample Size ZE
Many people focus on how many units are in a sample but ignore
the sample selection process. Explain why the method of selecting
units to be included in a sample is equally or more important than
sample size.
The response entered here will appear in the
performance dashboard and can be viewed by
your instructor.
Submit
4.4:  Producing a Simple
Random Sample
4.4 Evaluate the three main steps that are followed in
simple random sampling
The simple random sample is the basis from which other
types of random samples are modeled. In simple random
sampling, there are three main steps to follow:
1. Develop an accurate sampling frame.
2. Select elements from the frame based on a mathemati-
cally random selection procedure.
3. Locate the exact selected elements to be in the sample.
In practice, after we develop a sampling frame we begin
by numbering each element in the sampling frame, from 1
to the last element. Next, we obtain a set of randomly
generated numbers, from 1 to the largest number in the
sampling frame. Most people do this with a computer
program that asks for the size of sampling frame and the
size of the sample; the program then produces a list of
random numbers. There are many such inexpensive or

free random number generator programs available.
Notice that we must decide the sample size before select-
ing elements.
How large should the sample be?
This is not simple to answer. At this stage, remember that
using a bigger sample is not always better. The selection
process is more important than sample size for a good rep-
resentative sample.
You may ask, once I select an element from the sampling
frame, should I then return it to the sampling frame or
keep it separate?
You usually do not return it, or sample without replace-
ment. Pure unrestricted random sampling is random sam-
pling with replacement—i.e., replacing an element after
sampling it so that you can select it again. In most situa-
tions with people as the unit, this makes no sense.
We can see the logic of simple random sampling with
a favorite example of statisticians—sampling marbles
from a jar. Let us say you have a large jar full of 5,000
marbles (some blue and some white) and you want to
estimate a population parameter or the percentage of blue
marbles in the jar. The 5,000 marbles are the target popu-
lation. You randomly select 100 marbles. To do this, close
your eyes, shake the jar vigorously, reach in to pick out
one marble to set aside, and then repeat the procedure 100
times. You now have a random sample of 100 marbles.
Open your eyes and count the number of blue marbles in
the sample. This gives you an estimate the percentage of
blue versus white marbles in the jar, or population. While
it is time-consuming, this is a lot easier than counting all
5,000 marbles.
Let us say that your sample has 52 white and 48
blue marbles. Does this mean that the population
parameter is 48 percent blue marbles? Maybe or maybe
not. Because of random chance, a specific sample might
be off. You can also check the results by dumping the
100 marbles back in the jar, mixing the marbles, and
drawing a second random sample of 100 marbles. Let us
say that on the second try, your sample has 49 white
marbles and 51 blue ones. Which is correct? You may
ask, how good is this random sampling business if dif-
ferent samples from the same population can yield dif-
ferent results? Let us say you then repeat the procedure
over and over until you have drawn 130 different sam-
ples of 100 marbles. Most people might empty the jar
and just count all 5,000, but you want to see what is
going on. The results of your 130 different samples
reveal a clear pattern. The most common mix of blue
and white marbles is 50/50. Samples that are close to
that split are more frequent than those with uneven
splits. The population parameter looks like 50 percent
white and 50 percent blue marbles.
Sampling 77
WRITING PROMPT
Sampling Frames
In the ideal situation, we would have a list with every element in the
population from which we could then randomly select. However,
most of the time, we must substitute a sampling frame. Describe
how incompleteness in a sampling frame can cause distortions or
misrepresentation in a sample, even if the sampling process is good
and sample size large.
The response entered here will appear in the
performance dashboard and can be viewed by
your instructor.
Submit
4.4.1:  The Sampling Distribution
Mathematical proofs and tests like that with white and
blue marbles both show that the pattern in Figure 4.4
always happens when we draw many sample. The sam-
pling distribution shows the same bell-shaped pattern
whether the sample size is 1,000 instead of 100; if there
are 10 colors of marbles instead of 2; if the population has
100 marbles or 10 million marbles instead of 5,000; and if
the population of interest is people, automobiles, or col-
leges instead of marbles. In fact, the more random sam-
ples you draw, the clearer the pattern becomes. The
sampling distribution shows that over many separate
samples, the true population parameter (i.e., the 50/50
split in the preceding example) is the most frequent
result. Some samples deviate from it, but they are less
common. Plus, we can calculate mathematically how
“less common” they are. Perhaps you heard of the bell-
shaped or normal curve. If you plot many different ran-
dom samples in the graph, you will see the bell-shaped
curve in the sampling distribution. This curve is widely
used in probability theory (see Figure 4.4).
If you say “I only need one sample and do not have the
time or energy to draw many different samples,” you
would not be alone.
Researchers rarely draw numerous samples. They use
knowledge of what always happens to generalize from one
sample to the population. The mathematical proof is about
many samples, but it also allows us to calculate the proba-
bility of a particular sample being off from the population
parameter. This is because we know that any one sample is
somewhere in the bell-curve pattern of all possible samples
that make up the sampling distribution. The mathematics
of bell-shaped curves lets us estimate the odds that one
particular sample is near the center of the bell curve or off
in one of its tails.
Random sampling does not guarantee that every ran-
dom sample perfectly represents the population. It means
that most times, a proper random sample yields results
78  Chapter 4

Figure 4.4  Sampling Distribution Example

A sampling distribution
Blue White Number of Samples

42 58 1
43 57 1
45 55 2
46 54 4
47 53 8
48 52 12 Number of blue and white marbles that were
49 51 21 randomly drawn from a jar of 5,000 marbles
50 50 31 with 100 drawn each time, repeated 130
51 49 20 times for 130 independent random samples.
52 48 13
53 47 9
54 46 5
55 45 2
57 43 1
Total 130

Number of Samples

31 *
30 *
29 *
28 *
27 *
26 *
25 *
24 *
23 *
22 *
21 * *
20 * * *
19 * * *
18 * * *
17 * * *
16 * * *
15 * * *
14 * * *
13 * * * *
12 * * * * *
11 * * * * *
10 * * * * *
9 * * * * * *
8 * * * * * * *
7 * * * * * * *
6 * * * * * * *
5 * * * * * * * *
4 * * * * * * * * *
3 * * * * * * * * *
2 * * * * * * * * * * *
1 * * * * * * * * * * * * * *

42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57
Number of Blue Marbles in a Sample

that are close to the population parameter. The mathemat-
ics supporting random sampling enables us to estimate
how close results are to the population parameter. The
same logic lets us calculate sampling errors and estimate
the probability that a particular sample is unrepresenta-
tive. In short, information from one sample allows us to
estimate a sampling distribution of many samples. The
mathematics of sampling distributions can tell us the
chances that sample results deviate from the true popula-
tion parameter.
A central idea in random sampling is: We do not have
100 percent accuracy 100 percent of the time. In reality, such
accuracy is virtually impossible in most endeavors. Life is
full of risk and chance. Some activities have a high risk of
serious mishap or death, such as skydiving, speeding
while driving drunk, or defusing a live bomb. Other activi-
ties have a very low risk such as walking to the local store,
sitting in a classroom, or eating your lunch. We cannot pre-
dict an outcome with 100 percent accuracy, but we know
the probability of an outcome varies by type of activity.
 Sampling 79

Figure 4.4  Continued

Number of Samples

31 *
30 *
29 *
28 *
27 *
26 *
25 *
24 *
23 *
22 *
21 * *
20 * * *
19 * * *
18 * * *
17 * * *
16 * * *
15 * * *
14 * * *
13 * * * *
12 * * * * *
11 * * * * *
10 * * * * *
9 * * * * * *
8 * * * * * * *
7 * * * * * * *
6 * * * * * * *
5 * * * * * * * *
4 * * * * * * * * *
3 * * * * * * * * *
2 * * * * * * * * * * *
1 * * * * * * * * * * * * * *

42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57
Number of Blue Marbles in a Sample

Although random sampling is not 100 percent accu-

Summary Review
rate, we have something extremely powerful. We can cal-
culate the size of the probability that a sample differs
from the population. If the probably of a sample differing Twelve Terms in Random
is very small, we have confidence that what is in a sam- Sampling
ple holds true in the entire population. This confidence in
a sample representing the population applies more 1. Confidence interval
broadly. As many different studies with high confidence
2. Population
samples arrive at the same results, our belief that we
3. Population parameter
have an accurate picture of the social world strengthens.
4. Random selection process
Later, we will look at using these ideas to create a zone of
5. Sample
confidence.
80  Chapter 4
6. Sampling distribution
7. Sampling element
8. Sampling error
9. Sampling ratio
10. Sampling frame
11. Target population
12. Universe
WRITING PROMPT
Sampling Distributions
A sampling distribution is a collection, or distribution, of many sepa-
rate samples drawn from the same population. Explain how a collec-
tion of many samples can help us learn more about a population
than just taking one sample.
The response entered here will appear in the
performance dashboard and can be viewed by
your instructor.
Submit
4.5:  Types and Uses
of Random Samples
4.5 Identify the three main types and uses of random
sampling
Although we have talked about the random sample gener-
ally, there are actually several types:
• systematic
• stratified
• cluster sampling
All are slight variations on the true, simple random sample.
4.5.1:  Systematic Sample
Sometimes we do not have a computerized random num-
ber generator to create a pure random selection process
and might use a quasi-random method, systematic sam-
pling. People often used it before random number gener-
ators became widely available. It is still necessary to
number each element in the sampling frame, as in a sim-
ple random sample, but instead of using a pure random
process to create a list of random numbers, we calculate a
sampling interval. It tells us how many elements to skip
in the sampling frame before selecting one for the sample.
Calculating the sampling interval is very easy. Divide
the total number of elements in the frame by the sample
size, and round to the nearest whole number. For instance,
your sampling frame has 1,800 names of clients for a social
service agency, and you want a sample of 300. Divide the
number of clients by the sample size, or 1,800/300 = 6 to
get the sampling interval. It tells you to skip over five
names then pick the sixth one for the sample. Repeat the
process until 300 names are in the sample. In many
instances, the sample produced with systematic sampling
will differ little from random selection.
Avoid using systematic sampling if elements in the
sampling frame are grouped or organized in a pattern,
because the resulting sample can deviate significantly from
that produced by random selection. Grouped elements
make it possible to skip over elements with a sampling
interval. For example, the elements are individual names
grouped into four-person family units. With a sampling
interval of 16, the sample would contain one family mem-
ber from every fourth family unit and regularly skip over
other people from the same family unit or people in adja-
cent family units (see Table 4.1).
Table 4.1  Problems with Systematic Sampling
of Cyclical Data
Case
1 Husband 7 Husband
a
2 Wife 8 Wife
3 Husband 9 Husband
4
5
Wife
Husband
10a
11
Wife
Husband
6a Wife 12 Wife
Random start = 2; Sampling interval = 4.
a
Selected into sample.
Making It Practical: How to Draw Simple Ran-
dom and Systematic Samples  The processes of
drawing a simple random and a systematic sample are
slightly different, but they usually yield very similar results
(see Table 4.2).
1. Number each case in the sampling frame in sequence.
The list of 40 names is in alphabetical order, numbered
from 1 to 40.
2. Decide on a sample size. We will draw two 25 percent
(10-name) samples.
3. For a simple random sample, locate a random-number
table (see excerpt). Before using random-number
table, count the largest number of digits needed for
the sample (e.g., with 40 names, two digits are needed;
for 100 to 999, three digits; for 1,000 to 9,999, four dig-
its). Begin anywhere on the random number table (we
will begin in the upper left) and take a set of digits (we
will take the last two). Mark the number on the sam-
pling frame that corresponds to the chosen random
number to indicate that the case is in the sample. If the
number is too large (over 40), ignore it. If the number
appears more than once (10 and 21 occurred twice in
the example), ignore the second occurrence. Continue
 Sampling 81

until the number of cases in the sample (10 in our Simple

example) is reached. No. Name (Gender) Random Systematic
4. For a systematic sample, begin with a random start. You 33 McKinnon, K. (F)
can just point blindly at the random number table then 34 Min, H. (F) Yes Yes (4)
take the closest number that appears on the sampling 35 Moini, A. (F)
frame. In the example, 18 was chosen. Start with the 36 Navarre, H. (M)
random number. Next, count the sampling interval, or 37 O’Sullivan, C. (M)
4 in the example, to come to the first number. Mark it, 38 Oh, J. (M) Yes (5)
and then count the sampling interval for the next num- 39 Olson, J. (M)
ber. Continue to the end of the list, counting the sam- 40 Ortiz y Garcia, L. (F)
pling interval as if the beginning of the list was
attached to the end of the list (like a circle). Keep
counting until ending close to the start. It will be at the
start if the sampling interval divides evenly into the Excerpt from a Random-Number Table (for Simple Random
total of the sampling frame. Sample)
15010 18590 00102 42210 94174 22099
90122 38221 21529 00013 04734 60457
Table 4.2  Example of Systematic Sampling and Excerpt
from a Random-Number Table (for Simple Random Sample) 67256 13887 94119 11077 01061 27779
13761 23390 12947 21280 44506 36457
Simple
81994 66611 16597 44457 07621 51949
No. Name (Gender) Random Systematic
79180 25992 46178 23992 62108 43232
01 Abrams, J. (M)
07984 47169 88094 82752 15318 11921
02 Adams, H. (F) Yes Yes (6)
*Numbers that appeared twice in random numbers selected
03 Anderson, H. (M)
04 Arminond, L. (M)
05 Boorstein, A. (M)
06 Breitsprecher, P. (M) Yes Yes (7)
07
08
Brown, D. (F)
Cattelino, J. (F)
Tips for the Wise
09 Cidoni, S. (M) Researcher
10
11
Davis, L. (F)
Droullard, C. (M)
Yes*
Yes
Yes (8)
Drawing Systematic Samples
You do not want to start systematic sampling at the beginning
12 Durette, R. (F)
of the list. If everyone always started at the beginning, names
13 Elsnau, K. (F) Yes
at the very beginning would never be in a sample. This violates
14 Falconer, T. (M) Yes (9)
the principle of being mathematically random—each element
15 Fuerstenberg, J. (M) has an equal probability of being selected. The simple solution
16 Fulton, P. (F) is to pick a random starting place, then use your sampling
17 Gnewuch, S. (F) interval. Once you reach the end of the list, treat the list as if it
18 Green, C. (M) START, Yes (10) were a loop. Continue counting off based on the sampling
19 Goodwanda, T. (F) Yes interval until you are back where you began. In most cases, a
20 Harris, B. (M)
simple random sample and a systematic sample yield virtually
equivalent results.
21 Hjelmhaug, N. (M) Yes
You can usually substitute systematic sampling for a sim-
22 Huang, J. (F) Yes Yes (1)
ple random method, but there is a situation when you do not
23 Ivono, V. (F) want to use it. Do not use it when the elements in a sampling
24 Jaquees, J. (M) frame are in a cycle or repeated pattern. Referring back to
25 Johnson, A. (F) Table 4.1, you have a list of heterosexual couples who married
26 Kennedy, M. (F) Yes (2) in the past four years organized by couple, with the male name
27 Koschoreck, L. (F) first and the female second. Your sample will not be
28 Koykkar, J. (M) ­representative if you use systematic sampling. To illustrate
why, let us say you used systematic sampling with a sampling
29 Kozlowski, C. (F) Yes
interval of four. By taking every fourth name, your sample
30 Laurent, J. (M) Yes (3)
would include only the females. The easiest solution is to use a
31 Lee, R. (F)
simple random sample instead.
32 Ling, C. (M)
82  Chapter 4

WRITING PROMPT schools from each stratum to get eight schools. Next, she ran-
domly selected a single class from each grade level at each
Systematic Sampling
school. It was the largest required course (usually E
­ nglish). As a
Since systematic sampling is a type of “short-cut” method to get a result, she administered the survey in 32 classrooms, where 72.8
representative sample, in what ways is it faster or easier than draw- percent of the students completed the survey. This yielded 982
ing a true random sample? What are some of its limitations?
completed surveys. She then examined rates of hazardous
The response entered here will appear in the drinkers and other factors. She found that the strength of individ-
performance dashboard and can be viewed by ual-level risk and protective factors varied by type of neighbor-
your instructor. hood that fed into a school.

Submit

4.5.2:  The Stratified Sample Learning from History

In some situations, it is important to make certain to include
specific kinds of diversity in a sample. For example, you
learn that people with four physical disabilities (walking,
seeing, hearing, and speaking) are 8 percent of population
you want to study. You may want to be certain that they also
are 8 percent in your sample. In a simple random sample, it
is possible that you will randomly select more or less than 8
percent of this group of interest to you. Stratified sampling
addresses this situation. In stratified sampling, we first
divide the population into subpopulations (strata). Stratified
sampling requires that we have information about strata in
the population, and this can be a limitation. To draw a strati-
fied sample, we create multiple sampling frames, one for
each stratum or subpopulation (see Figure 4.5).
Figure 4.5  Stratified Sampling
Step 1: Divide the population into subpopulations or strata
8% have one of the
4 disabilities
10,000 employees
Step 2: Create a sampling frame for each strata
800 employees 9,200 employees who do not
who have one of the have any of the 4 disabilities
4 disabilities
Step 3: Decide on a sample size and randomly select from each sampling frame
General Social Survey
Oversample
In most situations, you want each part of a sample to be the
exact same proportion as in the population. However, at times
researchers select a proportion different from what it is in the
population. They do this when a segment of the population is
a small proportion of the whole and wish to examine it in
depth. If a segment of interest were 5 percent of a population
of people, there would only be 10 for a sample of 200. This is
too few to examine the effect of several factors in a research
question (e.g., do children of divorced parents suffer in school
more than nondivorced parents?). For this reason, research-
ers sometimes purposely oversample. They sample a larger
percent of a segment than its size in the population. This way
they can analyze details of the segment. The General Social
Survey, a sample of all adult Americans, in 1987 and 2006
oversampled African ­Americans. A purely random sample of
the U.S. population yielded 191 blacks, about 13 percent of
the sample and close to the actual percent of African Ameri-
92% do not have one
cans in the U.S. adult population. Because researchers
of the 4 disabilities
wanted to analyze details about African Americans, they drew
a separate sample of African Americans to increase the total
number of blacks to 544. The 544 blacks were 30 percent of
a disproportionate sample. The oversampling allowed a more
in-depth study of African Americans than using a sample of
only 191. The larger sample also better reflected the diversity
among African Americans. When researchers looked at the
entire U.S. population, they only used the 191 African
­Americans in the pure random sample.

16 184

WRITING PROMPT
Sample of 200
employees
Stratified Sampling

If done properly, stratified sampling can be even more accurate than

a simple random sample. Why is this the case?
Example Study List the extra information we need to have in order to draw a
stratified sample instead of a simple random sample.
Stratified Sample The response entered here will appear in the
performance dashboard and can be viewed by
Browning (2012) used stratified sampling to study alcohol use
your instructor.
among teens in Toronto, Canada. She selected high schools and
stratified them by region (city vs. outskirts) and socioeconomic
status of the school (low vs. other), then randomly selected two
Submit
 Sampling 83

4.5.3:  A Cluster Sample clusters. In the study that opened this chapter, the authors
began with the population of all Boston area high schools.
A cluster is a grouping of the elements in the population. In
They had a sampling frame of all English classes by grade
cluster sampling, we treat each cluster as temporary sam-
for the schools. They sampled one English class from each
pling element. Frequently, there is no good sampling frame
grade in each school. They did not need to have a sampling
for a smaller unit, but there is a good sampling frame for

Figure 4.6  Cluster Sampling Example

Step 1: Randomly select a county among the 3,143 counties and similar units in the United States.

(a) (b)
Step 2: Randomly select a city block or section of the county.

Step 3: Randomly select a household within the block or section.

Step 4: Randomly select an individual within the household.

84  Chapter 4
frame of individual students, but used a “cluster” of stu-
dents that was the English class.
Let us consider a multi-level cluster example. Perhaps
you want to sample all teachers employed at colleges in
North America that enroll over 800 students. For a simple
random sample, you need a sampling frame, but there is
no list of all teachers. Instead of using a single sampling
frame, you can use multiple-stage sampling with clusters.
In the example, you may not have a list of all teachers, but
you can get an accurate list of all 5,000 colleges that enroll
over 800 students. You can create an accurate sampling
frame of colleges, i.e., clusters of teachers. To proceed, you
first sample the clusters or the colleges. For example, you
randomly sample 150 from a sampling frame of all 5,000
colleges in North America. Each college maintains a list of
its teachers. You now can contact the 150 colleges in your
first sample to get a list of teachers at each, or a second
sampling frame. You then draw a second sample of teach-
ers from each selected cluster, or college. If you decided to
sample five teachers per college, you would have a final
sample of 5 × 150 = 750 teachers.
WRITING PROMPT
Cluster Sampling
Suppose you want to conduct face-to-face interviews with a sample
of 50 youth soccer players to learn what got them interested in soc-
cer. The official rules say that every team must have a roster with 18
players and all teams have the full amount allowed. There are 60
teams scattered across the town.
Describe how cluster sampling can make your task manageable
and help you get a good sample.
The response entered here will appear in the
performance dashboard and can be viewed by
your instructor.
Submit
Advantages and Limitations of Cluster Sam-
pling  Cluster sampling has a large practical advantage.
You can often create a good sampling frame of clusters,
even if it is impossible to create one for the final sampling
elements. With a sample of clusters, creating a sampling
frame for elements within each cluster is often manage-
able. This example had two stages, but it is possible to have
more than two stages or a set of larger clusters that contain
small clusters, which in turn contain elements.
Cluster sampling has practical advantages and is less
expensive than simple random sampling, but it is slightly
less accurate. Each stage in cluster sampling introduces
sampling errors. Generally, a multistage cluster sample has
more sampling errors than a single-stage random sample.
A major limitation in cluster sampling occurs when the
clusters are of unequal size (i.e., not have the same number
of elements). When this happens, we make adjustments.
Example
If all colleges in the North America had the exact same num-
ber of teachers, we could use the procedure outlined above
with a sample of five teachers from each college. However,
as you might know, some colleges have 50 teachers whereas
others have over 2,000. When the number of elements per
cluster is of unequal size, selecting the same number from
each cluster violates the principle of random selection that
each element has an equal chance of being in the sample.
Let us see what goes wrong when clusters are of
unequal size and you sample from each cluster is the same.
After sampling the 150 colleges, the probability of selecting a
teacher from a college that has 50 teachers is 5/50 or 10
percent; however, the probability of selecting a teacher from
a college with 2,000 teachers is 5 in 2,000, or 0.25 percent.
The odds of a teacher at a small college with 50 teachers
ending up in your sample are 10 percent, or 1 in 10 of the 50
goes into the sample. The odds of sampling at a large univer-
sity with 2,000 teachers are 0.25 percent, or 1 in every 400
goes into the sample. Clearly, all teachers do not have the
same chance of being selected.
We must adjust so that sample elements from each
cluster (or college in this example) have an equal chance of
being selected. The technical details of how to adjust sam-
ples with unequal clusters are not necessary for the purpose
of our discussion, but the principle is easy to see. To create a
representative cluster sample, you adjust the number of
teachers from a small versus a large college to make the
probability of selection equal. Let us say you sample one
teacher from a college that has 50. The odds of selecting a
teacher from that college are 1 in 50, or 2 percent. To sample
with the same chances (2 percent) from a large university
that has 2,000 teachers, you need to select 2 percent of
2,000, or 40 teachers. This means after sampling the clus-
ters (or colleges) and learning how many elements (teachers)
are in each, we adjust the sample sizes by size of the clus-
ters. We want the samples taken from each cluster to be
proportionate to the cluster’s size. Small clusters will have
small samples, and larger clusters will have larger samples.
Example Study
Cluster Sample
David Kirk and Andrew Papachristos (2011) studied whether
local beliefs about the law and law enforcement influence levels
of violence in U.S. urban neighborhoods. This study was men-
tioned in Chapter 3 when its footnote on funding sources was
quoted. The authors noted that levels violence in U.S. cities,
such as murders, have long been associated with structural fac-
tors, such as high poverty and unemployment rates, substan-
dard housing, female-headed households, and unstable
residence. They argued that a cultural belief, legal ­cynicism, may

also explain violence levels. If people trust and believe in the fair-
ness of the law, the courts, police, and law enforcement, they
will call the police and rely on the legal system to resolve con-
flicts. However, if they have high levels of legal cynicism, i.e.,
they distrust and fear the police and believe the legal system is
unfair or ineffective, they will instead rely on nonlegal methods to
resolve local conflicts (i.e., illegal gangs or individual acts of vio-
lence). To examine the legal cynicism, the authors looked at sur-
vey data from a large-scale study, the Project on Human
Development in Chicago Neighborhoods (PHDCN) that used
cluster sampling. The PHDCN grouped the 865 census tracts
in the City of Chicago into 343 homogenous “neighborhood
clusters.” Each cluster had similar social-economic features,
natural boundaries (e.g., railway tracks, major highways), and
was of equal size, about 8,000 people. The researchers drew a
random sample of 80 from the 384 clusters. In each of these 80
sampled neighborhoods, they randomly selected city blocks.
Next, they went down the randomly selected blocks and com-
piled a list of all dwellings on the blocks. This created a list of
roughly 40,000 dwellings. They next went to a sampled dweling
and screened people to randomly select 8,782 residents for
interviews. The multi-stage cluster sampling had four levels:
1. neighborhood clusters,
2. city blocks,
3. dwellings on city blocks, and
4. residents in the dwellings.
The researchers found that even in neighborhoods that
had improved with regard to structural factors, the level of vio-
lence remained high if the residents expressed high levels of
legal cynicism.
WRITING PROMPT
Neighborhoods as Clusters
Imagine that you want to sample individuals in a medium-sized city
and your first cluster was the neighborhood, how might you continue
after you drew a random sample of 15 neighborhoods (from among
55) of the city to get to the individual level? Would you want the 15
neighborhoods that are similar to each other, or very different types
of neighborhoods? Why?
The response entered here will appear in the
performance dashboard and can be viewed by
your instructor.
Submit
4.6:  Sampling in Difficult
or Specialized Situations
4.6 Identify potential challenges presented in
sampling for research and how to effectively
manage those difficulties
At times, we want to interview members of the public or a
large, spread-out collection of people by telephone. To get
Sampling 85
a good random sample, we need a complete sampling
frame but published telephone directories are not accept-
able. We miss four kinds of people using a telephone direc-
tory as the sampling frame:
• People without telephones
• People who have recently moved
• People with unlisted numbers
• People who only use a cell phone
Any telephone interview study will miss people without
phones (e.g., poor, uneducated, and transient people), but
this is usually not a big problem since nearly 97 percent of
people have phones. As more people got phones, the per-
centage of them with unlisted numbers also grew. In some
urban areas, over 50 percent of phone numbers are unlisted.
In addition, people change their residences, so directories
often have numbers for people who have left and do not
include those who have recently moved into an area. One
report suggests that as of 2014 about 40 percent of Ameri-
cans only have a cell phone, no landline.*
4.6.1:  Random-Digit Dialing
Random-digit dialing (RDD) avoids the problems of tele-
phone directories by randomly sampling possible tele-
phone numbers, not a list of people with telephones. This
avoids the bias of using listed numbers. RDD is not diffi-
cult, and there are several specialized computer programs
designed to make the calls. However, it takes time and can
frustrate the person doing the calling. Many of the num-
bers may not be operating. As with any sampling that uses
the telephone, you must retry reaching a selected, working
number many times (5–6 attempts at different times) before
giving up on it.
Making It Practical: How Random-Digit Dial-
ing Works  In the United States, a telephone number
has three parts: a three-digit area code, a three-digit ex-
change number or central office code, and a four-digit
number. It is easy to create a list of all possible phone num-
bers by getting a list of active area codes and three-digit
phone exchanges in the area code. Possible phone numbers
in an exchange go from 0000 to 9999. In RDD, a computer
randomly selects a number (0000 to 9999) in an exchange
and makes the call. Some selected numbers are out of ser-
vice, disconnected, pay phones, or numbers for businesses.
Only some numbers are what you want—a working resi-
dential phone number. Until you call, it is not possible to
know whether the number is a working residential num-
ber. This means spending a lot of time getting disconnected
numbers, numbers for businesses, and so forth. The sam-
pling element in RDD is the phone number, not the person
*
http://www.pewresearch.org/fact-tank/2014/07/08/two-of-every-five-u-s-
households-have-onlywireless-phones/
86  Chapter 4
or the household. Several families or individuals can share
the same phone number, or each person may have a sepa-
rate phone number or more than one number. This means
that after contacting a person at a residential phone, a sec-
ond stage is necessary, within-household sampling, to
select the person who you will interview.
WRITING PROMPT
Random Digit Dialing
Why is Random Digit Dialing (RDD) widely used in telephone survey
research? What are some of the limitations or drawbacks of RDD?
The response entered here will appear in the
performance dashboard and can be viewed by
your instructor.
Submit
4.6.2:  Within-Household Sampling
We use within-household sampling to address situations
in which we sample households. Building on the idea of
cluster sampling, the household is a kind of cluster with
multiple sampling elements, or individuals. We want to
avoid a possible bias of having one member of the house-
hold always being in a sample because he/she is the one
who regularly answers the phone, or mail, or door. One sit-
uation occurs with telephone interviewing when we have
one phone number for several people in a household. This
is a common situation when we use RDD and have a sam-
pling frame that is a list of phone numbers. Another situa-
tion occurs with face-to-face or mail surveys when we want
to sample the individuals in a town, city, or state but do not
have a current, complete list of each person in the area. It
might be possible to create a list of all addresses (i.e., all
houses, apartments, and similar residential locations).
In both situations, instead of picking the first person
who answers a phone, or who comes to the door or who
happens to read the mail first, we sample individuals within
a household. For within-household sampling, we first deter-
mine the number of eligible members in the household, such
as adults over a certain age. If there is only one person, we
choose that person. If there are two or more people, we ran-
domly pick one. Let us say you consider all adults over 18
years old who reside in a household. There are three eligible
people at one house: a 50-year-old woman, a 53-year-old
man, and a 20-year-old woman. To select a person randomly,
we use selection rules, such as the following:
• If the last sampled person interviewed was male, select
a female (or vice versa).
• If there is one male or one female, interview that person.
• If there are two females/males, select the oldest one
first, next time select the youngest.
The purpose of randomly selecting one person from the
household is to avoid the bias of including only certain
types of people, whomever always answers the phone or
door first. However, within household sampling often
adds time, because the individual of the household ran-
domly selected may not be immediately available.
4.6.3:  Sampling Hidden
Populations
Sometimes a particular type of person is very difficult to
locate, making sampling more complicated. A hidden
­population includes people who engage in concealed
activities. They are often central in the studies of deviant or
stigmatized behavior. Examples of hidden populations
include users of illegal drugs, prostitutes, people with
HIV/AIDS, people on parole, or homeless people.
We need to make adjustments to sample people in a
hidden population because they are more difficult to
locate than the general population of visible and accessi-
ble people. To sample hidden populations, we need to be
creative and use nonprobability sample techniques, such
as purposive or snowball sampling. As you have seen,
there are many different types of samples and the type of
sample is based on the study purpose and type of data
being gathered.
Example Study
Hidden Populations
Draus and associates (2005) sampled a hidden population in
a field research study of illicit drug users in four rural Ohio
counties. The researchers used a version of snowball
sampling, respondent-driven sampling (RDS), which is used
when members of a hidden population maintain contact with
one another.
RDS begins by identifying an eligible case or partici-
pant. The researchers give this person, called a “seed,” refer-
ral coupons to distribute among other eligible people who
engage in the same activity. For each successful referral, the
“seed” receives some money. This process is repeated with
several waves of new recruits until a point of saturation (no
new people). Draus and associates interviewed a drug-using
participant who was paid $50 for an initial two-hour interview
and $35 for an hour-long follow-up interview. The partici-
pants received three referral coupons at the end of the initial
interview. They got $10 for each eligible participant they
referred who completed an initial interview. No participant
received more than three referral coupons. Sometimes this
yielded no new participants, but at other times more than
three people were recruited. In one case, a young man heard
about the study at a local tattoo parlor. He called the study
office in July 2003. He (­participant 157) had been a powder
 Sampling 87

cocaine user. During his interview, he said that he knew

many other drug users. He referred two new participants 4.7:  Making Inferences
(participants 161 and 146), who came in about one month
later. Participant 161 did not refer anyone new, but partici- from a Sample
pant 146 referred four new people, and two of the four (154
and 148) referred still others. Participant 154 referred four to Population
new people and 146 referred one new person, and that one
4.7 Examine how sampling errors lead to incorrect
person (participant 158) referred four others. Between June
inferences
2002 and February 2004, the researchers interviewed 249
users of cocaine or methamphetamine with this sampling A probability sample allows us to make valid inferences from
process. the sample to the population. We measure or observe vari-
ables for the units in the sample, not all units in the popula-
tion, but we infer that what we see in the sample also is true
for the population. For example, you may have information
on 750 sampled teachers, not the hundreds of thousands of
Summary Review
North American teachers, or on 200 sampled employees, not
all 10,000 who work for Specialized Consulting Services, Inc.
Types of Samples A sample stands in for, or represents, the entire popu-
lation. We are not interested in a sample in itself; we want
Table 4.3  Nonrandom Samples to talk about the population, but can use the data on the
sample to do this. When we draw a random sample, we
Type of Sample Sample Selection
want to minimize any gap between the sample and popu-
Convenience Select any sample element that is
­convenient lation, or sampling error, to make inferences. Next, we will
Quota Select a fixed number of sample elements look at what affects sample error size.
into preset categories of the population
Purposive Use diverse methods to select sample
­elements that match narrowly pre-defined 4.7.1:  Sampling Errors
criteria
The mathematics to calculate the sampling error are
Snowball Select based on direct or indirect connec-
tions to a few sample elements
beyond this text, but a conceptual picture of it is not very
difficult. Imagine you are the manager of XYZ department
store. Last year, 90,000 shoppers visited the department
store. You do not know it, but exactly 69 percent were
Table 4.4  Random Samples female (i.e., the population parameter). For a sample, you
Type of Sample Sample selection picked 10 days at random. The store has only four
Simple Random Select elements from a sampling frame
entrances. You position cameras and station observers at
using a pure random process each entrance. During the 10 days, 800 shoppers entered.
Systematic Select elements from a sampling frame Based on the cameras and observers, you determined how
using a sampling interval many were male and how many were female. Purely by
Stratified Select elements from preset categories random chance, you get 66 percent female in the 10-day
using a pure random process
sample. You estimate the population parameter to be
Cluster Create multiple samples, first randomly
select clusters, then randomly select 66 percent female. You know you might be off from what is
­elements within each cluster to create the true in the total population. This is sampling error, an esti-
final sample
mated based on a sample that does not necessarily match
the population parameter. In this case, it is the 3 percent
difference between the 69 percent true population param-
Table 4.5  Special Techniques eter and 66 percent estimate based on the sample.
Type of Sample Sample selection Two factors influence the sampling error: the sample
Random Phone Use RDD to select phone numbers out of size and the diversity of cases in the sample:
all possible phone numbers
• The larger the sample size, the smaller the sampling
Within Household Randomly select individuals, usually by
Sample gender and age order, within a sampled error.
household • The greater the homogeneity (or the less the diversity),
Hidden Populations Use purposive or snowball methods to the smaller its sampling error will be.
select elements from difficult to reach pop-
ulations
If you have a large sample (8,000 randomly selected from
80,000) and there is very little diversity among cases, the
88  Chapter 4

sampling error will be tiny. If you have a small sample (80 size, shrinks. ­Without getting into the complexity of this,
randomly selected from 80,000) and there is great diversity three simple principles can help you make sense of what is
among cases, the sampling error will be very large and important:
your sample may not accurately represent the population.
1. For small populations (less than 500), you need one
half or more in your sample, and the required sample
4.7.2:  Sample Size size grows very fast as the population size gets smaller.
A large sample size alone does not guarantee a representa- 2. For target populations over 5,000, you need 17.5 per-
tive sample. A large sample without random sampling or cent or 27 percent of the population, depending on
with an inaccurate sampling frame will be less representa- degree of confidence. Sample size changes very little as
tive than a smaller sample that uses random sampling and the population size grows larger.
has an excellent sampling frame. 3. Once your target population is over 250,000, the sam-
Calculating sample size mathematically requires ple size hardly changes at all.
making assumptions, estimating population size and
Practically, this means that if you want to sample a very
diversity, knowing how many variables you plan to
small target population, such as the 50 employees of the
examine, determining how confident you want to be, and
local fast food outlet, just include everyone, or a sample
the degree of accuracy you require. The mathematics and
ratio of close to 100 percent. However, to sample a city of
calculations are beyond the level of this text. Most people
250,000 people, you can be equally accurate with a sam-
just use “rules of thumb.” These are rough approxima-
ple of 1,063 in your sample, or a sampling ratio of 0.4 per-
tions based on target population size, assuming you
cent. The main idea about sample sizes is that the smaller
examine one or a few variables, there is moderate popula-
the ­population, the bigger the sampling ratio must be for an
tion diversity, and you need medium accuracy (such as
accurate sample. Larger populations permit smaller sam-
getting within 3 percent of the population parameter).
pling ratios for equally good samples because as the pop-
Table 4.2 provides sample sizes for a range of target pop-
ulation gets bigger, returns in accuracy for sample size
ulation sizes (50 to 100 million).
quickly shrink. This is why random sampling is so pow-
erful and efficient when you want estimates about large
populations.
Table 4.6  Sample Sizes for Two Levels of Confidence and
Various Population Sizes
4.7.3:  Confidence Intervals
Target Population 95 Percent 99 Percent
Size Confident Confident The confidence interval expresses a familiar idea. When
50 48 49 reporters discuss polling results, they say “the margin of
200 168 180 error being plus or minus 2 percentage points.” They are
500 340 393 using a simple version of confidence intervals. After you
1,000 516 648 draw a random sample, you may look at a characteristic or
5,000 879 1347 measure you get from the sample, such as the average
25,000 1023 1717
income or the percentage saying “agree” to a question. From
100,000 1056 1810
the above discussion of sampling error, you know this does
250,000 1063 1830
not indicate that the sample measure (income or percent
agree) is identical to what is in the population, or the popula-
1,000,000 1066 1840
tion parameter. You use the sample results to estimate the
100,000,000 1067 1843
population parameter.
The mathematics around a sampling distribution
Notice from the table that the sampling ratio is very helps us estimate a zone or range around what we find in a
large when the target population is small. When your sample within which the population parameter will be.
population is under 500, you will need about one-half the Interval in confidence interval is the zone or range around
population in your sample to be highly confident and what you found in sample. Confidence in confidence interval
accurate. If your population is under 100, you might as refers to the probability that the population parameter falls
well take everyone in the sample. However, if your popu- within the interval. A typical level of confidence is
lation is 25,000, you need less than 10 percent. This per- 95 ­percent. It means you can be 95 confident that the true
centage gets smaller as the population grows in size. As population parameter falls within the interval. A higher
the population size increases, the number in the sample level of confidence (99 percent) requires you to have a
size might grow too, but proportion from the population slightly larger sample size, everything else being identical.
that is in the sample, or the ratio of sample to population Sample size affects the confidence interval. Everything else

being the same, as your sample size gets bigger, the interval
gets narrower.
The mathematical calculations for sampling errors or
confidence intervals build on the ideas of the sampling dis-
tribution. For example, you cannot say, “There are pre-
cisely 2,500 red marbles in the jar based on a random
sample.” However, you can say, “I am 95 percent certain
that the population parameter of red marbles is between
2,450 and 2,550.” You can combine characteristics of the
sample (e.g., its size and variation) to set up an interval
and level of confidence. Let us say you have two identical
samples except that one is larger. A larger sample will have
a smaller sampling error and a narrower confidence inter-
val. A narrow confidence interval lets you be more precise
when estimating the population parameter.
An example illustrates the basic ideas. You will see
how sample size reduces the sample error, which in turn
affects the confidence interval.
Example
Let us say you want to know how many people are likely to
support a referendum to add a tax for a new school in a city
that has 5,000 eligible voters (i.e., a target population). You
want to be 99 percent confident. You get a good sampling
frame and use simple random sampling. At first, you draw a
random sample of 100 people and find that 52 percent sup-
port the referendum. Your confidence interval is 3.6 percent
above or below that number, or 48.4 to 55.6 percent sup-
port. In other words, you can be 99 percent certain that the
population parameter (true vote intention in the population)
lies between 48.4 and 55.6. The vote is too close to call,
and it could go either way. You want to be more certain, so
Summary: What You Learned about Sampling
You learned about several types of sampling in this chap-
ter. Sampling is widely used in social research. Some
types are and some are not based on random processes.
Only some nonrandom sample types are acceptable, and
qualitative researchers frequently use these types. In gen-
eral, random or probability sampling yields the most rep-
resentative sample. Quantitative researchers use it
because they often want to generalize from a small num-
ber of cases to a much larger collection. Random sampling
can produce samples that represent the population accu-
rately. Random samples also allow you to use powerful
statistical techniques.
Sampling 89
you draw a second sample. This time you increase the sam-
ple size to 500 eligible voters. You find that 51 percent sup-
port the referendum, but your confidence interval with this
larger sample size is narrower. It is 1.5 percent above or
below, or from 50.5 to 53.5 percent. It now looks as if the
vote will be close, but the referendum is likely to pass (see
Figure 4.7).
Figure 4.7  Confidence Interval with Sample of 100. 99%
Confident
99% confidence interval with a sample size of 100
55.6
48.4
52% estimate
99% confidence interval with a sample size of 500
53.5
50.5
WRITING PROMPT
Confidence Intervals
Give an example from an area of daily life (e.g., sports, cooking,
­politics, weather, travel, appointments) where the core idea of
­confidence intervals can be found, and explain how it illustrates
the concept.
The response entered here will appear in the
performance dashboard and can be viewed by
your instructor.
Submit
In addition to simple random sampling, you learned
about systematic, stratified, and cluster sampling. This
course does not cover the statistical theory used in ran-
dom sampling. However, from discussions of sampling
error and sample size, you saw that random sampling
produces more accurate and precise sampling. You may
wish to express the results from a sample in terms of a
confidence interval.
Before moving on, let us review a fundamental princi-
ple of social research: Do not compartmentalize the steps of
the research process; rather, see the interconnections
between the steps. Research design, measurement,
90  Chapter 4
s­ ampling, and specific research techniques are interde-
pendent. Unfortunately, the constraints of presenting infor-
mation in a textbook necessitate presenting each part
separately, in sequence. In practice, researchers think about
data collection when they design research and develop
measures for variables. Likewise, sampling issues influ-
ence research design, measurement of variables, and data
collection strategies. As you will see in future chapters,
good social research depends on simultaneously control-
ling quality at several steps—research design, conceptual-
ization, measurement, sampling, and data collection and
handling. Making major errors at any one stage could
make an entire research project worthless.
Quick Review
Why Do We Use Samples?
1. We often invest at lot of effort into sampling design for
quantitative studies because our goal is to create a genu-
inely representative sample (i.e., a sample that has all the
features of the population) that will enable us to make
accurate generalizations about the entire population using
sample data. The best representative samples use a ran-
dom selection process.
2. The data from a well-designed random sample are equally
or even more accurate than if we tried to reach everyone in
the population. In addition, the samples are highly efficient
in terms of time and cost.
3. A census is an official government count carried out at
periodic intervals. Usually, it attempts to count everyone,
but this is less accurate than a well-designed random
sample.
4. In qualitative research, our goal differs from getting a rep-
resentative sample of a large population, so instead of
using random sampling we use nonrandom types of sam-
pling. Our goal is usually to learn how a small collection of
cases, units, or activities can illuminate key features of
social life. Some types of nonrandom sampling are better
designed for this purpose.
Types and Applications of
Nonrandom Samples
1. Random samples are best to use when the goal is to cre-
ate an accurate representation of a population, but they
can be difficult to conduct and are not appropriate for all
purposes. Four nonprobability techniques are also used.
2. Convenience or haphazard samples are very quick and
easy, but can be very inaccurate and nonrepresentative.
This makes them inappropriate for most purposes.
3. Quota sampling yields a somewhat representative sample,
and it much easier to produce than random sampling, but
it has important limitations.
4. Purposive sampling is a valuable type of nonprobability
sample that is appropriate to use, especially in qualitative
studies, when the goal is to focus on a specifically tar-
geted set of cases or units.
5. Snowball sampling is a valuable type of nonprobability
sample. Its special design enables it to capture cases or
units that are within of an interlinked network.
The Terminology Used to Discuss
Random Sampling
1. Random sampling has its own terminology. In addition to
the previously introduced terms of universe, population,
and sample, some of key terms include target population,
sampling element, and sampling frame. The terms refer
respectfully to the specific, concrete population of a study,
the cases or units of that population, and a specific list of
every element.
2. Three other important terms are sampling ratio,
population parameter, and sampling error. The sampling
ratio is the size of the sample relative to the size of the
population. The population parameter is some character-
istic of interest in the entire population; we often estimate it
using the sample. The sampling error is the degree of mis-
match or deviation of a sample from the population. In
theory, a perfectly representative sample has a sampling
error of zero. While this is very rare, we can use statistics
with a true random sample to estimate the size of the sam-
pling error.
Producing a Simple Random
Sample
1. The process of producing a simple random sample
requires that we first list all elements of the target popula-
tion in a sampling frame and then use a true mathemati-
cally random process to select some elements from the
frame.
2. Some people have the misconception that a bigger sam-
ple is always best. If the sampling frame is inaccurate, or if
a nonrandom selection process is used, the sample can
be significantly “off” or unrepresentative of the population,
even if the sample size is very large. However, for two
samples of the same target population that use equally
accurate sampling frames and equally random selection
processes, the sample that is larger is likely to be more
representative of the population.
3. The sampling distribution is a collection of different ran-
dom samples taken from the same population, and then
plotted on a graph. Used with the mathematics of proba-
bilities, it enables us to estimate the size of sampling error
and the location of the population parameter. Although
random sampling is not 100 percent accurate each time,
using it enables us to calculate the probability that a spe-
cific sample deviates from the population.

Types and Uses of Random
Samples
1. Systematic sampling is a less complicated version of the
simple random sample. You still need to have a good sam-
pling frame, but instead of a random selection process you
create a sampling interval. After picking a random starting
place, you select sampling elements using the sampling
interval. So long as there is no cycle, the results are only
slightly less accurate than the simple random sample.
2. In stratified sampling, you first subdivide the sampling
frame into categories, or strata, then you draw a random
sample from each (or can take the entire population of a
strata if it is small). This requires you to have knowledge of
strata in the population and be able to subdivide it. This
can be more accurate than a simple random sample.
3. Cluster sampling is a type of multi-stage sampling process.
It is often used when there is no good sampling frame for the
final sample elements, but there are good sample frames for
clusters of the elements. In addition, clusters can be nested
within one another. You sample clusters then once you have
clusters, you can create a sampling frame for elements
within that cluster for another stage of sampling.
Sampling in Difficult or Specialized
Situations
1. A special sampling situation occurs when using tele-
phones to contact participants. Because there are usually
no comprehensive sampling frames of all participants, we
can sample phone numbers instead. With Random Digit
Dialing, all possible phone numbers become the sampling
frame, and phone numbers are randomly selected.
Sampling 91
2. When sampling households, either as a type of cluster or
by phone where one phone is shared by a household, we
need to sample randomly within the household. This helps
ensure that one household member who may regularly
answer is not oversampled.
3. Some populations are “hidden” because they engage in
socially disapproved or illegal behavior, are highly transient,
or lack a stable residence (e.g., homeless). Sampling such
populations requires additional efforts that vary with the cir-
cumstances of the particular hidden population.
Shared Writing: Samples You Can Believe In
Sampling principles are the same whether you are sampling
leaves from a forest floor, fish from the ocean, products on an
assembly line, or the residents of a town. In your opinion, what
makes drawing a true random sample of individual people in a
town more difficult or complicated than sampling leaves, fish, or
products? When reading about a sample of individual people in a
study, what features of the sample would you be looking for to
give you great confidence that the sample truly represents the
entire population?
A minimum number of characters is required
to post and earn points. After posting, your
response can be viewed by your class and
instructor, and you can participate in the
class discussion.
Post 0 characters | 140 minimum
Chapter 5
Measuring Social Life
Learning Objectives
5.1 Analyze how measures in social life shape
research outcomes and larger social issues
5.2 Describe how measurement in the social
world extends the range of our natural senses
5.3 List four ways in which systematic data
may be collected
5.4 Apply reliability and measurement validity
to create good measures
Are you a “redneck”? Do you know “rednecks?” What is a
“redneck anyway?” Carla Shirley (2010) conducted a
study to address this question. She chose four rural com-
munities in two eastern Mississippi counties. After con-
tacting community development clubs and churches
explaining that she was doing research on the lives rural
92
5.5 Describe how principles of reliability and
validity operate for qualitative data
5.6 List the characteristics of the four levels of
measurement as they relate to quantitative
measurement
5.7 Apply the processes involved in
the construction and use of indexes
and scales
white S­ outherners, she used purposive and snowball
sampling to contact and secure cooperation from 42
respondents. She conducted 1–3 hour in-depth interviews
with an equal number of white men and white women
aged 24 to 84 in private settings. Half of the respondents
had lived in their community for their entire lives. The

remainder came from other rural communities or metro-
politan areas of the American South. She asked a range of
questions about the meaning of being a white Southerner
and the types of people called “rednecks.” She also asked
for a definition, whether the term’s meaning had changed,
and whether media portrayals of rednecks were accurate.
Two-thirds of respondents said that besides being a white
Southerner, a redneck had a set of values or lifestyle. Nearly
all (84 percent) said the media and non-Southerners associate
the term with a Southern white person who is backwards,
uneducated, and racially prejudiced. Other “redneck” charac-
teristics include being of lower social status and someone
who has lived in the same rural community for his or her
entire life. Interestingly, 95 percent of respondents said the
term refers more to males than females, especially a male who
chews tobacco, drives a pick-up truck, and spends a lot of lei-
sure time fishing and hunting. Negative characteristics
include being self-centered and rude, combative, and racist.
The more negative term “white trash” overlapped somewhat
with that of redneck. Over one-third of the respondents iden-
tified male relatives or themselves as “rednecks.” Many
acknowledged that outsiders used the term more broadly
than its use among white Southerners. The term “redneck”
has multiple meanings, but its core refers to a white, rural
male Southerner of lower social status. To study any topic,
including rednecks, you first must clarify the concept’s mean-
ing and think seriously about how you might measure it.
5.1:  Why Measure?
5.1 Analyze how measures in social life shape research
outcomes and larger social issues
Measurement is everywhere, the Stanford Binet IQ test to
measure intelligence, the Index of Dissimilarity to indicate
racial segregation in a city, the J. D. Powers customer satis-
faction measure for new cars, the poverty line to determine
whether someone is poor, and uniform crime reports to
assess crime rates. We measure for many reasons: to evalu-
ate an explanation, test a hypothesis, provide a service pro-
gram, determine proper medical treatment, or resolve an
applied issue. Measurement is a critical part of doing
research. It transforms ideas and casual observations into
specific, concrete data and enables us to share thoughts
and observations with others.
Measurement issues are more central for quantitative
than qualitative data. In a quantitative data study, you create
measures early in the research process before collecting any
data. You start with a clearly thought out idea or concept, then
create a way to capture it precisely and accurately as num-
bers. In a qualitative data study, you also want to capture
ideas or concepts but without numbers. This often involves
an inductive approach. In all social research, you will want to
Measuring Social Life 93
measure types, amounts, frequency, intensity, duration, loca-
tion, and so forth about the concepts you wish to study.
Measures can profoundly shape both research out-
comes and larger social issues. Consider intelligence. Psy-
chologists debate what intelligence means and how to
measure it. Most intelligence tests used in schools, on job
applications, or in statements about racial or other inherited
superiority measure only one type of intelligence, analytic
reasoning (i.e., a capacity to think abstractly and to infer
logically). Most experts agree that humans possess multiple
types of intelligence besides the analytic type, such as prac-
tical, creative, social-interpersonal, emotional, body-kines-
thetic, musical, or spatial intelligences. If there are many
types of intelligence but schools and businesses are only
looking at one type, then schools and businesses are limited
in how they are evaluating, promoting, and recognizing
people’s contributions. The way we measure intelligence
influences our ability to value diverse human abilities.
Here is another example. Human service agencies allo-
cate assistance from social programs (e.g., subsidized hous-
ing, food aid, health care, child care, etc.) to people identified
as being poor. Government agencies shift funding to an area
based on the number of poor people living there. Politicians
and economists argue over rising and falling poverty rates.
5.1.1:  Who Is Poor?
Measuring poverty is controversial.
Some say a person is poor if he or she cannot afford essential
food required to prevent malnutrition. Others say a person is
poor if his or her annual income is less than one-half of the
average (median) income of everyone else. Still others say
that someone is poor if he or she earns less than a “living
wage.” A living wage is the income required to meet minimal
community standards for health, safety, hygiene, housing,
clothing, diet, transportation, and so forth. How we define
and measure poverty greatly i­nfluences agency decision
making and the daily living conditions of millions of people.
94  Chapter 5
Learning from History
Who Is Poor?
There are many definitions of poverty and ways to measure it.
Most studies examine relative poverty (i.e., comparing people
to others) and absolute poverty (i.e., requirement for bare phys-
ical survival). The most common measure internationally is a
certain percentage (e.g., 50 percent) below the average income
(i.e., median or midpoint) level of a society. The United States
created its official “poverty line” in the 1960s. The poverty line
was a temporary measure with an arbitrary dollar amount cre-
ated for the bureaucratic purpose of estimating the size of the
poor population. The line was based on a preexisting study of
the U.S. Department of Agriculture on the amount a family
would need to spend on food to meet its minimal nutritional
needs. To estimate minimal living costs, officials multiplied min-
imal food costs by 3. This is because in the 1950s most poor
people devoted roughly one-third of their incomes to buying
food. Ever since, the poverty line has become a permanent fix-
ture for official reports and the delivery of a range of social pro-
grams in the United States. It has not been updated except for
inflation increases and adjustments for the number of family
members. Experts agree that the current measure is inade-
quate and have repeatedly made recommendations to recalcu-
late the poverty line. They note changes in living conditions,
new services provided to people, and that few people spend
one-third of their income on food. Every attempt at a change
has become embroiled in political controversies. This is
because almost every suggestion for an improvement in mea-
surement accuracy would classify far more people as being
poor than by the current poverty line. Few politicians want the
publicity or cost of suddenly having many more people officially
called poor, as a result, a poverty measure with widely acknowl-
edged flaws continues. The current measure allows charting
time trends, but most poverty scholars as well as officials in
various levels of government acknowledge the need for multi-
ple measures that can capture poverty’s multiple dimensions,
and that more accurately capture how poor families now live.1
WRITING PROMPT
Anything Can Be Measured
Social scientists hold that nearly anything in the social world we wish
to study (love, friendship, happiness, racism, and so forth) can be
measured. Describe an aspect of the social world that you would like
to have a study measure? Why do you choose this? How might you
start to measure it?
The response entered here will appear in the
performance dashboard and can be viewed by
your instructor.
Submit
1
See Edin and Kissan, 2010
5.2:  How Do We Make
the Social World Visible?
5.2 Describe how measurement in the social world
extends the range of our natural senses
You use many measures in daily life. For example, this
morning I woke up and hopped onto a bathroom scale to
see how well my diet is working. I glanced at a thermome-
ter to find out whether to wear a coat. Next, I got into my
car and checked the fuel gauge to be sure that I could make
it to campus. As I drove, I monitored the speedometer so
that I would not get a speeding ticket. By 8:00 A.M., I had
measured weight, temperature, fuel volume, and speed—
all measures of the physical world. Such precise, well-
developed measures used in daily life are fundamental in
the natural sciences. We also measure the social world, but
usually in less exact terms. We measure when we say that a
restaurant is excellent, that Pablo is smart, that Karen has a
negative attitude toward life, that Johnson is prejudiced,
that children in Sunny Valley School perform at below
average levels, or that the real estate market in East River
City is hot. However, such everyday judgments as “really
prejudiced” or “hot market” are imprecise, vague, intuitive
measures.
Measurement extends the range of natural senses,
just as the astronomer or biologist uses a telescope or the
microscope to extend his or her natural vision. In addi-
tion to extending our senses, scientific measurement pro-
duces a more exact measure than found in ordinary
experience. It varies less with the specific observer. A
thermometer extends describing feeling beyond “hot or
cold.” It gives specific, precise communicable informa-
tion about temperature, and varies less from person to
person than touch. Likewise, a good scale gives you more
specific, constant, and precise information about the
weight of a 5-year-old girl than you get by lifting her and
calling her “heavy” or “light.” Social scientific measures
also provide precise information about features of the
social world.
Measures make visible ideas we have about the social
or physical world that otherwise are unseen. For example,
you cannot see a magnetic field with your natural senses.
Scientists had the idea that magnetism existed as a physical
force but they could not see it directly, so they developed
ways to observe the effects of invisible magnetic fields
indirectly. For instance, metal flecks move near a magnet.
The magnet lets us “see” or measure invisible magnetic
fields. Natural scientists have invented thousands of mea-
sures to “see” very tiny things (molecules or insect organs)
or very large things (huge geological landmasses or plan-
ets) that are not observable through ordinary senses. By
extending the reach of our senses, measurement extends
our range of knowledge.

Some of what we want to measure (e.g., age, height,
skin tone, eye shape, etc.) is visible, but we cannot directly
observe many other things of interest (e.g., employee sat-
isfaction, poverty, a child’s self-esteem, religious commit-
ment, desire to purchase, or quality of life). Just as natural
scientists invent indirect ways to observe the “invisible”
forces of the physical world, social researchers create
measures to reveal difficult-to-see aspects of the social
world.
5.3:  Do We Measure with
Numbers or Words?
5.3 List four ways in which systematic data may
be collected
All researchers use careful, systematic methods to collect
data but the process varies in four ways depending on
whether the data are quantitative or qualitative: timing,
direction, form, and linkages.
1. Timing. For quantitative data, first convert concepts
into variables, next convert variables into specific mea-
surement actions at a planning stage before from gath-
ering or analyzing data. For qualitative data, create
measures of concepts while collecting data. The pro-
cesses of thinking about concepts, collecting data, and
starting to analyze qualitative data all blur together.
Measuring qualitative data is integrated with other
research activities and not a separate stage.
2. Direction. Recall the inductive versus deductive direc-
tions of research. Most quantitative data research fol-
lows a deductive route: Start with the abstract idea and
end with visible empirical data. Most qualitative data
research follows an inductive route: Start with empiri-
cal data and end with a mix of ideas and data. In both,
the process is interactive, with measuring processes
influencing ideas and vice versa.
3. Data form. In a quantitative data study, measures pro-
duce data in the form of numbers. You go from an
abstract idea to a data collection technique that will
yield precise numerical information. In a qualitative
data study, data might be in the form of numbers, but
more often, they are written or spoken words, actions,
sounds, symbols, physical objects, or visual images
(e.g., maps, photographs, videos, etc.). Instead of con-
verting all observations into a single medium, num-
bers, you leave the data in many diverse shapes, sizes,
and forms. They stay as words, images, quotes, and
descriptions instead of all becoming numbers.
4. Linkages. In all research, you link ideas to observable
empirical data. In a quantitative data study, you reflect
on and refine ideas and then create specific ­measurement
Measuring Social Life 95
techniques (such as a questionnaire) to capture the
ideas. Logic links ideas to the measures used to gather
empirical evidence. You may begin with a very abstract
idea (“quality of life”), link it to a less abstract idea
(“being socially engaged”), then link it to a specific mea-
surement act (answers to a survey question, “How often
do you see friends socially?”). In a qualitative data
study, you try to make sense of data by actively creating
new, or adapting existing concepts. You simultaneously
gather data and link data to ideas that clarify the data’s
meaning. You may first develop an idea (such as “din-
ing in isolation”) based on gathering the data (observe
many people eating silently alone at a nursing home).
The idea, once developed, may influence your subse-
quent observations.
5.3.1:  Two Parts of the
Measurement Process
Measurement builds on two processes: conceptualization
and operationalization. Often our ideas are fuzzy around
the edges and not fully developed within; we conceptual-
ize to sharpen and clarify ideas so that other people can
understand them better. During the process of converting
mental images or ideas into words, we develop a concep-
tual definition. This expresses what we mean by the idea.
To create a definition, we think carefully, are observant,
consult with others, read what others have written about
the idea or related ones, and try out several possible defini-
tions, making many readjustments to improve clarity. A
definition should identify the boundaries of the idea and
its core meaning.
Let us say you want to develop a conceptual definition
of discrimination. Perhaps you think it means a “negative
action.” To begin the conceptualization process, consider
your personal experiences, your inner thoughts about it,
discussions you have had with other people, and what you
read about it in the literature. Reflect on what you know,
ask others what they think, and look up multiple defini-
tions. As you ponder over the meaning of discrimination,
the core of the idea should become clearer. Eventually you
may collect several alternative potential definitions and
will need to sort through them. You settle on:
“Discrimination is the act of treating people unequally
simply because they belong to a social category or are the
member of a group.”
You conclude that discrimination involves ways of
behaving toward a type of people, and it refers to “others”
or to an outgroup (a group to which a person does not
belong). As your thinking expands, you may consider vari-
ous types of discrimination based on various social
­categories or groups—racial, religious, age, gender, lin-
guistic, national origin, sexual orientation, and so forth.
96  Chapter 5
As you conceptualize, consider the unit of analysis
and develop a measure for a unit of analysis that best fits
the conceptual definition. For example, you think discrimi-
nation is an action of individuals, and this makes the indi-
vidual the unit of analysis. However, organizations or
institutions (e.g., families, clubs, churches, companies,
schools, or media outlets) may also treat people in out-
groups unequally (e.g., not hiring an outgroup member). If
this is your focus, the group or organization becomes the
unit of analysis. To conceptualize and develop a measure,
you decide on a unit of analysis. Do you want to look at
discrimination only as individual actions or as actions by
groups, organizations, and institutions?
During the conceptualization process, it is important
to distinguish the idea, or concept, from closely related
ones. Ideas overlap with others and blur into one another.
Good measurement requires separating the concept you
want to study from others that are closely related. For
example:
How are prejudice, racism, and stereotypes similar to or differ-
ent from discrimination?
5.3.2:  Conceptualization and
Operationalization
Conceptualization requires thinking carefully about the
concept. You might define discrimination as “a negative
action of unequal treatment by a person directed toward
members of an outgroup that relies on stereotypes.” This is
more precise than your initial idea, “negative action.” It
now has links to other ideas, such as outgroup and stereo-
type. Repeatedly reevaluate each part of the definition.
Consider the term “negative action.” Can a positive action
be a kind of discrimination? Can there be positive discrimi-
nation, or unequal treatment in favor of a group based on
stereotypes? Conceptualization is a process that requires
systematical thinking, stating ideas clearly, and using pre-
cise terms.
Operationalization links a conceptual definition to a
specific set of measurement procedures, or its operational
definition. An operational definition restates the concep-
tual definition as measures. It might be one or more survey
questions, a way to observe events in a field setting, or
counting the appearance of symbols in a video. It is a spe-
cific activity in which we observe, document, or represent a
conceptual definition.
It is not always necessary to invent a new measure of a
concept if existing ones capture the idea adequately. Many
measures of racial discrimination are already in use. For
example, Gee et al. (2007) wanted to see whether Asian
Americans who experienced more everyday discrimina-
tion suffered worse health. Their operational definition of
“everyday discrimination” was how people answered nine
survey questions about routine unfair treatment:
1. You are treated with less courtesy than other people.
2. You are treated with less respect than other people.
3. You receive poorer service than other people at restau-
rants or stores.
4. People act as if they think you are not smart.
5. People act as if they are afraid of you.
6. People act as if they think you are dishonest.
7. People act as if you are not as good as they are.
8. You are called names or insulted.
9. You are threatened or harassed.
If a person’s answers indicated that many of these
actions occurred to him or her on a regular basis, the
researchers considered it as empirical evidence of routine
discrimination. Social scientists have created measures of
many widely used ideas, such as social distance.
Learning from History
Measuring Social Distance
The famous sociologist Emory Borgadus (1882–1973) wrote
275 books and articles. Twenty-seven dealt with the idea of
social distance. In a 1925 article (“Social Distance and Its
­Origins,” Journal of Applied Sociology 9:216–226), he outlined
the concept of social distance. He saw social distance as a
force that influenced most social relationships and indicated
the degree of social-emotional closeness and trust in others.
He tried to capture how close or distant people felt toward
members of different racial and ethnic categories. Researchers
still use variations of it today. To measure social distance,
­Borgadus asked people whether they were willing to interact
and establish specific social relationships with members of
racial and ethnic groups other than their own. He asked peo-
ple how they would feel to have a member of group X
a. in close kinship marriage
b. in my club as a personal chum
c. in my street as a neighbor
d. as a fellow employee in my occupation
e. as a fellow citizen in my country
f. as a visitor only in my country
g. I would exclude all members of X from my country.
In the original list, X stood for 30 racial-nationality groups.
After people rated how they felt toward each group, Borgadus
developed a picture of social distance among groups. The
results showed that the dominant white majority felt very socially
distant from some groups and socially close to others. Borgadus
created an average distance score for each of the 30 groups. In
subsequent years, researchers replicated the study and found
slight changes in how distant social groups feel from one another
(see Kleg and Yamamoto, 1998; Parrillo & Donoghue, 2013).

Tips for the Wise Consumer
Creating a Good Measure
1. Clarify the conceptual definition. Precisely match the
measure to a specific conceptual definition of the idea.
Without a clear conceptual definition, a good measure is
impossible.
2. Keep an open mind. Do not get locked into a single mea-
sure or type of measure; instead, be creative and con-
stantly look for better measures.
3. Borrow from others. Do not be afraid to borrow from other
researchers, as long as you give them credit. Other studies
often have very useful measures or ones you can modify.
4. Anticipate difficulties. Logical and practical problems
often arise when you try to measure. With careful fore-
thought and planning, you can anticipate and avoid some
problems, but often you must adjust and modify based on
experiences with after trying out measure.
5. Remember the unit of analysis. Tailor a measure to the
units of analysis of a study. You will want a measure that
generalizes to the universe of interest.
WRITING PROMPT
Social Distance
Consider the social distance measure that Emory Borgadus created
in the 1920s. Come up with another way to measure the amount of
social distance between different social groups in today’s society.
The response entered here will appear in the
performance dashboard and can be viewed by
your instructor.
Submit
5.3.3:  Quantitative Conceptualization
and Operationalization
Quantitative measuring is a three-part process that con-
nects three levels of reality, from the most abstract to the
most concrete: conceptual, operational, and empirical.
1. Conceptualization: think through the idea and create a
clear conceptual definition of it.
2. Operationalization: link the conceptual definition to its
operational definition, i.e., specific measurement pro-
cedures.
3. Measurement: apply the operational definition to col-
lect the empirical data.
At the most abstract level, you can specify the relation-
ship between at least two concepts to form a conceptual
hypothesis. After you develop a conceptual definition of
each variable and conceptual hypothesis, you create
Measuring Social Life 97
­ easures or operationalize the variables. Using the opera-
m
tional definitions of variables, you can create an empirical
hypothesis. It is a restatement of the conceptual hypothe-
sis in operational terms.
Lastly, at the level of the visible, concrete empirical
world, you use operational definitions of variables to gather
data. Often called indicators, these measures indicate the
presence and degree of variables. You can use statistics such
as correlations to determine the degree of empirical associa-
tion among the indicators. Findings of empirical associa-
tions among indicators test the empirical hypothesis that
connects logically back to the conceptual ­hypothesis.
A Study on Quantitative Measuring  To see
these three parts together, let us consider the explanatory
study by Sherkat, de Vries, and Creek (2010). They exam-
ined why African Americans oppose same-sex marriage
more than other groups. The researchers began with four
concepts in addition to a person’s self-described race:
1. Support for or opposition to couple of the same sex
marrying one another.
2. Religious factors, such as how religiously active a person
is and the views on same-sex marriage of the religious
sect or denomination to which a person is attached.
3. A person’s general political beliefs that may influence
views on same-sex marriage.
4. Age and social-demographic factors (e.g., income,
education, marital status) that may influence same-sex
marriage views.
In this study, the unit of analysis is the individual. The
researchers conceptualized the primary dependent variable,
individual acceptance of same-sex marriage, with a broad
set of views about sexuality and LGBT rights. They hypoth-
esized that African Americans oppose same-sex marriage
more than members of other racial groups due to religious
factors among African Americans. Based on past studies of
religion in America, they considered the strength of a per-
son’s religious attachments and beliefs of the religious sect
or denomination to which a person belongs.
The operational definition of the dependent variable was
a single ordinal-level question in a survey, “Homosexual
couples should have the right to marry one another,” with
five possible answers ranging from strongly agree to neu-
tral to strongly disagree. The researchers used two survey
questions to operationalize the independent variable. One
was frequency of attending religious services. It was a sur-
vey question that had eight response categories from never
to several times a week. The other measure was religious
affiliation and included no religion, Roman Catholic, main-
stream Protestant or other, or conservative Protestant sects
(includes Baptists, Churches of Christ, Assembly of God,
and others). The set of conservative Protestant sects had
been identified in several past studies.
98  Chapter 5
Figure 5.1  Conceptualization and Operationalization: Abstract Construct to Concrete Measure
This illustrates how the measurement process links the three levels (abstract theoretical, operational,
and concrete empirical). It shows the measurement process for two variables linked together with a
­hypothesis. You need to consider all three levels together.
Abstract Construct to Concrete Measure
Independent Variable
Hypothetical Causal
Abstract Construct
Relationship
Conceptualization
Conceptual Definition
Operationalization
Tested Empirical
Indicator or Measure
Hypothesis
Besides the two main variables, the researchers con-
sidered other factors that often shape views on same-sex
issues (e.g., liberal vs. conservative political views, educa-
tion level, and age) and had survey questions related to
each. They wanted to learn whether the religious factor
was a more powerful, separate force from the nonreligious
factors (e.g., political views, age, income) in predicting
African American views regarding same-sex marriage.
In a study with quantitative data, you move from the
abstract concept toward a concrete measure. Begin by con-
ceptualizing, i.e., create a clear conceptual definition for
each variable. Next, operationalize by giving each an opera-
tional definition. Next, gather empirical data following the
measurement operations described in the operational defi-
nition. Lastly, test the hypothesis empirically with the data.
The empirical tests are logically connected back to the con-
ceptual level. In this way, empirical tests provide evidence to
support or refute a conceptual hypothesis (see Figure 5.1).
5.3.4:  Qualitative Conceptualization
and Operationalization
In research using quantitative data, you give abstract ideas
a conceptual definition early in the research process. By
contrast, in qualitative data research you may begin by
using partly developed, rudimentary working ideas dur-
ing the data collection process. As you gather and analyze
qualitative data (i.e., field notes, photos and maps, histori-
cal documents, etc.), you rethink and refine your initial
ideas and develop new ones based on your data. As you
try to “make sense” of the data, you adjust or create clearer
definitions of the ideas. It is an iterative, back-and-forth,
process of gathering data, refining ideas, gathering more
data, then again refining ideas. Eventually, you connect
the ideas together into theoretical relationships. Often this
is grounded theory. The process requires you to be
Dependent Variable
Abstract Construct
Conceptualization Theoretical
Level
Conceptual Definition
Operational
Operationalization Level
Empirical
Indicator or Measure Level
s­ elf-aware and reflect continuously while you are in the
middle of doing research. You must simultaneously docu-
ment both the data and the process of how you gathered
the data, including emerging ideas about the data.
During conceptualization, you “interrogate” or ask
theoretically related questions about the data:
Is this a case of idea Z? What is the sequence of events,
and could it be different? Why did this happen here and
not somewhere else?
As in quantitative research, you conceptualize by
developing clear, explicit definitions. The definitions are
more abstract than your direct observations but are tied to
specific data. In effect, you anchor the conceptual defini-
tions in the specific words, events, or actions that are the
data. Because qualitative measurement is integrated with
other parts of a study and not a separate step, it may be
more difficult to carry out.
Although the above discussion emphasized quantita-
tive–qualitative differences, in practice, neither type of
research adheres to a rigid process. For both, ideas and
data mutually influence one another. For both, we draw on
ideas from beyond a specific research setting and blend
past techniques and concepts with new ones that emerge
during data collection. In this way, ideas and evidence
become mutually interdependent. Remember, all opera-
tionalization is a process of connecting ideas with data.
Example Study
Operationalizing Social Ties
Matthew Desmond (2012a, 2012b) studied evictions in
urban neighborhoods. Desmond noted that past studies
showed how the poor often rely on kinship networks to
 Measuring Social Life 99

teenage children on Myesha’s welfare check and odd jobs

survive. He c ­ onducted a year-long ethnographic study of
two low-income Milwaukee neighborhoods, a majority-
white trailer park and a majority-black inner-city neighbor-
hood. He purposely studied a majority-white and a
majority-black improvised neighborhood to see whether
the survival strategies used by of black inner-city residents
differed those of from whites in similar economic situations.
He followed low-income tenants who had been evicted
from their homes and explored how they used social net-
works during their time of crisis. Unlike research using
quantitative data where operationalization links ideas to a
preplanned technique of measuring before data gathering,
in a qualitative data study like that by Desmond, operation-
alization occurs while gathering data, i.e., while observing/
describing. In almost a reverse process from quantitative
data, operationalization takes place as researchers dis-
cover or create a concept that is useful in their observa-
tions and data gathering.
that Chester picks up.
As he explored barriers faced by residents of the two
neighborhoods when seeking aid from kin, he saw how they
developed and used short-term social relations, what he
describes as “disposable ties.”
Every evicted person relied on others; however, some-
times family members refused to provide aid due to past
conflicts or a lack of resources. Instead, Desmond observed
that people formed social ties with new acquaintances
based on spending a lot of time together and having recip-
rocal exchanges. People created the short-social connec-
tions and later dropped them. He noted that everyone uses
disposable ties. Middle-income people rely on them for
social or job advancement, sexual relations, financial trans-
actions, or specific services. However, the poor also use
them for basic human needs, such as securing food and
shelter. Desmond described how he saw many people in
desperate situations (e.g., being left out on the street) create
a disposable tie, such as asking a favor (e.g., a warm place
to sleep for a night) of someone they barely knew. He out-
lined how people created, used, and “burned” ties. Despite
some minor differences between poor blacks and poor
whites during evictions and the use of disposable ties,
including blatant racial discrimination against blacks, he
found many more similarities between the racial groups.
Desmond concluded that disposable ties are an alternative
survival option to isolated individualism and reliance on kin
networks for the urban poor.

Read More
Summary Review
In the Green Street Mobile Home Park and in a north side
impoverished neighborhood in Milwaukee, Desmond estab-
lished relationships with several families—11 of these families Table 5.1  Steps in Quantitative and Qualitative
went through evictions. Past ethnographic studies of low- Conceptualization and Operationalization
income neighborhoods showed that poor people relied on kin
Quantitative Qualitative
and nonkin social networks. Because many poor people are
1. Conceptualize variables by 1. G
 ather empirical data and
loners with few kin or have kin without resources, Desmond
developing a clear, complete simultaneously think about
wondered how low-income people used such networks in a written conceptual definition concepts to organize and
time of crisis, such as eviction. for the core idea of each. make sense of the data.
Build on past theories, Develop clear definitions for
Desmond operationalized major concepts of his study by
­consider the definitions others each concept by drawing on
describing specific residents and their experiences. He have used, and be clear past readings, newly created
described what he saw and offered quotes of what he heard, and logical. ideas, or from the ideas
used by the people you are
grounding his concepts in specific events and individuals. In
studying.
the trailer park, he introduced us to several people.
2. Operationalize variables by 2. As you gather data, be very
They include Teddy, a 52-year-old half-paralyzed man creating specific activities to aware of processes you use
who received disability payments; Pam and Ned, 32 and 42, measure each. This opera- to make sense of the data
respectfully, a white couple who raised five children working tional definition closely and interpret details of the
matches how you have data. Reflect on and describe
odd jobs for cash; and Tina, a 40-year-old white single mother defined the variable in its this process of linking ideas
of three with a seasonal phone answering job and who relies ­conceptual definition. to specific observations in
on unemployment for half the year. In the inner city black the data.

neighborhood, we learned of Arleen, a 38-year-old black 3. Gather empirical data using 3. Review and refine your defini-
the specific measurement tions and the descriptions of
women and mother of six who receives welfare. We also activities of your operational how you gathered data and
learned about Lamar, a 48-year-old wheelchair-bound black definition; this links data to made sense of it.
man and single father of two who receives welfare, and Ches- the conceptual ­definition.
ter and Myesha, a black couple, both 33, who support two
100  Chapter 5
WRITING PROMPT
Levels of Variables
Logic is what connects the three levels (abstract theoretical, opera-
tional, and concrete empirical) we use to measure a variable. Select
a social variable that most interests you (and was not given as an
example here) and create each of the three levels for it. Was it easier
to start with the concrete empirical level and work to the abstract, or
vice versa? Why?
The response entered here will appear in the
performance dashboard and can be viewed by
your instructor.
Submit
5.4:  How Can We Create
Good Measures?
5.4 Apply reliability and measurement validity
to create good measures
Thus far, we reviewed how the measurement process
works. Now we want to make certain that our measures
are strong or good ones. The terms reliability and validity
have multiple meanings. Here, they refer to desirable
aspects of scientific measurement. They describe the
strength of linking unobservable ideas “in your head” to
specific measurements in the visible world. We can never
achieve perfect reliability and validity. Rather, they are ide-
als toward which we strive. Measures that are highly reli-
able and valid are more credible or believable.
We can easily see measurement reliability in how we
measure the physical world. You step onto a bathroom
scale and measure your weight. Now you get off and get
back on repeatedly. A reliable scale gives the same weight
each time—assuming, of course, that you are not eating,
drinking, changing clothing, and so forth. An unreliable
scale registers different weights over time, even when your
“true weight” is unchanged. The analogy of measuring
weight on a bathroom scale carries over to other physical
world measures (e.g., speedometer for speed, thermometer
for temperature) and to measures of the social world (e.g.,
survey questions for attitudes). Reliability says that the
measurement method or instrument itself remains consis-
tent and dependable.
Specific measures are called indicators, because they are
empirical data that indicate (i.e., points to) an idea. A mea-
sure’s reliability increases when we have more than one
indicator or specific measure. For example, if you wanted to
measure discrimination, you could ask one survey question:
“Are you treated with less courtesy than other people?” It
would be a single indicator. If you asked nine related survey
questions, as in the study by Gee et al. (2007) previously
mentioned, you would have multiple indicators. There is
less consistency in detecting discrimination experiences
with one rather than several questions. The discrimination
measure would be even stronger with different types of
indicators. In addition to several survey questions, you
could observe behavior. With multiple different indicators,
the chances of accidental fluctuations or inconsistency are
less likely than relying on just one indicator.
Validity suggests truthfulness. It tells us how well a
mental picture, or idea about an aspect of the world,
matches measures of that picture in empirical reality. In
other words, how well the measure aligns with the ideas
we use to understand a specific aspect of reality.
Validity is an overused term that often means “true” or
“correct.” There are several kinds of validity. Here, we are
concerned with measurement validity or the fit between
conceptual and operational definitions—the better the fit,
the greater the validity. A measure can be valid for one pur-
pose (i.e., a research question with units of analysis and
universe) but invalid for others. For example, a measure of
job satisfaction might be valid for measuring job satisfac-
tion among retail clerks in a clothing store but invalid for
measuring it among police officers.
How can you improve reliability?
Compare Your Thoughts
You can improve a measure’s reliability in four ways:
1. Clearly conceptualize. Sloppy, loose, or fuzzy thinking weak-
ens reliability. Such thinking allows issues disconnected from the
core idea to enter a measure and allows omitting important
aspects. Reliability improves with sharp, unambiguous definitions
of concepts. When you define a concept to eliminate “noise” (i.e.,
interfering information) that can spill over from other concepts and
introduce instability, reliability improves. Each measure should
indicate one and only one central idea; otherwise, it is impossible
to determine exactly what concept your measure is indicating.
2. Increase the level of measurement. We will discuss levels of
measurement later in this chapter. At this point, the main idea is
that a higher or more precise level of measurement is more
­reliable than a less precise one. This is because the imprecise

measures pick up less detailed information. If you do not mea-
sure specific information, things other than the concept of inter-
est might slip in. The general principle is to try to measure at the
most precise level possible.
3. Use multiple indicators. You can improve reliability by using
multiple indicators because two (or more) indicators of the same
concept are better than one. Using multiple measures is a widely
accepted principle of good measurement. For example, I create
three indicators of employee satisfaction. My first indicator is an
attitude question on a survey. I ask research participants their
beliefs and feelings about different job areas. For a second indi-
cator, I observe research participants at work. I note whether
they smile, appear to be happy, maintain good relationships with
coworkers and customers, or appear stressed, complain fre-
quently, and act gruff. Lastly, I examine employment records for
absenteeism, disciplinary actions, length of service, and turn-
over rates. By using the three separate indicators—survey,
observation, and written records—I learn about satisfaction. If all
three show consistent satisfaction (high or low), I gain confi-
dence that I have a dependable measure. Multiple indicators let
you measure different aspects of the concept (such as employee
satisfaction with pay, with workplace conditions, with supervi-
sion) each with its own indicator. In addition, one indicator may
be imperfect or unstable, but several indicators are unlikely to
have the same (systematic) error.
4. Use pilot studies and replication. Trying pilot versions of a
measure can improve its reliability. Develop one or more draft or
preliminary versions of a measure and try them out before using
the final version. Of course, this takes more time and effort. You
can also replicate the measures other researchers have used. If
you find measures from past research, you can build on and use
them, citing the source, of course.
Validity is more difficult to achieve than reliability. We
never achieve absolute validity because it links invisible,
abstract ideas to specific empirical observations. A gap
always exists between our mental images about the world
and the concrete reality we experience. Nonetheless, some
measures are more valid than others are.
Measurement validity and sampling error are similar.
In sampling, you want minimal sampling error—i.e., you
want a specific sample, for which you have data, to repre-
sent the population that you cannot directly observe. A
sample with a small sampling error provides more
Figure 5.2  Illustration of Relationship between Reliability and Validity
Source: Adapted from Babbie (2004:145).
A Bull’s-Eye = A Perfect Measure
Low Reliability High Reliability
and Low Validity but Low Validity
Measuring Social Life 101
a­ ccurate estimates of the population parameter than a
sample with a large sampling error. In a parallel manner,
for the measurement process you want specific empirical
measures that represent abstract concepts that you cannot
directly observe. If you have a valid measure, it deviates
very little from the concept it represents.
5.4.1:  Relationship of Reliability
and Validity
We want both reliability and validity in measures. Reliabil-
ity is necessary for validity and is easier to achieve than
validity. Although reliability is necessary for a valid mea-
sure, it does not guarantee that a measure will be valid. It is
not a sufficient condition for validity. A measure can pro-
duce the same result over and over, but what it measures
may not match the definition of the construct (i.e., validity).
A reliable measure can be invalid.
Here is a simple example: You step onto a scale to get
weighed. The weight registered by the scale is the same each
time you get on and off. The scale is a reliable measure. You
go to another scale—an “official” one that measures true
weight—and it says that your weight is much heavier. The
first scale was reliable (i.e., dependable and consistent), but it
did not give a valid measure of weight. A diagram shows the
relationship between reliability and validity using the anal-
ogy of a target. The bull’s-eye represents a good fit between a
measure and the definition of the construct (see Figure 5.2).
WRITING PROMPT
Reliable Measures
Select one aspect from your daily life that is measured and you
check occasionally. It can be part of physical or social world. How
reliable is this measure? How would you know if it stopped being a
reliable measure? How would using multiple indicators help you gain
confidence in the reliability of your measure?
The response entered here will appear in the
performance dashboard and can be viewed by
your instructor.
Submit
High Reliability
and High Validity
102  Chapter 5
5.4.2:  Three Types of Measurement
Validity
Face validity  This is easiest to achieve and the most
basic kind of validity. It is a judgment by knowledgeable
people of how well an indicator measures the concept it
purports to measure. It addresses the question: On the face
of it, do more informed people believe that the conceptual
definition and measurement fit tightly together than who
believe that they do not fit? For example, no one would
accept a measure of college student math ability using a
question that asked students 2 + 2 = ?. This is not valid on
the face of it.
Content validity  Content validity is a special
type of face validity. The conceptual definition of a
sophisticated idea is a type of mental “space” that con-
tains areas with secondary or reinforcing ideas. Mea-
sures that capture all the areas of the conceptual space
have high content validity. For example, your concept of
discrimination has three aspects (e.g., interpersonal,
housing, employment). A measure that captures all three
aspects has content validity.
Criterion validity  Criterion validity uses a stan-
dard or criterion to indicate a concept. An indicator ’s
validity depends on comparing it with another mea-
sure of the same construct in which you have strong
confidence.
Summary Review
Table 5.2  Measurement Validity Types
Validity Type Features
Face There is widespread agreement among
informed people that the measure is valid
Content Measure captures the entire meaning of all
parts of a concept’s definition
Criterion Measure agrees with highly trusted other
measures of the same concept
Example Study
Content Validity and the
Contact Hypothesis
Scholars have examined “contact hypothesis” for 50 years,
and it is central to our understanding intergroup relations. The
hypothesis says, “contact, particularly close and sustained
contact, with members of other cultural groups provides
direct information about the values, lifestyles, and experi-
ences of members of those groups.” Information obtained in
this way is likely to be more favorable and accurate than infor-
mation gained through other, less direct sources. 2 Without
direct contact, people often rely on socialization, peers, and
the media—all that tend to be sources of stereotypes. Such
indirect information and images tend to be less accurate and
less favorable toward outgroups that are minorities. Ellison,
Shin, and Leal (2011) explored the contact hypothesis for
relations between non-Latinos and the Latino population of
the United States. They conceptualized the independent vari-
able, contact, to include non-Latinos having personal contact
with a Latino in four areas—in high school, as co-workers, as
friends or acquaintances, or as relatives. Their dependent
variable on views about Latinos had four dimensions: accep-
tance of negative stereotypes (e.g., lazy, unintelligent, lack a
strong family), degree of awareness and respect for Latino
achievements, high social distance measured by the Borga-
dus Social Distance Scale, and views on immigration. Thus,
they conceptualized both the independent and dependent
variables as having content in multiple areas. They used one
or more survey questions to ask about each content area of
both variables. This enabled them to measure both the
degree of contact and favorable–unfavorable views. They
also measured whether the general relationship held for all
content areas in the independent and dependent variables.
The researchers also controlled for several alternative expla-
nations, such as living in an area with many Latinos. They
found having personal contact with Latinos, especially as a
friend or relative, had a positive effect on all areas of the
dependent variable, including immigration views. Living in an
area with many Latinos by itself did not increase favorable
views; having personal contact was essential. Another finding
that was not part of the main hypothesis was that among
non-Latinos, African Americans had more positive views
toward Latinos than the white Anglos. Although the contact
hypothesis was developed in the United States to examine
racial intergroup interactions, it has been tested across a
2
Ellison, Shin, and Leal (2011 ) pp. 938–39
 Measuring Social Life 103

wide range of settings and for many issues, from defusing the
Israeli–Arab conflict, to “straight” Dutch teens attitudes about
gays and lesbians, to South Korean college students’ views
regarding and willingness to interact with foreign students in
their country.3
Validity links a concept to empirical measures. To be
valid, qualitative data measures should have authenticity.
The goal is less to match an abstract concept with a single,
fixed, standard version of reality, than to match concepts
with the understandings of the people being studied in a
manner that “rings true” to their life experiences.

5.5:  What Are the 5.6:  What Are the

Principles of Qualitative Principles of Quantitative
Measurement? Measurement?
5.6 List the characteristics of the four levels of
5.5 Describe how principles of reliability and validity
measurement as they relate to quantitative
operate for qualitative data
measurement
The principles of reliability and validity operate somewhat
Thus far, you have learned about basic measurement prin-
differently for qualitative data. Reliable qualitative data
ciples, including reliability and validity, and qualitative
means we have collected the data consistently. Although
measurement. All quantitative measures use levels of mea-
qualitative data techniques emphasize being flexible and
surement. These vary from a refined level with great preci-
adaptive to changing conditions, we do not want to be vac-
sion to rougher or less precise levels. The level of
illating and erratic. In qualitative studies, we frequently
measurement depends on your assumptions about charac-
interact with and develop deep social relations with people.
teristics of the concept you are measuring. The way in
The relationships can be a growing, evolving process. Thus,
which you conceptualize a variable limits the levels of
how we gather data, observe, and interact with others may
measurement you can use.
evolve over time, but this does not mean being sloppy,
You can begin by thinking of all variables as being
erratic, or inconsistent. To reliably measure concepts in
either continuous or discrete. A continuous variable has
qualitative data, measurement must be thoughtful, consis-
an infinite number of values that flow along a continuum.
tent, and dependable. For example, observing a field setting
It is possible to subdivide the variable into smaller and
on the first day may differ from how you observe after
smaller increments; the number of increments can be infi-
being in the field site for six weeks. This difference does not
nite. Such variables include temperature, age, income,
mean unreliable or inconsistent observing; rather, you care-
crime rate, and amount of schooling. A discrete variable
fully and consistently self-monitor the process of observing
has a fixed set of separate values or categories. Instead of a
over time.
smooth continuum of many values, discrete variables
In a qualitative data study, we consider a wide range
have two or a few separate, distinct categories. Examples
of data sources and measure in many diverse ways. The
include gender (male or female), religion (Protestant,
interactive and situational nature of qualitative data
Catholic, Jew, Muslim, atheist), and marital status (never
means that two researchers may not always get identical
married single, married, divorced or separated, ­widowed).
results. One researcher may use a mix of measures that
Whether a variable is continuous or discrete affects its
another researcher does not use. The diverse measures
level of measurement.
and differences among researchers can illuminate various
The idea of levels of measurement expands upon the
facets or dimensions of an issue or setting. Nonetheless,
continuous versus discrete variable distinction and orga-
different measures or specific researchers can yield reli-
nizes measurement precision. You may not find it easy to
able measurement. It is not reliable in the same way as
decide on the appropriate level for a concept at first. The
with quantitative data, i.e., creating a single, fixed, stan-
appropriate level of measurement depends on inherent
dard, and unchanging measure. Rather, it is reliable by
features of the variable and how you conceptualized it.
observing and measuring in a consistent and self-con-
Some concepts and variables can only be discrete. In the
scious way that has been adjusted to fit a specific research
social world, marital status has just a few basic categories.
situation.
Marital status is either married or not married, and not mar-
ried is subdivided into never married, divorced, separated,
3
For diverse recent examples of tests of the contact hypothesis, see Collier,
or widowed. Other concepts and variables can be either con-
et. al. (2012), Jon (2013), Pickett, et al. (2014), and Van Laar, et. al. (2005). tinuous or discrete. Age can be continuous, i­ndicating how
104  Chapter 5

old a person is in years, months, days, hours, and minutes. arbitrary zeros, not true zeros. This can create confusion
Age can also be a set of discrete categories, such as infancy, and lead to mistakes. For example, a rise in temperature
childhood, adolescence, young adulthood, middle age, and from 30° to 60° is not a doubling of the temperature. The
old age. Education can be continuous, indicating years of zero in measuring temperature is arbitrary because 0° is
schooling, or a set of discrete categories by degree/diploma, not the absence of all heat. With an arbitrary zero, the
such as less than high school, high school diploma, more numbers on a thermometer might double, but the actual
than high school but less than four-year college degree, four- temperature does not double.
year college degree, and graduate degree.
Practical Reasons to Conceptualize and
While it is possible to collapse most continuous vari-
Measure Variables at Higher Levels of Mea-
ables into a few discrete categories, you cannot do the
surement  You can always collapse a high level of
opposite—turn a discrete variable into a continuous mea-
measurement to a low level, but the reverse is not true. If
sure. For example, sex, religion, and marital status cannot
you measure a concept very precisely, you can decide
be conceptualized as continuous. Nevertheless, you can
later to “throw away” or ignore some of the precision. But
shift to related concepts that you conceptualize as continu-
you cannot measure a concept with little precision and
ous. Sex is discrete, but the “degree of femininity” and
then make it more precise later. You can turn a ratio-level
“degree of masculinity” are continuous variables. A spe-
measure (annual family income) into ordinal level (High,
cific religion (e.g., Catholic, Lutheran, or Baptist) is dis-
Medium, or Low income) or nominal level (same as or
crete, but the degree of commitment to religion is
different from others). However, the process does not
continuous. Marital status is discrete, but the number of
work in the opposite way. You cannot measure something
years a person has been married is continuous.
at the nominal level and then reorganize it to be ordinal,
The four levels of measurement, from lowest (discrete
interval, or ratio.
and least precise) to highest precision, are nominal, ordi-
nal, interval, and ratio. Each level provides different types
of information. Discrete variables are at the nominal and
ordinal levels, whereas you can measure continuous vari-
Summary Review
ables at the interval or ratio levels.

What are the levels of measurement? Table 5.3  Characteristics of the Four Levels
of Measurement
Compare Your Thoughts
Distance between
Nominal measures only indicate a difference among cate- Different Categories True
gories. Examples include gender: male or female; religion: Level Categories Ranked Measured Zero
Protestant, Catholic, Jew, Muslim, or other; racial heritage: Nominal Yes No No No
African, Asian, Caucasian, Latino, other; state/province or Ordinal Yes Yes No No
region of residence: Illinois, Ontario, New York, Texas, or Interval Yes Yes Yes No
Midwest, Northeast, and so on. Ordinal measures indicate Ratio Yes Yes Yes Yes
a difference among categories, and the categories can be
ordered or ranked. Examples include letter grades: A, B, C,
D, E; opinion measures: Strongly Agree, Agree, Disagree,
Strongly Disagree; quality ratings: Excellent, Very Good, When you use an ordinal measure, try to have at least
Good, Fair, Poor. Interval measures do everything the first five ordinal categories and obtain multiple observations
two do, plus they specify the distance between categories. for each. Interval measures can be confusing because
Examples include Fahrenheit or Celsius temperature: 5°, some measures, like temperature, use arbitrary zeros.
45°, 90°; and IQ scores: 95, 110, 125. Ratio measures do Arbitrary zeros in interval measures confuse many peo-
everything all the other levels do, plus they have a “true ple. The zeros are “arbitrary” or just there to keep score.
zero.” A true zero means a score of zero indicates a value They do not indicate a real zero. The temperature can be
of zero or nothing. Having a true zero makes it possible to zero, or below zero, but zero is an arbitrary number. Com-
talk about relationships in terms of proportion or ratios, pare 0°C with 0°F—they are different temperatures.
such as twice as much. Examples include money income: Because the zeros are arbitrary, doubling the degrees in
$10, $100, $500; years of formal schooling: 1 year, 10 one system does not double the degrees in the other. It
years, 13 years; age: 18 years, 32 years, 64 years. In most makes no sense to say that it is “twice as warm” if the
­situations, the distinction between interval and ratio levels temperature rises from 2° to 4°, from 15° to 30°, or from
makes little difference. Some interval-level variables have 40° to 80° (see Figure 5.3).
 Measuring Social Life 105

Figure 5.3  Examples of Levels of Measurement

Nominal Level

Protestant Catholic Muslim Jewish

Religious Faith

The categories differ, but we do not assume that one is “better/high” or “worse/lower” than others.

Ordinal Level

 
Very Happy Happy Neither Sad Very Sad

Degree of Happiness

These categories differ, plus we can order or rank them.

Interval Level

60 70 80 90 100 110 120 130 140

IQ Score

We can precisely determine the size of differences among score, but there is no absolute zero.

Ratio Level

2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50 52 54 56 58 60 62 64 66 68 70 72 74 76 78
Age
We can precisely determine the size of differences among scores, plus a true or absolute zero exists.

WRITING PROMPT 5.7:  How Do We Construct

Levels of Measurement

List four examples of nominal-level social variables not mentioned in
the discussion. Explain why each of your nominal-level example vari-
ables cannot be ranked as better/worse higher/lower.
Now list four examples of ordinal-level social variables. Identify
five categories from high/best to low/worse for each of them, where
the categories are about equal distance from one another.
The response entered here will appear in the
performance dashboard and can be viewed by
your instructor.
Submit
and Use Indexes
and Scales?
5.7 Apply the processes involved in the construction
and use of indexes and scales
Scales and indexes are information-filled quantitative data
measures. Researchers have created hundreds of indexes
and scales to measure the prestige of occupations, the
adjustment of people to a marriage, the intensity of group
interaction, the level of social activity in a community, the
106  Chapter 5
degree to which a state’s sexual assault laws reflect feminist
values, the level of socioeconomic development of a nation,
and much more. You can borrow already created scales/
indexes or create your own. In this section, we look at the
principles of scale and index construction and explore a few
major types. Before we begin, keep two things in mind:
1. We can develop a measure of every social phenome-
non. We can measure some concepts directly and pro-
duce precise numerical values (e.g., family income),
whereas for other concepts we use proxies to measure
a variable indirectly and it may not be as precise (e.g.,
predisposition to commit a crime).
2. You can learn a great deal by looking at measures cre-
ated by other researchers. You do not have to start
from scratch but can use a previously used scale or
index, or you can modify it.
Social researchers do not use a consistent nomencla-
ture to distinguish between index and scale. One person’s
scale can be another’s index. Both produce ordinal- or
interval-level measures. Also, you can combine scale and
index techniques to create a single measure. Compared to
basic single indicator measures, scales and indexes provide
more information, increase measurement reliability and
validity, and organize and condense complex information.
For most purposes, you can treat scales and indexes as
interchangeable, although they have distinct features.
A scale creates an ordinal measure of intensity, direc-
tion, level, or potency by arranging responses or observa-
tions along a continuum. It can be a single indicator or
multiple indicators. An index combines information from
multiple separate indicators into a one score, often a sum
of their values. Most indexes add the numerical values of
several items to yield a score at the interval level of mea-
surement. You can combine several indicators measured
with scales into a single composite index measure.
5.7.1:  Mutually Exclusive,
Exhaustive, and Unidimensional
Two features of all good measurement are that variable cate-
gories are mutually exclusive and exhaustive. For example,
a variable measuring type of religion—with the attributes
Christian, non-Christian, and Jewish—is not mutually exclu-
sive. Judaism is both a non-Christian religion and a Jewish
religion. A Jewish person fits into both the non-Christian and
the Jewish category. Likewise, type of city as river port city,
state capital, and interstate exit lacks mutually exclusive
attributes. One city could be all three (a river port state capi-
tal with an interstate exit), any one of the three, or none of the
three. For numerical data, you do not want any overlap. The
following question about years of school is not mutually
exclusive (it is exhaustive) Persons with exactly 6, 12 or 18
years of schooling can fit into two categories.:
How much schooling did you complete?
_____ 0–6 years _____ 6–10 years _____ 11–12 years
_____ 12–15 years _____ 16–18 years _____ 18 or more years
Exhaustive means all possibilities are included in the
measure of a variable. If you measure religion and ask
whether a person is Catholic, Protestant, or Jewish, it is not
exclusive. A Buddhist, a Muslim, or an agnostic does not fit
anywhere. You must have variable categories to include
every possible situation. For example, Catholic, Protestant,
Jewish, or other is both exclusive and mutually exclusive.
The following question about years of schooling is mutu-
ally exclusive but it is not exhaustive:
How much schooling did you complete?
_____ 6–10 years _____ 11–12 years _____ 13–15 years
_____ 16–18 years _____ 19 years
People with less than 6 years or more than 19 years of
schooling are not included.
In addition to being mutually exclusive and exhaus-
tive, scales and indexes should be unidimensional or have
one dimension. They all measure a single concept. Unidi-
mensionality says that if you combine several specific
pieces of information into a single score or measure, all the
parts should act alike and measure the same core concept.
Many indexes combine subparts of a concept into a sin-
gle measure. This appears to contradict the principle of uni-
dimensionality. It does not contradict the principle because
you can define concepts at different levels of abstraction.
You can define a general, abstract concept (e.g., happiness)
as having subparts (happiness with health, with job, with
marriage). Each subpart is an aspect of the concept’s con-
tent. One subpart, marital happiness, is at a lower level of
abstraction; yet, it too may have subparts (e.g., happy with
communication in a marital relationship, happy with sexual
intimacy, happy with sharing household tasks, and so forth).
A measure can indicate a unidimensional construct in
one situation but measure one part of a different, more
abstract concept in another situation. General happiness is
more abstract than marital happiness, which is more
abstract than happiness with communication in a m ­ arriage.
5.7.2:  Index Construction
News reports in the United States regularly mention the
Federal Bureau of Investigation (FBI) crime index, the con-
sumer price index (CPI), the index of leading economic
indicators, and the consumer confidence index. The FBI
index is the sum of police reports on seven so-called index
crimes (criminal homicide, aggravated assault, forcible
rape, robbery, burglary, larceny of $50 or more, and auto
theft). It began with the Uniform Crime Report in 1930. The
CPI, which is a measure of inflation, is created by totaling
the cost of buying a list of goods and services (e.g., food,
 Measuring Social Life 107

rent, and utilities) and comparing the total to the cost of B = the total black population of the geographic entity
buying the same list in the previous year. The U.S. Bureau (e.g., the entire city)
of Labor Statistics has used a CPI since 1919; wage increases, wi = the white population of the ith area
union contracts, and social security payments are based on W = the total white population of the large geographic
it. The Conference Board, a non-governmental organiza- entity
tion, produces the index of leading economic indicators The index of dissimilarity also shows us historical patterns of
(LEI) and the Consumer Confidence Index (CCI). The LEI segregation. For example, the black–white D index for Saint
tries to predict economic conditions in the near future by Louis was 39.1 in 1900, it rose to 54.3 by 1910, and to 92.6 in
adding scores on 11 items. The 11 items include average 1940, dropped to 89.3 in 1970 and to 74.3 by 2000 but rose
hours in employee work weeks, new unemployment to 78.0 in 2010. Researchers modified and extended the index
claims, new orders for consumer goods, new equipment to  other topics (e.g., gender segregation in occupations).
orders, building permits, stock prices (S&P 500), and so Table 5.4 illustrates the D indexes for racial relations in 10 U.S.
forth. The consumer confidence index is based on a monthly cities based on the 2010 census.
survey of a random sample of 5,000 households that has
been conducted since 1985. It contains questions about the Table 5.4  Index of Dissimilarity for 14 U.S. Cities, 2010
current situation and about future expectations. It is the
The table shows the index of dissimilarity scores. Black–white
sum of respondents’ appraisal of current business condi- segregation was highest in Milwaukee and lowest in Seattle.
tions, current employment conditions, and expectations six Asian–white segregation was lower than black–white
months hence regarding respondents’ employment condi- ­segregation in all cities, and white–­Hispanic segregation was
highest in New York City.
tions, total family income, and business conditions. Respon-
dent answers are scored positive, negative, or neutral. City/Metro White vs. White vs. White vs.
Area Black Asian Hispanic
To create an index, you combine two or more items into
Atlanta 59.0 48.5 49.5
a single numerical score. Indexes measure the most desir-
Boston 64.0 45.4 59.6
able place to live (based on unemployment, commuting
time, crime rate, recreation opportunities, weather, and so Chicago 76.4 44.9 56.3

on), the degree of crime (based on combining the occurrence Dallas 56.6 46.6 50.3

of different specific crimes), and a person’s mental health Denver 62.6 33.4 48.8

(based on the person’s adjustment in various areas of life). Detroit 75.3 50.6 52.5

Also called a summative or composite index, you add sev- Houston 61.4 50.4 52.5

eral specific numerical measures that represent parts of one Milwaukee 81.5 40.7 57.0
concept. Some indexes involve more than simply adding Minneapolis 52.9 42.8 42.5
items and are adjusted to give measure with a 0 to 1 score. New York City 78.0 42.3 62.0
Philadelphia 68.4 42.3 55.1
San Francisco 62.0 46.6 49.6

Learning from History Seattle

Tampa
49.1
56.2
37.6
35.3
32.8
40.7

Index of Dissimilarity Source: http://www.psc.isr.umich.edu/dis/census/segregation2010.html

(­accessed 12/2/14)

Social justice activists, religious leaders, social scientists, edu-

cators, and policy officials discuss racial segregation in the
United States, and it is connected to numerous other social
issues. How much residential racial segregation is there? Dun-
can and Duncan (1955) invented the Index of Dissimilarity, or D
index, to measure the integration or separation of groups
across all neighborhoods of a city or metropolitan area. The D
index reveals the degree of segregation in cities. It only com-
pares two groups and is based on city blocks. A D score of
zero means no segregation, whereas a score of 1.0 means
total segregation. If a city had a white–black D score of 65, it
means that 65 percent of white people must move to another
neighborhood to make an equal balance of whites and blacks
across all neighborhoods. The basic formula for the index of
dissimilarity is as follows:
WRITING PROMPT
Index of Dissimilarity
Since 1955, demographers and the U.S. Census Bureau measured
residential segregation with Index of Dissimilarity, but it has been
­critiqued on technical grounds. One criticism is that it only measures
two groups at a time. Look at the U.S. cities in the chart and com-
pare the three minority groups (black, Hispanic, Asian) to white–
Anglo majority. List the cities that stand out as very high or low on
the measure. Are cities with the greatest black–white segregation
also always highly segregated for other racial–ethnic groups?
The response entered here will appear in the
performance dashboard and can be viewed by
your instructor.

D = [.5 × sum of (bi/B − wi/W)] × 100

bi = the black population of the ith area (e.g., a city block)
Submit
108  Chapter 5
5.7.3:  Two Index Construction Issues
1. Count Items Equally or Weigh Them? If you find an index,
unless it is otherwise stated, assume an unweighted
index, i.e., each item has equal weight. Unless you have a
very good reason to do otherwise, add up the items with-
out modification, as if each were multiplied by 1 (or –1 for
items that are negative). In a weighted index, you value
or weight the items differently. The size of the weights
depends upon your assumptions, conceptual definition,
or specialized statistical techniques. Weighting can pro-
duce different scores than an unweighted index. For
example, in a weighted index of a desirable place to live,
the percentage of days with sunshine might be weighted
one-half the importance of a low crime rate or quality of
public schools but the same as amount of park land or
number of museums.
2. Missing Data. Missing data can become an issue when
constructing an index, threatening validity, and reli-
ability. For example, you construct an index of the
degree of societal development for 50 nations using
United Nations statistics. You sum four items:
• Average life expectancy
• Percentage of homes with indoor plumbing
• Percentage of population that is literate
• Number of telephones per 100 people
As you gather data, you discover that literacy data are
available for only 47 of the 50 countries. The remaining
three countries did not collect the data. You may drop
down to comparing only 47 nations, or you can substitute
weaker measures.
WRITING PROMPT
Index Construction
Think of some aspect of the social world that has multiple parts that
you might want to measure. Now create an index in which you
­identify 4–5 separate indicators that you could measure, then add
together to create a score. Do you believe all items should be equally
weighed or should you weigh different indicators more or less than
others? Explain.
The response entered here will appear in the
performance dashboard and can be viewed by
your instructor.
Submit
5.7.4:  Scale Construction
You can use scales to create an ordinal- or less commonly,
interval- or ratio-level measure. Most scales measure the
intensity, hardness, or extremity of a person’s feelings/opin-
ion at the ordinal level. The simplest scale is a visual rating. It
is easy to construct and use. You ask participants to indicate
their feelings or rating by checking a point on a line that runs
from one extreme to another. This conveys the idea of a con-
tinuum. Numbers on a line can help people think about quan-
tities. When using a scale, you assume that people with the
same subjective feeling mark the visual line at the same place.
Figure 5.4 is an example of a “feeling thermometer” scale.
Figure 5.4  Feeling Thermometer Graphic Rating Scale
A graphic that is used to measure feelings
100 Very Warm
90
80
70
60
50 Neither Warm nor Cold
40
30
20
10
Very Cold
0
Researchers used it to learn how people feel about var-
ious groups in society (e.g., the National Organization of
Women, the Ku Klux Klan, labor unions, physicians, etc.).
Political scientists have used it since 1964 in the National
Election Study to measure people’s feelings toward politi-
cal candidates, social groups, and public issues.
5.7.5:  Commonly Used Scales
In this section, we look at four widely used scales from
among the hundreds that researchers created: the Likert,
social distance, semantic differential, and Guttman scales.
Likert Scale  You have probably used a Likert scale
even if you did not know its name. Survey researchers use
Likert scales to measure opinions or ratings at the ordinal
level. The Likert scale offers a statement or question.
­Participants indicate their response by selecting one from a
set of answer choices, such as strongly agree, agree, dis-
agree, or strongly disagree. Other answer categories are
­possible—approve or disapprove; support or oppose;
believe the statement is always true, almost always true,
almost never true, or never true; do something frequently,
­sometimes, rarely, never. Answers in Likert scales need a
minimum of two categories. However, using only two
 Measuring Social Life 109

choices creates a crude measure and does not capture com- The index offers a score for each person, ranging from 0
plex distinctions. Usually you want to use four to six (strongly disagree to all) to 36 (strongly agree to all nine).
answer categories. If you use six categories, you can reduce
Making It Practical: Likert Scale and Mea-
to four or two categories after you collect the data. But, if
suring Self-Esteem  You have heard about self-
you collect data with just two categories, you cannot make
esteem and may have also heard that certain social
your data more precise later. Keep the number of answer
problems or personal issues stem from a lack of self-esteem.
choices to under nine. More distinctions than that are
Morris Rosenberg operationalized the idea into a widely
rarely meaningful, and people may become confused.
used measure in 1965 (Rosenberg, 1965). In the measure,
Always balance the answer choices (e.g., “strongly agree,”
people read a list of 10 statements dealing with general
“agree” with “strongly disagree,” “disagree”).
feelings and then answer using a Likert Scale from strongly
Should you use a “don’t know,” “undecided,”
agree to strongly disagree (see Table 5.5). Take this oppor-
“unsure,” “no opinion” category in addition to the direc-
tunity to measure your own self-esteem with it.
tional categories (e.g., “disagree,” “agree”) in a Likert
scale? Researchers are divided on this issue, but in most
situations include the “don’t know” category. It is better to Table 5.5  Rosenberg Self-Esteem Scale
have people who are uncertain or without an opinion indi- Strongly Strongly
cate that than to force them to guess. Agree Agree Disagree Disagree
Statements (SA) (A) (D) (SD)
Another issue arises when you have a long set of Lik-
  1. On the whole, I am ______ _____ ______ ______
ert scale questions on the same issue. If answer choices are
satisfied with myself.
set up such that answering one way (such as “strongly
  2. *At times, I think I am ______ _____ ______ ______
agree”) always indicates the same position on the issue, it no good at all.
can create problems. For example, you ask nine questions   3. I feel that I have a ______ _____ ______ ______
about the issue of a woman’s right to a legal abortion. If number of good
­qualities.
you word all nine such that the response “strongly oppose”
  4. I am able to do things ______ _____ ______ ______
means strong opposition to abortion, you might create a as well as most other
“response set” problem. Some people stop paying close people.
attention after several similar questions or have a tendency   5. *I feel I do not have ______ _____ ______ ______
much to be proud of.
to agree or disagree to similar Likert scale answer choices.
  6. *I certainly feel use- ______ _____ ______ ______
The solution is to reverse the wording of some questions. less at times.
For example, you could phrase three questions so that
  7. I feel that I’m a per- ______ _____ ______ ______
“strongly oppose” indicates opposition to legal abortion, son of worth, at least
and six so that “strongly oppose” indicates support for on an equal plane
with ­others.
legal abortion, then mix the questions. For example, after
  8. *I wish I could have ______ _____ ______ ______
the question “Do you support a woman’s choice for legal more respect for
abortion if she is pregnant due to rape?” you ask, “Do you myself.
oppose laws that restrict a woman’s access to legal abor-   9. *All in all, I am inclined ______ _____ ______ ______
to feel that I am a
tion clinics?” A person with a strong antiabortion view ­failure.
must switch from repeating “strongly disagree” to say 10. I take a positive atti- ______ _____ ______ ______
“strongly agree” to be consistent. tude toward myself.
You can combine several Likert-scaled items into an Scoring: SA = 3, A = 2, D = 1, SD = 0. Items marked with an asterisk (*) are reverse
scored: SA = 0, A = 1, D = 2, SD = 3. Sum the scores for the 10 items. The higher your
index if all measure a single concept. Consider the Self- score, the higher your self-esteem.

Esteem Index. Combining a Likert scale and index con-

struction improves reliability and validity. This is because
the index uses multiple indicators, a feature that improves
reliability. When the multiple indicators measure various
aspects of one concept, content validity is improved. In
addition, an index score can give you a more precise quan-
titative measure of a person’s opinion, raising the level of
measurement from ordinal to interval. For example, you
create an index with nine questions each having five Likert
scale answer categories. You assign a numerical score to
each answer choice (0 = strongly agree, 1 = agree, 2 =
­neutral, 3 = disagree, 4 = strongly disagree). You can create
composite measure a person’s opinion on all nine ­questions.
Likert scale answer categories are ordinal and indicate
rank. Numbering the categories does not change the scale
into a ratio level of measurement. It makes no difference
whether Strongly Agree, Agree, Disagree, Strongly Dis-
agree, and Don’t Know are score from 1 to 5, −2 to + 2, or
10, 20, 30, 40, 90. The basic scale remains at the ordinal level
of measurement. If you carefully assign numbers to catego-
ries (Strongly Disagree = 1, Agree = 2, Disagree = 3,
Strongly Disagree = 4) and assume equal distance among
each category, you can have interval but not ratio-level
measurement. Advanced statistics also provide ways to
convert ordinal into interval measures.
110  Chapter 5
WRITING PROMPT
Likert Scale
The Likert Scale is one of the most widely used formats for measur-
ing attitudes. Create a Likert scale for measuring an attitude of inter-
est to you. Give it five answer categories and number scoring.
The response entered here will appear in the
performance dashboard and can be viewed by
your instructor.
Submit
Borgadus Social Distance Scale  Social distance
is a widely used sociological concept. The Borgadus social
distance scale was created to measure the amount of social
distance separating groups. Here is how it works: People
respond to a series of ordered statements from most socially
distant to most socially intimate. For example, people from
Group X are entering your country, are in your town, work
at your place of employment, live in your neighborhood,
become your personal friends, and marry your brother, sis-
ter, son, or daughter to become part of your extended family.
People indicate their comfort with each statement. Starting
with the most distant statement, people indicate the point at
which they no longer feel comfortable. The scale assumes
that a person who refuses contact or is uncomfortable with a
distant item will refuse the socially closer items. The original
scale measured social distance among racial-ethnic groups,
but researchers adjusted the scale to measure social distance
in many other social relations, such as doctor-patient dis-
tance and social distance toward ex-convicts, toward people
with ­HIV-AIDS, or toward people with disabilities.
Example Study
Social Distance and
Immigration Views
Ayers et al. (2009) used a full social distance scale to assess
whether an aversion to Latinos by white Anglos predicted their
position on immigration policies. They conducted telephone
interviews of 549 randomly selected people who self-identified
as Anglos and lived in San Diego country. The social distance
measure was as follows:
“Would you be comfortable having a Latino person as
• a family member through marriage,
• a close personal friend,
• a co-worker or classmate,
• a next-door neighbor,
• or a resident of San Diego County?”
Answering “no” to all choices indicated a person’s strong aver-
sion to Latinos. The researchers found that aversion to Latinos,
measured with the five-question Bogardus social distance
scale, strongly predicted whether the person favored putting
limits on all legal immigration and specifically Mexican immigra-
tion. Their study confirmed findings from other studies that
white Anglo respondents applied their feelings regarding race-
ethnicity when they took positions on immigration policies.
Semantic Differential  We communicate our eval-
uations with adjectives in spoken and written language.
Many adjectives have polar opposites (e.g., light/dark,
hard/soft, slow/fast). The semantic differential measures a
person’s subjective feelings about a target concept, object,
or other person indirectly by asking for an indication on a
set of polar opposite adjectives.
The semantic differential scale uses connotations of gen-
eral adjectives that people associate with a target. To use the
semantic differential, you present research participants with
a list of about a dozen paired opposite adjectives on a visual
continuum showing 7 to 11 points between them. Partici-
pants then mark the place on the continuum that expresses
their inner feelings. The adjectives can be diverse and
include opposite positive and negative items mixed on each
side. Instead of asking a person directly, “Do you feel posi-
tive or negative toward Joan Jones a candidate for mayor?”
you ask for rating Joan Jones using general adjectives. You
might ask, “When thinking of Joan Jones, do you feel that
she is warm or cool, happy or sad, active or passive?” and so
forth. Participates give a rating from zero to seven for each
adjective pair. Researchers use the semantic differential for
many purposes. Marketing researchers use it to learn how
consumers feel about a product. Political advisers use it to
discover what voters think about a candidate or issue. Ther-
apists use it to determine how a client feels about relations
with other people. For example, you might discover that
younger voters perceive Joan Jones as traditional, weak, and
slow and as halfway between good and bad. Elderly voters
perceive her as leaning toward strong, fast, and good and as
halfway between traditional and modern. The pattern of
responses illustrates how the research participant feels over-
all. The analysis of semantic differential results is difficult.
You will need to learn rather complex statistical procedures
to analyze the data (see Heise, 1970).
Example Study
Semantic Differential and Wine
Researchers use the semantic differential to evaluate implicit
views that often are difficult to measure directly. Italian research-
ers noted a steady decline in per capita alcohol consumption,
mainly due to a decline in the consumption of wine. At that same
time, the consumption of beer and spirits, driven mainly by the
younger generations, had increased. They saw a large trend
away from the Mediterranean model of drinking. It is marked by
moderation and an association of wine with family meals. There
 Measuring Social Life 111

The researchers said that a wine preference increased

with age and was associated with more responsible drinking
habits. Also, increased wine consumption was accompanied
by a decrease of the consumption of other beverages. Some
respondents viewed wine was an “elite” product and but
lacked a knowledge of wine. This meant some respondents felt
unable to choose wine properly.

Guttman Scaling  The Guttman scale differs from

was a new trend toward North European model, characterized
by binge drinking and high consumption of beer and spirits out-
side of meals. In Tuscany region, where wine has had significant
socio-economic and cultural role and is connected with a territo-
rial identity, wine producers wanted information that might help
slow the trend away from the Mediterranean model. Marinellia et
al. (2012) used the Semantic Differential to understand the
meanings that various beverages have for Italians of “Generation
Y” (ages 16–35). They conducted face-to-face interviews with
430 Tuscan young adults aged 18–35 (the drinking age is 16, but
persons under 18 were excluded as legal minors).
After getting background information including drinking
habits, they asked respondents to complete a scale with 17
polar opposite adjectives. They used a visual scale that had
numbers 1–7 between the pairs of adjectives and had respon-
dents complete it four times—once for beer, once for wine,
and once for spirits (i.e., hard liquor), and once for soft drinks.
The 17 paired adjectives were as follows: cheap-expensive,
happy-sad, young-old, comfortable-uncomfortable, intimate-
collective; sophisticated-ordinary, pleasant-unpleasant;
usual-occasional, classic-modern, relaxing-exciting, not-
socializing-socializing, sacred-profane; euphoric-depressing;
quality-poor quality, status symbol-not status symbol; appeal-
ing-not appealing, trendy-not trendy.
Results showed that respondents perceived wine as a
sophisticated, classic, sacred, pleasant, and quality product.
They viewed beer as a collective and socializing drink and
associated it with “young,” “happy,” and “cheap.” They per-
ceived spirits as “exciting” and “trendy.” Soft beverages had
negative associations and generally were not a preferred bev-
erage. The researchers statistically analyzed the data and
found the respondents could be grouped into three clusters.
the previous scales or indexes because you use it to evalu-
ate collected data. This means designing a study with the
Guttman scaling technique in mind. Guttman scaling is a
powerful technique that tells you whether a particular
hierarchical pattern of people’s responses exists among a
set of items. Researchers applied the scale to many phe-
nomena (e.g., opinions on public issues, patterns of crime
or drug use, characteristics of societies or organizations,
voting or political participation, and psychological disor-
ders). The structured pattern is hierarchical, such that some
items are basic or lower, whereas others are advanced or
difficult. By ordering items from lower to advanced, we
can see whether the data fit a structured and hierarchical
pattern. In opinion studies, the structure is such that almost
everyone agrees with lower items, but fewer agree with
higher items. Plus, most who agree with the lower items
also agree higher ones too, but the opposite is not the case.
Example
Table 5.6  Guttman Answer Pattern: 16 Possible
Combinations for 4 Items
You have four items about what a five-year-old child knows. She knows
her age, her telephone number, whether her teacher is married, and the
mayor’s name. The little girl may know her age but no other answer, or
all three, or only her age and telephone number. In fact, for 4 items there
are 16 possible combinations of answers or patterns of responses.
Each item is “yes” or “no” for whether the child knows the answer.
Age Phone Number Teacher Marriage Mayor’s Name
1 YES YES YES YES
2 YES YES YES NO
3 YES YES NO YES
4 YES YES NO NO

1. The first cluster was older and evenly divided between 5 YES NO YES YES

males and females. They drank different beverages but 6 YES NO YES NO
tended toward wine, and they drank less than the other 7 YES NO NO YES
two clusters. Getting drunk was not a motivation to drink 8 YES NO NO NO
alcohol in this cluster. 9 NO YES YES YES
2. The second cluster was much younger and predominantly 10 NO YES YES NO
male. They mostly drank beer or spirits, and a majority said 11 NO YES NO YES
that getting drunk was their prime motivation for drinking.
12 NO YES NO NO
3. The third cluster was mixed male and female. They drank 13 NO NO YES YES
more than cluster 1 but less than cluster 2. When indoors
14 NO NO YES NO
they often drank wine, and when outdoors they drank
15 NO NO NO YES
either beer or wine. Getting drunk was a motivation for
16 NO NO NO NO
some in this group but not for the majority.
112  Chapter 5

At one extreme (sequence 1) is the child who knows all the
four items. At the opposite extreme (sequence 16) is the child
who knows none of the four. With a Guttman scale, you
hypothesize combinations that will be frequent and fit a
structured pattern for many participants. Let us say you think
the pattern is the following order: age, phone number,
teacher marital status, and mayor’s name. Most children
know their age even if nothing else, but few who know the
mayor’s name do not also know the lower or “easier”
­ uestions. A Guttman scale allows you to measure how well
q
the data fit a hierarchical pattern by seeing how many people
answer in the predicted pattern and how many answer in
other ways. Various statistical measures can tell you how
“scalable” the data are ranging from 0 to 100 percent (i.e.,
how well they fit the hierarchical pattern among items that
you hypothesized). A score of 100 percent indicates that
everyone’s answer fits the hierarchical or scaled pattern and
0 occurs with a random pattern, or an absence of hierarchy.

Example Study
Guttman Scaling and Neighborhood Preference
Xie and Zhou (2012) hypothesized that whites’ preference for hypothetical neighborhoods. Each of the neighborhoods had
housing in a racially mixed neighborhood with African ­Americans, varying numbers of blacks: 0, 1, 3, 5, and 8 of 14 immediate
based on the mix of percentage of African Americans versus neighbors, representing a mix of 0, 7, 21, 36, or 57 percent
whites, might fit the Guttman scaling pattern. They examined blacks. The percentage of white respondents willing to move
survey data on approximately 8,900 adults from the 1990s taken into each type of neighborhood indicated the whites’ neighbor-
from respondents in Detroit, Atlanta, Los Angeles, and Boston hood preferences.
metropolitan areas. The survey showed five images of 14 The authors hypothesized that if responses fit a Guttman
houses, i.e., five neighborhoods, with the houses filled-in white scale, any respondent willing to move to a neighborhood with a
or black to represent the race of the family in the house. It asked higher level of blacks’ presence would also be willing to move to
white respondents to express their willingness to live in the five a neighborhood with a lower level. This rank-order divides the
respondents into six categories. Five conform to a Guttman hier-
archical scale of varying tolerance of black neighbors. The last
category (#6) is for those who did not fit the Guttman scale
Table 5.7  Whites’ Preference for Different Racial requirement. We can see this in Table 5.8. For example, category
Mixes in a Neighborhood 1 are whites who cannot tolerate a single black out of 14 neigh-
Percentage of White Respondents bors. Category 2 consists of whites who can tolerate only one
Willing to Move into the Neighborhood black neighbor but not two or more black neighbors; and so
Percentage
of Blacks in a by City* forth. The next set of four columns to the right show percentages
Neighborhood Detroit Atlanta Los Angeles Boston of whites in the categories for the four metropolitan areas. The
0% 96% 96% 95% 94% last row (category 6) shows that the data fit the Guttman scale
7% 87% 88% 94% 92% very well. There are fewer whites in category 6, about 5 percent
than in any other. It appears that whites in Detroit and Atlanta are
21% 70% 74% 89% 85%
less tolerant of black neighbors than those in Los Angles and
36% 43% 50% 73% 62%
Boston. We see this clearly from the percentage of category 1. It
57% 29% 32% 59% 46%
is much higher for Detroit and Atlanta than for Los Angeles or
* rounded to nearest whole percent
Boston.

Table 5.8  Guttman Scale Pattern and Data for Whites’ Residential Preferences
Percentage of Whites with Category
GUTTMAN PATTERN Percentage of Blacks in Neighborhood Response by City
Category 0% 7% 21% 36% 57% Detroit Atlanta Los Angeles Boston
1 Y N N N N 10.5% 9.3% 3.1% 4.9%
2 Y Y N N N 18.1% 15.0% 6.8% 8.0%
3 Y Y Y N N 26.7% 22.9% 15.2% 22.2%
4 Y Y Y Y N 13.9% 18.2% 14.2% 14.4%
5 Y Y Y Y Y 26.6% 30.0% 55.9% 44.3%
6 Not Guttman Scalable 4.3% 4.4% 4.8% 6.2%
 Measuring Social Life 113

Summary Review WRITING PROMPT

Guttman Scale

The Guttman Scale helps tell us whether a pattern is operating, such

Table 5.9  Four Major Scales that nearly everyone who has a highest level of a characteristic also
has its medium level, and everyone with a medium level has the low
levels, yet some people remain at lower levels and do not have
Scale What Does the Scale
higher levels. Come up with an example of something which you are
Name Type of Scale Indicate?
familiar with that follows the Guttman Scale pattern.
Likert General Attitude The direction and intensity of an
Measure attitude with ranked answers that The response entered here will appear in the
show the degree of agreement/ performance dashboard and can be viewed by
support your instructor.
Borgadus Measure of Social Acceptance of various levels of
Distance social closeness or intimacy with
outgroups Submit
Semantic Indirect Measure Subjective feelings based on con-
Differential of Subjective notations in adjective sets
Evaluation
Guttman Measure of a Whether the responses a set of
Structured items correspond to a preset hier-
­Pattern of archical pattern.
Responses

Summary: What You Learned about Measuring Social Life

You learned about the principles of measurement in
quantitative and qualitative data studies. You also
learned that a vital task is to conceptualize your ideas,
i.e., refine and clarify them with very explicit conceptual
definitions. Another task is to operationalize the ideas,
i.e., develop a set of techniques that link each conceptual
definition to specific operations or measurement proce-
dures. With quantitative data, you generally follow a
deductive path, whereas for qualitative data you use an
inductive path. The goal remains the same: establish
unambiguous links between each abstract idea and the
empirical data.
You also learned about the principles of reliability and
validity. Reliability refers to the dependability or consist-
ency of a measure. Validity refers to a measure’s truthful-
ness; how well an idea and the measure for it fit together.
The principles operate differently for quantitative and
qualitative data; yet, in both you seek consistent and truth-
ful measures. In both you want a good fit between the
abstract ideas you use to understand social world and
observable empirical reality.
You also learned some of the ways that quantitative
researchers apply measurement principles to indexes and
scales. There are hundreds or different scales and ways to
use indexes. You read about a few of the major ones.
Two fundamental principles of good measurement
are to create clear definitions for concepts and to use
multiple measures or indicators. These principles hold
across all fields of study and across the many research
techniques (e.g., experiments, surveys, etc.). As you are
probably beginning to realize, research involves doing
a good job in each phase of a study. Even if you conduct
one or two phases of the research project in a flawless
manner, serious mistakes or sloppiness in another
phase can do irreparable damage to the results. You
must be careful and vigilant at each phase; the overall
quality of the study will depend on how well you con-
duct all phases.
Quick Review
Do We Measure with Numbers
or Words?
1. Measures make the ideas we have about the social or
physical world that otherwise are unseen visible and use-
able in social research.
2. The central processes in measurement are conceptual-
ization and operationalization. Conceptualization is
114  Chapter 5
­ arefully thinking through a concept or idea. Operational-
c 5. Three types of measurement validity are: (1) face, agree-
ization turns an idea into a set of operations, or what we
do to measure the idea.
3. The outcome of the conceptualization process is a con-
ceptual definition. It is stating our ideas precisely in words
and defining what we specifically mean by the idea in a
way others can see.
4. The outcome of the operationalization process is an
operational definition. This is another name for the very
specific actions we use to indicate or to measure a con-
cept.
5. We connect the operational definitions of variables to
their corresponding conceptual definitions using logic
and analytic reasoning. We connect the conceptual defi-
nitions of variables to the concepts of a theory using
logic, careful reasoning, and precise terminology. We
sometimes include examples as well to convey the
meaning of ideas clearly and unambiguously.
6. The basic principles of measurement and processes of
conceptualization and operationalization are used in both
quantitative and qualitative research.
7. Most quantitative research tests hypotheses. The con-
ceptual hypotheses link variables at the level of theory,
while an empirical hypothesis is the same hypothesis
restated at the level of data.
8. In quantitative research when we test the empirical
hypothesis using statistics, we use the logical connec-
tions of measurement process to connect the results
back to the conceptual hypothesis.
9. In research with qualitative data, we may begin with par-
tial conceptual definitions. These only become clearer
and more precise after gathering the data. Likewise, the
operational definitions begin semi-formed, but are
adjusted and refined during the data gathering process.
10. In qualitative research, conceptual definitions are
anchored in the specific observations of words, events,
or actions that are the data.
11. In qualitative research, the goal often involves theory
building. The measurement process is integrated with
other parts of a study and is not separate research step.
How Can We Create Good
Measures?
1. We want our measures to have both validity and reliability.
Reliability is necessary but not sufficient for validity.
2. Validity has multiple meanings; we are interested in mea-
surement validity. It means we are certain that our specific
measures accurately capture the core meaning of our
concepts.
3. Reliability means consistency, and no instability or
unwanted variation due the measurement itself.
4. There is an analogy between sampling error and measure-
ment error. In both, we try to match what we observe
empirically with unseen theoretical ideas, and a mismatch
is considered error.
ment by others; (2) content, measuring the full content of
the concept; (3) criterion, agreement with existing and
trusted measures of the same concept.
What Are the Principles of
Qualitative Measurement?
1. Reliable measures in qualitative data are thoughtful, con-
sistent, and dependable. Instead of a fixed, standard, and
unchanging measure, reliable means consistent but
adjusted to a specific setting.
2. In qualitative data, the idea of measurement validity often
takes the form of authenticity. This means an honest and
balanced account from the standpoint of a person stud-
ied, what is genuine for them.
3. We can classify variables as discrete or continuous. The
information in discrete variables is contained in a limited
number of separate categories, while continuous variables
have a very large number of categories that can be subdi-
vided into finer subparts.
What Are the Principles of
Quantitative Measurement?
1. All quantitative variables have a level of measurement. The
four basic levels from least precise to most precise in order
are: nominal, ordinal, interval, and ratio. Only ratio-level
variables are fully continuous and also have a “true zero.”
2. Among discrete variables, nominal-level variables only imply
difference, not any inequality or hierarchy of variable catego-
ries. Ordinal-level variables imply a ranking of variable cate-
gories. Interval-level variables are continuous and imply a
quantitatively measurable distance between categories.
3. Variables at a higher, more precise level (e.g., ratio) can be
collapsed into less precise ones (e.g., ordinal), but not the
other way around.
How Do We Construct and Use
Indexes and Scales?
1. The terms “scale” and “index” are not used consistently. In
general, a scale arranges responses as an ordinal mea-
sure with intensity, or direction using one or several indica-
tors. An index combines information from multiple separate
indicators into a one score, often a sum of their values.
2. The Likert Scale is a way to measure attitudes, especially
in survey research. Research participants respond to a
question or statement using ordinal answer categories.
3. The Borgadus Social Distance Scale is a measure of how
close or distant a person feels toward a group to which he
or she does not belong. Research participants consider
rank-ordered degrees of closeness to an outgroup and
pick the level at which they feel comfortable.
4. The Semantic Differential is an indirect measure of subjec-
tive feeling toward a person, object, or idea. Research par-
ticipants chose from a set of bipolar opposite adjectives
about the person, object, or idea.

5. The Guttman Scale is a hierarchically structured set of
responses. If research participants respond to a set of
questions or relations in a manner that indicates a
sequence of levels, one within the other, it fits the response
pattern of the Guttman Scale.
Shared Writing: Measuring Aspects
of the World
Beyond creating valid measures for scientific purposes, mea-
sures can have real consequences: such as the poverty line
determines who gets help and how well a society reduces the
number of poor. People who strongly dislike a study’s results
often criticize the measures as being inadequate so that they can
ignore the study. Controversies about measures of some aspect
of the social world are often in news.
Measuring Social Life 115
Describe a news item in which a measure of an aspect of the
social world was contested. What was the measure, and why was
it criticized?
Examine two of the new items presented by fellow class-
mates. Do you agree with the criticism of the measure, and if so
can you suggest a better measure?
A minimum number of characters is required to
post and earn points. After posting, your
response can be viewed by your class and
instructor, and you can participate in the
class discussion.
Post 0 characters | 140 minimum
Chapter 6
The Survey
Learning Objectives
6.1 Identify the ubiquitousness of social surveys
6.2 List the three stages of conducting a
research survey
6.3 Evaluate strategies for developing effective
survey questions
6.4 Analyze some of the features of an effective
questionnaire
Seventeen nations (including Argentina, Belgium, Brazil,
Canada, Denmark, France, the Netherlands, New ­Zealand,
Norway, South Africa, Spain, Sweden, and the United
Kingdom) recognize marriages between people of the
same sex. In the United States, this has been a heated,
divisive political issue, which surfaced in 1996 when the
U.S. Congress passed the Defense of Marriage  Act. Since
116
6.5 List the advantages and disadvantages
of various survey formats
6.6 Apply best practices for preparing
the interviewer to administer social
surveys
6.7 Analyze some of the ethical issues in
conducting survey research
then, many states passed laws banning same-sex mar-
riage. In November 2003, the issue exploded after the
Massachusetts Supreme Court ruled in Goodridge v.
Department of Public Health that it was unconstitutional to
ban same-sex marriages in that state. In the 2004
P residential election, social-religious conservatives
­
pushed same-sex marriage to the forefront, displacing the

a­ bortion issue. Since the first legalization in Massachu-
setts in 2003, the courts in 35 of the 50 states have
approved or overturned bans on same-sex marriage. By
2015 only 13 states either still had bans in place or had
mixed court rulings. Finally, in July 2015, in a split deci-
sion, the U.S. Supreme Court ruled that states cannot ban
same sex marriage.
Beyond the court battles, advocacy campaigns, and
political rhetoric, you may ask: What do ordinary Ameri-
cans think about the issue? Dozens of polls and surveys tell
us that 40 to 60 percent of the public oppose legal same-
sex marriage, 30 to 53 percent favor it, with 1 to 15 percent
uncertain. The most ardent opponents tend to be men,
older people, those who say religion is central in their
lives, people with less schooling, those with strong anti-
abortion views, and people who want to deny atheists
legal rights. Results vary somewhat by when and how a
survey asked the question. Did it ask about same-sex
marriage, gay marriage, homosexual marriage, or the
marriage of gay men and lesbians? Did it ask about per-
mitting or banning such marriages? Did it place the issue
at the federal level as a constitutional amendment or as a
state law, or did it ask about a personal moral position?
Did it ask about civil unions, specific legal rights for gay
people, or only about marriage? With this issue, as with
others, if you want to know what people think and why,
you must first understand how the survey research
method operates.
6.1:  What Is a Social
Survey?
6.1 Identify the ubiquitousness of social surveys
Nearly everyone has completed a survey or read about
survey results. The social survey may be too familiar
and popular. Many people say “do a survey” to get
information when they should ask, “What is the most
appropriate research technique for learning about this
issue?” A survey may be the method appropriate
depending on what you want to find out. Also, because
surveys are ubiquitous and asking questions is easy,
people find it easy to create a survey. Unfortunately, it is
even easier to create a survey that produces misleading
or worthless results. A good survey that yields accurate
data requires serious thought and effort. In this chapter,
you will learn what makes a quality social survey, limi-
tations of the survey method, and how to conduct your
own survey.
Survey data come from self-reports. It requires you to
phrase the research question and variables as questions
The Survey 117
that people are willing and able to answer without great
difficulty. Examples of these types of questions include:
how much schooling a person completed or whether a
person favors or opposes same-sex marriage. To learn
about things people are unaware of or are unwilling to
self-report on, such as illegal behaviors, require special
adjustments to surveys or may not be possible using the
survey technique.
In a social survey, respondents hear and give an
answer to the exact same questions. The questions may be
about past behaviors, experiences, opinions, and character-
istics. You can use one survey to measure many variables
and test multiple hypotheses at the same time. You can test
hypotheses by looking for patterns in the data (answers
given). For example, you might hypothesize that a per-
son’s views on other issues (e.g., gun control, immigration,
spanking a child) are associated with how a person feels
about same-sex marriage. In short, one attitude predicts
the other. Statistical techniques allow you to see whether
an association exists between other views and feelings
about same-sex marriage.
6.1.1:  How Does an Opinion Poll
Differ from a Social Survey?
The difference is minor. An opinion poll is a type of
­survey; it is a short survey about opinions on current
issues. Most polls look at a sample over a short time
period, such as one week. Many polling organizations
(Roper, Gallup), media organizations (CNN, ABC
News/Washington Post, New York Times/CBS, Fox News),
political organizations, and research centers (Pew
Research Center, National Opinion Research Center) con-
duct polls on current issues, such as same-sex marriage.
Most reputable polling organizations use techniques simi-
lar to those used by professional academic survey
researchers.
In addition to polls, there are other types of surveys.
Some use samples, others do not; some survey the public,
and others focus on a specific group. Beyond asking about
opinions on current issues, a survey may ask about
knowledge, social background characteristics, general
beliefs, or behaviors. For example, a business may con-
duct a survey to learn about employee job satisfaction or
customer product preferences, and a medical clinic may
conduct a survey to document patient health behaviors.
Polls rarely have more than a dozen questions, but sur-
veys can have over a hundred questions. Survey research-
ers measure many variables simultaneously and analyze
survey data to test hypotheses, explore relationships
among variables, and document people’s thoughts and
actions.
118  Chapter 6

Example Study
Views on Same-Sex Marriage
Between August 11 and 17, 2009, Princeton Survey Research same-sex marriage. Over time, the public moved to be more
Associates International conducted 2,010 ­ t elephone ­supportive of same-sex marriages. In 2001, only about one-third
­interviews for the Pew Forum on Religion & Public Life (see of American adults supported same-sex marriage (35 percent),
Figure 6.1). while 57 percent opposed it. By 2014, a majority (54 percent)
A total of 1,510 respondents over the age of 18 were inter- supported it (see Figure 6.2).
viewed on a landline telephone, and 500 were interviewed on a
cell phone. The sampling used a random digit dialing (RDD)
sample of landline and cell phone numbers in all 50 U.S. states Figure 6.2  U.S. Opinion Trends Regarding Same-Sex
and the District of Columbia. Telephone interviewers asked the Marriage, 1996–2014
following two questions (in random order) with Likert scale
Source: PEW Research Center
answer choices:
80
Favor Same-Sex Marriage
• Do you strongly favor, favor, oppose, or strongly oppose Oppose Same-Sex Marriage
70
allowing gay and lesbian couples to marry legally?
60
• Do you strongly favor, favor, oppose, or strongly oppose
allowing gay and lesbian couples to enter into legal agree- 50

Percent
ments with each other that would give them many of the 40
same rights as married couples? 30
Among respondents, a majority (57 percent) favored allow- 20
ing gay and lesbian couples to enter into legal agreements (civil
10
unions) with each other that give them many of the same rights as
0
married couples. In the survey, opponents of same-sex marriage
96
01
03
04

06

08
09
10
11
12
13
14
15
16
05

07
outnumbered supporters, 53 percent opposed allowing gays and
19
20
20
20

20

20
20
20
20
20
20
20
20
20
20

20
lesbians to marry legally, compared with 39 percent who support Year

Figure 6.1  Alternative Survey Questions

Examples of alternative survey questions about views on same sex marriage from two national survey organizations.
Survey questions asked by the Pew Center for Research for the People & The Press
Do you strongly favor, favor, oppose, or strongly oppose
[INSERT ITEM; ASK ITEMS IN ORDER].

a. Allowing gays and lesbians to marry legally?

b. Allowing gays and lesbians to enter into legal agreements with each other that would give them many of the same rights as married couples?
ASK IF FAVOR LEGAL AGREEMENTS AND OPPOSE SAME-SEX MARRIAGE
Why do you oppose allowing gays and lesbians to marry legally?
[OPEN END: ACCEPT UP TO THREE RESPONSES.] [IF RESPONDENT IS UNCLEAR/IF NECESSARY:
“You said you favor allowing legal agreements for gay and lesbian couples but that you oppose gay marriage, why is that?]
c. ASK ALL Regardless of your opinion about same-sex marriage, do you think legal recognition of it is inevitable, or not?

(SOURCE: “Political Survey” Pew Research Center For The People & The Press Final Topline, © May 2013 Pew Research Center.

The survey question asked by the National Opinion Research Center in the General Social Survey.

Do you agree or disagree? Homosexual couples should have the right to marry one another.

Values Categories
1 STRONGLY AGREE
2 AGREE
3 NEITHER AGREE NOR DISAGREE
4 DISAGREE
5 STRONGLY DISAGREE
0 NAP
8 CANT CHOOSE
9 NA

(SOURCE: General Social Surveys, © 1972-2006 National Opinion Research Center. 

 The Survey 119

6.1.2:  Survey Data and Cause-Effect
Explanations
A cause-effect explanation based on survey data differs
somewhat from other research techniques, such as the
experiment. Recall that to say one variable causes another,
three conditions must be met:
1. The independent variable must come before the depen-
dent in time
2. The two variables must be associated, or correlated
with one another
3. There is no alternative cause for the relationship, or
there is no spuriousness
Survey data are sometimes called correlational,
because they best satisfy the second condition of causal-
ity. Meeting the first condition, time order, can be compli-
cated with survey data because most survey data are
collected at a single time point. Without data at multiple
time points, we rely on logic to show that information
from one survey question (e.g., father’s occupation while
growing up) occurred earlier than that from another (the
person’s current income). Meeting the third condition, no
alternative cause, requires thinking of variables that could
be possible alternative causes and measuring them in the
survey. These are control variables, because we can sta-
tistically control for, or take into account, their effects
using statistics. For example, you find that widowed peo-
ple have more health problems than married people do.
Before you say that being widowed itself is the cause of
poor health, you must consider alternative causes that
might make the relationship spurious. If most widowed
people are older than most married people are, then you
must rule out the alternative of age as a cause of health
problems before you can say that widowhood causes
more health problems. You will want to ask about age in
addition to marital status and health. As you plan a sur-
vey, you need to measure variables from the main hypoth-
esis (dependent and independent variables) and variables
that represent potential alternative explanations (control
variables).
Making It Practical: Survey Research and
Control Variables  Control variables measure
variables from alternative explanations that compete
with the primary hypothesis you wish to test. Let us say
you think that gender influences differences in opinion
about same-sex marriage. An alternative explanation
might be that race, or how deeply religious a person
feels, causes the opinion. Recall the study by Sherkat, de
Vries, and Creek (2010) on African Americans, religion,
and views on same-sex marriage. You learned that reli-
gious views and attendance explained greater African
American opposition to same-sex marriage. In that study,
the authors looked at many independent variables
besides religious attendance and view, such as educa-
tion, age, gender, political belief, that influence views on
same-sex marriage. The authors found that religion had
the largest impact on African American views. They
looked at all variables and used statistical techniques to
see which one had the greatest predictive power. Reli-
gious attendance and views were the most powerful fac-
tors for African Americans, but the variables were not as
important for whites for whom political beliefs were the
dominant factor predicting views on same-sex marriage
(see Table 6.1).

Table 6.1  View on Homosexual Marriage Based on Gender, Age, and Religiosity
Gender of Respondent
View on Gay Marriage Male Female Total
Agree 24.0% 34.1% 29.6%
Neither 13.3% 15.6% 14.6%
Disagree 62.7% 50.2% 55.7% Gender gap 5 12.5%
Total (N) 525 659 1184
100.0% 100.0% 100.0%

Age of Respondent

Under 35 35–55 561 Total

Agree 41.0% 25.9% 22.4% 29.7%
Neither 14.6% 16.1% 2.1% 14.5%
Disagree 44.5% 58.0% 65.4% 55.8% Oldest/youngest gap 5 20.9%
Total (N) 371 491 321 1183
100.0% 100.0% 100.0% 100.0%
120  Chapter 6

Table 6.1  Continued

How Religious?
Not Very
View on Gay Marriage Strongly Religious Not Religious Total
Agree 21.6% 30.9% 47.0% 29.5%

Neither 10.4% 16.3% 20.2% 14.5%

Disagree 68.0% 52.9% 32.7% 55.9% Strong/no religion gap 5 35.3%

Total (N) 462 541 168 1171

100.0% 100.0% 100.0% 100.0%

Generated by author from General Social Survey 2004 data.

WRITING PROMPT Figure 6.3  Steps in the Process of Survey Research

Correlational Studies
Step 1:
Some people criticize survey research because it is “correlational” • Develop hypotheses
and does not demonstrate causality the way an experiment can. • Decide on type of survey
Identify aspects of the world or relationships for which correlational (mail, interview, telephone)
studies like surveys are all that is available.
• Write survey questions
The response entered here will appear in the • Decide on response categories
performance dashboard and can be viewed by • Design layout
your instructor.

Submit
Step 2:
• Plan how to record data
• Pilot test survey instrument
6.2:  How Do We Conduct
a Survey? Step 3:
6.2 List the three stages of conducting a research survey • Decide on target population
• Get sampling frame
Once you decide that the survey is an appropriate method • Decide on sample size
for gathering data to test a hypothesis, you proceed • Select sample
through three stages start-up, implementation and data
analysis. We can subdivide the stages into six steps (see
Figure 6.3). Step 4:
• Locate respondents
6.2.1:  Start-Up Stage • Conduct interviews
• Carefully record data
In this stage, you address the following three questions:
Who will be the respondents of your survey?
What information do you want to learn from them?
Step 5:
How can you effectively get that information? • Enter data into computers
The type of respondent influences the topics you ask about • Recheck all data
• Perform statistical analysis on data
and question wording. At the beginning, you need to think
about the respondents. Topics relevant in a survey of nurs-
ing home residents may not be relevant to a survey of college
students. Survey questions about the working conditions of Step 6:
part-time workers at a fast food outlet may differ from ques- • Describe methods and findings
tions on the same topic in a survey of medical doctors. in research report
• Present findings to others for
Second, you must be very clear about exactly what you
critique and evaluation
want to learn from each question. Survey questions are the

operationalization of variables and depend on how well
you conceptualized them. Before gathering data, you
should consider what the data might look like and how you
intend to use the results. Newcomers to survey research can
be disappointed because they discover that the data do not
allow them to answer their research question when they
failed to think clearly about what the results of the survey
questionnaire. Too often they ask questions unrelated to
their true concern or fail to ask specific enough questions.
To prepare a survey, follow these steps:
First, create an instrument—a survey questionnaire or
interview schedule—to measure variables. Respondents
may read the questions themselves and mark answers on a
questionnaire. Alternatively, you may prepare an inter-
view schedule—a set of survey questions designed so that
an interviewer can read them to a respondent. The inter-
view may be by phone or face-to-face. To simplify the dis-
cussion, I will use only the term questionnaire.
Once you decide on who the respondents will be,
exactly what you want to measure, it is time to start writ-
ing questions. Expect to write and rewrite questions sev-
eral times for clarity and completeness. It is important to
organize carefully the flow of questions on a questionnaire.
Base the flow of survey questions on the research question,
respondents, and the survey format (discussed later in this
chapter), and think ahead to how you will record and orga-
nize data for statistical analysis.
After you have written the questions and before collect-
ing the data, it is always best first to conduct a short pilot test
or “dry run” of the survey questionnaire. Pilot tests can
increase question clarity. Use a small set of respondents who
are similar to those in the final survey. After they answer,
briefly interview them and ask the pilot respondents whether
the questions were clear, whether they interpreted the ques-
tions with the intended meaning, and whether the answer
choices offered were sufficient. Based on pilot test feedback,
you may want to reword the questions or answer choices or
decide to reorganize items in the questionnaire for clarity. In
this start-up stage, you also draw the sample of respondents.
6.2.2:  Implementation Stage
After selecting respondents, writing questions, and revising
the questionnaire, you are ready to collect data. Many new-
comers are surprised that planning and preparation require
much time. After you have located the sampled respondents
in person, by telephone, over the Internet, or by mail, you
must provide each with information about the survey and
instructions on how to complete it. Survey questions follow
a simple stimulus/response or question/answer pattern.
You will need to create a system to record all responses
clearly and accurately immediately after respondents give
them. After a respondent finishes and you thank him or her,
you should also quickly review responses for completeness
The Survey 121
and clarity. After all respondents have completed all ques-
tionnaires, you then organize the recorded data and pre-
pare them for statistical analysis.
Large-scale survey research, such as with 2,000 respon-
dents located across a wide geographic area and asking 100
questions, can be very complex and expensive. It requires
coordinating many people and has dozens of steps. Such a
large survey research project requires excellent organiza-
tion and accurate recordkeeping to keep track of each
respondent, questionnaire, and interviewer. Similar proce-
dures apply for a small-scale survey, such as distributing
questionnaires to 80 people in one location with 20 ques-
tions, but they are more manageable.
Whether you conduct a large- or small-scale survey,
assign an identification number to each respondent. Place
the number on each questionnaire and then check com-
pleted questionnaires against a list of sampled respon-
dents. You should review the responses on each
questionnaire and transfer data from questionnaires to a
format for statistical analysis. Also, be sure to store the
original questionnaires (physical copies or electronic ones)
in a secure place. Meticulous bookkeeping and labeling are
essential. Otherwise, you may find that valuable data and
your efforts are lost through sloppiness.
6.2.3:  Data Analysis Stage
In this stage, you have all the data so you are ready for the
analysis, interpretation, and reporting of survey data. This
stage in a survey differs little from the types of analysis
and reporting you would use for other sources of quantita-
tive data (e.g., an experiment, content analysis).
6.3:  Writing Good Survey
Questions
6.3 Evaluate strategies for developing effective survey
questions
Excellent communication is essential to writing quality
survey questions. Two core principles guide writing sur-
vey questions: avoid confusion, and keep the respondent’s per-
spective in mind. Good survey questions provide a valid
and reliable measure of variables. They also help respon-
dents feel that they understand exactly what you are ask-
ing in a question and that their answers are meaningful.
When questions fail to mesh well with a respondent’s
viewpoint or respondents find them confusing, the survey
questions will not produce high-quality data.
You face a dilemma in survey research. You want each
respondent to hear the exact same question, because the
standard procedure is to measure each variable the same
way across many people. On the other hand, if respondents
122  Chapter 6
have diverse backgrounds and use different frames of
r­ eference, the exact same wording may not carry the exact
same meaning to all respondents. However, if you tailor
question wording to each respondent, it will make com-
parisons very difficult, because you will not know whether
the question wording or differences among the respon-
dents account for variation in answers.
Writing good survey questions takes practice, patience,
and creativity, even for experienced and skilled profession-
als. You can get a sense of the principles of survey question
writing by looking at things to do or to avoid when you
write survey questions.
What are three things to do when writing questions?
Compare Your Thoughts
1.  Keep it simple and write for someone reading at an eighth-
grade level  For the general public, use the language used on
television, i.e., for someone with no higher than an eighth-
grade vocabulary and reading level, and use simple, short
sentences. It also means writing in a way that anyone can
understand, avoiding jargon, slang, and abbreviations. Jar-
gon and technical terms come in many forms. Plumbers talk
about snakes, lawyers about a contract of uberrima fides,
psychologists about the Oedipus complex. Slang is a kind of
jargon within a subculture. Homeless people may talk about
a snowbird and skiers about a hotdog. Abbreviations are
problematic. NATO usually means North Atlantic Treaty Orga-
nization, but for a respondent, it might mean something else
(National Auto Tourist Organization, Native Alaskan Trade
Orbit, or North African Tea Office). Slang and jargon are only
useful when surveying highly specialized populations.
2.  Be clear, concise, and precise  You want to be very specific
to reduce respondent confusion and get more information.
Ambiguity and vagueness plague most question writers. Most
of us make assumptions without realizing it. When we write sur-
vey questions, we should be aware of our assumptions and be
sensitive to differences in the life situations among respondents.
Consider the question “What is your income?” It may appear
straightforward; however, it could mean weekly, monthly, or
annual; family or personal; before taxes or after taxes; for this
year or last year; from salary or from all sources. The confusion
can cause inconsistencies. Different respondents may assign
diverse meanings and answer the question as they interpret it,
not as you mean it. If you want before-tax annual family income
for the last year, be specific and ask for it. Ill-defined words or
response categories create ambiguity. An answer to the ques-
tion “Do you jog regularly? Yes _____ No _____” hinges on the
word regularly. Some respondents may define regularly as
“every day.” Others may think it means “once a week.”
3.  Use neutral language  Many words have more than diction-
ary definitions and contain emotional content. Advocates or
advertisers try to manipulate the emotional aspects of language
to persuade you, but when writing survey questions you want to
use neutral words. If you use words with a lot of emotional
“­
baggage,” respondents may react to the emotionally laden
words rather than directly answer your question. Asking “What
do you think about a policy to pay murderous terrorists who
threaten to steal the freedoms of peace-loving people?” is full of
emotional words—such as murderous, freedoms, steal, and
peace. It is not always easy to know what words have emotional
baggage or to avoid them. Survey questions refer to same-sex
marriage as “gender-neutral marriage,” “equal marriage,” “gay
marriage,” “lesbian marriage,” “homosexual marriage,” and
“same-gender marriage.” More people oppose “homosexual
marriage” than “gay marriage,” and more oppose “gay mar-
riage” than “same-gender marriage,” and more oppose it than
“equal marriage.” The phrase used can influence answers.
What are three things to avoid doing when writing
­survey questions?
Compare Your Thoughts
1.  Avoid prestige bias  Highly ranked positions in society
(e.g., judge, scientist, medical doctor, CEO, etc.) and celebri-
ties (e.g., movie stars, sports heroes) have a lot of prestige or
high social status. Prestige bias is linking an answer choice
for a survey question with a well-known or prestigious person
or group. Many respondents will answer based on their feel-
ings toward the prestigious person or group rather than
addressing the issue. Consider the question, “President
Obama supports legalizing same-sex marriage. Do you favor
or oppose a law to ban same-sex marriages?” With a ques-
tion like this, you cannot distinguish whether a respondent is
giving his or her own view on the issue of same-sex marriage
or expressing his or her feeling toward President Obama.
2.  Avoid writing double-barreled questions  Make each ques-
tion about one and only one issue. A double-barreled ques-
tion joins two or more questions together and makes a
respondent’s answer ambiguous. For example, consider the
question, “Do you support marriage and civil unions for gay
people?” A respondent who opposes marriage but supports
civil unions could respond either “yes” or “no.” If respondents
read the question very strictly, emphasizing the word “and” in
it, they would say no. However, if they interpret it weakly, read-
ing the word “and” meaning and/or, they may answer “yes”.
The problem is that you would not know what they really think.
If you want to ask about two things—for example, marriage
and civil unions—write two separate questions. Once you have
the results, you can look at the answers to both questions and
see whether a respondent supports one or the other position
or neither or both. You both avoid confusion and get more
information by using two separate questions.
3.  Avoid treating a respondent’s belief about a hypothesis as a
test of the hypothesis  People have beliefs about many things.
They may think exercise causes people to have a positive atti-
tude about life, or they may think having more education
increases acceptance of same-sex marriage. If you want to
 The Survey 123

examine the hypothesis that educated people are more
accepting of same-sex marriage, you need two questions,
one about each variable (education, view on same-sex mar-
riage). After you have the data, simple statistics allow you to
see whether the two variables are associated with one
another. The wrong way to test the hypothesis between edu-
cation and favoring same-sex marriage is to ask people, “Do
you think less educated people oppose same-sex marriage
more than highly educated people?” This asks people their
opinion about the hypothesis and can tell you about people’s
beliefs about the variables. It does not reveal the actual rela-
tionship among the two variables. The people’s beliefs might
be right or wrong about the actual relationship.
WRITING PROMPT
Specific Survey Questions for Targeted Groups
A dilemma when writing survey questions is between writing ques-
tions that a highly specific and homogenous type of respondent can
easily understand, and using the same survey questions for a large,
diverse collection of respondents.
Pick an issue and write two narrow and specific survey ques-
tions that would be well understood by group to which you belong.
Now, imagine a very different type of respondent (e.g., much dif-
ferent age, racial-ethnic or cultural background, education level).
Rewrite the questions so that this very different type of respondent
would also understand the questions.
The response entered here will appear in the
performance dashboard and can be viewed by
your instructor.
Submit
6.3.1:  What Are Leading Questions?
You’ve probably heard about “leading questions,” but what
are they? You should never intentionally use leading ques-
tions in an honest, ethical survey. A leading question prompts
the respondent to pick one response over another. They fre-
quently appear in dishonest surveys, in which someone tries
to manipulate results or mislead people. They also can occur
when an inexperienced survey question writer is unclear.
There are many types of leading questions. In a good survey
question, respondents do not know which answer you expect;
they feel free to state what they really think or feel.

Summary Review

Table 6.2  Survey Question Writing Pitfalls

Things to Avoid
  1. Jargon, slang, abbrevia-
tions
  2. Vagueness
  3. Emotional language
  4. Prestige bias
(also Double-barreled)
  5. Double-barreled
­questions
  6. Confuse beliefs about a
hypothesis with testing
the hypothesis
  7. Leading questions
  8. Issues beyond respond-
ent capabilities
  9. False premises
10. Distant future intentions
11. Double negatives
12. Unbalanced responses
Not Good
Did you drown in brew until you were totally blasted
last night?
Do you eat out often?
The respected Jasper Commission documents that a
staggering $500 Billion of your tax dollars are being
wasted on public schools through poorly trained or
incompetent teachers, outdated curriculum, weak man-
agement unable to deal with unions, “and other wasteful
practices. Is cutting excessive public school spending and
providing vouchers that will allow parents to choose effec-
tive private schools a top priority for you?”
Do you support or oppose raising social security ben-
efits and increased spending for the military?
Do you think more educated people smoke less?
Did you do your patriotic duty and vote in the last elec-
tion for mayor?
Two years ago, how many hours did you watch TV
every month?
When did you stop beating your girl/boyfriend?
After you graduate from college, get a job, and are set-
tled, will you invest a lot of money in the stock market?
Do you disagree with those who do not want to build a
new city swimming pool?
Did you find the service at our hotel to be: Outstand-
ing, Excellent, Superior, or Very Good?
A Possible Improvement
Last night, about how much beer did you drink?
In a typical week, about how many meals do you eat away
from home, at a restaurant, cafeteria, or other eating estab-
lishment where you pay for food?
How important is it to you to reduce wasteful public school
spending?
Very Important
Somewhat Important
Neither Important or Unimportant
Somewhat Unimportant
Not Important at All
Do you support or oppose raising social security benefits?
Do you support or oppose increasing spending on the military?
What is your education level? Do you smoke cigarettes?
Did you vote in last month’s mayoral election?
In the past two weeks, about how many hours do you think
you watched TV on a typical day?
Have you ever slapped, punched, or hit your girl/boyfriend?
Do you have definite plans to put some money into the
stock market within the coming two months?
There is a proposal to build a new city swimming pool. Do
you agree or disagree with the proposal?
Please rate the service at our hotel: Outstanding, Very
Good, Adequate, or Poor.
124  Chapter 6

Making It Practical: Improving Unclear WRITING PROMPT

Questions  Here are three survey questions written by
Leading Questions
experienced professional researchers. They revised the
original wording after a pilot test revealed that 15 percent Many survey questions asked in public opinion polls about public
issues are “leading” to some degree (especially those by political or
of respondents asked for clarification or gave inadequate advocacy groups, or by companies checking customer satisfaction).
answers (e.g., don’t know). As you can see, question word- Write a question that you believe to be “leading.” Describe what
ing is an art that may improve with practice, patience, and makes it a leading question.
pilot testing (see Table 6.3). The response entered here will appear in the
performance dashboard and can be viewed by
your instructor.
Table 6.3  Responses to Question and Percentage
Asking for Clarification
Submit
Original Question Problem Revised Question
Do you exercise or What counts as Do you do any sports
play sports regularly? ­exercise? or hobbies, physical
activities, or exercise, 6.3.2:  Getting Answers to Survey
including walking, on a
regular basis? Questions
What is the average Does margarine count The next question is
number of days each as butter? just about butter—not Should you use an open- or closed-ended response
week you have butter? including margarine. ­format?
How many days a
week do you have In an open-ended response format (also called unstruc-
butter?
tured or free response), respondents can give any answer,
[Following question on How many eggs is a On days when you eat
eggs] What is the serving? eggs, how many eggs whereas in the closed-ended response format (also called
number of servings in What is a typical day? do you usually have? structured or fixed response), they must choose among
a typical day?
fixed answer choices. Each format has advantages and dis-
Responses to Percentage Asking advantages. You must decide which format is most appro-
Question for Clarification
priate for the conditions or purpose of your particular
Original Revision Original Revision survey. This depends on your study’s goal and the practi-
Exercise ques- 48% 60% 5% 0% cal limitations of your research project.
tion (% saying
“yes”)
An open-ended response format takes more time for
Butter question 33% 55% 18% 13%
respondents to answer than a closed-ended format. It also
(% saying requires a respondent to have minimal writing or verbal
“none”) skills. It is easy for respondents to get off track and give
Egg question (% 80% 33% 33% 0% unrelated information in the open-ended format. Inter-
saying “one”)
viewers or recorders must get down extensive verbatim
Source: Adapted from Fowler (1992).
answers. Researchers must develop a coding system and
place open-ended responses into codes for data analysis,
Avoiding leading questions requires some thought.
which is more complex and time-consuming than for data
The question, “Stable gay and lesbian couples have same
in the closed-ended format. This makes the open-ended
rights as any other law-abiding citizens. Do you agree that
format impractical for all but small-scale samples (under
your fellow citizens who are gay or lesbian should have
200 people) with a few questions (under 20). Nonetheless,
equal rights to the benefits and responsibilities of a legal
it is very useful for exploratory research, when we know
marriage as anyone else?” may lead respondents to state
little about an issue. It is also more effective when the goal
that they favor same-sex marriage even if they are uncer-
is to capture a respondent’s thinking process.
tain. The phrases about the “same rights as other law-abid-
Writing survey questions with the closed-ended
ing citizens” and “your fellow citizens” make it leading.
response format requires making some decisions.
You can state leading questions to get a positive or a nega-
tive answer. For example, “Should we change laws to give • How many response choices should you offer?
sexually deviant people the privileges and benefits of a • Should you offer a middle or neutral choice?
marriage, or should we continue to uphold the natural
• How should you order answer choices?
basis of a marriage as only between one man and one
• How should you measure the direction of answers?
woman as it has been for centuries worldwide in custom,
law, and religious teachings?” leads a respondent to dis- Making such decisions is not easy. For example, two
agree. Leading questions are just one of several potential response choices are too few, but more than five choices
pitfalls to avoid when writing survey questions. can create confusion. You want to measure meaningful

­ istinctions and not collapse them. Very specific answer
d open format. Mixing formats also changes the pace and helps
choices yield more information, but being too specific may
create confusion for the respondents. You can offer more
choices and get more information without confusion if you
rephrase a yes/no question, “Are you satisfied with your
dentist?” by using a Likert scale format such as, “How sat-
isfied are you with your dentist: very satisfied, somewhat
satisfied, somewhat dissatisfied, or not satisfied at all?”
A closed-ended format is much faster and easier for both
respondents and the researcher. It permits large-scale sam-
ples and long questionnaires. Yet, you can lose important
insights by forcing an individual’s complex beliefs and feel-
ings into a few fixed categories. One way to reduce the disad-
vantages of a closed-ended format is to mix open-ended and
closed-ended formats in a questionnaire or to use a partially
Summary Review
Open- and Closed-Ended Response Formats
Table 6.4  Closed-Ended Response Formats
Advantages of Closed-Ended Format
• Easier and quicker for respondents to answer.
• Easier to compare answers across respondents.
• Easier to code and statistically analyze answers.
• The response choices can help clarify question meaning for
respondents.
• Respondents are more likely to answer about sensitive topics.
• There are fewer irrelevant or confused answers to questions.
• Less articulate or less literate respondents are not at a disad-
vantage.
• Replication of the survey is easier.
Table 6.5  Open-Ended Response Formats
Advantages of Open-Ended Format
• They permit an unlimited number of possible answers.
• Respondents can answer in detail and can qualify and clarify
their responses.
• Researchers can discover unanticipated findings.
• Respondents can give adequate answers to complex issues.
• Respondents can be creative, express themselves, and
provide rich detail.
• Respondents can reveal their thinking process and frame of
reference.
How Do You Write Good Closed-Format
Responses?  Most surveys offer preset answers from
which a respondent chooses. Writing good answer choices
is just as important as writing a good question. The answer
choices should have three features:
• Mutually exclusive. Response categories do not over-
lap. You can easily correct numerical ranges (e.g.,
5–10, 10–20, 20–30) that overlap (e.g., 5–9, 10–19,
The Survey 125
interviewers establish rapport. Periodic probes with closed-
ended answers can reveal a respondent’s reasoning. For
large-scale surveys, you can use open-ended format in small,
preliminary pilot tests, then develop closed-ended response
categories based on responses the open questions. The par-
tially open format uses a closed-ended format with an open-
ended “other” option. For example, the following is a survey
question used in CBS/Newsweek/New York Times polls that I
modified to make partially open:
Which of the following comes closest to your view?
_____ Gay couples should be allowed to marry, OR _____ form
civil unions, OR _____ there should be no legal recognition of
gay couple’s relationship, OR some other arrangement,
please specify ________________
Disadvantages of Closed-Ended Format
• Questions can suggest ideas that the respondent would not otherwise have.
• Respondents without an opinion or knowledge on an issue may answer any-
way, creating meaningless data.
• Respondents can be frustrated because their desired answer is not a choice.
• Respondents get confused if many (e.g., 15) response choices are offered.
• A respondent’s misinterpretation of a question can go unnoticed.
• Distinctions between respondent answers to a question can become blurred.
• Respondent mistakes in marking the wrong response are possible.
• Respondents are forced to give simplistic responses to complex issues.
• Respondents are forced to make choices they would not make in the real world.
Disadvantages of Open-Ended Format
• Different respondents give different degrees of detail in their answers.
• Respondent answers may be irrelevant or buried in useless detail.
• Data coding, comparisons, and statistical analysis become very difficult.
• Articulate and highly literate respondents have an advantage.
• Questions may be too general for respondents who lose direction or the ques-
tions may intimidate unsophisticated respondents.
• A greater amount of respondent time, thought, and effort is necessary.
• Answers require a lot of space and time to record.
20–29). The ambiguous verbal choice is another type
of overlapping response category—e.g., “Are you
satisfied with your job, or are there things you don’t
like about it?” It is not clear how a person who is
generally satisfied but has a few minor complaints
would answer this.
• Exhaustive. This means that each respondent has a
choice—a place to go. For example, asking respondents
126  Chapter 6
“Are you working or unemployed?” leaves out
respondents who are not working but do not con-
sider themselves unemployed (e.g., full-time home-
makers, people on vacation, students, people with
disabilities, or retired people). When writing a ques-
tion, first think about what you want to know and
then consider the circumstances of all possible
respondents. For example, if you ask about a respon-
dent’s employment, do you want information on the
primary job or on all jobs? Do you want both full-
and part-time work? Do you only want jobs for pay
or also unpaid and volunteer jobs as well? If some-
one is temporarily unemployed, do you want the last
job he or she held?
• Balanced. This means you offer the favorable or unfa-
vorable choices equally in a set of responses. A case of
unbalanced choices is the question, “What kind of job
is the mayor doing: outstanding, excellent, very good,
or satisfactory?” It offers three favorable and one
neutral response choice. Another type of unbalanced
question omits information—e.g., “Which of the five
candidates running for mayor do you favor: Eugene
Oswego or one of the others?” You can balance
responses by offering bipolar opposites. Unless there
is a specific purpose for doing otherwise, offer
respondents equal polar opposites at each end of a
continuum. Asking, “How strongly you support a
ban on same-sex marriage? Do you strongly support
it, somewhat support it, or just barely support it?” is
a set of unbalanced answers. To make it balanced,
you could ask, “How do you feel about a ban on
same-sex marriage; do you support it, oppose it, or
neither support or oppose it?”
Should You Offer a “Don’t Know” or “No
­Opinion” Response Choice?  Professional survey
researchers debate whether to include choices for neutral,
middle, and nonattitudes (e.g., “not sure,” “don’t know,”
“undecided,” or “no opinion”) in closed-ended questions.
They want to avoid two errors:
• Getting a “no opinion” or “don’t know” response when
a respondent actually holds a nonneutral opinion
• Forcing a respondent to choose a position when he or she
has no opinion on an issue or knows nothing about it.
We have three ways to address the “don’t know”
response in attitude questions:
1. standard-format
2. quasi-filter
3. full-filter questions.
A standard-format question does not offer a “don’t
know” choice, and respondents must volunteer their lack
of knowledge or opinion. A quasi-filter question offers
respondents a “don’t know” or “not certain” answer alter-
native. A full-filter question is a special type of contin-
gency question (contingency questions are discussed later).
It is a two-part question. It first asks whether respondents
have an opinion, and then asks for the opinion among
those who say that they have an opinion.
Studies of survey formats vary, but several suggest
that without a “no opinion” choice, as in the standard
question, some respondents will offer to answer a ques-
tion, even if they are very uncertain or unaware of an issue.
Many people find saying “I don’t know” or “I have no
opinion” difficult to assert or may feel embarrassed in
doing so. With a quasi-filter question, most such respon-
dents choose “don’t know” because it appears as a legiti-
mate response. A full-filtered question takes “no opinion”
or “don’t know” options one step higher. You should use a
full filter for issues about which many people may not be
informed or have a firm opinion. An option is to ask about
an opinion using a quasi-filter question, then follow up all
those with an opinion with a second question about how
strongly they feel.
Example
What is your opinion about the issue of global warming? Do you feel
in the future it is going to be a major threat, a minor threat, or no real
threat to how our society will be, or do you have no opinion?
_____  Major Threat
_____  Minor Threat
_____  No Real Threat
_____  No opinion {Go directly to next question}
[ASK ONLY IF FIRST THREE ANSWERS ARE GIVEN]
How strongly do you hold that opinion?
Do you hold the opinion _____ very strongly, _____ somewhat
strongly, or _____ not very strongly at all?
Example Study
Questionnaire Items from
the 2009 Pew Research
Center Survey
This marital status question has mutually exclusive and
exhaustive answer choices.
Are you currently married, living with a partner, divorced, sepa-
rated, widowed, or have you never been married? (IF R SAYS
“SINGLE,” PROBE TO DETERMINE WHICH CATEGORY IS
APPROPRIATE) {QID:MARITAL1}
1. Married
2. Living with a partner
3. Divorced
4. Separated

5. Widowed
6. Never been married
9. Don’t know/Refused (VOL.)
The question below reduces the social desirability bias (dis-
cussed later in this chapter) to say that a person registered by
offering the out of “can’t find the time to register.”
REGIST These days, many people are so busy they can’t find
time to register to vote, or move around so often they don’t get
a chance to re-register. Are you NOW registered to vote in your
precinct or election district or haven’t you been able to register
so far? [INSTRUCTION: IF RESPONDENT VOLUNTEERS THAT
THEY ARE IN NORTH DAKOTA AND DON’T HAVE TO REGIS-
TER, PUNCH 1 FOR REGIST AND REGICERT] {QID:REGIST}
1. Yes, registered
2. No, not registered
9. Don’t know/Refused (VOL.)
Making It Practical: Standard-Format, ­Quasi-
Filter, and Full-Filter Questions 
Standard Format  things occur at once, they can blur together in memory.
Here is a question about another country. Do you agree
or disagree with this statement? “The Russian leaders
are basically trying to get along with America.”
Quasi-Filter  • An issue or event that was not significant—people remem-
Here is a statement about another country: “The Russian
leaders are basically trying to get along with America.”
Do you agree, disagree, or have no opinion on that?
Full Filter  him/herself—people tend to selectively remember what
Here is a statement about another country. Not everyone
has an opinion on this. If you do not have an opinion, just
say so. Here’s the statement: “The Russian leaders are
basically trying to get along with America.” Do you have
an opinion on that? If yes, do you agree or disagree?
Source: Schuman and Presser. 1981:116–125.
Table 6.6  Example of Results from Different
Question Forms
Standard Form (%) Quasi-Filter (%) Full Filter (%)
Agree 48.2 27.7 22.9
Disagree 38.2 29.5 20.9
No opinion 13.6* 42.8 56.3
* Volunteered
Source: Adapted from Schuman and Presser (1981:116–125). Standard format is from
Fall 1978; quasi- and full-filter are from February 1977.
WRITING PROMPT
What If You Have No Opinion?
If you were presented with a survey question in which there was no
“Don’t Know” or “No Opinion” choice but you did not know or had
no opinion, how would it make you feel? What could you do in this
situation?
The Survey 127
The response entered here will appear in the
performance dashboard and can be viewed by
your instructor.
Submit
How Can You Help Respondents Remember? 
Many survey questions ask about past events, such as
when you last saw a doctor or when you last purchased a
cell phone. Recalling events accurately takes more time
and effort than the few seconds respondents typically use
when answering. Also, accurate answering declines over
time. Most respondents can recall highly significant events
that occurred in the past four to six weeks, but after that,
their recall accuracy erodes. The following might interfere
with good recall:
• A sensitive or threatening topic—people often suppress a
bad memory and forget unpleasant or embarrassing
events.
• Events that occurred simultaneously—when several
• Events that occurred after that being recalled—more recent
events can distort the memory of what happened
­earlier.
ber what was important for them but if they did not
consider it important, they are likely to forget it.
• A person’s need to be consistent and not appear to contradict
is consistent and forget what is contradictory.
Such influences on recall do not mean that you cannot
ask about past events, but when asking about the past you
need to customize questions and interpret results cau-
tiously. It is best to give respondents special instructions,
extra thinking time, and recall aids. One recall aid is to use
a fixed time frame or location.
Rather than ask:
“How often did you attend sporting events last year?”
Instead, ask it this way:
“I want to know how many sporting events you
attended last winter. Let’s go month by month. Think
back to December. Did you attend any sporting events
for which you paid an admission in December? Now,
think back to January. Did you attend any sporting
events in January?”
If you want information about activities over a long
period, it is best to ask about a short time frame that a
respondent is likely to know about, and then do the math
yourself afterward.
Instead of asking:
“How many hours did you watch TV in the past year?”
128  Chapter 6
Try asking:
“In a typical weekday during the past year, about how
long did you watch TV? What about in a typical weekend
day, how often?”
You can then multiply the respondent’s answers by the
number of weekdays and weekend days to create an esti-
mate for annual TV watching.
How Can You Ask Respondents about Sensi-
tive Issues?  Most respondents want to present a posi-
tive image of themselves. They may feel ashamed,
embarrassed, or afraid to give truthful answers regarding
unpleasant or unflattering behaviors or events. They may
find it emotionally painful to confront their own actions
honestly, let alone admit them to a stranger. People may
underreport or self-censor reports of behavior or attitudes
they wish to hide or believe to violate social norms such as
having an illness or disability (e.g., cancer, mental illness,
venereal disease) or engaging in illegal or deviant behavior
(e.g., evading taxes, taking illegal drugs, engaging in
uncommon sexual practices). They may be hesitant to
reveal their financial status (e.g., income, savings, or debts).
Alternatively, they may over report positive or generally
accepted behaviors.
Researchers developed several techniques to increase
getting truthful answers to sensitive issues. One is to alter
the context and question wording to be less threatening.
You should only ask about sensitive issues after a warm-
up, when respondents feel more trust in the survey or
interviewer. You can emphasize to respondents that you
want honest answers and reassure them of confidential-
ity. You can provide a context that makes it easier for
respondents to answer and appear less unusual. For
example, rather than asking “Have you stolen from a
store?,” you can ask, “In past surveys, many people have
reported that at some point they took items from a store
without paying. Have you ever taken something from a
store without paying for it?” Another technique is first to
ask about more serious activities, making the sensitive
question issue appear less unusual. A respondent may
hesitate to admit that he or she shoplifted. However, a
question about shoplifting appears after several questions
about armed robbery or burglary, respondents may admit
to shoplifting because it appears to be less serious than
the other crimes.
Social desirability bias occurs when respondents dis-
tort answers to look good or to conform to social norms.
Many people over report being highly cultured (i.e., read-
ing books, attending high-culture events), giving money to
charity, having a good marriage, loving their children, and
so forth. For example, one study found that one-third of
people who said they gave money to a local charity in a
survey really did not. Because a norm says that one should
vote in elections, people often say they voted when they
did not. To reduce social desirability bias, you can phrase
questions to make norm violation appear less objection-
able, or present a wider range of behavior as acceptable or
give respondents “face-saving” alternatives. The National
Election Survey asked about voting in the following way to
reduce the social desirability bias: “In talking to people
about elections, we often find that a lot of people were not
able to vote because they weren’t registered, they were
sick, or they just didn’t have time. Which of the following
best describes you?—One, I did not vote. Two, I thought
about voting this time but didn’t. Three—I usually vote,
but didn’t this time. Four—I am sure I voted.”
What Are Contingency Questions?  A contin-
gency question (also called screen or skip question) is a
two-question sequence that increases relevance. A first
question selects respondents for whom the second ques-
tion is relevant. It screens in/out respondents who get the
second part. The following example is a contingency
­question.
1. Did you vote in the mayoral election last April when Guo,
Smith, and Lopez were candidates?
[ ] Yes (GO TO QUESTION 2)
[ ] No (SKIP TO QUESTION 3)
2. Which candidate did you vote for? _____ Guo _____ Smith
_____Lopez _____ Don’t remember
3. What kind of overall job is the new mayor doing in your
opinion? _____ Excellent _____ Good _____ Fair _____
Poor
How Can You Avoid Specific Words That
Affect Answers?  Wording effects occur when a par-
ticular word evokes a response. Professional survey
researchers recognize that particular words in a survey
questions may trigger strong feelings or have connotations
that color answers. Because respondents react to one word
rather than thinking about the issue in a question, you
want to avoid such words in survey questions. It is easier
to write survey questions if you have a large vocabulary,
know the connotations and meanings of many words, and
are sensitive to the vocabulary of respondents. In general,
you want to use simple vocabulary and grammar to mini-
mize confusion. Unfortunately, it is not possible to know in
advance whether a word or phrase will affect responses.
Learning from History
The Power of Words
Survey researchers have uncovered several powerful wording
effects in surveys. One well-documented effect is the differ-
ence between forbid and not allow.1 Both terms mean the
1
See Foddy (1993) and Presser (1990).

same thing, but many more people are willing to “not allow”
something than to “forbid” it. In general, less educated
­respondents are most influenced by minor wording differ-
ences. Certain words trigger an emotional reaction or have
significant connotations that we are just beginning to learn
about. Smith (1987) found large differences (e.g., twice as
much support) in U.S. survey responses depending on
whether a question asked about spending “to help the poor”
or “for welfare.” Heated political attacks on welfare in the
1970s and 1980s changed connotations of the word welfare,
and it took on negative connotations that it did not previously
have. The once neutral word came to imply lazy and immoral
people as well as wasteful, ineffective, and expensive govern-
ment programs. Today, it is best to avoid using it. Likewise,
Hurwitz and Peffley (2005) discovered that in recent years
many Americans have come to associate the term inner city
with negative racial stereotypes about African Americans.
Racially prejudiced whites gave negative responses when the
phrase “inner city” appeared in a survey question but neutral
responses for the same issues when it did not appear. In a
2005 Pew Research survey, 51 percent of respondents said
they favored “making it legal for doctors to give terminally ill
patients the means to end their lives” but only 44 percent
favored “making it legal for doctors to assist terminally ill
patients in committing suicide.” Both questions asked about
the same thing, but the respondent reactions differed because
of the word “suicide.”
Respondents also can become confused about the
meaning or connotations of key words. One survey asked
respondents whether they thought television news was
“impartial.” Impartial is a ninth-grade vocabulary term, and
researchers assumed everyone knew its meaning. They later
learned that fewer than half of the respondents had inter-
preted the word with its proper meaning. Over one-fourth
had ignored it or had no idea of its meaning. Others gave it
unusual meanings, and one-tenth gave it a meaning directly
opposite to its true meaning. You need to be cautious,
because although a few wording effects (e.g., the difference
between forbid and not allow) are known, we are still learning
about the power of specific words to shape respondent
answers.
WRITING PROMPT
Wording Effects in Survey Questions
Sometimes a term with a strong emotional impact appears in a sur-
vey question and people answer based on their emotions about the
term rather than the word’s actual meaning. Brainstorm a short list
of words that you should avoid using in a survey question because
they have a lot of emotional impact.
The response entered here will appear in the
performance dashboard and can be viewed by
your instructor.
Submit
The Survey 129
6.4:  How Can You Design
an Effective Questionnaire?
6.4 Analyze some of the features of an effective
questionnaire
Once you have created a collection of survey questions,
you face two decisions in designing an effective question-
naire. First, how many questions can you include in the
questionnaire, or total questionnaire length. Second, how
should you organize or arrange the questions that are in
the questionnaire.
6.4.1:  Length of Survey
or Questionnaire
The length of a questionnaire depends on the format of
your survey and on respondent characteristics. A 3–5
minute telephone interview is rarely a problem. You can
often extend it to 10 minutes. Web surveys vary but few
people spend more than 10 minutes taking them. Mail
questionnaires are more variable. A short (three-page)
questionnaire is appropriate for the general population.
Some researchers used questionnaires as long as 10 pages
(about 100 items) with the general public, but responses
drop significantly for longer questionnaires. For highly
educated respondents and a salient topic, using question-
naires of 15 pages may be possible. Many face-to-face
interviews last a half-hour. In special situations, research-
ers have been able to conduct face-to-face interviews for
as long as 3–5 hours.
6.4.2:  Question Sequence
You face three survey question sequence issues when
designing a questionnaire:
1. How to organize questions on a questionnaire
2. How to reduce question order effects
3. How to control context effects
Several years ago, my students conducted a telephone
survey on two topics: concern about crime and attitudes
toward a new anti-drunk-driving law. A random half of the
respondents heard questions about the drunk-driving law
first; the other half heard about crime first. I examined the
results to see whether there was a context effect—a differ-
ence by topic order. This study found that respondents
asked about the drunk-driving law first expressed less fear
about crime than did those asked about crime first. Like-
wise, those asked first about drunk driving expressed more
support for a tough new drunk-driving law than those who
first heard about crime. Apparently, the first topic created a
context influencing answers to the second topic. After hear-
ing questions about crime in general and about violent
130  Chapter 6
crime, respondents may have considered drunk driving to
be a less serious offense. By contrast, respondents asked
about drunk driving first may have thought about it as
criminal behavior. When asked about crime in general, they
still are thinking of drunk driving as a type of crime.
How to Organize Questions on a Question-
naire  Every questionnaire has opening, middle, and
ending questions. Sequence the questions in a way to mini-
mize respondent discomfort and confusion. The early ques-
tions should help a respondent feel positive and comfortable
with the survey process. After an introduction that explains
the purpose of the survey, make opening questions pleasant,
interesting, and easy to answer. Demographic questions
(e.g., age, education level, and so forth) are easy but not
interesting, and should go toward the end. In addition, place
questions on the same topic together and introduce the sec-
tion with a short statement (e.g., “Now I would like to ask
you questions about housing”) to orient respondents. Ques-
tion topics should flow smoothly and logically. Organize
them to assist respondents’ memory and comfort levels.
How to Reduce Question Order Effects  The
order in which questions appear may influence a respon-
dent’s answers. Order effects are strongest for people who
lack strong views, for less educated respondents, and for
older respondents or those with memory loss. For exam-
ple, support for a single woman’s having an abortion pre-
dictably rises if it comes after a question about abortion
being acceptable when a fetus has serious defects, but not
when the question is by itself or it comes before a question
about fetus defects.
Making It Practical: The Effect of Previous
Questions  Previous questions in a questionnaire influ-
ence later ones in two ways:
1. Question content (i.e., the issue). This occurred in the
above example of my student’s study about drunk driv-
ing and crime. In another case, researchers compared
three forms to ask how much a respondent followed
politics: the question alone, after asking what the respon-
dent’s elected representative recently did, and after ask-
ing about what the representative did and about the
representative’s “public relations work” in the area. The
percentage of respondents’ reporting that they followed
politics “now and then” or “hardly at all” was 21 per-
cent, 39 percent, and 29 percent, respectively, for the
three forms. The second form apparently made respon-
dent’s feel they knew little, but the last form gave respon-
dents an excuse for not knowing the first question—they
could blame their representative for their ignorance.
2. Respondent’s response. A respondent having already
answered one part of an issue may assume no overlap.
For example, a respondent is asked, “How are you
doing in the classes in your academic major?” The next
question is, “How are you doing in your classes?” Most
respondents will assume that the second question only
means classes outside their major because they already
answered about the major. If you wanted to ask about
classes overall, then you should place that question
before the question about the classes in the major.
WRITING PROMPT
Survey Question Order
In many surveys, we ask about multiple topics. Can you think of two
topics for a survey where answering one topic first might shift how
people think about a second topic? What might we do to find out
whether topic order makes a difference, and if so how to deal with
the issue?
The response entered here will appear in the
performance dashboard and can be viewed by
your instructor.
Submit
How to Control Context Effects  Respondents
tend to answer questions based on a context of preceding
questions and the interview setting. Context effects are
strongest for ambiguous or unclear questions. This is
because respondents draw on the context when interpret-
ing a question. It is not always possible to control context
effects. A first step is to be aware of them. You want to ask
a more general question before a more specific one. It takes
a bit more work, but making two versions of the question-
naire and altering topics to two random parts of a sample
allows you to check whether context effects are operating.
Example Study
Question Order Effects
In a study with 9,000 respondents, Wilson (2010) found that
the ordering of survey questions influenced reports of interra-
cial prejudice expressed by American blacks and whites.
When questions specifically deal with salient emotional cues,
such as interracial hostility among blacks and whites, question
context and racial group membership strengthen in-group and
out-group categorization. In-group members (e.g., blacks)
tend to view members of an out-group (e.g., whites) as having
greater dislike toward them (e.g., whites dislike blacks) when
respondents were first asked about the in-group (e.g., blacks).
This produced a desire to favor one’s in-group over the out-
group—a contrast effect. When the opposite order occurred, a
norm of “evenhandedness” suggests that evaluating an out-
group in a noncomparative context, the in-group member feels
a need to appear balanced or evenhanded, particularly in the
presence of an interviewer. When first asked if their own group
dislikes the out-group, respondents felt compelled to say that
more of the out-group members disliked them in a follow-up
 The Survey 131

question. So, when black respondents were asked “Do whites
dislike blacks?” 41% said “yes.” When white respondents
were asked “Do blacks dislike whites?” 42.8% said “yes.”
However, when asked about their own in-group first, percep-
tions shifted. When black respondents were asked the same
question after they first answered whether “blacks dislike
whites,” then 55.4% said “yes” instead of 41%. When white
respondents were asked “Do blacks dislike whites?” after hav-
ing answered whether whites dislike blacks, 51% said “yes”
instead of 42.8%. In short, roughly 10 percent more people
said the out-group disliked their in-group in one situation over
the other. It all depended upon the context, or which question
they answered first. (See Figure 6.4.)
Figure 6.4  Race of Respondent by Question Order
Interaction Pattern (Predicted Probability of Perceiving
More Dislike toward Out-Group Members)
Answering Survey Questions Vary by Question Order and
Respondent’s Race
60
55.4
55
51
Percentage
Each survey format has advantages or disadvantages: self-
administered, mail, face-to-face-interview, phone inter-
view, and web survey.
6.5.1:  Mail and Self-Administered
Questionnaires
Advantages  Mail and self-administered question-
naires are popular because they are easy and inexpen-
sive. You distribute or mail questionnaires directly to
respondents, respondents read instructions and ques-
tions, and then record their answers. If you use mail, you
can cover a wide geographical area. In addition, a mail
survey allows respondents to check personal records at
home if necessary, offers anonymity, and avoids inter-
viewer bias.
Disadvantages  Physical location is limited for dis-
tributed, self-administrated questionnaires. Distribution
by mail provides large geographic area, but people do not
always complete and return questionnaires and the biggest
problem with mail questionnaires is a low response rate.

50 You might mail out 500 questionnaires but get back only
50. Increasing the number mailed out to 50,000 so that you
45
42.8 have 500 both becomes much more expensive and can cre-
41 ate a bias. The 10 percent of people who respond are
40
unlikely to be representative. Perhaps only people who are
highly interested in the survey topic or who have a lot of
35
Black White free time (e.g., unemployed, retired, traditional homemak-
Respondents Respondents ers) respond. The opinions, education, income, age, and
Whites dislike Blacks first asked other characteristics of those who respond may not ade-
Blacks dislike Whites first asked
quately reflect the entire sample. It might seriously distort
your data.
Another limitation is that you do not control the con-
ditions under which a person completes a mail question-
naire. A questionnaire completed during a drinking party
6.5:  The Advantages and by a dozen laughing people may be returned along with
one filled out by an earnest respondent. With mail ques-
Disadvantages of Different tionnaires, no one is present to clarify questions or to
probe for more information when respondents give
Survey Formats incomplete answers. Someone other than the sampled
6.5 List the advantages and disadvantages of various respondent (e.g., spouse, new resident, etc.) may open the
survey formats mail and complete the questionnaire. Respondents can
complete the questionnaire weeks apart or answer ques-
tions in an order different from that intended by research-
ers. Incomplete questionnaires can also be a serious
problem.
A mail questionnaire format limits the kinds of ques-
tions that a researcher can use. Questions that require
visual aids (e.g., look at this picture and tell me what you
see), open-ended questions, many contingency questions,
and complex questions do poorly in mail questionnaires.
Likewise, mail questionnaires are ill suited for the illiterate
or semi-illiterate in a country’s main language.
132  Chapter 6
6.5.2:  Telephone Interviews
Advantages  The telephone interview is a popular
survey method because you can reach about 95 percent of
the population by telephone. You call a respondent, ask
questions, and record answers. The sample of respondents
can come from lists, telephone directories, or random-digit
dialing (RDD) and can be from a wide geographical area.
A staff of interviewers can interview 1,500 respondents
across a nation within a few days and, get response rates as
high as 85 percent with 7 to 10 callbacks. The telephone
interview is a flexible method and has most of the strengths
of face-to-face interviews. Interviewers pick a specific
respondent, control the sequence of questions, and can use
some probes. A specific respondent answers the questions
alone. Interviewers can use contingency questions effec-
tively, especially with computer-assisted telephone inter-
viewing (CATI) (discussed later in this chapter). Also,
supervisors can monitor interview quality by listening in.
Disadvantages  High cost and limited interview
length are major disadvantages of telephone interviews.
Respondents without telephones are impossible to reach,
or the call may come at an inconvenient time. The use of an
interviewer reduces anonymity and introduces potential
interviewer bias. Open-ended questions are difficult to use,
and questions that require visual aids are impossible. Inter-
viewers can only note serious disruptions (e.g., back-
ground noise) and respondent tone of voice (e.g., anger or
flippancy) or hesitancy. Over the past 30 years, respondent
cooperation with telephone interviews has steadily
declined, with more refusals and inability to reach people,
even with 10 callbacks at different times.
6.5.3:  Face-to-Face Interviews
Advantages  Face-to-face interviews have the highest
response rates and permit the longest questionnaires.
Interviewers also can observe the surroundings and can
use nonverbal communication and visual aids. Well-
trained interviewers can ask all types of questions, can ask
complex questions, and can use extensive probes.
Disadvantages  The training, travel, supervision, and
personnel costs for interviews can be very high. Interviewer
bias is also greatest in face-to-face interviews. The appear-
ance, tone of voice, question wording, and so forth of the
interviewer may affect the respondent. In addition, inter-
viewer supervision is less than for telephone ­interviews.
6.5.4:  Web Surveys
Advantages  Web surveys came into widespread use
over the past 10 years. They are now the lowest cost format
and get answers the fastest. Three other advantages are
that web surveys can span a large geographic area, include
visual materials as well as the survey questions, and use
elaborate contingency questions.
Disadvantages  A drawback with web surveys is that
not everyone has Internet access, especially low-income,
elderly, and less educated people. Low response rates and
incomplete surveys are also common problems with web
surveys. As incentives for web survey participation
improve, these issues may become less important. Another
disadvantage is a lack of control over the conditions under
which a person completes a web survey. Someone who is
not serious or someone other than the selected respondent
may complete the web survey. Also, as with mailed ques-
tionnaires, no one is present to clarify questions or to probe
for more information when respondents give incomplete
answers.
WRITING PROMPT
Different Survey Formats
Of the various types of survey formats, what type of survey format
or presentation encourages you to take the survey most seriously?
Describe what qualities in that type of survey encourage more
respondents to take a survey questionnaire seriously and give the
most truthful, thoughtful answers.
The response entered here will appear in the
performance dashboard and can be viewed by
your instructor.
Submit
6.6:  How Do You Interview
in Survey Research?
6.6 Apply best practices for preparing the interviewer
to administer social surveys
Interviews are a special type of social relationship in which
one person asks the questions while another person pro-
vides answers. There are several types of interview. There
are police interrogation interviews, job candidate selection
interviews, and celebrity entertainment interviews. In sur-
vey research interviews, the primary goal is to obtain accu-
rate information from another person. The survey
interview is a short-term social interaction between two
strangers with expectations, social roles, and norms. Its
explicit purpose is for one person to obtain specific infor-
mation from the other. The roles are those of the inter-
viewer and the interviewee or respondent. The interviewer
obtains information by asking prearranged questions and
recording responses given by the respondent.
You have several choices if you plan to collect survey data
with interviewers. Will you interview by phone or face-
to-face? Will you personally do the interviews or have
others conduct interviews for you?

If you use other people to conduct interview, you need
to train them with the questionnaire. Interviewers must
become very familiar with the wording and purpose of
each question and practice following the flow of question-
naire items. They will also need to exhibit proper survey
interview behavior.
6.6.1:  The Interviewer’s Role
The interviewer’s task is complex. He or she must control
the interview and the flow of interaction, obtain coopera-
tion and build rapport, and simultaneously remain neutral
and objective. Interviewers encroach on the respondents’
time and privacy for information that may not directly
benefit the respondents. They try to reduce embarrass-
ment, fear, and suspicion so that a respondent feels com-
fortable revealing information. A good interviewer
constantly monitors the pace and direction of the social
interaction as well as the answers and the behavior of
respondents. He or she helps respondents to feel that they
should give complete, truthful answers.
It is important to note that not all participants will be
familiar with their role as a survey respondent. A survey
interviewer may need to clarify the roles in the social rela-
tionship and follow them consistently or explain the nature
of survey research. Some respondents will view the
research interview as an intimate conversation similar to a
therapy session, a boring bureaucratic exercise in complet-
ing forms, a referendum on public policy choices, or a com-
petitive testing situation. Some suspicious people even
treat it as a contest in which interviewers try to trick or
entrap respondents. A misunderstanding about the respon-
dent role can produce a misunderstanding of the meaning
of survey questions.
A survey interviewer is nonjudgmental and never
reveals his or her opinions, verbally or nonverbally (e.g.,
by a look of shock). If a respondent asks for an interview-
er’s opinion, the interviewer politely redirects the respon-
dent. For example, if a respondent asks “What do you
think?,” the interviewer may answer “Here we are inter-
ested in what you think; what I think doesn’t matter.” Like-
wise, if the respondent gives a shocking answer (e.g., “I
was arrested three times for beating my infant daughter
and burning her with cigarettes”), an interviewer should
not show shock, surprise, or disdain but treat the answer in
a matter-of-fact manner.
If the survey interviewer must be neutral and objective,
why not use a computer?
Compare Your Thoughts
Machine interviewing has many limitations. It lacks the human
warmth, sense of trust, motivation, and rapport that a human
interviewer creates. It cannot answer respondent questions
The Survey 133
or clarify misinterpretations. An interviewer helps to define
the situation for respondents, offers guidance and deter-
mines whether respondents have the information sought,
understands what is expected, and is providing relevant and
serious answers.
6.6.2:  The Interview Stages
A survey interview proceeds through three major stages:
introduction and entry, main part of the interview, and
the exit.
Stage 1: Introduction and Entry  The inter-
viewer gains access, provides authorization, and reassures
and secures cooperation from the respondent. He or she is
prepared for reactions, such as “How did you pick me?”
“What good will this do?” “I don’t know about this,”
“What’s this about?” The interviewer explains why the
specific respondent is being interviewed and not a substi-
tute. In this stage, the interviewer takes control, and estab-
lishes role expectations. Informed consent and assurances
of confidentiality occur in this stage.
Stage 2: The Main Part of the Interview  The
main part consists of asking questions and recording
answers. The interviewer follows the exact wording on the
questionnaire—no added or omitted words and no
rephrasing. In this stage, it is important to set a comfort-
able pace and give nondirective feedback to maintain
respondent interest.
Nondirective feedback might include a slight nod or
smile, making eye contact, or quietly saying “OK.”
He or she asks all applicable questions in order, with-
out returning to or skipping questions unless the directions
specify this. Interviewers need to be aware that respon-
dents often reinterpret survey questions to apply to their
personal situations or to make them easy to answer.
In addition to asking questions, the interviewer accu-
rately records all answers. This is relatively easy for closed-
ended questions, where interviewers just mark the correct
box. For open-ended questions, the interviewer’s job is
more difficult. An interviewer listens carefully, has legible
writing or good recording machine skills, and records what
a respondent says verbatim without correcting grammar or
slang. The interviewer does not summarize or paraphrase
because doing so leads to lost information or distorted
answers. For example, the respondent says, “I’m really
concerned about my daughter’s heart problem. She is only
10 years old and already she has trouble climbing stairs. I
don’t know what she’ll do when she gets older. Heart sur-
gery is too risky for her and it costs so much. I guess she’ll
have to learn to live with it.” If the interviewer summa-
rizes, “respondent is concerned about daughter’s health,”
much is lost.
134  Chapter 6
Stage 3: The Exit  In this last stage, the interviewer
thanks the respondent and leaves. He or she then goes to a
quiet, private place to edit the questionnaire. During this
time, other details like the date, time, and place of the inter-
view; a thumbnail sketch of the respondent and interview
situation; the respondent’s attitude (e.g., serious, angry, or
flippant); and any unusual circumstances (e.g., “Telephone
rang at question 27 and respondent talked for four minutes
before the interview started again”) should be recorded.
The interviewer will also note anything disruptive that
happened during the interview (e.g., “Teenage son entered
room, sat at opposite end, turned on television with the
volume loud, and watched a music video”). The inter-
viewer also records his or her personal feelings and any-
thing that was suspected (e.g., “Respondent became
nervous and fidgeted, changed answers once each on ques-
tions 14, 15, and 16 about his marriage”).
6.6.3:  Training Interviewers
Perhaps someday you may get a job interviewing for a
professional survey organization. A large-scale survey
employs many interviewers. Few people other than pro-
fessional survey researchers appreciate the difficulty of
the interviewer’s job. A professional-quality interview
requires a carefully selected and trained interviewer. As
with any employment situation, interviewers need ade-
quate pay and good supervision to perform consistently at
their peak. Good interviewers are pleasant, honest, accu-
rate, mature, responsible, intelligent, stable, and moti-
vated. They are patient and calm. They have experience
with many types of people and possess poise and tact.
Face-to-face interviewers must have a nonthreatening
appearance. If the survey involves face-to-face interview-
ing in high-crime areas, the interviewers need to have
proper “street smarts” and may require extra protection
(e.g., a partner or ­assistant).
When hiring interviewers, researchers consider the
candidate’s physical appearance, age, race, sex, languages
spoken, and even the sound of their voice. Most training
for professional interviewers requires full-time sessions
lasting one to two weeks. It usually includes lectures and
reading, observation of expert interviewers, mock
i­nterviews in the office and in the field that are recorded
and critiqued, many practice interviews, and role playing.
Interviewers are taught about survey research and the role
of the interviewer. They must become very familiar with
the questionnaire and the purpose of questions.
6.6.4:  Using Probes in Interviews
A good interviewer knows how and when to use probes.
Probes clarify a respondent’s ambiguous or irrelevant
response and confirm that respondents understand the
questions as intended. This means interviewers need to
understand the survey and recognize an irrelevant or inac-
curate answer. There are many types of probes. A three- to
five-second pause is often effective, as is nonverbal com-
munication (e.g., tilt of head, raised eyebrows, or eye con-
tact). The interviewer can repeat the question or the reply
and then pause.
He or she can ask a neutral question, such as:
“Any other reasons?”
“Can you tell me more about that?”
“How do you mean?”
“Could you explain more for me?”
Making It Practical: Using Probes
Interviewer Question: What is your occupation?
Respondent Answer: I work at General Motors.
Probe: What is your job at General Motors? What type
of work do you do there?
Interviewer Question: How long have you been
unemployed?
Respondent Answer: A long time.
Probe: Could you tell me more specifically when your
current period of unemployment began?
Interviewer Question: Considering the country as a
whole, do you think we will have good times during
the next year, or bad times, or what?
Respondent Answer: Maybe good, maybe bad, it
depends, who knows?
Probe: What do you expect to happen?
Record Response to a Closed Question
Interviewer Question: On a scale of 1 to 7, how do you
feel about capital punishment or the death penalty,
where 1 is strongly in favor of the death penalty, and 7
is strongly opposed to it?
(Favor) 1 ___ 2 ___ 3 ___ 4 ___ 5 ___ 6 ___ 7 ___
(Oppose)
Respondent Answer: About a 4 or 5. I think that all mur-
derers, rapists, and violent criminals should get death,
but I don’t favor it for minor crimes like stealing a car.
Interviewer: Would that be a 4 or 5 on this rating from
1 to 7?

Probes are not substitutes for writing clear ques-
tions or creating a framework of understanding for the
respondent. Unless carefully stated, probes might shape
the respondent’s answers. Yet flexible or conversational
interviewing, in which interviewers use many probes,
can improve accuracy on questions about complex
issues for which respondents do not clearly understand
basic terms or about which they have difficulty express-
ing their thoughts. For example, to the question, “Did
you do any work for money last week?,” a respondent
might hesitate and then reply, “Yes.” An interviewer can
probe, “Could you tell me exactly what work you did?”
The respondent may reply, “On Tuesday and Wednes-
day, I  spent each afternoon helping my buddy John
move into his new apartment. For that he gave me $50,
but I didn’t have any other job or get paid for doing any-
thing else.” If the question intended only to get reports
of regular paid employment, the probe revealed a
­misunderstanding.
WRITING PROMPT
Probes in Interviews
Does using many probes impede the goal of standardizations (i.e.,
each respondent hears the exact same survey question, so we
know all answers from respondents are to the same question)?
Explain your answer.
The response entered here will appear in the
performance dashboard and can be viewed by
your instructor.
Submit
6.6.5:  Interviewer Bias
Survey researchers prescribe interviewer behavior to
reduce bias. Bias is when a particular interviewer ’s
actions have influenced how a respondent answers, and
responses differ from what they would be if another
interviewer had asked the questions. Proper interviewer
behavior and exact question reading are critical, but there
is a larger issue.
An interviewer’s uncontrolled visible characteris-
tics, including race, stature, age and gender, often affect
interviews and respondent answers. This means noting
the characteristics of both interviewers and respondents,
especially in questions about issues related to visible
characteristics. For example, African American and His-
panic American respondents tend to express different
policy positions on race- or ethnic-related issues depend-
ing on the apparent race or ethnicity of the interviewer.
This occurs even with telephone interviews when a
respondent has clues about the interviewer’s race or
ethnicity.
The Survey 135
Gender also affects interviews in terms of both obvi-
ous issues, such as sexual behavior, and support for
­gender-related collective action or gender equality. In gen-
eral, interviewers of the same gender or ethnic-racial
group as the respondent tend to get the most accurate
answers.
Example Study
Interviewer Race Effects
Are Subtle and Pervasive
Survey researchers have been long aware that an interviewer’s
race can influence how respondents answer racially sensitive
questions. Other research documented that women or racial
minorities tend do poorly on certain tests administered by
members of outside groups (such as white males) because of
a stereotype that they will do poorly. They feel great pressure
to disconfirm the negative stereotype, but the pressure creates
test anxiety that lowers their test score. In contrast, when a
member of their same group administers the test, the stereo-
type threat is not activated, test anxiety is reduced, and they
score higher.
To test whether the stereotype-triggered test anxiety also
operates in survey interviews, Davis and Silver (2003) conducted
a telephone survey of whites and African Americans in the
Detroit area. They wanted to see whether a survey interviewer’s
race affected how a respondent answered. The research ques-
tion was, “Do African Americans score differently on survey
knowledge questions depending on whether they think the tele-
phone interviewer is white or African American?” Results showed
that white respondents answered the same irrespective of inter-
viewer’s race. However, ­African American respondents scored
higher on the knowledge questions when they believed their
interviewer was an African American. The authors concluded
that beyond widely known social desirability and racial confor-
mity issues in surveys, the race of an interviewer can create
subtle anxiety based on negative stereotypes that influence how
a respondent answers many survey questions.
6.6.6:  Computer-Assisted Telephone
Interviewing
Most professional survey organizations that do phone
interviewing have installed computer-assisted telephone
interviewing (CATI) systems. With CATI, the interviewer
sits in front of a computer and makes calls. Wearing a
136  Chapter 6
headset and microphone, the interviewer reads the
­questions from a computer screen for the specific respon-
dent, then enters the answer via the keyboard. Once he or
she enters an answer, the computer shows the next ques-
tion on the screen. CATI speeds interviewing and reduces
interviewer errors. It also eliminates the separate step of
entering information into a computer and speeds up data
processing. The CATI system works well for contingency
questions because the computer shows the questions
appropriate for a specific respondent; interviewers do not
have to look for the next appropriate question. In addition,
the computer can check an answer immediately after the
interviewer enters it. For example, the interviewer enters
an answer that is impossible or clearly an error (e.g., an H
instead of an M for “Male” or F for “Female”). The com-
puter catches the error immediately.
6.7:  How Can You Be
Ethical in Survey Research?
6.7 Analyze some of the ethical issues in conducting
survey research
There are ethical and unethical ways to conduct surveys.
A major ethical issue in survey research is the invasion of
privacy. People have a right to privacy, but asking about
intimate actions and personal beliefs could violate a per-
son’s privacy. You need to be careful and ask questions in
a polite, respective manner, and to protect the informa-
tion you obtain. Respondents are most likely to provide
accurate information when asked for it in a comfortable
context with mutual trust, when they believe that serious
answers are required from them for a legitimate purpose,
and if they believe that their answers will remain confi-
dential. This requires treating all respondents with dig-
nity and reducing any anxiety or discomfort. The
researcher is responsible for protecting the confidentiality
of data.
A second ethical issue involves voluntary participa-
tion by respondents. You can never force anyone to answer
a survey question and must obtain informed consent.
Respondents can withhold information or refuse to partici-
pate at any time. Because you depend on voluntary coop-
eration, you need to ask well-developed questions in a
sensitive way, treat respondents with respect, and keep
what you learn confidential.
A third ethical issue is the misuse of surveys. Because
surveys are so common, some people use the survey for-
mat for illegitimate purposes. When someone uses a sur-
vey format to attempt to persuade others to do something,
it is called a pseudosurvey. A charlatan may have no interest
in learning information from a respondent but only use a
survey to gain entry into homes, persuade a person to vote
in a certain way, or try to sell something in the guise of a
survey.
The mass media report more on surveys than other
types of social research; however, the way mass media
report on surveys permits abuse. Few people who read
poll results in a newspaper or hear them on television real-
ize that ethical codes require certain details about the sur-
vey method. The purpose of providing details on method
is to reduce misuse of survey research. Researchers urge
the media to include the information, but it is rarely
included. In addition to omitting critical details on how a
survey was conducted, the media often report on weak,
biased, and misleading surveys along with sound, rigor-
ous, professional ones without any distinction. This only
increases public confusion and a distrust of all surveys.
Making It Practical: Ten Items to Include
When Reporting Survey Research  Below are
10 items that national public opinion organizations rec-
ommend to include when reporting any poll or survey:
1. The sampling frame used (e.g., telephone directories)
2. The dates on which the survey was conducted
3. The population that the sample represents (e.g., U.S.
adults, Australian college students, housewives in
Singapore)
4. The size of the sample for which information was col-
lected
5. The sampling method (e.g., random)
6. The exact wording of the questions asked
7. The method of the survey (e.g., face to face, t­ elephone)
8. The organizations that sponsored the survey (paid for
it and conducted it)
9. The response rate or percentage of those contacted
who actually completed the questionnaire
10. Any missing information or “don’t know” responses
when results on specific questions are reported
WRITING PROMPT
Incomplete Reporting of Survey Research
For many years, national public opinion organizations outlined what
reports of survey findings should include so that the people would
be able to determine whether the survey was of poor quality and
unreliable. What three aspects of how a survey was conducted, if
omitted, would mislead people the most?
The response entered here will appear in the
performance dashboard and can be viewed by
your instructor.
Submit
Summary: What You Learned about Surveys
In this chapter, you learned about survey research. It is a
process of asking many people the same questions and
examining their answers. You also learned some principles
of writing good survey questions. There are many things to
avoid and to include when writing questions. You learned
about the advantages and disadvantages of mail and web
surveys, and of telephone and face-to-face interviews. You
saw that interviewing, especially face-to-face interviewing,
can be difficult. Although this chapter focused on survey
research, researchers use questionnaires to measure varia-
bles in other types of quantitative research (e.g., experi-
ments). The survey is often called the sample survey
because most surveys involve random sampling.
Survey researchers try to minimize errors, but survey
data often contain them. Errors in surveys can compound
each other. For example, errors can arise in sampling frames,
from people not answering survey questions, from question
wording or order, and from interviewer bias. Do not let the
existence of errors discourage you from using the survey.
Instead, use care when designing or conducting surveys and
exercise caution when generalizing from survey results.
Quick Review
What Is a Social Survey?
1. Survey data come from self-reports. To measure variables,
you must convert them into survey questions that people
are able and willing to answer.
2. It is possible to test multiple hypotheses at the same time
in one survey.
3. In survey research, control variables measure variables
that might be alternative explanations in a causal
­relationship.
4. In survey research, you often must use logic to meet the
requirement of time order of variables in a causal ­relationship.
How Do We Conduct a Survey?
1. You should tailor survey questions to the type of respon-
dent who is in the survey and to the format of the survey.
2. You need to very clear and explicit about the type of infor-
mation you request in each question.
3. You should create a system to number each respondent’s
questionnaire and way to record responses during survey
design.
4. It is always best to pilot test a questionnaire before using it
in the research project.
Writing Good Survey Questions
1. Avoid leading questions in surveys, i.e., questions that
push respondents toward specific answers.
2. Also avoid vague questions, double-barreled questions,
and keep the language of questions simple.
3. Be aware of prestige bias and false premises in survey
questions, and ask about separate variables of a hypoth-
esis, not whether respondents accept the hypothesis
you are testing.
4. Weigh the advantages and disadvantages of open-
ended versus closed-ended response formats when
designing a survey.
5. Keep the answer choices mutually exclusive, exhaustive,
balanced, and provide adequate number (over three but
less than nine) of choices.
6. Phrase recall questions in ways that make it easy for
respondents to remember and phrase sensitive ques-
tions in ways to increase honest, accurate answers.
7. Offer a “do not know” or “no opinion” choice most of the
time and use a full-filter question when asking about top-
ics about which few respondents may be informed.
8. Be aware of social desirability bias and avoid it when
possible.
9. Use contingency questions to increase the relevance of
questions for specific respondents.
10. Be sensitive to the emotional impact of specific words or
phrases.
Shared Writing: What To Do When
You Get a Bad Survey
Give an example of a survey question that you considered to be
highly biased or leading. How would you respond?
Choose two of your classmates’ examples and provide
advice that you would give to people when they receive a ques-
tionnaire or hear an interviewer ask this question?
A minimum number of characters is required to
post and earn points. After posting, your
response can be viewed by your class and
instructor, and you can participate in the
class discussion.
Post 0 characters | 140 minimum
137
Chapter 7
The Experiment
Learning Objectives
7.1 Identify how experiments are the strongest
type of social research for testing cause–
effect relationships
7.2 Report the need to make valid comparisons
in experiments
7.3 Analyze the function of independent and
dependent variables as used in
experimental research
In terms of politics, many observers like journalists, policy
makers, informed-educated citizens, and researchers in the
United States and in other countries see Americans as a
deeply polarized society. Recent news stories report solid
red states versus blue states, refusals to compromise, and
political gridlock. Poll data show that divisions by political
138
7.4 Describe combining and managing as the
two main tasks in experimental designs
7.5 Report some of the issues that might
weaken internal and external validity
7.6 Identify the importance of comparisons in
experimental research
7.7 Analyze why ethical concerns often arise
in experimental research
party in the 2008 and 2012 elections were sharper than any
time in the past 40 years. Others claim that most ordinary
people are not polarized and share many common beliefs.
What influences perceptions of political polarization? Why
do some perceive sharp partisan divides whereas others do
not? Van Boven, Judd, and Sherman (2012) thought that

people who were intensely committed to a particular point
of view and held extreme partisan attitudes might be pro-
jecting their own polarization onto others. In short, not
everyone is polarized; rather the people who feel most
polarized see others in highly polarized terms.
The researchers identified three social psychological
processes to account for this situation:
1. a general tendency to categorize others
2. a shift from categorizing to a dividing of people be-
tween in-groups (“the people like me”) and out-groups
(“others”)
3. “naïve realism.” Naïve realism is a tendency people
have to see their own attitudes on partisan issues as un-
biased and rational, and the attitudes of people in out-
groups as influenced by bias, self-interest, and ­ideology.
Essentially, polarized people assume that they are not
biased but that others are. The researchers called these com-
bined processes polarization projection. They created a social
experiment to test polarized projection using a fictional
political issue that had the essential features of political par-
tisan conflict: well-defined groups with polarized attitudes
toward a policy concerning the allocation of scarce
resources. They told the experimental participants, under-
graduate students at a large public university, that the uni-
versity administration was seriously considering adoption
of a fictional Nonresident Attraction and ­Retention Pro-
gram (NARP). The NARP would give nonresident, out-of-
state students (who pay higher tuition than in-state
students) privileged benefits, such as priority access to
desirable dorms, priority class registration, and free print-
ing. The researchers thought resident and nonresident stu-
dents would develop partisan attitudes toward NARP.
In the experiment, 101 undergraduate students (41
men and 60 women) at the University of Colorado-Boulder
first completed a questionnaire on the NARP policy using
a 5-point Likert Scale. It asked questions such as what stu-
dents liked or disliked about the policy and whether it vio-
lated their sense of fairness. The researchers next randomly
assigned participants into two conditions, either the intro-
spection group or the control group. They gave the intro-
spection group time and cues to ponder why they held
their views before they were asked to estimate the distribu-
tion of all other students’ attitudes on campus. The control
group students just went to estimation.
The researchers asked participants in the introspection
condition to describe their reasoning processes with ques-
tions like: What factors, different thought processes, and
experiences might have caused you to hold your stance?
Did you consider if the proposal was in your self-interest?
Did you engage in careful and extensive thought? After-
wards, the introspection participants estimated the distri-
bution of other students’ attitudes toward NARP.
The Experiment 139
The results revealed that the participants who had
introspected about their attitudinal processes showed
more polarization projection than did participants in the
control condition. The study demonstrated the most
highly committed people overestimate the polarization
among others. When polarized people are stimulated to
think about why they hold their views (i.e., naïve realism),
they become even more polarized and see people who
hold an opposing position as stereotypic caricatures. This
especially happens when they believe they have exercised
careful reasoning and believe those holding an opposing
position engaged in biased, self-­interested ­reasoning.
7.1:  When Are Experiments
Most Useful?
7.1 Identify how experiments are the strongest type of
social research for testing cause-effect relationships
Compared to the other social research techniques, experi-
ments provide the strongest means for testing causal rela-
tionships. In many ways, a research experiment is an
extension of commonsense logic. The key difference is that
everyday experiments are less careful and systematic than
scientific ones. In commonsense language, an experiment
refers to two situations:
Before-and-After Comparison  You modify some-
thing and then compare an outcome to what existed prior
to the modification. For example, one morning, you try to
start your car. To your surprise, it does not start. You
“experiment” by cleaning off the battery connections and
then try to start it again. You modified something (cleaned
the connections) and compared the outcome (whether the
car started) to the prior situation (it did not start). Your
“hypothesis” was that the car did not start because of a
buildup of crud on the connections, and once you removed
the crud, the car would start.
Side-by-Side Comparison  You have two similar
things, and then you modify one but not the other and
compare the two. For example, you watch a young boy
playing with a soft drink can. He vigorously shakes it. You
hold a can that is not shaken. You then open both cans for
him. He laughs when the shaken can “explodes” with a
fizzy mess, but the one you held does not explode. You
began with two similar things (soft drink cans) and modi-
fied one (shook up a can) but not the other. Before you
opened the cans, you hypothesized that the shaken can but
not the other one would make a fizzy mess.
Compared to other social research methods, experi-
ments best satisfy the three conditions needed to demon-
strate causality—temporal order, association, and no
140  Chapter 7
alternative explanations. In order to demonstrate causality
in an experiment, we follow three basic steps:
1. Start with a cause–effect hypothesis.
2. Modify a situation or introduce a change.
3. Compare outcomes with and without the modification.
7.1.1:  Questions You Can Answer
with the Experimental Method
Experimental research has a clear, simple logic that offers a
very powerful way to focus narrowly and demonstrate
causal relations among a few variables in controlled situa-
tions. Research questions most appropriate for an experi-
ment fit its strengths and limitations that include the
following features:
• It can isolate and identify a causal mechanism.
• It targets two or three specific variables, and it is nar-
row in scope.
• It is limited by the practical and ethical aspects of the
situations you can impose on humans.
In general, the social experiment is better for targeted
micro-level concerns (e.g., individual or small-group phe-
nomena) than for complex macro-level issues containing
many factors that operate together. In other words, they
are excellent when you can isolate few variables in a con-
trolled, small-scale setting but are not well suited for ques-
tions involving many diverse influences that operate across
an entire society or over decades. For example, to track
changes in public attitudes about same-sex marriage across
the entire society over the past 20 years, a survey would be
much better than an experiment. We can, however, provide
insights into larger issues when we integrate and synthe-
size the results from many narrowly focused experiments.
WRITING PROMPT
Limitations of Experiments
Give an example where you could conduct a survey and experiment
on the same topic. What could you learn from a survey that you
could not learn from an experiment?
The response entered here will appear in the
performance dashboard and can be viewed by
your instructor.
Submit
7.1.2:  Limitations of Experimental
Research
Ethical and practical constraints limit what you can study
with the experimental method. Let us say you want to know
whether biracial children are more likely to develop an inter-
est in a career in human services than single-race children
are. You can study this issue using a survey or existing sta-
tistics research, but not the experimental method. For a true
experiment, you would force some parents to have biracial
children and not others, force all the parents to raise their
children similarly, and then wait until the children grow up
to examine their career choices. This is both unethical and
impractical. Despite its great strength at demonstrating
causal relations, the experiment is limited in the questions
you can ask, the variables you can measure, and your ability
to generalize from one experiment to larger society.
The ideal solution is to study an issue using both
experimental and a nonexperimental methods and then
combine what you learn from both. Maybe you are inter-
ested in attitudes toward people in wheelchairs. You could
conduct an experiment asking participants to respond to
photos of people in wheelchairs and not in wheelchairs,
with the different research participants seeing the same
person in or not in a wheelchair. You could ask them sur-
vey questions like the following: Would you hire this per-
son? How comfortable would you be if this person asked
you for a date? You could also conduct a social survey on
attitudes about disability issues. In addition, you could
conduct field research and observe people’s reactions to a
person in a wheelchair in a natural setting, or you might be
in a wheelchair and carefully note the reactions of others to
you. Our greatest confidence comes when many well-
designed studies conducted by different researchers yield
similar results.
WRITING PROMPT
True Experiments
A true experiment often requires manipulating situations or aspects
of people lives but within practical and ethical limits. What practical
or ethical limits can you see restricting your ability to carry out true
experiment on social situations or relations?
The response entered here will appear in the
performance dashboard and can be viewed by
your instructor.
Submit
7.2:  Why Assign People
Randomly?
7.2 Report the need to make valid comparisons in
experiments
The cliché “compare apples to apples, don’t compare apples
to oranges” is not about fruit; it is about making valid com-
parisons. In experimental social research, what you want to
compare must be fundamentally alike. You can facilitate
comparison by creating similar groups of research partici-
pants. To make a valid comparison, you want to compare

participants who do not differ with regard to variables that
could be alternative explanations to your hypothesis. Let us
say you want to learn whether completing a college course
affects a person’s skill level. You have two groups of partici-
pants: one group completed the course whereas the other
did not. To make a valid comparison, you want participants
in the groups to be similar in every respect, except the one
you are examining in the study—taking the course.
Why is it important to have participants in the groups
to be similar in every respect?
Compare Your Thoughts
Let’s imagine that participants who completed the course
are also two years older than those who did not. You could
not know whether it was completing the course or being
older that accounted for group differences in skill levels.
7.2.1:  Random Assignment
and Random Sampling
Many experiments use random assignment, also called
randomization. To compare groups of participants, you do
not assign them based on your feelings, their personal
preference, or their specific features. Random assignment
is unbiased because your desire to confirm a hypothesis or
a participant’s personal interests is removed from the
­selection process. It is random in a statistical or mathemati-
cal sense, not in the everyday sense of unplanned, haphaz-
ard, or accidental. In probability theory, random is a process
Figure 7.1  Random Assignment and Random Sampling
Random Sampling
Population (Sampling Frame)
Random Assignment
Step 1: Begin with a collection of subjects.
Step 2: Devise a method to randomize that is purely mechanical (e.g., flip a coin).
Step 3: Assign subjects with “Heads” to one group
Control Group
The Experiment 141
in which each case has a known and equal chance of selec-
tion. It obeys mathematical laws, which make precise cal-
culations possible.
Both random sampling and random assignment use a
mathematically random process. When you randomly
sample, you use a random process to select a smaller sub-
set of people (sample) from a larger pool (population).
When you randomly assign, you use a random process to
sort a collection of participants into two or more groups.
You can both randomly sample and randomly assign. You
can first sample to obtain a set of participants (e.g., 150
­students out of 15,000), then randomly assign to divide the
sampled participants into groups (e.g., divide the 150
­people into 3 groups of 50 each). Combining random sam-
pling and random assignment is the ideal; however,
because of the extra effort required, it is rare in social
experiments (see Figure 7.1).
If the purpose of random assignment is to create two
(or more) equivalent groups, you may ask whether it might
be simpler to match participants’ characteristics in each
group. Professional researchers occasionally match partici-
pants on certain characteristics, such as age and sex, but
this is infrequent. True equal matching falls apart as the
number of relevant characteristics expand, and soon find-
ing exact matches is impossible. Individuals differ in thou-
sands of ways. You cannot know which traits are relevant
and influence your variables. Let us say you compare two
groups of 15 students. Group 1 has eight males. To match,
group 2 must also have eight males. Two males in group 1
are only children; the rest have one or more siblings. Three
Sample
Random
Process
and “Tails” to the other group.
Experimental Group
142  Chapter 7
are from a divorced family. One male is tall, slender, and
Jewish; another is short, heavy, and Methodist; and so
forth. To match groups, you have to find a tall Jewish male
only child from a divorced home and a short Methodist
male only child from an intact home. True matching on
more than two characteristics soon becomes impossible.
For this reason, randomization is preferred. With a true
random process, over the long run, the odds are that the
groups will be equal. If the groups are equal, the experi-
ment’s internal logic is stronger.
Making It Practical: How Do You Conduct
Random Assignment?
Random assignment is simple in practice. Start with a collec-
tion of people, such as 50 volunteers who show up to par-
ticipate in a study. Next, divide them into two or more
equal-sized groups using a true random process, such as
tossing a coin or throwing dice. If you toss a coin, you may
assign all for whom heads appear in one group and the rest
in another. You will probably have about 25 participants in
each group. What if you have 80 people and want to assign
them to four groups? You can use coin toss (every other toss
goes to a group), dice, or another pure random process, such
as a randomizing software application. The key feature of
the process is that all have an equal, one-in-four chance, of
ending up in a group. Nothing about a specific participant
or an experimenter’s preference affects who goes to which
group. It is entirely due to pure mathematical chance.
WRITING PROMPT
Random Assignment or Matching
Imagine that you have 100 volunteers to be participants in an experi-
ment, and you want to randomly assign them into four groups of
equal size (i.e., 25 each). How would you randomly assign the 100
into four groups? Be very specific in explaining what you would do.
The response entered here will appear in the
performance dashboard and can be viewed by
your instructor.
Submit
7.3:  How Do We Use
Variables in an Experiment?
7.3 Analyze the function of independent and dependent
variables as used in experimental research
Experimental research has its own terms, concepts, and
logic.
In the experimental context, the terms independent variable
and dependent variable have specific meanings. In experi-
ments, the independent variable is a condition or situa-
tion you modify or alter. For this reason, we also call it the
treatment, manipulation, stimulus, or intervention.
Experimental researchers strive to create realistic condi-
tions to produce specific reactions and feelings within the
experimental participants. For the independent variable,
an experimenter often manipulates what different sets of
participants see or think is happening. He or she may
give them different instructions, show them different situ-
ations using visual images or elaborate equipment, use
different physical settings, or stage different contrived
social situations.
Example Study
Was It a Gun or a Tool?
In many situations, police officers must respond quickly and
accurately to a potentially dangerous situation, such as trying
to determine if an individual is holding a gun. In 2014, national
outrage erupted when a white police officer shot an unarmed
black teen in Ferguson, Missouri. A few weeks later, a white
police officer shot and killed a 12-year-old black boy in a park
holding a toy gun just 2 seconds after pulling up in his squad
car. Earlier, in 1999, four white New York City police officers
shot and killed an unarmed immigrant from West Africa who
was holding out his wallet which the police mistakenly thought
was a gun.
Payne (2001) created two experiments to learn whether
racial stereotypes about “dangerous people” interferes with the
person’s ability to make accurate split-second judgments.
Payne built on past studies of priming that found people link

visual and other images to preexisting negative stereotypes.
The images prime or activate a negative response, often auto-
matically and unconsciously, in people who hold stereotypes.
If revealing a negative racial stereotype is socially inappropriate,
people will try to control or hide its priming effects. However,
hiding the expression of a stereotype slows a person’s deci-
sion making. For a short time period (perhaps several sec-
onds), people must reconsider their public responses in order
to make them more socially acceptable.
In the experiment, Payne had 31 white undergraduate
students complete an attitude survey. Next, they looked at
visual images on a computer. The experimenter told partici-
pants he was measuring their speed and accuracy of identify-
ing visual images. Participants practiced using the equipment
and classifying 48 photos. Payne then told them that they
would see a pair of photos. The first one was a warning that
the second would soon appear. The first photo was always the
face of either a white or a black man. The second photo was of
a handgun or a tool (hand drill or socket wrench). The first
photo appeared for 200 milliseconds; the second photo
appeared for 200 milliseconds. After a participant responded,
the screen went blank. Participants did not have a fixed time by
which they had to respond. After the participants gave a
response, the next pair of photos appeared. Participants iden-
tified 192 photos with the race of the first photo and tool or gun
randomly mixed. Thus, Payne randomly mixed what each par-
ticipant saw instead of randomly assigning the participants.
In a second experiment, everything was the same except
that Payne added pressure. If participants failed to respond
within 500 milliseconds after the second photo, they saw a dra-
matic red warning and had 1 second to respond. In the first
experiment, the participants showed no differences in accuracy.
In the second experiment, with time pressure, the error rate was
much higher. Many of the participants who first saw the black
man’s face mistook a hand tool for a gun. However, their errors
of seeing a tool as being a handgun did not increase after they
saw a white man’s face. The participants who made the most
errors, i.e., mistaking a tool for a gun, were participants who most
strongly accepted negative stereotypes about African Americans
based on answers to the survey at the start of the study.
Analysis of Was It a Gun or a Tool?
In the experiment described in Example Study: Was It a
Gun or a Tool?, the independent variable is different prim-
ing that the experimenter created by showing participants
either a white man or black man’s face.
Experimenters measure dependent variables in many
ways, including response times, percent accurate scores,
social behaviors, attitudes, feelings, and beliefs. In the
experiment described in the Example Study, the dependent
variable was participant accuracy about whether a second
photo showed a gun or a tool. You can measure dependent
variables by paper-and-pencil indicators, such as a
­questionnaire, by interviews, by observing behaviors
(making a choice or response time), or by physiological
responses (e.g., heartbeat or sweating palms).
The Experiment 143
In many experimental designs, you measure the
dependent variable more than once, both before introduc-
ing the independent variable, in a pretest, and again after
introducing it, in a posttest.
When introducing an independent variable, researchers
can use two or more groups that experience different situa-
tions or a single participant group that experiences different
situations one after the other. If there are two or more groups,
it is an independent group design. In the study that opened
this chapter, the independent variable was to have one
group of participants “introspect” and seriously consider
their reasoning for holding a partisan position, whereas the
control group did not. The group that introspected was the
experimental group. When the independent variable has
many different values, you may have more than one experi-
mental group, called comparison groups, one for each level
of the independent variable. At other times, experimenters
use a repeated measures design. There is one group and the
same participants experience multiple situations over time
as was the case in the study about viewing images of a black
or white face then a gun or tool.
7.3.1:  Planning an Experiment
To plan an experiment, you must decide on a specific experi-
mental design and plan what participants will experience
from beginning to end. Planning includes decisions about
the number of groups, how and when to create independent
variable conditions, and how often to measure the depen-
dent variable. Overall, you need to plan and configure seven
parts of a design. Not all experiments have all seven parts,
but a few experimental designs have more than these seven.
1. Independent variable
2. Dependent variable
3. Pretest
4. Posttest
5. Experimental group
6. Control group
7. Random assignment
You also develop measures of the dependent variable
and pilot test the experiment.
Summary Review
Steps in Conducting
an Experiment
1. Begin with a straightforward hypothesis that is ­appropriate
for experimental research.
2. Decide on an experimental design that will test the
hypothesis within practical limitations.
144  Chapter 7
3. Decide how to introduce the independent variable or cre-
ate situations to induce it.
4. Develop a valid and reliable measure of the dependent
variable.
5. Set up an experimental setting and conduct a pilot test of
the variables.
6. Locate appropriate participants.
7. Randomly assign participants to groups and give them
instructions.
8. Gather data for the pretest measure of the dependent
variable for all groups.
9. Introduce the independent variable to the experimental
group only (or to relevant groups if there are multiple
experimental groups) and monitor all groups.
10. Gather data for posttest measure of the dependent
­variable.
11. Debrief the participants. This means you ask partici-
pants what they thought was occurring and reveal the
true purpose and situation you deceived them about in
any aspect of the experiment.
12. Examine data collected and make comparisons between
different groups using statistics and graphs to determine
whether the data support the hypothesis.
7.4:  How Do We Combine
Parts into Experimental
Designs?
7.4 Describe combining and managing as the two
main tasks in experimental designs
Designing an experiment involves two tasks:
1. Combining several parts into a specific configuration
2. Managing the overall experiment
The logic of experimental research requires that you
carefully manage all aspects of the experimental setting. In
order to accomplish this, you must isolate the effects of the
independent variable and eliminate alternative explana-
tions. Any aspects of an experimental situation that you do
not control might become an alternative explanation for
changes in the dependent variable. Alternative explana-
tions weaken your ability to show a causal connection
between the independent and dependent variables.
One way that experimenters control the experimental
setting is with deception. Carefully applied, this can help
to prevent participants from altering their behavior or
opinions to fit the researcher’s hypothesis. By focusing
attention on a false topic, the unaware participants will act
more “natural” with regard to the variables of interest.
Experimenters have deceived participants by intentionally
misleading them with written or verbal instructions, by
using the actions of helpers or confederates, or by arrang-
ing a setting. Experimenters also invent false dependent
variable measures to keep the true ones hidden. Obviously,
using deception raises ethical concerns.
7.4.1:  Types of Experimental Design
To plan an experiment, you combine the parts of an experi-
ment (e.g., pretests, control groups, etc.) into an experi-
mental design. Experimental designs vary in their
components: Some do not use random assignment, some
lack pretests, some do not have control groups, and others
have several experimental groups. In research reports,
experimenters name widely used designs. To understand
experimental design, it is helpful to learn the standard
designs that illustrate several ways to combine the parts of
an experiment.
We can illustrate the standard experimental designs by
looking at variations on this simple example: You want to
test whether servers (waiters and waitresses) receive big-
ger tips if they introduce themselves by first name before
taking an order and return 8 to 10 minutes after delivering
food to ask, “Is everything fine?” The independent variable
is server behavior. The dependent variable is the size of the
tip received.
We can divide standard designs into three groups:
• True experimental
• Pre-experimental
• Quasi-experimental designs
7.4.2:  True Experimental Designs
Classical Experimental Design  We will start
with the “gold standard” of the true experimental designs,
the classical experimental design. All other designs are
variations on it. Classical experimental design has ran-
dom assignment, a pretest and a posttest, an experimental
group, and a control group.
Example.  You give 40 newly hired servers a train-
ing session and instruct them to follow a script. They are
not to introduce themselves by first name or return dur-
ing the meal to check on the customers. You randomly
divide the participants into two equal groups of 20. You
record the average weekly tips for all participants for one
month (pretest score). Next, you retrain one group of 20
participants (the experimental group). You instruct them
henceforth to introduce themselves to customers by first
name when taking an order and to check on the custom-
ers 10 minutes after delivering food. They are to smile and
ask, “Is everything fine?” You instruct the other partici-
pants (control group) to continue without an introduction
or checking. Over a second month, you record average
weekly tips for both groups (posttest score).
 The Experiment 145

Figure 7.2  Example Using Classical Experimental Design

Month 1 Month 2
Group 1 Pretest Independent Variable Present Posttest
Serve food without (amount of tips) Self-introduction and return to (amount of tips)
Randomly assign introduction or checking check on customer
participants to
training sessions Group 2 Pretest Independent Variable Absent Posttest
Serve food without (amount of tips) Serve food without introduction (amount of tips)
introduction or checking or checking

Two-Group Posttest-Only Design  This design
has all the parts of the classical design except a pretest. It is
similar to the static group comparison, a type of two-group
pre-experimental design. The two-group pretest-only
design includes random assignment absent in the static
group comparison. Random assignment improves the
chance that the groups are equivalent, but without a pre-
test you cannot be as certain that the groups really began
equal on the dependent variable.
Example.  You randomly divide 40 newly hired serv-
ers into two groups and give all training. You instruct one
group not to introduce themselves by first name or to return
during the meal to check on the customers. You instruct
participants in the other group to introduce themselves to
customers by first name and to check on the customers 10
minutes after delivering food. You record average weekly
tips for both groups (posttest score) (see Figure 7.3).
Solomon Four-Group Design  It is possible that
your pretest measure sensitizes participants to your
­ ependent variable or improves their posttest score. Rich-
d
ard Solomon developed a design to address this issue by
combining the classical experimental design with the two-
group posttest-only design and randomly assigning par-
ticipants to one of four groups.
Example.  You randomly divide 80 newly hired serv-
ers into four groups and give them training. You instruct
participants in groups 1, 2, and 4 not to introduce them-
selves or to return during the meal to check on the cus-
tomers. You instruct participants in group 3 (experimental
group 2) to introduce themselves and to return during the
meal to check on the customers. During the first month,
you count average weekly tips for groups 1 and 2 only
(pretest). After the first month, you “retrain” group 1 par-
ticipants (experimental group 1) henceforth to introduce
themselves to customers by first name and to return dur-
ing the meal to check on the customers 10 minutes after
delivering food. All other groups continue as first in-
structed. During the second month, you record average
weekly tips for all groups (posttest) (see Figure 7.4).

Figure 7.3  Example with Two-Group Posttest-Only Experimental Design

Month 1 Month 2
Group 1 Independent Variable Present Posttest
Serve food without Self-introduction and return (amount of tips)
Randomly assign introduction or checking to check on customer
participants to
training sessions Group 2 Independent Variable Absent Posttest
Serve food without Continue to serve food without (amount of tips)
introduction or checking introduction or checking

Figure 7.4  Example with Solomon 4-Group Experimental Design

As you can see, groups 2 and 4 are identical, except that group 2 has the pretest. Groups 1 and 3 are also the same. Both use a self-­
introduction and checking on customers.

Month 1 Month 2
Randomly Group 1 Pretest Independent Variable Present Posttest
assign Serve food without (amount of tips) Self-introduction and return to (amount of tips)
participants to introduction or checking check on customer
training
Group 2 Pretest Independent Variable Absent Posttest
sessions
Serve food without (amount of tips) Continue to serve food without (amount of tips)
introduction or checking introduction or checking
Group 3 Independent Variable Present Posttest
Serve food with Continue self-introduction and (amount of tips)
introduction and checking return to check on customer
Group 4 Independent Variable Absent Posttest
Serve food without Continue to serve food without (amount of tips)
introduction and checking introduction or checking