 During the 2010 congressional elections and even now, Republicans repeatedly asserted

that the Bush/Obama fiscal stimulus did not work. Rather, it just built up trillions in
federal debt that future generations would have to pay off.

 For example, our newly elected local congressman, Bill Flores, advertised then and
recently said that joblessness increased by over seven million during the time the stimulus
was occurring. What we need is to allow the private sector to work by getting government
out of the economy.

 While these arguments undoubtedly paid political capital and may have sounded good to
those inclined to vote for him, the science behind these arguments is problematic. Why?

 There was no control. In order to be able to conclude anything at all about the success of
the stimulus, one would need to know what happened to a group which did NOT receive
the stimulus. We need a both a treatment and a control in order to make causal
explanations.
 Perhaps the stimulus did actually work, but that other factors were
functioning simultaneously to make the economy worse.
 Thus, as scientists we need to ask:
 What would the economy have been like without the stimulus?
 For those who did not receive the stimulus (an imaginary group), what
would have happened.
 This is the issue of control for rival explanations.
 The issue of control is fundamental to being able to make causal
explanations. Politicians and people live in a much simpler world and are
often willing to accept explanations offered by those they trust, especially if
they coincide with one’s predispositions.
 However, as scientists it is our obligation to flesh out the truth whenever it
is possible to do so, regardless of its implications for our political views.
 The political world is a very complicated place. For every explanation there are
often competing explanations. Our job is to determine which of the competing
explanations is true.

 In the case of the Bush/Obama economic stimulus it will probably not be possible to
flesh out the truth. Why?

 There was no control group which did not receive the stimulus. It was applied
uniformly to the entire nation. It would have been unethical to withhold the stimulus
because of the economic implications for those who did not receive it.

 Similarly, there are often rival explanations for most things we want to explain.
 Consider our earlier discussion on concealed handguns on college campuses.
 We hypothesized that partisanship was probably a good explanation for supporting
this initiative.
 However, there are other possible rival explanations which require “controlling for”
if we are to be able to make causal statements.
 Gender might explain this phenomenon. Women are more likely to be Democrats
and Men are more likely to be Republicans. Further, Women are generally stronger
gun control advocates than men.
 These rival explanations undermine our ability to reach causal conclusions about
how partisanship affects support for guns on campus.
 Is the independent variable, partisanship, causing support/non-support for guns on
campus. Or, is some other variable at work such as gender, distorting our results and
leading us to erroneous conclusions.
 Our ability to rule out competing explanations depends on the power of our research
design.

 We can approach this problem by designing an experimental study. An experimental

study typically contains a treatment group and a control group. The experimental
subjects are similar in every way, except that the treatment group receives the
stimulus and the control group does not. This isolation of only one group receiving
the stimulus allows us to make causal explanations.

 We can also approach this problem by designing an observational study. Here the
researcher makes controlled comparisons. That is, the researcher observes the effect
of the independent variable of interest on the dependent variable, while holding
constant all other plausible causes of the dependent variable.
 In all experiments, the investigator manipulates a treatment group and a control
group in such a way that, in the beginning, the two groups are virtually identical in
every way.

 Measurement is taken prior to the application of a stimulus.

 The two groups then receive different values of the independent variable of interest.
Typically, the treatment group receives the stimulus, while the control group does
not.

 Measurement is then taken of both groups after the application of the stimulus.

 Since the two groups are identical in every way, except in their receipt of the
stimulus, any observed differences in the dependent variable cannot be attributed to
rival explanations.
 In a laboratory experiment, the treatment group and control group are studied in an
environment wholly created by the investigator. For example, we have an experimental
lab in this building where students participate in experiments while sitting in front of a
computer screen. Participants are generally aware that an experiment is taking place, but
are often unaware of the purpose. In laboratory experiments subjects are recruited to a
common location, the experiment is largely conducted at that location, and the researcher
controls almost all aspects in the environment in that location except for subject behavior.

 In a field experiment, the treatment and control group are studied in their natural
environment, living their lives normally, probably unaware that an experiment is taking
place. In a field experiment the researcher’s intervention takes place in an environment
where the researcher has only limited control beyond the intervention conducted.
 Both types of experiments generally depend on random assignment for control. Random
assignment occurs when each prospective participant has an equal probability of being in
the treatment and control groups.
 Suppose we want to study the effect of candidates’ racial attributes on people’s
political behavior. For example, I have a colleague who studied the effect of racial
attributes in Mexican election behavior.

 In Mexico there is a diversity of racial features among the population. There are
people of Spanish origin who appear caucasian; there are people of Mulatto origin
who are very dark; there are people of Indian origin whose features are less dark but
have distinctly Mongol features.

 The treatment is showing participants pictures of three candidates.

• One picture depicts Candidate A with obvious Spanish features.
• A second picture depicts Candidate B who is very dark, probably mulatto (mix
between caucasian and African).
• A third picture shows Candidate C who has clear Indian features, probably
descended from mixed caucasion/Mayan or Aztec populations.
 The participants are read identical candidate profiles for the three. Each is well-
educated, has a legal background, and is very successful.

 Participants are randomly assigned to three groups. Each of the three groups is
shown a different picture at the same time they see the candidate profiles.

 Participants in each group are then asked which candidate they prefer in the
upcoming Mexican election to represent their district.

 A significant difference is observed between groups in preferences for the three

candidates. The candidate who appears Spanish is preferred over the candidate who
appears Indian, who is in turn preferred over the candidate who appears African.

 What can we conclude? Why?

 What is the role of “control” in allowing us to make these conclusions?

 In order to make conclusions we need good random assignment. In other words, the
three groups need to be identical in every way.

 Random assignment is the great equalizer of rival explanations.

 Internal validity means that the experiment successfully controls for rival
explanations that might cause the treatment response.
 External validity refers to whether or not the results from an experiment are
“generalizable” to the population.
 Laboratory experiments generally have high internal validity, but low
external validity. For example, the preceding experiment involved
participants who knew they were participating in an experiment.
 How might this have affected the results?
 It is common for political scientists to use “college students” or “internet
respondents” when conducting laboratory experiments. How might this
affect the external validity of experiments?
 Field experiments are conducted in the real world and can sometimes overcome the limitations
of laboratory experiments.

 Gerber, Karlan, and Bergen (2007) conducted a field experiment on how exposure to various
media outlets affects voting behavior. They randomly selected households in Prince William
County Virginia to receive treatments prior to the 2005 Virginia gubernatorial election..

 Postcard to Subjects

Congratulations!
You have won a free
Ten week subscription to The Washington Post/Washington Times!
We have held a drawing to award free ten-week subscriptions of The Washington Post to households in Prince
William County. Delivery begins this week. Delivery will automatically end after ten weeks, you do not
need to call to cancel. However, if you want to cancel before the end of the ten weeks, please call 1-
800-635-9224 and we will remove you from this promotion. Thank you for trying out the newspaper.
 Group A was selected to receive a free newspaper subscription to the Washington Post (a
liberal media outlet).

 Group B was selected to receive a free newspaper subscription to the Washington Times (a
conservative media outlet).

 Group C was a control group which received no free newspaper subscription.

 A public opinion survey was administered to all subjects after the 2005 Virginia
gubernatorial election.

 Those receiving the subscription to the Washington Post were eight percentage points more
likely to have voted for the Democrat than those in the control group. A subscription to the
Washington Times produced no change in voting behavior relative to the control group.

 Did this field experiment have internal and external validity? What, if any, are the
limitations?
 Experiments are the “gold standard” for research in political science.

 However, many research questions are not suited for conducting experiments. As
political scientists we study concepts as we find them naturally in society. We
generally cannot manipulate variables such as people’s party id, their relative
liberalism, a state’s level of economic development, a nation’s institutional design,
people’s gender, or education levels. The list is long of the things we would find
difficult to manipulate in an experiment, but which are deemed important political
variables.

 Hence, we must often rely on observational studies. An observational study is one in

which we make controlled comparisons of data where we find it.

 In some cases we use surveys based on random selection of respondents from a

population. Then, we compare groups in this randomly selected sample.
 However, if randomization is not complete, there may be factors which “creep into”
our observational studies which can affect the outcome.

 One potential problem is generally labeled “selection bias.” Selection bias occurs
when subjects find their way into the treatment group based on some systematic
factor relating to the dependent variable.

 Examples:

 It was widely predicted in 1948 that Thomas Dewey would win the 1948
presidential election over Harry Truman. These predictions were based on telephone
surveys of respondents voting intentions prior to the election. Why were they
wrong?

 Suppose we conduct an observational study of how a person’s income affects their

propensity to vote. However, our sample contains only respondents from
neighborhoods where people could be safely surveyed. As a result, we have too few
low income respondents in our sample. What is the result of this procedure?
 Wood and Vedlitz (2007) conducted a survey experiment concerning the
determinants of people’s views on global climate change.

 At the start of the survey, they asked people how concerned they were about various
issues facing the nation. The issues included global terrorism, global climate
change, the economy, discrimination, deteriorating moral values, etc. At this stage
people had no idea that the researchers’ primary interest was their level of concern
about global climate change.

 About 30 minutes into the survey after questioning people about various issues,
people were randomly selected to receive three different scenarios.

 The three groups randomly received the scenarios: Suppose I told you that 70/50/30
percent of Americans believe that global climate change is a serious problem.
 Respondents were then asked how concerned they are about the issue of global
climate change.

 Comparing the scenario responses to the pre-measured level of concern, those

receiving the high treatment (70) were more concerned about global climate change
than those in the mid-treatment (50), who were more concerned than those in the
low treatment (30).

 The authors claimed that this difference suggested the importance of social pressure
to people’s level of concern.
 A controlled comparison is accomplished by examining the relationship between an
independent and dependent variable, while holding constant other variables,
especially those suggested by rival explanations.
 Returning to our example of “support for guns on college campuses.” Our theory
says that this support is based on a person’s party identification. However, a rival
explanation is gender determines support. How could we evaluate which is correct?
 We could look at support for guns on campus only among women, and only among
men. If it differs between Republican and Democrat women, and it differs between
Republican and Democrat men, then we would know that partisanship is an
important explanation.
 More generally, we can also accomplish the same thing statistically, rather than by
splitting the sample into women and men. You will learn how to do this in future
weeks. For now, it is enough to note the concept.
 This brings us back to our earlier discussion of causality, spuriousness, mediating
relationships, and interactions.
Here, Gender affects Partisan ID and Gender affects Gun control opinion. If the relation
between X and Y disappears when Z enters the relationship, then we say the relation
between X and Y is spurious.
One way of seeing a spurious relationship is simply to construct a graph of the
two groups. Above we can see that Republican and Democrat women are
about the same in their opinions about gun control. Similarly, men Republicans
and Democats are also similar. Therefore, the relationship between party id
and opinions on gun control is spurious, fully determined by gender.
However, suppose that Gender does not fully explain both partisan id and opinion on
gun control. Then we can say that Gender both directly and indirectly affects opinion
on gun control. Gender affects gun control opinion directly (Z to Y). It also affects gun
control opinion indirectly (Z to X to Y). The two effects are called additive.
Here is the same line chart as before, but showing that partisanship affects gun
control opinion. Note that the two lines are parallel. However, this need not be
the case. The relationship can also be “interactive.”
 Imagine a scenario in which support for gun control is such that women do not
differ by partisanship, but that men do differ by partisanship. We call such a
relationship an “interactive” relationship.

 In other words, gender affects support for gun control, but partisanship only affects
it for men.