Experimentation

Experimentation 1
 Learning Objectives
Upon completion of this chapter, you will be able to:

 Understand the concept of experimentation and causality

 Understand the internal and external validity issues in
experimentation
 Understand the threats to internal and external validity in
experimentation
 Understand the ways to control extraneous variables
 Understand and compare a laboratory experiment and field
experiment
 Understand the concept of experimental designs and deal with
various types of experimental designs
 Understand the limitations of experimentation
 Understand the concept of test marketing and different types of test
marketing techniques
Experimentation 2
 Defining Experiments
 Experiments can be defined as the systematic study in which a
researcher controls or manipulates one or more independent
(experiment) variables to test a hypothesis about the
independent variable.
 Independent variables are the variables that are manipulated or
controlled by the researcher.
 On the other hand, dependent variables are variables on which
a researcher has little or no control over the research process,
but the researcher is keen to note a change in it with the
corresponding change in the independent variables.
 Tull and Hawkins (1984) stated that the essence of
experimentation is the manipulation of one or more variables
by the experimenter in such a way that its effect on one or more
variables can be measured.

Experimentation 3
 Concept of Causality
 Before detailing experimentation, we will first focus on the
concept of causality, a prerequisite for experimentation. We do
things in the world by exploiting our knowledge of what cause
what (Hobbs, 2005).
 In fact, causality is a conditional phenomenon between
variables in the form “if x, then y.” Causality is an important
aspect of how we construct reality (Cavazza et al., 2007).
 There exist four formal conditions for causality: covariation,
time order of occurrence of variable, systematic elimination of
other causal variable, and experimental designs.

Experimentation 4
 Concept of causality (Cont.)
 Covariation is the extent to which a caused variable occurs with
the causal variable together or vary together as the framed
hypothesis under consideration. For example, sales (dependent
variable) move upward with the lowered status of the price
(independent variable).
 The second condition, time order of occurrence of variable,
explains that the causal variable changes prior to or
simultaneously with the caused variable; hence, it cannot occur
afterwards. For example, when prices are increased on the first
day of the month, sales go down on the remaining days of the
month.
 The third condition, systematic elimination of other causal
variables, indicates that the variable being investigated should be
the only causal explanation of any change in the dependent
variable.
 The experimental designs are a set of procedures conducted to
Experimentation 5
Some Basic Symbols and Notations
in Conducting Experiments

 O = Observation or measurement made on dependent variable

as a part of the experiment
 X = Exposure of the test unit under investigation to independent
variable or experimental manipulation or treatment
 EG = Experimental group that is exposed to the experimental
manipulation or treatment
 CG = Control group participating in the experiment but has no
exposure to the experimental manipulation or treatment
 R = Random assignment of test units and experimental
manipulation or treatments to the groups
 M = Match of experimental group and control group on the basis
of some concerned related characteristics

Experimentation 6
Internal and External Validity in the
Experimentation
 If the design and structure of a study are such that one can
confidently conclude that the independent variable caused
systematic changes in the dependent variable, then the study is
said to have a high internal validity.
 On the other hand, if the study gives us a plausible alternative
interpretation of the observed relationship between the
independent and dependent variable, then it is said to have a
low internal validity.
 The external validity typically refers to the generalizability of
the results of a study to other (usually real world) settings or
populations (Anderson & Bushman, 1997).

Experimentation 7
Internal and External Validity in the
Experimentation (cont.)
 The problems of the external validity generally relate to the
possibility that a specific, but time limited, set of experimental
conditions may not deal with the interpretations of untested
variables in the real world (Zikmund, 2007).
 The lack of external validity puts a researcher in a difficult
situation to repeat the experiment for a different set of subjects
in different time span. If an experiment lacks the internal
validity, then there is no meaning in generalizing the result of
the experiment.
 Factors that are of serious concern for the internal validity of
the experiment may also jeopardize the external validity of the
experiment. The major source of threat to the internal validity of
the experiment is the impact of different extraneous variables.

Experimentation 8
 Threats to the Internal Validity
of the Experiment

 History
 Maturation
 Testing
 Instrumentation
 statistical regression
 selection bias
 mortality

Experimentation 9
 Threats to the Internal Validity
of the Experiment

 History effect refers to a specific event in the external

environment that occurs between the commencements of
experiment and when the experiment ends.
 In an experiment, maturation takes place when the subjects
become older, bored, experienced, or disinterested during the
experiment.
 A testing effect occurs when a pre-test measurement sensitizes
the subjects to the nature of the experiment.
 The instrumentation effect is said to be occurred in an
experiment when either the measuring instrument or the
observer changes during the experiment.

Experimentation 10
 Threats to the Internal Validity
of the Experiment

 Statistical regression is the tendency of the subjects with

extreme scores to migrate (regress) towards the average scores
during the experiment.
 Selection bias occurs when an experimental group significantly
differs from the target population or control group.
 Mortality effect occurs when the subjects drop out while the
experiment is in progress. The subjects refuse to participate in
the experiment because of various reasons such as lack of time,
loss of interest in the experiment, and so on.

Experimentation 11
 Threats to the External Validity
of the Experiment

 If an experiment lacks the internal validity, then there is no

meaning in generalizing the result of the experiment.
 In other words, the internal validity can be viewed as a
necessary, but not sufficient, condition for external validity
(Parasuraman et al., 2004).
 Four biases, such as reactive effect, interaction bias, multiple
treatment effect, and non- representativeness of the samples,
that seriously jeopardize the external validity of an experiment.

Experimentation 12
 Threats to the External Validity
of the Experiment
 Reactive effect occurs when the respondents exhibit an unusual
behaviour knowing that they are participating in an experiment.
 Interaction bias occurs when a pre-test increases or decreases
the sensitization of the respondent to the experimental
treatment.
 Multiple treatment effect occurs when a participant is exposed
to multiple treatments. This is quite obvious that the impact of
previous treatments cannot be fully erased from the mind of
the respondents.
 Sometimes, it happens that a researcher knowingly or
unknowingly selects the subjects who may not be a true
representative of the population. When this happens in an
experiment, it lacks external validity.

Experimentation 13
Ways to Control Extraneous Variables

 Presence of extraneous variable poses a serious threat to the

internal as well as external validity of the experiment.
 The four ways to control the extraneous variable are
 Randomization
 Matching
 Statistical control
 Design control

Experimentation 14
Ways to Control Extraneous Variables
(cont.)
 Randomization refers to the random assignment of the subjects
and experimental treatment to experimental group to equally
distribute the effect of extraneous variables.
 When a researcher suspects that the extraneous variables may
affect the dependent variable, he or she applies the technique of
Matching, which involves matching each group on some
pertinent characteristics or some pertinent background variables.
 With the help of a Statistical control, a researcher measures the
effect of extraneous variable and adjusts its impact with a
sophisticated statistical analysis. In this context, a statistical
technique such as ANCOVA can be applied.
 Design control suggests the use of an appropriate experimental
design to control the effect of extraneous variable.

Experimentation 15
Laboratory Versus Field Experiment
 The laboratory experiment is conducted in a laboratory or
artificial setting. A researcher applies or controls the
experimental manipulation or treatment in an artificial
environment.
 A field experiment is conducted in the field or a natural setting.
In the field experiment, the effect of experimental manipulation
or independent variables on dependent variable is observed in a
natural setting.
 The main objective of experimentation is to detect or confirm
the causal relationship between the dependent variable and
independent variables and quantify them.
 In every study, there is a trade-off between internal and
external validity.

Experimentation 16
 Experimental Designs and its
Classification

 An experimental design is a sketch to execute an experiment

where a researcher is able to control or manipulate at least one
independent variable. The experimental designs can be broadly
segregated into two groups: classical experimental designs and
statistical experimental designs.
 Classical experimental designs consider the impact of only one
treatment level of independent variable taken for the study at a
time, whereas statistical experimental design considers the
impact of different treatment levels of independent
(explanatory) variable as well as the impact of two or more
independent variables.

Experimentation 17
Figure 8.1 : A classification of experimental designs

Experimentation 18
 Classical Experimental Designs
1. Pre-Experimental Design

 Pre-experimental design is an exploratory type of research

design and has no control over extraneous factors.
 These designs are exploratory in nature and are mainly used to
frame the hypotheses about the casual relationship and that is
why a detailed description of these designs is of paramount
importance.
 There are four commonly used pre-experimental designs. These
are one-group, after-only design; one-group, before-after
design; non-matched with control group design, and matched
with control group design.

Experimentation 19
1 (a) One-Group, After-Only Design

One-group, after-only design is the most basic experimental design.

One-group, after-only experimental design involves the exposure of
single group test unit to a Treatment X and then taking a single
measurement on the dependent variable (O).

Experimentation 20
1 (b) One-Group, Before-After Design
One-group, before-after design involves testing the test units twice.
The first observation is made without exposing the test units to any
treatment, and the second observation is made after exposing the
test unit to treatment.
The treatment effect can be determined as the difference between
the first observation and the second observation, that is, O1 − O2.

Experimentation 21
 1 (c) Non-Matched With Control
Group Design
Non-matched with control group design involves the introduction of
control group in the experiment. This group does not receive any
experimental treatment. In this design, the control group is
introduced, so that it can be compared with the experimental group.
The treatment effect can be determined as the difference between
the observations from the experimental group that receives the
treatment (O1) and the observations from the control group that
receives no treatment (O2). Hence, the result of interest is O1 − O2.

Experimentation 22
1 (d) Matched with Control Group Design

To address the problem of selection bias, matched with control

group design involves the matching of experimental group and
control group on the basis of some relevant characteristics.
M indicates that experimental group and control group are matched
on the basis of some relevant characteristics.

Experimentation 23
 Classical Experimental Designs
2. True-Experimental Design

 “Randomization” is the key of difference between pre-

experimental design and true experimental design.
 True experimental design involves the random assignment of
test units to the experimental group and various treatments to
the experimental groups.
 True experimental designs are commonly classified as two-
group, before-after design; two-group, after-only design; and
Solomon four-group design.

Experimentation 24
2 (a) Two-Group, Before-After Design
Two-group, before-after design is also known as pre-test–post-test
control group design. This design involves the random assignment
of test units to either the experimental group or the control group.

The control group is treated with extraneous variable only and not
with the experimental treatment. Let the impact of experimental
variable be EV and impact of extraneous variable be UV. Hence, the
impact of the experimental treatment can be computed as

Experimentation 25
2 (b) Two-Group, After-Only Design
Two-group, after-only design is similar to the matched with control
group design with one difference in terms of assignment of units
(or treatments) to experimental group and control group in a
random manner.

This design is mainly susceptible to two extraneous variables:

selection bias and mortality.

Experimentation 26
 2 (c) Solomon Four-Group Design
To handle the problems of two supplement groups, before-after
design is supplemented by an after-only design and is referred to as
Solomon four-group design. This design is also known as four-group-
six-study design.

Experimentation 27
 Classical Experimental Designs
3. Quasi-Experimental Designs

 In quasi-experimental design, a researcher lacks full control

over the when and whom part of the experiment and often
non-randomly selects the group members.
 The quasi-experimental designs are useful because these are
less expensive and they save time. These designs can also be
used when true experimental designs cannot be used. The
widely used quasi-experimental designs are time series designs
and multiple time series designs.

Experimentation 28
 3 (a) Time Series Designs
Time series designs are like one-group, before-after design except
that the periodic measurement is employed on the dependent
variable for a group of test units.
In the time series designs, treatments are either administered by
the researcher or it occurs naturally.

Experimentation 29
Figure 8.2: Time series design

Experimentation 30
3 (b) Multiple Time Series Designs
In a multiple time series design, another group of test units is
incorporated to serve as a control group. This design may be a better
alternative as compared with the time series designs subject to a
cautious selection of the control group.

Experimentation 31
Statistical Experimental Designs
 Statistical experimental designs involve the conduction of a
series of experiments to statistically control the extraneous
variables and to measure the impact of multiple independent
variables.
 As different from the classical experimental designs, statistical
experimental designs allow a researcher to examine the impact
of different treatment levels of the experimental variable.
 These designs also provide an opportunity to determine the
impact of two or more independent variables. The most widely
and commonly used statistical designs are completely
randomized design, randomized block design, Latin square
design, and factorial design.

Experimentation 32
1. Completely Randomized Design
In a completely randomized design, the experimental treatments are
randomly assigned to the test units. By randomly assigning
treatments to the test units, a researcher tries to check the impact of
extraneous variables through manipulation of the treatment
variable.

The statistical technique applied to analyse the result of this type of

experimental design is known as “Analysis of Variance” commonly
known as ANOVA.

Experimentation 33
 2. Randomized Block Design
 Randomized block design involves in random assignment of
treatments to the experimental group and control group.
 A randomized block design is useful when there is one major
external variable, such as sales, store size, or income of the
respondent that might influence the dependent variable
(Malhotra & Dash, 2009).
 In fact, the randomized block design is a combination of
randomization and matching.
 Thus, control variable defines groups and the randomized
experiment is conducted within each group.

Experimentation 34
 Randomized Block Design (Cont.)
A randomized block design can be symbolically represented as

For understanding the concept of randomized block design, let us

take a simple example of a company that considers the difference in
sales in four different “price levels”. It is also believed by the
company that the geographic regions may also have an impact on
the sales.

Experimentation 35
 Randomized Block Design (Cont.)
The research question may be do the regions significantly differ in
terms of generating sales. The second question may be do the price
levels significantly differ in terms of generating sales. This example
can be symbolically represented as

Experimentation 36
 Randomized Block Design (Cont.)
The sales (in thousands) results are exhibited in the Table 8.2.

Experimentation 37
 3. Latin Square Design
 Latin square design allows a researcher to control two external
variables (non-interacting) along with the manipulation of independent
variable.
 In Latin square design, the test units are grouped according to the two
external variables considered in the study. The test units are
systematically blocked in two directions provided by two external
variables.
 Let us assume that another external variable “size of the showroom”
may provide the basis for difference in quarterly sales (in million
rupees) of the company.
 In addition, the company has also considered one more region “south
region” as the basis for difference in sales volume. The company has
four different sizes of showrooms across the country. So, the design
based on the two external variables “different regions” and “size of the
showroom” will appear as shown in Table 8.3.

Experimentation 38
 Latin Square Design (Cont.)
Treatments (levels of independent variable) are applied to the cells
in such a way that each level of independent variable is applied to
each cell only once as exhibited in Table 8.3.
In Latin square design, each cell is treated with only one treatment
level.

Experimentation 39
 4. Factorial Design
 In the statistical designs discussed so far, the effect of only one
independent (experimental) variable on dependent variable was
studied.
 In a factorial design, two or more experimental variables are
simultaneously considered.
 In a factorial experiment, more than one type of independent
variables are varied at a time but in a structured way (Shaw et al.,
2002); for example, a fast moving consumer goods company wants
to test a new product in 30 cities.
 Three different customer groups with respect to income: high-
income group, middle-income group, and low-income group are to
be tested. In addition, two price levels: high price and moderate
price are considered in the experiment. Hence, this will be 3 × 2
factorial, as there are three different levels of income and two
different treatment levels.

Experimentation 40
 Factorial Design (Cont.)
 So, this design will have the following six experimental groups:
 X1 = High-income group; high price
 X2 = High-income group; moderate price
 X3 = Middle-income group; high price
 X4 = Middle-income group; moderate price
 X5 = Low-income group; high price
 X6 = Low-income group; moderate price

Experimentation 41
 Factorial Design (Cont.)
The effect of each independent variable on dependent variable is
referred to as the main effect. Hence, the impact of “different
income levels” on sales and “price levels” on sales is termed as the
main effect.
Interaction takes place when the simultaneous effect of two or more
experimental variables is different from the sum of their separate
effect.

Experimentation 42
 Limitations of Experimentation
 Time
 Cost
 Secrecy
 Implementation problems

Experimentation 43
 Test Marketing
 Test marketing means conducting an experiment in a field
setting.
 Test markets are generally classified into four types: standard,
controlled, electronic, and simulated.
 In standard test market, a company uses its own distribution
channel network to test a new product or market mix variables.
 In controlled test market, a company hires an outside research
agency to conduct the study.
 An electronic test market gathers data from the consumers who
agree to carry an identification card that they present when
buying goods and services at participating retailers in the
selected cities (Hair et al., 2002).

Experimentation 44
 Test Marketing (Cont.)
 Simulated test market is an artificial technique of test marketing.
A simulated test market occurs in a laboratory, where the
potential consumers of a particular product are exposed to a new
product or competitive product or any other marketing stimuli.

Experimentation 45