Experimental

Designs
Research
4
CHAPTER

Learning Objectives
By the end of the chapter, you should be
able to:
Define an experiment and explain the
concept of causality.
Discuss the necessary conditions for drawing
3. Explain the basic concepts that are causal inferences.
used in experiments.
4. Explain the difference
between internal and external validity of theexperiment.
5. Explain the factors affecting internal
validity of the
6 Describe the factors affecting external validity of theexperiment.
7. Discuss the methods to experiment.
control extraneous variables.
8 Distinguish between laboratory and field experiments.
9. Explain the classification of experimental designs into
four categories--pre-experimental, quasi
experimental, true experimental design and statistical designs.

In 1991 Bajaj Enterprises set up a chain of supermarkets in all the Indian metros. These
supermarkets sell a broad line
of household and kitchen appliances. While the supermarkets in other metros were doing well, the one in
Delhi NCR
was showing a stagnant growth of 2-2.5 per cent per annum. The General Manager (Sales) was concened and was
thinking of ways to boost the sales. A meeting of the senior marketing officials was called to discuss the issue. Many
Suggestions came up including increasing the advertising budget, reducing the prices of slow-moving items, and giving
a discount to loyal customers. One of the suggestions was to offer a discount of 5 per cent in the form of coupons to
customers who opt for a bulk purchase of2,500/- and above. It was decided that these customers would be given 5per
cent discount coupons that they could redeem within a three-month period. It was argued that this would gradually result
n increasing sales and profits of the supermarkets. However, amarket researcher whowas part of the discussion team
argued that the sale increase depended upon a host of factors such as the size of the supermarket, location, the layout,
point-of-purchase (POP) displays, competitor's prices and competitor's advertising expenses besides other variables.
Ihe regulation of many of these was beyond their control. The GM (Sales) also gave athought to designing astudy in
order to examine the impact of the entire idea of discount on the bulk purchase scheme and gradually on the net sales
and profits of thesupermarkets. The members also realized that the extraneous factors would have to be controlled s0
as to infer a
causality.
 Research Methodology
70
EXPERIMENT?
WHAT IS AN
LEARNING OBJECTIVE 1
Define an experiment
and expiain the concept
of caUsality.
NECESSARY CONDITIONS FOR MAKING CAUSAL INFERENCES
LEARNING OBJECTIVE 2
Discuss the necesSary
conditions for drawing
Causal inferences.
2. Time order of
Concomitant variation is the
extent to which a cause Xand
effect Yoccur together or vary
together.
This chapterdiscussestheissuesinvolvedin inferringa cause and effectrelationship.
Anumber ofconcepts would be discussed which wouldihelpin setting up experiments
limitations of various designs in removing thei
to establish
causality.
extraneous variables
The
will also be covered
under this chapter.
influence of
An experimentis generally used toinfer a causality. In an experiment, a researcher
causal variables and measures their
dependenteffects
manipulates one or more changes in the on
actively any
the dependent variables
of interest. Since variable
variables, the relationship between cause and
number of other
may be caused by a probabilistic in nature. It is virtually impossible to prove a
often tends to be
causality. One can onlyinfer a cause-and-effecttrelationship. It is, therefore, essential
effect
understand the whole concept of causality. To illustrate this concept, an example
to
follows in the paragraph below.
Causality
some ofhis sales personnal
The sales manager of a soft drink bottling company sends
from the training
for a new sales training programme. Three months after they return increases hy
programme, the sales in the territory where this sales force was Working
20 per cent. The sales manager concludes that the training programmne is very
effective and, therefore, the sales force from the other territories should also he
sent for the same. What the sales manager is trying to infer is that the sales training
is a causal variable and increased sales is an effect variable. Do we agree to this
statement? This statement may not be true as the increase in sales may not be due to
the sales training programme alone. It could occur because of a host of factors e.g.,
reduction in the price of the soft drink,a strikeat the competitor's plant,increase in
the price of the competitor's product, reduction in the quality of competing products,
weather conditions and so on. Therefore, it is very important that the sales manager
understands the conditions under which such causal statements can be made. There
are three necessary conditions for making causal inferences.
The following are the necessary conditions for making
causal inferences:
1. Concomitant variation: Concomitant variation is the extent to which a
and effect Yoccurtogether orvarytogether. This cause A
means that there hasto bea strong
association between the training programme and increased sales.
of them need to occur
together. However, a strong association betweenMoreover, bon
does not imply causality. The he t
be due to the influence of otherhigh association between these two variables coud
the variables or it may be the ofextraneous factors which may be influencing bou
result of random variations.
occurrence of
variable must occur prior to variables: This condition means that the
or caus
means that sales training must have simultaneously with the effect
taken place either before or
variable. This
with the increased sales.
an increase in sales However, just because sales simultaneousy
place prior
to
will not help in training took
a mere inferring causality.
coincidence and thus, cannot help in inferring It might have been dueto
causality.
 Experimental Research Designs 71

urthermore, it is quite possible for each of the two events to be both cause and
effect of each other., In the illustrated example,the sales training programme may
cause an increase in sales, and increased sales may result in keeping company
some spare funds for training etc. Therefore, the relationship between the two
variables could be that they alternatively 'feed' each other.
Even if it can be shown that there is a concomitant variation between the
sales training programme and the increased sales and the time occurrence of
all variables, there is still a question left unanswered whether other variables
which could 'cause' increased sales have remained ina constant position. This is
explained in the next point.
3. Absence of other possible causal factors: As mentioned earlier, the increase in
an
Theobjective of measure the sales of soft drink could have been due tomany other factors besides the sales
experimentis to training. There could be a strike at the competitor's plant, resulting in an overall
independent
influenceofthe reduction in supply, weather conditions, the increased price of the competitor's
dependent
variables on a product or a problem at the distribution channel at the competitor's end. The sales
variable while keeping the training programme may be a causal variable if all the other factors mentioned
effect of other extraneous above were kept constant or otherwise controlled.
variables constant.
Asamatter offact, the researcher cannotrule outthe influence ofother causal factors
such as the weather condition. However, it will be seen later that it may be possible to
control some or nmore of the extraneous variables by the use of experimental design. It
may be possible to balance the effect of some uncontrolled factors. This may help in
measuring random variations resulting from uncontrolled measures.
Experiments are used to seekhelp in identifying acause-and-effectrelationship.
The objective of an experiment is to measure the influence of the independent
variables on a dependent variable while keeping the effect of other extraneous
variables constant. Experiments may be used to arrive at conclusive answers in
the following situations:
Can a change in the package design of a product enhance its sales?
purchase to
" Should a supermarket introduce a discount scheme on bulk
increase its sales?
particular
Will an increase in the shelf space allocated to a brand of a
product increase its sales?
restaurant increase
. Will a reduction in the price of the menu items of a
sales?
on their sales?
What will be the impact of POP display of Arrow shirts
increasing the
Which of several promotional techniques is most effective in
sales of a product?
of female counter clerks
What is the impact of increasing the proportion
the store?
from 30 to 60 per cent on the sales of
person to the organizational
Does mentoring help in acclimatizing a
culture? company?
Does organizational climate impact the quality of workinglife of ainvestment
home loan rates on the investor
" What is the impact of change in
in real estate?
understanding of experimentation, it would be useful to
In order to have a good
definition used in experiments.
learn some basic concepts and

1. Define the term 'experiment.

CONCEPT What is a concomitant variation?
2 establishing causality?
CHECK order of occurrence of variables in
3 What is the significance of the time
 Researh Methoooloay
74
AFFECTING
FACTORS
EARNIN OBIECTVES
Exploin the foctors
offecting intemal validity
of the experiment.
Nistory, nths conlexl refers
hese spe fcevents that are
etema to the experment but
Ocu at the same time as the
expenment
The main testing effect occurs
when the first observation
nfiuences the second
observation.
4.
EXPERIMENT
INTERNAL VALIDITY OF THE
to Control influence of extraneous
the
As discussed
earlier,
thatthe
thereis a need
experiment has not
been confounded. Thee
experiment. following varextiraaneous
bles so
astoensure threaten the internalvalidity of an refer to the
variables may contextdoes not 0CCurrence of
Historyinthe
present
sto those specific events that. are
History hererefers t time as the
events
1. History:
experiment. experiment. externalthe
beforethe
to the
experiment but occur at
the same Consider
following experiment:
0, X O,
(sales training programme)
and the
where X denotes
treatment
and
symbols
after the training programme. The
0.
the sale before
and 0, may represent indicate the treatment effect. Even if this
difference
difference (0, - 0,) may attributed tothe training programme as this may be due
is
positive, this may not be condition between 0, and 0,.
improvement in the general economic This
positiv,
to an is not the only variable causing a
is because the training programmematter of lact, the higher the time differon
difference between 0, and 0,.As a
chances of history confounding o
between the two observations, higher are the
experiment.
Concerned with the
2. Maturation: Maturation is similar to history except that it is
passage of time. These changes are nos
changes in a test unit occurring with the becoming
include people
due to the impact of treatments. Examples of maturation example.
older, more experienced, tired, or uninterested. Referring to our
sales people might have gained maturity as with passage of time they become
experienced and understand their job better. It is not only people who chanop
over time, so do stores, geographic regions and organizations. Stores change over
time in terms of physical layout, décor, traffic and composition. Again, longer
the time difference between O, and O,,the greater are the chances of maturation
effect to OCcur.
3. Testing: It is concerned with the possible effect on the experiment of taking a
measurement on the dependent variable before presentation of the treatment.
Testing effects are of two kinds: (i) main testing effect and (ii) reactive or
interactive testing effect. The main testing effect occurs when the frst observation
inluences the second observation. This is responsible for compromising with the
internal validity of the experiment. Consider, as an example, aquestionnaire filled
up by the respondents before being exposed to the treatment. Now, after being
subjected to the treatment, they are likely to respond differently. This is because
they are now 'experts' with the questionnaire.
Consider the example of the sales training programme mentioned earlier. If the
respondents become aware during the experimentation that their behaviour 1s
being measured, this can sensitize and bias the responses. For
people know that they are being sent for the training to know example, if sales
its efectiveness)
they would become 'sensitized' and
behave differently.
Instrumentation: It refers to the effect caused by the
changes in measul'b
Instrument used for taking an observation. At times ameasurement instrume
may be modified during the course of an
that particular experiment. experiment resulting in confounais

Statistical regression Ocurs
when the testunits
with
sores are hosen for
etree
e20sure to the treatment.
FACTORS AFFECTING EXTERNAL VALIDITY
LEARNING OBJECTIVE 6
Describe the factors
affecting external validity
of the experiment.
Experimerntal Research Designs 75
Suppose the diference in 'rupee' sales before and 'after' the training
programme is used to measure the effectiveness of the training programme, a
price difference during the time interval could make a substantial difference in
the inference. A'change in price' would be the change of
instrumentation.
Presenting the pre and post-test questionnaire in a different fashion, experience
of the invigilator, and a change in the mood of the investigators are some of the
examples of changing instrumentation.
5. Statistical regression: The effect of statistical regression occurs when the test
units with extreme scores (either extremely favourable or extremely unfavourable)
are chosen for exposure to the treatment. The effect is that test units with extreme
sCores tend to move towards an average score with the passage of time. Suppose
in the example of the sales training programme, the sales people with extremely
poor performance are sent for the training programme. An increase in sales
after the training programme may be attributed to the regression effect. This is
because test units with extreme score have more room for a change, so a variation
0s more likely to be there. Random occurrences (weather, luck, festive seasons),
might have helped good and poor performance of sales people in the pre-test
measurement. These random occurrences will turn some of the poor performers,
into better performers thereby confounding the experiment.
6. Selection bias: This refers to the improper assignments oftest units to treatments.
Test units may be assigned to the treatment groups in such a way that the groups
differ on the dependent variable prior to the presentation of the treatment.
Selection bias can occur if test units self-select their groups or are assigned to the
groups on the basis of the researcher's judgment. The selection of test units to the
treatment group should be random.
7. Test unit mortality: Some of the test units might drop out from the experiment
while it is in progress or some may refuse to continue with the experiment. In the
case of sales training example, some sales people may quit the organization before
completing the training successfully. There is no way of finding outwhether those
who were not improving quit the organization. It is also not possible to measure
whether those who left would have produced the same results as those who
completed the training programme.
The types of extraneous variables discussed above are not mutually exclusive.
They can occur together and interact with each other. These extraneous variables
can provide alternative explanations regarding what is being observed in an
experiment and our objective should be to eliminate the possibility of these effects
confounding the results.
the absence of all
While the internal validity of an experiment is concerned with
examined, external
possible causal factors except the one whose influence is being
The factors affecting
validity raises the issues of generalizability of the findings.
external validity of the experiment are listed below:
different from the environment
The environment at the time of test may be
be generalized. For example, a
of the real world where these results are to prospective customers
commercial advertisement may be shown to a set offavourable. However, if
may be very
and their reaction to the advertisement respondents are watching TV at
while the
the same advertisement appears
 see it
Research Methodology members, they may not
liketo
the two
and switch to
theirfamily example, the environment in
76
with
home channel. Inthis
another different and has come in the
way to generalize
test may
the results.
not be
situations is
completely experimentation
for
of the
experiments are to be to similar
.
Population usedwhere the results of the
population perform a task that
college are asked to performance. could
applied.
of a
the
Suppose the students
the effects on their However,
the
manipulated to study be generalizedto the real world when the samne
be studycannot organization. This is
findings ofthis employees of an because
the
assigned to the ofjobin this particular organization may be quite
task is nature
employeesandthe
not hold in an application of 12
different. a 5-6 week test may cream in Delhi NCR. The
in
Results obtained company wants to launch ice
a months of May and June may be
months. Suppose conducted duringthe
results ofthe
survey
favourable.
wouldl certainly not be
These results January, thereby raising questions applicableduring ENVIR
extremely and on
winter months in December
the results. LEARNIN
generalizability ofthe different from the treatment of the
the
the time of the test may be the effect of a treatment, Distingu
Treatment at
happen when whiletesting laborat
real world. This
can it is given as a part of a
form of a pill and in reality experir
it is administered in the
cereal.

used in experiments?
1 What are the concepts and 'external validity'?
terms 'internalvalidity'
CONCEPT 2 What is meant by the experimental research.
symbols commonly used in experiment.
CHECK 3. Define the set of
affect the internal and external validity of a particular
4. Name the prime factors that In a la
the re
artific
EXTRANEOUS VARIABLES
METHODS TO CONTROL Condu

discussed in the previous sections, extraneous variables pose a threat to the field
LEARNING OBJECTIVE 7 As
Discuss the methods internal and external validity of the experiment. They affect the dependent variable mar

Therefore, there is a need to control the

to controlextraneous and confound the results of the experiment.
explanations of crucial
variabies. the extraneous variables as they represent alternative
experimental results.
The researcher has four methods to control the effect of extraneous variables.
These are randomization, matching, use of specific experimental design and
statistical control. These methods are discussed below:
1. Randomization: It refers to the random assignments of test units
experimental groups. Treatments are also randomly assigned to the experimental
grOups. Because of random assignment, extraneous factors will be operating n
experimental groups. However, for randomization to be effective, a large sample
size is required.
2. Matching: Another way of controlling extraneous variables is to match the
various groups by confounding variables. Suppose there are 120 people to be
distributed in three groups. If there are 45 females among the 120 members,
then each of the three groups is
gender can be distributed among all assigned 15 females. This way, the e d
three groups. Likewise, other contou
variables like age, income, years of work
the three groups. The experience could be distributedamong
size or location of store.other examples of matching variables can be price, sales,
However, there are two drawbacks of matching. It may

ENVIRONMENTS OF CONDUCTING EXPERIMENTS
LEARNING OBJECTIVE 8
Distinguish between
laboratory and field
experiments.
Inalaboratory experiment
the researcher works in an
artificial environment to
conduct a study whereas in a
field experiement an actual
market condition is used for
the same.
A
CLASSIFICATION OF EXPERIMENTAL DESIGNS
Experirnental Research DesIgns 77
be not possible to match allthe confounding variables to various groups. Further,
matched characteristics may not be relevant to the dependent variable.
3. Use of experimental designs: Some of the
experimental designs may be very
useful in eliminating the influence of extraneous variables. In the subsequent
sections, these experimental designs and their role in eliminating the extraneous
factors will be discussed.
4. Statistical control: If all the above discussed methods fail to eliminate the effect
of extraneous variables among the treatment group, then the experiment in
question gets confounded and it is not possible to make any causal inferences.
However, there is still one way of handling the confounding variable. It may
be possible to statistically control the effects of this variable on the dependent
variable by the use of a technique called analysis of covariance (ANCOVA). This
topic is beyond the scope of this text.
There are two types of environments in which the experiment can be conducted.
These are called laboratory environment and field environment. In a laboratory
experiment, the researcher conducts the experiment in an artificial environment
constructed exclusively for the experiment. Suppose the interest is in studying the
effectiveness of a TVcommercial. If the test units are made to see a test commercial
in atheatre or in a room, theenvironment would of a laboratory experiment. Field
experiment is conducted in actual market conditions. There is no attempt to change
the real-life nature of the environment. Showing of test commnercial in an actual TV
telecast is an example of a field experiment.
There are certain advantages of laboratory experiments over field experiments.
Laboratoryexperiments have higher internal validity as they provide the researcher
with maximum control over the maximum number of confounding variables. Since
the laboratory experiment is conducted in a carefully monitored environment, the
effect of history can be minimized. The results of alaboratory experiment could be
repeated with almost similar subjects and environments. Laboratory experiments
are generally shorter in duration, mnake use of smaller number of test units, easier to
conduct and relatively less expensive than field experiments.
However, laboratory experiments lack in external validityi.e., it is not possible to
generalize the results of the experiment. Experiments conducted in the field have
lower internal validity. The ability to generalize the results of the experiment is
possible in case of a field experiment, thereby leading to higher external validity. In
the light of the above-mentioned facts, researchers need to take a decision whether
to use a laboratory experiment or a field experiment. These two types of experiments
play complementary roles in real life situations.
quasi-experimental,
Experimental design can be classified as pre-experimental,
Pre-experimental designs include the one
true experimental and statistical.pre-test-post-test design and the static group
shot case study, the one-group quasi-experimental designs are time series
comparison. Tests included under
True-experimental designs include pre-test-post-test
and multiple time series. The
group, and Solomon four-group design.
control group, post-test-only control
 78 Research Methodology

completely randomized design,

LEARNING OBJECTIVE 9
Explain the classification
of expeimental designs
into four categories
pre-experimental design,
quasi-experimental
design, true expenimental
design and statistical
design.
One-shot case study is also
called the after-only design
and may be symbolically
presented as:
XO
statistical designs include randomized
factorial andLatinsquare
presented in Figure 4.1.
designs. To have aglimnpse of the classification, blocks,
these are
Pre-experimental Designs
designs do not make use of any randomization
Pre-experimental
control the extraneous variables. Therefore, the internal validity of such designs isprocedures to
questionable. Three designs includedl in this category areeelaborated below:
design is also known as the after-only
design and may
1, One-shot case study: This
be presented symbolically as:
XO
This means that only one test group is subjected to the treatment Xand then
a measurement on the dependent variable is taken 0. It may be noted that the
symbol R does not appear in this design. This means there was no random
assignment of test units to the treatment group. This means that the test units
were either self-selected or arbitrarily selected by the researcher. In the cal
those et
training programmeexample, the sales manager might have chosen
people whom he likes or may askthe sales people to volunteer for the training
programme.

FIGURE 4.1
Experimental
Cassification of Design
ëxerimental design

Pre Quasi True q Statistical

Experimental Experimental Experimental

One-Shot Case Pre-test-Post-test Completely

Time Series
Study Control Group Randomized

One-Group Pre Multiple Time Post-test-Only Randomized

test-Post-test Series Control Group Blocks

Static Group Solomon Four Latin Square

Group

Factorial

One-group pre-test-post-test 2. One-group pre-test-post-test design:internally
design is also known as before
after without control group design without control group design. This design may be written symbolically as:
and may be symbolically written
Static group comparison
Uses two treatment groups
in which test units are not
selected at random.This
design is presented as:
Group 1-X0,
Group 2- O,
Experimental Research Designs 79
Let us examine another example here. The objective is to study the impact of
an extra ten days' credit period (X) on a credit card
payment time (0) and one
decides to study the relationship/impact by offering this to the customers who
make an average usage of 25,000/- per month. The problem in this case would
be that no measure was taken to establish their
payment behaviour prior to the
extended period. Hence, no valid conclusion can be made from this design. There
1S no pre-treatment observation on performance. The level of'O' might be
by several uncontrolled extraneous factors like history, affected
maturation, selection bias
and test unit mortality. These uncontrolled extraneous variables will confound
the experiment and render the design
invalid.
This design is also called before-after
0, X 0,
In this design also, test units are not selected at random as the symbol 'R is not
appearing here. The test units are subjected to the treatmentX and both pre
treatment (0,) and post-treatment measurement (0,) are taken. For instance,
in the credit card example, one might take the payment time
before and after
the extended ten-days' period. One may be tempted to compute
treatment
effect as 0, - 0,, which may not be really so, as this difference could be the
result of many uncontrolled extraneous factors like history, maturation, testing,
instrumentation, regression, selection and mortality. This would make the
design invalid for making any causal inferences on account of the following
reasons:
Theeconomic condition might have changed during the two periods (history).
The test units may mature over time (maturation).
The pre-test measurement on the test units may influence the performance
(testing).
" The prices of goods might havechanged over time (instrumentation).
Test units might not have been selected at random (selection bias).
" Some test units might have left before the experiment was complete (mortality).
Test units might be self-selected on the basis of the current poor performance
and may have a better period ahead because of sheer luck (regression).
3. Static group comparison: This design is symbolically written as:
Group 1 X
Group 2
This design uses two treatment groups. Test units in both the groups are not
selected at random. The first group, called the experimental group, is subjected
to the treatmentX, whereas the second group, namely, the control group, is not
subjected to any treatmnent. Both groups are measured only after the treatment has
been presented. Thus, it is critical to understand that in this design the exposure
as well as the experimental treatment not under the control of the researcher.
Consider the following example:
Astudy wants to assess the relationship of 'family support' (measured by the
presence of domestic help or spouse/family's help in carrying out domestic
chores)with the work-life balance of BPO women employees. Here, the presence
or absence of help is ascertained and then we can mneasure the work-life balance.
Thusthe design is essentially ex-post facto and any segregation into experimental
or controlgroup is not made by the researcher.
 80 Research Methodology
Quasi experimental design
lacks complete control of
scheduling of treatment
and also lacks the ability to
randomize test units'exposure
to treatments.
The results of atime series
design may be affected
by an interactive testing
effect because multiple
measurements are made on
these test units.
could be measured by 0, - 0, However,
The treatment effect selection bias this
could be attributedto at
least
units are not selected at random,
the
sufficient to
two
e
make
and mortality. Moreover.
groups
the
could I differ
design invalid
prior to
for
dif erence
thesiapplication
nce the test
of treatment. All these
inferences.
are
drawing any causal
Quasi-experimental Designs
In quasi-experimental design the researcher can control when measurements are
However, this design lacks
taken and on whomthey are taken. complete
scheduling of treatment and also lacks the
ability to randomize test
units' control
of
to treatments. As the experimental
control is lacking, the possibility
of exposure
aware getting
confounded results is very high. Therefore, the researchersshould be
such variables should be of what
variables are not controlled and the effects of
into the findings. There are two forms of quasi-experimental designs. incorporated
1. Time series design: This design involves aseries ofperiodic1 measurementsSon
o the
dependent variable for agroup of test unit. The treatment Xis then
and a series of periodicmeasurements are again taken to measure the efert
treatment. This design may be written symbolically as:
administered
0, O, O, 0, X O, O, 0, 0,
The above is a quasi-experimental design since there is no randomization of
treatment to test units. Further, the timing of treatment presentation as well
as which of the test units are exposed to the treatment may not be within the
researcher's control. Because of the multiple observations in time series design
the effect of maturation, main testing effect, instrumentation and statistical
regression can be ruled out. If test units are selected at random, selection bias
can be reduced. Further, if a strong measure like giving certain incentives to the
respondents is introduced, mortality effect can more or less be controlled.
The major drawback of this experiment is the inability of a
control the effect of history. The results of the experiment may beresearcher by
to
an interactive testing effect because multiple affected
measurements
test units. If a researcher could keep a record of key
are made on these
economic activities and if no changes are found, onechanges
in various unusual
that the treatment has exerted an effect on test unit. can reasonably conclude
This design may look similar to the one
by 0, X0,. However, there are group pre-test-post-test design given
differences as in case of time series design, a
number of periodic measurements are taken both before and after the application
of the treatment. But in the
case of one group
measurement is taken prior to the treatment and one pre-test-post-test
after
design, One
The results of taking multiple that.
design. This ismeasurements
can be compared with one group
pre-test-post-test
the new advertising shown in Figure 4.2, where X (treatment) 1s
campaign and the measurement on
represents the market share at certain periodic dependent variab
(A to F) are intervals. Six different scenari0s
The case presented.
of one group
and the analysis of the pre-test-post-test
results would indicatedesign
some
would be shown as 0,
advertising campaign in situations A, B, Dand E, positive effects Of ulGand FC
advertising would be having any effect. The conclusion
series design would not
whereas in situations C time
in the case of
be
In situation A, the as follows:
share was sustained.campaign had ashort-run positive effect, after which market

FIGURE4,2
(%)
Share
Market
2. Multiple time series design: In this design, one more group
Multiple time series design
involves the addition of the
tontrol group' which is not
subjected to any treatment.
Experimnental Research Designs 81
70
60
50
B
-AC
40
30
20
10
2 3 4 X 5 6 7 8
Source: Adopted with modification from Thomas C. Kinnear &James R.
Taylor,
"Marketing Research: An Applied Approach" McGraw-Hil, Inc., Fifth Edition
" In situation B, the new advertising
campaign had a short-run positive effect.
The rise in market share was temporary. The
market share reverts to the level
which was there before the application of the
" In situation C, the treatment had a delayed treatment.
took longer time to appear.
positive effect and, accordingly, it
" In situation D, E, and F the changes that occur after the
application of
are in line with what occurred prior to the application of treatment. treatment
the new advertising campaign had no effect on the market share. Therefore,
Therefore it is seen that by taking multiple observations, the results have
altogether different interpretations and inferences.
called the 'control
group' is added to the time series design. The design may be diagrammed
symbolically as:
Experimental Group: X 0,
Control Group:
The experimental group is subjected to the treatment X, whereas the control
group is without any treatment. Taking the example of the sales training
programme, the sales training would represent treatment, and observations
OO, O,. would represent sales volume of this group. The test unit of the
control group would compromise sales people who are not sent for the training
programme. The measurement on the sales volume is denoted by 0,, o, 0,, .
etc. The measurement on the sales for both the groups is taken after the training
programme. The treatment effect (sales training) is found by comparing the
average sales of the two groups before and after the training programme. The
major drawback of this design is the possibility of the interactive effect in the
experimental group.
 82 Research Methodology
In the true experimental
design, the researcher is
able to elinmnate the effet
ofextraneous varnables from
both the experimental and the
control group
2. Post-test-only control group design: This design is also named as
True Experimental Designs
researchers can randomly assign test units and
is able to
designs, researcher
In true
experimental
treatments to an
experimental group.
HHere, the experimental
both the
from
and control elimgroup.
inate
extraneous variables researcher the use of: statistical
the effect of
Randonization procedure
allows the
Included in this category are the
techniques
for
following:
experimentalresults. design is also called before-after with
analysingthe group: This
Pre-test-post-test control
1. symbolicallypresernted as:
controlgroup. It is R 0, X
Experimental Group: O,
ControlGroup:
and control group are selected at
test units in both experimental subjected| to the treatmentX.
In this design, time. The experimental group is
random at the same there is no treatment
applied. Pre-test measurements
the control group, and control group at the same time,
whereas in experimental
taken in the the experimental
0, and 0, are measurements 0, and O, are taken for eauall
Similarly, post-test time. Allthe extraneous variables operate
the control group at the
same randomization. Therefor
experimental and control group because of
on both the treatment in the experimental
the only difference in the two groups is the effect of
of experimental
group.
difference in the post-test and pre-test measurements
If the
and control group is denoted
by A and B respectively, then
extraneous variables
A=0,-0,= Treatment +
variables
B =0, -0,= Extraneous
include history, maturation, testing.
The extraneous variables would and test unit mortality.
instrumentation, statistical regression, selection bias
effect would be present
However, it may be worth noting that the interactive testing the control group. This
missing in
only in the experimental group and would'be
subjected to the treatment. Therefore
is because only the experimental group is would include interactive
A -B= (O, - 0) - (0, - 0,) = treatment effect which
testing efect. Therefore, it is doubtful to generalize the results of the experiment.
after-only with
one control group and is presented symbolically as:
Experimental Group: R X 0,
Control Group: R
Here, the test units in both the experimental and the control group are selected
at random. The experimental group is subjected to the treatment X, and post-test
measurements are taken on both experimental (O,) and control group (0,)at tne
same time. The post-test measurement (0,) on experimental group comprises
treatment effect and all other extraneous variables, whereas O, comprises only
extraneous variables. Therefore, the difference in the post-test measurement of
experimental and control group is taken as a measure of treatment effect. Hente
0, -, = (Treatmenteffect +extraneous factors)- (extraneous factors)
Treatment effect
As pre-test measurement is absent, the effect of instrumentation
andinteracuvo
testing effect is ruled out. As there is a random assignment of test units to both uio
groups, itcan be approximately assumed that both the groups were equal prior o

TheSolomonfour-group
toas"deal
designisreferred as it
ontrolledexperiment"
remove
Nolns the researcher to
extraneous
theinfluenceof the
wanablesandthatof
nteractivetestingeffect.
Experimental Research Designs 83
he application of treatment to the
assume that the test units' mortalityexperimental group. Further, one can always
affects each group equally.One can always
Justity these assumptions by taking a large
widely used in marketing research. randomized sample. This design is
3. Solomon four-group
design: This design is also called four-group
design. This is also referred to as 'ideal controlled six-study
this design helps the researcher to remove the experiment. As will be seen,
and also that of the interactive testing effect. Thisinfluenceis extraneous variables
of
design symbolically presented
as:
Experiment Group 1 R X
Control Group 1 R
Experiment Group 2 R X
Control Group 2 R
In the above design test units are selected at random in all the
four groups. It is
seen that the experimental group 2 and control group 2 are not given any
pre-test
measurement, whereas experimental group land control group 1 are subjected
to pre-test measurement 0, and O, respectively. Both experimental groups l and
2 are subjected to the same treatment Xat the same time.
As the experimental group 2 and control group 2 are not subjected to pre
test measrement, we would need their estimates to remove the influence of
extraneous variables and interactive testing effect. As test units fron all the
four groups are chosen at random, it can be assumed that all the four groups
are equal before experiment. Therefore, the pre-test measurements 0, and 0,
on experimental and control group l can be used as an estimate of the pre-test
measurement of experimental and control group 2. The results of difference of
various post-test and pre-test measurement would give the following results:
Experimental Group 1:
0,- 0= Treatment effect +extraneous factors without interactive
testing effect +interactive testingeffect ..)
Control Group 1:
O, -0,= Extraneous factors without interactive testing effect ..(i)
As this group was not subjected to any treatment, there would not be any
interactive testing effect.
Experimental Group 2:
0.-0,=Treatment effect +extraneous factors without interactive
testing effect ..tii)
0. -0,= Treatment effect + extraneous factors
without testing effect ..iv)
As there was actually no pre-test measurement, the interactive testing effect
cannot Occur here.
Control Group 2:
0,-0=(Extraneous factors without testing effect) ..(v)
..(vi)
0,-0,= (Extraneous factors without testing effect)
 84 Research Methodoloay
The Solomon four-group
design is usetul for businesses
where establshing cause-and
eftect relationship is crucal for
Survval
Compieteily randomized
design alloWs aresearcher to
nvestigate the effect of one
ndependet variable on the
dependent variable.
the difference in
subjected to any treatment,
As the group was not
would only indicate the
effect of extraneous
factors
measuretment
without interactive
esting
effect.
(v) and (vi), one gets:
By taking the average of
0, +0, =(Extraneous factors without testing effect)
2
(iii) and (iv), one obtains:
By taking the average of
o-9Treatment effect +extraneous factors withouttesting effect
2
one obtains:
..vii)
By subtracting (vii) from (viii),
lo-,-0-0,= Treatment efect
By subtracting (vii) from (i), one obtains:
q,-0,-0,-Interacting testing effect
Therefore, this design has helped not only in measuring the effect of treatment.
but also in obtaining magnitude of the interactive testing effect and extraneous
factors.
To conduct this experimental design, the time and cost required are enormous
and therefore, this design is not commonly used in research. However, as seen,
this experimental design guarantees the maximum internal validity. In businesses
where establishing cause-and-effect relationship is very crucial for survival, this
design is useful.
Statistical Designs
Statistical designs allow for statistical control and analysis of external variables. The
main advantages of statistical design are the following:
The effect of more than one level of independent variable on the dependent
variable can be manipulated.
" The effect of more than one independent variable can be examined.
" The effect of specific extraneous variable can be controlled.
Included in this category are the following designs:
1. Completely randomized design: This design is used when a researcher is
investigating the effect of one independent variable on the dependent variable.
The independent variable is required to be measured in nominal scale i.e. it
should have a number of categories. Each of the
variable is considered as the treatment. The basiccategories
of the independent
assumption of this design S
that there are no differences in the test units. All the test units are treated alike and
randomly assigned to the test groups. This means that there are no
variables thatcould influence the outcome. extraneous
Suppose we know that the sales of a product is
In this case, sales are a dependent variable and influenced by the price level.
the price is the independent
variable. Let there be three levels of price, namely, low,
wish to determine the most effective price level, ie.. at medium and high. we
which price level the sale
 Erpernental Research Designs 85

is highest. Here the test units are the stores which are randomly assigned to the
three treatment 1levels. The average sales for each price level is computed and
examined to see whether there is anysignificant difference in the sale at various
price levels. The statistical technique to test for such a difference is called analysis
of variance (ANOVA).
mtaion of the This design suffers from the main limitation that it does not take into account
h aan
completely tandomized the effect of extraneous variables on the dependent
does not variable. The possible
design s ihat it extraneous variables in the present example could be the size of the store, the
cffect of
ake nto a«ount the competitor's price and price of the substitute product in question. This design
variables on the
NaneoU assumes that all the extraneous factors have the same influence on all the test
enendent variable units which may not betrue in reality. This design is very simple and inexpensive
to conduct.

2. Randomized block design: As discussed, the main limitation of the completely

randomizeddesignisthatallextraneousvariableswere assumedtobeconstantover
all the treatment groups. This may not be true. There may be extraneous variables
influencing the dependent variable. In the randomized block design it is possible
to separate the influence of one extraneous variable on a particular dependent
variable, thereby providing a clear picture of the impact of treatment on test
units.
In the example considered in the completely randomized design, the price level
(low, medium and high)was considered as an independent variable and all the
test units (stores) were assumed to be more or less equal. However, all stores may
not be of the same size and, therefore, can be classified as small, medium and
large size stores. In this design, the extraneous variable, like the size of the store
could be treated as different blocks. Now the treatments are randomly assigned to
the blocks in sucha way that each treatment appears in each block at least once.
The purpose of forming these blocksis that it is hoped that the scores of the test
units within each block would be more or less homogeneous when the treatment
is absent. What is assumed here is that block (size of the store) is correlated with
the dependent variable (sales). It may be noted that blocking is done prior to the
application of the treatment.
In this experiment one might randomly assign 12 small-sized stores to three
ina randomized block price levels in such a way that there are four stores for each of the three price
Design, it is assumed that levels. Similarly, 12 medium-sized stores and 12 large -sized stores may be
block is correlated with the
dependent variable. Blocking is randomly assigned to three price levels. Now the technique of analysis of variance
done prior to the application of could be employed to analyse the effect of treatmeñt on the dependent variable
the
the treatment. and to separate out the influence of extraneous variable (size of store) from
experiment.
the researcher is interested
3. Latin square design: This design is employed when variables. Suppose the interest
in separating out the influence of twoextraneous
Latin square design has Let there be three levels of
avery cormplex setup and is isto study the influence of price (treatment) on sales. (X)). The sales could be
and high
quite expensive to execute but price categories, namely, low (X)), medium (X) store size and type of packaging.
theips to rmeasure statistically influenced by two extraneous variables, namely, number of categories of two
the
For the application of the Latin square design,
the effect of atreatment on the
number of levels of treatments. This
dependent variable. extraneousvariables should be egual to the store could be of
necessary condition for the use of Latin square design. The
is a
and type of packaging could be I, II and
(3)
size - small (1), medium (2) and largelayout of the Latin square design.
III. The Table 4.1 below presents the
 86
Research Methodology
TABLE 4.1
Latinsquare deslqn for
various levels of price
Afactorial design is
4. Factorial design: A factorial design may be employed to measure the
employed to measure
the effect of two or more
independent variables at
various levels.
TABLE 4.2
3x2 factorial design for
price level and store size
Store Size
Packaging
1(Small) X, X, X,
2 (Medium) X, X, X.
3 (Large) X X X
It may be noted that the rOws and columns represent those
extraneous variable.
whose effect is to be controlled and measured. There are three categories of rou
variable (size of store) and three categories of column variable (type of packaging)
This would result in 3 x 3 Latin square.
One point that has to be kept in mnind is that the treatment should be assion ed
randomly tocells in such a way that each treatment occurs once and only once in
each row and in each column. The treatments exhibited in Table 4.1 satisfy this
condition.
Use of this design helps to measure statistically the effect of a treatment on
the dependent variable andalso the measurement of an error resulting from two
extraneous variables. This design, indeed has a very complex setup and isquite
expensive to execute.
effect of
two or more independent variables at various levels. The factorial designs allow
interaction between the variables. An interaction is said to take place when the
simultaneous effect of two or more variables is different from the sum of their
individual effects. An individual may have a high preference for mangoes and may
also like ice-cream, which does not mean that he would
like mango ice cream,
leading to an interaction.
The sales of a product may be influenced by two factors,
and store size. There may be three levels of price--low namely, price level
(A,), medium (A) and
high (A,). The store, size could be categorized into small (B,) and big
could be conceptualized as a two-factor design with (B,). This
form of a table. In the table, each level of one information reported in the
and each level of another variable would be factor may be presented as a roW
presented as a
could be summarized in the form of a table having three column. Thisexample
This would require 3 x 2 = 6 cells. Therefore, six rows and twO columns.
different levels of treatment
combinations would be produced, each with a specific
size. The respondents would be randomly selected and level of price and store
six cells. The tabular presentation of 3 × 2 randomly assigned to the
factorial design is given in Table 4.2.
Price Store
Small (B,)
Low Level (A,)
Big (B,)
A,B, A,B,
Medium Level (A,) A,B, A,B,
High Level (A,) A,B, A,B,
Respondents in each cell receive a specified treatment
example, respondents in the upper left hand corner cell
combination. ro
price and small store. Similarly, the would face small level o
cell will be respondents in the lower right hand
subjected to both high price level and big store. cornel
 Experinental Research Designs 87

The main advantagesof factorialdesign are:

" It is possible to measure the main effects and interaction effect of two or more
independent variables at various levels.
It allows a saving of time and effort because all observations are employed to
study the effects of each factor.
" The conclusion reached using factorial design has broader applications as each
factor is studied with different combinations of other factors.
The limitation of this design is that the number of combinations (number of cells)
increases with increased number of factors and levels. However, a fractional
tactorial design couldbeused ifinterestisin studyingonlyafewofthe interactions
or main effects.

1. How would you control the appearance of extraneous variables in an experiment?

CONCEPT 2 What is the influence exerted by an environment upon the conducting of an experiment?
CHECK 3
Glassify and segregate the various types of experimental designs. Which, according to you, is the most
effective and why?

SUMMARY

¢ Experiments are used to infer causality where the researcher actively manipulates one or more causal variables
and measure their effects on the dependent variable. There are three necessary conditions for inferring causality: 0)
concomitant variation (i) time order of occurrence of variables, and (ii) the absence of other possible causal factors.
Various concepts like independent variables (treatments), test units, dependent variables, exogenous variables
are used in conducting an experiment. An experiment can be conducted under different environmental conditions,
namely, laboratory and field. The researcher has two goals while conducting an experiment: () to keep the internal
wider popu
validity of the experiment very high and (ii) to make generalization of the results of the experiments to a wh0se influ
the causal factors except the one
lation. Internalvalidity is concerned with examining the absence of all
refers to the generalization
ence is being examined on the dependent variable. External validity, on the other hand, These are
validity of the experiment.
of the results of the experiment. There are various factors affecting the internal
statistical regression, selection bias and test units' mortality. Similarly,
history, maturation, testing, instrumentation,
factors may be common to both the
there are factors influencing the external validity of an experiment. Some of the
effects of extraneous variables
internal and the external validity of the experiment. The methods of controlling the
are also discussed.
true-experimental, and statistical
Experimental designs are classified into pre-experimental, quasi-experimental, (ii) one-group pre-test-post-test
design. Under pre-experimental design are included (0) one-shot case study,
designs do not make use of randomization pro
design and (ii) static group comparison. The pre-experimental
the internal validity of such experiments remains
cedure in order to control the extraneous variables. Therefore,
series design and (ii) multiple time series de
doubtful. Under quasi-experimental design are discussed (i) time
when the measurements are to be taken and on whom
sign. In these designs the researcher has control over ability to
they are taken. However, the design lacks complete control of scheduling of treatment and also lacks
category of true-experimental design are (i) pre
randomize test units exposure to treatments. Included in the
and (ii) Solomon four-group design. In these de
test-post-test control group, (ü) post-test-only control group researcher is
units and treatments to experimental groups. The
Signs, the researcher can randomly assign test statistical de
to eliminate the effect of extraneous variables from both control and experimental groups. The design,
able Latin square
randomized design, (iü) randomized block design, (ii) level of independent
SIgns covered here are (i) completely effect of more than one
designs help to (i) study the
and (iv) factorial design. The statistical study the effect of more than one independent variable and (ii) the effect
Variables on the dependent variable: (ii)
of specific extraneous variables.