AK0047

ACCOUNTING
RESEARCH
METHODOLOGY
CHAPTER 8
Experimental Designs

ACCOUNTING
PROGRAM
Learning Objective
After completing Chapter 8 students should be able to:
• Describe lab experiments and discuss the internal and
external validity of this type of experiment.
• Describe field experiments and discuss the internal and
external validity of this type of experiment.
• Describe, discuss, and identify threats to internal and
external validity and make a trade-off between internal
and external validity.
• Describe the different types of experimental designs.
• Discuss when and why simulation might be a good
alternative to lab and field experiments.
• Discuss the role of the manager in experimental designs.
• Discuss the role of ethics in experimental designs.
Subject
1. The Lab Experiment
2. The Field Experiment
3. External and Internal Validity in Experiment
4. Types of Experimental Design and Validity
5. Simulation
6. Ethical Issues in Experimental Design
Research
7. Manager Implication
The Lab Experiment
• Lab experiment is done in an artificial or
contrived environments.
• It is also necessary to manipulate the independent
variable so that the extent of its casual effects
can be established.
• The control and manipulation are best done in
an artificial setting (the laboratory), where the
causal effects can be tested.
• When control and manipulation are introduced to
establish cause-and-effect relationships in an
artificial setting, we have laboratory experimental
designs, also known as Lab experiments.
The Lab Experiment
Control
• When we postulate cause-and-effect
relationships between two variable X and
Y, it is possible that some other factors,
say A, might also influence the
dependent variable Y. In such a case, it
will not be possible to determine the
extent to which Y occurred only because
of X, since we do not know how much of
the total cariation in Y was caused by the
presence oh the factor A.
The Lab Experiment
Manipulation
• Manipulation simply means that we
create different lavels of the independent
variable to assess the impact on the
dependent variable.
• The manipulation of the dependent
variable is also known as the treatment,
and the results of the treatment are
calles treatment effects.
The Lab Experiment
Controlling the contaminating
exogeneous or “nuisance” variables
1. Matching groups
One way of controlling the
contaminating or “nuisance” variables
is to match the various group by
picking the confounding characteristics
and deliberately spreading the across
groups.
The Lab Experiment
Controlling the contaminating
exogeneous or “nuisance” variables
2. Randomization
In randomization, the process by which
individuals are drawn and their
assignment to any particular group are
both random.
By thus randomly assigning members
to the groups we are distributing the
confounding variables among the group
The Lab Experiment
Controlling the contaminating exogeneous or
“nuisance” variables
3. Advantages of randomization
• The process of randomization will distribute the
inequalities among the groups, based on the laws
of normal distributions.
• Randomizations will take care, since all the
contaminatinf factore will be spread across all
groups.
• Randomizations solves through the proves of their
matching or randomization, and the manipulation
of the treatment.
The Lab Experiment
Internal Validity of Lab Experiments
• Internal validity refers to the confidence
we place in the cause-and-effect
relationship.
• In research with high interal validity,
causality connot be inferred at all.
• In lab experiments where cause-and-
effect relationships are substantiated,
internal validity can be said to be high.
The Lab Experiment
External validity or generalizability of
lab experimets.
• If we do find a cause-and-effect
relationship after conducting a lab
experiment, can we them confidently say
that the same-caused-effect relationship
will alsohold true in the organizational
setting
The Field Experiment
• A field experiment, as the name
implies, is an experiment done in the
natural environment in which work (or
life) goes on as usual, but treatments are
given to one or more groups.
• In the Field Eksperiment even though it
may not possible to control all the
nuisance variables because members can
not be either randomly assigned to
groups, or matched, the treatment can
still manipulated.
External and Internal
Validity In Experiments
• External validity refers to the extent of
generalizability of the results of a causal
study to other settings, people, or
events.
• Internal validity refers to degree of our
confidence in the causal effects.
• Fields experiments have more external
validity, but less internal validity.
• In the lab experiments the reverse is
true: the internal validity is high but the
external validity is rather low.
External and Internal
Validity In Experiments
Trade-off between internal and external
validity
• If we want high internal validity, we should
be willing to sttle for lower external validity
and vice versa.
• To ensuer both types of validity, researchers
usually try first tp test the causal
relationsheps in a tightly controlled artificial
or lab setting, and once the relationship has
been established, they try causal relationship
in a field experiment.
External and Internal
Validity In Experiments
Factors affecting the validity of
experiments
• Even the best designed lab studies may be
influenced by factors that might affect the
internal validity of the lab experiment.
• That is, some confounding factors might still
be present that could offer rival explanations
as to what is causing the dependent variable.
• These possible confounding factors pose a
threat to internal validity.
External and Internal
Validity In Experiments
Factors affecting the validity of experiments
The seven major threats to internal validity are:
1. History effects
2. Maturation effects
3. Testign effects
4. Selection effects
5. Mortality effects
6. Statistical regression effects
7. Intrumentation effects
Types of Experimental
Design and Validity
• Quasi-experimental designs
 Pretest and posttest experimental group design
 Posttests only with experimental and control groups
 Time series design
• True experimental designs
 Pretest and posttest experimental and control group
design
 Solomon four-group design
 Solomon foru-group design and threats to valifity
 Double-blind studies
• Ex post facto designs
 Types of Experimental
Design and Validity
• Quasi-experimental designs
Experimental is the weakest of all designs, and it does not
measure the true cause-and –effect relationship.
 Pretest and posttest experimental group design
An experimental group may be given a pretest, exposed to a
treatment, and then given a posttest to measure the effects to
the treatment.
 Posttests only with experimental and control groups
Some experimental designs are set up with an experimental and
a control group, the former alone being exposed to a treatment
and not the latter. The effects of the treatment are studied by
assessing the difference in the outcomes-that is, the posttest
scores of the experimental and control groups.
 Time series design
A time series design thus allows the reasercer to assess the
impact of a treatment over time.
Types of Experimental
Design and Validity
• True experimental designs
Experimental designs that include both the
treatment and control groups and record
information both before and after the experimental
group is exposed to the treatment, are known as
ex post faco experimental designs.
 Pretest and posttest experimental and control
group design
 Solomon four-group design
 Solomon foru-group design and threats to valifity
 Double-blind studdies
Simulation
• A simulation can be thought of as an
experiment conducted in a specially
created setting that very closely
represents the natural environment in
which activities are usually carried out.
• In that sense, the simulation lies
somewhere between a lab and a field
experiment, insofar as the environment is
artificially created but not too different
from “reality.”
Ethical Issues in
Experimental Design
Research
It is appropriate at this juncture to briefly discuss a few of
the many ethical issues involved in doing research, some a
which are particularly relevant to conducting lab
experiments.
The following practices are considered unethical:
• Putting pressure on individuals to participate in
experiments through coercion, or applying social pressure.
• Giving menial tasks and asking demeaning questions that
diminish participants’ self respect
• Deceiving subjects by deliberately misleading them as to
the true purpose of the research.
• Exposing pasticipants to physical or mental stress.
• Not allowing subjects to withdraw from the research when
they want to. (continued…)
 Ethical Issues in
Experimental Design
Research
It is appropriate at this juncture to briefly discuss a few of the
many ethical issues involved in doing research, some a which
are particularly relevant to conducting lab experiments.
The following practices are considered unethical: (…
continued)
• Using the research result to disadvantage the participants,
or for purpose not to their liking.
• Not explaining the procedures to be followed in the
experiment.
• Exposing respondents to hazardous and unsafe
environments.
• Not debriefing participants fully and accurately after the
experiment is over.
• Not preserving the privacy and confidentiality of the
information given by the participants
Managerial Implication
A good knowledge of experimental designs
could foster some pilot studies to be
undertaken to examine whether factor
such as bonus system, piece rates, rest
pauses, and so on lead the positive
outcomes sich as better motivation.
Improved job performance, and other
favourable working condition at the
workplace.
Manager could use experimental desings
to study the effect on sales
Questions and Answers