You are on page 1of 41

Research...

The systematic application of a family of methods employed to provide trustworthy information about problems
an ongoing process based on many accumulated understandings and explanations that, when taken together lead to generalizations about problems and the development of theories

The basic steps of research...
Scientific and disciplined inquiry is an orderly process, involving: • recognition and identification of a topic to be studied (“problem”)

• description and execution of procedures to collection information (“method”)
• objective data analysis • statement of findings (“results”)

Research methods...
Quantitative… …collects and analyzes numerical data obtained from formal instruments

Quantitative methods...


descriptive research (―survey research‖) correlational research experimental research

applies the treatment(s) to the groups and measures the effects upon the groups .experimental research  …the researcher selects participants and divides them into two or more groups having similar characteristics and. then.

Conducting an experimental study… select the problem select participants and instrument selection and execution of a research plan data analysis and formulation of conclusions .

comparison of an existing approach to a new approach (A and ~ A) 3.Types of experimental comparison… 1. comparison of differing amounts of a single approach (A and a or a and A) . comparison of two different approaches (A versus B) 2.

where: A – experimental (―treatment‖) group B – control (―no treatment.‖ ―nonmanipulated‖) group .

g. aptitude) that can assume any one of a range of values .. intelligence..Variable. height..  …a concept (e.

.Research variables. Independent… …an activity of characteristic believed to make a difference with respect to some behavior …(syn. active variable.. cause. treatment .) experimental variable.

assigned variable. outcome. effect.Dependent… …the change or difference occurring as a result of the independent variable …(syn.) criterion variable. posttest .

effect. assigned variable.) criterion variable. outcome.Confounding… …the fact that the effects of the independent variable may intertwine with extraneous variables. such that it is difficult to determine the unique effects of each variable …(syn. posttest .

Validity of Experimental Design   Internal validity—The accuracy in concluding that the outcome of an experiment is due to the independent variable External validity—The extent to which the results of an experiment can be generalized .

INTERNAL OR EXTERNAL VALIDITY: A TRADE-OFF?    Too much control reduces ability to generalize Too little control reduces ability to make causal statements Attempt to find a good balance Internal Validity External Validity .

The events occurring between the first and second measurements in addition to the experimental variable which might affect the measurement. Example: Researcher collects gross sales data before and after a 5 day 50% off sale.Factors Jeopardizing Internal Validity  History . . not the sale. During the sale a hurricane occurs and results of the study may be affected because of the hurricane.

and their responses on the Posttest are affected. growing more tired. or similar changes. Example: Subjects become tired after completing a training session.The process of maturing which takes place in the individual during the duration of the experiment which is not a result of specific events but of simply growing older.  Maturation . .

. Example: Subjects take a Pretest and think about some of the items.  Pre-testing . On the Posttest they change to answers they feel are more acceptable. Experimental group learns from the pretest.The effect created on the second measurement by having a measurement before the experiment.

observers. . 5th. calibration of instruments.Changes in instruments. or scorers may cause changes in the measurements. Measuring Instruments . 6th become fatigued and are less careful and make errors. Example: Interviewers are very careful with their first two or three interviews but on the 4th.

. Example: Managers who are performing poorly are selected for training. Their average Posttest scores will be higher than their Pretest scores because of statistical regression. even if no training were given. those scores or measurements tend to move toward the mean with repeated measurements even without an experimental variable.  Statistical Regression .Groups are chosen because of extreme scores of measurements.

Example: A group of subjects who have viewed a TV program is compared with a group which has not. .Different individuals or groups would have different previous knowledge or ability which would affect the final measurement if not taken into account.  Differential Selection . There is no way of knowing that the groups would have been equivalent since they were not randomly assigned to view the TV program.

Not only is there differential loss in the two groups. 12 accountants drop out of the experimental group and none drop out of the control group. but the 12 dropouts may be very different from those who remained in the experimental group. Groups to be compared need to be the same after as before the experiment. .The loss of subjects from comparison groups could greatly affect the comparisons because of unique characteristics of those subjects. Example: Over a 6 month experiment aimed to change accounting practices. Experimental Mortality .

.Combinations of these factors may interact especially in multiple group comparisons to produce erroneous measurements. . Interaction of Factors. such as Selection Maturation. etc.

Taking the Pretest may increase the effect of the film. a group of subjects is given a 60 item antichemical test. The film may not be effective for a nonpretested group. Example: Prior to viewing a film on Environmental Effects of Chemical.Factors Jeopardizing External Validity or Generalizability  Pre-Testing -Individuals who were pretested might be less or more sensitive to the experimental variable or might have "learned" from the pre-test making them unrepresentative of the population who had not been pre-tested. .

Thus subjects in the 12th corporation may be more accepting or sensitive to the treatment. Differential Selection . The 12th corporation is obviously different then the others because they accepted. but the 12th corporation grant permission. . is turned down by 11 corporations.The selection of the subjects determines how the findings can be generalized. requesting permission to conduct experiment. Subjects selected from a small group or one with particular characteristics would limit generalizability. Example: Researcher. Randomly chosen subjects from the entire population could be generalized to the entire population.

Generalization to persons not in the experimental setting may be precluded. . Example: Department heads realize they are being studied. Experimental Procedures .The experimental procedures and arrangements have a certain amount of effect on the subjects in the experimental settings. try to guess what the experimenter wants and respond accordingly rather than respond to the treatment.

Example: A group of CPA’s is given training in working with managers followed by training in working with comptrollers. the first training will affect the second.If the subjects are exposed to more than one treatment then the findings could only be generalized to individuals exposed to the same treatments in the same order of presentation. Multiple Treatment Interference . Since training effects cannot be deleted. .

Tools of Experimental Design Used to Control Factors Jeopardizing Validity   Pre-Test . or measurement before the experiment begins. It can aid control of experimental mortality because the subjects can be removed from the entire comparison by removing their pre-tests. pre-tests cause problems by their effect on the second measurement and by causing generalizability problems to a population not pretested and those with no experimental arrangements.The pre-test. . However. can aid control for differential selection by determining the presence or knowledge of the experimental variable before the experiment begins.

Maturation. The control group is exposed to all conditions of the experiment except the experimental variable. Instrumentation. . Control Group -The use of a matched or similar group which is not exposed to the experimental variable can help reduce the effect of History. and Interaction of Factors.

Differential Selection. Randomization . It greatly increases generalizability by helping make the groups representative of the populations .Use of random selection procedures for subjects can aid in control of Statistical Regression. and the Interaction of Factors.

They would have to be used in conjunction with other pre-tested groups or other factors jeopardizing validity would be present. . Additional Groups .The effects of Pre-tests and Experimental Procedures can be partially controlled through the use of groups which were not pre-tested or exposed to experimental arrangements.

Group experimental designs… 1. single-variable 2. factorial .

single-variable designs …involve one manipulated independent variable  pre-experimental  quasi-experimental  true experimental .

greater control of validity Quasi-Experimental Design .not randomly selected . could be biased True Experimental Design .loose in structure.greater control and refinement.Experimental Designs    Pre-Experimental Design .

Weakness lies in no examination of preexperimental equivalence of groups. Conclusion is reached by comparing the performance of each group to determine the effect of a variable on one of them.Pre-Experimental Design Aim of the Research Name of the Design Notation Paradigm X» O Comments An approach that prematurely links antecedents and consequences. An approach that provides a measure of change but can provide no conclusive results.» O .O » X » O posttest Static group comparison Group 1: X » O Group 2: . To attempt to One-shot explain a consequent experimental case by an antecedent study To evaluate the influence of a variable To determine the influence of a variable on one group and not on another One group pretest. The least reliable of all experimental approaches.

.True Experimental Design Aim of the Research To study the effect of an influence on a carefully controlled sample Name of the Design Pretest-posttest control group Notation Paradigm R--[O»X»O [O»-»O Comments This design has been called "the old workhorse of traditional experimentation." If effectively carried out. this design controls for eight threats of internal validity. Data are analyzed by analysis of covariance.

Data are analyzed by analysis of variance on posttest scores .» X » O [-»-»O Comments This is an extension of the pretest-posttest control group design and probably the most powerful experimental approach.Experimental design (cont) Aim of the Research To minimize the effect of pretesting Name of the Notation Design Paradigm Solomon four-group design R--[O»X»O [O»-»O [.

Randomness is critical. . the simplest and best test for significance in this design is the t-test. Probably.Experimental design (cont) Aim of the Research To evaluate a situation that cannot be pretested Name of the Notation Design Paradigm Posttest only control group R--[ X»O [-»O Comments An adaptation of the last two groups in the Solomon four-group design.

Quasi-Experimental Design Aim of the Research To investigate a situation in which random selection and assignment are not possible Name of the Notation Design Paradigm Nonrandomized control group pretestposttest O»X»O O»-»O Comments One of the strongest and most widely used quasi-experimental designs. Differs from experimental designs because test and control groups are not equivalent. Comparing pretest results will indicate degree of equivalency between experimental and control groups. .

To increase external validity. . repeat the experiment in different places under different conditions. then the variable can be suspect as to the cause of the change.Quasi-Experimental Design (cont) Aim of the Research Name of the Notation Design Paradigm Comments To determine the influence of a variable introduced only after a series of initial observations and only where one group is available Time series experiment O»O»X»O»O If substantial change follows introduction of the variable.

.Quasi-Experimental Design (cont) Aim of the Research To bolster the validity of the above design with the addition of a control group Name of the Design Control group time series Notation Paradigm O»O»X»O»O O»O»-»O»O Comments A variant of the above design by accompanying it with a parallel set of observations without the introduction of the experimental variable.

. sometimes absent.Quasi-Experimental Design (cont) Aim of the Research To control history in time designs with a variant of the above design Name of the Design Equivalent timesamples Notation Paradigm [X1 » O1] »[X0 » O2] » [x1 » O3] Comments An on-again. off-again design in which the experimental variable is sometimes present.