Educational Research: CausalCausal -Comparative Studies

Research...

The systematic application of a family of methods employed to provide trustworthy information about problems
an ongoing process based on many accumulated understandings and explanations that, when taken together lead to generalizations about problems and the development of theories

EDU 8603 Educational Research Richard M. Jacobs, OSA, Ph.D.

The basic steps of research...

Scientific and disciplined inquiry is an orderly process, involving: recognition g and identification of a topic to be studied (problem problem) description and execution of procedures to collection information (method method) objective data analysis statement of findings (results results)

Research methods...
Quantitative Quantitative collects and analyzes numerical data obtained bt i d from f formal f l instruments i t t

Quantitative methods...
descriptive research (survey research) correlational research causal-comparative research (ex post facto research) experimental research

causal-comparative research (ex post causalfacto research) at at least two different groups are compared on a dependent variable or measure of performance (called the effect) because the independent variable (called the cause) has already occurred or cannot be manipulated

Reflective Practice: Using Theory and Skill to Inform Practice

Research variables...
Independent Independent an activity of characteristic b li believed d to t make k a difference diff with respect to some behavior (syn.) experimental variable, cause, treatment dependent variable variable the change or difference occurring as a result lt of f the th independent i d d t variable (syn.) criterion variable, effect, outcome, posttest

A causal causal-comparative study

a study in which the researcher attempts to determine the cause, or reason, for f pre-existing i ti differences diff in i groups of individuals called an ex post facto study because both the effect and the alleged cause have already occurred and must be studied in retrospect

Differences in causalcausal-comparison and correlational studies

causal-comparative studies
attempt to identify cause-effect p relationships

correlational studies
attempt to identify relationships

causal-comparative studies
involve two (or more) groups and one p variable independent

causal-comparative studies
involve making comparisons

correlational studies studies

involve establishing relationships

correlational studies
typically involve two (or more) variables and one group

Reflective Practice: Using Theory and Skill to Inform Practice

Differences in causalcausal-comparison and experimental studies

causal-comparative studies
individuals are not randomly selected but y belong g to g groups p selected because they

causal-comparative studies
the researcher cannot manipulate the p variable independent

experimental studies
individuals are randomly selected and assigned to two (or more) groups

experimental studies
the researcher manipulates the independent variable

causal-comparative studies
the independent variable has already p occurred and cannot be manipulated

causal-comparative studies
the random sample is selected from two y g populations p p already-existing

experimental studies
the researcher manipulates the independent variable to determine its effects

experimental studies
the random sample is selected from a single population

Conducting a causalcausal-comparative study

1. select the problem 2. select participants and instrument 3. design and procedure 4. data analysis and interpretation 1. select the problem the researcher starts with an effect and seeks its causes the independent variable cannot or should not be manipulated

Reflective Practice: Using Theory and Skill to Inform Practice

2. select the participants and instrument select samples representative of their respective populations and similar with respect to critical variables other than the independent variable called comparison comparison groups groups

3. design and procedure the performance of the groups is compared using some valid dependent variable measure (instrument instrument) lack of randomization, manipulation, and control are sources of weakness

control the process by which the researcher attempts to ensure that the findings are as free of researcher bias and error as possible

types of control random assignment of participants to g groups p pair-wise matching comparing homogeneous groups comparing homogeneous subgroups factorial analysis of variance analysis of covariance

random assignment of participants to groups not possible in causal-comparative studies because the groups already exist and have already received the treatment

pairpair -wise matching

first: find a participant in the second (third, f fourth, th etc.) t ) group with ith the th same or similar i il score on the control (nonmanipulated) variable as the participant in the first group second: if a participant in either group does not have a suitable match, the participant is eliminated from the study

Reflective Practice: Using Theory and Skill to Inform Practice

comparing homogeneous groups control for extraneous variables that are homogeneous with respect to the extraneous variables limitation: lowers the number of participants in the study and, of course, limits the generalizability of the findings

comparing homogeneous subgroups

form subgroups within each group that p all levels of the control represent (nonmanipulated) variable controls for the variable and also permits the researcher to determine whether the independent variable affects the dependent variable differently at different levels of the control (nonmanipulated) variable

factorial analysis of variance (FANOVA)

building building the control (nonmanipulated) variable into the research design then use FANOVA to analyze the results to determine the effect of the independent and control (nonmanipulated) variable on the dependent variable, both separately and in combination

FANOVA allows the researcher to determine if there is an interaction b t between the th independent i d d t variable i bl and the dependent variable such that the independent variable operates differently at different levels of the independent variable building it into the research design

analysis of covariance (ANCOVA)

statistically adjusts initial group p variable for differences on a dependent initial differences on some other variable related to performance on the dependent variable removes initial differences so that the results can be fairly compared as if the two groups started equally

symbolic representation of the basic causalcausal -comparative design

Group (E) (C) Independent Variable (X) Dependent Variable O O

Where: E (experimental group); C (control group); X (independent variable); O (dependent variable)

Reflective Practice: Using Theory and Skill to Inform Practice

Group (E) (C)

Independent Variable (X1) (X2)

Dependent Variable O O

Where: E (experimental group); C (control group); X (independent variable); O (dependent variable)

4. Data analysis and interpretation researcher uses a variety of descriptive and inferential statistics: t-test mean analysis of standard variance deviation chi squared

mean the descriptive statistic indicating the average performance of an individual or group on a measure of some variable

standard deviation the descriptive statistic indicating the spread of a set of scores around the mean

t-test the inferential statistic indicating whether the means of two groups are significantly different from one another

analysis of variance (ANOVA) the inferential statistic indicating the presence of a significant difference among the means of three or more groups

Reflective Practice: Using Theory and Skill to Inform Practice

MiniMini -Quiz

chi squared (2) the inferential statistic indicating that there is a greater than expected difference among group frequencies

True and false causal-comparative causal-comparative studies attempt to identify the causeeffect relationships; correlational studies do not
True

causal-comparative studies typically involve two (or more) groups and one independent variable, variable whereas correlational studies typically involve two (or more) variables and one group
True

causal-comparative studies involve relation, whereas correlational studies involve cause

False

oftentimes, causal-comparative research is undertaken because the independent variable could be manipulated but should not
True

one of the most important reasons for conducting causal-comparative research is to identify variables worthy of experimental investigation
True

Reflective Practice: Using Theory and Skill to Inform Practice

lack of control means that the researcher can and should manipulate the independent variable
False

each group in a causal-comparative study represents a different population

True

the more similar two groups are on all relevant variables except the independent variable, variable the stronger the study is
True

there is random assignment to treatment groups from a single population in causal causal-comparative comparative studies
False

lack of randomization, manipulation of the independent variable, and control are all sources of weakness in a causal-comparative design
True

matching, comparing homogenous groups or subgroups, and covariate analysis are strategies that enable researchers to overcome problems of initial group differences on an extraneous variable
True

Reflective Practice: Using Theory and Skill to Inform Practice

interpretation of the findings in a causal-comparative study requires considerable caution because the cause may be the effect and the effect may be the cause
True

extraneous variables or confounding factors may be the real cause of both the independent and dependent variables
True

Fill in the blank groups groups selected for a causalcomparative study which differ on some independent variable and comparing them on some dependent variable
comparison groups

Fill in the blank unexplained unexplained variables that influence a dependent variable
confounding factors extraneous variables

Fill in the blank a a method for controlling extraneous variables by comparing groups that are homogeneous with respect to the extraneous variable
comparing homogeneous groups

Fill in the blank a a method for controlling extraneous variables by forming subgroups within each group that represent all levels of the control variable
comparing homogeneous subgroups

Reflective Practice: Using Theory and Skill to Inform Practice

Fill in the blank a a statistical tool to determine the effects of the independent variable and the control variable on the dependent variable, both separately and in combination
factorial analysis of variance

Fill in the blank a a statistical tool to adjust initial group differences on variables
analysis of covariance

Fill in the blank the the descriptive statistic indicating the average performance of a group on a measure of some variable
mean

Fill in the blank the the descriptive statistic indicating how clustered or spread out around the mean a set of scores is
standard deviation

Fill in the blank the the inferential statistic determining whether there is a significant difference between the means of two groups
t-test

Fill in the blank the the inferential statistic determining whether there is a significant difference between the means of three or more groups
analysis of variance

Reflective Practice: Using Theory and Skill to Inform Practice

10

Fill in the blank the the inferential statistic determining whether there is a greater than expected difference among group frequencies
chi squared

Fill in the blank activities activities by which a researcher endeavors to ensure that the results of a causal-comparative study are not tainted by extraneous variables
control

This module has focused on...

causal-comparative studies
which identify the cause, or reason, for existing differences in the behavior or status of groups

The next module will focus on...

experimental studies
...which test hypotheses to establish cause-and-effect relationships

Reflective Practice: Using Theory and Skill to Inform Practice

11