MA in TESOL

Course: TESOL 2002

Lecturer: SAM Chanphirun

Chapter Review: Validity and Reliability

Jack R. Fraenkel (2012) Chapter 8

Submitted by: Din Koldevinkea

The Importance of Valid Instrumentation

 The quality of the instruments used in research is very important, for the conclusions researchers draw are
based on the information they obtain using these instruments.
 Validity refers to the appropriateness, meaningfulness, correctness, and usefulness of the inferences a
researcher makes.
 Reliability refers to the consistency of scores or answers from one administration of an instrument to another,
and from one set of items to another.

 Both concepts are important to the selection or design of the instruments a researcher intends to use.

Validity is the most important idea to consider when preparing or selecting an instrument for use, for researchers want
the information they obtain through the use of an instrument to serve their purposes.
Validation is the process of collecting and analyzing evidence to support such inferences.
The important point here is to realize that validity refers to the degree to which evidence supports any inferences a
researcher makes based on the data he or she collects using a particular instrument.
But unless we can make inferences that mean something from the information we obtain, it is of little use. The purpose
of research is not merely to collect data but to use such data to draw warranted conclusions about the people (and others
like them) on whom the data were collected.
What kinds of evidence might a researcher collect?
Content-related evidence of validity refers to the content and format of the instrument.
Criterion-related evidence of validity refers to the relationship between scores obtained using the instrument
and scores obtained using one or more other instruments or measures (often called a criterion).
Classroom Discussion
Study Design from the perspective of the types of investigation:
 Experimental Study
 Quasi Experimental Study
 Non-Experimental Study

Before we can distinguish, we need to look at the possible causes or effects.

When you can identify the problems or the impacts or the effects, and you want to know the
possible factors or causes; or you go from the Effects towards Causes: you are to start doing a Non-
experimental Study.

When you want to find out the effects of something because you already know the possible causes;
you start from the Causes towards the Effects, you are to do an Experimental Study. You are testing
an Intervention or Treatment. You have 2 groups of participants: one Experimental Group
(Treatment Group) and another Controlled Group. Say, you want to test if Blended Learning is
effective. You divide you class into 2 group, one will receive the Blending Learning and the other
don’t. The first is Experimental Group and the Latter is the Controlled. Also, you need to have a
baseline which will be used to compare the results after the experimental group has been given the
treatment. Say, in this case, there should be 2 Tests, Pre-Test before the Treatment to get the
Baseline data, and Post-Test, after the treatment has been conducted to see if the Experimental
Group has some sorts of improvement for us to make a conclusion, which will be decided using the
application/ software.
Also, in Experimental Study, you sample selection is very crucial with good reasons because
inaccurate and improper selection of samples will devalue or discredit your study.
So, the samples have to be selected randomly. You are not to choose the students intentionally at
all. It’s very difficult to achieve pure experimental study.

But if you don’t choose the samples using randomization, you are using Quasi-Experimental Study;
half Experimental and half Non. You don’t use randomization but you are looking to find effects of
some sorts of causes. You still have 2 groups, one experimental and one controlled. This is more
practical comparing to Experimental.
Pre-Test Treatment Post-Test
Experimental Group Yes Yes Yes
Controlled Group No NO Yes
To get Base Line
Types of Quantitative Research Study Design: we have 2, experimental (aka scientific study) and
non-experimental (aka survey research).