HMEF5014 Eduacational Research Methodology

HMEF5014
EDUCATIONAL
RESEARCH
METHODOLOGY
Prof Dr John Arul Phillips
Copyright © Open University Malaysia (OUM)

Project Directors: Prof Dr Mansor Fadzil
Prof Dr Mohd Ghazali Mohayidin
Open University Malaysia
Module Writer: Prof Dr John Arul Phillips
Moderator: Assoc Prof Dr Chung Han Tek

Developed by: Centre for Instructional Design and Technology

Printed by: Meteor Doc. Sdn. Bhd.

Lot 47-48, Jalan SR 1/9, Seksyen 9,
Jalan Serdang Raya, Taman Serdang Raya,
43300 Seri Kembangan, Selangor Darul Ehsan
First Edition, January 2007

Second Edition, August 2011
Third Edition, April 2013 (rs)
Copyright © Open University Malaysia (OUM), April 2013, HMEF5014
All rights reserved. No part of this work may be reproduced in any form or by any means without
the written permission of the President, Open University Malaysia (OUM).

Table of Contents
Course Guide xiăxviii
Course Assignment Guide xxiăxxiii
Topic 1 The Educational Research Process 1

1.1 Methods of Acquiring Knowledge 2
1.2 Research Defined 4
1.3 Characteristics of Research 5
1.4 What is Educational Research? 7
1.5 The Research Process 8
1.5.1 Generating Research Ideas 8
1.5.2 Formulate the Research Problem 10
1.5.3 Develop Hypotheses or Research Questions 12
1.5.4 Design Study to Test Hypotheses or
Research Questions 12
1.5.5 Collect Data 13
1.5.6 Analyse and Interpret Data 14
1.5.7 Communicate Results 15
1.6 Constructs and Variables 15
1.7 Types of Variables 16
Summary 18
Key Terms 19
References 20
Topic 2 Theory and Review of Literature 21

2.1 What is Theory? 22
2.2 Confirming or Disconfirming a Theory 23
2.3 What is the Review of Literature? 26
2.4 The Importance of the Review of Literature 27
2.5 Steps in Developing a Literature Review 28
2.6 Common Weaknesses 30
2.7 Sources 31
2.8 Evaluating Research Articles 33
Summary 37
Key Terms 37
References 38

iv  TABLE OF CONTENTS
Topic 3 Experimental Methodology 39

3.1 The Experimental Method 40
3.2 Controlling Extraneous Variables to Enhance Internal
Validity of Experiments 42
3.2.1 Time Interval and Threats to Internal Validity 43
3.2.2 Other Threats to Internal Validity 44
3.2.3 Subject-Experimenter Effects to be Controlled 45
3.3 Random Assignment to Enhance Internal Validity 46
3.4 Other Techniques to Ensure Groups are Equivalent 47
3.4.1 Matching 48
3.4.2 Holding One or More Variables Constant 48
3.4.3 Including an Extraneous Variable in the
Research Design 49
3.4.4 Analysis of Covariance (ANCOVA) 49
3.5 Hypothesis Testing 50
3.5.1 The Research Hypothesis and the Null Hypothesis 51
3.6 Test of Significance 52
Summary 54
Key Terms 55
References 56
Topic 4 Experimental Research Designs 57

4.1 Symbols Used in Experimental Research Designs 58
4.2 Weak Designs 58
4.2.1 One-shot Design 59
4.2.2 One-group Pretest-posttest Design 59
4.2.3 Non-equivalent Posttest-only Design 60
4.3 True Designs 61
4.3.1 After-only Research Design 62
4.3.2 Factorial Research Design 63
4.3.3 Before-after Research Design 68
4.4 Quasi-experimental Design 68
4.4.1 Non-equivalent Control-group Design 69
4.4.2 Interrupted Time Series Design 70
4.5 Ethics in Experimental Research 72
Summary 74
Key Terms 74
References 75

TABLE OF CONTENTS  v
Topic 5 Survey Research Methodology 77

5.1 What is a Survey? 79
5.2 Types of Surveys 80
5.2.1 Cross-sectional Survey 80
5.2.2 Longitudinal Survey 81
5.3 Sampling 82
5.4 Sampling Techniques 83
5.4.1 Probability Sampling Techniques 84
5.4.2 Non-probability Sampling 88
5.5 Sample Size 90
5.6 The Process of Conducting a Survey 91
5.7 Data Collection Methods Using Surveys 96
5.8 Ethics in Surveys 98
Summary 99
Key Terms 100
References 101
Topic 6 Instrumentation and Data Analysis 102

6.1 The Questionnaire 104
6.2 Question Response Format 105
6.2.1 Structured Questions 105
6.2.2 Types of Structured Questions 106
6.2.3 Unstructured Questions 108
6.3 Guidelines on Questionnaire Design 109
6.4 Pilot Testing the Questionnaire 111
6.5 Designing an Attitude Test 112
6.6 Quantitative Data Analysis 116
6.6.1 Mean and Standard Deviation 116
6.6.2 Testing for Significant Differences between
Two Means Using the StudentÊs t-Test
(Independent Groups) 118
6.6.3 Testing for Significant Differences between
Two Means Using the t-Test (Dependent Groups) 120
6.6.4 Testing for Differences between Means Using
One-way Analysis of Variance (ANOVA) 121
6.6.5 Testing Differences Using the Chi-square (2) 122
6.6.6 Testing Relationship Using the Correlation
Coefficient 123
Summary 125
Key Terms 126
References 127

vi  TABLE OF CONTENTS
Topic 7 Qualitative Research Methods 128

7.1 Evolution of Qualitative Research 129
7.2 Differences between Quantitative and Qualitative
Research 132
7.3 Qualitative Research Methods in Education 134
7.4 Ethnography 134
7.4.1 What is Ethnography? 135
7.4.2 Role of Researcher in an Ethnographic Study 136
7.4.3 Fieldsites 136
7.4.4 Some Practical Concerns 137
7.4.5 Fieldnotes 138
7.5 Case Study 140
7.5.1 The Method of Case Studies 140
7.5.2 Techniques for Gathering Data 141
7.6 Action Research 142
7.6.1 What is Action Research? 142
7.6.2 Why Action Research? 143
7.6.3 Concept of Action Research 143
7.7 Generic Qualitative Research Methods 145
Summary 148
Key Terms 149
References 149
Topic 8 Qualitative Data Collection Methods 151

8.1 Document Examination 152
8.1.1 Content Analysis 152
8.2 Observation 154
8.3 Interviews 156
8.4 Triangulation 157
8.5 Skills Required of the Researcher 159
8.6 Length of Time Spent in Collecting Data 160
8.7 Sampling 162
8.8 Validity of Qualitative Research 164
8.8.1 Types of Validity 164
8.8.2 External Validity of Qualitative Research 165
8.8.3 Enhancing Internal Validity of Qualitative
Research 166
Summary 169
Key Terms 169
References 170

TABLE OF CONTENTS  vii
Topic 9 Qualitative Data Analysis 171

9.1 Stages in Qualitative Data Analysis 173
9.2 Writing the Qualitative Research Report 179
Summary 182
Key Terms 183
References 183
Topic 10 Writing the Research Proposal and Research Report 185

10.1 What is a Research Proposal? 186
10.2 Relationship between the Research Proposal and
the Final Research Report 190
10.3 The Research Report 190

viii  TABLE OF CONTENTS

COURSE GUIDE

x  COURSE GUIDE

COURSE GUIDE  xi
WELCOME TO HMEF5014
HMEF5014 Educational Research Methodology is one of the required courses
for the Master of Education (MEd) programme. The course assumes no previous
knowledge and experience in educational research but you are encouraged to
tap into your experiences as a teacher, instructor, lecturer or trainer and relate
them to the concepts and principles discussed. This is a three (3) credit course
conducted over a semester of 14 weeks.
WHAT WILL YOU GET FROM DOING THIS

COURSE?
Description of the Course
The course begins with discussion of the research process followed by techniques
in writing a review of literature. Both quantitative and qualitative research methods
are examined. Among the quantitative methods emphasised are experimental
research designs and the survey method. Various kinds of experimental research
designs are introduced together with the procedure for conducting a survey. The
design and development of instruments and issues relating to reliability and
validity are also discussed.
The qualitative and quantitative approaches are compared in terms of objectives,

data collection methods and techniques of data analysis. In addition, data collection
methods such as observation, interviews and documents are explored. This is
followed by an examination of data analysis methods with emphasis on coding
and reporting of qualitative data. The final part of the course focuses on the
writing of the research proposal.
Aim of the Course

The main aim of the course is to provide you with a foundation on the principles
and practices of educational research and its application in the design of a research
project.

xii  COURSE GUIDE
Objectives of the Course

By the end of this course, you should be able to:
1. Identify the characteristics of educational research;
2. Compare between quantitative and qualitative methods in educational research;
3. Discuss the differences between quantitative and qualitative methods of data

collection;
4. Critique research in education and write a review of literature; and
5. Demonstrate an ability to write a research proposal.
HOW CAN YOU GET THE MOST FROM THIS

COURSE?
Learning Package
In this Learning Package you are provided with THREE kinds of course materials:
1. The Course Guide you are currently reading;
2. The Course Content (consisting of 10 topics); and
3. The Course Assessment Guide (which describes the assignments to be

submitted and the examinations you have to sit for).
Please ensure that you have all of these materials.
COURSE SYNOPSIS
Topic 1 examines what is education, the scientific method, the criteria for good
research and the steps in the research process.
The first part of the Topic 2 defines theory and the role of theory in educational
research. The second part deals with the role of the literature review in research
and guidelines in writing a review.

COURSE GUIDE  xiii
Topic 3 introduces the concept of the experiment in educational research,

emphasising the issue of internal validity, random assignment, hypothesis testing
and test of significance.
Topic 4 presents different types of weak designs, true experiments and quasi-
experiments. Focus is on the reasons for using a particular design and the ethics
involved in experiments using human subjects.
Topic 5 examines the different types of surveys and the procedure in conducting
a survey. Sampling and generalisation of findings are discussed together with the
methods of data collection. The ethics of conducting surveys are also discussed.
Topic 6 is devoted towards explaining the design and development of

instruments such as the survey questionnaire, attitude scales and use of
commercially available instruments. The issues of reliability and validity of
instruments are explored.
Topic 7 introduces qualitative approaches in educational research and compares

this with the quantitative approach. The issues of reliability and validity in
qualitative research are discussed.
Topic 8 focuses on common data collection methods adopted in qualitative

research, focussing on observation, interviews and collection of documents.
Topic 9 examines the techniques of analysing qualitative data, focussing on

transcribing, coding and presentation of qualitative data.
Topic 10 outlines the sections of a research proposal and suggestions on how to

go about writing the document.
Organisation of Course Content

In distance learning, the module replaces the university lecturer. This is one of
the main advantages of distance learning where specially designed learning
materials allow you to study at your own pace, anywhere and anytime. Think of
it as reading the lecture instead of listening to a lecturer. In the same way that a
lecturer might assign something for you to read or do, the module tells you what
to read, when to read and when to do the activities. Just as a lecturer might ask
you questions in class, your module provides exercises for you to complete at
appropriate junctures.

xiv  COURSE GUIDE
To help you read and understand the individual topics, numerous realistic
examples are given to clarify definitions, concepts and theories. Diagrams and
text are combined in a visually appealing, easy-to-read module. The use of
illustrations, tables and charts reinforce important points and simplify the more
complex concepts.
Each topic contains the following features:
 INTRODUCTION
Lists the headings and subheadings of each topic to provide an overview of the
contents and the major concepts to be studied.
LEARNING OUTCOMES
This is a listing of what you should be able to do after successful
completion of a topic. In other words, whether you are be able to
explain, compare, evaluate, distinguish, list, describe, relate and so forth.
You should use these indicators to guide your study. When you have
finished a topic, you must go back and check whether you have achieved
the learning outcomes or be able to do what is required of you. If you
make a habit of doing this, you will improve your chances of
understanding the contents of the course.
SELF-CHECK
Questions are interspersed at strategic points in the topic to encourage

review of what you have just read and retention of recently learned
material. The answers to these questions are found in the paragraphs
before the questions. This is to test immediately whether you understand
the few paragraphs of text you have read. Working through these tests
will help you determine whether you understand the topic and prepare
you for the assignments and the examination.

COURSE GUIDE  xv
ACTIVITY
These are situations drawn from research projects to show how

knowledge of the principles of research methodology may be applied to
real-world situations. The activities illustrate key points and concepts
dealt with in each topic.
The main ideas of each topic are listed in brief sentences to provide a review of
the content. You should ensure that you understand every statement listed. If
you do not, go back to the topic and find out what you do not know.
Key Terms discussed in the topic are placed at end of each topic to make you
aware of the main ideas. If you are unable to explain these terms, you should go
back to the topic to clarify.
DISCUSSION QUESTIONS:
At the end of each topic a list of questions is presented that are best solved
through group interaction and discussion. You can answer the questions
yourself. But, you are encouraged to work with your course-mates and discuss
online and during the seminar sessions.
At the end of each topic a list of articles and books is provided that is directly
related to the contents of the topic. As far as possible the articles and books
suggested for further reading are accessible at OUMÊs Digital Library. In
addition, relevant internet resources are available to enhance your understanding
of selected curriculum concepts and principles as applied in real-world situations.

xvi  COURSE GUIDE
ASSESSMENT FORMAT
Please refer to myVLE.
WHAT SUPPORT WILL YOU GET IN STUDYING THIS

COURSE?
Seminars
There are 15 hours of seminars or face-to-face interaction supporting the course.
This consists of FIVE seminar sessions of 3 hours each. You will be notified of the
dates, times and location of these tutorials, together with the name and phone
number of your facilitator, as soon as you are allocated a seminar group.
myVLE Online Discussion

Besides the face-to-face seminar sessions, you have the support of online
discussions. You should interact with other students and your facilitator using
myVLE. Your contributions to the online discussion will greatly enhance your
understanding of course content, besides giving you pointers on how to go about
doing your assignment and preparing for the exam.
Facilitator
Your facilitator will mark your assignments and provide assistance during the
course. Do not hesitate to discuss during the seminar sessions or online if:
Ć You do not understand any part of the course content or the assigned
readings;
Ć You have difficulty with the self-tests and activities; or
Ć You have a question or problem with the assignments.

COURSE GUIDE  xvii
HOW SHOULD YOU STUDY FOR THIS COURSE?

1. Time Commitment for Studying
You should plan to spend about six to eight hours per topic, reading the
notes, doing the self-tests and activities and referring to the suggested
readings. You must schedule your time to discuss online. It is often more
convenient for you to distribute the hours over a number of days rather than
spend one whole day per week on study. Some topics may require more
work than others, although on average, it is suggested that you spend
approximately three days per topic.
2. Proposed Study Strategy

The following is a proposed strategy for working through the course. If you
run into any trouble, discuss it with your tutor either online or during the
tutorial sessions. Remember, the tutor is there to help you.
(a) The most important step is to read the contents of this Course Guide
thoroughly.
(b) Organise a study schedule. Note the time you are expected to spend on
each topic the date for submission of assignments as well as seminar
and examination dates. These are stated in your Course Assessment
Guide. Put all this information in one place, such as your diary or a wall
calendar. Whatever method you choose to use, you should decide on
and jot down your own dates for working on each topic. You have some
flexibility as there are 10 topics spread over a period of 14 weeks.
(c) Once you have created your own study schedule, make every effort to
„stick to it‰. The main reason students are unable to cope is that they
get behind in their coursework.
(d) To begin reading a topic:

 Remember in distance learning much of your time will be spent,
READING the course content. Study the list of topics given at the
beginning of each topic and examine the relationship of the topic to
the other nine topics.
 Read the topic overview showing the headings and subheadings to
get a broad picture of the topic.
 Read the topic learning outcomes (what is expected of you). Do you
already know some of the things to be discussed? What are the
things you do not know?

xviii  COURSE GUIDE
 Read the introduction (see how it is related to the previous topic).

 Work through the topic. (The contents of the topic has been
arranged to provide a sequence for you to follow)
 As you work through the topic, you will be asked to do the self-test
at appropriate points in the topic. This is to find out if you
understand what you have just read.
 Do the activities (to see if you can apply the concepts learned to
real-life situations)
3. When you have completed the topic, review the learning outcomes to
confirm that you have achieved them and are able to do what is required.
4. If you are confident, you can proceed to the next topic. Proceed topic by topic
through the course and try to pace your study so that you keep yourself on
schedule.
5. After completing all topics, review the course and prepare yourself for the
final examination. Check that you have achieved all topic learning outcomes
and the course objectives (listed in this Course Guide).
FINAL REMARKS
One again, welcome to the course. To maximise your gain from this course you
should try at all times to relate what you are studying with the real life. Look at
the environment in your institution and ask yourself whether they provide
opportunities for research. Most of the ideas, concepts and principles you learn
in this course have practical applications. It is important to realise that much of
what we do in education and training has to be based on sound theoretical
foundations. The contents of this course provide the principles for doing research
in education, whether it is in a school, college, university or training organisation.
We wish you success with the course and hope that you will find it interesting,
useful and relevant towards your development as a professional. We hope you
enjoy your experience with OUM and we would like to end with this saying,
„Education is a lifetime of learning, relearning and unlearning‰ by Alvin Toffler
(an adaptation).

COURSE ASSIGNMENT
GUIDE

xx  COURSE ASSIGNMENT GUIDE

COURSE ASSIGNMENT GUIDE  xxi
INTRODUCTION
This guide explains the basis on which you will be assessed in this course during
the semester. It contains details of the facilitator-marked assignments, final
examination and participation required for the course.
One element in the assessment strategy of the course is that, all students should
have the same information as facilitators about the answers to be assessed.
Therefore, this guide also contains the marking criteria that facilitators will use in
assessing your work.
Please read through the whole guide at the beginning of the course.
ACADEMIC WRITING
(a) Plagiarism
(i) What is Plagiarism?

Any written assignment (essays, project, take-home examinations, etc)
submitted by a student must not be deceptive regarding the abilities,
knowledge or amount of work contributed by the student. There are
many ways that this rule can be violated. Among them are:
Paraphrases: A closely reasoned argument of an author is paraphrased but

the student does not acknowledge doing so. (Clearly, all our
knowledge is derived from somewhere, but detailed arguments
from clearly identifiable sources must be acknowledged.)
Outright Large sections of the paper are simply copied from other sources
plagiarism: and the copied parts are not acknowledged as quotations.
Other These often include essays written by other students or sold
sources: by unscrupulous organisations. Quoting from such papers is
perfectly legitimate if quotation marks are used and the source
is cited.
Works by Taking credit deliberately or not deliberately for works
others: produced by others without giving proper acknowledgement.
These works include photographs, charts, graphs, drawings,
statistics, video clips, audio clips, verbal exchanges, such as
interviews or lectures, performances on television and texts
printed on the Web.
Duplication The student submits the same essay for two or more courses.

xxii  COURSE ASSIGNMENT GUIDE
(ii) How Can I Avoid Plagiarism?
 Insert quotation marks around „copy and paste‰ clause, phrase,

sentence, paragraph and cite the original source.
 Paraphrase clause, phrase, sentence or paragraph in your own

words and cite your source
 Adhere to the APA (American Psychological Association) stylistic

format, whichever applicable, when citing a source and when
writing out the bibliography or reference page
 Attempt to write independently without being overly dependent

on information from anotherÊs original works
 Educate yourself on what may be considered as common

knowledge (no copyright necessary), public domain (copyright
has expired or not protected under copyright law), or copyright
(legally protected).
(b) Documenting Sources

Whenever you quote, paraphrase, summarise or otherwise refer to the
work of others, you are required to cite its original source documentation.
Offered here are some of the most commonly cited forms of material.
 Direct Citation Simply having a thinking skill is no assurance

that children will use it. In order for such skills
to become part of day-to-day behaviour, they must
be cultivated in an environment that values and
sustains them. „Just as childrenÊs musical skills will
likely lay fallow in an environment that doesnÊt
encourage music, learnerÊs thinking skills tend
to languish in a culture that doesnÊt encourage
thinking‰ (Tishman, Perkins and Jay, 1995, p. 5)
 Indirect Citation According to Wurman (1988), the new disease of

the 21st century will be information anxiety, which
has been defined as the ever-widening gap between
what one understands and what one thinks one
should understand.

COURSE ASSIGNMENT GUIDE  xxiii
(c) Referencing
All sources that you cite in your paper should be listed in the Reference
section at the end of your paper. HereÊs how you should do your Reference.
Journal Article DuFour, R. (2002). The learning-centred principal.

Educational Leadership, 59(8). 12ă15.
Online Journal Evnine, S. J. (2001). The universality of logic: On the
connection between rationality and logical ability
[Electronic version]. Mind, 110, 335ă367.
Webpage National Park Service. (2003, February 11). Abraham
Lincoln Birthplace National Historic Site. Retrieved
February 13, 2003, from http://www.nps.gov/abli/
Book Naisbitt, J., & Aburdence, M. (1989). Megatrends 2000.
London: Pan Books.
Article in a Nickerson, R. (1987). Why teach thinking? In J. B. Baron,
Book & R. J. Sternberg (Eds). Teaching thinking skills:
Theory and practice. New York: W. H. Freeman and
Company. 27ă37.
Printed Holden, S. (1998, May 16). Frank Sinatra dies at 82:
Newspaper Matchless stylist of pop. The New York Times,
pp. A1, A22ăA23.
ASSESSMENT
Please refer to myVLE.
TAN SRI DR ABDULLAH SANUSI (TSDAS)

DIGITAL LIBRARY
The TSDAS Digital Library has a wide range of print and online resources for
the use of its learners. This comprehensive digital library, which is accessible
through the OUM portal, provides access to more than 30 online databases
comprising e-journals, e-theses, e-books and more. Examples of databases available
are EBSCOhost, ProQuest, SpringerLink, Books247, InfoSci Books, Emerald
Management Plus and Ebrary Electronic Books. As an OUM learner, you are
encouraged to make full use of the resources available through this library.

xxiv  COURSE ASSIGNMENT GUIDE

Topic  The
1 Educational
Research
Process
LEARNING OUTCOMES
By the end of this topic, you should be able to:
1. Identify the different methods of acquiring knowledge;
2. Describe what is educational research;
3. Discuss the importance of educational research and its characteristics;
4. Identify the criteria for research;
5. Describe the steps involved in the research process;
6. Identify a research problem; and
7. List the criteria of a good research problem.
 INTRODUCTION
„Research has shown that four out of five dentists interviewed used Plantoid
Herbal Toothpaste‰
„Years of research has shown that SlimTex Capsules will reduce your weight
in a matter of weeks‰
„Azlan is doing research browsing through consumer reports, catalogues,
brochures and market surveys before deciding what car to buy‰
„Kong Beng is surfing the internet doing research on the symptoms of HIV
AIDS‰

2  TOPIC 1 THE EDUCATIONAL RESEARCH PROCESS
The word „research‰ has been used in many different ways and sometimes
rather loosely giving rise to confusion and sometimes with the intention to
deceive. The claim that the majority of dentists used the particular brand of
toothpaste is misleading as there is no mention as to how many dentists were
interviewed and whether they are representative of all dentists in the country.
With regards to the advertisement on slimming pills, research on the use of
various types of medicine and supplements by humans tend to be inconclusive. It
would be difficult to make a claim that it will reduce weight as there are many
other contributory factors such as gender, level of health, weight and so forth. In
the strictest sense of the term what Azlan and Kong Beng are doing is not
research. Looking up facts and writing them down is nothing more than fact
finding and fact transcribing (Leedy, 1974). There is even the tendency to call the
written work a „research report‰ which is not accurate. It may be a report but not
a research report. There needs to be a distinction between true research and the
accumulation of facts. Research is a way of thinking; it involves thinking what we
want to study, how we go about collecting data, analysing the data and deriving
conclusions.
ACTIVITY 1.1
Discuss the different ways in which the word „research‰ has been used
in the statements in the box.
1.1 METHODS OF ACQUIRING KNOWLEDGE

Over the centuries, humans have accumulated a vast amount of knowledge
and the amount of knowledge produced is doubling every two years. There are
many ways in which we obtain knowledge about a given phenomenon, event or
situation. There are six ways in acquiring knowledge and they are; through our
beliefs, intuition, authority, empiricism, rationalism and science (see Figure 1.1).

TOPIC 1 THE EDUCATIONAL RESEARCH PROCESS  3
Beliefs Intuition
Rationalism METHODS OF ACQUIRING Authority

KNOWLEDGE
Experience Science
Figure 1.1: Six methods of acquiring knowledge

Source: Helmstadter (1964)
(a) Beliefs
These are all superstitious beliefs people hold on to as though they are
fact. For example, wear your lucky shirt when making deals (the one you
strongly believe will bring you good fortune).
(b) Intuition
An approach to acquiring knowledge that is not based on reasoning
or inferring. Intuitive knowledge is not scientific but is knowledge that
originates from gut feeling or predictions by soothsayers, astrologers and
fortune-tellers.
(c) Authority
Knowledge that originates from persons or sources that are highly
respected. For example, various religions have a sacred text that represents
the facts, which are considered indisputable, final and cannot be challenged.
(d) Experience
This approach of acquiring knowledge is based on the statement which
says, „If I have experienced it, then it is valid and true‰. In other
words, only facts that are in agreement with experience are accepted, and
those that do not are rejected. However, reliance on experience has its
shortcomings because our perceptions of people, events and objects are
affected by many factors. For example, we constantly add, delete and
reconstruct our experiences.

(e) Rationalism
This approach uses reasoning to arrive at knowledge and assumes that
valid knowledge is acquired through correct reasoning.
Ancient philosophers believed that knowledge derived from reasoning was

just as valid as knowledge gained from observation. Reasoning is regarded
as the beginning of the scientific process where hypotheses are proposed.
(f) Science
It is a process that is followed in generating knowledge and has been
accepted as the best method of acquiring knowledge. It lists a series of steps
to be followed when acquiring knowledge using the scientific method.
However, it has been argued that strictly following the scientific method
prevents us from studying in depth human behaviour (we will discuss this
issue in Topics 7ă9 under qualitative research).
ACTIVITY 1.2
1. Identify the six different methods by which we acquire knowledge.

Give specific examples for each method.
2. Which method has contributed most towards our understanding of
how children learn? Give specific examples.
1.2 RESEARCH DEFINED

According to the WebsterÊs dictionary, research is diligent scientific search or
inquiry to discover facts. Wikipedia describes research as an active, diligent and
systematic process of inquiry in order to discover, interpret or revise facts,
events, behaviours or theories. Kerlinger defines research as „the systematic,
controlled, empirical and critical investigation of natural phenomena guided by
theory and hypotheses about the presumed relations among the phenomena‰
(p. 10). You will notice certain key words in these definitions of research:
 Scientific  Controlled
 Systematic process  Theory, hypotheses
 Inquiry or Investigation  Presumed relations
 Discovery  Natural phenomena

It is a scientific (or systematic) process of gathering information about the

hypothesised relations between phenomena. For example, to investigate if there
is a relationship between a studentÊs attitude towards mathematics and his or
her performance in mathematics. The scientific method was popularised by John
Dewey in 1933 and lists the following four steps:
1. Formulation of a hypothesis (a tentative statement about the relationship

between two or more theoretical constructs. e.g. attitude and mathematical
performance).
2. Test the hypothesis (design a study to establish whether the relationship

between the constructs are as hypothesised).
3. Collect data (e.g. collect data on attitude towards mathematics and

mathematical performance).
4. Decide to accept or reject the hypothesis (e.g. correlation between attitudes

towards mathematics and mathematical performance).
The purpose of using the scientific method is to enable the researcher to describe
(the relationship between factors); to predict (given what is known we can we
predict what might happen); to control (when certain variables are manipulated,
does it lead to a particular condition), and to explain (can a theory be formulated
to explain the phenomena being investigated).
1.3 CHARACTERISTICS OF RESEARCH

Research is a way of thinking and to qualify as a research, it needs to have six
characteristics such as follows (adaptation of Leedy, 1993, Borg & Borg, 1983,
Mitchell & Jolley, 1988).
(a) Research Begins with a Question in the Mind of the Researcher

You need only to look around and you will see phenomena that will
arouse your curiosity. For example, why are children in this school unable
to read? Why are girls performing better than boys? These are questions
which beg for answers. By asking relevant questions we create an
inquisitive environment which is the prerequisite for research. Research
arises from an observed phenomenon puzzling the researcher.

(b) Research Requires a Plan

One does not discover the truth or explanations about a phenomenon
without serious and meticulous planning. Research is not looking up
something in the hope of coming across the solution to your problem.
Rather it entails a definite plan, direction and design.
(c) Research Demands a Clear Statement of the Problem

Successful research begins with a clear and simple statement of the
problem. The statement of the problem should be stated precisely and
concisely, so that the researcher is sure of what he or she is seeking to
discover.
(d) Research Deals with the Main Problem through Subproblems

Divide the main problem into appropriate subproblems, all of which when
resolved will result in the solution of the main research problem.
(e) Research Seeks Direction Through Appropriate Hypotheses

Having stated the problem and the related subproblems, each of the
subproblems is then each viewed through logical constructs called
hypotheses. A hypothesis is a logical supposition, a reasonable guess or an
educated conjecture which may give direction to solving the problem.
(f) Research Deals with Facts and their Meaning

Having defined the problem, the subproblems and hypothesis, the next
step is to collect whatever facts pertinent to the problem. Organise the data
collected into a form that is potentially meaningful.
SELF-CHECK 1.1
1. Define research in your own words.

2. Identify the steps to be followed when pursuing knowledge
through the scientific method.
3. What are the four objectives of science?
4. What are the six characteristics of research?

1.4 WHAT IS EDUCATIONAL RESEARCH?

What is educational research? Generally, educational research is described as
research that investigates the behaviour of students, teachers, administrators,
parents and other members of the community who interact with educational
institutions. The word behaviour is taken broadly to mean such phenomena as
learning, attitudes, aptitudes, abilities, interests, practices, processes, emotions
and so forth.
What is the purpose of educational research? Since education is fundamentally

the development of individuals then the central purpose of educational research
is to find ways to improve student learning. It has been argued that educational
research that does not have this as its ultimate motivation and objective is not
educational research. Anyone who is engaged in a systematic search of ways
to improve student learning is doing educational research. For example, a
classroom teacher experimenting with alternative ways of explaining the laws of
physics and a full-time researcher comparing the effectiveness of different
reading methods in early literacy programmes are both engaged in a search for
ways to improve student learning, and, in this sense, are both engaged in
educational research. The leadership styles of the school principal will influence
teacher morale and job satisfaction which will translate to how teachers behave in
the classroom. So, the behaviour of school principals and headmasters can
influence student learning and attitudes.
ACTIVITY 1.3
1. Do you agree with the statement „⁄ the central purpose of

educational research is to find ways to improve student learning‰?
2. Give some examples of findings from educational research in your
area of interest.

1.5 THE RESEARCH PROCESS

As discussed earlier, research is a systematic process which means there are
definite steps involved. Figure 1.2 lists the seven steps. These are elaborated in
the following subtopics.
Figure 1.2: The seven steps of the research process
1.5.1 Generating Research Ideas

For many beginning researchers and graduate students „the problem of finding
a problem‰ can be difficult. There are cases of graduate students who have
completed all coursework requirements and „get stuck‰ at the thesis stage and
some never graduate. Hence, it is advisable that students search for a suitable
problem early on which they can explore throughout their coursework. The word
„problem‰ means there is a dispute, controversy, debate or disagreement that
needs to be addressed, solved or answered. For example, why do young learners
have more difficulty with multiplication and division operations compared to
addition and subtraction operations in mathematics?
Where does one find research problems in education? They are all around you!
There are abundant research problems or unresolved issues everywhere. Look at
the 14 year old who says, „I hate history‰. Do you know why? Do you want to
know why? In fact, whatever that arouses your interest for which there are as yet
no answers or are inconclusive have the potential of being a research problem.
Often one starts with a rather general, diffuse and even confused notion of the
problem. Do not worry, this is the nature and complexity of the research process.
It is the first step towards becoming a mature researcher. The following three
steps are to help you get a research problem.
(a) Step 1: Identify a broad problem in your area that is of interest to you
and related to your professional goals. Your research should ultimately
contribute to the corpus of knowledge in that area of interest. For example,
you may be interested in how young children learn mathematics; or
how to get teenagers interested in science; how to enhance the teaching
of moral education; how to improve training in the corporate sector using
e-Learning; or how a headmasterÊs leadership style affects teacher morale,
etc.
(b) Step 2: Systematic programme of reading within your broad area of study.
For example if you are interested in why children have difficulty in learning
mathematics, you could start with textbooks in the area or topics of
textbooks. Textbooks explain the basic concepts and facts related to the
issue and may cite research in the area which will be listed in the
„references‰ which you could further explore. Journals such as the Review
of Research in Education and Review of Educational Research provide
valuable information about a particular field as they review related
literature. Next, is to read articles in the relevant journals in the field. For
example if you are interested in reading research then you should read
„Journal of Reading‰ and the „Reading Research Quarterly‰. You should
look through the catalogue both in the library and from on-line resources
and identify the journals in your field.
Students do not adequately refer to journals in the field but instead tend to
cite from popular sources such as newspapers, magazines and speeches.
Journals report empirical evidence about the field you are interested in and
they indicate the current thinking about research and the trend of research
efforts in the field. This will give you a grasp of leading edge research in the
United States, Britain, Europe and Australia and how you might do the
same in Malaysia.
(c) Step 3: Relate your research problem to a theory in the field. A theory is an
explanation of events or phenomena or behaviour. For example, if you
are interested in finding out whether providing children with multimedia
presentations explaining science concepts will enhance understanding,
you may want to explore the underlying theories of visual learning.
Many phenomena in education are explained drawing upon theories from
cognitive psychology, sociology, psycholinguistics, management, computer
science and so forth. It should be remembered that theory provides the
direction for the research (we will discuss in more detail the role of theory
in Topic 2).

SELF-CHECK 1.2
1. How do you go about finding a research problem in education?

2. What other sources provide research problems in education?
1.5.2 Formulate the Research Problem

Upon having a broad idea of what you want to investigate, the next step is to
formulate the problem simply, clearly and completely. This is what we mean by
„statement of the problem‰. An adequate statement of the research problem
is an important step in the research process. Obviously, if you want to solve
a problem, you must know what is the problem. What is a good problem
statement? Although research problems differ greatly, and although there is no
one „right‰ way to state one, certain characteristics of problems and problem
statements can be learned and used to good advantage.
Example: There is a relationship between self-esteem and attitudes toward

science and academic performance in science.
This is an example of a research problem stating the relationship between

three factors or variables (self-esteem, attitudes and academic performance). A
problem is an interrogative sentence or statement that asks: What relationship
exists between the variables? The answer to this question will be sought by
conducting the research.

Three Criteria of Good Problem Statements
1. The problem should express a relationship between two or more variables

Is A related to B?
How are A and B related?
How is A related to B under condition C?
Is there a difference between A and B in terms of C?
2. The problem should be stated clearly and unambiguously preferably in

question form. Instead of saying, „The problem is ....‰, or „The purpose of
this study is ....‰; ask a question. Questions have the advantage of posing
problems directly. The purpose of a study is not necessarily the same
as the problem of the study. For example, the purpose of the study is
to throw light on the relationship between academic performance and
self-esteem. The problem is stated as a question: Is self-esteem related to
academic performance?
3. The problem should be such as to imply possibilities of empirical testing.

A problem that does not contain implications for testing its relationship
not a scientific problem. So, if you can measure the constructs self-esteem
and academic performance, then the problem is considered a good
problem.
SELF-CHECK 1.3
1. What is a good problem statement?

2. Why do graduate students have difficulty in stating research
problems?

1.5.3 Develop Hypotheses or Research Questions

Let us look at the following example:
Your car will not start. You put forward the hypothesis that „the car does not
start because there is no petrol‰. You check the fuel gauge to either reject or
accept the hypothesis. If you find there is petrol, you reject the hypothesis.
Next, you hypothesise that „the car did not start because the spark plugs are
dirty‰. You check the spark plugs to determine if they are dirty and accept or
reject the hypothesis accordingly.
Similarly, in educational research you have to put forward hypotheses or

research questions that state a relationship between the variables or constructs
you are studying. After having established the research problem or area that you
plan to investigate, the next step is to break down the problem into subproblems
called hypotheses or research questions. A hypothesis or research question is
an „educated guess‰ or a hunch about possible relationships or differences. The
hypothesis or research question guides the selection of appropriate research
methods, data collection techniques, data analysis techniques (e.g. the statistical
analysis to be used), etc.
Hence, hypotheses or research questions have to be clearly stated and you

should be prepared to defend or support your choice of hypotheses or research
questions. For example, you hypothesise that „Students taught science using a
problem-based approach will be more creative‰. You are seeking to confirm
empirically that the problem-based approach in science teaching will enhance
creativity of learners.
1.5.4 Design Study to Test Hypotheses or Research

Questions
Having determined the hypotheses or research questions, the next step is to
design the study. We often hear of graduate students saying they want to do
an „experiment‰ or a „survey‰. But, they are unable to state with clarity and
precision the hypotheses or research questions they intend to answer. It is like
„putting the cart before the horse‰. The hypotheses or research questions
determine the design of the study. If you intend to test the effectiveness of an
educational phenomenon such as a teaching method or a counselling technique,
the logical choice would be to design an experiment. If you intend to find out
whether teachers are satisfied with their profession or how they perceive their

principals, than the survey would be the appropriate research design. If your
intention is to study inter-racial mixing among students in the school canteen,
than a qualitative approach using the observation technique might be more
appropriate.
Hence, the decision on which methodology to use will depend on the research
problem and the research questions or hypotheses. It is not good practice to
decide on a methodology and then work on the research questions. You have
to be able to state clearly what you intend to study and then decide on an
appropriate methodology. If you are clear about your research problem and
research questions, you will find it easy to get assistance from your supervisor
and other students in designing a study to find answers to your research
questions.
ACTIVITY 1.4
The following are research problems taken from research literature.
Study them carefully and construct one or two hypotheses or research
questions based on them.
(a) How do self-esteem and level of aspiration influence academic
achievement?
(b) Does providing learners with graphic organisers enhance their
understanding of science text materials?
(c) How does the organisational climate in schools affect teacher
satisfaction and morale?
1.5.5 Collect Data

The research question determines the design of the study and method of data
collection to answer the question. Say for example, one of your research
questions is to determine whether there are differences in self-esteem between
male and female 16 year old students in secondary schools. To answer this
question you have to collect data on the self-esteem of students. This may be
done by developing a self-esteem instrument (or using an available instrument)
and administering it to a sample of secondary school students. The sample will
have to be representative of 16 year old students in secondary schools to allow
you to generalise the results obtained to the population. Here you are using
quantitative data collection methods (we will discuss in detail quantitative data
collection methods in Topics 3ă5).

On the other hand, if you are interested in studying student-teacher interaction,

you may have to spend time in classrooms. You can use a structured observation
checklist or leave it open-ended and record all the processes that occur in the
classroom. Here you are using qualitative data collection methods (we will
discuss in detail qualitative data collection methods in Topics 7ă9).
ACTIVITY 1.5
You have stopped at a junction because the lights have just turned red.
But, on your left a motorcyclist and a car beat the lights. You are most
annoyed. Being a good social scientist, you ask yourself why people
beat traffic lights. If you were to convert the problem into a hypothesis,
which of the following would be the best hypothesis? Explain your
answer.
(a) Some people beat traffic lights more frequently than others.
(b) People who beat traffic lights are a danger to themselves and
other road users.
(c) Beating traffic lights is common in crowded cities.
(d) Road users are more likely to beat traffic lights when the traffic is
light than when it is heavy.
1.5.6 Analyse and Interpret Data

The data collected from subjects (e.g. students, teachers, school administrators
and others) will have to be analysed. If your study involves quantitative data
then statistical procedures will be used to analyse the data. The analysed data
is usually presented as tables and graphs. Based on the statistical analysis, the
researcher interprets the data in relation to the research questions or hypotheses.
In the case of qualitative data, information is coded and presented anecdotally.
Instead of numbers, data is presented in the form of words and sentences.
Similarly, the data is interpreted in relation to the research questions or objectives
of the study (we will discuss how to analyse and interpret qualitative data in
Topic 9).

1.5.7 Communicate Results

Data that has been analysed and interpreted will have to be communicated to the
community of fellow researchers and practitioners. This is most commonly done
through the numerous journals on education available that report on areas such
as early childhood, reading, second language learning, educational psychology,
adolescents, mathematics teaching and so forth). If you are a graduate student,
you will most probably be communicating the results of your study in the form
of a thesis or dissertation or even a research practicum. There is an established
format of presenting the findings of your study which will be discussed in detail
in Topic 10.
ACTIVITY 1.6
1. Identify a research problem that you are interested in investigating.

2. Formulate TWO research questions or hypotheses based on the
research problem.
1.6 CONSTRUCTS AND VARIABLES

A construct is deliberately and consciously invented or adopted for a special
scientific purpose. „Intelligence‰ is a construct based on observation of presumably
intelligent and less intelligent behaviours or having a value of more or less.
Constructs are used in theoretical schemes and is related in various ways to
other constructs, e.g. school achievement is in part a function of intelligence.
Intelligence is so defined and specified that it can be observed and measured, e.g.
administering intelligence tests or interviewing teachers about their students.
Researchers somewhat loosely call the constructs or properties they study as

„variables‰, e.g. gender, social class, etc. A variable is something that varies. A
variable is a symbol to which numerals of values are assigned. For example, the
symbol „intelligence‰ is assigned a set of numerical values which may be IQ
scores ranging from 50 to 150. In the case of the variable „gender‰ there are only
two values and they are called dichotomous variables, i.e. male and female.
Other examples of two-value variables are: graduate-nongraduate, low income-
high income, citizen-noncitizen. Besides dichotomous variables, some variables
are polytomies, e.g. religion ă Islam, Christianity, Buddhism, Hinduism, etc.

1.7 TYPES OF VARIABLES

There are many ways of classifying variables but in educational research, the
most common methods of classification are as follows:
(a) Independent and Dependent Variables

An independent variable (IV) is the variable that is presumed to cause a
change in the dependent variable (DV). The independent variable is the
antecedent while the dependent variable is the consequent. See Figure 1.3
which describes a study to determine which teaching method (independent
variable) is effective in enhancing the academic performance (dependent
variable) of students.
(i) The independent variable (teaching method) can be manipulated
„Manipulated‰ means the variable can manoeuvred, and in this case
it is divided into „discussion method‰ and „lecture method‰. Other
examples of independent variables are gender (male-female), race
(Malay, Chinese, Indian), socioeconomic status (high, middle, low).
Other names for the independent variable are treatment, factor and
predictor variable.
(ii) The dependent variable in this study is academic performance which
cannot be manipulated by the researcher. Academic performance is a
score and other examples of dependent variables are IQ (score from
IQ tests), attitude (score on an attitude scale), self-esteem (score
from a self-esteem test) and so forth. Other names for the dependent
variable are outcome variable, results variable and criterion variable.
Put in another way, the DV is the variable predicted to, whereas the IV is
predicted from. The DV is the presumed effect, which varies with any change or
variation in the IV.
Figure 1.3: Independent and dependent variables

(b) Continuous and Categorical Variables

A continuous variable (also called ordinal variable) is capable of taking on
an ordered set of values within a certain range. For example, an attitude
scale towards smoking may have values ranging from five to 20 which
expresses differing amounts of attitude towards smoking. A categorical
variable (also called nominal variable) may be made up of two or more
subsets or categories. Each subset or category possesses certain characteristics,
and individuals are categorised by their possession of those characteristics
that defines a subset. For example, the variable socio-economic class (SES)
may consist of three values such as high SES, middle SES and low SES.
Now, let us look at operational definition of variables. „If you lead a good life,
you will not suffer‰. This is a specific prediction of the future, but it cannot be
scientifically tested. Such a prediction is not scientifically tested because we
cannot define it operationally. How do you define „good life‰ and how do you
define „suffer‰. According to Bridgman, 1927, operational definition means that
variables used in the study must be defined as it is used in the context of the
study and publicly observable. This is done to facilitate measurement and to
eliminate confusion. For example, when you state in your study that you are
studying „excellent principals‰, you should be able to explain what „excellent‰
means. Once the behaviours of an excellent principal have been identified the
operational definition will be unique to your study (see Figure 1.4).
Operational Definition
The person:
Excellent  listens to teachers
Principal  looks after the welfare of teachers
 acknowledges effort
 consults teachers
 motivates teachers
Figure 1.4: Example of an operational definition of an excellent principal
However, it should be borne in mind that in education not all variables are
directly observable. For example, we cannot really observe learning, memory,
reasoning, and so forth. Though they cannot be observed they can be measured
to see their traces. With enough indirect evidence, researchers can make a
convincing case for the existence of these invisible variables (Mitchell & Jolley,
1988). For example, though we cannot observe learning directly, we can see its
effect on performance, i.e. we can operationally define learning as an increase in
performance. Thus, if we see students improve their performance after practicing
a task, we conclude that learning has occurred.

Similarly, we can provide operational definitions for such intangible variables

such as self-esteem, racial stereotype, attitudes and so forth.
SELF-CHECK 1.4
1. What is a variable?
2. What is the difference between an Independent Variable and a
Dependent Variable? Give specific examples.
3. Why do you need to define variables operationally?
 The word „research‰ has been used in many different ways and sometimes
rather loosely, giving rise to confusion and sometimes with the intention to
deceive.
 There are six ways in acquiring knowledge and they are through: our beliefs,
intuition, authority, empiricism, rationalism and science.
 Research is defined as the systematic, controlled, empirical and critical

investigation of natural phenomena guided by theory and hypotheses about
the presumed relations among the phenomena.
 The purpose of the scientific method is to describe, explain, control and

predict phenomena.
 Educational research is described as research that investigates the behaviour

of students, teachers, administrators, parents and so forth.
 The seven steps of the educational research process is a useful guide for
beginning researchers to follow.
 A good research problem is stated clearly, expresses a relationship between

variables and can be tested empirically.
 An independent variable can be manipulated to see its effect on a dependent

variable.
 Operational definitions of variables are necessary to allow measurement and

elimination of confusion.

Categorical variable Research

Continuous variable The research process
Dependent variable Research question
Educational research Research questions
Hypothesis Scientific method
Independent variable Variables
Operational definition
1. Write down your definition of research.

2. Suggest how you will go about finding a research problem that you
propose to investigate.
3. Discuss some educational research you have read.
[Go to OUMÊs Digital Library and click on „ProQuest‰ which has a
good collection of journals in education]
4. List the current thinking on research in your area of interest.
5. Make up operational definitions for the following variables:
(a) Underachievement
(b) Parental bonding
(c) Aspirations [of teenagers]
(d) Autocratic leader [principal or headmaster]
(e) Teacher burnout
(f) Socioeconomic status
(g) Leadership
(h) Reading ability
(i) Delinquency

Bechhofer, F. (2000). Principles of research design in the Social Sciences. London:

Routledge.
Chapter 1: Fundamentals [available at eBrary].
Borg, W., & Borg, M. (1988). Educational research: An introduction. New York:
Longman.
Chapter 3: The research problem, research plan and pilot study. 71ă106.
Kerlinger, F. (1990). Foundations of behavioural research. New York: Allyn and

Bacon.
Chapter 3: Constructs, variables and definitions. 28ă46.
Mitchell, M., & Jolley, J. (1988). Research design explained. New York: Holt,
Rinehart and Winston.
Chapter 2: Generating the research hypothesis. 14ă36.
Chan, A. (1998). Defining the research problem. Retrieved from:

http://www2.fhs.usyd.edu.au/well/knowbase/defresp.htm
Mikijanis, M., & Thom, D. (n.d.). The research question outline. Retrieved from:
http://kancrn.kckps.k12.ks.us/guide/question.html
Trochim, W. K. (2006). Problem formulation. Retrieved from:

http://www.socialresearchmethods.net/kb/probform.htm

T op i c  Theory and
2 Review of
Literature
LEARNING OUTCOMES
1. Define „theory‰;
2. Explain the role of theory in educational research;
3. Justify the need for the review of literature;
4. Identify the criteria for a good review of literature; and
5. Critique a research article.
 INTRODUCTION
Without some viable theory to serve as a guide, many studies address trivial
questions or contribute nothing to the slow accumulation of knowledge
needed for advancement of the science of education.
Source: Borg, W., & Borg, M. (1983). Educational research: An introduction

(pp. 23ă24). New York: Longman. 79ă80
Do you agree with the statement by W. Borg and M. Borg? Theory has not
been properly understood by some graduate students. Some students are of the
opinion that theory is not relevant to practice. Oftentimes, we hear students
remark that a particular course is „too theoretical‰ or they prefer courses that are
„practical-oriented and not too theoretical‰. Some go to the extent to denounce
theory as useless! Actually, this reflects a lack of understanding on what is theory
and what is practice or practical. According to W. Borg and M. Borg, theory

22  TOPIC 2 THEORY AND REVIEW OF LITERATURE
serves as a guide for research to avoid investigating phenomena that is irrelevant

and does not contribute to our understanding.
2.1 WHAT IS THEORY?

A theory is a large body of interconnected propositions about how some portion
of the social world operates (Kidder, 1980). It is statement or set of statements
that explain and predict phenomena. It is a statement of a relationship between
two or more events. The more „powerful‰ a theory is, the more events can be
explained by it. Theories consist of generalisations and in the physical sciences
some of them are called laws (for example, Archimedes Principle, BoyleÊs Law).
(a) According to Social Learning Theory by Albert Bandura, the observer will
imitate the modelÊs behaviour if the model possesses characteristics such as
talent, intelligence, power, good looks or popularity that the observer finds
attractive or desirable.
(b) According to the Theory of Meaningful Learning by David Ausubel,

learning takes place when the learner subsumes new information with old
information or oneÊs cognitive structure.
(c) According to B. F. SkinnerÊs Theory of Reinforcement, a learner will repeat

performance of a task if he or she is reinforced with a system of rewards or
punishment.
Note that each of these theories explain learning and the variables or factors that
determines learning. Assuming that each of these theories are true we can predict
that learning will take place when a student connects new information with
old information, is reinforced through a system of rewards and reproduces a
modelled behaviour if it brings pleasure. However, many areas of education
have virtually no theoretical foundation and have to rely on other behavioural
sciences, such as psychology, sociology, anthropology and social psychology.
Can a theory be „true‰? Not necessarily. The scientific method makes it

impossible to conclude that a theory in the behavioural sciences can be definitely
true. It is possible only to disconfirm or confirm a hypothesis or theory. We
cannot say that we have verified a theory because there is always the possibility
that some future research will disconfirm it or that some other theory will
account for the same results. Theories, therefore, are always tentative. They
represent the best of our knowledge for the time being but they do not represent
some absolute truth. They await revision of replacement (Borg & Borg, 1983).

TOPIC 2 THEORY AND REVIEW OF LITERATURE  23
2.2 CONFIRMING OR DISCONFIRMING A

THEORY
Let us take an example of a theory and examine how we go about confirming
or disconfirming it (Figure 2.1). According to Jean PiagetÊs theory of cognitive
development, „Children at the pre-operational stage (age 0ă5) are not able to
perform conservation tasks‰. Conservation is defined as the ability to recognise
that an object remains unchanged when its volume or length has undergone a
transformation with nothing added or taken away. From this broad theory a
hypothesis is derived which is a small version of the theory.
Figure 2.1: Confirming or disconfirming a theory
Then, an experiment is set up in which young children are shown two identical
containers (A and B) with the same amount of water. Then, the water from
container B is poured into a flat container C (see Figure 2.2). Children are asked
whether the amount of water in container A is the same as container C (Is A =
C?). Based on these observations, the researcher concludes whether to confirm or
disconfirm the theory. If the theory is confirmed, children are unable to perform
conservation tasks. In other words, children replied that container C had more
water. If the theory is disconfirmed, then the majority of children answered that
the amount of water in container A and C is the same.

A B C
Figure 2.2: Containers A, B and C filled with water
SELF-CHECK 2.1
1. What is a theory?
2. Why are there few laws in the field of education?
3. What do you mean by confirming or disconfirming a theory?
4. Identify some theories in your field of interest.
5. What are the implications of PiagetÊs theory on childrenÊs cognition
in the classroom?
Let us examine another well-known theory in psychology which has been

used widely in education. David Clarence McClelland developed a theory of
motivation in 1988, which states that a person is motivated to do something
because of a desire or need for achievement, authority or affiliation or a
combination of the three characteristics illustrated in Figure 2.3.
Achievement
Authority/ Power
Motivation (n-ach)
Motivation (n-pow)
Affiliation Motivation
(n-affil)
Figure 2.3: Three characteristics that motivate a person to do something

(a) Achievement Motivation (represented by „n-ach‰) includes persons who

are driven by the need or desire to achieve, attain realistic but challenging
goals, and advance in their job.
(b) Authority/Power Motivation (represented by „n-pow‰) includes persons

who are driven by the need or desire to be influential, effective and to make
an impact.
(c) Affiliation Motivation (represented by „n-affil‰) includes persons who are

driven by the need or desire for friendly relationships and are motivated
towards interaction with other people.
As research is focused in testing behavioural science theory, we gain a better

understanding of the theory, which in turn leads to modification in the theory
and eventually to its acceptance or rejection. It is hoped that this better
understanding of human behaviour will inform practice in terms of what
teachers do in the classroom, how administrators manage and lead their
organisations, and most importantly how students learn and conduct themselves.
Theory can provide a rational basis for explaining or interpreting the results of
research. Studies without a theoretical foundation often produce results that the
researcher is at a loss to explain. Also, studies based on theory enable the
researcher to make predictions about a wide range of situations. For example,
McClellandÊs theory of motivation could be employed to determine what
motivates teachers, students and school administrators.
ACTIVITY 2.1
1. To what extent does McClellandÊs Motivation Theory describe your

motivation to do something in your daily life?
2. Briefly explain how you would attempt to confirm or disconfirm
McClellandÊs Theory.
[You can find more information about this theory at this site:
Chapman, A. (1995). David McCllelandÊs motivational needs theory
[Electronic version].
http://www.businessballs.com/davidmcclleland.htm]
3. Identify some major theories in your area of interest.

2.3 WHAT IS THE REVIEW OF LITERATURE?
Until you know what others have done in your area and what has not been
done, you cannot convincingly carry out research that will contribute to
furthering knowledge in your field. Thus, the literature in any field forms the
foundation upon which all future work must be built. If you fail to build this
foundation of knowledge provided by the review of literature, your work is
likely to be shallow and naive, and will often duplicate what has already been
done better by someone else.
Source: Borg, W., & Borg, M. (1983). Educational research: An introduction.

Longman: New York. p. 142
The review of literature is usually a standard topic of the research report, thesis
or dissertation. It is an account of the research done in the field of study. The
review forms an important topic in a thesis or dissertation where its purpose
is to provide the background to and justification for the research undertaken.
The review would usually consist of empirical studies done in the area being
investigated. It also includes theoretical positions or proposals related to the
study, which are not necessarily empirical in nature.
The aim of the literature review is to show what has been done in the field and
how your study relates to earlier research. It also indicates the approaches, the
samples used, the variables examined, the statistical procedure used and most
important of all, the findings obtained. The review gives an overview of the
findings of various previous studies. The review traces the general patterns of the
findings and the conclusions that can be made based on the findings.
It also provides an insight into how your study is similar or different from
previous studies. For example; Is your study an extension of what others have
done? Are you examining variables that have not been attended to in earlier
studies? Are you attempting to replicate earlier studies in a different cultural
context? Are you applying statistical procedures that have not been attempted by
others?
The review of literature requires you to locate, read and evaluate reports of
research as well as reports of opinions and proposals. The review must be
extensive and thorough because you are aiming to obtain a detailed account of
the topic being studied.

2.4 THE IMPORTANCE OF THE REVIEW OF

LITERATURE
The review of literature is an important part of the research process because:
(a) It forms the basis of any research and puts your work into perspective;
(b) It gives an understanding of previous work that has been done (seminal
works in the field); and
(c) It familiarises you with the personalities doing research in the field and to
demonstrate that you can access such works.
The review of literature helps the student in delimiting the research problem by
setting the parameters. By setting the limits of your study, you avoid being
questioned „why didnÊt you do this or do that?‰ You can confidently reply that
your study is confined to studying what you had set out to study. Delimiting the
research problem can be achieved if you read extensively and intensively the
problem you plan to investigate and from the literature specify clearly what is it
you want to study.
The literature also provides insight into the approaches and methodologies
adopted by different researchers. A common mistake made by students is to pay
attention to only the findings of studies. Besides findings, students should also
examine the methodologies used. There could be unique approaches adopted
which you might want to replicate in the Malaysian context. For example, in
most studies reviewed the sample used tended to be university students and
your study is an attempt to use secondary school students, which could be a
unique contribution to the field because it is different.
Some of the research studies include a section on Recommendations for further

research which indicates whether you are on the right track in studying
something that has not been explored before. These suggestions are significant
because they express the insight of the researcher after having studied the
phenomenon.
SELF-CHECK 2.2
1. What is the Review of Literature?

2. What is the purpose of the Review of Literature?

2.5 STEPS IN DEVELOPING A LITERATURE

REVIEW
Three steps in developing a literature review are:
(a) Step 1: Select a Research Topic
(i) Choose a topic of current interest ă your goal is to summarise and

evaluate findings of a line of research. Pick a research topic about
which articles are continuing to be published.
(ii) Choose a well-researched area ă an area that is well-defined and well

studied will give you more lines of research to choose from. A line of
research is a series of studies by the same individual. An area of major
research interest will have several lines of research.
(iii) Narrow your topic ă It is far more satisfying, to both the writer
and the reader, to restrict your topic and cover it in depth.
Comprehensiveness and narrowness of topic go hand in hand.
(iv) Write about what interests you ă If you are interested in the topic, you
are likely to already know something about it, which will make it
easier to gather information.
(b) Step 2: Collect and Read the Relevant Articles
(i) Do a preliminary search ă visit the library or do an online literature

search before you even decide on a topic.
(ii) Search for helpful articles ă Some articles will contribute more than
others to your understanding of a topic. Sometimes you can find a
pivotal article that can serve as a foundation for your study (the
references will lead you to other similar studies).
(iii) Find readable articles ă some areas of research will be harder to

understand than others. Scan the research articles in the topic areas to
decide on their readability.

(iv) „Read, Read, Read‰. That is the bottom line of doing a review. People
have different ways of doing a literature review. A common technique
used by many graduate students is to use a „Note Card‰ (see below).
It may be a rather old-fashioned technique, but has proven to be
most effective. Many graduate students will testify to this, despite
advancements in computer technology.
 Read the easier articles first.
 Identify (1) the problem statement, (2) the research questions or
hypotheses, (3) method used, (4) the findings, and (5) how the
findings were interpreted.
 Jot down the contents of the article using the following Note Card
illustrated in Figure 2.4.
Title: …………………………………………………………………….
Author/s: ………………………………………………………………..
Source: …………………………………………………………………..
Problem Statement: ………………………………………………………
Methodology:…………………………………………………………….
Findings…………………………………………………………………
Conclusion:……………………………………………………………….
Comments:……………………………………………………………….
Figure 2.4: Note card
(c) Step 3: Write the Review
(i) Introduce your research questions (what it is, why it is worth

examining). Begin your review with some theme (or point) that you
want to emphasise. [REMEMBER YOU ARE WRITING FOR THE
READER and not for yourself].

(ii) Briefly outline the organisation of the paper. Organisation is of utmost

importance, so make the structure known to your reader. For example,
tell the reader that you will present research supporting first, one
side, then the other. Alternatively, if addressing three methodologies,
briefly describe them and state that you will compare the results from
the three methods.
(iii) Describe, compare and evaluate studies in terms of the:

 Research assumptions;
 Theories;
 Hypotheses stated;
 Research designs used;
 Variables selected (independent and dependent); and
 Researcher speculations about future studies.
(iv) Discuss the implications of studies (your judgment or what the

studies show, and where to go from here).
(v) Most important of all, avoid plagiarism. Give due recognition to the
works of other people. It does not cost anything to acknowledge
sources. In fact, it shows the breadth and depth of your review, and
the thoroughness of your work.
2.6 COMMON WEAKNESSES

In writing the review of literature, beginning researchers make the following
common errors:
(a) The presentation is a mere description of various studies without making

an effort to show how the studies are related to the main aim of your study
and the research questions of your study.
(b) The presentation is a mere listing of the studies without an attempt to show
how each study is similar or different. Use connectives such as however, on
the other hand, similarly, but and so forth.
(c) Poor citations; in education it is normal practice to adopt the format

proposed in the manual published by the American Psychological
Association (commonly known as the „APA style‰).

(d) Hurriedly reviews the literature and relies too heavily upon secondary
sources.
(e) At times there is evidence to suggest that students have not read the
original works but instead have taken someoneÊs work and cited it as
though they had read the primary source.
(f) There is also evidence of „cut and paste‰ which SHOULD NOT be
encouraged. You must have read the original works and know in detail
every study that you cite.
(g) Articles or reports that are included are not critically evaluated. Critically
evaluate the research questions, the methodology used, the statistics used,
the conclusion arrived at and recommendations made by the researcher
(details about evaluation of articles is discussed in 2.7).
SELF-CHECK 2.3
1. „Read, Read, Read‰. Comment on this phrase.

2. What are some weaknesses of graduate students when writing the
Review of Literature?
2.7 SOURCES
A good literature review requires knowledge of the use of indexes and abstracts,
and the ability to conduct exhaustive bibliographic searches. You should be able
to organise the material meaningfully, describe, critique and relate each source to
the subject of the inquiry, and present the organised review logically and most
importantly to correctly cite all sources mentioned (Afolabi, 1992). Generally,
there are two main sources of materials:
(a) Secondary Sources: This includes materials written by an author/s who

was not a direct observer or participant in the events described. If you read
a textbook on „Educational Psychology‰, it would be a compilation of the
views and empirical works of other authors rearranged into a textbook. The
textbook is a review of research done by others and interpreted by the
author. This interpretation by the author of the textbook would be classified
as a secondary source (be aware that the interpretation may be biased).
Secondary sources are useful because they provide a quick and relatively
easy method of getting an overview of current thinking in the field.

(b) Primary Sources: This includes direct descriptions of events by a person

who actually conducted the investigation. Most primary sources are found
in research journals. However, there are also abundant reports of research
conducted by individuals, groups of individuals and organisations.
How do you search for research articles, research reports, etc.? You can start by
referring to preliminary reference sources such as, indexes and abstracts intended
to help you identify and locate research articles and other primary sources of
information. The following are well-known indexes and abstracts in education:
(a) Education Index ă provides a list of articles

published in education journals and books about
education.
(b) Psychological Abstracts ă contains abstracts of
articles appearing in over 900 journals and other
sources in psychology and related issues.
(c) Educational Resources Information Center (ERIC) ă transmits the
findings of current educational research to teachers, administrators,
researchers and the public.
(d) Resources in Education (RIE) ă provides abstracts of papers presented at
education conferences, progress reports and final reports of projects
which may not appear in education journals.
(e) Current Index to Journals in Education (CIE) ă indexes over
800 education journals and includes more than 1000 articles each month.
Specialised Areas
(a) Child Development Abstracts and Bibliography
(b) Exceptional Child Education Resources (ECER)
(c) Education Administration Abstracts
(d) Physical Education Index
Another way to obtain information about research done is to examine journals,

handbooks and encyclopaedias that locate and review research for particular
topics. The following are the more well known materials in education.

(a) Review of Educational Research ă journal published by the American

Educational Research Association (AERA) covering critical issues and
reviews of research literature on important topics and issues.
(b) Review of Research in Education ă presents critical essays that survey
and synthesise educational research in important problem areas.
(c) Encyclopedia of Educational Research ă best single source of information
on educational research with contributions from among leading
educational researchers.
(d) Handbook of Research on Teaching ă contains reviews of various aspects
of research on teaching such as methods and techniques of teaching,
teaching specific school subjects, and problems of teaching.
Other sources of information are theses and dissertations that have never been
published. The following are important sources that provide abstracts of mastersÊ
theses and doctoral dissertations in education:
(a) Dissertation Abstracts International ă a compilation of abstracts of

doctoral dissertations. Under the education section are subtopics such as
adult education, art education, preschool, teacher training and so forth.
(b) MastersÊ Theses in Education ă this is a listing of mastersÊ theses in about
40 major educational topics. It includes the names of authors, titles and
institutions.
ACTIVITY 2.2
What do you think are some of the problems graduate students face
when doing the Review of Literature for their theses or research projects?
2.8 EVALUATING RESEARCH ARTICLES

As mentioned earlier, in writing your Review of Literature it is essential that
you are able to interpret the works of others. How do you go about evaluating
research articles? The procedure for evaluating research articles is shown in
Figure 2.5. You should keep in mind that in educational research, the findings of
previous research tends to be inconclusive as results are often contradictory. This
may leave you at a loss to decide which, if any, to accept. However, this problem

can be resolved through a critical evaluation of previous work in which the

strengths and weaknesses of each study are carefully weighed (Borg and Borg,
1983). The procedure proposed below consists of two parts: the first relates to
DESCRIBING the research article and the second part relates to CRITIQUING the
article or doing a critique of the article.
Figure 2.5: The five-step procedure of evaluating an article
(a) Step 1: Read the Abstract

(i) What is the research about? Did the study specify the purpose or
objective?
(ii) Did the researcher describe the design that was used?
(iii) What was the rationale or reasons for the research?
(b) Step 2: Read the Introduction

(i) Describe:
 Keep in mind that the writer is assuming that the reader is an
expert in the field or at least has some background knowledge
about the field.
 References made may be brief because it is assumed you know
the people in the field (e.g. if you are reading about „intelligence‰
then names like Sternberg, Gardner, Thurstone, Spearman, should
be known to you).
 Writer assumes you know the concepts in the field (e.g. burnout,
metacognition, inductive reasoning, organisational climate).
 The rationale given for the study and why the research questions
or hypotheses were put forward.

(ii) Critique:
 Was the reason for answering the research question or testing the
hypotheses convincing or just attempting to appeal to your emotion
or merely seeking endorsement from well-know authorities in the
field?
 Did the research questions or hypotheses follow logically from
theory?
 Was there a tendency to oversimplify the theories or studies
reviewed?
(c) Step 3: Read the Methods Section

(i) Describe:
 The writer(s) should describe the background of the subjects used,
the number of subjects and the method used to collect data.
 The design of the study should be described in sufficient detail.
There should also be justification for the study.
 Development of the instrument(s) is described (or use of
someoneÊs instrument) and there is mention about pilot-testing the
instrument(s) and reliability and validity figures are given.
(ii) Critique:
 Was it clear how the subjects were selected?
 Was it adequately explained how the instrument or treatment was
administered?
 Did the researcher discuss the issues of validity and reliability?
 Was the design of the study appropriate? How did it reduce
different types of biases?
 What were the independent and dependent variables?
 Were the statistical procedures used appropriate?
 Were the variables operationally defined?
 If the study was an experiment, was the treatment explained in
sufficient detail? Could it have been done in another way?

(d) Step 4: Read the Results or Analysis Section

(i) Describe:
 Connection between the results and the research questions or
hypotheses.
 Report results relating to the research questions or hypotheses
(whether results were statistically significant).
 Report other statistically significant results.
(ii) Critique:
 Were the results clearly reported and illustrated? (e.g. tables,
graphs).
 Do the statistics test the predictions made in the „Introduction‰?
(e) Step 5: Read the Discussion section

(i) Describe:
 Lists the main findings.
 Relate findings to what was mentioned in the „Introduction‰.
 Speculate about the reasons for the results.
(ii) Critique:
 Is the authorÊs way the only way to interpret the predicted results?
 Can you explain any of the findings the author/s did not explain
or were unable to explain?
 What are the weaknesses or limitations identified by the author or
which you found but were not mentioned?
SELF-CHECK 2.4
1. What are the major aspects of a study you would examine when
describing a research article?
2. When you critique the Methods section and the Results section,
what are you looking for?

Ć A theory is a statement or set of statements that explain and predict

phenomena. It is a statement of a relationship between two or more events.
Ć Theories are tentative and they can be confirmed or disconfirmed.
Ć The review of literature shows what has been done in the field and how the
intended study relates to earlier research.
Ć The review of literature consists of research evidence as well as propositions

and opinions of personalities in the field.
Ć The review of literature delimits the study; relates the methods used by
others; describes the recommendations of earlier works; and provides the
basis for the intended study.
Ć There are primary and secondary sources of material in education.
Ć The best sources of primary material are found in research journals.
Ć All material included in the review should be critically evaluated.
Abstracts of research Importance

Confirming a theory Indexes in education
Critique of research articles Primary sources
Disconfirming a theory Review of Literature
Dissertation and theses abstracts Secondary sources
Evaluating information Theory

1. Select a research topic in which you are interested and locate a

primary and secondary source related to it. Explain why each is a
primary or secondary source.
2. Locate full-text journals in your areas of interest that are available
free on the Internet. Check to see if they are referred journals. Share
what you have found with your coursemates.
3. How will you define plagiarism? What constitutes plagiarism?
4. „Who am I to critique the research of experts in the field, I am only a
student‰. Discuss.
Longman.
Chapter 5: Reviewing the literature. 142ă191.
Jackson, G. (1980). Methods of integrative review. Review of educational

research, vol. 50, 438ă460.
Academic Grammar (n.d.). Literature review: An introduction. Retrieved from:

http://ecdev.hku.hk/acadgrammar/litrev/min.htm
Taylor, D. (n.d.). The literature review: A few tips on conducting it.

Retrieved from: http://www.utoronto.ca/writing/litre.html
Tools for preparing literature review ă A web tutorial (n.d.). Retrieved from:
http://www.gwu.edu/~litrev/
Vaverka, K., & Fenn, S. (n.d.). Background research: The review of literature
Retrieved from: http://kancrn.kckps.k12.ks.us/guide/literature.html

T op i c  Experimental
3 Methodology
LEARNING OUTCOMES
1. Describe what an experiment is;
2. Explain the components of an experiment in education;
3. Identify the threats to internal validity of experiments;
4. Explain how to control the extraneous variables that affect the
internal validity of experiments; and
5. Describe how random assignment is performed;
 INTRODUCTION
„Remember more and think faster with BE SMART‰
„Rewarding pre-schoolers with chocolates has improved
attention in class‰
„The Mental Awareness Approach has proven to be an
effective way to help smokers give up the habit‰
„Cognitive therapy is an effective method for treating
drug addicts‰
„Enhancing self-esteem improves academic performance‰ Source:
faculty.washington.
edu/chudler

40  TOPIC 3 EXPERIMENTAL METHODOLOGY
You may have come across these statements or somewhat similar statements
before. Note that each of them is making a claim that their proposed product,
technique or procedure is effective in enhancing human performance. Obviously,
you would like to know how they went about proving „effectiveness‰. Of
all available research methods, the experimental method is the best. You may
even have conducted science experiments in the laboratory or in the field!
The experimental method was originally used in the field of agriculture
where experiments were conducted to test the effectiveness of various kinds of
treatments such as fertilisers, water and sunlight on plant growth. The method is
also used in medical sciences especially in testing the effectiveness of various
kinds of drugs, procedures and therapy on patients. The experimental method is
widely used in education in which researchers observe the occurrence of a
phenomenon as a consequence of a particular action or intervention.
3.1 THE EXPERIMENTAL METHOD

An experiment is a research method used to determine the effectiveness of a
particular action or treatment on a single or group of organisms. To show that
a particular treatment has an effect or brings about a particular change, the
researcher has to control all other factors that might influence the occurrence of
the particular change. The experimental method is the best method to show
effectiveness of a particular treatment (e.g. teaching method, curriculum
innovation). Experiments are ideally suited for the task of causal analysis (claim
to show „cause and effect‰). No other method of scientific inquiry permits the
researcher to say with confidence that „X (praising young learners) caused Y (to
repeat the task) to happen‰.
Hence, it is important that you use the word „effectiveness‰ carefully, as it only
applies if you are using the experimental method.
See Figure 3.1 which shows a simple experiment to determine whether teaching
young learners using analogies (e.g. blood circulation is like a river and its
tributaries) „causes‰ them to perform better academically in science („effect‰).
The experiment involves administering a treatment (Independent Variable) such
as teaching science using analogies. A pretest (Dependent Variable) is given
before the experiment and the same test or equivalent test is given after the
experiment. The differences between pretest scores and posttest scores will
determine whether teaching using analogies improves performance in science.

TOPIC 3 EXPERIMENTAL METHODOLOGY  41
PRETEST TREATMENT POSTTEST

(Science test) (Teaching using analogies) (Science test)
extraneous variable X extraneous variable Z
extraneous variable Y
Figure 3.1: A simple experiment
However, experiments are difficult to conduct. Many experiments in education

are concerned with testing the effectiveness of certain interventions or educational
practices on student learning, attitudes, perceptions and so forth. A key problem
in conducting experiments is establishing suitable control, so that any change in
behaviour can be attributed only to the treatment introduced by the researcher.
Control means ruling out other possible causes for the changes in the behaviour
of subjects (see Figure 3.1). There are many extraneous variables (irrelevant or
unrelated or unconnected factors) that need to be controlled so that they do not
contaminate or interfere with the findings of the study. Once an extraneous
variable creeps into an experiment, the researcher can no longer draw any
conclusion regarding the causal relationship that exists between the independent
and the dependent variable (Christensen, 1988).
In education, many experiments are conducted in the classroom (natural setting)

and so many factors not related to the treatment may influence performance in
the posttest. With reference to Figure 3.1, some students may have discussed the
topic with their friends at home while others may have viewed a programme
on the topic on TV. So, improved performance on the posttest should not be
attributed solely to the treatment; it could be due to the influence of other factors.
Therefore, it is necessary to control for the influence of these outside factors or
variables in order to attain internal validity.

Some experiments have both an experimental group and a control group. An

experimental group consists of subjects who are exposed to the treatment. For
example, a particular counselling technique is used for a group of juvenile
delinquents. The control group consists of subjects who do not receive the
treatment (i.e. they are not „treated‰ with the counselling technique). A
comparison between the experimental group and the control group determines
the effectiveness of the counselling technique. In some experiments there may be
more than one experimental group; subjects treated with two or three different
methods, techniques or procedures are compared with the control group who do
not receive any of the treatments. You can also compare the effectiveness of
different treatments on the dependent variable.
SELF-CHECK 3.1
1. What is unique about the experimental method compared to other

methods of research?
2. What is „treatment‰?
3. What is the difference between an experimental group and a
control group? Why do you need these two groups?
3.2 CONTROLLING EXTRANEOUS VARIABLES

TO ENHANCE INTERNAL VALIDITY OF
EXPERIMENTS
In conducting experiments, you should ensure that your design has the highest
internal validity possible. What is internal validity? The internal validity of an
experiment is the extent to which extraneous variables (irrelevant variables) have
been controlled or ruled out by the researcher. Internal validity is an indication
that the results you obtain are caused by the treatment you administered and not
some other variable or factor. For example, in your experiment you taught
(treated) one group of four year olds with the whole word method of reading
and discovered that their reading ability increased by 50% compared to the
group who were taught (treated) with the „phonics method of reading‰. How
can you be sure that the increase in reading scores of the whole-word method
group is DUE to the method taught and not some other factors or variables?
Generally, in experiments with high internal validity, the probability that the
treatment caused the change is higher.

Campbell and Stanley (1963) state that experiments are internally valid when the
obtained effect can be attributed to the manipulation of the independent variable.
In other words, if the effects (e.g. improved scores in mathematics) obtained in
the experiment are due only to the experimental conditions manipulated by the
researcher and not to any other variables (factors), the experiment has internal
validity. In any experiment there are always some other factors than the
independent variable (treatment) that can influence the observed effects
(dependent variable). These variables must be identified and dealt with, or at
least, held constant. Cook and Campbell (1979) list a number of factors that can
threaten the validity of experiments. It is important therefore, that you know
these threats in order to take the necessary steps to control their influence.
3.2.1 Time Interval and Threats to Internal Validity

In conducting an experiment, a pretest and posttest is administered to subjects
undergoing the treatment. The time interval between the pre- and post
measurement of the dependent variable can introduce extraneous factors (see
Figure 3.2).
 HISTORY
 MATURATION
 INTRUMENTATION
 TESTING
Pretest TIME INTERVAL Posttest
Figure 3.2: Time interval between pretest and posttest and threats to internal validity
(a) The first is History which includes events that have occurred in the
subjectsÊ environment between the pre-test and the posttest that might
affect the scores. For example, the subjects may have experienced events
during the time lapse that affected their attitude and this is reflected in
the scores of the dependent measure. Generally, the longer the duration
between the pre- and the posttest, the greater the possibility of history
threatening internal validity. But even short time lapses can generate the
history effect.

(b) The second is Maturation in which subjects may change between the pre-
test and posttest resulting in subjects becoming more mature. The change
could be both biological and psychological such as age, learning, fatigue,
boredom and hunger that are not related to specific external events but
reside within the individual.
(c) The third is Instrumentation whereby a change in the instruments used

in the pretest and the posttest can lead to changes in measurement. For
example, an easier test used in the posttest will result in better performance
in the posttest because of the instrument and not the treatment.
Alternatively, in your pretest you used a multiple-choice test to measure
the effect of the treatment. In the posttest an essay test was used.
(d) The fourth is Testing whereby subjects remember the questions in the
pretest and if the same test is given as a posttest, the chances are they may
score higher in the posttest i.e. they have become „test-wise‰. The time
period between the pretest and the posttest should not be too short such
that subjects can recall.
3.2.2 Other Threats to Internal Validity

Besides the above four factors threatening internal validity, there are three other
factors. First is Mortality, which is sometimes referred to as „attrition‰ when
subjects drop out from the experiment which can affect the experiment. This is
especially serious when subjects of a particular characteristic (e.g. high ability)
systematically drop out. Second is Selection Bias when the subjects selected for
the experimental group and the control group are not equivalent before the
treatment, leading to a misleading conclusion. For example, if the experimental
group consists of 50% high ability subjects while the control group consists of
only 25% high ability subjects, higher performance on the posttest may not be
attributed to the treatment but due to non-equivalent subjects in terms of ability.
Third is Regression to the Mean; when subjects with extreme scores on a test are
selected, there is a likelihood that when they are retested later on, a measure that
is correlated with the first test, their scores will move towards the mean. For
example, students who performed poorly are selected for training; their average
posttest scores will be higher than their pretest scores because of statistical
regression even if no training were given.

3.2.3 Subject-Experimenter Effects to be Controlled

You should know that in an experiment the experimenter or researcher interacts
with the subjects. We assume that the subjects taking part in the experiment will
listen to the instructions and perform all tasks according to the way you planned
it. However, you well know that in reality this does not happen because your
subjects are of varied backgrounds and have their own perceptions and opinions.
This may lead to subjects responding to the experiment in different ways that
may affect the experiment; this has been termed „subject-experimenter effects‰.
(a) Subject Effect

The perception of subjects, when they enter the experiment, can affect how
they respond to the tasks required of them. Their perception of the purpose
of the experiment, the task required and the rumours they hear about the
experiment may cause them to behave differently. For example, subjects
who realise that the experiment is about speed of learning and intelligence
will tend to learn the material presented as rapidly as possible to appear
intelligent. Similarly, if the task suggests something about emotional stability,
subjects may respond in such a way as to appear more emotionally stable.
If there is an experimental group and a control group, the tendency is for

the subjects in the experimental group to succumb to the novelty effect
because the treatment given is different from what they are used to.
Subjects tend to be enthusiastic especially in the beginning, which may
wear off as the treatment continues.
(b) Experimenter Effect

The experimenter has a motive for conducting the experiment. He or
she is attempting to uncover the laws of human behaviour through
experimentation. Towards this goal, the experimenter expects subjects to be
perfect respondents who will cooperate and follow instructions carefully.
The experimenter may be too keen to obtain findings that confirm the
hypotheses and this desire is communicated unconsciously to subjects. The
subtle cues presented by the experimenter are picked up by subjects, and
influence their performance in the direction desired by the experimenter.
Certain attributes of the experimenter have been shown to influence
subjects. For example, in some experiments young children respond more
readily to women experimenters compared to their male counterparts.
If there is an experimental as well as a control group, the researcher, if not

careful, may pay special attention to the subjects in the experimental group
which may influence their behaviour. This is called the Hawthorne Effect.

SELF-CHECK 3.2
1. What is meant by internal validity and why is it an important

ingredient in experimental research?
2. Identify the major extraneous variables that need to be controlled in
an experiment.
3. How do these extraneous variables affect the internal validity of
experiments?
4. Explain how subjects and the experimenter can bias the results of
an experiment.
3.3 RANDOM ASSIGNMENT TO ENHANCE

INTERNAL VALIDITY
An important issue when conducting experiments is how subjects are assigned
to the groups. This is important because it determines whether your study is a
true experiment or a quasi-experiment. We will discuss this issue in Topic 4:
Experimental Research Designs. Random assignment means that each sampling
unit (e.g. student, teacher, class, etc) has an equal chance of being selected in the
experiment. In designing an experiment, you should ensure random assignment
as it is the best technique available in establishing that the two or more groups
are equivalent. Equivalent means that the subjects in the two or more groups
have more or less similar characteristics, such as similar ability levels, similar
attitudes, similar number of males and females, similar experiences, similar
socio-economic backgrounds and so forth. If the subjects are not randomly
assigned, there is the possibility that you may have disproportionately high
ability subjects in one group. If they score highly after the treatment, it may be
attributed to the larger number of high ability subjects rather than the treatment.
A popular technique to ensure random assignment is to use the Table of Random

Numbers. Say you have 70 subjects to be assigned to two groups (see Figure 3.3).
Assign number 1 to 70 to the subjects. Then, refer to the Table of Random
Numbers (see Table 3.1) and select a starting point, let us say you take the third
column which has the numbers 26, 54, 37, 98, 39 and so forth. You will select
subject No. 26 assigned to Group 1 followed by subject no. 54 assigned to the
Group 2. Of course you will ignore number 98 because it is outside the
70 subjects. You will continue this procedure until all 70 subjects have been
assigned to the two groups.

Group 1: Students taught by

their peers (n=35)
Subjects
Target selected
Population randomly
(N=70)
Group 2: Students taught by the
teacher (n=35)
Figure 3.3: Random assignment of subjects to two groups
Table 3.1: Table of Random Numbers
23 34 26 91 73 93 83 59 50 51
76 79 54 45 65 13 11 56 91 27
68 57 37 38 45 45 04 85 66 12
45 25 98 63 52 23 03 36 06 08
89 3 39 34 91 94 12 39 13 31
90 26 83 26 21 34 82 07 34 67
23 61 64 65 37 06 54 26 29 75
87 82 51 02 95 64 62 35 96 49
90 71 25 86 62 39 53 49 48 52
12 38 67 09 67 31 45 40 28 31
SELF-CHECK 3.3
Why is it important that subjects are assigned randomly in an

experiment?
3.4 OTHER TECHIQUES TO ENSURE GROUPS

ARE EQUIVALENT
One of the difficult tasks for a researcher using the experimental method is
getting two or more equivalent groups. Imagine the difficulty of finding two
people who are similar in every characteristic such as IQ, attitude, aptitude,
mathematical ability and so forth. As mentioned earlier, random assignment is a

powerful way to ensure that subjects assigned to the various groups will have
more or less similar characteristics. Other techniques can be used to increase the
probability of subjects in two or more groups being equivalent.
3.4.1 Matching
Determine a particular factor, for example, academic performance, that can be
measurable and categorised as High and Low. From the sample, select two
High Ability subjects and randomly assign them to the control group and the
experimental group. Next, select two Low Ability subjects and assign them
randomly to the control group and the experimental. Continue doing this until
all subjects have been assigned and your two groups are matched in terms of
academic performance.
Another technique of matching is to give the pretest, and based on the scores
obtained, assign subjects to the control group and the experimental group.
However, you should ensure that the average score or mean score of the pretest
should be the same for the two groups. e.g. two subjects with mean of 23; two
subjects with mean 30; two subjects with mean 34 and so forth.
3.4.2 Holding One or More Variables Constant

Another method is to hold a particular variable constant. For example, in
an experiment you have difficulty ensuring that the control group and
the experimental have an equal number of high socioeconomic and low
socioeconomic subjects. You could take only low socioeconomic subjects and
assign them randomly to the control group and experimental group if you are
not interested in comparing high and low socioeconomic subjects (see Figure 3.4).
What you have done is to eliminate the socioeconomic factor or variable
by including only low socioeconomic subjects; i.e. controlling by holding the
socioeconomic factor constant across the two groups you are comparing.
Sample of subjects
from low
socioeconomic status
Random Assignment
Group 1 Group 2
Figure 3.4: Control by holding a variable constant

3.4.3 Including an Extraneous Variable in the

Research Design
You could control a variable by including it in the design and making it another
independent variable. For example, you design an experiment to test the
effectiveness of getting students to define concepts using their own words on
performance in economics (see Figure 3.5). However, you find it difficult to
control for prior knowledge in economics among your subjects. You could
include only those who have low prior knowledge based on a test on economics
you administered or you could categorise prior knowledge as High, Medium and
Low based on test scores and treat prior knowledge levels as an independent
variable. However, you should use this technique only if you are interested in the
influence of prior knowledge on performance. What you have done is to control
the influence of prior knowledge on other independent variables by including it
in the research design.
Sample of Subjects with

Different Prior Knowledge
of Economics
High Level of Average Level Low Level of

Prior of Prior Prior
Knowledge Knowledge
Knowledge
Figure 3.5: Including extraneous variable in the design
3.4.4 Analysis of Covariance (ANCOVA)

This is a statistical method used to ensure that the subjects in the control group
and the experimental group are equivalent in various factors. ANCOVA adjusts
the scores on the dependent measure for the differences found on the pretest
and statistically equates the subjects in the control and experimental group. For
example, you are conducting a study on the effectiveness of metacognitive
training on the critical thinking skills of Form 4 students. However, you find that
some subjects in your experiment are high achievers while the others are low
achievers which may influence performance on the critical thinking test (i.e. the
dependent variable). To ensure that all subjects in the control group and the

experimental group are equal in academic achievement, the ANCOVA is used

to adjust scores on the critical thinking test for the difference in academic
achievement.
„You should keep in mind that ANCOVA is an imperfect statistical technique for
equating experimental groups prior to the treatment period. Only the variables
that are measured can be used as covariates. The groups may differ on other
variables, but if these variables have not been measured, they cannot be entered
into the ANCOVA‰ (Borg, W., & Borg, M. (1988). Educational research: An
introduction, p. 684).
SELF-CHECK 3.4
1. Besides randomisation, what are three techniques of increasing the

probability of subjects of two or more groups being equivalent?
2. Explain the differences among these three techniques.
3.5 HYPOTHESIS TESTING

A hypothesis is a statement created by a researcher to speculate the outcome of
the research that he intends to carry out. In other words, a research hypothesis is
a conjecture about the presumed relations between the variables under study. A
research usually begins with the research problem. The research problem may be
framed in the form of a research question. However, the research question may
be too broad or not specific enough for the purpose of conducting statistical
testing. The conversion of a research question into the form of a hypothesis
makes it more realistic and testable.
There are two types of hypothesis, the null hypothesis and the alternative
hypothesis; (the latter denoted by H1 the null hypothesis denoted by H0).
Hypothesis testing involves the following steps:

(a) State the null hypothesis and the alternative hypothesis;
(b) Select a research method that the testing of the null hypothesis is to be
carried out;
(c) Gather the empirical data; and

(d) Use inferential statistical calculation to derive at one of the following

possible outcomes:
(i) Outcome 1: Reject the null hypothesis in favour of the alternative
hypothesis; or
(ii) Outcome 2: Do not reject the null hypothesis.
3.5.1 The Research Hypothesis and the Null

Hypothesis
The research hypothesis or the alternative hypothesis is the experimental
outcome that the researcher conjectures. For example, a researcher believes that a
certain treatment (e.g. inductive approach) has a positive effect (e.g. enhancing
creative thinking ability) of primary school students. In order to test his
conjecture, he carries out an experimental study involving two groups, the
experimental group and the control group. The experimental group receives the
inductive approach treatment but the control does not undergo such treatment.
At the end of the experiment, both groups were evaluated using a test instrument
to measure creative learning abilities of both groups.
It is however impossible to test the research hypothesis directly. It is necessary

first to state a null hypothesis and then to assess the probability that this hull
hypothesis is true. Here, the null hypothesis states the negation of what the
researcher conjectures. In this example, the null hypothesis states that there is no
difference between the two groups in their creative thinking ability (test scores).
Statistically, the hypothesis can be expressed as follows:
Ho: 1 =  2
OR
Ho:  1 ă  2 = 0
Where ø1 is the mean test score for the experimental group (Group 1);
and ø2 is the mean test score for the control group (Group 2)
Both equation A and equation B indicate that there is no significant difference

between the mean test scores of Group 1 and Group 2.

In this example, since the researcher conjectures that the inductive approach
helps in improving creative thinking ability, the alternative hypothesis therefore
can be statistically expressed as follows:
Ho: 1 >  2
OR
Ho:  1 ă  2 > 0
3.6 TEST OF SIGNIFICANCE
In order to reject the null hypothesis, it is necessary to analyse the data

statistically. Why is this necessary? For example, in your experiment you
obtained the following:
Mean Standard
Deviation
Experimental Group 30.4 3.7
Control Broup 28.3 4.1
To the naive person, he or she might conclude that the experimental group
performed better than the control group because the mean score is higher by 2.1,
thus proving that the treatment is effective. This is misleading because it is likely
that the differences in the mean between the experimental group and control
group could have occurred by chance. In order for you to accept or reject the null
hypothesis, it is necessary that you analyse the data statistically because you
want to be sure that the treatment administered produced a real effect. How
do you determine that the difference between the two groups is caused by
the treatment and not some other extraneous variable? You could repeat the
experiment and see if you get the same results which will provide evidence of the
reliability of the obtained findings.
However, this is not an economical approach and for this reason statistical tests
are preferred. The test of significance enables one to determine whether the
amount of difference between the two groups is due to chance or due to the
treatment. Does a large difference between the mean score of the experimental
and control group indicate that the difference is real? Even large differences
could occur by chance, although the probability of this happening would be
very low. The most common practice is to state a significance level that must
be reached; which is a statement of the probability that an observed difference

is a chance difference. The most common significance levels are .05 and .01;
regardless whether you are using the t-test, F-test or the chi-square.
If you decide from the onset of the experiment that the .05 significance level is to
be used, it means that you will accept as a real difference only one that is so large
that it could have occurred by chance only 5 times in 100 (i.e. 95% not due to
chance). If the .01 significance level is selected, then the difference can be
expected to occur only 1 time in 100 by chance (i.e. 99% not due to chance).
SELF-CHECK 3.5
1. Why is the statistical test of significance used to determine

differences between means?
2. Explain the difference between .05 and .01 level of significance.
ACTIVITY 3.1
Age Group Pretest Mean Posttest Mean

Males 52.4 57.2
Females 53.1 64.5*
note: * significant at p < .05
The table above shows the pretest and posttest means on a critical
thinking skills test. The subjects were taught critical thinking skills for
one period (40 minutes) a week for six weeks.
1. Give a title for the study.
2. State TWO null hypotheses based on the data above.
3. State TWO conclusions based on the findings.
4. What are the independent and dependent variables?
5. Provide an operational definition for the treatment.

 An experiment is a research method used to determine the effectiveness of a

particular action or treatment on a single or group of organisms.
 The experimental method is the best method to show the effectiveness of a

particular treatment.
 The internal validity of an experiment is the extent to which extraneous

variables have been controlled or ruled out by the researcher.
 History, maturation, testing, selection and instrumentation threaten the

internal validity of experiments.
 Random assignment means that each sampling unit has an equal chance of
being selected in the experiment.
 Random assignment increases the likelihood that groups are equivalent.
 Other methods of ensuring equivalence of groups are matching, holding

variables constant, including variables in the design and ANCOVA.
 A hypothesis is a conjecture (guess or speculation) about the presumed

relations between variables.
 The test of significance enables one to determine whether the amount of

difference between two groups is due to chance or due to the treatment.

ANCOVA Matching
Control & experimental groups Maturation
Directional hypothesis Null hypothesis
Equivalent groups Pretest & posttest
Experimenter effect Random assignment
History Selection
Holding a variable Subject effects
Hypothesis testing Table of random numbers
Including the variable Test of significance
Instrumentation Testing
Internal validity The experiment
1. Explain when you would use an experiment in educational

research.
2. What do you mean by the statement that „Experiments allow the
researcher to make causal statements‰?
3. Why should you be concerned about the internal validity of an
experiment?


Routledge.
Chapter 2: Experiments [available at eBrary].
Johnson, B., & Christensen, L. (2000). Educational research: Quantitative and

qualitative approaches. Needham Heights, MA: Pearson Education.
Chapter 8: Experimental research.
Rinehart & Winston.
Chapter 4: Internal validity: Why researchers value experimental designs.
Abrahams, D. (n.d.). Introduction to research design. Retrieved from:

http://www.socialresearchmethods.net/tutorial/Abrahams/sbk16.htm
Abrahams, D. (n.d.). Design notation. Retrieved from:

http://www.socialresearchmethods.net/tutorial/Abrahams/dnote.htm

Topic  Experimental
4 Research
Designs
LEARNING OUTCOMES
1. Define what a research design is;
2. Distinguish the ways in which good research designs differ from
weak research designs;
3. Explain the differences between a true experimental design and a
quasi-experimental design;
4. Elaborate on the concept of main effects and interaction; and
5. Discuss the role of hypothesis testing in an experiment.
 INTRODUCTION
What is meant by research design? According to Christensen (1988) the „research
design refers to the outline, plan, or strategy specifying the procedure to be
used in seeking an answer to the research question. It specifies „how to collect
and analyse the data‰ (p. 219). The design of an experiment will show how
extraneous variables are controlled or included in the study (refer to the control
techniques discussed in Topic 3). The design will determine the types of analysis
that can be done to answer your research questions and the conclusions that can
be drawn. To what extent your design is good or bad will depend on whether
you are able to get the answers to your research questions. If your design is
faulty, the results of the experiment will also be faulty. How do you go about
getting a good research design that will provide answers to the questions asked?
It is not easy and there is no fixed way to go about it. The logical thing to do
would be to examine different research designs, discover their strengths and
weaknesses, and make a decision.

58  TOPIC 4 EXPERIMENTAL RESEARCH DESIGNS
You should have an in depth understanding of your research problem; such as

the treatment you want to administer, the extraneous variables or factors you
want to control and the strengths and weaknesses of the different alternative
designs. You should be clear about your research question/s and what it is you
intend to establish. You should avoid selecting a design and then trying to fit
the research question to the design. It should be the other way round! Most
important is to see if the design will enable you to answer the research question.
You should be clear what factors you wish to control so that you can arrive at a
convincing conclusion. Choose a design that will give you maximum control over
variables or factors that explain the results obtained.
4.1 SYMBOLS USED IN EXPERIMENTAL

RESEARCH DESIGNS
A research design can be thought of as the structure of research, i.e. it is
the „glue‰ that holds all of the elements in a research project together. In
experimental research, a few selected symbols are used to show the design of a
study.
O = Observation or Measurement (e.g. mathematics score, score on

an attitude scale, weight of subjects, etc.).
O1, O2, O3 ⁄⁄⁄⁄ On = more than one observation or measurement.
R = Random assignment: subjects are randomly assigned to the

various groups.
X = Treatment which may be a teaching method, counselling

techniques, reading strategy, frequency of questioning and so
forth.
4.2 WEAK DESIGNS

We will discuss three types of weak designs which are one-shot design, one-
group pretest-posttest design and non-equivalent posttest-only.

TOPIC 4 EXPERIMENTAL RESEARCH DESIGNS  59
4.2.1 One-shot Design

For example, you want to determine whether praising primary school children
makes them do better in arithmetic (see Figure 4.1). You measure arithmetic
achievement with a test. To test this idea, you choose a class of year 4 pupils and
increase praising of children and you find that their mathematics performance
significantly improved.
X O
(praise) (scores on a
mathematics test)
Figure 4.1: One-shot design
You conclude that praising children increases their mathematics score. This
design is weak for the following reasons:
(a) Selection Bias: It is possible that the pupils you selected as subjects were
already good in mathematics.
(b) History: The school had organised a motivation course on mathematics for
year 4 pupils. So, it is possible it might influence their performance.
4.2.2 One-group Pretest-posttest Design

To ensure that there was no pre-existing characteristic among the school children,
a pretest may be administered (see Figure 4.2). If the children performed better in
mathematics after praising compared to the pretest, then you can attribute it to
the practice of praising.
This design is weak for the following reasons:

(a) Maturation: If time between the pretest and posttest is long, it is possible
that the subjects may have matured because of developmental changes.
(b) Testing: Sometimes the period between the pretest and the posttest is too
short and there is the possibility that subjects can remember the questions
and answers.
O1 X O2
(mathematics pretest) (praise) (mathematics posttest)
Figure 4.2: One-group pretest-posttest design

ACTIVITY 4.1
Twenty pupils who were weak in arithmetic were taught using the
Zandox method. Three weeks later when they were tested, their
arithmetic scores improved. Thus the Zandox method improves
arithmetic performance.
(a) Which type of research design is this study based on?
(b) What are some problems with this design?
4.2.3 Non-equivalent Posttest-only Design

The main weakness of the previous two designs is the lack of a comparison
group and the consequent difficulty of saying conclusively that the treatment
(„praising‰) contributed to increased mathematics score. In the Non-equivalent
Posttest-only Design an attempt is made to include a comparison group (i.e.
control group) that did not receive „praise‰ (see Figure 4.3). The dashed lines
separating the experimental group and the control group indicates that the
children were not randomly assigned to the two groups. Hence, the two groups
are non-equivalent. Matching can be used but there is no assurance that the two
groups can be equated (see Topic 3). The only way one can have assurance that
the two groups are equated is to assign the children randomly.
EXPERIMENTAL GROUP X O
- - - - - - - - - -- - - - - - - - - - - - - - -- - - - -- - - - - - - - - -- - - - -- - - - - - - - - -
O
CONTROL GROUP (Praise) (mathematics posttest)
Figure 4.3: Non-equivalent posttest-only design
This design is weak for the following reason:
Selection Bias: Since there was no random assignment, it cannot be established

that the two groups are equivalent. So, any differences in the posttest may not be
attributable to giving praise but other factors such as ability, IQ, interest and so
forth.
The three designs described are „weak‰ research designs because they do not
allow for the controlling of extraneous factors that might creep into the
experiment. These extraneous factors may affect the results of the dependent
measure. For example, if attitude towards mathematics and outside tuition in

mathematics are not controlled it may not be possible to conclude that „praise‰
(treatment) affects mathematics performance (dependent variable). Also, weak
research designs do not attempt to randomly assign subjects to the groups being
compared, which could introduce extraneous factors affecting the dependent
measure. Random assignment controls for both known and unknown extraneous
variables might affect the results of the experiment.
SELF-CHECK 4.1
1. Identify the major differences between the one-shot design, one-

group pretest-posttest design and non-equivalent posttest only
design.
2. Why are these designs considered weak?
ACTIVITY 4.2
A teacher assigns one class of pupils to be the experimental group

and another class the control group. Both groups are given a science
posttest. The pupils in the experimental group are taught by their peers
while pupils in the control group are taught by their teacher.
1. Which research design is the teacher using?
2. How will you challenge the findings of the experiment?
4.3 TRUE DESIGNS

In this section, we will discuss some „true‰ experimental research designs. What
is a „true‰ experimental design? According to Christensen (1988), „to be a true
experimental design, a research design must enable the researcher to maintain
control over the situation in terms of assignment of subjects to groups, in terms of
who gets the treatment condition, and in terms of the amount of treatment
condition that the subjects receive‰ (p. 231).
In this topic, we will discuss two major types of true designs (see Figure 4.4):
(a) After-only design; and
(b) Before-after design.
What is the difference between the two designs? The after-only design relies only
a posttest while the before-after design (as the name suggests) relies on both a
pretest and a posttest.
After-Only Research
Design Factorial
Design
True Designs
Before-After Research
Design
Figure 4.4: Types of true experiments
4.3.1 After-only Research Design

The after-only research design gets its name from the fact that the dependent
variable is measured only once after the experimental treatment. In other words,
the posttest is administered once to the experimental group and the control
group (see Figure 4.5). It shows an experiment in which the researcher is
attempting to show the effectiveness of the inductive method in improving the
science problem skills of 17 year old secondary school students. The sample was
drawn from a population and randomly assigned to the experimental and control
group. The experimental group were taught science using the inductive approach
while students in the control group were not taught using the inductive
approach. Instead students in this group were taught the same science content
using the traditional didactic approach („chalk and talk‰ method).
Treatment Posttest
(inductive Method) (Science Problem
Solving Test Scores)
EXPERIMENTAL GROUP R X O
CONTROL GROUP R ă O
Note: R ă random assignment

Figure 4.5: After-only research design
SELF-CHECK 4.2
1. What is the main strength of „true‰ experiments?

2. What is the major difference between the two types of true
experiments discussed?

4.3.2 Factorial Research Design

The factorial design is an after-design research design that allows the study of
two or more independent variables simultaneously and their interactive effects
on the dependent variable. To understand the factorial design, a hypothetical
example is shown in Figure 4.6. The experiment (2  2 factorial design) aims to
examine the effectiveness of two teaching methods (Independent Variable A) on
performance in History (Dependent Variable) among a sample of 17 year old
students of different ability levels (Independent Variable B).
(a) Method (independent variable A) is made of two methods:

(i) Deductive teaching method: In this method, students are presented
with a concept followed by the examples.
(ii) Inductive teaching method: In this method, students are presented
with examples and from these examples they derive the concept.
(b) Ability (independent variable B) is divided into two levels:

(i) High ability: based on their academic performance scores.
(ii) Low ability: based on their academic performance scores.
Independent Variable A
METHODS
Inductive Deductive
(A1) (A2)
Independent ACADEMIC
High (B1) A1 B1 A2 B1
Variable B ABILITY
Low (B2) A1 B2 A2 B2
Figure 4.6: A 2  2 factorial design
So, there are four possible combinations of the two independent variables. Each
of these treatment combinations are referred to as cells (i.e. A1B1; A2B1; A2B1
and A2B2). Subjects are randomly assigned to these four cells within the design.
For the experiment using this factorial design, you are looking for THREE
different kinds of effects: the main effect of the method, the main effect for ability
and the interaction between method and ability.

In this example, you are able to test three null hypotheses:

(a) There is no significant difference between the inductive method and
deductive method on performance in History (MAIN EFFECT ă Method).
(b) There is no significant difference between high and low ability students on
performance in History (MAIN EFFECT ă Ability).
(c) There is no significant interaction between method and ability
(INTERACTION EFFECT).
Figure 4.7: Factorial design showing means for ability and method
The results of the hypothetical experiment are shown in Figure 4.7. The main
effects for methods (variable A) showed that there was a significant difference in
History performance between students taught the inductive (Mean = 50.0) and
the deductive approach (M = 40.0). This means that the method had an „effect‰
on History performance. There was also a main effect for ability (variable B)
where a significant difference was observed between high (M = 55.0) and low
ability (M = 35.0) students on performance in history. Similarly, it means that
ability had an „effect‰ on History performance.
However, the interaction effect was not statistically significant. What is an

interaction? A psychologist is asked, „Does listening to a motivation talk
improve academic performance?‰ When the psychologist replies, „Yes, but it
depends on ⁄⁄..‰ or „It is more complicated than that,‰ he or she is referring
to „interaction‰. An interaction effect tells us about the influence of one
independent variable on another. In the case of our hypothetical example, it
is whether the combination of „method‰ and „ability‰ produced an effect on

performance in history. Though the two variables (method and ability) by

themselves had a significant effect, the combination of method and ability did
not produce an effect on performance in History.
In a 2  2 experiment (the hypothetical experiment), you can obtain 8 basic

patterns of results (see Figure 4.8):
1. A main effect for method, no main effect for ability and No interaction.
2. No main effect for method, a main effect for ability and No interaction.
3. A main effect for method, a main affect for ability and No interaction.
4. A main effect for method, a main effect for ability and an interaction.
5. No main effect for either method or ability, but an interaction.
6. A main effect for method, no main effect for ability, and an interaction.
7. No main effect for method, a main effect for ability and an interaction.
8. No main effects (method & ability) or interaction.
Figure 4.8: Patterns of effects in a 2  2 factorial design
(a) Main Effect for Method and Main Effect for Ability and No Interaction (No: 3)
Let us examine what this means with our hypothetical example. The data in
Figure 4.7 indicates that you have main effects for both method and ability.
Look at the first row. You can see that high ability learners treated with the
inductive method (M = 60.0) scored higher than high ability learners treated
with the deductive method (M = 50.0). Looking at the next row, you see that
low ability learners treated with the inductive method (M = 40.0) scored
higher than low ability learners treated with the deductive method (30.0).
You can see this in Figure 4.8.
60 High ability
Score
on the
History 50
Test
40
Low ability
30
Deductive Inductive
METHOD
Figure 4.9: Graph showing no interaction

Looking at the columns tells you about the effect of ability. You see
that high ability learners treated with the inductive method scored 60.0
compared to their low ability counterparts who scored 40.0 (i.e. 20 more) in
the same treatment group. Looking at the second column, you learn that
high ability learners (M = 50.0) treated with the deductive method scored
higher than low ability learners (M = 30.0) treated with the same method.
Thus, it appears that in addition to the method main effect, you have an
ability main effect.
Finally, you also know that there is no interaction because method has not
affected the ability level of students. As Figure 4.7 demonstrates, the effect
of method is independent of ability level and the effect of ability level is
independent of method of instruction. If you graph the means, your graph
should look something like Figure 4.8. The graph confirms what you saw in
Figure 4.7. The high ability line is above the low ability line. Similarly,
ability increases as shown by the fact that both lines slope upwards as they
go from deductive to inductive method. Finally, the graph tells you that
there is no interaction between method and ability on performance in
History because the lines are parallel.
(b) No Main Effect for Method and No Main Effect for Ability but an
Interaction (No: 7)
Let us examine what this means to our hypothetical experiment. According
to Figure 4.10, the means obtained on History performance according to
method reveals no significant difference between the inductive method
(M = 55.0) and the deductive method (M = 55.0). Similarly, for ability there
was no significance difference between high ability and low ability
students. However, there was an interaction and the interaction was
significant (see Figure 4.11). In this figure, you notice that the lines are not
parallel (as in Figure 4.9). Therefore, you have an interaction. What is the
meaning of this interaction since there was no effect for either method or
ability? You could say that method has an effect, but its effect depends on
ability level. Alternatively, you could say that ability has an effect but that
effect depends on the type of method students had been treated with.

Figure 4.10: Factorial design showing means for methods and ability
60 High ability
Score
on the
History 50 Low ability
Test
40
Deductive Inductive
METHOD
Figure 4.11: Graph showing an interaction
SELF-CHECK 4.3
1. What is the main advantage of using the factorial design?

2. Why is the factorial design considered a true experiment?
3. Identify the differences between main effects and interaction effect?

ACTIVITY 4.3
A lecturer conducting an experiment finds that students who were

given lecture notes but did not attend the lecture, performed better
than those who attended the lecture. Refine the study by using a 2  2
factorial design.
4.3.3 Before-after Research Design

The Before-after Research Design is perhaps the best example of a true research
design that incorporates both an experimental and control group to which the
subjects are randomly assigned (see Figure 4.12). This research design is a good
experimental design because it does a good job of controlling extraneous factors
such as history, maturation, instrumentation, selection bias and regression to the
mean. How is this done? Any history events (e.g. certain events subjects may
have been exposed to) that may have produced a difference in the experimental
group would also have produced a difference in the control group. Here it is
assumed that subjects in both groups have experienced the same set events.
Pretest Treatment Posttest
Experimental Group R O X O
Control Group R O - O
Figure 4.12: Before-after research design
4.4 QUASI-EXPERIMENTAL DESIGN

So far, we have examined both weak and strong experimental research designs.
However, in educational research there are times when investigators are faced
with a situation in which all the requirements of a true experiment cannot be met.

For example, sometimes it is not possible to assign students to groups which are
a requirement of strong experimental research. Due to logistical reasons it is
difficult to randomly assign subjects to groups and so, intact groups such as a
class may have to be used. Is it still possible to do an experiment despite these
limitations? The answer is „yes‰; you can use a quasi-experimental design.
According to Christensen and Johnson (2000), a quasi-experimental design is „an

experimental research design that does not provide for full control of potential
confounding variables. In most instances, the primary reason that full control is
not achieved is because participants cannot be randomly assigned‰ (p. 255).
4.4.1 Non-equivalent Control-group Design
Pretest Treatment Posttest
Experimental Group O X O
Control Group O - O
Figure 4.13: Non-equivalent control-group design
The non-equivalent control-group design contains an experimental and control

group, but the subjects are not randomly assigned to groups (see Figure 4.13).
The fact there is no random assignment means that subjects in the experimental
group and control group may not be equivalent on all variables. For example,
you could have more low ability students in the control group compared to the
experimental group. Hence, it may be difficult to establish whether the better
performance of the experimental group is due to the treatment or because there
are more high ability students in the group.
In the Nonequivalent control-group design both groups are given first a pretest
and then a posttest [after the treatment is given to the experimental group]. The
pretest score and the posttest score are compared to determine if there are
significant differences.

When you cannot randomly assign subjects, you can be sure that extraneous
variables or factors will creep into the experiment and threaten its internal
validity. (We have discussed in Topic 3, the factors that threaten the internal
validity of experiments). Do you leave it alone or do something about it?
Knowing that extraneous factors will creep into a quasi-experiment, a good

researcher will take steps to ensure that the subjects in the experimental group
and control group are as similar as possible, especially on important variables
such as academic ability, attitude, interest, socioeconomic status and so forth.
How do you go about doing this?
Cook and Campbell (1979) proposed the following steps to enhance the internal
validity of the non-equivalent control-group design or quasi-experiments in
general:
(a) Selection: ensure that subjects in the experimental and control are
matched in terms of important variables that may affect the results of
the experiment. For example; match subjects in terms of academic ability,
IQ, attitudes, interests, gender, socioeconomic background and so forth).
(b) Testing: ensure that the time period between the pretest and posttest is not
too short such that subjects are able to remember the questions given to
them earlier.
(c) History: ensure that events outside the experiment do not affect the
experiment. The problem is most serious when only subjects from one of
the groups are exposed to such events (e.g. motivation talks, private tuition)
(d) Instrumentation: ensure that the pretest and the posttest are similar. If
a different test is used, you should make sure that the two tests are
equivalent in terms of what it is measuring (i.e. high reliability and
validity).
4.4.2 Interrupted Time Series Design

The interrupted time-series design requires the researcher to take a series of
measurements both before and after the treatment. A single group of subjects are
pretested a number of times during the baseline phase, exposed to the treatment,
and then posted a number of times after the treatment. „Baseline‰ refers to the
testing done before the treatment is designed to alter behaviour.

A hypothetical example may illustrate how the interrupted time series design is
used. Say that you want to determine whether positive reinforcement encourages
inattentive low ability learners to be more attentive. You identify a group of
11 year old low ability learners and get them to attend an experimental classroom
for at least one period each school day (see Figure 4.14). In this classroom,
subjects are taught reading skills in a positive environment where they are
praised and rewarded for attentive behaviour that is focussed on the given task
or activities. Before the students are sent to the positive treatment classroom their
behaviour is observed over three sessions in their regular classroom with regards
to their attentiveness. This is to obtain baseline data where their behaviour is
recorded in its freely occurring state. The treatment lasts for three weeks and
after the treatment, subjects are observed for their attentiveness and focussed
behaviour.
Multiple Multiple
PRETESTS TREATMENT POSTTESTS
O1 O2 O3 X O4 O5 O6
Figure 4.14: Interrupted time-series design
Multiple Pretests Multiple Posttests

80 (Baseline)
Percentage
of 60
students
who were
attentive
40
20
1 2 3 4 5 6
Sessions
Figure 4.15: Percentage of students observed to be attentive and focussed

The result of the hypothetical experiment is shown in Figure 4.15 which

illustrates the percentage of students who were attentive and focussed on the
given task. From this graph you can see that the percentage of students who were
attentive and focussed who were assessed multiple times prior to and after
implementation of the positive classroom environment, making it an interrupted
time-series design. This assessment reveals that the percentage of students who
were attentive and focussed remained rather constant during the first three
baseline class sessions, or the class sessions prior to the implementation of the
positive classroom environment. After implementation of the positive classroom
environment, the percentage of attentive behaviour consistently increased over
the next three class sessions, suggesting that the implementation of the positive
approach had a beneficial effect on the behaviour of inattentive students.
SELF-CHECK 4.4
1. What is the meaning of non-equivalent in the non-equivalent

control group design?
2. How can you enhance the internal validity of quasi-experimental
research designs?
3. When would you use the interrupted time-series design?
4.5 ETHICS IN EXPERIMENTAL RESEARCH

During World War II, Nazi scientists conducted some gross experiments such as
immersing people in ice water to determine how long it would take them to
freeze to death. They also injected prisoners with newly developed drugs to
determine their effectiveness and many died in the process. However, these
experiments were conducted by individuals living in a demented society and
universally condemned as being unethical and inhumane. Research in education
involves humans as subjects: students, teachers, school administrators, parents
and so forth. These individuals have certain rights, such as the right to privacy
that may be violated if you are to attempt to arrive at answers to many significant
questions. Obviously, this becomes a dilemma for the researcher as to whether to
conduct the experiment and violate the rights of subjects, or abandon the study.
Surely, you have heard people say: „I guess we are the guinea pigs in this study!‰
„We are your lab rats!‰

Any researcher conducting an experiment must ensure that the dignity and
welfare of the subjects are maintained. The American Psychological Association
published the Ethical Principles in the Conduct of Research with Human
Participants in 1982. The document listed the following principles:
(a) In planning a study, the researcher must take responsibility to ensure that
the study respects human values and protect the rights of human subjects.
(b) The researcher should determine the degree of risk imposed on subjects by
the study (e.g. stress on subjects, subjects required to take drugs).
(c) The principal researcher is responsible for the ethical conduct of the study
and be responsible for assistants or other researchers involved.
(d) The researcher should make it clear to the subjects before they participate in
the study regarding their obligations and responsibilities. The researcher
should inform subjects of all aspects of the research that might influence
their decision to participate.
(e) If the researcher cannot tell everything about the experiment because it is
too technical or it will affect the study, then the researcher must inform
subjects after the experiment.
(f) The researcher should respect the individualÊs freedom to decline to

participate in or withdraw from the experiment at any time.
(g) The researcher should protect subjects from physical and mental
discomfort, harm, and danger that may arise from the experiment. If there
are risks involved, the researcher must inform the subjects of that fact.
(h) Information obtained from the subjects in the experiment is confidential

unless otherwise agreed upon. Data should be reported as group
performance and not individual performance.
SELF-CHECK 4.5
What are some ethical principles proposed by the American

Psychological Association with regards to doing experiments involving
human subjects?

Ć A research design is a plan or strategy specifying the procedure in seeking an

answer to the research question.
Ć „Weak‰ research designs do not allow for the controlling of extraneous

factors that might creep into the experiment.
Ć Examples of weak designs: one shot design, one-group pretest-posttest

design and non-equivalent posttest-only design.
Ć „True‰ experimental designs enable the researcher to maintain control over

the situation in terms of assignment of subjects to groups.
Ć Examples of true designs: after-only research design, factorial design and

before-after research design.
Ć A quasi-experimental design is a design that does not provide for full control
of potential confounding variables.
Ć Examples of quasi-experimental designs: non-equivalent control-group

design and interrupted time-series.
Ć Researchers conducting experiments involving human subjects should

respect the privacy and confidentiality of subjects.
Experimental design Factorial design

Quasi-experimental design Before-after design
Non-equivalent design Weak research designs
Time series design One-shot design
True research designs One-group pretest-posttest
After-only design Non-equivalent posttest only

1. Make a case for the superiority of true experimental designs.

2. What are quasi-experimental research designs and how do they
differ from true experiments?
3. Discuss the circumstances in which researchers have to use intact
groups.
4. What can a researcher do to increase the equivalence of subjects in
the control and experimental groups in a quasi-experiment design?
5. Graph the following data from an experiment on the effect of
lighting and music on anxiety. The scores represent the mean
results in an anxiety test.
Music
Classical Rock
Lighting Level Dim 45 11
Bright 12 44
Is there an interaction? How do you know?
Christensen, L. (1988). Experimental methodology. Boston: Allyn and Bacon Inc.

Chapter 8: Experimental research designs.
Johnson, B., & Christensen, L. (2000). Educational research: Quantitative and

qualitative approaches. Needham Heights, MA: Pearson Education.
Chapter 9: Quasi-experimental and single-case designs.
Rinehart & Winston.
Chapter 7: Expanding the simple experiment: factorial designs.
Abrahams, D. (n.d.). True experimental designs and their meaning. Retrieved

from:
http://www.socialresearchmethods.net/tutorial/Abrahams/true.htm

Abrahams, D. (n.d.). Pre-experimental designs and their meaning. Retrieved from:

http://www.socialresearchmethods.net/tutorial/Abrahams/preex.htm
Connor, T. (n.d.). Experimental and quasi-experimental research design.

Retrieved from: http://faculty.ncwc.edu/toconnor/308/308lect06.htm

T op i c  Survey
5 Research
Methodology
LEARNING OUTCOMES
1. Define what a survey is;
2. Compare the different types of survey methods;
3. Explain the process of selecting a sample using different techniques;
4. List the seven major steps in conducting survey research;
5. Discuss the advantages and disadvantages of different survey data
collection methods; and
6. Elaborate on the ethics involved in conducting surveys.
 INTRODUCTION
Do you have a curiosity as to what people are thinking, feeling or doing about
various aspects of life? If you do, than the survey would be the best research
method for finding out. You could survey them orally through an interview or
you could give them a questionnaire. By asking questions, you are tapping into
their attitudes, beliefs, and behaviour. The survey has proven to be a powerful
tool in gathering information about peopleÊs perceptions of and intentions
concerning different social, cultural, economic and political issues.
It is quite common for newspapers, the radio and television to report on the
opinions and perceptions of people obtained through surveys (at times using
survey data to sensationalise issues). Surveys are used by various government
agencies, non-governmental agencies, business and scientific organisations
probing into peopleÊs reactions to different issues and trends. For example:

78  TOPIC 5 SURVEY RESEARCH METHODOLOGY
(a) TV stations rely on surveys to profile the people who watch programmes
over their stations.
(b) Automobile companies use surveys to find out customer satisfaction with
their cars.
(c) Statistics departments conduct surveys to measure the consumer price
index with the purpose of advising policy.
(d) Engineering companies gather information about the habits of road users.
(e) Magazine companies conduct surveys to find out about the reading habits
and interests of their subscribers.
(f) Manufacturing companies conduct surveys to determine consumer
acceptance of their products.
Malaysian Teens Grade Dad a Că
A sample of 3212 young people from Hong Kong, Indonesia, Philippines,

Singapore, South Korea, Taiwan, Thailand and Malaysia aged between 14 and 18
took part in the survey. The survey required participants to grade their parents
from A to F for 35 statements given.
The survey was designed to allow children to reflect on and rank their parentÊs
ability to show respect and love, have integrity and patience and be open-
minded, among a myriad of other skills and characteristics.
Although the Malaysian dad ranks lower than the Malaysian mum, a general
analysis of the survey results reveals fascinating details of Malaysian fathers as
seen through the eyes of young adults. For instance:
(a) Dad got a B for telling jokes and C for being fashionable.
(b) Dad does not really allow them much independence (Că) but respects their
privacy (B) and trusts them (B).
(c) Dad takes the trouble to teach them right from wrong (B+).
(d) Dad accepts their own taste in music (B).
(e) Dad talking about sex (D).
Source: Sunday Star, 27 July, 2005, p. 22

TOPIC 5 SURVEY RESEARCH METHODOLOGY  79
ACTIVITY 5.1
Read the newspaper report on ÂMalaysian teens grade dad a CăÂ

1. What are your views on the findings of the survey?
2. What other information would you need about the survey to accept
the findings?
5.1 WHAT IS A SURVEY?

What is a survey? According to Kerlinger (1973), survey research involves
the studying of large and small populations selecting and studying samples
chosen from the populations to discover the relative incidence, distribution
and interrelations of sociological and psychological variables. It is a method of
obtaining information about a population from a sample of individuals. Surveys
can provide a quick, inexpensive and accurate means of obtaining information
from a large group of people. If you want to know about the opinions, attitudes
and perceptions of respondents, the survey is an appropriate method of
collecting data. Besides, describing surveys can also be used to explain the
relationship and differences between variables. The term sample survey is often
used because a sample which is representative of the target population is used.
The survey method is widely used in the social sciences, education, business and
medicine. Basically, information is obtained by asking people questions either
orally or by responding to a written paper or computer screen concerning:
(a) What they know (Who was the first Prime Minister of Malaysia?)
(b) What they believe (Should students be given freedom to express
themselves?)
(c) What they expect (Do you expect to be a famous person?)
(d) What they feel (Do you think your school principal is fair?)
(e) What they have done (How often do you use the computer in a week?)
(f) What they plan (Do you intend to continue studying or start working?)

5.2 TYPES OF SURVEYS

Surveys provide an important source of basic scientific knowledge. Economists,
psychologists, health professionals, political scientists, educationists and
sociologists conduct surveys to study such matters as income and expenditure
patterns among households, the roots of ethnic or racial prejudice, the
implications of health problems on peopleÊs lives, comparative voting behaviour,
factors influencing academic performance, the effects on family life of women
working outside the home and so forth. To serve these different needs, there are
two main types of survey (see Figure 5.1). The types of survey used will depend
on the objectives of the study. If the study aims to get a snapshot of opinions and
practices than the cross-sectional survey would be most appropriate. If the
objective is to compare differences in opinion and practices over time than the
longitudinal survey would be the obvious choice.
Figure 5.1: Types of survey
5.2.1 Cross-sectional Survey

Just like all surveys, the cross-sectional collects information from a sample drawn
from a population. It involves collecting data at one point of time. However, the
time period for collection of data can vary from one week to six months. If you
are using a questionnaire to collect data, you can ask respondents about the past,
present or the future. For example, you administered a questionnaire on the
habits and attitudes towards smoking to 500 students in secondary school aged
between 14ă16 years on 6th September, 2005. The students included male and
females from different socioeconomic backgrounds in the state of Selangor. The
data you obtained is a cross-section of the population at one point of time.

5.2.2 Longitudinal Survey

In longitudinal surveys, data are collected at different points in time in order to
study changes. There are two common types of longitudinal surveys:
(a) Cohort Studies

You identify a specific population (e.g. teachers in Perak who have a
masterÊs degree) and list the names of all members of this population. At
each data collection point, you select a sample of respondents from the
population of Perak teachers with a masterÊs degree and administer a
questionnaire (e.g. about their aspirations). At another point you might
select another sample from the same population of teachers and administer
the same questionnaire. Thus, although the population remains the same,
different individuals are sampled each time. Your aim is to see if there are
changes in perceptions or trends that are present.
(b) Panel Studies

You identify a sample from the beginning and follow the individuals over a
period of time with the aim of noting changes in specific respondents and
explore reasons why these individuals have changed. For example, you
want to find out about changes in racial attitudes among a group of
primary school children. You administer an attitude scale at Year 5 and
than administer the same scale when they are in Year 6 and so on. You
analyse the data to see if there are changes in racial attitudes as children
grow older. The only problem is the loss of subjects which you cannot
replace.
SELF-CHECK 5.1
1. Define survey in your own words.

2. What is the main difference between cross-sectional surveys and
longitudinal surveys?

5.3 SAMPLING
Unlike a census, where all members of the population are studied, surveys gather
information from only a portion of the population of interest ă the size of the
sample will depend on the purpose of the study we will discuss this issue later.
In a good survey the sample is not selected haphazardly or only from persons
who volunteer to participate. It is scientifically chosen so that each person in the
population will have a measurable chance of being selected. This procedure is
called sampling. This way the results can be reliably projected from the sample to
the larger population. Two key words involved in sampling are population and
sample. The word population is defined as all people, objects or events found in a
particular group the researcher is planning to generalise on (Borg & Borg, 1983).
For example, the population of primary school students; graduate teachers;
medical doctors in Malaysia; senior citizens in Selangor; secondary school
students in Sarawak; and so forth.
If the population is small (e.g. school principals in Kuala Lumpur), the researcher
may decide to study all the subjects. When the population is large, the researcher
is not able to study all individuals. It would be too expensive and time
consuming even for large research organisations. Only in a census is the whole
population studied involving enormous expenditure, time and many research
assistants. Just imagine having to administer a questionnaire to 100,000 sixteen
year olds in Malaysian schools!
So the researcher has to select a certain number of subjects or a sample from the
population to study. Regardless of the method used in selection of the sample,
the most important thing is to ensure that the sample is representative of the
population. See Figure 5.2 which shows a population of 10,000 individuals who
possess certain characteristics in terms of gender, socioeconomic status and
ethnicity. If you intend to select 5% of individuals from the population to form a
sample, you should ensure that the sample has similar characteristics to the
population. This is called population validity where the researcher is trying to
show that the sample is representative of the population according to the
variables specified. Using a sample is more economical and if the sample is
selected appropriately, the researcher can make conclusions about the population
based on the results from the sample. This is called generalisation.

POPULATION:
(Total = 10,000 persons)
Characteristics:
- Gender: 5100 males & 4900 females
- Socioeconomic status: (2500 high, 4000
middle, 3500 low)
- Ethnicity: 5000 Malays, 3000 Chinese,
1000 Indians, 1000 Ibans.
GENERALISATION
SAMPLING
SAMPLE (n=500) Sample

Characteristics: Selected
- Gender: 255 males & 245 females from
- Socioeconomic status: (125 high, 200 population
middle & 175 low) (5%)
- Ethnicity: 250 Malays, 150 Chinese,
50 Indians & 50 Ibans.
Figure 5.2: Selection of a sample representative of the population
ACTIVITY 5.2
Refer to Figure 5.2 and explain how the sample selected is representative
of the population.
5.4 SAMPLING TECHNIQUES

As mentioned earlier, surveys rely on samples to make projections about the
population. How does one select a sample? The sampling techniques used will
depend on the objectives of the study and the resources available. Generally,
there are two types of sampling techniques:
(a) Probability sampling: Probability sampling includes techniques that select

samples based on the concept of random selection. Among the techniques
that are based on the concept of random sampling are random sampling,
systematic sampling, stratified sampling and cluster sampling.
(b) Non-probability sampling: Non-probability sampling techniques are not

based on random selection. Among the common techniques are quota
sampling, purposive sampling and convenience sampling.

5.4.1 Probability Sampling Techniques

Let us now look at some examples of probability sampling techniques.
(a) Random Sampling

One of the techniques to increase the probability that the sample selected
represents the population is random sampling. Using this technique, each
member of a population has an equal chance of being selected (refer
to Topic 2 on „random assignment‰). For example you are interested in
surveying the attitudes of graduate teachers toward the teaching profession.
The sample you select will have to represent the attitudes of the target
population (see Figure 5.3). Due to financial and time constraints you are
unable to survey the attitudes of all graduate teachers across the whole of
Malaysia (N = 100,000) and so you decide to confine your study to graduate
teachers in Perak (n = 15,000) which is called the accessible population.
From the accessible population a sample of 100 teachers is drawn.
TARGET POPULATION Graduate Teachers in Malaysia

(N= 100,000)
ACCESSIBLE POPULATION All Graduate Teachers in Perak

(N=10,000)
SAMPLE Sample Randomly Selected

(n= 100)
Figure 5.3: Random selection of a sample
Even though the sample is selected from the accessible population, you
may want to know the degree to which the results can be generalised to the
target population which requires two steps. In the first step, you generalise
from the results of the sample to the accessible population (teachers in
Perak). Second, you generalise from the accessible population to the target
population (graduate teachers in Malaysia). The leap from sample to the
accessible population presents no problem if a random sample of the
accessible population is obtained.
However, in order to make the leap from the accessible population to

the target population, you must gather data to determine the degree of
similarity between graduate teachers in Perak and graduate teachers in the
whole of Malaysia. If you can demonstrate, based on a number of variables
(such as gender, age, experience, ethnicity) that the accessible population

is closely comparable to the target population, you have established

population validity, i.e. the accessible population is representative of the
target population.
How do you select a sample randomly? The usual definition of a random

sample is that it is a procedure in which all individuals in the defined
population have an equal and independent chance of being selected as a
member of the sample. Independent means the selection of an individual is
not affected by the selection of another individual. In other words, each
individual, event or object has the same probability of being selected. For
example, the number of graduate teachers in Perak is 10,000 and you intend
to draw a sample of 100 teachers (see Figure 5.3). When you select the first
teacher, he or she has a 1:10,000 chance of being selected. Once this teacher
is selected, however there are only 9999 cases remaining so that each
teacher has 1:9999 of being selected as the second case. Thus, as each case is
selected, the probability of being selected next changes slightly because the
population from which you are selecting has become one case smaller.
The table of random numbers is often used in the selection of a random

sample (see Table 5.1). You need to obtain a list of the 10,000 graduate
teachers in Perak and assign number 1 to 10,000. Using the table of random
numbers, randomly select a row or column as the starting point. Say, that
you select Column 5. Select all the numbers that follow in that column. So,
you will select teacher assigned number 7332, followed by teacher assigned
number 6516, then teacher assigned number 4553 and so forth. If you need
more numbers, you can proceed to the next column until you have enough
numbers to make up the desired sample (i.e. n = 100).
Table 5.1: Table of Random Numbers
1 2 3 4 5 6 7 8 9 10
1 2345 3445 2678 9156 7332 9332 8345 5950 5023 5189
2 7612 7989 5456 4523 6516 1345 1123 5636 9189 27452
3 6823 5732 3702 3808 4553 4589 0467 8506 6612 12136
4 4598 2564 9860 6360 5245 2347 0391 3623 0620 0850
5 8956 357 3934 3495 9112 9472 1254 3998 1390 3194
6 9059 2691 8395 2634 2189 3465 8223 0745 3487 6709
7 2312 6120 6425 6556 3720 0639 5490 2614 2950 7556
8 8787 8236 5153 0202 9530 6490 6220 3523 9691 4917
9 9063 7190 2590 8696 6267 3923 5360 4937 4854 5223
10 1298 3820 6737 0932 6719 3154 4532 4046 2860 3191

(b) Systematic Sampling

Similar to random sampling, systematic sampling is used to draw a sample
from a population. It is often used instead of random sampling. It is also
called an Nth name selection technique. You begin by having a list of the
names of members in the population in random order.
(i) You want to select a sample of 100 students from a population of
1000 students.
(ii) You divide the population by the sample needed (1000/100) = 10.
(iii) You select at random a number smaller than 10.
(iv) You start with that number (e.g. 7) and select every seventh name
from the list of the population.
After the required sample size has been calculated, every Nth record is
selected from a list of population members. As long as the list does not
contain any hidden order, this sampling method is as good as the random
sampling method. Its only advantage over the random sampling technique
is simplicity. Systematic sampling is frequently used to select a specified
number of records from a computer file.
(c) Stratified Sampling

In some surveys, you want to ensure that individuals with certain
characteristics are included in the sample to be studied. For this purpose,
the stratified sampling technique is used. For example, if you are interested
in studying inductive reasoning among 12 year olds according to ability
and gender in Petaling District, you would want to ensure that you have a
proportionate number of high and low academic achievers as well as a
proportionate number of males and females. In order to avoid a sample that
does not include a sufficient number of students of each sex at each ability
level, a stratified sample may be selected. All 12 year old students in the
district are divided into one of the following four groups: male high
achievers, female high achievers, male low achievers and female low
achievers. Subsamples are then selected at random from the population to
fill each of the four groups (see Table 5.2).
Table 5.2: Sample Stratified According to Gender and Academic Performance
Males Females
High achievers n=? n=?
Low achievers n=? n=?

However, the proportion of students randomly selected to fill each of the

groups is the same as the population. For example, if male high achievers
made up 25% of all 12 year olds in Petaling District, the proportion of male
high achievers in the sample should also be 25%. If you do not follow this
procedure, the results obtained from the sample will produce an inaccurate
view of the population. Stratified samples are most appropriate when you
want to make comparisons between various subgroups and to ensure the
sample is representative of the population in terms of the critical factors
you want to study.
ACTIVITY 5.3
You have been appointed to lead a research team assigned with the
task of finding the reasons teenagers smoke. The team has decided to
conduct a nation-wide survey involving students between 14ă16 years
of age in secondary schools.
1. Suggest THREE research questions that will guide data collection.
2. Explain how you plan to draw the sample of students using
stratified sampling. What subgroups would you include?
3. What further information do you need to draw a representative
sample?
(d) Cluster Sampling

In the techniques of sampling discussed thus far, the unit of sampling is
the individual student, teacher or principal. In cluster sampling, the unit
of sampling is not the individual but rather a group of individuals. For
example, in a district there are 25 villages and in each village there are
an estimated 20 pre-school children. You need a sample of about 100 pre-
schoolers but this would mean going to many villagers if random sampling
is used. In cluster sampling, you select five villages randomly from the total
25 villages. You study all the pre-school children in the five villages you
have selected. The advantage of using cluster sampling is that it saves time
and money, especially if the population is dispersed. The weakness is that it
is less accurate compared to the other techniques of sampling because the
subgroups may be more heterogeneous rather than homogeneous.

SELF-CHECK 5.2
1. What is meant by random sampling?

2. What is the difference between stratified sampling and systematic
sampling?
3. How is cluster sampling different from all the other types of
sampling techniques discussed?
ACTIVITY 5.4
N=118
N=134 N=105 N=113
N=154
N=129 N=98
N=111 N=109
The above is a diagram showing the location of nine housing areas

dispersed all over a city with the number of senior citizens indicated in
each area.
1. Explain how you would select a sample of about 200 senior citizens
you intend to interview regarding how they spend their time using
cluster sampling.
2. What factors should you consider when using cluster sampling?
5.4.2 Non-probability Sampling

In many situations it is not possible to ensure that the sample will be selected
based on random selection. So the sample has to be chosen by some other
way. Non-probability sampling is based on the researcherÊs judgment and hence
biasness will enter in sample selection and distort the findings of the study.
Nonetheless, non-probability sampling techniques are used because of practical
reasons. For example, non-probability techniques are used to save costs and time;
when its use can satisfactorily meet the objectives of the study and it may be the
only feasible method, given the spread and features of the population. Among
the more common non-probability techniques are quota sampling, purposive
sampling and convenience sampling.

(a) Quota Sampling

In situations where you are unable to ensure that subjects are randomly
assigned to the subgroups you can use quota sampling. Like stratified
sampling, you first identify the subgroups and their proportions as they
are represented in the population. Following which you select subjects
based on convenience and your judgment to fill each subgroup. When
using this method, you must be confident that the chosen sample is truly
representative of the population. Obviously, you should state clearly the
criteria for your selection of the sample; especially when you make
generalisations of the results to the population.
(b) Purposive Sampling

The sample is selected on the basis that members conform to certain criteria.
The researcher uses his or her judgement to select cases to answer certain
research questions. The form of sampling is usually used when the
population is small (such as in case study research) and when the main
purpose is to select cases that are particularly informative. It is very useful
in the early stages of an exploratory study. The main weakness of this
technique is that the sample may have characteristics which differ from
population characteristics.
(c) Convenience Sampling

In this sampling technique, researchers have the freedom to choose
whomever they find. You simply sample people who are easy to survey.
The sample is chosen rather „haphazardly‰ until the required sample size is
met. It is less expensive and easy to conduct; and is considered the most
useful procedure to test ideas in exploratory research. This sampling
technique is considered the least reliable because of its high incidence of
biasness. Newspapers ask people to mail in their responses to a survey
question; radio stations ask people to call in their opinions to a question. As
you can see, the convenience sample can provide you with a lot of data but
you do not know whether the sample represents your population.
SELF-CHECK 5.3
1. What is the major difference between probability and non-

probability sampling techniques?
2. Why are non-probability sampling techniques used despite their
many weaknesses?

5.5 SAMPLE SIZE

How large must the sample size be? This is a question uppermost in the minds
of many beginning researchers. The general rule is to use the largest sample
possible. The rule is good because we are interested in learning about the
population based on the results of the sample from which it is drawn. The larger
the sample, the more likely the sample mean and standard deviation will be
representative of the population mean and standard deviation. The sample size
required for a survey partly depends on the statistical quality needed for survey
findings; this, in turn, relates to how the results will be used. Even so, there is no
simple rule for sample size that can be used for all surveys. Much depends on
the professional and financial resources available. Often a moderate sample
size that is sufficient statistically and operationally is preferred. For example,
the well-known Gallup Poll in the United States frequently uses samples of
about 1,000 persons to get reasonable information about national attitudes and
opinions.
The following are some guidelines you can use in deciding how large your
sample should be:
(a) When the sample selected has to be broken down into smaller groups
involving comparisons of groups, a larger sample is advisable. For
example, you want to compare the self-esteem of males from low
socioeconomic backgrounds from single-parent families against males from
high socioeconomic backgrounds from single-parent families.
(b) When attrition is expected, especially in longitudinal studies; the longer the
duration of a study, the higher will be the number of subjects who drop out.
To reduce attrition you should keep demands on subjects to the minimum,
fully inform subjects about the study, obtain a strong commitment from
subjects before the study and make frequent contacts with subjects to
maintain interest and rapport.
(c) When the population is highly heterogeneous on the variables being

studied, you need to have a larger sample so that the different characteristics
of individuals are satisfactorily represented. If members in the population
are more or less similar, then you will need a small sample as most of the
characteristics can be captured.

SELF-CHECK 5.4
What are the factors you will consider when making decisions about
sample size in a survey?
ACTIVITY 5.5
Sample Size Calculator (www.surveysystem.com/sscalc.htm)

Go to the above website to access a Sample Size Calculator.
1. What is meant by confidence level and confidence interval?
2. Calculate the sample size required from a population of 9000
students with a confidence level of 95% and confidence interval
(or margin of error) of 5.
5.6 THE PROCESS OF CONDUCTING A SURVEY
The seven steps for conducting a survey are (see Figure 5.4):
Figure 5.4: The seven steps of the survey process

(a) Step 1: Defining the Objectives of the Study

A successful survey requires careful preparation. The first and often the
most difficult question is „Why am I doing this survey?‰ Many studies
start with a general hope that something interesting will emerge, and they
often end in frustration. Putting together a group of questions you feel are
important to students does not constitute a valid and reliable survey.
This initial planning requires some idea of the final analysis; and it may be
useful at the outset to outline the key tables for the final report (better still if
you are able to create dummy tables), and to consider the number of cases
expected in each major cell or subgroup. Conduct a review of the literature
to define terms and justify the theory or theories underlying research
questions.
(b) Step 2: Writing the Items and Construction of the Questionnaire

Survey data is mostly obtained by means of questionnaires. Generally,
questionnaires are an inexpensive way to gather data from a large number
of respondents. They may be either self administered (that is, completed by
the subject), administered at an interview or administered to a group in a
face-to-face session. The crux of a survey is the questionnaire (i.e. what we
ask our respondents). A well-designed questionnaire can provide valuable
information about the opinions, beliefs and practices of groups of
individuals. Questionnaire design is a long process that demands careful
attention and should not be taken lightly. We will discuss in detail how to
design questionnaires in Topic 6 „Instrumentation‰. In this section the
general qualities of a good questionnaire are discussed briefly.
(i) Clarity: Questions must be clear, precise and unambiguous. This is to

eliminate the chance that the question will mean different things to
different people. Avoid the use of colloquial or ethnically sensitive
terms. Technical terms that assume a certain background should also
be avoided.
(ii) Leading Questions: A leading question forces or implies a certain type

of answer. The researcher feels strongly about an issue and assumes
that everyone will be of the same opinion. e.g. Does your principal
treat men and women teachers differently?
(iii) Hypothetical Questions: A hypothetical question is one in which you

are asking respondents to indicate what they think they would do
under particular imaginary circumstances. While they are used in
some attitudinal research, they are difficult to interpret and often give
rise to unreliable answers. e.g. If you were buying a house, what
features would you most want it to have?
(iv) Questions Requiring Prior Knowledge: Questions that tax a

respondentÊs memory may lead to non-responses or inaccurate
replies. e.g. What did you do in the last school holidays?
(v) Order of Questions:

 Take into account the sensitivities of the person to whom they are
addressed ă it is better to start with „What is your date of birth?‰
rather than jump straight into „Has there been a death in the family?‰
 Begin with interesting items. If the first items are boring, there is
little chance that the person will complete the questionnaire.
(c) Step 3: Method of Conducting the Survey
(i) You have to decide the procedures for conducting the survey; how
many people you will survey (the sample size and how they will be
obtained)
(ii) Decide how you will survey your subjects (by phone, in class, mail
format, interview).
(iii) Decide how you intend to distribute and collect the questionnaires;
make follow-up contacts.
(iv) Decide on the level of response that is acceptable (refer to your

research questions and the population the question is being asked).
What percentage of questionnaires returned or answered will you
accept? What will you do if you do not have enough respondents or a
certain sector of respondents is under-represented (e.g. insufficient
number of females)?.
(d) Step 4: Pilot-Testing of the Questionnaire
(i) Pilot-testing a questionnaire is necessary to avoid problems that may

arise when the questionnaire is administered to the whole sample.
Unfortunately, some people consider pilot-testing nothing more than
a ritual while others do not do it at all.
(ii) Before pilot-testing, get some of your colleagues to review the

questionnaire, particularly those who are interested or familiar with
what you are doing.

(iii) The questionnaire should be pilot-tested with a small group of

individuals who are in many respects similar to the sample in the
actual study. When they have finished, ask for their opinions and
suggestions (e.g. Was it too long? Which items were difficult?).
(iv) The pilot-test helps you find out if the content and form of the
questions are satisfactory. You can also get information about:
 The length of time needed to complete the questionnaire;
 The appropriate order of the questions;
 Sufficient space needed for responses;
 Whether the instructions or directions are understood;
 The extent to which the questionnaire is reliable, e.g. if you gave
the questionnaire again four weeks later would you get the same
responses?; and
 The validity of the questionnaire (does it measure what it is
supposed to measure?) We will discuss further these issues in
Topic 6.
(e) Step 5: Administering the Questionnaire
(i) You should gain approval from the respondents you intend to survey.
For example, permission from the Ministry of Education (Educational,
Planning and Research Division), State Education Departments and
school principals should be sought.
(ii) In a small study you can administer the questionnaire yourself. But
in a large survey, you will need helpers. Your field assistants have
to be trained, especially if they are required to observe or conduct
interviews. Such training should be completed before the start of the
main study.
(iii) Teachers are gatekeepers to the students. Gaining the cooperation of

teachers is very important when conducting a survey involving
students. Teachers will be more receptive in conducting the survey
during class time if you explain to them clearly the objectives and
contents of the survey. It would even be better if you could
demonstrate that the survey can be incorporated into the curriculum.
Schedule the survey far enough in advance to allow teachers to make
class time available.

(iv) Consent may be necessary for students to participate in the survey. If

so, parents may play a key role in studentsÊ participation. Ensure that
parents understand why you are asking their childrenÊs opinions.
(v) Do not administer the questionnaire on a day when there are school
activities such as open day, celebration of festivals, sports events or
examinations. Students will be distracted with these events and may
not give the survey their full attention.
(vi) If you are surveying parents, you could send the questionnaire home
with the students. How can you ensure receiving enough returned
surveys? One suggestion is to reward students for returning
questionnaires, e.g. a coupon for a free soft drink.
(vii) Thank the groups that assisted in the survey process. An appreciation
goes a long way toward support and participation.
(f) Step 6: Data Entry and Analysis
(i) Coding and entry of data. Check to ensure accuracy of data entry
and ensure that all codes are valid (for example, „1‰ is for male, „2‰
for female and there should not be a „3‰!) and look for any
inconsistencies.
(ii) Selection of software package to analyse data. Statistical analysis

should only begin when the data set is as „clean‰ as possible.
(iii) If you are unclear of certain statistical procedures, obtain advice from
a person who is well-versed in statistics.
(g) Step 7: Writing the Report

The final phase of any research process is documentation or writing the
research report. The normal components of a research report are:
(i) The introduction (background of the study, rationale for the study,
the aims and research questions, limitations and significance of the
survey);
(ii) The review of literature (previous work done in the field, underlying
theory or core of theories);
(iii) Methodology (description of the sample, how the sample was

drawn, description of the questionnaire used, administration of the
questionnaire);

(iv) Results (data presented in the form of tables and graphs; statistical
analysis; description of the findings);
(v) Conclusion (explanation for the findings, relate to previous literature

and theory, making generalisations and recommendations for future
research) We will discuss in detail „Writing the research report‰ in
Topic 10.
SELF-CHECK 5.5
1. List some of the qualities of a good questionnaire.

2. Why is pilot-testing necessary?
5.7 DATA COLLECTION METHODS USING

SURVEYS
Surveys can be classified by their method of data collection. Generally, there are
two main types of data collection methods: self-administered and investigator
administered (see Figure 5.5). The most common self-administered method of
data collection is the mail survey and more recently the web survey where the
respondents are expected to respond to the questionnaire without the presence of
the investigator.
The investigator administered method of data collection requires the presence of

the investigator such as the telephone interview, face-to-face interview and group
administered questionnaire. Besides the above, extracting data from samples of
medical and other records are also frequently done. In newer methods of data
collection, information is entered directly into computers using devices attached
to TV sets that automatically record the channels being watched.
(a) Mail Survey

Surely you would at one time or another received a questionnaire in the
mail (e.g. credit card companies, automobile companies). There are many
advantages to mail surveys. This method of data collection can be relatively
low in cost. You can send the exact same questionnaire to many people
and they allow respondents to fill it out at their own convenience. Mail
surveys can be most effective when directed at particular groups, such
as subscribers to a specialised magazine or members of a professional

association. The disadvantage of the mail survey is the low response rate.
Also, since the researcher is not present, there is no way for the respondent
to seek clarification if questions are unclear.
(b) Web Survey

A more recent method of data collection is using the web. The
questionnaire is uploaded to a website and respondents are invited to
respond to the questionnaire. While it is less expensive and you can reach
out to a large audience, there are many weaknesses with this method of
data collection. The authenticity of the person responding can be difficult
to prove, response rate may be low and persons responding to the
questionnaire would be confined to those who have internet access which
may not be representative of the population.
(c) Telephone Interview

Telephone interviews are an efficient method of collecting some types of
data and are being increasingly used. They lend themselves particularly
well to situations where timeliness is a factor and the length of the
survey is limited. The telephone interview gives respondents the feeling
of anonymity since the interviewer cannot see them.
(d) Face-to-face Interview

Face-to-face or in-person interviews in a respondentÊs home or office
are much more expensive than mail or telephone surveys. They may be
necessary, however, especially when complex information is to be collected.
(e) Group-Administered Questionnaire

A sample of respondents are brought together and invited to respond to a
structured sequence of questions (Figure 5.5). This is convenient method
because you are able to capture a relatively large sample of respondents
in one sitting (e.g. classroom). Also, the response rate is relatively high. If
the respondents are unclear about the meaning of questions they could
ask for clarification. However, the presence of the researcher may make
respondents feel that their answers are less anonymous and as such they
may be less candid.

Figure 5.5: Data collection methods using a questionnaire
5.8 ETHICS IN SURVEYS

What about confidentiality and integrity in surveys? The confidentiality of the
data supplied by respondents is of prime concern to all who conduct surveys.
For example, in many countries the data collected by the census department
is protected by law. There are acts that guarantee the confidentiality of data
collected by the relevant agencies. Several professional organisations dealing
with survey methods have codes of ethics that prescribe rules for keeping survey
responses confidential. The recommended policy for organisations or individuals
to safeguard such confidentiality includes:
(a) As far as possible use only number codes to link the respondent to a
questionnaire and store the name-to-code linkage information separately
from the questionnaires. For example 001 for respondent Azlina Darus, 002
for respondent Ong Mei Ling and so forth.
(b) The names and addresses of survey respondents should not be made
available to anyone outside those involved in the survey after the responses
have been entered into the computer (individuals and organisations have
been known to sell such databases to companies for marketing purposes
without the consent of individuals involved!).
(c) Omitting the names and addresses of survey respondents from computer
files used for analysis.
(d) Presenting statistical tabulations using broad enough categories so that

individual respondents cannot be singled out.

Respondents should be informed about the purpose of the survey and have the
option not to participate or not to divulge information that he or she feels not
comfortable with. For example, respondents may be reluctant to disclose their
incomes. To overcome this, you may want to use categories (e.g. RM1500ă
RM2000 per month) which may be less intrusive. You should determine in the
pilot-test which items respondents are uncomfortable with, so that you do not
have too many unanswered questions to the point that some research questions
cannot be answered. The questions asked should not in any way attempt to
deceive respondents. The integrity of a survey is enhanced if respondents are
clear about the purpose of the study.
Ć The survey is a method of obtaining information about a population from a

sample of individuals.
Ć Surveys are useful in gathering data about what people are thinking, feeling
or doing.
Ć There are two types of survey: cross-sectional surveys (taken at a particular

time) and longitudinal surveys (compare changes over time).
Ć Since the entire population would be too costly and time consuming to
survey, a sample is drawn from the population.
Ć Two techniques of sampling are probability and non-probability.
Ć Probability sampling is based on random selection while non-probability

sampling is not based on random selection.
Ć The questionnaire is the main tool for data collection in surveys.
Ć There are two methods of data collection using the questionnaire: self-
administered survey and investigator administered survey.
Ć The self administered survey is the mail survey and web survey while
investigator administered surveys include telephone interviews, face-to-face
interviews and group administered questionnaires.
Ć The main ethical issues in conducting a survey are confidentiality and

integrity.

Cluster Sample size

Cross-sectional Sampling techniques
Face-to-face interview Self-administered survey
Group administered questionnaire Stratified
Investigator administered survey Survey
Longitudinal Survey process
Mail survey Systematic
Questionnaire Telephone interview
Random
1. You wish to study moral reasoning among fifteen year olds

in a large district. A total of 4100 fifteen year olds are enrolled
in 105 classrooms in the district. You plan to obtain a total of
250 students using the cluster sampling technique. Describe the
steps you would take in selecting the sample.
2. Discuss some of the problems with telephone interviews that may
affect the results of a survey.
3. One of the drawbacks of mail surveys is the low response rate.
Suggest how you would increase the response rate.
4. When would you use a longitudinal survey rather than a cross-
sectional survey?
5. What are some problems with surveys conducted over the
Internet? How can you overcome these problems?


Routledge. [available at eBrary].
Chapter 5: To interview or not to interview
Chapter 5: Representativeness
Longman.
Chapter 6: Population and sample.
Rinehart and Winston.
Chapter 12: An introduction to survey research. 284ă311.
Garson, D. (n.d.). Survey research. Retrieved from:

http://www2.chass.ncsu.edu/garson/pa765/survey.htm
Trochim, W. K. (n.d.). Survey methods. Retrieved from:

http://www.socialresearchmethods.net/kb/survsel.htm

Topic  Instrumentation
6 and Data
Analysis
LEARNING OUTCOMES
1. Differentiate between instrument and instrumentation;
2. Define what a questionnaire is;
3. Explain the construction of a questionnaire;
4. Design and develop an attitude scale;
5. Analyse data to show significant differences; and
6. Analyse data to show correlations or relationships.
 INTRODUCTION
The most important and sometimes difficult aspect of educational research is
determining the instruments to be used in data collection (see the Self-Esteem
Test in Figure 6.1 which is an example of an instrument). There are numerous
tools that may be used to obtain information. In this topic we will discuss in
detail the process of instrumentation and look at some examples of widely
used types of instruments. We will also discuss two very important concepts
related to instrumentation, namely; reliability and validity. Fraenkel and
Wallen (2001) make a distinction between instrument and instrumentation. An
instrument is a device or procedure for systematically collecting information
while instrumentation includes both the instrument and „the conditions under
which it is used, when it is used, and by whom it is used‰ (p. 81). Examples of
instruments are tests, questionnaires, rating scales, inventories and checklists.

TOPIC 6 INSTRUMENTATION AND DATA ANALYSIS  103
Self-Esteem Test
This test is designed to test your level of self-esteem. Read each item and state whether
you strongly agree, agree, disagree or strongly disagree with each statement by
referring to yourself.
Strongly Strongly
Agree Agree Disagree Disagree
4 3 2 1
1. I am an attractive person.
2. People will dislike me if I am myself.
3. I am happy with my present height.
4. I like my hair.
5. I have something to say in social

situations.
6. I am looked down because I am not good
looking.
7. I am a friend to myself and take care of
myself.
8. I am popular among my friends.
9. I like the colour of my skin.
10. I think I am useless in the company of

others.
Source: Reprinted with permission: © CoPS Project Malaysia (2005). John Arul Phillips
Figure 6.1: Example of Self-Esteem Test
ACTIVITY 6.1
Examine each of the 10 items of the Self-Esteem Test.

1. What do you understand by self-esteem?
2. Identify the two aspects of self-esteem being measured.
3. Why do you think some items are stated in the positive while others
are stated in the negative?

104  TOPIC 6 INSTRUMENTATION AND DATA ANALYSIS
6.1 THE QUESTIONNAIRE

In Topic 5, we discussed in detail the Survey Method where Step 2 involves
„writing the items and constructing the questionnaire‰. When you conduct a
survey, a questionnaire is administered in which subjects respond in writing or
orally in the case of an interview. The subjectsÊ responses to the questions are
summarised, analysed, and reported. The aim is to obtain information about the
characteristics of a particular group. The results from the sample are generalised to
a larger group or the population, if the sample surveyed was randomly selected.
Questionnaires are widely used because they are cost effective and can be
administered to large groups of people. There is greater standardisation in
questionnaires as each respondent receives the same set of questions which
allows interpreting from a large sample. The questionnaire if properly designed
can address many issues in a relatively efficient way. Also, there is assurance of
anonymity which increases the likelihood that the responses are genuine and
reflective of the opinions, perceptions and beliefs of respondents. However, the
quality of data obtained from questionnaires will depend on how well it is
designed. Fowler (1984) suggests the following important criteria when designing
a questionnaire:
(a) Is this a question that can be asked exactly the way it is written?
(b) Is this a question that will mean the same thing to everyone?
(c) Is this a question that people can answer?
(d) Is this a question that people will be willing to answer?
Designing a questionnaire is both an art and a science; and takes time and
precision. Before setting out to design a questionnaire from scratch, it is a good
idea to find out whether such questionnaires already exist that could be used or
modified to gather the information that you seek. A little bit of research could
save you a lot of work in creating an entirely new questionnaire.
In designing a questionnaire you have to determine the questions that you want
to ask; select the question response format for each question, decide on question
sequence and overall layout, pilot-test and implement the instrument. Deciding
what questions to ask will depend on the aims of your survey. For example, if the
aim of your study is to survey the reading habits and interests of teenagers, one
of the questions you will ask is: „What kinds of books do you like to read?‰ You
should ensure that there is a close link between the research aims and the
individual questions asked. You do not want a situation where the data collected
does not answer the research questions.

6.2 QUESTION RESPONSE FORMAT

The questionnaire is designed with the purpose of getting people to respond
to a set of questions. In most questionnaires, you will find two broad types of
questions used: Structured (closed) and Unstructured (open).
6.2.1 Structured Questions

Structured questions are questions in which all possible answers (or responses)
are identified and the respondent is asked to choose one or more among the
answers.
Advantages of Structured Questions

(a) Structured or closed questions are useful when the answers are limited and
clear-cut. They are most appropriate for asking questions about factual
information; but they are also widely used in obtaining data about opinions
and attitudes. We will discuss Attitude Scales later.
(b) Structured questions allow for greater control as they limit the choice of
answers and force the respondent to answer.
(c) It is easier for subjects to respond to structured questions, thus saving time.
(d) Since it does not require writing, it may encourage unmotivated subjects to
respond.
(e) The limited responses make the task of coding data easier.
Disadvantages of Structured Questions

(a) The answers are limited to what is stated which may not capture responses
other than those listed.
(b) Questions that are poorly designed may mislead and frustrate respondents.

6.2.2 Types of Structured Questions

Some types of structured questions are:
(a) Dichotomous questions: These items require two possible responses such as
Agree/Disagree or True/False or Yes/No.
Should the driving age be increased from 17 to 21 years?

Yes
No
(b) Check Only One Answer from a List: The respondent is required to check
or tick only ONE answer.
What is your fatherÊs highest level of education?

Primary school
Secondary school
Form 6/Matriculation
Degree
(c) Checking More than One Answer from a List: The respondent is required to
place a check or tick next to one or more answers.
Which of the following are reasons for school bullying?

Violence on TV
Lack of discipline at home
Insecurity within themselves
Peer pressure
(d) Ranked items: Respondents can also be asked their preferences by ranking
the items from 1, 2 and so forth. We want the respondent to put a 1, 2, 3, 4
or 5 next to the brand, where 1 is the respondentÊs first choice, 2 the second
choice and so forth.

Rank the following brands of cars in terms of prestige:

(1 ă most prestigious, 2 ă second most prestigious and so forth)
Proton _____
Toyota _____
Honda _____
Nissan _____
Ford _____
(e) Rated responses: The Likert Scale has become a popular tool in getting people
to respond to questions (The Self-Esteem Test presented at the beginning of the
Topic consists of items using the Likert Scale). The Likert scale is a rating scale
that measures the strength of agreement with a statement.
Common types of rating scales:
AGREEMENT
Strongly Strongly
Disagree Disagree Undecided Agree Agree
1 2 3 4 5
FREQUENCY
Very
Rarely Very rarely Occasionally Frequently Frequently
1 2 3 4 5
IMPORTANCE
Of little Moderately Very
Unimportant importance important Important important
1 2 3 4 5
(f) Filter questions: This format is used when you want to obtain information
from people about the habits (whether they „do‰ or „do not‰ engage in
something), beliefs in or opinions of something (i.e. you are filtering).
(i) Do you smoke cigarettes Yes No
If „YES‰, answer Question 2ă3. If „NO‰, continue to Question 4.

(ii) How many sticks of cigarettes do you smoke a day?
<5 sticks a day 11ă15 sticks a day

6ă10 sticks a day >15 sticks a day
(iii) When did you start smoking?
Primary school Upper secondary school

Lower secondary school After secondary school
6.2.3 Unstructured Questions

In the design of a questionnaire, one issue that will arise is whether you should
use open-ended questions. Some researchers are against open-ended questions
because of the problems associated with using such a format. However, open-
ended questions can be useful. For example, the open-ended questions:
What do you think are the reasons for school bullying?

What do you like most about the learning materials?
Such questions would provide many kinds of responses. Some would be long
answers while others would be just a phrase. If you are interested in getting a
variety of reasons and also some in depth reasons, then the unstructured or open-
ended format would be useful. Unstructured or open questions should be used
when you feel that the particular question cannot be categorised to include all
possible responses. However, if you are concerned about the time-consuming
task of processing many different responses, than closed questions should be
used; but, you should be aware of the disadvantages of such questions.
SELF-CHECK 6.1
1. What are the advantages and disadvantages of using structured or

closed questions in a questionnaire?
2. What is the difference between ranked responses and rated
responses?
3. When would you use a ranked response and a rated response? Give
specific examples.

6.3 GUIDELINES ON QUESTIONNAIRE DESIGN

Most problems associated with questionnaires can be traced back to the design
phase. Your questionnaire would be better designed if you are clear about the
purpose of your study. You should clarify your goals in relation to how you
intend to use the information. Unfortunately, many researchers neglect this
aspect of the questionnaire design process.
(a) Questions Related to Objectives:

Only ask questions that directly address the objectives of your study. For
example, if you do not intend to compare differences between ethnic
groups, you do not need to ask about ethnic origin. Avoid the temptation to
ask questions because it would be „interesting to know‰ or „maybe IÊll need
the data later‰ or „what harm is there in collecting more than I need‰.
(b) Length of the Questionnaire:

Generally, long questionnaires get fewer responses than short ones. You
should keep the questionnaire short to sustain respondent interest and not
exhaust your subjects (you can guess how a person will respond if he or she
is tired!). However, if the questions are interesting, respondents might
overlook the length of the questionnaire.
(c) Instructions:
(i) Instructions on how to answer the questions should be clear and
concise, using words that are not difficult to understand.
(ii) State the purpose of the survey and identify who is administering or
sponsoring the survey.
(iii) Indicate how confidentiality is protected. This assurance may increase
the likelihood of honest responses.
(iv) Indicate who the respondent can call or write to if they have
questions, concerns or want a copy of the survey results.
(d) Order of Questions:

(i) Begin with questions that you think are interesting to your
respondents. For example, questions about their hobbies and interests
would attract the interest of adolescents. Also, the questions should
not be too threatening. Questions that are boring and threatening will
cause respondents not to continue.

(ii) Group together questions that are similar in terms of format and the
specifics covered (e.g. group together questions asking respondents
about their background such as occupation, income, age, etc.).
(iii) The transition from one question to another should not be too abrupt.
Avoid questions that jump from one unrelated topic to another.
(e) Scales:
(i) When you want subjects to respond to a scale from lower to higher, it
is usually better to place the lower end of the scale on the left and the
higher end of the scale to the right.
1. Never 2. Seldom 3. Occasionally 4. Always
(ii) Avoid having too many divisions in your scale which strains the
respondent (see example below). Usually a 4 or 5 point scale would be
adequate.
1. Never 2. Seldom 3. Occasionally 4. Fairly often
5. Often 6. Very Often 7. Always
(f) Wordings of questions:

(i) Avoid leading questions such as the following:
 Is it important to treat the handicapped fairly?
 Do you agree with most people that the traffic system is getting
worse?
(ii) Watch out for words that are ambiguous or have more than one
meaning:
Which teacher is the best?
(iii) Avoid jargon, acronyms or terms that only a few people may know.
Acronyms should be expanded or stated in full unless the target
audience commonly knows them.
 Do you favour inclusion? (Are you sure most of your subjects
know what „inclusion‰ is?).
 What are your SPM results? (Foreigners may not know what SPM
stands for).

(iv) Use simple and direct language which is easily understood by the
respondent. Do not use words that will probably not be understood to
avoid misinterpretation. You could bold or underline words that you
want to emphasise to eliminate misunderstanding.
(v) Avoid double barrelled questions such as the following:
Do you think Malaysians should eat less and do more exercise?
Break up the question into two: one asking about „eating less‰ and the
other asking about „exercise‰.
(vi) Avoid questions involving negatives. Do not confuse respondents
with questions such as the following:
 Are you against the ban on smoking?
 Do you oppose the ban on public speaking?
(g) Layout and Design:

(i) Do not clutter the questionnaire with unnecessary headings and
numbers.
(ii) Avoid using lots of lines, borders and boxes since these can make the
page look too „dense‰.
(iii) Ensure that the questionnaire has a title and a brief introductory
statement on the purpose of the study.
(iv) Use a good legible font (e.g. Times Roman). Make use of italics and
bold type for instructions as well as headings.
SELF-CHECK 6.2
1. What are some common mistakes in the design of questionnaires?

2. What is the purpose of pilot-testing the questionnaire?
6.4 PILOT TESTING THE QUESTIONNAIRE

Test the questionnaire on a small sample of subjects who are closely similar to
your final sample. For example, if you are planning to survey adolescents, make
sure that you pilot-test the questionnaire with a group of adolescents. Encourage
respondents to make comments on each of the questions, on the order and format
of the questions, or on the nature of the questionnaire. You could also have
respondents discuss in small groups to provide insights into the questionnaires.

Note whether any questions are frequently unanswered or are frequently

misunderstood and answered incorrectly. Probably the questions are poorly
worded, they are located wrongly in the questionnaire or they are too difficult to
answer or too sensitive to answer. Also, take note of the time taken to complete
the questionnaire. The aim of the pilot-test is to detect any mistakes in your
questioning and to correct them before the main survey. The pilot-testing may
also allow you to convert an unstructured or open-ended question to a structured
or closed question by determining the range of possible answers or responses.
Based on the pilot-test, you will be able to make changes that will help maximise
response rate and minimise confusion among respondents.
ACTIVITY 6.2
Design a 15-item Teacher Workload Questionnaire focusing on the

following:
(a) Planning for teaching;
(b) Preparing for assessment and marking papers;
(c) Clerical and record keeping; and
(d) Attending meetings.
6.5 DESIGNING AN ATTITUDE TEST

What is an attitude? An attitude is a pattern of belief that is enduring which may
influence behaviour. So if someone has a positive attitude about his or her job,
you would expect the person to be committed to their job. The Likert scale was
introduced by Renis Likert in 1932 as the familiar five-point response format. The
scale requests people to indicate how much they agree or disagree or approve or
disapprove. An attitude test or inventory is a cluster of items (or questions) that
measures a unitary dimension or single attitude. The respondent is provided
with a range of possible responses. An appropriate scoring scheme is associated
with each of the five possible responses. For example; strongly agree = 5, agree = 4,
undecided = 3, disagree = 2 and strongly disagree = 1. This could of course be
reversed if desired. Sometimes, if the researcher wants to avoid an undecided
category, then they may choose to use an even number of choices, that is; 4 or 6.

Steps in Developing an Attitude Test
(a) Define the Construct: Define the attitude (or construct) that you wish to
measure. You should review the literature in the field to see how the
attitude has been defined. Examples of attitudes are; attitudes toward racial
integration, attitudes toward women bosses, attitudes toward smoking,
attitudes toward water conservation and so forth.
(b) Break down the Construct: Usually an attitude (or construct) needs to
broken down into a number of categories. In other words, you are
providing an operational definition of the attitude. For example, racial
attitudes can be broken down into: marriage, social interaction, cultural
heritage, workplace and so forth.
(c) Brainstorm: Discuss and come up with a list of about eight to ten statements
for each category of the attitude. Develop an equal number of positive and
negative statements about each category of the attitude object.
(d) Rating the Items: Next is to have a group of judges or panel of experts, rate
each statement on a 5 point rating scale to ensure content validity (or face
validity). Do the statements cover the breadth of the attitude being
assessed? Measure what it is supposed to measure. For example, is the
statement: I would be or have been in a romantic relationship with a person
of another race; describing a racial attitude.
1 = strongly does not describe the attitude
2 = somewhat does not describe the attitude
3 = undecided
4 = somewhat describes the attitude
5 = strongly describes the attitude
The above scale is suggested to guide judges or experts in rating the

statements. Note that the judges are not telling you what they believe, but
the extent to which they perceive the statements as describing the attitude
(or construct) of interest.
(e) Scale: Decide on an appropriate scale such as the following which has
five possible responses: 1 = strongly disagree, 2 = somewhat agree, 3 =
undecided, 4 = somewhat agree, and 5 = strongly agree. If you want to
avoid an „undecided‰ category, you can choose to use an even number of
choices, i.e. a 4 point Likert scale or a 6 point Likert scale.

(f) Pilot-Testing: Administer the attitude scale to a group of subjects and

score the instrument. Monitor to see if respondents had problems in
understanding the items. The final score for the respondent on the scale is
the sum of their ratings for all of the items. (this is sometimes called a
„summated‰ scale). On some scales, you will have items that are reversed
in meaning from the overall direction of the scale. These are called reversal
items. You will need to reverse the response value for each of these items
before summing for the total. That is, if the respondent gave a 1, you make
it a 5; if they gave a 2 you make it a 4; 3 = 3; 4 = 2; and, 5 = 1.
(g) Reliability: You have to examine the test for reliability which means
whatever the instrument measures, it measures consistently. CronbachÊs
alpha measures how well a set of items measures a single construct (or
attitude). It measures consistency within the instrument where all items are
compared with each other. Alpha coefficient ranges in value from 0 to 1 and
the higher the score, the more reliable is the attitude scale. Nunnaly (1978)
has indicated 0.7 to be an acceptable reliability coefficient but lower
thresholds are sometimes used in the literature.
(h) Reliability Analysis: See Figure 6.2 which is a printout of the reliability
analysis done for a sample of items. The Cronbach alpha for the instrument
is 0.77 (rounded from 0.77102). The second column shows the correlation
between each item with the total or rest of the items. The third column
shows the correlation coefficient obtained if a particular item is deleted.
Note that if you were to delete Item 4, the Cronbach alpha will increase to
0.82 (rounded from 0.816080).
Correlation with Alpha when Item

Item Total Deleted
Item 1 0.358869 0.772209
Item 2 0.350085 0.772623
Item 3 0.434180 0.781626
Item 4 0.176243 0.816080
Item 5 0.443533 0.769178
Item 6 0.418211 0.773390
Item 7 0.434247 0.770623
Cronbach Coefficient Alpha: 0.77102
Figure 6.2: Printout of reliability analysis

(i) Final Items: Based on the reliability analysis, final items to be included in
the instrument can be decided. Generally, the reliability of the scales tends
to increase with the number of items. However, as the number of items in a
scale increases, so the time taken to complete the attitude test will also
increase, and this may demotivate respondents. There is no hard and fast
rule to determine the final number of items in a scale and this will reflect
the nature and complexity of the attitude being assessed. Generally fewer
than 20 items may reduce reliability acceptably, but more than 30 will begin
to demotivate the respondent.
Example of an Attitude Scale:
Attitude towards Sex Education in Schools

1 = Strongly disagree 2 = Disagree 3 = Agree 4 = Strongly agree
1 2 3 4
1. Sex education should be the main responsibility of
parents and not teachers.
2. Rather than have students learn about sex from their
peers or on their own, it is better if sex education is
taught in schools.
3. Since some students are likely to experiment with
sex, it is a good idea to have sex education taught in
the public schools.
4. Allowing sex education to be taught in the schools
would lead to an increase in teenage pregnancy.
5. Sex education should not be taught in public schools
because it will lead to students experimenting with
sex earlier than they might, otherwise.
6. The reproductive system taught in science and
biology is not sufficient for students to know about
sex.

ACTIVITY 6.3
Referring to the „Attitude towards Sex Education Scale‰, do the
following:
1. Identify the positive and negative statements.
2. Complete the attitude scale by adding three positive statements and
three negative statements.
3. Try out the complete attitude scale with some of your colleagues
and friends.
6.6 QUANTITATIVE DATA ANALYSIS
This section will discuss quantitative data analysis such as the t-test, ANOVA,
chi-square and correlation coefficient.
6.6.1 Mean and Standard Deviation

The data collected from questionnaires, attitudes scales, achievement tests
and other instruments have to be analysed and interpreted. Certain statistical
procedures are used for quantitative data. We will discuss the more common
statistical procedures used without going into the mathematics behind the
procedures. Emphasis will be on a conceptual understanding of these procedures,
which might be used in analysing quantitative data.
(a) Mean
The mean is what is commonly called the average. It is the sum of all the
scores, divided by the number of scores. The mean is the most commonly
used measure of central tendency or the location of the middle or the centre
of a distribution of scores. For example, the mean for the following set of
scores: 5, 3, 6, 4 & 7 = 5.
(b) Standard Deviation

The standard deviation is a figure that tells you how close the scores
are clustered around the mean. Look at Figure 6.3 which shows the spread
of scores obtained on a geography test for two groups of students who
obtained the same mean of 20. When the scores are bunched together
around the mean, the standard deviation is small and the bell-shaped curve

is steep (as in the case for Group A). When the scores are spread away from
the mean, the standard deviation is large and the bell curve is relatively flat
(as in the case of Group B).
Figure 6.3: Distribution of scores on a geography test
To further understand what standard deviation means, refer to the graph in

Figure 6.4. The mean is 20 and the standard deviation (SD) is 5.
Figure 6.4: Distribution of scores on a geography test
(i) ONE standard deviation (SD = 5) from the mean in either direction on
the horizontal axis accounts for around 68% of the students in this
group. In other words, 68% of students obtained between 15 and
25 marks.
(ii) TWO standard deviations (5 + 5 = 10) away from the mean accounts
for roughly 95% of students. In other words, 95% of students obtained
between 10 and 30 marks.

(iii) THREE standard deviations (5 + 5 + 5 = 15) away from the mean

accounts for roughly 99% of students. In other words, 99% of students
obtained between 5 and 35 marks.
Much of quantitative educational research is focused on answering two

important questions.
(i) Is there a significant difference between variables?
(ii) Is there a relationship (or correlation) between variables?
In the following section we will focus in answering these two questions and
the more commonly used statistical tools. We will talk about these tools
without going into the mathematics involved. You should be aware that we
will discuss the use of these tools at a conceptual level and many issues
(such as assumptions) related to each tool will not be mentioned. (You are
advised to refer to a book on statistics for the details and the mathematics
involved).
6.6.2 Testing for Significant Differences between Two

Means Using the Student’s t-Test (Independent
Groups)
Say for example, you conducted an experiment to compare the effectiveness of
the „discovery method‰ (independent variable) in enhancing science achievement
(dependent variable) among primary school children. The mean scores and
standard deviations obtained for the science test is shown in Table 6.1. You want
to test the null hypothesis:
H0: There is no significant difference between the experimental group and the
control group in terms of science achievement
Steps:
(a) You can use a statistical tool called the StudentÊs t-test to obtain the t-value
for independent means. („Independent‰ means that the two groups consist
of different subjects). The t-test gives the probability that the difference
between the two means is caused by chance. To test the significance, you
need to set a risk level called the alpha level. In most educational research,
the „rule of thumb‰ is to set the alpha level at .05.
This means that an obtained result that is significant at the .05 level could
occur by chance only 5 times in 100 trials.

Table 6.1: Means and Standard Deviations Obtained

for the Experimental and Control Groups
n Mean Standard Deviation
Experimental group 10 13.8 2.10

Control group 10 11.4 1.96
t value = 2.65; degrees of freedom = 18; p < 0.02
(b) Table 6.1 shows that a t-value of 2.65 was obtained. If you are using a
statistical software, the probability level is given (i.e. p < 0.02). You could
also look it up in a table of critical values to find out whether the t-value is
large enough to say that the difference between the groups is not likely to
have been a chance finding.
(c) You determine the degrees of freedom (df) for the test which is the sum of
the persons in both groups minus 2 (i.e. nă2). Given the alpha level, the df,
and the t-value, you can look up the t-value in the table of critical values
(available as an appendix in the back of most statistics texts) to determine
whether the t-value is large enough to be significant.
(d) See Table 6.2. The obtained t-value (2.65) is bigger than the critical value
(2.1009) for 18 degree of freedom (20 ă 2 = 18). From this you can conclude
that the difference between the means for the two groups is significantly
different at the 0.05 level of significance.
Table 6.2: Extract from the Table of Critical Values
df 0.05 0.01
17 2.1098 2.8982
18 2.1009 2.8784
19 2.1009 2.8609
(e) Note that the difference is NOT SIGNIFICANT at the 0.01 level of
significance because the t-value (2.65) is smaller than the critical value
(2.8784) for 18 degrees of freedom.

6.6.3 Testing for Significant Differences between Two

Means Using the t-test (Dependent groups)
Say for example, you conducted an experiment to compare the effectiveness of
the „discovery method‰ (independent variable) in enhancing science achievement
(dependent variable) among ONE group of primary school children. You gave a
pretest and after teaching the students using the discovery method, you gave a
posttest. Here, the same group of subjects were tested twice. The mean score and
standard deviation obtained for the science pretest and posttest is in Table 6.3.
You want to test the null hypothesis:
H0: There is no significant difference between the pretest mean and the
posttest mean in terms of science achievement.
Table 6.3: Means and Standard Deviations Obtained for the Pretest and Posttest Scores
Mean Standard Deviation

Pretest 9.90 1.66
Posttest 10.90 0.99
n = 10 t value = 1.94; degrees of freedom = 9; p < 0.09
Steps:
(a) Using the t-test for dependent groups (also called paired groups or
correlated groups), you obtain a t-value of 1.94. („Dependent‰ means that
the two means are obtained from the same groups of subjects).The degrees
of freedom (df) for the test is the persons in the one group minus 1 (nă1).
You set the alpha level at 0.05 and with the t-value you look up the table of
critical values
(b) From Table 6.4 you find that for 9 df the critical value is 2.2622 which is
larger than the t-value of 1.94. You conclude that the means is NOT
significantly different at the 0.05 level of significance.
Table 6.4: Extract from the Table of Critical Values of t
df 0.05 0.01
8 2.3060 3.3554
9 2.2622 3.2498
10 2.2281 3.1693

ACTIVITY 6.4
Using the t-test calculator at this website:

http://www.physics.csbsju.edu/stats/t-test_NROW_form.html OR
http://www.physics.csbsju.edu/stats/t-test_bulk_form.html
Key in the mathematics test scores for the two groups shown below and
calculate to determine if the means for the two groups are significantly
different at the .05 level of significance:
Mathematical Test Scores
Control group: (n = 10) 20, 23, 25, 22, 27, 19, 28, 24, 25 & 26.
Experimental group: (n = 10) 29, 30, 28, 32, 33, 37, 27, 36, 31 & 29.
6.6.4 Testing for Differences between Means Using

One-way Analysis of Variance (ANOVA)
In the analysis of variance, the approach is conceptually similar to the t-test,
although the method differs. When you want to compare more than two
means, the One-way Analysis of Variance (ANOVA) is used. Say, for example
you conducted an experiment in which you compared the effectiveness of
three teaching methods in enhancing Reading comprehension. The means and
standard deviations obtained are shown in Table 6.5.
Table 6.5: Means and Standard Deviations Obtained for

Science Achievement Among Students in the Three Groups
N Mean Standard Deviation

Method 1 10 14.6 1.83
Method 2 10 15.6 2.22
Method 3 10 18.0 2.10
Summary Sum of square df Mean squares F p
Treatment 61.066 2 30.533 7.1811 0.003

Within 114.800 27 -
Total 175.866 29
Pos Hoc Comparison Using the Tukey Test

Method 1 vs Method 2 - not significant
Method 1 vs Method 3 - significant at p<.01
Method 2 vs Method 3 - significant at p<.05

You used the One-way ANOVA and obtained an F-value of 7.1811, which is
significant at 0.003. Hence, the null hypothesis of no differences between means
is rejected. However, you are not sure which differences contribute to the
significance. In other words, whether all three methods are different or whether
only two methods are significantly different.
So, you have to perform another statistical test called Pos Hoc Tests or Pos Hoc
Comparisons. Two common tests used are the Tukey Test and the Scheffe Test
which is usually applied after an analysis of variance. The Tukey Test in Table 6.5
shows that there is no significant difference between the performance of subjects
taught science using Method 1 and Method 2. Subjects taught with Method 3
performed significantly better than subjects taught with Method 2 at the
0.05 level of significance. Also, subjects taught with Method 3 significantly
outperformed subjects taught with Method 1 at the 0.01 level of significance.
6.6.5 Testing Differences Using the Chi-square (2)

The analysis of variance t-test assumes that the population from which you drew
the sample is normal. What if you cannot assume that your sample is normal?
In this context, the chi-square (pronounced as „kai‰ square) represented by the
symbol χ2 is used. Chi-square is a non-parametric test of statistical significance
for analysis of crossbreaks based on frequencies.
Table 6.6: Male and Female Undergraduates Opinions on Violence on TV
Is there too much violence on TV?

Yes No Total
Males 15 38 53
Females 30 20 50
Total 45 58 103
df = (2 ă 1)  (2 ă 1) = 1 2 = 10.5077 p < 0.01
The chi-square can be used to test whether or not two or more samples are
different and can be generalised to the population from which the samples were
drawn. The chi-square is widely used in analysing data obtained from surveys
using questionnaires. Say for example, you ask a group of 103 undergraduates
„Is there too much violence on TV?‰ The null hypothesis you want to test is:
H0: There is no significant difference between the opinions of male and

female undergraduates regarding violence on TV.

The frequencies of their responses are shown in Table 6.6. The degrees of
freedom (df) are obtained by the formula df = (ră1)(că). That is, the df equals the
number of rows in the table minus one multiplied by the number of columns in
the table minus one. So the df for Table 6.6 is 1.
Table 6.7: Extract from the Chi-Square Critical Values
df 0.05 0.01
1 3.841 6.634
2 5.991 9.210
The chi-square (2) obtained is 10.5077 and with 1 degree of freedom, it is more
than the critical value of 3.84 at the 0.05 level of significance and 6.634 at the
0.01 level of significance (see Table 6.7). So, you can reject the null hypothesis
and affirm that male and female undergraduates differ in their opinion about
violence on TV.
ACTIVITY 6.5
Use the Chi-square calculator at this website:

http://www.georgetown.edu/faculty/ballc/webtools/web_chi.html
Enter the number of rows and columns and press Generate table. Enter
the following data in the cells for the Rows and Columns and press
Calculate chi square. Test the null hypothesis that there is no
significant difference in the opinions of final year and first year
undergraduates regarding the death sentence.
Should the death sentence be abolished?
Yes No
Final Year Undergraduate 54 83
First Year Undergraduates 75 55
6.6.6 Testing Relationships Using the Correlation

Coefficient
When you are interested in finding out the relationship or correlation between
two sets of variables, the statistic used is the correlation coefficient. For example,
in a study you collected data on „self-esteem‰ and also „academic performance‰.

You want to find out whether there is a correlation between self-esteem and
academic performance. A correlation has direction and can be either positive or
negative.
(a) With a positive correlation, individuals who score high (or low) on one
measure (or variable) tend to score similarly on the other variable.
(b) With a negative correlation, individuals who score high on one measure (or
variable) tend to score low on the other (or vice versa).
The Pearson product-moment correlation coefficient (represented by „r‰) is used
to show the degree or strength of the relationship between two variables. The
coefficient can range from r = +1.00 to ă1.00. See Figure 6.3 which illustrates what
the coefficient means.
(a) Figure 6.5(a) shows a perfect positive correlation (r = +1.00) which means
an increase in variable y is also followed by an increase in variable x.
(b) Figure 6.5(b) shows a perfect negative correlation (r = ă1.00) which means
an increase in variable y is followed by a decrease in variable x (or vice
versa).
Figure 6.5: Graphs showing different correlation coefficients

(c) Figure 6.5(c) shows a zero correlation (r = 0.00) which means there is no
relationship between variable y and variable x.
(d) Figure 6.5(d) shows a positive correlation (r = +0.65) which means an

increase in variable y is also followed by an increase in variable x but not
perfectly. For example, if the correlation coefficient between self-esteem and
academic performance is 0.65; you can conclude that there is a positive
correlation between the two variables. In other words, individuals who
have self-esteem also are academically high achievers. However, the
relationship is not perfect, indicating that there are individuals who may
have high self-esteem but are not academic high achievers and vice-versa.
You should be cautious not to assume that correlation equals causation. In
other words, you cannot say that self-esteem „causes‰ high academic
performance. You can only say that there is a relationship between the two
variables.
ACTIVITY 6.6
Use the Correlation calculator at this website:

http://calculators.stat.uda.edu/correlation.php
Enter the following data in the Input data column and press Submit.
What is the correlation between attitude and performance?
Attitude towards mathematics: 23 25 26 12 20 18 21

Mathematics score: 55 50 60 41 49 52 51
Ć Questionnaires are widely used because they are cost effective and can be
administered to large groups of people. If properly designed, questionnaires
can address many issues in a relatively efficient way.
Ć Questions in a questionnaire can be structured and unstructured.
Ć The questions in a questionnaire should be clear, not ambiguous and related

to the goals of the study.
Ć Pilot-testing of the questionnaire is essential to detect weaknesses of the items.

Ć An attitude is a pattern of belief that is enduring which may influence

behaviour.
Ć Attitude scales may be designed to measure attitudes using the Likert scale.
Ć The t-test is used to test for significant differences between means for
independent and dependent groups.
Ć One-way ANOVA is used when comparing the means of more than two groups.
Ć Chi-square is a non-parametric test of statistical significance for analysis of

crossbreaks based on frequencies.
Ć The correlation coefficient is used to test the strength or degree of the

relationship between two variables.
Attitude scale Questionnaire response format

Chi-square Ranked response
Correlation coefficient Rated response
Dichotomous response Reliability analysis
One-way ANOVA Structured questions
Pilot-testing questionnaire t-test
Questionnaire Unstructured questions
1. Why do you think some people are unable and unwilling to

respond to questions in a questionnaire?
2. What are some of the common errors you have found in
questionnaires?
3. How reliable do you think is the attitude scale in measuring
attitudes?

Brace, I. (2004). Questionnaire design. London: Kogan Page.
Muijs, D. (2004). Doing quantitative research in education with SPSS. London: Sage.
Chapter 7: Bivariate analysis: comparing two groups.
Chapter 10: Using analysis of variance to compare more than two groups.
Reynaldo, J., & Santos, A. (1999). CronbachÊs Alpha: A tool for assessing the
reliability scales. Journal of Extension, 37 (2). Retrieved from:
http://www.joe.org/joe/1999april/tt3.html
Siegle, D. (n.d.). Beginning steps in developing an attitude instrument.
StatPac. Inc. (n.d.). Questionnaire design: A free web tutorial. Retrieved from:
http://www.statpac.com/surveys/

T op i c  Qualitative
7 Research
Methods
LEARNING OUTCOMES
1. Define what is qualitative research;
2. Trace the evolution of qualitative research;
3. Compare qualitative and quantitative research approaches;
4. Describe the characteristics of ethnography;
5. Explain the reasons for conducting action research;
6. Elaborate on the features of a case study; and
7. Describe the characteristics of the generic qualitative method.
 INTRODUCTION
The term qualitative research is a general definition that includes many different
methods used in understanding and explaining social phenomena with minimum
interference with the natural environment.
(a) According to Denzin and Lincoln (1994), qualitative research focuses on

interpretation of phenomena in their natural settings to make sense in terms
of the meanings people bring to these settings. Qualitative research involves
collecting information about personal experiences and significant moments
in peoplesÊ lives through introspections, life stories, interviews, observations,
historical interactions and visual texts.

TOPIC 7 QUALITATIVE RESEARCH METHODS  129
(b) Patton (2002) defined qualitative research as attempting to understand the

unique interactions in a particular situation. The purpose of understanding
is not necessarily to predict what might occur, but rather to understand in
depth the characteristics of the situation and the meaning brought by
participants and what is happening to them at the moment. The aim of
qualitative research is to truthfully present findings to others who are
interested in what you are doing.
(c) According to Pope and May (1995), qualitative researchers study things in
their natural settings in an effort to discover the meanings seen by those
who are being researched rather than that of the researcher.
Qualitative research begins by accepting that there are many different ways of
understanding and of making sense of the world. You are not attempting to
predict what may happen in the future. You want to understand the people in
that setting. „What are their lives like? What is going on for them? What beliefs
do they hold about the world?‰ In short, qualitative research is concerned with
the social aspects of our world and seeks to find answers to the following
questions:
(a) Why do people behave the way they do?
(b) How are opinions and attitudes formed?
(c) How are people affected by the events that go on around them?
(d) How and why have cultures developed in the way they have?
(e) What are the differences between social groups?
7.1 EVOLUTION OF QUALITATIVE RESEARCH

In the 1950s and Ê60s, research in education was very much influenced by the
behaviouristic perspective which used the scientific method in studying animal
behaviour which later was generalised to humans. This is described as the
quantitative approach which dominated much of educational research until the
publication of the book The Structure of Scientific Revolutions by Thomas Kuhn.
The ideas proposed by Kuhn played a significant role in influencing scientific
thinking. He introduced the concept of „paradigm‰ which was identified as the
scientific achievements and discoveries which provided solutions and explanations
of various phenomena at a particular point in time. He further suggested that
when the paradigm at that point of time is unable to explain satisfactorily
phenomena, a „paradigm shift‰ should occur together within the existing
paradigm. A paradigm shift will lead to the introduction of new research
methods and tools and how the researcher sees the world.

130  TOPIC 7 QUALITATIVE RESEARCH METHODS
At the same time, the Ê60s was a period of turmoil in the United States and
Europe. Society was going through radical changes, being most concerned about
issues such as racial integration, poverty, womenÊs rights and the cold war (the
threat of Russia). People began to question the use of quantitative methods (such
as experiments and surveys) in explaining social phenomena such as juvenile
delinquency, drug addiction, truancy and so forth).
Referring specifically to the school system, interest shifted towards understanding

„school culture‰. This gave rise to the use of qualitative methods in education.
For example, the ethnographic method which focuses on studying processes and
practices in the classroom became a popular technique in educational research.
Increasingly, it was realised that quantitative methods were not able to explain in
detail what was happening in the classroom and the individuals involved.
In education, Carr and Kemmis (1986) differentiated the two types of research
as positivism and phenomenology. According to the positivist approach,
knowledge obtained using the scientific method is objective and measurable.
„Reality‰ according to this perspective is stable, observable and can be measured.
On the other hand, the phenomenological perspective in education focuses on the
processes and experiences one goes through. Literally, phenomenology is the
study of „phenomena‰ or the things as they appear in our experience or the ways
we experience things.
Example:
You are interested in investigating the experiences of a group of adults learning
how to use the computer for the first time. You get them to relate their
experiences and how they feel about touching the keyboard and looking at the
computer screen for the first time in their lives. You also study their experiences
which include their perceptions, their misconceptions, their emotions (feelings),
their desires, their actions and their thoughts. In conducting the study you are
seeking to understand the processes and experiences these adult learners are
going through.
SELF-CHECK 7.1
1. What is positivism?
2. What is phenomenology?
3. Identify FIVE common features of qualitative research in the
definitions given by scholars in the field.

To illustrate, let us examine how a quantitative researcher (positivist approach)

and a qualitative researcher (phenomenological approach) would study Reasons
for Dropping Out of Secondary Schools.
Issue: Study on Reasons for Dropping out of School
(a) Positivism (Quantitative Approach)

You begin by suggesting the factors influencing your beliefs as to why
students drop out from school. Some of the factors you may have identified
are poverty, low self-esteem, poor academic performance, peer pressure and
so forth. Based on these beliefs you develop a questionnaire and administer it
to a sample of students who had dropped out of school. You analyse the data
and identify the factors explaining why students drop out from school. You
might rank the factors or reasons for students dropping out of school.
(b) Phenomenology (Qualitative Approach)

You do not propose any factors or attempt to measure anything. You do not
begin with any beliefs or preconceived ideas about the reasons for students
dropping out of school. You are more interested in understanding the
experience of dropping out of school. You interview and interact with a
small group of school dropouts. You observe their behaviours and record
what they talk about. You also examine documents such as reports by
counsellors and their school progress reports.
If you are asked why you have chosen the qualitative perspective and your
answer is, „⁄. because no statistics is involved!‰; you do not know in depth the
philosophy and orientation of qualitative research. Your choice of using the
qualitative approach should be based on the basic question, „Is the quantitative
or the qualitative approach appropriate in answering your research question?‰
The decision to conduct research using the qualitative approach should be based
on your orientation as a researcher towards issues such as;
(a) Reality
You must accept the fact that when you use qualitative research methods,
you are interested in „multiple realities‰ or multiple interpretations and not
just one conception of reality or one interpretation.
(b) Aims of the Study

You are interested in patterns when analysing qualitative data rather than
one right answer.
(c) Knowledge
The knowledge produced from your study will „emerge‰ and you may be
interested in developing a theory rather confirming a theory.

7.2 DIFFERENCES BETWEEN QUANTITATIVE

AND QUALITATIVE RESEARCH
Proponents of qualitative and quantitative research have and still argue about the
differences between the two approaches of conducting research. Proponents of
quantitative research have been critical of qualitative research arguing that it is
not scientific and too subjective. Table 7.1 lists the differences between qualitative
and quantitative research.
(a) Qualitative research focuses on the perspective of the subject or participants

rather than the perspective of the researcher. This has been termed as the
emic or insider perspective as against etic or the outsiderÊs perspective.
(b) In qualitative research, the researcher is the main instrument in data

collection and data analysis and not a questionnaire or tests as in the case of
quantitative research. The researcher being the main instrument of data
collection is more responsive to the situation and he or she is able to adapt
to the changing conditions. For example, the researcher is more sensitive to
the reactions of participants and the data can be immediately processed, i.e.
he or she is able to take whatever action to check and confirm with the
subject if there are any doubts or uncertainties.
(c) Qualitative research involves field work; that is, the researcher must himself
or herself „walk the factory floor‰ or go into the setting where there are
people, and observe or interview them in their natural setting.
Table 7.1: Differences between Quantitative and Qualitative Research
Qualitative Quantitative
Focus: Quality (features) Quantity (how much, numbers)
Philosophy: Phenomenology Positivism
Method: Ethnography/Observation Experiments/Correlation
Goal: Understand, meaning Prediction, test hypothesis
Design: Flexible, emerging Structured, predetermined
Sample: Small, purposeful Large, random, representation
Data collection: Interviews, observation Questionnaire, scales, tests,
documents, artefacts inventories
Analysis: Inductive (by the researcher) Deductive (by statistical methods)
Findings: Comprehensive, description
Precise, numerical
detailed, holistic
Researcher: Immersed Detached
Source: Adaptation from Merriam, 1999; Firestone, 1987; Potter, 1996

(d) Qualitative research adopts the inductive approach (see Figure 7.1). Such
research is conducted because of a lack of theory or, existing theories are
unable to explain a phenomenon convincingly. Because of this, no
hypotheses are put forward to guide research. The qualitative researcher
begins by observing phenomena and continues to find patterns in the form
of themes, categories, concepts and typologies that emerge. Tentative
hypothesis are introduced and additional information are collected to
explain the phenomenon.
Figure 7.1: Inductive approaches in qualitative approach
(e) Lastly, qualitative research focusses on process, meaning and understanding

based on thick and rich descriptions. Words and pictures, not numbers, are
used to explain phenomena. Also emphasised are the description about the
situation, the people involved and the activities observed. Data in the form
of communication of the participants themselves, extracts from documents,
video and audio recordings, support the findings of the study.
Other than that, qualitative research is also described as emerging. However, this
is not characteristic of all qualitative research. Graduate students doing masters
or doctoral thesis do not have much time at their disposal and so they may not be
able to see the emergence of theories. The sampling in qualitative research is
small and not chosen randomly. Rather, the choice of a sample is purposeful. For
example, if you intend to study drug addicts at a drug rehabilitation centre you
will select a few addicts to study. A long time is required to enable the researcher
to interact with other samples in different situations.
SELF-CHECK 7.2
1. What are the major differences between quantitative and qualitative

research methods?
2. When would you select the qualitative approach in doing research?

7.3 QUALITATIVE RESEARCH METHODS IN

EDUCATION
Different authors have discussed qualitative data collection methods
differently. For example Patton (1990) identifies 10 different types of qualitative
research methods based on the kinds of questions a particular researcher will
ask. They are ethnography, phenomenology, heuristics, ethnomethodology,
symbolic interactionism, ecological psychology, system theory, chaos theory,
hermeneutics and orientational inquiry. Denzin and Lincoln (1994) have classified
the different methods based on their strategies of inquiry and listed the
following: ethnography, phenomenology, historiography, participant observation,
ethnomethdology, grounded theory, biographical method, historical method and
clinical research. Others such as Tecsch (1990) and Merriam (1992) have included
case studies, content analysis and action research to the list.
This topic examines four qualitative methods commonly used in educational

research namely: ethnography, case study, action research and the basic qualitative
method (see Figure 7.2).
Qualitative Research Methods
Ethnography Action
Research
Generic Qualitative
Case Study
Method
Figure 7.2: Qualitative research methods
7.4 ETHNOGRAPHY
We will discuss the definition, the role of researcher fieldsites, fieldnotes and
some practical aspects of ethnography.

7.4.1 What is Ethnography?

Ethnography is a type of qualitative research
method employed by anthropologists to
study human society and culture. The term
„ethnography‰ means „portrait of a people‰
which involves describing a people and its
culture. Culture has many definitions but
essentially it refers to the beliefs, values
and attitudes that influence the behavioural
patterns of a specific group of people. When
you say something is cultural, it means that a
particular belief, value or attitude is shared
by a significant number in the group; it is practised by the group and is passed
down to the next generation. For example, when „punctuality‰ is cultural for a
particular group, it is a shared belief practised by most members of the group
and passed down to their children. Hammersly and Atkinson (1989), in their
book Ethnography: Principles in Practice stated that:
„⁄ ethnographers have developed an alternative view of the proper nature

of social research, often termed naturalism ⁄ which propose that the social
world should be studied in its natural state, undisturbed by the researcher⁄.
The research should be carried out in ways that are sensitive to the nature of
the setting ⁄ because human actions are based upon, or infused by, social
meanings: intentions, motives, attitudes, and beliefs ⁄ which mean different
things to different people, and, indeed to the same person at different times‰.
(p. 6ă7)
The ethnographic method when applied in education focuses on studying the

culture of the school community. Regardless, whether you are studying the
teachers, the students, the principal, the curriculum or co-curricular activities;
in an ethnographic study your focus will be on a sociocultural interpretation
of what you observe. For example, an ethnographic study of a Vision School
(Sekolah Wawasan) would take into account the community at large and its
cultural context. The characteristics of the neighbourhood, socioeconomic
factors of parents, the communityÊs ethnic makeup, the attitudes of parents,
teachers and school administrators. All these are important considerations in this
ethnographic study. Common ethnographic techniques used in education are
interviews, examination of documents and artefacts, and observations. These
techniques are discussed in more detail in Topic 8: Qualitative Data Collection
Methods. Merely using these techniques does not mean you are using
ethnography.

7.4.2 Role of the Researcher in an Ethnographic

Study
Have you been to a party or social gathering (such as a wedding) where you
did not know many people? Unless you are an extremely confident person, this
can be a tough situation for many people. How do you get yourself introduced to
the strangers in the party? How do you avoid giving the impression that you
are a stranger who does not know anybody? Doing an ethnographic study is
something like this. You have to first decide on your role when you enter the
setting. You could wander around with a notebook but that would appear to be
very intrusive and may affect the setting you are trying to observe. In other
words, you might affect the natural state.
One solution which has been adopted by some researchers, is to become

completely part of the group one wants to research on. A classic example in
the field of medicine was reported by Rosenhan (1973), in which researchers
managed to get themselves admitted as patients to a mental hospital. Once
admitted, they openly took notes, yet because the staff were used to odd
behaviour, did not think this unusual and indicative that the patients were
„genuinely‰ ill. The researcher gained in-depth knowledge of the hospital,
including the experience of being a patient. This kind of study is called
participant observation, in which the researcher assumes two roles ă an observer
and as a participant. Most of the time, it is not really possible to become a
participant and the researcher assumes the role of a non-participant observer. We
will discuss this in more detail in Topic 8: Qualitative Data Collection Methods.
7.4.3 Fieldsites
Traditionally, anthropologists have undertaken ethnographic research in small
bounded villages while living among the inhabitants. These ethnographers may
have been one of few non-natives the villagers may have seen. Today, however,
fieldsites can be nearly anywhere. While research may still focus on villages, an
increasing number of ethnographic studies have been conducted in urban areas.
Sometimes the „group‰ does not live in one location. A fieldsite could be a bank,
a religious centre, a school, a faculty and so forth. Once a potential fieldsite has
been selected, ethnographers must negotiate entry or access.
Say for example you entered a primary school and proceeded to the teacherÊs
lounge, you will certainly be questioned by the school authorities. Obviously,
you will not do this but made an effort to seek permission to gain access.
According to Hammersley and Atkinson (1989), gaining access requires the

researcher to draw on his/her interpersonal resources and strategies. It also

requires the researcher to be aware of the obstacles to access and effective means
of overcoming them by understanding the social setting.
(a) Getting Permission

Gaining access is not a single event. You may have to write a letter to
the headmaster requesting permission to observe the happenings in the
teacherÊs lounge. Even though you are given permission, you have to
explain who you are and what you want to do to the teachers. So access is
continuously negotiated throughout the period of fieldwork.
(b) Gaining Access from the Gatekeepers

Getting permission from the headmaster is only „getting your foot at the
door‰. Imagine entering the teacherÊs lounge and being reluctantly received
by the assistant headmaster or the senior assistant who gives various
excuses and imposes various restrictions. Gaining adequate access requires
you to know who has control of the setting and the power to grant access,
i.e. the gatekeepers. You may be seen as in „cahoots‰ with the authorities
and teachers may not like the fact you have come to watch their work. You
may be seen as a „management spy‰. Because of this, your data may be
comprised as the teachers may not behave „naturally‰. You may have to
constantly ask yourself whether your presence will in any way affect how
your subjects behave.
7.4.4 Some Practical Concerns

Having gained access to the setting, the first things you have to decide are:
(a) WHEN to do your observation which may not be obvious when you enter
the setting. For example, if you are interested in what goes on in the
morning session, than you have to arrive before school begins in the
morning.
(b) HOW OFTEN and for HOW LONG you intend to observe. You may not be
able to determine precisely until you get into the field and find out what
will be needed in order to gain an adequate picture We will discuss this
issue in more detail in Topic 8 under 8.6 Sampling. The usual criterion used
is that it should not be too short (there is insufficient data) or too long (until
it becomes too costly).
(c) WHERE to POSITION yourself. In other words, you will have to find an
appropriate place to stand that is unobtrusive (un-noticeable as possible).
You have to position yourself in such a way so that people around you

feel comfortable with your presence and you can explain to them (when
necessary) that you are not planted by the management or those in
authority. Saying that you are „doing research‰ can be quite comforting
and puts most people at ease about your presence.
7.4.5 Fieldnotes
Deciding what to record and what to leave out when observing people in a
setting can be a difficult task. In any setting there are many things happening and
it is not possible, nor desirable to record everything. Imagine the variety of things
happening in the teacherÊs lounge when it is recess and about 60ă70 teachers
gather in the room! So, you have to be selective and what you observe and record
will depend on the questions you want answered.
Note-taking is the main method of recording data and in most cases it is

handwritten taken either at the time, or immediately afterwards. If you use
audio-recording you may be able to capture conversations but you still need
to jot down non-verbal communication or the movements of people involved.
Note-taking is done at two levels. On the surface level are the facts, which are
direct descriptions of what was observed and the verbatim recordings of what
was overheard. On another level are the observerÊs comments about what was
observed. „These are recorded in order to provide a context for the raw facts and
to add speculation about what the researcher thinks it means‰ (Potter, 1996,
p. 99).
To help you focus on what to record, Spradley (1980) suggests the following
checklist:
(a) Space: the physical place or places;
(b) Actors: the people involved;
(c) Activities: a set of related acts people do;
(d) Objects: the physical things which are present;
(e) Acts: single actions people do;
(f) Events: a set of related activities that people carry out;
(g) Time: the sequencing that takes place over time;
(h) Goal: the things people are trying to accomplish; and
(i) Feelings: the emotions felt and expressed.

Ethnographic studies involve extensive fieldwork by the researcher. Because of

this, ethnography is extremely time consuming as it involves the researcher
spending long periods of time in the field observing and taking notes. The notes
contain rich, detailed descriptions of everything that went on called thick
description. At this stage the researcher does not attempt to summarise,
generalise or hypothesise. The notes capture and describe what happened to
permit interpretations, and most of all, to later infer cultural meaning. The
following are some standard rules for taking field notes suggested by Neuman
and Wiegand (2000).
(a) Take notes as soon as possible, and do not talk to anyone before note
taking;
(b) Count the number of key words or phrases used by subjects;
(c) Carefully record the order or sequence of events, and how long each
sequence lasts;
(d) Do not regard anything as insignificant; record even the smallest things;
(e) Draw maps or diagrams of the location, including your movements and
any reaction by others;
(f) Write quickly and do not worry about spelling; devise you own system of
punctuation;
(g) Avoid evaluative judgements or summarising; for example do not call
somebody „a bully‰, just describe his or her actions;
(h) Include your own thought and feelings in a separate section; your later
thoughts in another section; and
(i) Always make backup copies of your notes and keep them in a separate
location.
SELF-CHECK 7.3
1. Define ethnography.
2. List what should be included in fieldnotes.
3. What do you mean by access or entry into the fieldsite?

7.5 CASE STUDY

The Case Study is a method used in both quantitative and qualitative research.
As expected there are varying definitions of the case study as a method in
qualitative research. Perhaps the most well-known case study is The Man in the
PrincipalÊs Office by Harry Wolcott (1973) who studied one principal. Merriam
(1988) defines „a qualitative case study as an intensive, holistic description and
analysis of a single instance, phenomenon, or social unit (p. 21).‰ Smith (1995)
viewed the case study as a bounded system while Stake (1995) sees it as an
integrated system. Miles and Huberman (1994) presented a graphic meaning of a
case study by suggesting that it is like a circle with a heart in the middle (Figure
7.3). The heart is the focus of the study, while the circle defines the edge or
boundary of the case. What is beyond the edge or boundary will not be studied.
Boundary
Focus of the study
Figure 7.3: Graphical representation of a case study
In other words, to qualify as a case study, you have to state the boundary or
delimit what you want to study. For example, you have to limit the number of
people you intend to interview, you have to limit the amount of time you intend
to spend. „If there is no end, actually or theoretically, to the number of people
who could be interviewed or to observations that could be conducted, then the
phenomenon is not bounded enough to qualify as a case (Merriam, 1998, p. 28).
7.5.1 The Method of Case Studies

The case study has been widely used as a research method in law and medicine
and is increasingly used in education. Yin (1994) identified the following steps in
conducting any case study.
(a) The first relates to the research questions which are most likely to be „how‰
and „why‰questions. For example, „How do students interact in the school
canteen?‰

(b) Second relates to the unit of analysis which could be an individual, a group
of individuals, or an organisation.
(c) Third relates to linking the data collected with the research questions.
(d) Fourth relates to the interpretation of findings. A useful technique is

„pattern-matching‰ where data collected from the case may be related to
some theoretical proposition (Campbell, 1975).
Case studies can be either a single-case design or a multiple-case design.

According to Yin (1994), a single-case design is ideal for studying unique or
extreme cases, to confirm or challenge a theory or for cases where the researcher
did not have access to before. However, the researcher should be careful not to
misrepresent what was observed. Multiple-case designs are more suited when
the researcher is interested in using more than one case to gather data from
various sources and draw conclusions from the facts. They serve to confirm or
corroborate evidence which enhances validity of the study. Multiple-design cases
may require more than one investigator and training may be required covering
aspects such as the reason for the study, the type of evidence to collect and what
variations might be expected (Tellis, 1997).
7.5.2 Techniques for Gathering Data

The following are some types of data collection techniques employed in case
studies (Stake, 1995 and Yin, 1994):
(a) Interviews
The interview is an important technique for data collection and there are
two forms of interviews: Closed or Structured Interviews and Open-Ended
Interviews. Open-ended interviews allow subjects to express themselves
more freely and provide insights into events.
(b) Observations
This could be direct observation of events and behaviours as well as
participant-observation where the researcher is an active participant in the
events being studied.
(c) Documents
These could be letters, memos, agendas, administrative documents,
newspaper articles and any other relevant documents. Documents are
useful for making inferences about events. Documents are communications
between persons in the study.

(d) Physical Artefacts

These are objects collected from the setting which could be products
made by students and other individuals; the objects used such as tools or
instruments.
SELF-CHECK 7.4
1. List the characteristics of a case study.
2. When would you use a case study?
7.6 ACTION RESEARCH

This section will discuss the definition, motive and concept of action research.
7.6.1 What is Action Research?

Action research is a qualitative research method that has become increasingly
popular in education and can be used in any social organisation. It encourages
the practitioner (or teacher) to be reflective of his or her own practice with the
aim of improving the system (McNiff, 1994). Action research is based on the
belief that the practitioner (or teacher) is the best judge of his or her practice (or
teaching) and is encouraged to develop his or her own personal theories of
practice (or education), thus bridging the gap between theory and practice.
Teachers employ action research to test their personal theories in the classroom.
In education, action research has also been called classroom research (Hopkins,
1985) and self-reflective inquiry (Kemmis, 1982). It refers to those activities that
are designed to improve the quality of education. It refers to;
⁄. a form of self-reflective enquiry undertaken by participants (teachers,

students or principals, for example) in social (including educational) situations
in order to improve the rationality and justice of
(a) Their own social or educational practices;
(b) Their understanding of these practices; and
(c) The situations (and institutions) in which these practices are carried out.
Source: W. Carr and S. Kemmis (1986)

7.6.2 Why Action Research?

There are two main reasons for action research. One is to involve practioners
(such as teachers) in their work. The other is to encourage practioners (or
teachers) to be researchers with the purpose of bringing about improvement in
what they are doing. Action research means ACTION, both of the system under
consideration and of the people involved in that system. Here, „system‰ could
mean schools, factories, offices, airlines and so forth while „people‰ refers to the
teachers, managers, workers, supervisors, principals and so forth. For example, a
teacher who discovered that when he adopted an alternative style of dealing with
students with disciplinary problems, student attention in class greatly improved.
He therefore recommends the alternative method to his colleagues and soon the
whole school is seen practicing the method in all the classes. The action of action
research, whether on a small or large scale, implies change in peopleÊs lives, and
consequently, the system in which they live. „Applied to classrooms, action
research is an approach to improving education through change, by encouraging
teachers to be aware of their own practice, to be critical of that practice, and to be
prepared to change it‰ (McNiff, 1994. p. 4).
7.6.3 Concept of Action Research

Stephen Kemmis and Robin McTaggart (1988) developed a concept for action
research. He proposed a spiral model comprising four steps: planning, acting,
observing and reflecting (see Figure 7.4). The diagram shows the four steps
in action; the movement from one critical phase to another; and the way in
which progress may be made through the system. Action research is all about
what happens in the classroom. Teachers are encouraged to be researchers
investigating what is happening in their classrooms.

Figure 7.4: Kemmis and McTaggartÊs concept of action research (1988)
(a) Phase One
(i) Plan: My students find geography lessons boring. How can I make the
lessons more interesting and improve their thinking skills? Bring real-
world situations to the classroom using computer simulations.
(ii) Act: Take students to the computer lab and show them how to use
the simulation software. Select topics that are appropriate for using
computer simulations.
(iii) Observe: Record student interactions and the kind of questions asked.
Videotape a few lessons to see what is happening. Keep notes of my
impressions in a notebook.
(iv) Reflect: The lesson is lively but not all students are asking
questions. They are not discussing with others. They are glued to their
individual computers.

(b) Phase Two
(i) Revise Plan: Break up students into groups of three with each group
working on one computer on a simulation.
(ii) Act: Record their interactions by placing a tape-recorder in each

group. Students continue working on the computer simulations and
answer questions on the worksheets provided.
(iii) Observe: Students are really enjoying themselves. There is greater

group discussion and consensus in decision making.
(iv) Reflect: Can I continue with this teaching method? Can I use the
Internet instead? I am worried about practical problems such as the
availability of computer labs; will the school purchase more such
computer software? If the Internet is used, will there be broadband
access?
7.7 GENERIC QUALITATIVE RESEARCH

METHODS
Many qualitative studies in education are NOT about culture as in ethnographic
studies, or attempting to improve educational practice as in action research or
intensive case studies of an individual or group of individuals. Neither is there
any attempt to develop a theory. These qualitative methods do not fit within an
established qualitative approach. Many terms have been used to describe this
qualitative method. Thorne (1997) used the term „noncategorical qualitative
research‰, Sandelowski (2000) put forward the term „fundamental qualitative
method‰ while Merriam (1998) used the term „generic qualitative research‰. We
will use the term „generic‰ which is a method that „simply seeks to discover and
understand a phenomenon, a process or the perspectives and worldviews of the
people involved‰ (Merriam, 1998. p. 11).
The Generic Qualitative Method does not have a guiding set of philosophic
assumptions in the form of one established qualitative methodology. The
Generic Qualitative Method exhibits some or all of the characteristics of other
methodologies or approaches but makes no claim to any particular qualitative
method. In other words, the Generic Qualitative Method will use the techniques
of ethnography, the case study method, grounded theory and the techniques of
action research, but does not claim it is ethnography, case study, grounded
theory or action research.

Unfortunately, guidelines for conducting research using the Generic Qualitative

Method have been lacking. However, this has not prevented researchers from
using such a methodology. It is not surprising that graduate students face
many problems, especially when their thesis or dissertation supervisors are
themselves not clear about the methodology of the Generic Qualitative Method.
Interestingly, using the Generic Qualitative Method is becoming more common
in education research. Perhaps, the constraints of time and resources have
encouraged this. Also, because of its flexibility and not having to subscribe to any
particular methodology, it is becoming a preferred method in explaining various
kinds of social phenomena in education such as the activities in the classroom,
the playground, the hallways, the staff room, the principalÊs office, science
teachers, mathematics teachers, social studies curriculum materials and so forth.
The Generic Qualitative Method may appear to be a loose method incorporating

the use of interviews, observations and document analysis in data collection.
However, it does not mean that it is less rigorous. You have to be clear about
your objectives and ensure that internal validity is maintained while your values
do not influence the interpretation of data.
As mentioned earlier, the Generic Qualitative Method may incorporate many of

the elements of ethnography, action research and the case study, but cannot be
considered to belong purely to any of these qualitative methods. The focus is
on the identification of patterns and categories with the aim of describing
phenomena. Besides description, the data is interpreted to explain phenomena
but not with the intention of building or developing theory. The following is
a list of techniques that may be employed in a Generic Qualitative Method
research depending on the objectives of the study:
(a) To answer specific research questions or begins the study with rather
general or broad questions;
(b) The researcher could be a passive observer or an active participant;
(c) The methods of data collection could include interviews, observations or

document examination or a combination of the three;
(d) Semi-structured or unstructured interviews;
(e) Various documents examined such as teaching materials, newsletters,

lesson plans, memos, circulars and many other kinds of written items;
(f) Content analysis may be used to analyse communication (oral or written);
(g) Information is provided about the sample selected;

(h) There is no specific length of data gathering but a rationale is given for the
time spent gathering data from observations/interviews;
(i) Subjects could be interviewed individually or in focus groups;
(j) The biases of the researcher are made explicit;
(k) Entry or access to the setting is described in detail; and
(l) Fieldnotes are kept of all observations: jottings, description, analysis and
reflection.
The list of possible research techniques that may be adopted in the Generic
Qualitative Method is very much dependent on the objectives of the study and
the researcherÊs beliefs on what constitutes knowledge. What information will be
needed to explain the phenomenon examined? How can others be convinced that
the findings of the study are credible? The following are some examples of
phenomena that may be studied using the Generic Qualitative Method:
(a) Racial integration in the school canteen during recess in a primary school;
(b) Use of portfolios in assessing literacy in primary school children;
(c) Interaction in a technology-based classroom using the Internet to teach

economics;
(d) Cognitive level of questions in an inquiry-based science classroom;
(e) Teacher burnout and attitudes;
(f) Being in an academically weak class for three years: Perceptions and
attitudes of three students;
(g) Analysis of online collaboration in a biology class; and
(h) Comparative analysis of staff meetings in two schools.

Ć When a paradigm at a certain point in time is unable to explain phenomena

satisfactorily, a paradigm shift should occur together within the existing
paradigm.
Ć A paradigm shift will lead to the introduction of new research methods and
tools, and a whole new way of how the researcher sees the world.
Ć According to the positivist approach, knowledge obtained using the scientific

method is objective and measurable.
Ć According to the phenomenological perspective, focus is on the processes and

experiences one goes through.
Ć Qualitative research focuses on interpretation of phenomena in their natural

settings; to make sense of the meanings people bring to these settings.
Ć Inductive approach: The qualitative researcher begins by observing

phenomena and continues to find patterns in the form of emerging themes,
categories, concepts and typologies.
Ć Ethnography is a type of qualitative research method employed by

anthropologists to study human society and culture.
Ć The term „ethnography‰ means „portrait of a people‰ which involves a

description of a people and its culture.
Ć A qualitative case study is an intensive, holistic description and analysis of a

single instance, phenomenon, or social unit.
Ć Action research is based on the belief that the practitioner is the best judge of
his or her practice; it encourages the researcher to put personal theories into
practice.
Ć Action research adopts a spiral approach comprising four steps: planning,

acting, observing and reflecting.
Ć The Generic Qualitative Method does not have a guiding set of philosophic
assumptions in the form of one established qualitative methodology, but
instead employs the techniques of data collection of other qualitative
methods.

Action Research Natural Setting

Case Study Paradigm Shift
Culture Phenomenonology
Ethnography Positivism
Fieldnotes Qualitative methods
Generic Qualitative Method Thick description
1. Consider the following list of research problems and explain what

would be the most appropriate qualitative research method for each
one. If you think that more than one method would be appropriate,
explain why.
(a) An evaluation of a drug rehabilitation centre.
(b) The role of counsellors in primary schools.
(c) Leadership styles of principals.
2. „The flexibility of the Generic Qualitative Method may explain its

popularity in educational research‰. Discuss.
Dick, B. (n.d.). Action learning and action research. Retrieved from:

http://www.scu.edu.au/schools/gcm/ar/arp/aandr.html
Genzuk, M. (n.d). A synthesis of ethnographic research. Retrieved from:

http://www-rcf.usc.edu/~genzuk/Ethnographic_Research.html
Massey, A. (n.d.). The way we do things around here: The culture of ethnography.
Retrieved from:
http://www.freeyourvoice.co.uk/htm/waywedo.htm#harg
Qualitative social science research methodology. (n. d.). Retrieved from:

http://faculty.ncwc.edu/toconnor/308/308lect09.html

Tellis, W. (July. 1997). Introduction to case study. The Qualitative Report, 3(2).
Retrieved from: http://www.nova.edu/sss/QR/QR3-2/tellis1.html
Tellis, W. (September, 1997). Application of a case study methodology. The

Qualitative Report, 3(3). Retrieved from:
http://www.nova.edu/sss/QR/QR3-3/tellis2.html

Topic  Qualitative
8 Data Collection
Methods
LEARNING OUTCOMES
1. Describe the major types of data gathering techniques;
2. Discuss the time spent in collecting data;
3. Elaborate on sampling issues in qualitative research;
4. Identify the skills required of the qualitative researcher; and
5. Critically evaluate validity issues in qualitative research.
 INTRODUCTION
In Topic 7 we discussed four main methods of qualitative research commonly
adopted in educational research, namely; ethnography, case study, action research
and the basic qualitative method. Irrespective of the research method adopted,
the techniques of data collection are more or less similar. In this topic, we will
discuss in detail three common data collection or evidence-gathering techniques
employed in qualitative research. For example, the experimental method in
quantitative research uses tests or attitude scales to collect data. Similarly, the
survey method use questionnaires and interview checklists to collect data. So
for ethnographies or case studies, the data collection techniques could involve
observations, interviews or examining documents, or a combination of three
techniques.

152  TOPIC 8 QUALITATIVE DATA COLLECTION METHODS
8.1 DOCUMENT EXAMINATION

If the focus of your study is the examination of documents, than you should
have access to such materials as letters, memos, notes, diaries, photographs,
audiotapes, videotapes, films, articles, books, manuscripts, e-mails, online
discussions and so forth. In general, documents are any preserved recordings
of a personÊs thoughts, actions or creations (Potter, 1996). Documents may be
examined to investigate patterns and trends of the past as is commonly done by
historians. In this case, documents are the primary source of data. Document
examination may also provide confirmatory evidence of the information obtained
from interviews and observations.
8.1.1 Content Analysis

Content analysis is a technique for analysing the content of documents which
would be in the form of words, phrases, sentences, paragraphs, pictures, symbols
or ideas. Content analysis is widely used by law enforcement agencies to analyse
e-mails, letters and telephone conversations. Content analysis can be done both
quantitatively as well as qualitatively. Very often researchers use computer
programmes to assist in analysis. Most content analysis in education has been
aimed at answering questions directly relating to the materials analysed. For
example, content analysis of student essays can provide information about
grammatical and spelling errors which may be applied in the development of
remedial programmes. A content analysis of textbooks can tell us such things as
topics covered, emphasis in each topic, sequence of topics and so forth. Other
areas of education that have been studied using content analysis include the
treatment of women and minorities in textbooks, newspapers, literature and
television, the communication taking place in the classroom, etc.
For example, you could use content analysis to examine teacher feedback in the
classroom based on phrases such as, „All right‰, „Good‰, „Why do you say
that?‰, „Can anyone give another factor?‰
(a) The first step in content analysis is to establish the objectives of the
investigation. For example, you may be interested in finding evidence for
critical thinking activities in science textbooks or the infusion of values in
economics textbooks or the treatment of world events in history textbooks.

TOPIC 8 QUALITATIVE DATA COLLECTION METHODS  153
(b) The second step in the content analysis process is to locate data that is
relevant to the objectives. For example, if the objective of your study is to
examine how women are treated in Malaysian novels for teenagers, you
have to locate these novels and decide on the number and variety of novels
to select.
(c) The third step is to establish an empirical link between the data selected
and the inferences to be made from the data (Borg & Borg, 1988). In other
words, you need to provide some theory or model based on reviewing
previous research or citing expert opinion that supports the relationship
between the data and the objectives upon which the study is based. For
example, a review of previous literature may reveal that novels for
teenagers tend to portray women as helpless, „waiting for their prince
charming‰ and so forth.
(d) The fourth step is to sort the content into themes or categories. For example,
if you are examining the minutes of meetings, you might have themes such
as issues, delegation and action. The next step is to devise a coding scheme
which could record frequency (the number of issues discussed and how
often they are discussed), direction (who is the content directed to),
intensity (power of content), and space (size of content). However, not
everything always fits into the themes or categories and there is some
leftover content to be accounted for. This content must be interpreted by a
knowledgeable researcher who knows something about the culture of his
or her subjects. The researcher should be careful about making inferences
about motivation or intent and as far as possible content analysis should
be confined to what is in the text. For example, you cannot use content
analysis to prove that newspapers mislead the public or that certain articles
in newspapers have a certain effect on public attitudes. However, you can
use concepts such as „unconscious bias‰ or „unintended consequences‰
when interpreting the text.
SELF-CHECK 8.1
1. What do you understand by document examination? Give specific

examples.
2. Explain what content analysis is.

8.2 OBSERVATION
Imagine that you want to find out what goes on
in the teachersÊ lounge or staff room. You could
interview those involved, or maybe even send
out a questionnaire. Using the interview or the
questionnaire, you would be getting what people
thought about what was going on. Sometimes,
the best way to gain a „rich picture‰ of a setting
such as the staff room, the school canteen, a staff
meeting, the playground or the classroom is to
see for yourself what is happening, rather than
depending on your respondents.
Observation is the technique of obtaining data

through direct contact with a person or a group of persons. Since, the main focus
of qualitative research is naturalism; the researcher has to observe the person or
persons in their natural state as undisturbed as possible (Christians & Carey,
1989; Smith, 1987). The role of the researcher may be viewed as a continuum. On
one extreme, the researcher is a passive observer and on the other extreme
the researcher is a participant observer. In between these two extremes, the
researcher may be an active observer (Potter, 1996).
Passive Observer Active Observer Participant Observer
(a) Passive Observer

The best way to be not involved and keep your distance from your
subjects is to be a passive observer. As a passive observer, you simply
gather documents and observe the individual or individuals without
doing anything to disturb the situation. The researcher is unobtrusive and
watches the group from the outside; i.e. the ethic or outsiderÊs perspective.
To do so, the researcher must gain access and be accepted by the individual
or individuals being observed. For example, in collecting e-mails or essays
written by subjects or learning journals of students, the researcher examines
them without being involved. Similarly, a researcher interested in studying
children interacting in school canteens or the playground, merely observes
them without being involved. A certain amount of distance is maintained
between the researcher and the person or persons being observed.

(b) Participant Observation

As the name „participant‰ suggests, the researcher participates in the
activities of the persons being observed rather than being an observer. The
researcher has two roles ă as observer and as participant. The researcher
participates as much as possible in the daily life of the subjects while also
carefully observing everything he or she can, about it. Through this, the
researcher is seeking to gain what is called an „emic‰ perspective or the
nativeÊs point of view or the insiderÊs perspective. The researcher records
detailed field notes, conducts interviews based on open-ended questions
and gathers whatever site documents might be available in the setting as
data. Participation can take many forms.
For example, the researcher could show a film or video to stimulate

discussion or question subjects and observe how they would react to the
stimulus. The researcher takes an active position with the purpose of
stimulating subjects to think about things they might never have thought
about before. However, as pointed out by Hammersley and Atkinson
(1983), there is the danger of the researcher „going native‰ which means
being too involved or having too close a rapport with the person or persons
being observed to the extent that you loose objectivity.
(c) Active Observer

Between being a passive observer and an active participant, the researcher
could take a middle position of being an active observer. Here, participation
is allowed but limited. The researcher may intrude into the lives of subjects
such as entering their homes or their communities but remains passive once
inside the environment so as not to influence the natural occurring
behaviours and conversations. For example, a researcher interested in TV
viewing habits may enter a household, eat with the family, play with the
children and take part in family activities. Family members are told not to
change their routines in order to accommodate the observer. However, the
researcher tries as far as possible to be passive, saying as little as possible so
as not to influence the behaviours and conversations of subjects.
SELF-CHECK 8.2
1. What is the difference between passive observation and participant

observation?
2. Give an example of passive and participant observation of
classroom processes in a primary or secondary school.

ACTIVITY 8.1
To practise being a passive observer, undertake a 15 minute observation

of an educational setting. This setting could be the teachersÊ lounge (staff
room), the school canteen, playground, staff meeting, classroom or any
setting in your place of work. Make notes on what you observed and
from the notes, write down some of your thoughts on the interesting
issues observed.
8.3 INTERVIEWS
Interviewing is a technique for gathering data from humans
by asking them questions and getting them to react verbally.
There are many different ways of conducting interviews (see
Figure 8.1). Structured interviews use an interview schedule
that is similar to the survey questionnaire. You could phrase
the question in such a way that you have a limited range of
responses.
For example, „Do you think the image of teachers in society has gone down?‰
Strongly agree, agree, somewhat agree, disagree and strongly disagree.
Structured interviews are widely used in surveying opinions, beliefs and
perceptions of people. Individual interviews are expensive and you should
consider whether the same amount of data can be more efficiently collected using
written questionnaires.
Qualitative Research
Structured interviews Semi-structured interviews Unstructured interviews
Individual Individual Focus Group

Figure 8.1: Types of interviews
Semi-structured interviews and unstructured interviews are widely used in

qualitative research. Semi-structured interviews consist of a list of open-ended
questions based on the topic areas the researcher intends to study. The open-
ended nature of the questions provides opportunities for both the interviewer

and interviewee to discuss certain topics in more detail. If the interviewee has
difficulty answering a question or hesitates, the interviewer will probe. Three
types of probes commonly used by the interviewer are:
(a) Detail-oriented probe

When did it happen to you?
Who was with you?
(b) Elaboration probe

Tell me more about the incident.
Can you give an example.
(c) Clarification probe

IÊm not sure I understand what you mean by „hanging out‰. Can you
explain further?
You said that your principal is extremely autocratic. What do you mean
by autocratic?
Unstructured interviews aim to acquire in-depth perspectives. Only a limited

number of topics are discussed, sometimes as few as one or two topics. Although
only a few topics are discussed, they are covered in great detail. The interview
may begin with a question such as „IÊd like to hear your views on school
discipline‰. Subsequent questions would follow from the intervieweeÊs responses.
Unstructured interviews are used to find out about a specific topic but have no
structured or preconceived plan or expectation as to how the interview will
proceed.
Face-to-face or personal interviews are labour intensive but can be the best way
of collecting high quality data, especially when the subject matter is very
sensitive, if the questions are very complex or if the interview is likely to be
lengthy (Mathers, Fox & Hunn, 2002).
8.4 TRIANGULATION
Triangulation is the process of strengthening the findings obtained from a
qualitative inquiry by cross-checking information. A researcher who argues that
his or her findings are derived from many different kinds of people across many
different situations will be more convincing than another researcher whose
conclusions are based on observations of one person in one setting (Potter, 1996).

Document
Examination
Interview Observation
Figure 8.2: Methods triangulation
There are three types of triangulation:
(a) Methods Triangulation ă The use of multiple research methods to study a

phenomenon (see Figure 8.2). For example, you examine the lesson plans of
a history teacher (document examination) and then observe her teaching
using the lesson plan (observation). At the end the lesson, you follow-up
by interviewing the teacher regarding what was planned and what was
actually performed in the classroom.
(b) Investigator Triangulation ă The use of multiple investigators (i.e. multiple

researchers) in collecting and interpreting the data.
(c) Theory Triangulation ă The use of multiple theories and perspectives to

help interpret and explain the data.
Triangulation is used in bringing together different sources of information to

converge or conform to one interpretation. With the convergence of information
from different sources (documents, interviews and observations), settings and
investigators, the researcher can make a powerful argument that the interpretation
is more credible.
SELF-CHECK 8.3
1. When would you use unstructured interviews?

2. What is triangulation?

8.5 SKILLS REQUIRED OF THE RESEARCHER

In a qualitative study the investigator is the primary instrument for gathering
and analysing data. So, it is very important that the researcher maintain
impartiality if the findings are to be accepted. While it is well established that
humans make mistakes and to err is human; it essential that the researcher is
aware how his or her personal biases may influence interpretation of data. Just as
any research instrument is fallible, the human instrument is even more
susceptible to error. Merriam (1998) identified the following skills required of
qualitative researchers:
(a) First, as a qualitative researcher, you must have an enormous tolerance for
ambiguity. Tolerance for ambiguity means the ability to tolerate or accept
inconsistencies and uncertainties. In qualitative research, there are no set
procedures or guidelines to be followed step by step. From the design stage
to the data collection and data analysis stages the researcher has to be
prepared to face unforeseen events and change in direction. Merriam (1998)
compares the role of the qualitative researcher to that of a detective who
looks for clues, finds the missing clues and putting the pieces together.
If you are a person who likes structured situations and have no patience
with ambiguity, you should choose quantitative methods which are more
structured and have step by step guidelines.
(b) Second, a qualitative researcher should be sensitive. Sensitive refers

specifically to data collection. How should the researcher be sensitive when
collecting or gathering data?
(i) The researcher should be sensitive to the obvious or explicit
information and the not so obvious or implicit information such as the
nonverbal behaviour of people (such as gestures, silence, etc).
(ii) The researcher should be sensitive to the information collected in
terms of what it reveals and how it reflects what is happening.
(iii) The researcher should have a keen sense of timing.
 When observing he/she knows when it is enough and to stop
observing.
 When interviewing, the researcher should know:
ă When to probe;
ă When to allow for silence; and
ă When to change the direction of the interview.

(c) Third, the qualitative researcher should be able to detect personal biases. It
should be remembered that the primary instrument is the researcher and
being human there is a likelihood that his/her values might creep into the
observations and interviews conducted. The researcher will bring his/her
perceptions or interpretations into the phenomenon being studied. Some
scholars have argued that this is unavoidable and we will have to accept
this as part and parcel of qualitative research. However, if the findings of
any qualitative research are to be accepted by others, there is a need to
ensure that infiltration of the researcherÊs values is minimised.
So the qualitative researcher should be able to understand how biases or

subjectivity shape an investigation and interpretation of findings.
(d) Fourth, the qualitative researcher must also be a good communicator.

„A good communicator empathises with respondents, establishes rapport,
asks good questions, and listens intently‰ (Merriam, 1998. p. 23). The
extent to which qualitative researchers are able to communicate warmth
and empathy often marks them as good or not-so-good data collectors
(Guba & Lincoln, 1981). Another vital communication skill is listening. It is
only by listening to individuals that a researcher obtains good information
whether in interviews and even observations („hearing‰ the implied
meanings of communication). Besides having oral skills, a good qualitative
researcher must also be a good writer. Writing is needed when taking notes
and writing the report of findings. Qualitative research requires a lot more
writing than quantitative research.
SELF-CHECK 8.4
1. What are the skills required of a qualitative researcher?

2. List other qualities of a good qualitative researcher.
8.6 LENGTH OF TIME SPENT IN COLLECTING

DATA
Generally, it has been argued that when doing qualitative research, a substantial
amount of time should be spent in the field collecting data. However, there is less
agreement on how long the researcher should be in the field collecting data. If
you are not sure how much time is spent in collecting data, read the literature in
your area of research. If you find that the research study you are referring to is

too superficial and there is a lack of in-depth explanation, then you may conclude
that more time should be spent in conducting the study. The length of time spent
in collecting can be viewed from two angles:
(a) Span of Time ă how long? 2 weeks, 1 month, 3 months and so forth.
(b) Degree of Contact ă The number of contact hours the researcher spends
with the person or persons.
For example, Researcher A who observes a group of low ability students at the
rate of 1 hour per week for a span of 6 months which comes to a total of 24 hours.
Researcher B who moves in with the family of a low ability student and observes
the learner during all his waking hours. Who spent more time collecting data?
Researcher A took more time gathering data if you are focusing on span of time.
If you are focusing on contact hours, than Researcher B took more time collecting
data. Both conceptions of time (span of time and contact hours) are important. A
long span of time enables you to see broad and overall patterns while long
contact hours allow you to detect micro patterns or details in the environment.
While it is not advisable to prescribe the length of data collection, it would be

wise for the researcher to address the issue of time both in terms of span and
degree of contact. This is a minimum requirement. It is best that the researcher
justify the span of time and degree of contact proposed. For instance, you might
state that the X number of contact hours would be sufficient to capture
interesting patterns about the phenomenon being observed.
Examples of Studies: Length of Time Spent
(a) Cox (1980) observed a kindergarten class for two hours a day (though
not every day), between September and June. Thus about 132 hours of
observation took place over a 66 day period.
(b) Bossert (1974) watched two classrooms over a six month period about
3ă4 hours each week. He observed for 40ă60 minutes each time, and
observations were rotated so that all classroom activities could be sampled.
(c) Baker (1985) observed children in the playground twice a day during each
of two recess periods. These observations took 30ă40 minutes each day over
a one year period.

SELF-CHECK 8.5
1. What is the difference between the span of time and the contact
hours in terms of length of time spent in data collection?
2. How should a researcher determine the length of time spent in
collecting data?
8.7 SAMPLING
In quantitative research, sampling is the selection of a group of persons from
a population with each person having an equal chance of being selected. The
objective is to draw a representative sample and the results obtained from the
sample can be generalised to the population. How is the issue of sampling dealt
with in qualitative research? In qualitative research the concern is with the issue
of „access‰. What is meant by access?
When the researcher intends to observe or interview an individual or a group of

persons, he or she must gain access which means getting permission to be
physically present to gather the data. Having gained access, the researcher is
obliged to follow certain social rules so as to maintain access. This is especially
important in relatively private settings where people do not want an outsider to
interview or observe them. Related to the issue of access is the rationale or reason
for selection of the particular sample (which could be an individual or a group of
individuals). In qualitative research, there are two main reasons for selection of
the sample (Potter, 1996).
(a) First, the researcher might select a person or persons to investigate because
of efficiency or convenience.
(b) Evidence is collected from people who are easily available to support the
researcherÊs arguments. For example:
(i) A teacher might ask his or her students in the class to allow themselves
to be interviewed.
(ii) The researcher might go into the community to which he or she

belongs and observe the behaviours of easily accessible parents.

(iii) The researcher might use the „snowball technique‰ of interviewing

those people who are available and then asking them to suggest
others who might be willing to be interviewed. In the process, the
number of people gradually expands to those who have been referred.
(c) Second, the researcher might want to select a sample based on

representativeness or a critical case or a typical case.
(i) If the researcher wants a sample that is representative, than the
method of sample selection used in quantitative research is adopted
where each individual case has an equal chance of being selected.
(ii) If the researcher wants a critical case, all possible cases are examined
until a critical case is found that best captures the unique features of
what he or she wants to illustrate.
(iii) If the researcher wants a typical case, he or she looks for the case that
best exemplifies the norm, and the extent to which cases are different
or similar to the typical case.
Lincoln and Guba (1985) emphasise that the guiding principle of sampling in
qualitative research is one of convenience. An important consideration in
sampling is whether there are people available who will allow the researcher to
collect data about them. For example, the researcher interested in preschoolers
interacting during recess may call up several kindergartens until he or she finds
kindergartens that will allow him or her to observe children during recess.
If you read research using qualitative methods, you will see a range of positions
on the issue of sampling. Some studies do not give much information about how
the people interviewed or observed were selected. Other studies give some
information about why they selected certain people and how they gained access.
The main issue with sampling is the extent to which readers trust the findings of
the research. If the researcher fails to provide sufficient information about how
he or she collected evidence especially in relation to how and why particular
persons were selected, it would be difficult for the reader to trust the findings.
On the other hand, if the researcher provides a detailed description about the
process of gaining access and selection of the persons interviewed or observed,
readers will be more inclined to trust the findings.
SELF-CHECK 8.6
How do you decide on who and how many subjects to include in your
sample?

8.8 VALIDITY OF QUALITATIVE RESEARCH

The shift in focus from a quantitative paradigm to a qualitative paradigm in
educational research has raised many questions about the latter approach.
Various books and articles and conferences have and continue to discuss
important issues with regards to qualitative research. Among the issues raised is
whether qualitative research can be considered as scientific. How can the
qualitative method be considered scientific if the probability of the values of the
researcher creeping into the research is very high? Can the results of qualitative
research be generalised to other situations or other individuals? Still others
question whether quantitative which is considered „more‰ scientific is truly free
from the influence of values and beliefs of the researcher.
8.8.1 Types of Validity

The issues of validity have been discussed in great length by qualitative
researchers. Validity in qualitative research is defined as whether the data is
plausible, credible and reliable, and can be defended when challenged. Unlike
validity in quantitative research which is more definite, validity in qualitative
research is debatable. There are some researchers who believe that the concept of
validity as understood in quantitative research is not congruent with qualitative
research and as such should be ignored. Others are of the opinion that effort
should be made to ensure validity if the results of qualitative research are to be
believed. Maxwell (1996) identified three types of validity that should be given
attention in qualitative research.
(a) Descriptive Validity

This is defined as the accuracy of the behaviours, events, objects, settings
and others reported by the researcher. For example, that which is reported
is actually what happened or what was heard or observed.
(b) Interpretive Validity

This is defined as the accuracy of interpretation as to what happened in
the minds of subjects and the extent to which the researcher understands
exactly the opinions, thinking, feelings, intentions and experiences of
subjects.
(c) Theoretical Validity

This is defined as the extent to which the theoretical explanations developed
are in congruence with the data and is reliable and can be defended.

8.8.2 External Validity of Qualitative Research

External validity is defined as the extent to which the findings of a study may be
generalised to another setting or another group of people. Benz and Newman
(1998), state that:
"That one should be able to generalise underlies science. However, we are

unwilling to accept fully that generalisability is consistent with the
qualitative paradigm...... in principle; generalisability is the purpose of
quantitative, not qualitative research."
"In fact, we have assumed that, if the purpose of the research is to generalise,
one should employ quantitative methodology."
(p. 54)
They are of the opinion that generalisation is not important and is not consistent
with the qualitative paradigm or perspective. If generalisation is the objective, than
quantitative methods should be used and not qualitative methods. However, there
are other researchers who believe that efforts should be made to generalise
findings of qualitative research. These researchers argue that the in-depth
description of a particular phenomenon is sufficient for the researcher to make
generalisations to other individuals or individuals. To enable the findings of
qualitative research to be generalised, researchers have proposed ways in which
validity can he enhanced. Benz and Newman (1998) proposed the following terms
while discussing the issue of generalisation of qualitative research findings:
(a) Applicability
Can the study be applied to another sample? It should be remembered that
there is no „significant difference‰ and it is difficult to generalise to the
population based on the findings of a sample. The deep description of the
characteristics of the subject being studied may allow one to conclude
the extent to which it is comparable to other subjects. If the subjects are
comparable, then one would be more comfortable to make generalisations.
For example, you observe three science teachers. Can you generalise what
you observed to all science teachers in the country? It has been argued that
the in-depth description of subjects or the sample studies enables the
researcher to decide the extent to which it is the same with other subjects or
another sample. The greater the similarity between subjects, the higher is
the possibility of making generalisations.

(b) Context Dependent

Can the findings of a study be generalised to another setting or context? Say
for example, what was observed is not dependent on the context. Or what
was observed was not so much context dependent. In these situations the
findings may be „transferred‰ (Guba & Lincoln, 1989) to another context or
situation. Than you may generalise the findings.
(c) Replicability
What is the likelihood that a particular product or event will recur given
similar conditions? It is difficult to replicate or repeat a qualitative research
because the natural setting is constantly changing. Unless you have data
showing the changes, you are advised to be cautious when making claims
that a study can be replicated.
On the issue of generalisation or external validity, Lincoln and Guba (1985)

suggest the term „transferability‰. They warned researchers to be careful when
suggesting the generalisation of qualitative research findings. If the readers of the
report or other researchers wish to generalise the findings of one study to another
setting, it is their responsibility. It is not necessary for the original researcher to
show that the findings can be generalised.
SELF-CHECK 8.7
1. Discuss the issue of validity in qualitative research.

2. To what extent can you generalise the findings of qualitative
studies?
8.8.3 Enhancing Internal Validity of Qualitative

Research
Johnson (1997) and Benz & Newman (1998) discussed in detail the issue of
internal validity of qualitative research. They identified the following strategies
that should be considered by researchers if they wish to enhance the internal
validity of their studies.
(a) Triangulation
This involves the cross-checking of information from different dimensions.
Data triangulation is when the researcher refers to different sources of data
in understanding a particular phenomenon. Methods triangulation is when
the researcher uses different methods to study a particular phenomenon.

Investigator triangulation is when the research involves different investigators

or researchers in interpreting and explaining the data. Theory triangulation
is when different theories are used to explain the data.
(b) Longer Period of Time

Obviously, given the time and resources, researchers would like to remain
in the field as long as possible collecting data to provide a more accurate
picture of the phenomenon observed. For example, a researcher studying
classroom processes will encounter problems when the school is preparing
for sports day. It is possible that classes may be cancelled or no proper
teaching will be conducted as students and teachers are busy with various
activities. So, it is essential that adequate time be allotted to enable the
researcher to obtain a more accurate and holistic picture of classroom
processes.
(c) Member Checking

To enhance internal validity, a researcher could return to the subjects who
were interviewed and check whether what was recorded was what they
had said in the interview. For example, „Is this what you meant when you
said ⁄?‰ Or go back to the subjects you had observed and ask them
whether what had been recorded about their behaviour is accurate. For
example, „Did you do this?‰ Through this process of verification, the
internal validity of qualitative research can be enhanced.
(d) Peer Review

Discuss the interpretations and conclusions of the findings with a peer.
Choose a peer who is not involved in the study but is interested in what
you are doing. A person who is interested in the study is more likely to be
critical and challenge what you wrote. Involving a peer who is interested in
your study will enhance the likelihood that he or she will give an in-depth
opinion about your data.
(e) Low Inference Descriptors

Inference descriptors are words or phrases used to describe a collection of
information. By using low inference descriptors you are using descriptors
that are close to what the subjects said or what you had recorded as field
notes. The closer the raw data is with the inference descriptors the lesser is
the inference made and more accurate is the description. For example, if
you use the inference descriptor „autocratic‰ to describe the behaviour of a
principal you are making an inference based on evidence such as „one-man
show‰, „refuses to listen‰, „his way is always right‰. The closer your
descriptor is to the data, less inference is made by the researcher and
„interpretative validity‰ is enhanced.

(f) Negative Case Sampling

To enhance internal validity, you could pick cases that are different to the
cases you are studying. The aim is to confirm or disconfirm the findings of
your study by comparing it with the findings of cases that are opposite or
different. For example, you are studying the leadership behaviour of a
principal based on interviews with selected teachers and your findings
indicate the principal studied is autocratic. To enhance the validity of your
findings, you interview teachers from another school where you suspect the
principal is less autocratic and compare the opinions of these teachers.
(g) Reflexivity
One of the main issues with the validity of qualitative research is the
likelihood of the researcherÊs values creeping into the interpretation of data.
To minimise this happening you examine yourself critically to detect any
potential bias and inclination that may influence the conclusions you make
about the data. Though no data can be a hundred percent value free and
objective, the researcher should make serious effort to convince others that
the level of objectivity of your study has been maintained. If you had used
other investigators in your study, you should provide evidence that there
was high inter-rate reliability and consistency between investigators.
(h) Audit Trail

An audit trail is the keeping of detailed and accurate records of everything
the researcher did and of the data collected. Such records should be
documented and organised appropriately for retrieval purposes. These
records should be made available as evidence of data collected when
challenged as well as for validating the interpretation of data.
SELF-CHECK 8.8
What should a researcher do to enhance the internal validity of

qualitative research?

Ć Examination of documents such as letters, memos, notes, diaries, photographs,

audiotapes, videotapes, films, articles, books, manuscripts, e-mails, online
discussions reveal peopleÊs thoughts, actions and creations.
Ć Observation is the technique of obtaining data through direct contact with a

persons or group of persons.
Ć Interviewing is a technique of gathering data from humans by asking them

questions and getting them to react verbally.
Ć Triangulation is the process of strengthening the findings obtained from a

qualitative inquiry by cross-checking information.
Ć If you feel that the research study to is too superficial and there is a lack of
in-depth explanation, than it may be concluded that more time should be
spent in conducting the study.
Ć Validity is the credibility of findings and includes descriptive validity,

interpretive validity and theoretical validity.
Ć External validity is defined as the extent to which the findings of a study may
be generalised to another setting or another group of people.
Ć Triangulation, member checking, negative case sampling and audit trail are
some techniques to enhance qualitative research.
Descriptive, interpretive Observer

Document examination Participant observer
External validity Sampling
Internal validity Skills of a qualitative researcher
Interviews Structured interviews
Length of data gathering Triangulation
Observations Unstructured interviews

1. „Characteristics of Racial Interaction in the Playground of a Vision

School‰
(a) What data collection techniques would you use for the above
study?
(b) How long do you think you should be in the field?
(c) How would you would select the subjects for your study?
2. „Comparative Study of the Leadership Style of Two School

Principals‰
(a) What data collection techniques would you use for the above
study?
(b) How long do you think you should be in the field?
(c) How would you select the subjects for your study?
Chapter 15: Interviewing in qualitative research (n. d.). Retrieved from:

http://www.oup.co.uk/pdf/0-19-874204-5chap15.pdf
Golafshani, N. (2003). Understanding reliability and validity in qualitative

research. The Qualitative Report, 8(4). Retrieved from:
http://www.nova.edu/ssss/QR/QR8-4/golafshani.pdf
Module 3: Essentials of participant observation (n.d.). Retrieved from:

http://jan.ucc.nau.edu/~mid/edr725/class/observation/
Observation with Intervention (n. d.). Retrieved from:

http://www.ccny.cuny.edu/bbpsy/modules/observ_with_interv.htm
Palmquist, M. (n.d.). Content analysis. Retrieved from:

http://www.gslis.utexas.edu/~palmquis/courses/content.html
Two explanations of content analysis (n. d.). Retrieved from:

http://www.uiowa.edu/~commstud/adclass/research/content_analysis.html

Topic  Qualitative
9 Data Analysis
LEARNING OUTCOMES
1. Define what qualitative data analysis is;
2. Compare the approaches in analysing qualitative data;
3. Describe the stages involved in qualitative data analysis; and
4. Develop the categories in qualitative data analysis.
 INTRODUCTION
The Leadership Behaviour of a Principal as Perceived by
Teachers in a School
The following are probable qualitative data collection methods that
could be used to investigate the above topic:
(a) Field notes from observations of staff meetings.
(b) Focus groups interview transcripts.
(c) Copies of diary entries teachers have been asked to complete
each day.
(d) Researcher memos and reflections.
(e) Audio recordings.
The above is an example of a qualitative study investigating the leadership

behaviour of a principal. Note the range of techniques employed to study the
principal. At the end of the study you will have large piles of field notes, audio
recordings, documents (minutes of staff meetings), dairy entries and reflections

172  TOPIC 9 QUALITATIVE DATA ANALYSIS
sitting on your desk waiting to be analysed. How do you go about making some
sense of qualitative data? Qualitative data is mostly in the form of words and
narratives, while some may include visual images, audio and video recordings.
Quantitative data is usually a mass of numbers that is processed, summarised

and presented in the form of tables, charts and graphs. Data may be cross
tabulated while the means and standard deviations are calculated to establish
significant differences; and correlation coefficients used to determine relationships.
Based on these initial findings, more advanced statistical procedures such as
multivariate analysis may be performed to seek patterns and relationships in the
data if the research question demands it.
What about qualitative data? How is qualitative data analysed? Qualitative

data is a mass of words obtained from recordings of interviews, fieldnotes of
observations, and analysis of documents as well as reflective notes of the
researcher. This mass of information has to be organised, summarised, described
and interpreted. Any statistical package will not tell you which of the many
statistical tests to use to analyse numerical data. Similarly, there are many
different ways of analysing qualitative data as there are qualitative researchers
doing it. However, there is more agreement in the analysis of quantitative
data and there is less agreement on how to analyse qualitative data. Different
researchers have proposed different ways of analysing qualitative data.
Fortunately, there are some common procedures in the analysis of qualitative
data.
Generally, since numbers are not used, the qualitative researcher looks for
categories or themes from the raw data to describe and explain phenomena.
He/she analyses the relationships and patterns between the categories or themes
that have been identified. These categories or themes may be derived using two
approaches:
(a) Deductively ă whereby at the very beginning or half-way through you

identify the categories or themes and „fit‰ the data into the categories and
themes.
(b) Inductively ă whereby the categories or themes are allowed to „emerge‰

from the data gradually. This has been termed as „grounded theory‰. We
will discuss this later in the topic.

TOPIC 9 QUALITATIVE DATA ANALYSIS  173
9.1 STAGES IN QUALITATIVE DATA ANALYSIS

The stages in the analysis of qualitative data are shown in Figure 9.1. It usually
begins with familiarisation of the data, transcription, organisation, coding, analysis
(grounded theory or framework analysis) and reporting (though the order may
vary).
Familiarisation Transcription Organisation
Analysis
Grounded
theory
Report Coding
Writing Analysis
Framework
analysis
Figure 9.1: Stages in qualitative data analysis
(a) Familiarisation
The first step of data analysis is familiarisation in which you listen to tapes
and watch video material, reading and re-reading the field notes, making
memos and summaries before formal analysis begins. The is especially
important when besides you, others were also involved in data collection.
You have got to be familiar with the field notes they made (perhaps trying
to decipher their handwriting!).
(b) Transcription
Almost all qualitative research studies involve some degree of transcription.
What is transcription? Transcription is the process of converting audio or
video-recorded data obtained from interviews and focus groups as well as
handwritten fieldnotes into verbatim form (i.e. written or printed) for easy
reading. Why do you have to do this? If you were to analyse directly from
an audio or video recording, there is the likelihood that you might include
those sections that seem relevant or interesting to you and ignore others.
With a transcript of everything that you observed and recorded (audio,
video or fieldnotes), you get the whole picture of what happened and the
chances of your analysis being biased is minimised.

You should not forget to include non-verbal cues in the transcript such as silence
(which may indicate embarrassment or emotional distress), pause for thought
(such as „well ... er ... I suppose ...) laughter, gestures (which may add meaning
to the spoken word) and so forth. If someone else is transcribing your material,
make sure to tell him or her how much of this non-verbal information to include.
If you have never transcribed material, it is useful to do a little yourself. Try
doing Activity 9.1.
ACTIVITY 9.1
Find a member of your family, or a friend or colleague and interview

the person for about 10 minutes concerning „What are the characteristics
of a good teacher?‰. Try to probe what it is that makes a good teacher.
Tape record the interview, then transcribe into a word processor in
your own time, including as much non-verbal material as you can.
1. How long did the transcription take you, compared with the
original interview?
2. Highlight the non-verbal communication you were able to include.
What does it tell you, in addition to the words you have recorded?
3. Look at the questions you asked, and any comments you made.
Had you at any point led the respondent in any way, or missed
important cues given by the respondent?
4. Listen to the tape again with the transcript in front of you. Did you
change any of the words from the tape? Did you transcribe
everything accurately?
(c) Organisation
After transcription, it is necessary to organise your data into sections that
are easy to retrieve. What does this mean? Say for example, in your study
you interviewed 10 teachers (30 minutes each) on their opinion about the
leadership style of their principal. It is advisable that you give each teacher
a pseudonym (e.g. Elvis, Jagger, Dina ⁄ not their real names) or referred to
by a code number (e.g. T1, T2 ... T10). You need to keep a file that links the
pseudonyms or code numbers to the original informants which have to be
kept confidential and destroyed after completion of the research. Names
and other identifiable material should be removed from the transcripts.

The narrative data you obtained from the 10 teachers needs to be numbered
depending on your unit of analysis. In other words, you have to determine
whether you intend to analyse at the word level, sentence level or paragraph
level and they have to be numbered accordingly. Make sure that the unit of
text you use can be traced back to its original context.
You have at your disposal TWO approaches to analyse the data.
(i) If you are interested in conducting an exploratory study and are more
concerned with theory generation, than the grounded theory approach
should be your choice of analysis.
(ii) If you are interested in finding answers to pre-determined questions

(a priori questions) than framework analysis would be the logical
option.
SELF-CHECK 9.1
What do you mean by familiarisation, transcription and organisation in

the analysis of qualitative data?
(d) Coding
Coding is the process of examining the raw qualitative data in the
transcripts and extracting sections of text units (words, phrases, sentences
or paragraphs) and assigning different codes. This is done by marking
sections of the transcript and giving a numerical reference, symbol,
descriptive words or category words. Most of the text (or transcript) will be
marked and given different codes which will be later refined or combined
to form themes or categories. To help you with the practicalities of coding,
you could:
(i) Cut and Paste ă you can literally cut your transcripts into smaller
units of analysis which could be individual words, phrases, sentences
or paragraphs. You could paste these text units on to cards which you
could sort and re-sort easily. Keep in mind that each text unit needs to
be traceable to its original context. Sometimes, a text unit may have to
be sorted into two different categories or themes. So you will need
to make several copies of a text unit to be sorted into two or more
categories.

(ii) Colour code ă you could also use highlighting pens to highlight text
units or coloured pens to underline units of text. There could be a
problem when there are hundreds of text units and you will need
hundreds of colours which could pose a problem differentiating the
colours. The advantage of using coloured pens or highlighters is that
you do not need to cut up the transcripts. Colour coding would be the
choice if you do not have too many categories or text units.
(iii) Combination ă perhaps a preferred technique would be to use a

combination of cut and paste and colour coding.
(e) Analysis (Grounded Theory Approach)

Based on the research questions and your objective for conducting the
study, you determine the approach of analysing the data. If you are
interested in generating theory and not sure what to expect, the grounded
theory approach would be a logical choice. The grounded theory approach
offers a rigorous approach in generating theory from qualitative data. It is
particularly well suited for exploratory studies where little is known.
Grounded theory evolved from the work of sociologists Glaser and Strauss
(1967). Grounded theory is a method and approach in doing qualitative
research. It is an inductive method of qualitative research in which theory is
systematically generated from data. However, many studies in education,
business, management and in the health field (especially in nursing), have
adopted grounded theory as a procedure for conceptualising and analysing
data without taking on the whole methodology. The appeal of grounded
theory analysis is that it allows for the theory to „emerge‰ from the data
through a process of rigorous analysis (see Figure 9.1). The word „theory‰
is used to mean the relationships that exist among concepts that come from
the data and help us understand our social world more clearly (Strauss and
Corbin, 1998).
The main feature of the grounded theory procedure is the use of the
constant comparison technique. Using this technique, categories or concepts
that emerge from one stage of analysis are compared with categories
or concepts that emerge from the previous stage (see Figure 9.2). The
researcher continues with this technique until what is called „theoretical
saturation‰ is reached or no new significant categories or concepts emerge.
The grounded theory procedure is cyclical involving frequent revisiting of
data in the light of emergence of new categories or concepts as data analysis
progresses. The theory that develops is best seen as provisional until
proven by the data and validation from others.

Figure 9.2: The Grounded Theory Approach in qualitative data analysis
Strauss and Corbin (1998) suggest what is called open coding. Open coding
is where you „sweep‰ through the data, marking the text. It is a good idea
to leave a column at the side of your data so you can write your codes next
to the segments you are coding. LetÊs assume you are interested in how a
group of teachers view the behaviour of their principal in staff meetings.
Refer to an extract of an interview with a teacher and the key phrases
extracted as show in the right margin.
Extraction of key
phrases
R: How long have you been a teacher in this school?
T: For about 10 years.
R: Your principle, how would you describe him?
T: Quite a hot-tempered guy. hot-tempered
R. What do you mean hot-tempered?
T: Well, in the last staff meeting, I objected to his idea of cutting down the number
of fieldtrips for students. He argued that that it was too much of a responsibility
for the school. Also, it was getting more and more expensive for the school. lost his
R: What happened than? cool
T: Before I could say anything, he lost his cool and came for me. refused to
He refused to listen to what I had to say….he just went on and on. listen
R: What do you think? just went on
T: Personally, I think it was not fair of him to scold me. After all this is a democracy and on
and he should at least listen to what I had to say. It was very unpleasant and
many of my colleagues were very disturbed over the incident.
R: How do the others feel? not fair
T. Many of us prefer to keep quite and suffer in silence. You know, he is quite scold
close with the higher-ups. Anyone who questions his decisions are ridiculed
You know he determines whether we get promoted or not. You know, it’s the
usual thing!
R: How often does this happen? ridiculed
T: Almost always…..all meetings becomes a one man show …it’s all …talk….talk. for
You have uncovered eight descriptions of behaviour and the questioning
ll i d
You have uncovered eight descriptions of behaviour and the following

codes are assigned.
B1 ă hot tempered;
B2 ă lost his cool
B3 ă refused to listen
B4 ă just went on and on
B6 ă scolds
B7 ă ridiculed for questioning
B8 ă one man show
Next you may want to recode the eight descriptions into one or two
categories or themes. In other words, the category or theme emerges from
the data. You may have to assign a name for the category or theme. In this
example, B3 and B8 could be recoded to A1 and assigned the category or
theme „self-centred‰. You go on doing this until you have exhausted the
data in terms of developing any new codes.
(f) Analysis (Framework Analysis Approach)

Another approach to qualitative data analysis is called framework analysis
(Ritchie & Spencer, 1994). In contrast to the grounded theory procedure,
framework analysis was explicitly developed for applied research. In
applied research, the findings and recommendations of research need to be
obtained within a short period to be adopted. The general approach of
framework analysis shares many of the common features with the
grounded theory approach discussed earlier. This approach to qualitative
data analysis allows the researcher to set the categories and themes from
the beginning of the study. However, this approach also allows for
categories and themes that may emerge during data analysis which the
researcher had not stated at the beginning of the study.
Once the categories to themes have been pre-determined, specific pieces of

data are identified which correspond to the different themes or categories.
For a change, let us take an example form medicine. You may want to
know, for instance, about how people who had had a heart attack
conceptualise the causes of the attack. From existing literature, you may
know that these can be divided into physical causes, psychological causes,
ideas of luck, genetic inheritance and so forth. You interview people who
have had a heart attack and from the interview transcript you search the
data for material that could be coded under these headings.

Using the headings, you can create charts of your data so that you can
easily read across the whole dataset. Charts can be either thematic for each
theme or category across all respondents (cases) or by case for each
respondent across all themes:
(i) Thematic Chart

Theme Case 2 Case 3
„The stress at office is
Psychological „Business was bad. Had to
too much. Got to work
cause close shop.‰
late.‰
(ii) Case Chart

Theme 1 Theme 2
Genetic inheritance Physical cause
Case 1 „My younger brother and „I hardly do any exercise. I am
father died of a heart attack.‰ too busy to do any exercise.‰
In the chart boxes, you could put line and page references to relevant
passages in the interview transcript. You might also want to include some
text; e.g. key words or quotations as a reminder of what is being referred to
(see the charts above). For example, under the theme Psychological Causes,
Case 2 talked about „stress in the workplace‰ while Case 3 talked about
„business failure‰.
SELF-CHECK 9.2
What is the difference between the grounded theory approach and the
framework analysis approach in the analysis of qualitative data?
9.2 WRITING THE QUALITATIVE RESEARCH

REPORT
The steps involved in writing qualitative research reports are as follows:
(a) Introducing your Study
(i) Begin with something interesting, e.g. a quote or story, to capture the
readerÊs interest.

(ii) Introduce your question or curiosity. What is it that you want to

know or understand? How did you get interested in the topic?
(iii) Tell why thereÊs a need for the study. Cite relevant literature that
calls for the need for the research in this area, or demonstrate the
lack of attention to the topic. In your own words, describe how you
think this study will be useful.
(iv) Describe the intended audience for your research (e.g., the public,
family therapists).
(b) Research Method
(i) Identify and generally describe your research method (e.g.

ethnographic field study, single case study), and your research
procedures (e.g. long interviews, observation, etc).
(ii) Cite the major authors who have described your research method.
(iii) Explain how you selected your subjects and gained entry into the
research context (if relevant).
(iv) Describe the procedures you took to protect the rights of your
subjects (e.g. informed consent, human subjectÊs approval,
debriefing, etc).
(v) Describe the kind of relationship you had with the subjects. Will you
be neutral, collaborative or objective?
(vi) Describe the kind of data you collected (e.g. field notes from
memory, audio tapes, video tapes, transcripts of conversations,
examination of existing documents, etc).
(vii) Describe the procedures used in data collection. If interviews were

used, list your question(s) or attach as an appendix. Describe any
equipment used.
(viii) Describe the procedures you used to keep track of the research
process. E.g. your audit trail.
 Process notes: Day to day activities, methodological notes, or

decision making procedures.
 Materials relating to intentions and reactions: personal notes

about motivations, experiences with informants, etc.

 Instrument development information: revisions of interview

questions, etc.
(ix) Describe your data analysis procedures (coding, sorting, etc.)
 Data reduction: Write-ups of field notes, transcription

procedures and conventions, computer programs used, etc.
 Data reconstruction: development of categories, findings,

conclusions, connections to existing literature, integration of
concepts.
(x) Describe how you ensured "reliability" and "validity." Mention

whether you used triangulation, member checking, peer debriefing,
or auditing.
(xi) Summarise and reference all of the relevant literature that you have
reviewed.
(xii) Describe how you reviewed the literature and how it has influenced
the way you approached the research.
(xiii) Discuss how your previous experience with your topic has
influenced the way you have conceptualised this research.
(xiv) Summarise relevant personal and professional experiences, if you

have not done so in the Introduction.
Analysing qualitative data is not a simple or quick task. Done properly, it is

systematic and rigorous, and therefore labour-intensive and time-consuming.
The major element of qualitative analysis is to find, build, clarify, illustrate and
explain an argument or issue. The analysis should take the form of a research
essay containing certain expected elements: How you introduce them and
sequence the elements must be logical and help readers to „get it‰.
An adequate research report not only explains but also persuades. Being
persuasive is very much an issue of good clear writing. The way you write should
help readers to „see for themselves‰ what you claim to find in and make of the
data. The evidence is the data you collected and from which you choose carefully
an excerpt or excerpts to illustrate „points‰ in your report. It must be the right and
sufficient data to illustrate clearly and logically what is being claimed. Also, the
relevant evidence must be presented within a description that displays in narrative
form „the point‰ being made. Successful qualitative analysis tells a good,
absorbing, and understandable story. The story makes sense because you have
made an effort to make it so and you have communicated this to your reader.
The bottom line is credibility. It refers to the accuracy of your description as

shown in your report. It should be remembered that words are all you have to
describe phenomenon unlike quantitative research which uses numbers to
describe phenomenon. If you want to convince your reader that the findings you
obtained are credible (or accurate) you need to state precisely the parameters of
the study. What is meant by parameters? Parameters involve who was studied,
where and when, and methods used. If you are able to state these aspects clearly,
you enhance the credibility of the study.
Ć Qualitative data is a mass of words obtained from the recordings of

interviews, fieldnotes of observations, analysis of documents as well as
reflective notes of the researcher.
Ć Familiarisation is when you listen to tapes and watch video material, reading
and re-reading the field notes, making memos and summaries before formal
analysis begins.
Ć Transcription is the process of converting audio or video-recorded data

obtained from interviews and focus groups as well as handwritten fieldnotes
into verbatim form.
Ć After transcription, it is necessary to organise your data into sections that are
easy to retrieve.
Ć Coding is the process of examining the raw qualitative data in the transcripts
and extracting sections of text units (words, phrases, sentences or paragraphs)
and assigning different codes to them.
Ć Grounded theory is a method and approach in doing qualitative research. It

is an inductive method in which theory is systematically generated from data.
Ć The framework analysis approach allows the researcher to set the categories
and themes from the beginning of the study.

Case chart Grounded theory approach

Coding Organisation
Familiarisation Qualitative data
Framework analysis approach Thematic chart
1. When would you use the grounded theory approach instead of the
framework analysis approach when analysing qualitative data?
2. What are some of the elements you would include when writing a
qualitative research report?
3. Conduct a 20 minute observation of a classroom (primary or
secondary) and jot down in a notebook whatever you see. Analyse
the data using either the grounded theory approach or the
framework analysis approach.
Bernard, R. (1996). Qualitative data, quantitative analysis. Cultural Anthropology

Methods Journal, vol. 8 no. 1, 9ă11. Retrieve from:
http://web.missouri.edu/~anthgr/papers/Bernardqualquant.htm
Carney, J. & Joiner, J. (1997). Categorising, coding and manipulating qualitative

data using the WordPerfect word processor. The Qualitative Report, 3(2).
Retrieve from:
http://www.nova.edu/sss/QR/QR3-1/carney.html
Chapter 15: Qualitative data analysis. Retrieve from:

http://www.southalabama.edu/coe/bset/johnson/dr_johnson/lectures/l
ec17.pdf

Morgan. L. (n. d.). Module: Qualitative data analysis . Retrieve from:

http://comm2.fsu.edu/programs/commdis/ddseminar/QualitativeAnaly
sis.htm
Ratcliff, D. (n. d.). 15 methods of data analysis in qualitative research. Retrieve from:
http://www.vanguard.edu/uploadedFiles/faculty/dratcliff/qualresource
s/15methods.pdf

Topic  Writing the
10 Research
Proposal and
Research
Report
LEARNING OUTCOMES
1. Describe the elements of a research proposal;
2. Write a research proposal according to the proposed format; and
3. Explain the elements of a research report.
 INTRODUCTION
From Topic 1 to Topic 9, we discussed the methods
and techniques of conducting research, both from
the quantitative and qualitative perspective. This
topic will focus on the writing process and will
address two important tasks that most graduate
students will have to do. First, the writing of the
research proposal and second the writing of the
research report.

186  TOPIC 10 WRITING THE RESEARCH PROPOSAL AND RESEARCH REPORT
10.1 WHAT IS A RESEARCH PROPOSAL?

All graduate students pursuing a programme requiring the submission of a
project paper, thesis or dissertation will have to write a research proposal. A
research proposal is a short document written to inform others (your supervisor
or graduate committee) of a proposed piece of research. You should be aware
that a research proposal can be rejected as „unsuitable‰ or „poorly designed‰. So
the proposal is obviously an important document. Therefore it is important that
you spend some time getting it right. A well planned proposal will save you a lot
of time in the long run. If the proposal is well-designed, it will form the outline of
your project paper, thesis or dissertation that you can follow. In other words,
the proposal maps out the different parts of the final project paper, thesis or
dissertation. The elements that are usually included in any research proposal are
shown in Figure 10.1.
Chapter 1 Introduction
1.9 Background to the study
1.10 Problem statement
1.11 Objectives of the study
1.12 Research questions
1.13 Research hypotheses (if any)
1.14 Significance of the study
1.15 Limitations of the study
1.16 Definitions of terms
Chapter 2 Review of Literature
2.1 Previous studies
2.2 Theoretical framework
2.3 Methodological issues
Chapter 3 Methodology
3.1 Sample
3.4 Instrumentation
3.5 Data collection procedures
3.6 Proposed framework for data
analysis
References
Appendices
Note: The headings in each chapter may vary in
different studies.
Figure 10.1: Format for the research proposal

TOPIC 10 WRITING THE RESEARCH PROPOSAL AND RESEARCH REPORT  187
(a) Introduction
(i) This is where you provide an introduction or background to the

research problem or issue that you intend to study. This should be as
brief as possible (1 page). Do not ramble! The introduction should be
clear and straight to the point. Describe the general field of research
and than narrow down to the specific area you are concerned with.
(ii) Show that there is an issue that needs to be addressed or a „gap‰ in

the research that you will fill. When you are able to identify the issue
or gap, then the research question will fall in place naturally. Think of
the Introduction as follows:
Imagine a group of academics discussing in general the area of study

you are interested in. You join in the conversation (assuming they allow
you to!) and draw their attention to your specific problem of interest.
You tell them that there is something that has not been resolved or there
is a gap or problem. You argue that this gap or problem has to be
addressed and go on to describe it in detail. Than you tell your listeners
how your study will attempt to answer the research question.
(iii) Remember, the problem statement may only be tentative at this stage
as the research has not been carried out yet. It is not expected in a
proposal that you have an answer to your research question. It helps
if you have a tentative answer, however. A hypothesis is useful for
this purpose, though this might only be necessary for more empirical
subjects.
(iv) You should use simple and jargon-free language. The introduction
must actually narrow down; not get wider. You must demonstrate
that you understand well the issues in the area and that you are
focussing on a particular issue.
(b) The Research Question
(i) The research question may not necessarily be a „question‰ as such,

but can be a statement of a problem to be investigated.
(ii) It can be phrased in the form of a question or formal hypothesis. Refer

to Topic 1: The Research Process for more details about „The Research
Question‰.

(iii) State the Research Question clearly as it will influence research

methodology and the type of data analysis to be performed.
(c) Significance of the proposed research (Justification)
(i) Briefly tell the reader the significance of the study (justify doing the
study).
(ii) You can argue the significance of your study based on the following
criteria:
 The problem or gap demands attention because the findings could

influence practice and policy.
 The methodology you are using is unusual.
 You are studying certain variables that have not been given
attention in previous studies.
 Your study will contribute to the body of knowledge in the field.
 The outcome could be the extension of a theoretical model.
(d) Preliminary Literature Review
(i) This is where you provide more details on what others have done in
the area, and what you propose to do. Refer to Topic 2: Theory and
Review of Literature for more details
(ii) You need to cover the following:
 The major issues or schools of thought.
 The gaps in the literature (in more detail than that provided in the
introduction).
 Research questions (for qualitative research and hypotheses (for

quantitative research) which are connected carefully to the literature
being reviewed.

 Definition of key terms (this can be done when you introduce each
idea, or in a definition sub-section).
 Questions arising from the gaps that can be the focus of data
collection or analysis.
(iii) The theoretical framework usually forms the final part of the Literature
Review section. It describes the model that you are using in the thesis
to demonstrate your point.
(iv) Read a thesis in a similar area to get a feel for what is required in this
section.
(e) Proposed Research Methodology
(i) You do not have to describe the methodology used in great detail (this
will be done in the thesis) but you should justify its use over other
similar methodologies.
(ii) For example, you could explain:
 Why you are using a certain paradigm or theory.
 Why you are using qualitative or quantitative research.
 Why you are using a case study of a specific kind.
 Why you are using surveys, correlational experiments, field

studies, specific statistical measurements, etc.
 Why you are using a certain dependent or independent or

moderating variables.
 Why you have chosen a sampling frame and the size of a certain
sample.
 How you are proposing to have access to the data.
 How you are proposing to analyse the data.
(iii) You also need to provide operational (testable or at least well-

supported in the literature) definitions of key terms used.

(f) Project Timetable
(i) Provide a project timetable specifying how long you will take to
complete the project paper, thesis or dissertation. For example,
indicate how long you will take to collect data, analyse the data and
write up the final report.
(ii) It gives you a framework on the direction your proposed thesis

will take. It shows the reader that the project is well-organised and
achievable in the time available.
(g) List of References
(i) This must be provided in the usual scholarly fashion. It helps to

convince your reader that your proposal is worth pursuing if you can
identify literature in the field and demonstrate that you understand it.
(ii) Use the citation style proposed by the Manual of the American
Psychological Association (APA style for short).
10.2 RELATIONSHIP BETWEEN THE RESEARCH

PROPOSAL AND THE FINAL RESEARCH
REPORT
Note finally that while the proposal can be mapped onto the final thesis, much
work needs to be done. The proposal merely provides a „shell‰. The thesis fills in
the details. Parts of the proposal are not required in a final thesis (for example,
resources, and timetable). The order and arrangement of each document is
slightly different.
10.3 THE RESEARCH REPORT

The purpose of this section is to give you a general guide for preparing the
project paper, thesis or the dissertation. The format discussed may vary with
respect to different institutions. See Figure 10.2 which shows widely used
formats for the research report.

Format of the Research Report

Preliminary Materials
TITLE PAGE
Acknowledgements
Table of contents
List of tables (if any)
List of figures (if any)
Body of the Report

Chapter 1 Introduction
1.1 Background to the study
1.2 Problem statement
1.3 Objectives of the study
1.4 Research questions
1.5 Research hypotheses (if any)
1.6 Significance of the study
1.7 Limitations of the study
1.8 Definitions of terms
Chapter 2 Review of Literature
2.1 Previous studies
2.2 Theoretical framework
2.3 Methodological issues
Chapter 3 Methodology
3.1 Sample
3.1 Instrumentation
3.2 Data collection procedures
3.3 Framework for data analysis
Chapter 4 Data Analysis and Results
(Description of statistical analyses in relation to research
questions/hypotheses/objectives and presentation of relevant tables
and figures)
Chapter 5 Discussion and Conclusion
5.1 Summary of main findings
5.2 Discussion
5.3 Implications
5.4 Directions for future research
Supporting Materials
References
Note: The headings in each chapter may vary in different studies.
Figure 10.2: Format of a Research Report

(a) Preliminary Materials

The preliminary materials include the title page, the abstract,
acknowledgements, table of content, list of tables and list of figures.
(i) Title Page

Although title page may differ from one institution to another, they
usually include: (1) the name of the topic, (2) the name of the author,
(3) the relationship of the report to a degree requirement, (4) the name
of the institution where it is to be submitted, and (5) the date of
presentation (see Figure 10.3). The title should be concise and should
indicate clearly the purposes of the study. Keep in mind its possible
usefulness to the reader who may research the database in which it
may be listed. The title should not claim more than the study actually
delivers. It should not to be stated broadly and make it difficult for the
reader to pin point what the study is about.
For example, the title „The Self-Concepts of Urban Poor Children‰ is

too general and a more precise title would be „The Self-Concepts of a
Group of Urban Poor Children in Kuala Lumpur‰.
The title should be in capital letters, single-spaced and centred

between the right and left margins of the page. If the title goes beyond
one line, the words in the title should be divided into lines so that
each successive line is shorter than the one above it and is centred
below it in an inverted pyramid style (see Figure 10.3).
MATHEMATICS ACHIEVEMENT OF POOR CHILDREN

IN AN URBAN SCHOOL IN SARAWAK
EVY SOFIAH ISMAIL
A dissertation submitted in partial fulfilment

of the requirement for the degree of
Master of Education
Faculty of Education and Languages

2008
Figure 10.3: Example of a title page

(ii) Acknowledgement
An acknowledgment page is included if you have received unusual
assistance in the conduct of the study. The acknowledgement should
be simple and restrained. Do not indulge in flattery and excessive
recognition for routine participation of family members, lecturers,
supervisors, librarians and clerical helpers.
(iii) Table of Contents

A table of contents serves an important purpose in providing an
outline of the contents of the report. Differentiate between headings
and subheadings using capitalisation and small letters. Page references
for each topic should be indicated.
(iv) List of Tables and Figures

If tables and figures are included in the report, a separate section
should be included to list each table or figure. The full titles of figures
and tables, worded exactly as they appear in the text, are presented
with corresponding numbers and page locations.
Note: All pages in the preliminary section are numbered at the centre of the
bottom margin with lower-case Roman numerals (i, ii, iii, iv).
(b) Body of the Report

It is usual for this section to be divided into five sections or topics.
(i) Topic 1: INTRODUCTION

The first topic serves as the introduction to the area under consideration.
A clear statement of the problem with specific questions to be answered
or hypothesis to be tested is presented. You should present the
significance of the problem and its historical background appropriately.
Also, include assumptions and limitations of the study. All important
terms that are operationally defined should be included in this topic.
This is important because terms such as gifted, underachiever and
many other terms are defined differently by different researchers.
(ii) Topic 2: REVIEW OF RELATED LITERATURE

This topic is a review of important literature related to your study.
Extracts from previous research studies and significant writings of
authorities in the area studied are reviewed. This topic provides a
background for the development of your study and brings the reader
up to date about research and thinking in the field. It also gives
evidence of your knowledge of the field. You should avoid an article-
by-article presentation but should indicate areas of agreement or

disagreement in findings or gaps in existing knowledge. The journal

Review of Educational Research can be referred to for examples of
good critical reviews of the literature. Also, avoid excessive use of
quotations. Nothing is more tiresome or difficult to follow than a
review of literature that is merely an accumulation of quotations.
(iii) Topic 3: METHODOLOGY

This topic explains the design of the study in detail.
Ć Sampling: It is here that you explain the size of the samples and
how you selected them. Indicate the extent to which the sample is
representative of the population. Did you use random sampling?
Did you use stratified sampling?
Ć Setting: If you are doing a qualitative study, you have to explain in

detail the setting, the characteristics of your subjects, how you
gained access or entry to the setting and your role in the study
(e.g. observer, participant observer).
Ć Instrumentation: You should include a description of the data

collection techniques or instruments you used. For example, if
your study is a survey you have to explain how you designed and
developed the questionnaire or interview checklist. Explain the
number and types of items included in the questionnaire. If you
had used attitude scales, achievement tests and other psychological
tests; you have to give evidence regarding the reliability and
validity of the instruments. You may also describe the scoring
procedures adopted for the instruments used.
(iv) Topic 4: ANALYSIS OF DATA

In this topic, you present the findings of the study after having
processed and analysed the data. This is the heart of the research
report. If you are doing a qualitative study, tables and graphs are
commonly used to organise and present numerical data. Tables and
graphs are useful in presenting an overall picture of the data as
well as showing trends that have emerged from the analysis. If you
did a qualitative study, there would be less numerical data. Instead
you data would consist of concepts, categories or themes which
may be presented in table form. You would also be presenting data
in the form of anecdotes or excerpts of interviews, observations
and documents to support your arguments. You are advised to refer
to the Journal of Educational Psychology and American Educational
Research Journal to see how tables and graphs are presented and

explained. For qualitative studies, The Qualitative Report is a useful

journal which presents reports of qualitative studies in education,
nursing and medicine.
(v) Topic 5: DISCUSSION AND CONCLUSION

The last topic comprises two parts. The first part includes a brief
summary of the problem, the methodology and results. Focus should
be on a summary of the findings and it should be as brief as possible.
Some researchers present the main findings in the form of a list. The
second part is a discussion of the findings. Here, you identify and
interpret the findings. You give possible reasons why the results
occurred. You could provide reasons by referring to the findings of
previous research (This is where the studies cited in Topic 2 are
useful). Because you are the one who conducted the study, you
should have a deeper understanding compared to your readers
whom you are expected to discuss your findings and opinions with.
One of the most common weaknesses found in the writing of graduate
students is that their reports present important and interesting
findings but fail to provide a thoughtful interpretation of the findings.
On the other hand, there is the tendency for beginning researchers
to overgeneralise on the basis of their limited data. Remember, your
study is not supposed to be an attempt to change the whole education
system!
You should keep in mind that this topic is the most used part of the
research report by other readers. Readers who scan research literature
to find significant studies examine this topic before deciding whether
or not further examination of the report is worthwhile.
(c) Reference Materials

This section of the report comprises the References and Appendices (if any).
References are arranged in alphabetical order with the last name of the
author listed first. Here you would include journal articles, books, topics in
books, monographs, reports, newspaper articles, etc, you have cited in the
report. The common mistakes with the Reference section are:
(i) You had cited an author in the report but it is not listed in the
References and vice-versa.
(ii) The method of referencing does not follow a consistent format

(In education, the format used is by the American Psychological
Association ă APA format).

The appendix is indicated by the word APPENDIX, capitalised and centred

on the page. The first page of the appendix is entitled APPENDIX A
followed by APPENDIX B and so forth. What may be included in the
appendix?
(i) Tables and data ă important, but not essential to the understanding of
the report.
(ii) Copies of cover letters used, and printed forms of the questionnaire,
tests and other dataăgathering devices.
(iii) Item-analysis data and other materials pertinent to measures.
(iv) Scoring procedures.

MODULE FEEDBACK
MAKLUM BALAS MODUL
If you have any comment or feedback, you are welcome to:
1. E-mail your comment or feedback to modulefeedback@oum.edu.my
OR
2. Fill in the Print Module online evaluation form available on myVLE.
Thank you.
Centre for Instructional Design and Technology

(Pusat Reka Bentuk Pengajaran dan Teknologi)
Tel No.: 03-27732578
Fax No.: 03-26978702

HMEF5014 Eduacational Research Methodology

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

HMEF5014 Eduacational Research Methodology

Uploaded by

Copyright:

Available Formats

HMEF5014

Copyright © Open University Malaysia (OUM)

Module Writer: Prof Dr John Arul Phillips

Moderator: Assoc Prof Dr Chung Han Tek

Developed by: Centre for Instructional Design and Technology

Printed by: Meteor Doc. Sdn. Bhd.

First Edition, January 2007

Copyright © Open University Malaysia (OUM)

Topic 1 The Educational Research Process 1

Topic 2 Theory and Review of Literature 21

Copyright © Open University Malaysia (OUM)

Topic 3 Experimental Methodology 39

Topic 4 Experimental Research Designs 57

Copyright © Open University Malaysia (OUM)

Topic 5 Survey Research Methodology 77

Topic 6 Instrumentation and Data Analysis 102

Copyright © Open University Malaysia (OUM)

Topic 7 Qualitative Research Methods 128

Topic 8 Qualitative Data Collection Methods 151

Copyright © Open University Malaysia (OUM)

Topic 9 Qualitative Data Analysis 171

Topic 10 Writing the Research Proposal and Research Report 185

Copyright © Open University Malaysia (OUM)

Copyright © Open University Malaysia (OUM)

Copyright © Open University Malaysia (OUM)

Copyright © Open University Malaysia (OUM)

WHAT WILL YOU GET FROM DOING THIS

The qualitative and quantitative approaches are compared in terms of objectives,

Aim of the Course

Copyright © Open University Malaysia (OUM)

Objectives of the Course

1. Identify the characteristics of educational research;

2. Compare between quantitative and qualitative methods in educational research;

3. Discuss the differences between quantitative and qualitative methods of data

4. Critique research in education and write a review of literature; and

5. Demonstrate an ability to write a research proposal.

HOW CAN YOU GET THE MOST FROM THIS

1. The Course Guide you are currently reading;

2. The Course Content (consisting of 10 topics); and

3. The Course Assessment Guide (which describes the assignments to be

Please ensure that you have all of these materials.

Copyright © Open University Malaysia (OUM)

Topic 3 introduces the concept of the experiment in educational research,

Topic 6 is devoted towards explaining the design and development of

Topic 7 introduces qualitative approaches in educational research and compares

Topic 8 focuses on common data collection methods adopted in qualitative

Topic 9 examines the techniques of analysing qualitative data, focussing on

Topic 10 outlines the sections of a research proposal and suggestions on how to

Organisation of Course Content

Copyright © Open University Malaysia (OUM)

Each topic contains the following features:

Questions are interspersed at strategic points in the topic to encourage

Copyright © Open University Malaysia (OUM)

These are situations drawn from research projects to show how

Copyright © Open University Malaysia (OUM)

WHAT SUPPORT WILL YOU GET IN STUDYING THIS

myVLE Online Discussion

Copyright © Open University Malaysia (OUM)

HOW SHOULD YOU STUDY FOR THIS COURSE?

2. Proposed Study Strategy

(d) To begin reading a topic:

Copyright © Open University Malaysia (OUM)

 Read the introduction (see how it is related to the previous topic).

Copyright © Open University Malaysia (OUM)