Research methods in

mathematics and science

education
 What is educational research?
 “…the distinctive focus of educational research must be upon the quality of
learning and thereby of teaching.” (Pring, 2000: 27).
 “Central to educational research… is the attempt to make sense of the
activities, policies and institutions which, through the organisation of
learning, help to transform the capacities of people to live a fuller and more
distinctively human life.” (Pring, 2000: 17).
Pring, R. (2000). The focus of educational research - educational practice and
policy. In G. Thomas and R. Pring (Eds.), Philosophy of educational research. London:
Continuum

 So, how do we do educational research?

 To understand this, I think we must understand the carpenter.

So. imagine becoming a carpenter
 Imagine that you have seen someone using a hammer.
 You like the rhythm, the sound, and the fact that people with hammers tend to
build things quickly. Everything about a hammer feels right!
 So you decide to learn how to use a hammer and you become an expert on
hammering, maybe even the world’s best hammerer.
 Now, imagine you want to make a table.
 What sort of table can you make if all you have is a hammer?
 To become a carpenter
 you must learn to use a range of tools
 You must understand what each tool does

 Only then will you be able to make a beautiful and functional table.
 Hammer

No Hammer

However, even if I know what I want to make,

the quality of the output always depends on
my using the right tools properly.
The aim of this course is to help you not only
find the right tools but to use them properly
Talking of tools…
Speaking of
tools
Becoming a specialist
You can’t make this with a hammer

Grinling Gibbons: Wren Library, Trinity College, Cambridge, 1600s

And you can’t make this with a sculptor’s chisels

Cressing Temple barn, Essex, England, 1200s

So, when designing your research
 Researchers should consider the research question to be more important than
either the method they use or the paradigm that underlies the method (Teddlie
& Tashakkori, 2003, p.20)
 The selection of methods should not be made on the basis of sympathies
toward a certain methodological camp or school. Methods are tools for the
answering of research questions and not vice versa (Erzberger & Kelle, 2003,
p.482)
 In other words, don’t start with the tools, start with what you want to make.
No one ever became a carpenter just because they wanted to use a hammer!
 Then decide which are the best tools for the job; and to do this you need to
know what tools are available to you. This leads me to some name-dropping
In other words…
 I don’t usually name-drop but this talk warrants it
 I was having dinner with Paul Cobb in Stockholm last summer.
 We were talking about methodological problems in mathematics
education research.
 He commented that we should “be wary of the ideologues”
 That is, don’t trust those who say, even before you discuss your
research question, “Oh, you must use Dewey, Bloom, Foucault,
Bourdieu, Engeström, Marton, Bernstein or who ever…”
 Any of them may be relevant, but not until you have decided on
your research question and then asked, “what tools do I need now?”
Goldfinch or steglits
A good research question is a finch
 Focused: It should be well-defined and clearly manageable.
 Informed: It should be located in a deep understanding of the research
literature, both locally and internationally.
 Novel: It should be new to you and, hopefully, others.
 Contextualised: It should be located within an appropriate context;
individual, classroom, school, community, system and professionally
relevant.
 Hooked: The question must hook you like an angler hooks a fish. There is
absolutely no point working on a project that does not excite you

 Once you have narrowed down your research question, then you can start
thinking about the tools you intend to use and how you will use them
So, what tools are available?
 There are a number of tools, many of which are familiar to you.
 Interviews
 Tests
 Observations
 Questionnaires
 Reflective journals
 Autobiographies
 This makes it all look very simple, except each tool can be used in many
different ways that are dependent on the aims embedded in the research
questions.
 For example, interviews can be open, closed, structured, semi-structured,
unstructured, individual, group, focus group, elite, life history, clinical,
cognitive and many others.
 This leads us to…
Forms of research (understanding genomgång)
● Exploratory research aims to identify new phenomena. It is not about testing hypotheses
but focuses on questions that will “open your eyes and broaden your vision”. What is this
thing called genomgång?
● Descriptive research identifies and classifies the characteristics of the subject under
scrutiny. How is genomgång enacted?
● Explanatory research aims to provide explanations to phenomena already identified.
Where does genomgång come from?
● Emancipatory research examines the politics of the possible by confronting social
oppression at whatever levels it occurs. Who does genomgång include or exclude?
● Confirmatory research confirms previously identified outcomes. This could be a
replication study to help create a critical mass of findings to influence policy or a testing of
a previously developed hypothesis. Does genomgång exist everywhere?
● Predictive research is based on close analysis of available evidence of cause and effect, it
aims to speculate intelligently on future possibilities. Is genomgång effective and are there
more effective alternatives?
● All these make different demands on the toolkit available to us.
This course
Experimental research (typical case)
 Researchers believe that a particular way of teaching about force will improve
students’ learning
 They design a series of lessons to be taught to an experiemental group, which will
be taught in parallel with a control group who will not get the novel teaching.
 Students in both groups will typically be tested before, after and again much later
 Both groups need to be matched on as many characteristics as possible
 The main focus is statistical and researchers are interested in demonstrating a
statistically significant improvement in the experimental group’s performance over
time. In this sense, it is confirmatory in its framing.
 The design of the experimental teaching and the assessment measures is
complicated and takes much time.
 Knowledge derives from the quality of the tools and the statistics. Little interest is
shown in individual students.
Phenomongraphic enquiry (typical case)
 Researchers are interested, for example, in differences in the ways that students
and teachers perceive the same classroom phenomenon
 Researchers video record the phenomenon in a range of different classroom
settings and undertake video stimulated recall interviews with participants
 The analysis, acknowledging that there is no objective reality, is usually
unstructured by other’s frameworks and comparative in nature.
 The end result is typically descriptive but may lead to theory generation

 It focuses on the individual experience of the phenomenon and is uninterested

in statistical generalities. In this sense it is exploratory in its framing.
 Here, knowledge derives from the experiences of the individuals under
scrutiny and may or may not, depending on the researcher’s goals, exploit
external frameworks.
Ethnography (typical case)
 Here researchers are often concerned with identifying how people become
members of a particular community
 For example, how do teachers help students to beome safety aware in
laboratories or mathematical reasoners?
 Researchers would typically follow a small number of cases over a period
of time. Data derive from observations, interviews, students’ written work,
teachers’ lesson plans and any data relevant to the goals of the study.
 Researchers may even become participant observers.
 The use of external frameworks for analysis will depend on the
researcher’s goals and whether they are thought to be appropriate.
 While attention is paid to the individual, the goal is theory.
 Ehtnographic research can be exploratory, descriptive, emancipatory
Survey research (in slightly more depth)
 Researchers are interested in a quick form of data collection that yields
statistically valid and generalisable outcomes (However, designing the
questionnaire is not a quick step).
 Although not exclusively so (I have conducted a semi-structured interview
survey*), surveys typically draw on questionnaires.
 These are made available to a representative sample of the target population by
various means – post, internet, telephone and so on.
 Questionnaire format is important with simple questions first and harder
questions last – if you want to ask open questions, they must come last.
 The primary aim is statistically rigorous and generalisable results.
 The typical questionnaire has a psychological underpinning and this leads to a
major problem of questionnaire design
*Andrews, P. (2007). The curricular importance of mathematics: A comparison of English and
Hungarian teachers' espoused beliefs. Journal of Curriculum Studies, 39 (3), 317-338.
Questionnaire design
 Typically, researchers are not interested in participants’ responses to single
items.
 They are interested in constructs like self-efficacy or self-esteem that cannot be
evaluated directly.
 Typically each construct is attacked by means of several statements, each set
against a four or five point scale from strongly agree to strongly disagree.
 Each statement must be simple and address one and only one issue. For
example, a statement like, I work hard in mathematics because it will help me get a
better job and make more money, is a poor statement.
 Is it about getting a better job or more money? They are different things and
should be reflected in two statements.
 Researchers, having identified a series of statements for each construct, must
then pilot the questionnaire, analyse the results and make necessary changes.
Questionnaire example
 In a recent study Jose Diego Mantecón and I devised a questionnaire for
studying England and Spanish 13 years olds’ beliefs about school
mathematics.
 One construct of the twelve we examined was mathematical anxiety, which
we addressed through four statements. Each has one cause and one effect.
 In each case the cause (doing mathematics) is the same, but sometimes it
comes first and sometimes it comes second
 I become nervous when I do maths
 When I do maths I become tense
 I get stressed easily when I do maths
 Doing maths makes me uneasy
 These were distributed among the other items of the questionnaire to give,
in our case, 48 items.
However,
 It took us four iterations to get a decent questionnaire and
 Because we were developing the instrument for cross cultural use we had guarantee
several equivalences
 Conceptual: Are the concepts the same – house warming party
 Linguistic: Are there words in both languages
 Measurement; does the issue have the same degree of salience
 Sampling: Are we confident that comparable groups are involved
 Cognitive interviews were used to identify problematic areas; the word problem
itself was problematic, meaning different things to Spanish students from the
English
 So if you intend to translate a questionnaire, be warned. It is not as simple as just
translating the words
Andrews, P., & Diego-Mantecón, J. (2015). Instrument adaptation in cross-cultural studies of students’
mathematics-related beliefs: Learning from healthcare research. Compare: A Journal of Comparative and International
Education, 45(4), 545-567.
Approaches to statistical analysis
 Reliability analyses confirm that were the questionnaire to be used with a
different sample the same results would be obtained. Poor items are removed.
 Factor analyses confirm whether or not the items actually attack (load on) the
intended constructs. Poor items are removed from the analysis
 Factor scores are calculated from the remaining items and it is on these that the
main analysis will be undertaken.
 Factor scores will be compared for different groups to see how the different
constructs play out.
 Factors scores will also be examined for their impact on other factor scores
and so on. This will involve analyses of variance, tests of difference and so on.
 In short, analysing questionnaire data properly requires an understanding of
some fairly sophisticated statistical techniques.
 In other words, if you do not know before you collect your data how you will
analyses those data, you should not be collecting it.
Finally
 Over the coming months you will be exposed to many words. Some of these will be
familiar and others will not.
 Of all these words, four will become constants. These are ontology, epistemology,
methodology and method.
 Whenever you undertake research your questions should guide you to the following
 What is the nature of the knowledge you seek (ontology)
 How is that knowledge generated (epistemology)
 What principles guide data collection and analysis (methodology) and
 What tools will be used in data collection (methods)
 Each of the above paradigms (another word for you), whether experimental, survey,
phenomenological, ethnographic or pragmatic, has its own ontology, epistemology,
methodology and methods.
 These may not be unique. It is perfectly feasible to use interviews in survey,
phenomenological and ethnographical research. But the goals for doing so will be
different.
A provocative question
 Every theoretical framework linked to a particular person derives from a culturally
unique set of events. For example
 Many of philosopher Foucault’s ideas stem from his work in French prisons and his
desire to understand institutional power.
 Much of sociologist Bernstein’s work derives from his Marxist desire to understand how
social class constructed educational opportunities for English children, typically through
the medium of language.
 Much of sociologist Bourdieu’s work focused on the reproduction of culture and the
ways in which various groups are implicated in both inclusion and exclusion.
 Engeström’s perspectives on activity theory draw extensively on Russian scholars like
Leont’ev and Vygotsky. It is a descriptive rather than predictive theory.
 Marton’s variation theory derives from his studies of Chinese classrooms, mathematics
in particular, and how subtle variation of a problem underpins new knowledge.
 Acknowledging the cultural nature of such theories, can you be confident that any of
them has an explicit relevance to your particular cultural context? This is particularly the
case in light of Weber’s fascinating analysis of the functioning of Protestant and
Catholic cultures in Europé.
Good luck