Chris Butler Research methods in the study of language: an introduction

with particular reference to statistical techniques

Course summary
Session 1, Monday 5 September

Topic 1: Introduction to types of research methodology in

linguistics

• Qualitative methods
o Characteristics of qualitative research methods
o Types of qualitative research method
o An example: Linguistic Ethnography and Interactional Sociolinguistics
in the study of identity
• Quantitative methods
o Characteristics of quantitative research methods
o An example of use of quantitative methods: the role of phonological
decoding in the inference of word meaning in a second language
• Mixed methods

Topic 2: Why do (some) language researchers need to know

something about statistics?

• Presented as answers to a series of questions, including:

o Why do linguists need statistics at all?
o Do all linguists need statistics?
o Surely statistics is associated with formal, 'scientific' kinds of language
study?
o So what kinds of things can I do using statistics?
o What is meant by 'data', and what are 'variables'?
o What is meant by the ‘level of measurement’ of a variable?
o What is a hypothesis and how can I test it?
o Are statistics and research design closely linked?
o What should I do to find out more about statistics in relation to language
studies?

SdC, IWoDA’11, University of Santiago de Compostela, 5-6 September 2011 1

Chris Butler Research methods in the study of language: an introduction
with particular reference to statistical techniques

Topic 3: Descriptive statistics

[All aspects will be illustrated with concrete examples, analysed by means of SPSS.]
• Frequency distributions
o Frequency tables
o Graphs: histograms, frequency polygons, bar charts
o The importance of the ‘normal’ distribution

• Measures of ‘average’ value

o Mean
o Median
o Mode

• Measures of variability
o Mean deviation
o Standard deviation
o Interquartile range

Topic 4: Hypothesis testing: fundamental concepts

• Why statistical tests are (often) necessary to test a hypothesis.

• The stages in carrying out a hypothesis test:

o Stating the null and alternative hypotheses
o Selecting an appropriate test of the null hypothesis
o Carrying out the test and producing the value of the test statistic
o Determining the significance level associated with this value of the test
statistic.

• The factors involved in choosing an appropriate test:

o The level of measurement of the data (truly quantitative, ordinal,
nominal): the difference between parametric and non-parametric tests
o The characteristics of the frequency distribution of the data
o The type of research design (independent design vs repeated measures
or matched pairs designs)
o The number of groups you want to compare.

• The concept of a test statistic and the associated significance levels.

• Effect sizes.

SdC, IWoDA’11, University of Santiago de Compostela, 5-6 September 2011 2

Chris Butler Research methods in the study of language: an introduction
with particular reference to statistical techniques

Session 2, Tuesday 6 September

Topic 5: Parametric and non-parametric tests for comparison of

‘averages’ of two groups: summary and illustrative tests

• Conditions for use of parametric and non-parametric tests.

• Tests for differences between the ‘averages’ of two groups

o Table showing tests for different levels of measurement and types of
research design.
o An example of a parametric test (the t-test for independent groups), as
carried out using SPSS.
o An example of a non-parametric test (the Mann-Whitney test), as carried
out using SPSS.

Topic 6: (1) Summary of parametric and non-parametric tests for

comparison of ‘averages’ of more than two groups. (2)
Correlation

• Tests for differences between the ‘averages’ of more than two groups
o Table showing the tests available and their requirements

• Correlation
o What is correlation?
o What tests are available for assessing the significance of correlations?
o Example of a Pearson and Spearman correlation tests, as carried out
using SPSS.

Topic 7: A test for nominal variables: the chi-square test

• Contingency tables
• The chi square test.
o An example of a chi square test, as carried out using SPSS.

SdC, IWoDA’11, University of Santiago de Compostela, 5-6 September 2011 3

Chris Butler Research methods in the study of language: an introduction
with particular reference to statistical techniques

Topic 8: An introduction to selected exploratory multivariate

methods

• Techniques which can be used for isolating patterns in data sets which have
been classified according to a number of different variables
o Factor analysis
o Principal components analysis
o Cluster analysis
o Multidimensional scaling
• Examples of cluster analysis and multidimensional scaling, as carried out using
SPSS, will be shown.

Some useful references

Paltridge, Brian and Aek Phakiti (eds.) (2010) Continuum Companion to Research
Methods in Applied Linguistics. London: Continuum.
[Covers both qualitative and quantitative methods. Introduces various research
methods and approaches: experimental research, survey research, analysing
quantitative data, ethnography, case studies, action research, analysing qualitative data,
research syntheses, critical research in applied linguistics. Also looks at various areas in
depth: speaking, listening, reading, writing, grammar, vocabulary, pragmatics,
discourse, language classrooms, language testing, moribation, language and gender,
language identity.]

Heigham, Juanita and Robert A. Croker (2009) Qualitative Research in Applied

Linguistics: A Practical Introduction. Houndmills, UK: Palgrave Macmillan.
[Introduces core qualitative research approaches and frequently used data collection
methods, with examples from applied linguistics research. Also chapters on the
qualitative research cycle, ethics, writing up research.]

Litosselilti, Lia (ed.) (2010) Research Methods in Lingusitics. London: Continuum.

[Chapters on research questions in linguistics, combining qualitative and quantitative
methods, organising and processing data, corpus methods in linguistics, discourse
analytic approaches to text and talk, interviews and focus groups, multimodal
analysis, narrative analysis.]

Dörnyei, Zoltán (2007) Research Methods in Applied Linguistics: Quantitative,

Qualitative and Mixed Methodologies.
[Describes the various stages of qualitative and quantitative studies, from data
collection to reporting the results. Also include discussion of combinations of the two
methodologies.]

SdC, IWoDA’11, University of Santiago de Compostela, 5-6 September 2011 4

Chris Butler Research methods in the study of language: an introduction
with particular reference to statistical techniques

Perry, Fred L., Jr. (2005) Research in Applied Linguistics: Becoming a Discerning
Consumer. Mahwah, NJ: Lawrence Erlbaum.
[Provides an introduction to the roundations of research methods and discusses
matters concerned with understanding and using published research.]

Butler, C. S. (1985) Statistics in Linguistics. Oxford: Blackwell.

[Covers basic techniques only; explains the maths on the way; many examples. Now
out of print, but available for free download via the internet at
http://www.uwe.ac.uk/hlss/llas/statistics-in-linguistics/bkindex.shtml]

Hatch, E. & A. Lazaraton (1991) The Research Manual: Design and Statistics for Applied
Linguistics. Boston: Heinle & Heinle.
[Fairly high level treatment, with many useful examples.]

Oakes, M. P. (1998) Statistics for Corpus Linguistics. Edinburgh Textbooks in

Linguistics. Edinburgh: Edinburgh University Press.
[One chapter covers basic techniques rather briefly. Good discussion of multivariate
methods. Also deals with other statistically-related procedures relevant to corpus
linguistics, such as word tagging.]

Woods, A., P. Fletcher & A. Hughes (1986) Statistics in Language Studies. Cambridge:
Cambridge University Press.
[A more comprehensive treatment than Butler 1985, and at a considerably higher
level, so correspondingly more difficult to read. Covers multivariate techniques,
regression, etc in some detail. Very good examples.]

Johnson, K. (2008) Quantitative Methods in Linguistics. Oxford, Malden MA (USA) and

Carlton, Victoria (Australia): Blackwell Publishing.
[A fairly advanced treatment of a wide range of statistical techniques for the study of
language, with statistical routines in the programming environment ‘R’. Deals with the
basics rather quickly, but also covers regression methods in detail, something which is
outside the scope of the present course.

Rasinger, S. M. (2008) Quantitative Research in Linguistics: An Introduction. London:

Continuum.
[Contains chapters on: basic issues in quantitative research; research design and
sampling; questionnaire design and coding;data, its description and analysis; testing
hypotheses; dealing with difficult data. There is also a useful appendix containing a

SdC, IWoDA’11, University of Santiago de Compostela, 5-6 September 2011 5

Chris Butler Research methods in the study of language: an introduction
with particular reference to statistical techniques
summary of statistical formulae, Excel commands for producing statistical analyses, and
statistical tables.]
Field, A. (2009) Discovering Statistics Using SPSS for Windows. 3rd edn. London, Thousand
Oaks and New Delhi: Sage Publications.
[Not only illustrates the use of the latest version of SPSS, but also discusses in detail the
underlying methods. Highly recommended for more advanced study, especially if SPSS is
available.]

Larson-Hall, Jennifer (2009) A Guide to Doing Statistics in Second Language Research

Using SPSS. London: Routledge.
[Shows second language researchers how to use SPSS to use the statistical tests often
done in SLA research, such as chi square, t tests, ANOVA, correlation, multiple
regression and non-parametric analogues of these tests. Also deals with descriptive
statistics, how to choose a test and how to carry them out and interpret the results.
Many examples taken from actual research projects.]

SdC, IWoDA’11, University of Santiago de Compostela, 5-6 September 2011 6