ELICITATION

TECHNIQUES
Dadang Sudana
Source: Nunan, 1992: 136 - 158
 Introduction
• “Elicitation techniques have been a feature of SLA
research since the original morpheme order studies of the
1970s, collecting data through the use of Bilingual Syntax
Measure (BSM).”
• “Elicitation techniques vary enormously in scope, aim, and
purpose.”
• “They include studies which obtain data by means of a
stimulus, such as a picture, diagram, or standardized test,
as well as those based on questionnaire, survey, and
interview data, some of which will be discussed here.”
 Production tasks
• “The problem with naturalistic samples from learners as
they interact in the target language is that it can be
extremely time-consuming and difficult.
• It may not result in the outcomes one desires.”
• “Focused investigations attempt to overcome these
problems by employing elicitation to obtain samples of
learner language for linguistic analysis.”
• “Bilingual Syntax Measure (BSM) consisted of a series of
cartoonlike drawings.”
• Subjects were shown the drawings and asked a series of
questions which were designed to elicit the target language
items under investigation.”
• The slide illustrating BSM below is from the internet:
• “A similar technique has been used to obtain elicitation data from
immigrants.”
• “The Interview Test of English as a Second Language (ITESL) is
designed to obtain information from immigrants in Australia for
diagnosis, placement, and remediation (Griffin, 1986).”
• The test consists of twenty items designed to elicit twenty target
grammatical items.
• Each item contains a stimulus picture, cue questions, instructions for the
test administrator, and a set of scoring criteria.” ( look the illustration on
page 137)
• “The authors of this test claim that in the course of developing it, they
discovered a developmental order of grammatical acquisition…. This
order is at variance with that established by other studies and is, I
believe, an artifact of the elicitation instrument…” (NOTE: ‘artifact’)
• “Researchers need to be aware of two possible threats to
the validity of such investigations.”

• “The first is that by determining in advance what is going

to be considered relevant, other potentially important
phenomena might be overlooked.”

• “The other danger, and one which needs to be considered

when evaluating research utilizing such devices, is the
extent to which the results obtained are an artifact of the
elicitation devices employed.”
• “One needs to be particularly cautious in making claims
about acquisition orders based on elicited data, as Ellis
(1985) has pointed out.”

• “There is evidence that the so-called developmental order

of acquisition uncovered by the ITSEL is an artifact of the
test itself.” (look at page 139)
• “The dangers of making claims about acquisition based
on a single production task is underlined by Eisenstein,
Bailey, and Madden (1982), who carried out a study into
the acquisition of verb tenses.”

• “They use two different tasks to elicit data from their

subjects.”

• “The first of these was a production task similar to the

BSM, and the second was an imitation task –the
researcher read aloud sentences containing the target
items, and the subjects were required to imitate these.”
• “Eisenstein at al. concluded that the second task provided
more accurate information on the subjects’ current state of
morphosyntactic development.”

• “The major problem , as they saw it, with the production

task, was interpreting the data that resulted from the
task.”
• “Perhaps the best way of guarding against threats to the
reliability and validity of studies employing production
tasks is to do as Eisenstein at al. did and obtain one’s
data from more than one source.”

• “While this increase the practical problems of carrying out

the research, it greatly enhances the internal validity of
the study and enables the researchers to be more
confident in the claims they make.”
 Survey
• “The purpose of survey is generally to obtain a snapshot of
conditions, attitudes, and/or events at a single point in time.”

• “Survey research is distinguished from experimental studies

in a number of important respects.”

• “The most significant difference concerns the role of the

researcher…the experimental researcher manipulates the
environment in order to examine the interaction among
variables.”
• “In survey research, on the other hand, the researcher
doesn’t ‘do’ anything to the object of research, except
observe them or ask them to provide data.”

• “The research consists of collecting data on things or

people as they are, without trying to alter anything.”

• “In carrying out a survey, one works through a series of

steps similar to those of other types of research.” (Look at
Figure 7.1 on page 141)
 Steps in carrying out a survey
Step 1: Define objectives
Step 2: Identify target population
Step 3: Literature review
Step 4: Determine sample
Step 5: Identify survey instruments
Step 6: Design survey procedures
Step 7: Identify analytical procedures
Step 8: Determine reporting procedures
What do we want to find out?
Who do we want to know
about?
What have others
said/discovered about the
issue?
How many subjects should we
survey, and how will we
identify these?
How will the data be collected:
questionnaire/interview?
How will the data collection
actually be carried out?
How will the data be
assembled and analyzed?
How will the results be written
up and presented?
• “One of the most important question a survey researcher
must confront is: What is the population covered by the
survey?”

• “Of course it would not be practicable to obtain data from

the entire population, and a major task for the researcher
is to select a representative sample from the population
as a whole.”

• “The concern here is to ensure that the sample is

representative of the population as a whole.”
• In Table 7.2 “the first four strategies yield probability
samples. Using these procedures, we can determine the
probability of the selection of each respondent.”

• “Procedures 5 and 6, on the other hand, will result in non-

probability samples, in which the probability of selection is
unknown.”
 Strategies for survey sampling

Strategy Procedure
1. Simple random Select subjects at random from a list of the population.
2. Systematic Select subjects in systematic rather than random fashion
(e.g., select every twentieth person).
3. Stratified Subdivide population into subgroups (e.g.. Male/female)
and randomly sample from the subgroups.
4. Cluster Restrict one’s selection to a particular subgroup from
within the population (e.g., randomly selecting schools
from within a particular school district rather than the
entire state or country).
5. Convenience Choose nearest individuals and continue the process
until the requisite number has been obtained.
6. Purposive Subjects are handpicked by the researcher on the basis
of his/her own estimate of their typicality.
• “Perhaps the most comprehensive type of survey is the
national census, which aims the data on every individual
in the nation.”

• “When it is not feasible to collect data from the entire

population, the researcher must resort to sampling.”
• Fowler (1988: 41) “dismisses the common misconception
that the adequacy of a sample depends on the fraction of
the population included in that sample, arguing that a
sample of 150 people will describe a population of 15.000
or 15 million with virtually the same degree of accuracy,
assuming all other aspects of the sample design and
sampling procedures were the same.”

• “Key consideration in determining sample size concerns

the subgroups within a population that may need to be
separately identified – for example, whether to separate
men from women, or whether to separate the responses
from members of different ethnic groups.”
 Questionnaires

• “Survey data are collected through questionnaires or

interviews, or a combination of questionnaire and interview.”

• “Data from questionnaire are more amenable to

quantification than discursive data such as free-form field
notes, participant observers’ journals, the transcripts of oral
language.”
• Types of questions: “questionnaire can be relatively closed or
open ended.”

• Closed question types: list, category, ranking, scale,

quantity/frequency, grid. (p. 144).

• Question wording: “it is particularly important that the

researchers not reveal their own attitudes through leading
questions such as the following: ‘Do you think that the concept of
learner-centredness is utopian and unrealistic?”

• “Questions should not be complex and confusing, nor should

they ask more than one thing at a time.” (Example on page 143)
• “Another danger to avoid is culturally biased questions.”

• The following should be taken into consideration:

- the willingness of respondents to make critical
statements;
- to discuss certain personal topics, such as age, salary, or
opinions on political and social issues;
- the shared values which can be assumed such as the
concept of freedom of the press;
- the shared attitudes which can be assumed, for example,
the commonly held belief among many educators in
Western countries that classroom learning should be a
source of enjoyment for the learner.
Interpreting responses:
• First of all to be very clear about the objectives of the
study, and each item should be directly referenced
against one or more of the research objectives.”

• “you should also determine in advance how the data

to be gathered will be analyzed.”

• “It is imperative to pilot any questionnaire which is

developed.”
• “Having constructed, piloted, and administered a
questionnaire, one is faced with the task of collating and
interpreting the responses.”

• “One of the great advantages of closed questions is that

they yield responses which can readily be quantified and
analyzed, particularly if one has access to computer
statistics packages.”

• Free-form responses from open questions, although they

may result in more useful/insightful data, are much more
difficult to quantify, although there are ways of quantifying
this qualitative data.”
• reducing free-form data to manageable proportions from
the question: “State three beliefs you have about language
development that determine the way you teach.”

• Samples from 372 responses:

- children learn by using language;
- all children benefit from immersion of written print;
- language is developed through all curriculum areas;
- Children should be allowed to learn new concepts in their
native language if possible;
- Children learn best when there is a positive encouraging
environment.
• “to synthesize fifteen pages of statements in such
a way as to reveal possible patterns, yet without
distorting or misrepresenting the data.”

• “conducting a key word analysis, generating

categories such as ‘immersion’ and ‘learning by
doing/experiential.”
• Examples of categories: IMMERSION; LEARNING
BY DOING/EXPERIENTIAL; LANGUAGE ACROSS
THE CURRICULUM; GRAMMAR, STRUCTURE,
CORRECTNESS; ORAL/WRITTEN LANGUAGE
RELATIONSHIP; CREATION OF RICH, POSITIVE
ENVIRONMENT”, and many more.

• “Categories were then grouped together according

to whether they referred to language/learning, the
learner, or the climate/environment of learning.”
(see p.146 – 147)
• Quantifying qualitative data: “for example, the
researchers might want to know whether more
experience teachers, or teachers with higher
degrees, gave types of responses which were
different from the less experienced or less qualified
teachers.”

• “Using the procedure for calculating chi-square, the

researchers can determine whether these different
responses are significant.”
• “The final, and most difficult, task is to interpret the results.”

• “If, in the foregoing example, it should transpire that the

relatively more experienced teachers made significantly
more references to principles of language and learning,
one might be led to hypothesized that as teachers increase
their of technical aspects of instruction and knowledge of
language, their teaching practices become less dependent
on local environmental and affective factors.”

• “Such an observation would have implications for teacher

education.”
 Experienced Inexperienced Total
teachers (5+ teachers (1 – 5
years’ years’
experience) experience)
Reference to language 126 76 202
learning
Reference to 35 67 102
environment/climate
Reference to the learner 19 49 68
Total 180 192 372
 Interview
• “In addition to its use in survey research, the oral interview
has been used by second language acquisition researchers
seeking data on stages and processes of acquisition
(Johnston, 1985), and also by language testers, who use the
oral interview as a means of assessing proficiency (Ingram,
1984).”

• “The ‘sociolinguistic interview’ has also bee used to

investigate linguistic variation, conversational analysis,
pragmatics, and cross-cultural communication.”

• “Interview can be characterized in terms of their degree of

formality, and most can be placed on a continuum ranging
from unstructured through semi-structured to structured.”
• “An unstructured interview is guided by the
responses of the interviewee rather than the
agenda of the researcher.”

• “The researcher exercise little or no control, and the

direction of the interview is relatively unpredictable.”

• “In a semi-structured interview, the interviewer has

a general idea of where she or he want to go, and
what should come out of it, but does not enter the
interview with a list of predetermined questions.”
• “In the most formal type, the structured interview, the
agenda is totally predetermined by the researcher, who
works through a list of set questions in a predetermined
order.”

• “One source of bias in the oral interview is the

asymmetrical relationship between the participants: the
interviewer has much more power than the interviewee.”

• Numerous practical suggestions: preparing the interview

schedule; piloting; selection of subjects; elements of the
interview – briefing and explanation, questioning.”
THE END