Teachers of English to Speakers of Other Languages, Inc.

(TESOL)

Current Trends in Second Language Testing

Author(s): Eugène J. Brière
Source: TESOL Quarterly, Vol. 3, No. 4 (Dec., 1969), pp. 333-340
Published by: Teachers of English to Speakers of Other Languages, Inc. (TESOL)
Stable URL: http://www.jstor.org/stable/3586127 .
Accessed: 12/10/2014 15:07

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .
http://www.jstor.org/page/info/about/policies/terms.jsp

.
JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of
content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms
of scholarship. For more information about JSTOR, please contact support@jstor.org.

Teachers of English to Speakers of Other Languages, Inc. (TESOL) is collaborating with JSTOR to digitize,
preserve and extend access to TESOL Quarterly.

http://www.jstor.org

This content downloaded from 71.235.80.231 on Sun, 12 Oct 2014 15:07:39 PM

All use subject to JSTOR Terms and Conditions
CurrentTrendsin SecondLanguageTesting
Eugene J. Briere

The second conference on "Problems in Foreign Language Testing" was

held at the Idyllwild campus of the University of Southern California
November 7-9, 1968. Eleven papers were presented by the participants.
The papers covered many aspects of the field of language testing and ranged
from the highly theoretical to the highly pragmatic. This paper is an
attempt to summarize the content of some of the papers presented at that
conference in order to report to you some of the current thinking in second
language testing.
Because the theoretical papers have such far-reaching implications for
the fields of language testing and language teaching, this paper will empha-
size the contents of the theoretical papers. Unfortunately, due to space
limitations, the summaries of the "pragmatic" papers must necessarily be
shortened to a mere sentence or two.
As a footnote, I would like to point out that (a) although the three
theoretical papers were written independently, there is an amazing cohesive-
ness of thought among all three and (b) except for a few occasions, quota-
tion marks should be placed at the beginning of this paper and at the end
of the paper because most of the remarks are those of the authors, not mine.
It is interesting to note that the major concern of the first three papers
is the very real problem of identifying many of the variables which define
the competence of a speaker or listener in any act of communication. The
general agreement among the three papers is that traditional linguistic
definitions of the notion of competence in a language are too narrow and
are inadequate in identifying all of the skills involved when two people
communicate. Consequently, language tests based on the narrow linguistic
definition of competence (and 99 and 44/100% of them are) will be inade-
quate. At best, such tests only give us some kind of measure of behavior
which I will call "surface" behavior based on the analogy of the floating
iceberg. The part of the iceberg which is seen floating on top of the water
is but a small fraction of what lies underneath the water. So it is with
language competence. The three papers argue that language tests today
are limited to measuring that which is on the "surface" and can give us no
information about what is "underneath." However, probably it is precisely
these unidentified and unmeasured variables "underneath" which constitute

* This paper was presented at the TESOL Convention, March 1969.

Mr. Briere, Associate Professor of Linguistics, University of Southern California,
has published articles in Language, IJAL, Language Learning, Journal of Verbal Learn-
ing and Verbal Behavior, and previously in TESOL Quarterly, III, 1 (March 1969).
333

This content downloaded from 71.235.80.231 on Sun, 12 Oct 2014 15:07:39 PM

All use subject to JSTOR Terms and Conditions
334 TESOL QUARTERLY

the "bulk" of language competence. What is needed is a serious attempt to

develop a model which will identify and measure those variables which, at
the moment, are "underneath the surface." The three papers discussed in
this report are definitely "serious attempts" to develop such a model.
The papers are summarized in the order in which they were presented
at the Conference.
Bernard Spolsky, University of New Mexico, gave a paper entitled
"What Does It Mean to Know a Language, or How Do You Get Someone
to Perform His Competence?" In this paper, Spolsky shows that although
Fries rejected the layman's notion that knowing a certain number of words
in a language is the criterion for knowing that language, he still maintained
the related notion that knowing a language involves knowing a set of items.
This accounts for most of our language testing techniques used today.
Spolsky suggests, however, that testing of individual elements which could
be mastered, such as sound segments, sentence patterns, or lexical items,
is still inadequate.
He points out that the layman's criterion for knowing a language is
usually expressed in some type of functional statement. For example,
"I know enough French to read a newspaper." Statements such as these
refer to language use and not to grammar or phonology. The question then
arises, how does one go about deciding when someone knows enough
language to carry out a specified function. One approach is to give someone
a language-using test to perform such as having a physics major listen to a
lecture on thermodynamics, then checking the comprehension. Another
approach is to characterize in linguistic terms the knowledge of the language
required to function in this way, that is, to describe the linguistic knowledge
which correlates with the functional ability. However, one of the funda-
mental reasons that this approach has not proved successful is that it fails
to take into account the fact that language is redundant and that it is
creative.
Redundance (part of the statistical theory of communication) is present
in all natural languages since more units are used to convey a message than
are theoretically needed.
Spolsky cites the work which he had done experimentally using re-
dundancy as a testing technique. In this experiment noise was added to
messages and the tapes were played to native and non-native speakers. The
non-natives' inability to function with reduced redundancy suggested that
the key thing missing was the richness of knowledge of probability on all
levels-phonological, grammatical, lexical, and semantic. Two implications
followed from this experiment. The first is that knowing a language involves
the ability to understand a message with reduced redundancy. A model of
understanding speech must then include the ability to make valid guesses
about a certain percentage of omitted elements. The second implication is
to raise some serious theoretical questions about the value of deciding a
person knows a language because he knows certain items in the language.

This content downloaded from 71.235.80.231 on Sun, 12 Oct 2014 15:07:39 PM

All use subject to JSTOR Terms and Conditions
 TRENDS IN SECOND LANGUAGE 335

The principle of redundancy suggests that it will not be possible to demon-

strate that any given language item is essential to successful communication,
nor to establish the functional load of any given item in communication. He
then makes the distinction between language-like behavior, for example, the
utterances of the parrot, and knowing a language on the basis of creativeness.
The central fact in support of the creative aspect of language is that
humans produce (and of course understand) many sentences that they
never heard before.
One fundamental factor involved in the speaker-hearer's performance is
his knowledge of the grammar that determines an intrinsic connection of
sound and meaning for each sentence. We refer to this knowledge-for the
most part, obviously, unconscious knowledge-as the speaker-hearer's"com-
petence." Performance, that is, what the speaker-hearer actually does, is
based not only on his knowledge of the language, but on many other factors
as well-factors such as memory restrictions, inattention, distraction, etc.
Therefore, in searching for a test of overall proficiency, we must try to
find some way to get beyond the limitation of testing a sample of surface
features, and seek rather to tap underlying linguistic competence. This can
only be done with any degree of certainty if we can be sure that we are
presenting the subject with novel utterances, or calling on him to produce
utterances that he has not heard before.
Spolsky, then, has underlined the inadequacy of testing selected items
and has suggested added dimensions, such as redundancy, to the testing
situation.
The second paper presented at the conference also suggested added
dimensions and suggested a small scale model of an internal machine which
would specify some of these added variables. John Upshur, University of
Michigan, in his paper "Measurement of Oral Communication" first traces
the attempts that have been made so far in trying to find a measure of a
"general proficiency factor" in testing production of language. He feels that
these attempts have been essentially unsuccessful primarily because of the
lack of any performance theory generally available to, and useful for, those
who might prepare production tests.
Upshur suggests that in the act of communication, a Speaker's Meaning
(SM) be distinguished from Utterance Meaning (UM) or Word Meaning
(WM). In communication it is the task of the producer (a task certainly
shared by the receiver or audience) to "induce" in, or transmit to, the
audience a meaning (AM) which has as a part an equivalent of SM.
Because communication requires that AM contain SM, because UM is a
medium through which this is accomplished, and because SM need not be
equivalent to UM, more is required than that S (and A) have competence in
some language and that both have the ability to make a "sufficiently small"
number of errors in applying the rules of their competence components.
SM is, first of all, a transitory concept with a propositional constituent
much as Fillmore has described it: viz., "entities" have among them a

This content downloaded from 71.235.80.231 on Sun, 12 Oct 2014 15:07:39 PM

All use subject to JSTOR Terms and Conditions
336 TESOL QUARTERLY

certain set of case relationships. For S to get his meaning across to A (i.e.,
to communicate, to have AM contain SM), it is necessary (1) for A to get
the word meaning, (2) for A to know the case relations for each W (this
seems to be a part of UM), and (3) for A to get the relations between a
proposition and other concepts. A performance theory which explains only
the successes and failures of S (or A) to apply his competence may account
for communication of case relations; it can perhaps account for the com-
munication of WM; but it cannot altogether account for the communication
of relations between propositions and other concepts.
Upshur then develops a model for A which could account for the pro-
cessing of SM in A. The kinds of "components"in A which he suggests are
such things as: perceptions of the outer world (PO); a store of concepts
(AMs) resulting from the current communication transaction (CCS); a
semantic net (NET); a linguistic competence (COMP) and several others.
He then suggests the processing sequence that occurs when A receives a
message from S.
From S's point of view, he must have a concept to communicate (SM),
and some reason for doing so. S has the belief that A lacks the concept SM.
and cares to have it. His communication ability is then a function of (1) his
success in determining the constraints imposed by the contents of A's com-
ponents, (2) his success in altering the contents of those components, and
(3) his success (in language communication) in adapting his own com-
petence.
The model suggests that oral production testing, viewed as one of the
four skill components of the Carroll model, is but one part of speaker com-
munication testing. Communication measurement involves a matching of
SM and AM; therefore, "precise" measurement is not likely without com-
parable measures of both.
Upshur suggests several requirements which must be met in order to
translate different communication situations into language test situations.
First, a naive A receives a signal from S. A must be able to communicate
in the test language. We assume A has no knowledge of SM before receiving
a signal from S. The second requirement is that A have some way of com-
municating to a grader of AM. The third requirement is that the grader
also know SM. Grading is a process of determining the extent to which SM
is present in AM.
One experimental form to test one kind of communication situation is
currently being investigated by Upshur. In this technique, a set of 86 four-
picture items was prepared. The S's specific task was to communicate to
a remote A which one of the four pictures was identical to a single picture
shown to him by an examiner. The set contained pictures of four different
objects or actions being performed, four different values of some attribute,
etc. Six students of English as a foreign language took the initial 86 item
test. Utterances were recorded. Four naive A's listened to all six tapes. The
inner judge reliability for correct items was .872. Uniformly high coefficients

This content downloaded from 71.235.80.231 on Sun, 12 Oct 2014 15:07:39 PM

All use subject to JSTOR Terms and Conditions
 TRENDS IN SECOND LANGUAGE 337

were found between raw scores, total response, and communication rate
scores with composition and achievement tests scores. (Incidentally a modi-
fication of this technique is currently being used by the writer with American
Indian elementary school children.)
The third paper at the conference which stressed the need for added
dimensions to language-testing techniques was presented by Leon Jakobovits
from the Center for Comparative Psycholinguistics at the University of
Illinois. His paper, entitled "A Functional Approach to the Assessment of
Language Skills," pointed out that there is an obvious difference between
linguistic competence as it is traditionally defined and communicative com-
petence. The latter involves wider considerations of the communication act
itself-considerations which the linguists have dismissed in their definitions
of linguistic competence as being primarily the concern of paralinguistics,
exolinguistics, sociolinguistics, and psycholinguistics. Since the authors of
language tests are aware that the study of language use must necessarily
encompass the wider competencies involved in communicative competence,
the development of language tests must move from the present position of
measuring merely linguistic competence to the position of measuring com-
municative competence.
Jakobovits points out that speakers of a language have a command of
various codes that can be defined as a set of restriction rules that determine
the choice of phonological, syntactic, and lexical items in sentences. For
example the choice of address form in English, "using the title Mr. followed
by the last name versus first name," is determined by the social variable
which relates the status relation between the speaker and the listener. These
selection rules and others of this type are as necessary a part of the linguistic
competence of the speaker as those with which we are more familiar in
syntax, such as accord in gender, number and tense; and it would seem to be
entirely arbitrary to exclude them from a description that deals with lin-
guistic competence.
In order to be able to account for the minimum range of linguistic
phenomena in communicative competence, it will be necessary to incorporate
in the analysis of the utterance three levels of meaning, namely linguistic,
implicit, and implicative.
By "linguistic meaning" Jakobovits refers to the traditional concerns
of linguists such as Chomsky and Katz. This includes a dictionary of lexical
meanings and their projection rules, syntactic relations as defined by a
derivative transformational theory and phonological actualization rules.
By "implicit meaning" he refers to the elliptically derived conceptual
event which an utterance represents. By this is meant that particular impli-
cations for homonymous utterances are a function of the situational contexts
in which the utterance is used. In order to recover the particular meanings
of the word intended by the speaker, the listener must engage in an infer-
ential process which makes use of his knowledge of the dictionary meaning

This content downloaded from 71.235.80.231 on Sun, 12 Oct 2014 15:07:39 PM

All use subject to JSTOR Terms and Conditions
338 TESOL QUARTERLY

of words as well as his knowledge of the overall situation to which the

sentence as a whole refers.
"Implicative meaning" refers to the information in an utterance about
the speaker himself, e.g., his intention, his psychological state, his definition
of the interaction, etc. In some cases these implications are necessary to
recover the intended meaning of the utterance: "Do you have a match?"
is not a question to be answered verbally, but a request for fire to light a
cigarette.
The problem, then, of assessing language skills becomes the problem of
describing the specific manner in which an individual functions at the three
levels of meaning identified above.
Language tests then must take into account the full range of phenomena
in communicative competence if language use is to be tested.
Jakobovits outlines the major elements to be included in the model of
language use. The following is an extremely curtailed description of his
outline:
1. A Functional analysis of meaning
1.1 Linguistic meaning
1.2 Implicit meaning-Variables suggested for consideration in an
analysis of implicit meaning are aspectual, affective, and situational.
1.3 Implicative meaning-Sociolinguistic factors, intention, and rela-
tional status should be considered.
2. Elements of style-This would include such variables as internal con-
sistency and appropriateness.
3. Selection factors in the act of composing-Within this category one
should consider such things as audience, situation, and style.
4. The structure of the message-Syntactic and conceptual variables
should be considered within this category.
Jackobovits suggests that the development of a testing program based on
the classification scheme just described would require a team of workers with
specialized skills. He then makes some tentative suggestions with respect to
some methodological approaches which may be used in connection with his
classification scheme. Some of the methods suggested are as follows:
A. Judgments of Acceptability-Ask a subject to judge the acceptability
of an utterance or pick the most appropriate of two similar utterances.
Aspectual qualities such as extended duration (e.g., "to stare" versus "to
notice") and limited duration (e.g., "to strike" versus "to beat") can be
determined by this method.
B. Semantic differential techniques-Subjects rate a word on a seven
point bi-polar adjectival scale according to the Osgood Method. This
method would help us to assess the quality of intensity, for example, by
asking a subject to say which of two choices is more or less intense-"icy"
verus "cold."

This content downloaded from 71.235.80.231 on Sun, 12 Oct 2014 15:07:39 PM

All use subject to JSTOR Terms and Conditions
 TRENDS IN SECOND LANGUAGE 339

C. Acting out situations-Asking a subject how he would say something

under specified conditions becomes an encoding task that assesses cor-
responding active skills. Since encoding and decoding skills are not neces-
sarily related, this method could be used in addition to others which really
measure only passive skills.
In addition to the three papers just discussed, the following papers were
also delivered in the following order.
Ted Plaister, Chairman of the Department of English as a Second
Language at the University of Hawaii, gave a paper entitled "Measuring
Control of English Structure of Non-Native Speakers of English by Means
of Responses to a Scale of Grammaticalness." In this paper, Plaister re-
ports an experiment in which he tested non-native speakers' reactions to
degrees of grammaticalness based on the E.B. Coleman scale. The scores of
the Ss were correlated with scores on "structure" tests but were, for the
most part, statistically nonsignificant. This leads us to the question of
whether the "structure" tests were valid or whether the test of determining
degrees of grammaticality is valid.
Leslie Palmer, Georgetown University, in his paper entitled "Proposed
Analysis of Some Factors Affecting the Rating of English Spoken by Foreign
Speakers of the Language," reports the results of his most recent experi-
mental investigations of ratings made by native speakers of the utterances
made in English by foreign language speakers. Many of us are looking
forward to a completion of this project because we feel that many of the
variables he has isolated in his design will tell us a great deal more about
how to go about grading samples of oral speech.
Robert Wilson, University of California, Los Angeles, in his "ESL Mate-
rials for the Navajo: A New Approach," presented some novel approaches
to teaching English as a second language that excited the group and raised
many questions among the authors of tests as to how we will be able to
evaluate the new procedures he is proposing.
Newton Metfessel, University of Southern California, in a paper entitled
"Assessment of Cognitive Abilities of Children and Youth from the Culture
of Poverty," reported his recent investigations in the ghetto areas and
inspired a spirited discussion of his methods and findings.
Briere, in a rambling report called "ESL Placement Tests for the Bureau
of Indian Affairs," reported on the recent activities occurring among his
group to develop placement tests in English as a second language for the
American Indian Children attending BIA schools.
Mohamed Youseff, from the American University in Cairo, Egypt, re-
ported his recent investigations of "Facilitation Effects of Multiple Choice
Items in Grammar Testing." In this experiment Youseff used a cloze tech-
nique with items in a test and checked the effect of this technique on
different language groups.
David P. Harris, Georgetown University, in his paper "Testing English
in Difficult Circumstances" discussed the administrative problems he en-

This content downloaded from 71.235.80.231 on Sun, 12 Oct 2014 15:07:39 PM

All use subject to JSTOR Terms and Conditions
340 TESOL QUARTERLY

countered in testing English in Greece. Those of us who work in the field

felt that our lots are indeed very simple in comparison to what he experienced.
Robert Jones and Robert Kaplan, both from the University of Southern
California, presented a paper called "A Cloze Procedure Test of Listening
for University Level Students of English as a Second Language." In this
experiment they found that the cloze procedure might prove fruitful in tests
of listening comprehension for non-native speakers of English.
As frequently happens at small, informal yet stimulating conferences,
the group disbanded on a note of renewed interest in trying to solve some
of the complex questions which resulted from the papers and discussions.

This content downloaded from 71.235.80.231 on Sun, 12 Oct 2014 15:07:39 PM

All use subject to JSTOR Terms and Conditions