Book reviews

Madsen, H.S. 1983: Techniques in testing. New York and Oxford:

Oxford University Press. viii + 212 pp. ISBN 0-19-434132-1.

This book is intended for practising and student teachers of English

as a Second or Foreign Language. As such it is something of a para-
dox : it is a useful book, but one whose merits are finally outweighed
by its defects.
After a brief introduction which justifies the role of testing in
language teaching, and gives a brief history of testing, Part I deals
in three chapters with ’testing language subskills’. Part II then deals
in four chapters with ’testing communication skills’. There is a final
chapter on ’evaluating tests’ and an appendix describing some cur-
rently availablc languagc tcsts in thc USA and thc UK. Ncxt thcre is
a brief section of references (though none are to the ’standard’

language testing books for teachers, such as Harris (1969), Valette

(1977), or Heaton (1975)), and finally a subject index. There is no
bibliography.
One of the book’s chief positive points is that it is true to its
title. Techniques are promised and techniques are delivered. There
will be few readers who fail to find one or two new techniques.
In addition, the techniques are for the most part clearly described
and amply exemplified. Practical advice abounds, in particular on
the writing of multiple-choice items (e.g. pp. 39-42) and on scoring
spoken language (p. 166 ff). Each section ends with a useful sum-
mary listing advantages and limitations of the technique that has
been described. A pedagogically fruitful feature of the book is the
’Activities’ sections which allow ’you to work with those items you
need further practice with’ (p. 10). This should provide teacher
trainers/trainees with plenty of ideas for tasks as well as ’leading
out’ to issues beyond those covered in the book.
The credit side, then, is considerable; the debit side, however, is
even more considerable. First, though the promise of the title is

Downloaded from ltj.sagepub.com at University of Otago Library on March 14, 2015

106

honoured, the major parts: ’Testing language sub-

titles of the two
skills’ and ’Testing communication skills’ are misleading. The ’sub
skills’ in fact turn out to be none other than vocabulary, grammar
and pronunciation. In defence of Madsen’s use of the term ’subskill’
for what are more generally referred to (Carroll, 1961; Harris,
1969) as ’components’, ’elements’ or ’levels’, it might be said that
an author is free to innovate in the use of terms. If the term is not
to mislead, however, such innovative use must be justified. In this
case, the term ’subskill’ surely implies mastery or ability of some
sort. Thus ’vocabulary’ is not, in itself, a subskill, though ’using
vocabulary appropriately’ is. I have laboured this point, but it is
symptomatic of a cavalier approach to phraseology that is not
appropriate in a book that claims to deal with language.
Even more misleading is the heading for Part II, ’Testing com-
munication skills’. This part is divided into tests of reading, writing,
listening and speaking. While communication clearly takes place
through these skills, the question of whether a given test is an
adequate instrument for measuring communicative ability will
depend largely upon the nature of the test. In my view most of the
techniques proposed in Part II are integrative, but not communi-
cative. Moreover, it is not the case that Madsen has an idiosyn-
cratic interpretation of the heading, for he proceeds to make his
meaning quite explicit in his claim that Part II will ’look at tests
that measure actual communication in the real world’ (p. 75).
This is a very strong claim. One suspects again a loose use of language,
and that what Madsen really means is that Part II will look at tests
that measure an individual’s ability to communicate by means of
tests that replicate real-life tasks. However, the majority of the test
techniques described do not even match up to this modified aim,
for they are overwhelmingly language-oriented, and concerned with
formal accuracy. Of course, we must accept that accuracy of form
will (in all but exceptional cases) contribute to effective communi-
cation, and that Madsen’s techniques may therefore provide measureb
of proficiency in the skills underlying communication. The claims
made should, however, have been modified to take account of this.
A possible reason for the weakness of the book in Part II is that
Madsen makes no attempt to outline any theory of ’actual com-
munication’, or even to provide a simple description of it, apart
from saying it is ’more complex’ than adding pronunciation, vocabu-
lary and grammar (p. 75). Granted, but the nature of the complexity
should have received more attention than vague statements such as:
’Moreover, broader communication is concerned not with bits and
pieces of language, but with the exchange of facts and ideas, as well

Downloaded from ltj.sagepub.com at University of Otago Library on March 14, 2015

 107

as interpreting the speaker’s intentions’ (p. 127). This, quite simply,

is not good enough: why ’broader’ communication, why ’bits and
pieces’ of language? What is the relationship between ’facts and
ideas’ and the suddenly introduced issue of ’interpreting the speaker’s
intentions’? Come to that, why ’speaker’? What about listener,
reader, writer, even user? The excuse that this is deliberate simpli-
fication for unsophisticated teachers will not wash, for the un-
sophisticated should have concepts explained more fully rather than
less, particularly in the opaque area of communicative language
testing.
Alderson (1981: 56) raises the issue of whether ’tests are mirrors
of reality, or constructed instruments from a theory of what lan-
guage is, what language processing and producing are, what language
learning is’. What Madsen seems to want to describe is tests that are
mirrors of reality, but what he in fact describes are tests that are
’constructed instruments from a theory of what language is’. The
theory that he utilizes is one of ’language-as-object’, the well-known
descriptive theory based on analysis into three components (vocabul-
ary, grammar and pronunciation - one might, in passing, ask why
the graphological component is absent) and four skills (reading,
writing, listening, speaking). This theory has proved extremely pro-
ductive but is not an appropriate underpinning for tests of com-
municative ability. Theory for the sake of theory would of course be
out of place given the audience envisaged by Madsen. Nonetheless
theory, even if not articulated, underlies techniques, and the prob-
lem with Part II is that the underlying theory of language as object
does not help Madsen to come to grips with the aim he has set
himself, in that it tends to yield test techniques that fail to incor-
porate the features that distinguish communicative tests from inte-
grative tests. What the features of communicative tests might be
has received attention elsewhere (cf. Morrow, 1979; Porter, 1983).
The features selected, and the importance attached to them, can
vary depending on whether one inclines to a ’communication theory’
approach (cf. Cherry, 1957) which gives prominence to the trans-
mission of information, to a ’communicative competence’ approach
(cf. Hymes, 1971) which stresses sociolinguistic norms, or even to a
’communicative syllabus design’ approach (cf. Munby, 1978) which
emphasizes learners’ needs. Elements from all three approaches are
included in the following three questions (but not on a one-to-one
basis); the answers to them can provide a crude indication of a test’s
‘communicativeness’:
i) is the test task a reflection of
a ’real-life’ situation (with all that is

thereby implied in terms of ’real message’ transmission and of

appropriacy situation,
to discourse and intention)?

Downloaded from ltj.sagepub.com at University of Otago Library on March 14, 2015

108

ii) in scoring, is priority given to meaning rather than formal

accuracy?
iii)
I
is the test task one that the testee perceives as relevant?
Note that Question iii suggests that ’communicativeness’ is not an
intrinsic feature of a test, but is to a degree testee-dependent. Note
too that Question i suggests that communicative tests will tend to-
wards simulation, with the attendant problems of extrapolation.
The observation that tests ’are themselves authentic situations’
(Alderson, 1981: 57) (and that they need therefore not attempt to
mirror reality), is incontrovertible, but does not advance the dis-
cussion. While admitting that a ’real-life’ communicative test is for
practical reasons difficult (and some would claim in principle
impossible), it would have been useful if Madsen had discussed some
of the implications for classroom testing of the issues raised.
The purpose of the foregoing is not to argue for the superiority of
communicative over non-communicative tests, but simply to point
out that they are different. This implies that I take issue with
Madsen’s statement that ’actual communication in the real world’ is
measured through techniques such as the following: in reading,
matching visual and sentence, multiple choice, labeling statements
as true or false, identifying intrusive words; in writing, grammatical

transformation, guided writing, dictation; in listening, listen and

judge statements as true or false in relation to a picture, multiple
choice; in speaking, directed response (e.g. cue: ’Tell me he went
home’, response: ’He went home’), questions about pictures and
reading aloud. The claim (p. 149) that these three speaking tech-
niques ’appear in everyday communication’ is debatable. All the
above techniques may be integrative, but they are not necessarily
communicative. If Madsen is simply reclassifying integrative tests
as communicative tests, then he should have spelt this out clearly.

However, alongside these integrative test techniques, Madsen does

discuss other techniques that may have a primp facie claim to being
tests of communication. Examples are: listen and draw (p. 132),
listen and follow a map (p. 133); rather different instances are
role play (p. 160) and oral interview (p. 162). These perhaps do
begin to justify the claims made.
Not only is the components/skill approach inappropriate to
Madsen’s aims in Part II, but the fact that he deals with each com-
ponent and each skill separately leads to the problem of how to
categorize certain techniques, e.g. ’hearing identification’ (p. 61).
Madsen recognizes this problem: ’since listening includes the recog-
nition of words and structures and pronunciation features, the
difference between subskill tests using listening as a tool, and in-

Downloaded from ltj.sagepub.com at University of Otago Library on March 14, 2015

 109

tegrative listening comprehension tests can be blurred at times’.

(p. 127). Although Madsen copes with the problem, it might have
been avoided altogether had the book been organized differently.
A more irritating effect of this component-by-component plus
skill-by-skill approach is that it inevitably commits Madsen to a
repetition of techniques. This is particularly the case with multiple
choice. Together with the congeneric true/false, this technique takes
up approximately 20 per cent of the text and receives attention
on at least eight separate occasions. Despite this, Madsen fails to

point out the major drawbacks of multiple choice, namely that a

good distractor may distract all too well and confuse a testee who
might have obtained the correct answer if tested through a different
technique. Thus good distractors may be bad for learners.
Rather than employ the components/skills framework in the
way he does, it might have been more useful if Madsen had opted
for a more straightforward ’cookbook’ which simply listed and
exemplified techniques. While such an approach might be criticized
for being non-theoretical, teachers could select techniques which
matched their teaching approach, guided by discussion of their
advantages and limitations.
A point in the book that is unclear is the relationship between
norm and criterion-referenced tests on the one hand, and achieve-
ment tests on the other. Madsen claims that ’norm-referenced tests
compare each student with his classmates’ (p. 9) and that ’criterion-
referenced exams rate students against certain standards regardless
of how other students do’ (p. 9). He also points out that ’achieve-
ment tests ... measure progress’. From this it must surely follow
that most classroom tests are criterion-referenced, measuring whether
what has been taught has been learned. Yet Madsen claims (p. 9)
that ’most classroom tests’ are norm-referenced, without further
justification. Of course a criterion-referenced test may discriminate
and even yield normal distribution over a given class of testees:
however this will be strictly coincidental to the test’s purpose which
is to indicate whether each testee achieves a satisfactory perfor-
mance (i.e. reaches criterion).

Chapter 9, ’Evaluating tests’, treats evaluation only from the

standpoint of item analysis, and again seems overly concerned with
discrimination. In talking of facility value, Madsen says that an item
is ’too easy if more than 90 percent get it right’ and ’too difficult
if fewer than 30 percent get it right’ (pp. 181-82). Although he goes
on to say that ’it is not completely accurate to think of very diffi-
cult and very easy items as ’weak’ questions’ (p. 182), he does not
clearly make the point that there is nothing wrong in principle with
an item in an achievement test which all testees get right.

Downloaded from ltj.sagepub.com at University of Otago Library on March 14, 2015

 110

The statistical procedures in this chapter are also rather unusual.

The most accurate picture of the difficulty level (or facility value) of
an item would surely be gained by expressing the number of correct
answers as a percentage of the number of testees, not as a percentage
of the sum of testees in the top and bottom thirds (p. 182). Again
when calculating the discrimination level of an item the divisor is
normally the number of testees in the top subgroup (which is the
same as that in the bottom subgroup) and not, as Madsen suggests
on p. 183, the total number of testees in both subgroups. Madsen’s
method is not ’wrong’ of course, simply unorthodox in that it can
yield no result higher than 50 per cent, a somewhat perverse appli-
cation of the concept percentage.
On a more trivial level, the book seems out of touch with recent
developments. There is little to indicate that Madsen is aware of
work done in communicative testing in Britain. Again, the series
editors say ’seldom are books written that present practical infor-
mation that relates directly to daily classroom instruction’ (p. vii).
Since the late 1970s this has not been true, as witness the series for
teachers by Longman, Cambridge, Heinemann and Macmillan. If
- it is true for the USA, then a British edition of this book should
have been produced (which might, incidentally, have dealt with
points such as the US, but not British, minimal pair ’leather/lever’
on p. 67).
The book also bears signs of having been hurriedly put together
and carelessly edited. Three examples will suffice. On p. 80 there is
a picture which apparently contains two robbers, though the associ-
ated test items require one robber only. Secondly, on p. 29 Madsen
quite rightly says that reading comprehension questions should not
be answerable from general knowledge; yet on pp. 87-88 there is a
reading text followed by these true/false items:
Jazz is related to the work songs of American blacks.
Work songs are not helpful in getting more work done.
While admitting that general knowledge varies from person to
person, surely more appropriate examples could have been chosen.
Third, on p. 86, we read that ’it is interesting nowadays to notice
how many unusual ’passages’ or ’contexts’ that students are intro-
duced to in an ESL class’. Infelicities of this type should have been
spotted at some stage.
To conclude, one cannot help feeling that Madsen would have
done himself more justice with a less constraining framework than
the components/skills one. As it stands, this is an unsatisfactory

Downloaded from ltj.sagepub.com at University of Otago Library on March 14, 2015

 111

book, and OUP will have disappointed many people. Gaps in the list
should be filled more carefully.

University of Reading Eddie Williams

Alderson, J.C. 1981: Report of the discussion on communicative

language testing. In Alderson, J.C. and Hughes, A., editors,
ELT documents 111 - issues in language testing, London:
The British Council.
Byrne, D. series editor: Handbooks for language teachers. Harlow:
London.
Carroll, J.B. 1961: Fundamental considerations in testing for English
language proficiency of foreign students. In Testing the English
proficiency of foreign students, Washington: Center for Applied
Linguistics.
Cherry, C. 1957: On human communication. Cambridge, Massa-
chusetts: MIT Press.
Flavell, R.H. series editor, Essential language teaching series. London:
Macmillan.
Geddes, M. and Sturtridge, G. series editors, Practical language
teaching. London: Heinemann Educational.
Harris, D.P. 1969: Testing English as a second language. New York:
McGraw-Hill.
Heaton, J.B. 1975: Writing English language tests. Harlow: Longman.
Morrow, K. 1979: Communicative language testing: revolution or
evolution? In Brumfit, C.J. and Johnson, K., editors, The
communicative approach to language teaching, Oxford: Oxford
University Press.
Munby, J. 1978: Communicative syllabus design. Cambridge: Cam-
bridge University Press.
Porter, D. 1983: Assessing communicative proficiency: the search
for validity. In Johnson, K. and Porter, D., editors, Perspectives
in communicative language teaching, New York: Academic
Press.
Swann, M. series editor: Handbooks for language teachers. Cam-
bridge: Cambridge University Press.
Valette, R.M. 1977: Modern language testing. New York: Harcourt
Brace Jovanovich.

Downloaded from ltj.sagepub.com at University of Otago Library on March 14, 2015