3 C7

117
Longman Handbooks for Language Teachers
J. B. Heaton
New Edition
Consutant editors! Jerem" Harmon and #e" $ingsbur"
London and New %ork
11
1
.ongman &rou' L($ Limited
ongman House, Burnt Mill, Harlow,
sse) CM202JE, England
rid Associated Companies throughout the world.
ubished in the *nited +tates of ,merica
- Longman (nc.. New %ork
Longman &rou' *$ Limited 1/00
It rights reserved no part o! this pu"lication ma# "e reproduced, stored in a
retrieval s#stem, r transmitted in an# !orm or "# an# means. electronic,
mechanical, photocop#ing, recording, ,$otherwise, without the prior written
permission o! the %u"lishers.
1first 'ubished 1/72
hird im'ression 1//3
#(T(+H L(B#,#% C,T,L4&*(N& (N 5*BL(C,T(4N 6,T,
eaton. J. B.
7riting Engish anguage tests. 8 New ed. 8 9Longman handbooks for anguage teachers:.
1. Engish anguage 8 +tud" and teaching 8 ;oreign s'eakers
<. Engish anguage 8
,biit" testing
1.
Tite
=<0.<>=>37? 5E11<0.,<
.+BN 3822<833<3783
(B#,#% 4; C4N&#E++ C,T,L4&*(N& (N 5*BL(C,T(4N 6,T,
eaton. J. B. 9John Brian:
7riting Engish anguage tests.
9Longman handbooks for anguage teachers:
Bibiogra'h"! '.
(ncudes inde).
1.
Engish anguage 8 +tud" and teaching 8 ;oreign s'eakers.
<. Engish anguage 8
)aminations 8 ,uthorshi'. 3. Engish anguage 8 ,biit" testing. (. Tite. ((. +eries.
11<0.,<H3/= 1/00 =<0.37? 0782<73
@8in Times #oman
8oduced b" Longman &rou' 9;E: Ltd.
8inted in Hong $ong
iustrated b" 6a-id 5arkins
.C$N47LE6&EAENT+
e are gratefu to the foowing for 'ermission to re'roduce co'"right materia!
he author. John Bright and the *ni-ersit" of Cambridge Loca E)aminations +"ndicate for
n e)tract from his critiBue on s'ecimen e)amination Buestions@ Har'er C #ow 5ubishers
for a tabe from '. 1=3 DE+L Com'osition 5rofie> from &eaching E'( Compositi
on "#
ene B. Hughe". 6eanna #. 7ormuth. E. ;a"e Hartfied and Ho" L. Jacobs 9Newbur"
8ouse: Co'"right F 1/03 b" Newbur" House 5ubishers (nc@ the author. #osaind Hawkins.
Thief E)aminer for *CLE+ 5reiminar" Engish Test and the *ni-ersit" of Cambridge Loca
)aminations +"ndicate for e)tracts from sam'e test materias@ Hong $ong Education
Ge'artment for e)tracts from the Hong $ong Engish +choo Certificate E)amination 1/?0
nd the Hong $ong +econdar" +choos Entrance E)amination 1/?0@ Longman &rou' *$ Ltd
:r e)tracts from Composition &hrough %ictures b" J. B. Heaton.
'tud#ing in English b" J.
B.
caton and
)riting &hrough %ictures "# J. B. Heaton@ The author. ,nthon" Tucker for an
+treet from his artice in &he &uardian 2th +e'tember 1/?/@ and the foowing e)amination
oards for 'ermission to re'roduce Buestions from 'ast e)amination 'a'ers! Joint
Htatricuation Board@ North 7est #egiona E)aminations Board@ The #o"a +ociet" of ,rts
)aminations Board! *ni-ersit" of Cambridge Loca E)aminations +"ndicate@ *ni-ersit" of
I:)ford 6eegac" of Loca E)aminations and the ,res E)aminations Trust.
o dents
1 (ntroduction to anguage testing 2
1. 1 Testing and teaching 2
1.< 7h" testG ?
1 3 7hat shoud be tested and to
what standardG 7
1.= Testing the anguage skis 0
1.2 Testing anguage areas /
1 ? Language skis and anguage
eements 13
1 7 #ecognition and 'roduction 11
1 0 5robems of sam'ing 1<
1 / ,-oiding tra's for the students 1= 2
< ,''roaches to anguage testing 12
<.1 Background 12
<.< The essa"8transation a''roach 12
<. 3 The structuraist a''roach 12
<.= The integrati-e a''roach 1?
<.2 The communicati-e a''roach 1/
3 4bJecti-e testing <2 ?
3.1 +ubJecti-e and obJecti-e testing <2
3.< 4bJecti-e tests <?
3.3 Auti'e8choice items! genera <7
3.= Auti'e8choice items! the stemK
the correct o'tionKthe distractors 33
3.2 7riting the test 33
= Tests of grammar and usage 3=
=.1 (ntroduction 3=
=.< Auti'e8choice grammar items! 7
item t"'es 3=
=.3 Constructing muti'e8choice items 37
=.= Constructing error8recognition
muti'e8choice items 3/
=.2 Constructing rearrangement items =1
=.? Constructing com'etion items =<
=.7 Constructing transformation items =?
=.0 Constructing items in-o-ing the
changing of words =0
=./ Constructing >broken sentence> items =/
=.13 Constructing 'airing and
matching items =/
=.11 Constructing combination and
addition items 23
Testing -ocabuar" 21
2.1 +eection of items 21
2.< Auti'e8choice items 9,: 2<
2.3 Auti'e8choice items 9B: 882?
2.= +ets 9associated words: 20
2.2 Aatching items 20
2.? Aore obJecti-e items ?3
2.7 Com'etion items ?<
Listening com'rehension tests *+
?.1 &enera ?=
?.< 5honeme discrimination tests ?2
?.3 Tests of stress and intonation ?0
?.= +tatements and diaogues ?/
?.2 Testing com'rehension through
-isua materia@s 7G
?.? *nderstanding taks and
ectures 0<
4ra 'roduction tests 00
7.1 +ome difficuties in testing the
s'eaking skis 00
7.< #eading aoud 0/
7.3 Con-ersationa e)changes /3
7 = *sing 'ictures for assessing ora
'roduction
7 2 The ora inter-iew
7 ? +ome other techniBues for ora
e)amining
0 Testing reading com'rehension
0 1 The nature of the reading skis
0 < (nitia stages of reading!
matching tests
0.3 (ntermediate and ad-anced88888
stages of reading! matching tests
0.= TrueKfase reading tests
0.2 Auti'e8choice items 9,:! short
te)ts
0.? Auti'e8choice items 9B:!
onger te)ts
0.7 Com'etion items
0.0 #earrangement items
0./ CoLe 'rocedure
0. 13 4'en8ended and misceaneous
items
0.11 Cursor-8reading I
/ Testing the writing skis
/.1 The writing skis
/.< Testing com'osition writing
/.3 +etting the com'osition
/.= &rading the com'osition
/.2 Treatment of written errors
/.? 4bJecti-e tests! mechanics
/.7 4bJecti-e tests! st"e and
register
/.0 Controed writing
=
13 Criteria and t"'es of tests 12/
/< 13.1 Eaidit" 12/
/? 13.< #eiabiit" 1?<
13.3 #eiabiit" -ersus -aidit" 1?=
13< 13.= 6iscrimination 1?2
13.2 ,dministration 1?7
132 13.? Test instructions to the
132 candidate 1?0
13.7 Backwash effects 173
137 13.0 T"'es of tests 171
113 11 (nter'reting test scores 17=
113 11.1 ;reBuenc" distribution 17=
11.< Aeasures of centra tendenc" 172
11? 1 1 .3 Aeasures of dis'ersion 17?
11.= (tem ana"sis 170
117 11.2 Aoderating 102
1<= 11.? (tem cards and banks 102
1</
131 +eected bibiogra'h" 1 00
(nde) 1 /1
133
133
132
132
13?
130
1==
1=/
123
12<
12=
(ntroduction to anguage testing
1.1 Testing and
, arge number of e)aminations in the 'ast ha-e encouraged a tendenc" ti
teaching se'arate testing from teaching. Both testing and teaching are so cose"
interreated that it is -irtua" im'ossibe to work in either fied without
being constant" concerned with the other. Tests ma" be constructed
'rimari" as de-ices to reinforce earning and to moti-ate the student or
'rimari" as a means of assessing the student>s 'erformance in the
anguage. (n the former case. the test is geared to the teaching that has
taken 'ace. whereas in the atter case the teaching is often geared arge"
to the test. +tandardised tests and 'ubic e)aminations. in fact. ma" e)ert
such a considerabe infuence on the a-erage teacher that the" are often
instrumenta in determining the kind of teaching that takes 'ace before
the test.
, anguage test which seeks to find out what candidates can do with
anguage 'ro-ides a focus for 'ur'osefu. e-er"da" communication
acti-ities. +uch a test wi ha-e a more usefu effect on the earning of a
'articuar anguage than a mechanica test of structure. (n the 'ast e-en
good tests of grammar. transation or anguage mani'uation had a
negati-e and e-en harmfu effect on teaching. , good communicati-e test
of anguage. howe-er. shoud ha-e a much more 'ositi-e effect on earnin
and teaching and shoud genera" resut in im'ro-ed earning habits.
Com'are the effect of the foowing two t"'es of test items on the
teaching of Engish!
1
%ou wi now hear a short tak. Listen carefu" and com'ete the
foowing 'aragra'h b" writing one word on each ine!
(f "ou go to ... . on hoida". "ou ma" ha-e to wait a ong time
at the ... . as the 'orters are on ...
. Howe-er. it wi
not be as bad as at most ...
9etc.:
< %ou wi now hear a short weather and tra-e re'ort on the radio.
Beforr
"ou isten to the tak. choose one of the 'aces ,. B or C and 'ut a cross
9M: in the bo) ne)t to the 'ace "ou choose.
5ace ,8 +outhern +'ain 9b" air:.
5ace B 8 Northern ;rance 9b" car:.

5ace C 8 +witLerand 9b" rai:.
5ut crosses in the correct bo)es beow after istening to the 'rogramme.
#emember to concentrate on" on the information a''ro'riate to the 'ace
which "ou ha-e chosen.
No tra-e 'robems
, few tra-e 'robems
+erious tra-e 'robems
+unn"
;ine but coud"
#ain
&ood hotes
,-erage hotes
5oor hotes
9etc.:
;ortunate". a number of we8known 'ubic e)amining bodies now
attem't to measure the candidates> success in 'erforming 'ur'osefu and
ree-ant tasks and their actua abiit" to communicate in the anguage. (n
this sense. such e)aminations undoubted" e)ert a far more beneficia
infuence on s"abuses and teaching strategies than in the 'ast. Howe-er.
e-en the best 'ubic e)aminations are sti 'rimari" instruments for
measuring each student>s 'erformance in com'arison with the 'erformance
of other students or with certain estabished norms.
1.< 7h" testG The function indicated in the 'receding 'aragra'h 'ro-ides one of the
answers to the Buestion! 7h" testG But it must be em'hasised that the
e-auation of student 'erformance for 'ur'oses of com'arison or seection is
on" one of the functions of a test. ;urthermore. as far as the 'ractising
teacher is concerned. it shoud rare" be either the soe 'ur'ose or e-en the
chief 'ur'ose of testing in schoos.
,though most teachers aso wish to e-auate indi-idua 'erformance.
the aim of the cassroom test is different from that of the e)terna
e)amination. 7hie the atter is genera" concerned with e-auation for
the 'ur'ose of seection. the cassroom test is concerned with e-auation
for the 'ur'ose of enabing teachers to increase their own effecti-eness b"
making adJustments in their teaching to enabe certain grou's of students
or indi-iduas in the cass to benefit more. Too man" teachers gear their
teaching towards an i8defined Da-erage> grou' without taking into account
the abiities of those students in the cass who are at either end of the scae.
, good cassroom test wi aso he' to ocate the 'recise areas of
difficut" encountered b" the cass or b" the indi-idua student. Just as it is
necessar" for the doctor first to diagnose the 'atient>s iness. so it is
eBua" necessar" for the teacher to diagnose the student>s weaknesses and
difficuties. *ness the teacher is abe to identif" and ana"se the errors a
student makes in handing the target anguage. he or she wi be in no
'osition to render an" assistance at a through a''ro'riate antici'ation.
remedia work and additiona 'ractice.
?
1
The test shoud aso enabe the teacher to ascertain which 'arts of the
anguage 'rogramme ha-e been found difficut b" the cass. (n this wa".
the teacher can e-auate the effecti-eness of the s"abus as we as the
methods and materias he or she is using. The test resuts ma" indicate. fo
e)am'e. certain areas of the anguage s"abus which ha-e not taken
sufficient account of8foreign earner difficuties or which. for some reason.
ha-e been gossed o-er. (n such cases the teacher wi be concerned with
those 'robem areas encountered b" grou's of students rather than bN the
indi-idua student. (f. for e)am'e. one or two students in a cass of 33 or
=3 confuse the 'resent 'erfect tense with the 'resent sim'e tense 9e.g.
aread" see that fim>:. the teacher ma" sim'" wish to correct the error
before mo-ing on to a different area. Howe-er. if se-en or eight students
make this mistake. the teacher wi take this 'robem area into account
when 'anning remedia or further teaching.
, test which sets out to measure students> 'erformances as fair" as
'ossibe without in an" wa" setting tra's for them can be effecti-e" used
to moti-ate them. , we8constructed cassroom test wi 'ro-ide the
students with an o''ortunit" to show their abiit" to 'erform certain task!
in the anguage. 5ro-ided that detais of their 'erformance are gi-en as
soon as 'ossibe after the test. the students shoud be abe to earn from
their weaknesses. (n this wa" a good test can be used as a -auabe teachi
de-ice.
1.3
7hat shoud be
8The de-eo'ment of modern inguistic theor" has he'ed to make anguaO
tested and to what
teachers and testers aware of the im'ortance of ana"sing the anguage
standardG
being tested. Aodern descri'ti-e8grammars 9though not "et 'rimari"
intended for foreign anguage teaching 'ur'oses: are re'acing the oder
Latin8based 'rescri'ti-e grammars! inguists are e)amining the whoe
com'e) s"stem of anguage skis and 'atterns of inguistic beha-iour.
(ndeed. anguage skis are so com'e) and so cose" reated to the tota
conte)t in which the" are used as we as to man" non8inguistic skis
9gestures. e"e8mo-ements. etc.: hat it ma" often seem im'ossibe to
se'arate them for the 'ur'ose of an" kind of assessment. , 'erson awa..
s'eaks and communicates in a 'articuar situation at a 'articuar time.
7ithout this kind of conte)t. anguage ma" ose much of its meaning.
Before a test is constructed. it is im'ortant to Buestion the standards
which are being set. 7hat standards shoud be demanded of earners of
foreign anguageG ;or e)am'e. shoud foreign anguage earners after a
certain number of months or "ears be e)'ected to communicate with the
same ease and fuenc" as nati-e s'eakersG ,re certain habits of second
anguage eaine%s8regar>d8ed asmistakes8when these8saute8habits8wou nn.
constitute mistakes when beonging to nati-e s'eakersG 7hat. indeed. is
Dcorrect> EngishG
E)aminations in the written anguage ha-e in the 'ast set artificia
standards e-en for nati-e s'eakers and ha-e often demanded skis simi to
those acBuired b" the great Engish essa"ists and critics. (n imitating t
anguage e)aminations of written Engish. howe-er. second anguage
e)aminations ha-e 'ro-ed far more unreaistic in their e)'ectations of tt .
'erformances of foreign earners. who ha-e been reBuired to rewrite son of
the greatest iterar" master'ieces in their own words or to write origin! essa"s
in anguage be"ond their ca'acit".
1.= Testing the ;our maJor skis in communicating through anguage are often broad"
anguage skis defined as istening. istening and s'eaking. reading and writing. (n man"
situations where Engish is taught for genera 'ur'oses. these skis shoud
be carefu" integrated and used to 'erform as man" genuine"
communicati-e tasks as 'ossibe. 7here this is the case. it is im'ortant for
the test writer to concentrate on those t"'es of test items which a''ear
direct" ree-ant to the abiit" to use anguage for rea8ife communication.
es'ecia" in ora interaction. Thus. Buestions which test the abiit" to
understand and res'ond a''ro'riate" to 'oite reBuests. ad-ice.
instructions. etc. woud be 'referred to tests of reading aoud or teing
stories. (n the written section of a test. Buestions reBuiring students to
write etters. memos. re'orts and messages woud be used in 'ace of man"
of the more traditiona com'ositions used in the 'ast. (n istening and
reading8tee8B stiatas8in8whic >maw their abiit"8to8e)tract
s'ecific information of a 'ractica nature woud be 'referred to Buestions
testing the com'rehension of unim'ortant and irree-ant detais. ,bo-e
a. there woud be no rigid distinction drawn between the four different
skis as in most traditiona tests in the 'ast. a test of reading now being
used to 'ro-ide the basis for a reated test of writing or s'eaking.
+uccess in traditiona tests a too often sim'" demonstrates that the
student has been abe to 'erform we in the test he or she has taken 8 and
-er" itte ese. ;or e)am'e. the traditiona reading com'rehension test
9often in-o-ing the com'rehension of meaningess and irree-ant bits of
information: measures a ski which is more cose" associated with
e)aminations and answering techniBues than with the abiit" to read or
scan in order to e)tract s'ecific information8for a 'articuar 'ur'ose. (n this
sense. the traditiona test ma" te us reati-e" itte about the student>s
genera fuenc" and abiit" to hande the target anguage. athough it ma"
gi-e some indication of the student>s schoastic abiit" in some of the skis
he or she needs as a student.
7a"s of assessing 'erformance in the four maJor skis ma" take the
form of tests of!
8 istening 9auditor": com'rehension. in which short utterances. diaogues.
taks and ectures are gi-en to the testees@
8 s'eaking abiit". usua" in the form of an inter-iew. a 'icture
descri'tion. roe 'a"8 and a 'robem8so-ing task in-o-ing 'air work or
grou' work@
8 reading com'rehension. in which Buestions are set to test the students>
abiit" to understand the gist of a te)t and to e)tract ke" information on
s'ecific 'oints in the te)t@ and
8 writing abiit". usua" in the form of etters. re'orts. memos. messages.
instructions. and accounts of 'ast e-ents. etc.
(t is the test constructor>s task to assess the reati-e im'ortance of
these skis at the -arious e-es and to de-ise an accurate means of
measuring the student>s success in de-eo'ing these skis. +e-era test
writers sti consider that their 'ur'ose can best be achie-ed if each
se'arate ski can be measured on its own. But it is usua" e)treme"
difficut to se'arate one ski from another. for the -er" di-ision of the four
skis is an artificia one and the conce't itsef constitutes a -ast o-er8
sim'ification of the issues in-o-ed in communication.
+
1.2 Testing anguage (n an attem't to isoate the anguage areas earnt. a considerabe number
areas of tests incude sections on!
8 grammar and usage.
8 -ocabuar" 9concerned with word meanings. word formation and
coocations:@
8 'honoog" 9concerned with 'honemes. stress and intonation:.
&ests o! grammar andThese tests measure students> abiit" to recognise a''ro'riate grammatica
usage
forms and to mani'uate structures.
>,though it 91: ... . Buite warm now. 9<: ... . wi change
ater toda". B" tomorrow morning. it 93: ... . much coder and
there ma" e-en be a itte snow ... 9etc.:
91: ,. seems B. wi seem C. seemed 6. had seemed
9<: ,. weather B. the weather C. a weather 6. some
weather
93: ,. is B. wi go to8be C. is going to be 6. woud be 9etc.:
Note that this 'articuar t"'e of Buestion is caed a multiple$choice
item. The term multiple$choice is used because the students are reBuired
to
seect the correct answer from a choice of se-era answers. 94n" one
answer is norma" correct for each item.: The word item is used in
'reference to the word ,uestion because the atter word s"tggests the
interrogati-e form@ man" test items are. in fact. written in the form of
statements.
Not a grammar tests. howe-er. need com'rise muti'e8choice items.
The foowing com'etion item iustrates Just one of se-era other t"'es of O
grammar items freBuent" used in tests!
,! . does Eictor Luo ... G
B! ( think his fat is on the outskirts of $uaa Lum'ur.
9etc.:
&ests o! voca"ular# , test of -ocabuar" measures students> knowedge of the meaning of
certain words as we as the 'atterns and coocations in which the" occur.
+uch a test ma" test their active -ocabuar" 9the words the" shoud be abe
to use in s'eaking and in writing: or their passive -ocabuar" 9the words
the" shoud be abe to recognise and understand when the" are istening to
someone or when the" are reading:. 4b-ious". in this kind of test the
method used to seect the -ocabuar" items 91 sam'ing: is of the utmost
im'ortance. .
(n the foowing item students are instructed to circe the etter at the side
of the word I7hich best com'etes ttie Isentence.I I
6id "ou ...
. that book from the schoo ibrar"G
,. beg B. borrow C. hire
6. end
E. ask
(n another common t"'e of -ocabuar" test students are gi-en a
'assage to read and reBuired to re'ace certain words isted at the end of the
'assage with their eBui-aents in the 'assage.
Tests o! phonolog# Test items designed to test 'honoog" might attem't to assess the foowing
sub8skis! abiit" to recognise and 'ronounce the significant sound
contrasts of a anguage. abiit" to recognise and use the stress 'atterns of a
/
anguage. and abiit" to hear and 'roduce the meod" or 'atterns of the
tunes of a anguage 9i.e. the rise and fa of the -oice:.
(n the foowing item. students are reBuired to indicate which of the
three sentences the" hear are the same!
'po-en.
Just ook at that arge shi' o-er there.
Just ook at that arge shee' o-er there.
Just ook at that arge shi' o-er there.
,though this item. which used to be 'o'uar in certain tests. is now -er"
rare" incuded as a se'arate item in 'ubic e)aminations. it is sometimes
a''ro'riate for incusion in a cass 'rogress or achie-ement test at an
eementar" e-e. +uccessfu 'erformance in this fied. howe-er. shoud not
8888berc8tar8=et8N8as necessari" indicating an8abiit" to s'eak. 8
1.? Lwaguage skis and (tems designed to test areas of grammar and -ocabuar" wi be e)amined
anguage eements
in detai ater in the a''ro'riate cha'ters. The Buestion now 'osed is! to
what e)tent shoud we concentrate on testing students> abiit" to hande
these eements of the anguage and to what e)tent shoud we concentrate
on testing the integrated skisG 4ur attitude towards this Buestion must
de'end on both the e-e and the 'ur'ose of the test. (f the students ha-e
been earning Engish for on" a reati-e" brief 'eriod. it is high" ike"
that we sha be chief" concerned with their abiit" to hande the anguage
eements correct". Aoreo-er. if the aim of the test is to sam'e as wide a
f i ed as 'ossibe. a batter" of tests of the anguage eements wi be usefu
not on" in 'ro-iding a wide co-erage of this abiit" but aso in ocating
'articuar 'robem areas. Tests designed to assess master" of the anguage
eements enabe the test writer to determine e)act" what is being tested and to
're8test items.
Howe-er. at a e-es but the most eementar". it is genera"
ad-isabe to incude test items which measure the abiit" to communicate in
the target anguage. How im'ortant. for e)am'ie. is the abiit" to
discriminate between the 'honemes Ki!K and Kt
KG E-en if the" are confused
.b" a testee and he or she sa"s
(oo- at that sheep sailing slowl# out o! the
har"our, it is unike" that misunderstanding wi resut because the conte)t
'ro-ides other cues to the meaning. , anguages contain numerous so8
caed Dredundancies> which he' to o-ercome 'robems of this nature.
;urthermore. no student can be described as being 'roficient in a
anguage sim'" because he or she is abe to discriminate between two
sounds or has mastered a number of structures of the anguage. +uccessfu
communication in situations which simuate rea ife is the best test of
master" of a anguage. (t can thus be argued that fuenc" in Engish 8 a
'erson>s abiit" to e)'ress facts. ideas. feeings and attitudes cear" and
with ease. in s'eech or in writing. and the abiit" to understand what he or
she hears and reads 8 can best be measured b" tests which e-auate
'erformance in the anguage skis. Listening and reading com'rehension
tests. ora inter-iews and etter8writing assess 'erformance in those
anguage skis used in rea ife.
Too great a concentration on the testing of the anguage eements ma"
indeed ha-e a harmfu effect on the communicati-e teaching of the
anguage. There is aso at 'resent insufficient knowedge about the
weighting which ought to be gi-en to s'ecific anguage eements. How
im'ortant are artices. for e)am'e. in reation to 're'ositions or
13
'ronounsG +uch a Buestion cannot be answered unti we know wore abw8B the
degrees of im'ortance of the -arious eements at different stages of earning
a anguage.
1.7 #ecognition and Aethods of testing the recognition of correct words and forms of ae
'roduction
often take the foowing form in tests!
Choose the correct answer and write ,. B. C or 6.
(>-e been standing here ... . haf an hour.
,. since B. during C. whie 6. for
This muti'e8choice test item tests students> abiit" to recognise t tc
correct form! this abiit" is ob-ious" not Buite the same as tGe )bm" t o
'roduce and use the correct form in rea8ife situations. Howe-er. this t"'e
of item has the ad-antage of being eas" to e)amine statistica".
(f the four choices were omitted. the item woud come coser to being a
test of production.
Com'ete each bank with the correct word.
(>-e been standing here ... . haf an hour.
+tudents woud then be reBuired to 'roduce the correct answer /0 !or1. (n
man" cases. there woud on" be one 'ossibe correct answer. but
'roduction items do not awa"s guarantee that students wi dea with the
s'ecific matter the e)aminer had in mind 9as most recognition items =3:. (n
this 'articuar case the test item is not entire" satisfactor". for students are
com'ete" Justified in writing nearl#2almost2over in the bank. (t woud
not then test their abiit" to discriminate between !or with 'eriods of time
9e.g. !or hal! an hour, !or two #ears1 and since with 'oints of time 9e.g.
since 2.30, since Christmas1.
The foowing e)am'es aso iustrate the difference between testing
recognition and testing 'roduction. (n the first. students are instructed to
choose the best re'" in List B for each sentence in List , and to write the
etter in the s'ace 'ro-ided. (n the second. the" ha-e to com'ete a
diaogue.
9i: List , List B
1. 7hat>s the forecast for
a+oon after unch. ( think.
tomorrowG ...
<. 7oud "ou ike to go swimmingG
b7e can take our
umbreas.
3. 7here sha we goG ...
c, afternoon.
=. ;ine. 7hat time sha we set offG
d%es. that>s a good ism.
2. How ong sha we s'end thereG e(t> be Buite Not.
?. 7hat sha we do if it rainsG ...
fHow about Cearwater
Ba"G
9ii:
7rite B>s 'art in the foowing diaogue.
1.
,> 7hat>s the forecast for tomorrowG
a! (t> be Buite hot.
<. ,! 7oud "ou ike to go swimmingG
a!
11
3. ,! 7here sha we goG
0!
9etc.:
, good anguage test ma" contain either recognition8t"'e items or.
'roduction8t"'e items. or a combination of both. Each t"'e has its
uniBue functions. and these wi be treated in dctai ater.
1.0 5robems of The actua Buestion of what is to be incuded in a test is often difficut
sam'ing sim'" because a master" of anguage skis is being assessed rather than
areas of knowedge 9i.e. content: as in other subJects ike geogra'h".
'h"sics. etc. ,though the construction of a anguage test at the end of the
f i rst or second "ear of earning Engish is reati-e" eas" if we are famiiar
with the s"abus8co-ered. the construction of a test at a fair" ad-anced
e-e where the s"abus is not cear" defined is much more difficut.
The onger the test. the more reiabe a measuring instrument it wi be
9athough ength. itsef. is no guarantee of a good test:. ;ew students
woud want to s'end se-era hours being tested 8 and indeed this woud be
undesirabe both for the tester and the testees. But the construction of
short tests which function efficient" is often a difficut matter. +am'ing
now becomes of 'aramount im'ortance. The test must co-er an adeBuate
and re'resentati-e section of those areas and skis it is desired to test.
(f a the students who take the test ha-e foowed the same earning
'rogramme. we can sim'" choose areas from this 'rogramme. seeking to
maintain a carefu baance between tense forms. 're'ositions. artices.
e)ica items. etc. ,bo-e a. the kind of anguage to be tested woud be
the anguage used in the cassroom and in the students> immediate
surroundings or the 8anguage reBuired for the schoo or the work for which
the student is being assessed.
(f the same mother8tongue is shared b" a the testees. the task of
sam'ing is made sight" easier e-en though the" ma" ha-e attended
different schoos or foowed different courses. The" wi a e)'erience
'robems of a simiar nature as a resut of the interference of their first8
anguage habits. (t is not a difficut matter to identif" these 'robem areas
and to incude a cross8section of them in the test. 'articuar" in those
sections of the test concerned with the anguage eements. The foowing
two e)am'es based on interference of first8anguage habits wi suffice at
this stage. The first e)am'e concerns the use of the 'resent sim'e for the
'resent 'erfect tense! man" students from certain anguage backgrounds
write such sentences as &elevision e4ists onl# !or the last !ort# or
5!t# #ears
instead of &elevision has e4isted onl# !or the last !ort# or 5!t#
#ears.
, test
item based on this 'robem area might be!
7rite down ,. B. C. 6 or E according to the best aternati-e needed to
com'ete the sentence.
Tee-ision ... . on" for the ast fift" "ears.
, e)ists 6. e)isted
B. was e)isting
E. is e)isting
C. has e)isted
The second e)am'e has been taken from a test of -ocabuar" and
concerns confusion in the use of
loo- !or it is directed chief" at ,rabic
and Chinese earners of Engish. The word !etched has been incuded in the
ist of choices because there is no distinction in ,rabic between the two
1<
conce'ts e)'ressed in Engish b" !etch and loo- !or, whie account has aso
been taken of the difficut" man# Chinese earners e)'erience as a resut of
the ack of distinction in Aandarin between loo- !or and 5nd. Choices 6
and E might aso a''ear 'ausibe to other students unsure of the correct
use of loo- !or.
>Here>s "our book. John. %ou eft it on m" desk.>
>Thanks. (>-e ... . it e-er"where.>
,. ooked for 6. attended to
0. fetched E. watched.o-er
C. found
(t must be em'hasised that items based on contrasti-e ana"sis can
on" be used effecti-e" when the students come from the same anguage
area. (f most of them do not share the same first anguage. the test must be
uni-ersa b" nature and sam'e a fair cross8section of the anguage. (t wi
scarce" matter then if students from certain anguage areas find it easier
than others! in actua anguage.earning situations the" ma" ha-e an
ad-antage sim'" because their first anguage ha''ens to be more cose"
reated to Engish than certain other anguages are. ;ew woud wish to
den" that. gi-en the same anguage8earning conditions. ;rench students
earning Engish wi e)'erience fewer difficuties than their Chinese
counter'arts.
Before starting to write an" test items. the test constructor shoud
draw u' a detaied tabe of s'ecifications showing as'ects of the skis being
tested and gi-ing a com'rehensi-e co-erage of the s'ecific anguage
eements to be incuded. , cassroom test shoud be cose" reated to the
ground co-ered in the cass teaching. an attem't being made to reate the
different areas co-ered in the test to the ength of time s'ent on teaching
those areas in cass. There is a constant danger of concentrating too much
on testing those areas and skis which most easi" end themse-es to being
tested. (t ma" be he'fu for the teacher to draw u' a rough in-entor" of
those areas 9usua" grammatica features or functions and notions: which
he or she wishes to test. assigning to each one a 'ercentage according to
im'ortance. ;or e)am'e. a teacher wishing to construct a test of grammar
might start b" e)amining the reati-e weighting to be gi-en to the -arious
areas in the ight of the teaching that has Just taken 'ace! sa". the. contrast
between the 'ast continuous and 'ast sim'e tenses 9=3 'er cent:. artices
912 'er cent:. time 're'ositions 912 'er cent:. wish and
hope 913 'er cent:.
concord 913 'er cent:. the infiniti-e of 'ur'ose 913 'er cent:.
,nother teacher wishing to ado't a more communicati-e a''roach to
anguage testing might consider the foowing s'ecifications in the ight of
the earning 'rogramme8greeting 'eo'e 928'er cent:.. int oducing onesef
92 'er cent:. describing 'aces 912 'er cent:. taking about the future 9<3
'er cent:. making suggestions 92 'er cent:. asking for information 9<3 'er
cent:. understanding sim'e instructions 912 'er cent:. taking about 'ast
e-ents 912 'er cent:. 9(t must be em'hasised that these ists are mere" two
e)am'es of the kinds of in-entories which can be drawn u' beforehand
and are not intended to re'resent a 'articuar set of 'riorities.: (n e-er"
case. it is im'ortant that a test refects the actua teaching and the course
being foowed. (n other words. if a more traditiona. structura a''roach
to anguage earning has been ado'ted. the test s'ecifications shoud
cose" refect such a structura a''roach. (f. on the other hand. a
communicati-e a''roach to anguage earning has been ado'ted. the test
13
i
g.B ee fr
'
Notes and nre s
1=
(
s'ecifications shoud be based on the t"'es of anguage tasks incuded in the
earning 'rogramme. (t is cear" unfair to administer a test de-ised entire"
aong communicati-e ines to those students who ha-e foowed a course
concentrating on the earning of structures and grammar.
, good test shoud ne-er be constructed in such a wa" as to tra' the students
into gi-ing an incorrect answer. 7hen techniBues of error ana"sis are used.
the setting of deiberate tra's or 'itfas for unwar" students shoud be
a-oided. Aan" testers. themse-es. are caught out b"
constructing test items which succeed on" in tra''ing the more abe students.
Care shoud be taken to a-oid tra''ing students b" incuding grammatica and
-ocabuar" items which ha-e ne-er been taught.
(n the foowing e)am'e. students ha-e to seect the correct answer
9C:.8but8ti88P8whoe8itenris8constructed so as to tra' them into making88888
choice B or 6. 7hen this item actua" a''eared in a test. it was found that
the more 'roficient students. in fact. chose B and 6. as the" had
de-eo'ed the correct habit of associating the tense forms ha-e seen and ha-e
"een seeing with since and !or. The" had not been taught the com'ete
'attern 9as used in this sentence:. +e-era of the ess 'roficient students. who
had not earnt to associate the 'erfect tense forms with since and !or, chose
the Dcorrect> answer.
7hen ( met Tim "esterda". it was the first time ( ... . him since
Christmas.
,. saw C. had seen
B. ha-e seen 6. ha-e been seeing
+imiar". the foowing item tra''ed the more 'roficient students in
a
grou' b" encouraging them to consider the correct answer. Dsafet">. as too sim'e to be
right. Aan" of these students seected the res'onse
Dsaturation> since the" knew -ague" that this word was concerned with immersion in water.
The ess 'roficient students. on the other hand. sim'" chose Dsafet"> without further
thought.
The animas tried to find ... . from the fire b" running into the
ake.
,. sanitation C. saturation
B. safet" 6. sautation
To summarise. a tests shoud be constructed 'rimari" with the
intention of finding out what students know 8 not of tra''ing them. B"
attem'ting to construct effecti-e anguage tests. the teacher can gain a
dee'er insight into the anguage he or she
is testing and
the
anguage8
earning 'rocesses in-o-ed.
Auti'e8choice items of this nature ha-e ong been used in the *nited +tates b"
such we8known testing organisations as T4E;L /&est o! English as a 6oreign
(anguage, Educationa Testing +er-ice. 5rinceton. New Jerse": and the
Michigan &est o! English (anguage %ro5cienc# 9*ni-ersit" of Aichigan.
,nn ,rbor. Aichigan: to test grammar and -ocabuar". Auti'e8choice items
ha-e aso been wide" used in modern anguage testing in Britain and esewhere
throughout the word. #obert La8do /(anguage &esting, Longman 1/?1.
1/?=: was one of the first to de-eo' the muti'e8choice techniBue in testing the
s'oken anguage.
,''roaches to anguage testing
<.1 Background Language tests can be rough" cassified according to four main a''roaches
to testing! 9i: the essa"8transation a''roach@ 9ii: the structuraist
a''roach@ 9iii: the integrati-e a''roach@ and 9i-: the communicati-e
a''roach. ,though these a''roaches are isted here in chronoogica.
order. the" shoud not be regarded as being strict" confined to certain
'eriods in the de-eo'ment of anguage testing. Nor are the four
a''roaches awa"s mutua" e)cusi-e. , usefu test wi genera"
incor'orate features of se-era of these a''roaches. (ndeed. a test ma"
ha-e certain inherent weaknesses sim'" because it is imited to one
a''roach. howe-er attracti-e that a''roach ma" a''ear.
<.< The essa"8
This a''roach is common" referred to as the 're8scientific stage of
transation a''roach anguage testing. No s'ecia ski or e)'ertise in testing is reBuired! the
subJecti-e Judgement of the teacher is considered to be of 'aramount
im'ortance. Tests usua" consist of essa" writing. transation. and
gramrr...tica ana"sis 9often in the form of comments a"out the anguage
being earnt:. The tests aso ha-e a hea-" iterar" and cutura bias. 5ubic
e)aminations 9e.g. secondar" schoo ea-ing e)aminations: resuting from
the essa"8transation a''roach sometimes ha-e an aura ora com'onent at
the u''er intermediate and ad-anced e-es 8 though this has sometimes
been regarded in the 'ast as something additiona and in no wa" an
integra 'art of the s"abus or e)amination.
<.3 The structuraist
This a''roach is characterised b" the -iew that anguage earning is chief"
a''roach concerned with the s"stematic acBuisition of a set of habits. (t draws on the
work of structura inguistics. in 'articuar the im'ortance of contrasti-e
ana"sts and8f the need to i enh " an measure the earner> s master i4f the
se'arate eements of the target anguage! 'honoog". -ocabuar" and
grammar. +uch master" is tested using words and sentences com'ete"
di-orced from an" conte)t on the grounds that a arger sam'e of anguage
forms can be co-ered in the test in a com'arati-e" short time. The skis of
istening. s'eaking. reading and writing are aso se'arated from one
another as much as 'ossibe because it is considered essentia to test orse
thing at a time.
+uch features of the structuraist a''roach are. of course. sti -aid for
certain t"'es of test and for certain 'ur'oses. ;or e)am'e. the desire to
concentrate on the testees> abiit" to write b" attem'ting to se'arate a
12 8
;
<.= The (ntegrati-e
a''roach
1?
i
com'osition test from reading 9i.e. b" making it who" inde'endent of the
abiit" to read ong and com'icated instructions or -erba stimui: is
commendabe in certain res'ects. (ndeed. there are se-era features of this
a''roach which merit consideration when constructing an" good test.
The 's"chometric a''roach to measuretent with its em'hasis on
reiabiit" and obJecti-it" forms an integra 'art of structuraist testing.
5s"chometrists ha-e been abe to show cear" that such traditiona
e)aminations as essa" writing are high" subJecti-e and unreiabe. ,s a resut.
the need for statistica measures of reiabiit" and -aidit" is
considered to be of the utmost im'ortance in testing! hence the 'o'uarit"
of>the muti'e8choice item 8 a t"'e of item which ends itsef admirab" to
statistica ana"sis.
,t this 'oint. howe-er. the danger of confusing
methods of testing
with 8a''ruachesto8testing8shoud be8st"8ssed. The issue is not basica" a
Buestion of muti'e8choice testing -ersus communicati-e testing. There is
sti a imited use for muti'e8choice items in man" communicati-e tests.
es'ecia" for reading and istening com'rehension 'ur'oses. E)act" the
same argument can be a''ied to the use of se-era othei item t"'es.
This a''roach in-o-es the testing of anguage in conte)t and is thus
concerned 'rimari" with meaning and the tota communicati-e effect of
discourse. ConseBuent". integrati-e tests do not seek to se'arate anguage
skis into neat di-isions in order to im'ro-e test reiabiit"! instead. the"
are often designed to assess the earner>s abiit" to use two or more skis
simutaneous". Thus. integrati-e tests are concerned with a goba -iew of
'roficienc" 8 an under"ing anguage com'etence or Dgrammar of
e)'ectanc"I. which it is argued e-er" earner 'ossesses regardess of the
'ur'ose for which the anguage is being earnt. (ntegrati-e testing in-o-es
Dfunctiona anguage>< but not the use of functiona anguage. (ntegrati-e
tests are best characterised b" the use of doLe testing and of dictation.
4ra inter-iews. transation and essa" writing are aso incuded in man"
integrati-e tests 8 a 'oint freBuent" o-erooked b" those who take too
narrow a -iew of integrati-e testing.
The 'rinci'e of doLe testing is based on the &estat theor" of
Dcosure> 9cosing ga's in 'atterns subconscious":. Thus. doLe tests
measure the reader>s abiit" to decode odP >interru'ted> or Dmutiated> messages b" making the
most acce'tabe substitutions from a the conte)tua cues a-aiabe. E-er" nth word is
deeted in a te)t 9usua" e-er" fifth. si)th or se-enth word:. and students ha-e to com'ete
each ga' in the te)t. using the most a''ro'riate word. The foowing is an e)tract from an
ad-anced8
e-e doLe 'assage in which e-er" se-enth word has been deeted!
The mark assigned to a student ... . surrounded b" an area of
uncertaint" ... . is the cumuati-e effect of a ... . of
sam'ing errors. 4ne sam'e of ... . student>s beha-iour is
e)hibited on one ... . occasion in res'onse to one
sam'e ... . set b" one sam'e of e)aminers ...
'ossib" marked b" one other. Each ... . the sam'ing errors is
amost insignificant ... . itsef.8Howe-er. when each sam'ing
error ... . added to the others. the tota ... . of 'ossibe
sam'ing errors becomes significant.
The te)t used for the doLe test shoud be ong enough to aow a
reasonabe number of deetions 8 idea" =3 or 23 banks. The more banks
contained in the te)t. the more reiabe the doLe test wi genera" 'ro-e.
There are two methods of scoring a doLe test! one mark ma" be
awarded for each accepta"le answer or ese one mark ma" be awarded for
each er.icr answer. Both methods ha-e been found reiabe! some argue
that the former method is -er" itte better than the atter and does not
rea" Justif" the additiona work entaied in defining what constitutes an
acce'tabe answer for each item. Ne-ertheess. it a''ears a fairer test for
the student if an" reasonabe eBui-aent is acce'ted. (n addition. no
student shoud be 'enaised for miss'eings uness a word is so bad" s'et
that it cannot be understood. &rammatica errors. howe-er. shoud be
'enaised in those doLe tests which are designed to measure famiiarit"
with the grammar of the anguage rather than reading.
7here 'ossibe. students shoud be reBuired to fi in each bank in the
te)t it.ef. This 'rocedure a''ro)imates more cose" to the rea8ife tasks
in-o-ed than an" method which reBuires them to write the deeted items
on a se'arate answer sheet or ist. (f the te)t chosen for a doLe test
contain. a ot of facts or if it concerns a 'articuar subJect. some students
ma" be abe to make the reBuired com'etions from their background
knowedge without understanding much of the te)t. ConseBuent". it is
essentia in doLe tests 9as in other t"'es of reading tests: to draw u'on a
subJect which is neutra in both content and anguage -ariet" used. ;ina".
it is awa"s ad-antageous to 'ro-ide a Dead8in>! thus no deetions shoud he
made in the first few sentences so that the students ha-e a chance to
become famiiar with the author>s st"e and a''roach to the subJect of the
te)t.
CoLe 'rocedure as a measure of reading difficut" and reading
com'rehension wi be treated brief" in the ree-ant section of the cha'ter
on testing reading com'rehension. #esearch studies. howe-er. ha-e shown
that 'erformance on doLe tests correates high" with the istening. writing
and s'eaking abiities. (n other words. doLe testing is a good indicator of
genera inguistic abiit". incuding the abiit" to use anguage a''ro'riate"
according to 'articuar inguistic and situationa conte)ts. (t is argued that
three t"'es of knowedge are reBuired in order to 'erform successfu" on a
doLe test! inguistic knowedge. te)tua knowedge. and knowedge of the
word.> ,s it resut of such research findings. doLe tests are now used not
on" in genera achie-ement and 'roficienc" tests but aso in some
cassroom 'acement tests and diagnostic tests.
6ictation. another maJor t"'e of integrati-e test. was 're-ious"
regarded soe" as a means of measuring students> skis of istening
OcoOm'rehension. Thus. the com'e) eements in-o-ed in tests of dictation
were arge" o-erooked unti fair" recent". The inhegratte8a
is tn-oI8 Q-e
in tests of dictation incude auditor" discrimination. the auditor" memor"
s'an. s'eing. the recognition of sound segments. a famiiarit" with the
grammatica and e)ica 'atterning of the anguage. and o-era te)tua
com'rehension. *nfortunate". howe-er. there is no reiabe wa" of
assessing the reati-e im'ortance of the different abiities reBuired. and
each error in the dictation is usua" 'enaised in e)act" the same wa".
6ictation tests can 'ro-e good 'redictors of goba anguage abiit"
e-en though some recent research< has found that dictation tends to
measure ower8order anguage skis such as straightforward
17
/
com'rehension rather than the higher8order skis such as inference. The
dictation of onger 'ieces of discourse 9i.e. 7 to 13 words at a time: is
recommended as being 'referabe to the dictation of shorter word grou's
9i.e. three to fi-e words at a time: as in the traditiona dictations of the 'ast.
*sed in this wa". dictation in-o-es a d"namic 'rocess of ana"sis b"
s"nthesis. drawing on a earner>s >grammar of e)'ectanc"I and resuting in
the constructi-e 'rocessing of the message heard.
(f there is no cose reationshi' between the sounds of a anguage and
the s"mbos re'resenting them. it ma" be 'ossibe to understand what is
being s'oken without being abe to write it down. Howe-er. in Engish.
where there is a fair" cose reationshi' between the sounds and the
s'eing s"stem. it is sometimes 'ossibe to recognise the indi-idua sound
eements without fu" understanding the meaning of what is s'oken.
(ndeed. some a''ied inguists and teachers argue tha. dictation
encourages the student to focus his or her attention too much on the
indi-idua sounds rather than on the meaning of the te)t as a whoe. +uch
concentration on singe sound segments in itsef is sufficient to im'air the
auditor" memor" s'an. thus making it difficut for the students to retain
e-er"thing the" hear.
7hen dictation is gi-en. it is ad-isabe to read through the whoe
dictation 'assage at a''roaching norma con-ersationE ( s'eed first of a.
Ne)t. the teach8r shoud begin to dictate 9either once or twice: in
meaningfu units of sufficient ength to chaenge the stude. is short8term
memor" s'an. 9+ome teachers mistaken" fee that the" can make the
dictation easier b" reading out the te)t word b" word! this 'rocedure can
be e)treme" harmfu and on" ser-es to increase the difficut" of the
dictation b" obscuring the meaning of each 'hrase.: ;ina". after the
dictation. the whoe 'assage is read once more at sight" sower than
norma s'eed.
The foowing is an e)am'e of 'art of a dictation 'assage. suitabe for
use at an intermediate or fair" ad-anced e-e. The obiBue strokes denote
the units which the e)aminer must obser-e when dictating.
Before the second haf of the nineteenth centur" K the taest bocks of
officesKwere on" three or four store"s high.KK,s business e)'andedK
and the need for office accommodation grew more and more acute.K
architects began to 'an taer buidings.KK7ood and iron. howe-er.K
were not strong enough materias from which to construct ta buidings.KK
;urthermore. the in-ention of stee now made it 'ossibe Kto construct
frames so strong Kthat the" woud su''ort the -er" taest of buidings.
Two other t"'es of integrati-e tests 9ora inter-iews and com'osition
writing: wi he treated at ength ater in this hook. The remaining t"'e of
integrati-e test not "et treated is transation. Tests of transation. howe-er.
tend to be unreiabe because of the com'e) nature of the -arious skis
in-o-ed and the methods of scoring. (n too man" instances. the unreaistic
e)'ectations of e)aminers resut in the setting of high" artificia sentences
and iterar" te)ts for transation. +tudents are e)'ected to dis'a" an abiit"
to make fine s"ntactica Judgements and a''ro'riate e)ica distinctions 8
an abiit" which can on" be acBuired after achie-ing a high degree of
'roficienc" not on" in Engish and the mother8tongue but aso in
com'arati-e st"istics and transation methods.
7hen the tota skis of transation are tested. the test writer shoud
endea-our to 'resent a task which is meaningfu and ree-ant to the
situation of the students. Thus. for e)am'e. students might be reBuired to
write a re'ort in the mother8tongue based on information 'resented in
Engish. (n this case. the test writer shoud constant" be aert to the
com'e) range of skis being tested. ,bo-e a. word8for8word transation of
difficut iterar" e)tracts shoud be a-oided.
<.2 The The communicati-e a''roach to anguage testing is sometimes inked to
communicati-e the integrati-e a''roach. Howe-er. athough both a''roaches em'hasise
a''roach
the im'ortance of the meaning of utterances rather than their form and
structure. there are ne-ertheess fundamenta differences between the two
a''roaches. Communicati-e tests are concerned 'rimari" 9if not tota":
with how anguage is used in communication. ConseBuent". most aim to
incor'orate tasks which a''ro)imate as cose" as 'ossibe to those facing
the students in rea ife. +uccess is Judged in terms of the effecti-eness of
the communication which takes 'ace rather than forma inguistic
accurac". Language >use>3 is often em'hasised to the e)cusion of anguage
Dusage>. D*se> is concerned with how 'eo'e actua"
use anguage for a
mutitude of different 'ur'oses whie >usage> concerns the forma 'atterns
of anguage 9described in 'rescri'ti-e grammars and e)icons:. (n 'ractice.
howe-er. some tests of a communicati-e nature incude the testing of usage
and aso assess abiit" to hande the forma 'atterns of the target anguage.
(ndeed. few su''orters of the communicati-e a''roach woud argue that
communicati-e com'etence can e-er be achie-ed without a considerabe
master" of the grammar of a anguage.
The attem't to measure different anguage skis in communicati-e
tests is based on a -iew of anguage referred to as the di-isibiit"
h"'othesis. Communicati-e testing resuts in an attem't to obtain different
'rofies of a earner>s 'erformance in the anguage. The earner ma". for
e)am'e. ha-e a 'oor abiit" in using the s'oken anguage in informa
con-ersations but ma" score Buite high" on tests of reading
com'rehension. (n this sense. communicati-e testing draws hea-i" on the
recent work on a'titude testing 9where it has ong been caimed that the
most successfu tests are those which measure se'arate" such ree-ant
skis as the abiit" to transate news re'orts. the abiit" to understand radio
broadcasts. or the abiit" to inter'ret s'eech utterances:. The score
obtained on a communicati-e test wi thus resut in se-era measures of
'roficienc" rather than sim'" one o-era measure. (n the foowing tabe.
for e)am'e. the four basic skis are shown 9each with si) bo)es to indicate
the different e-es of students> 'erformances:.
? 2 = 3 < 1
I8L>iDstening
#eading
Listening and s'eaking
7riting
+uch a tabe woud norma" be ada'ted to gi-e different 'rofie.
ree-ant to s'ecific situations or needs. The degree of detai in the -arious
'rofies isted wi de'end arge" on the t"'e of test and the 'ur'ose for
which it is being constructed. The foowing is an e)am'e of one wa" in
which the tabe coud be ada'ted.
>/
i
? 2 = 3 .< 1
Listening to s'eciaist subJect ectures
#eading te)tbooks and Journas
Contributing to seminar discussions
7riting aborator" re'orts
7riting a thesis
;rom this a''roach. a new and interesting -iew of assessment
emerges! name". that it is 'ossibe for a nati-e s'eaker to score ess than a
non8nati-e s'eaker on a test of Engish for +'ecific 5ur'oses 8 sa". on a
stud" skis test of Aedicine. (ris argued that a nati-e s'eaker>s a @.iit"8to
use anguage for the 'articuar 'ur'ose being tested 9e.g. Engish for
stud"ing Aedicine: ma" actua" be inferior to a foreign earner>s abiit".
This is indeed a most contro-ersia caim as it might be Justifiab" argued
that ow scores on such a test are the resut of ack of moti-ation or of
knowedge of the subJect itsef rather than an inferior abiit" to use Engish
for the 'articuar 'ur'ose being tested.
*nike the se'arate testing of skis in the structuraist a''roach.
moreo-er. it is fet in communicati-e testing that sometimes the assessment
of anguage skis in isoation ma" ha-e on" a -er" imited ree-ance to
rea ife. ;or e)am'e. reading woud rare" be undertaken soe" for its
own sake in academic stud" but rather for subseBuent transfer of the
information obtained to writing or s'eaking.
+ince anguage is decoiite)tuaised in 's"chometric8structura tests. it
is often a sim'e matter for the same test to be used goba" for an"
countr" in the word. Communicati-e tests. on the other hand8. must of
necessit" refect the cuture of a 'articuar countr" because of their
em'hasis on conte)t and the use of authentic materias. Not on" shoud
test content he tota" ree-ant for a 'articuar grou' of testees but the
tasks set shoud reate to rea8ife situations. usua" s'ecific to a 'articuar
countr" or cuture. (n the ora com'onent of a certain test written in
Britain and triaed in Ja'an. for e)am'e. it was found that man" students
had e)'erienced. difficut" when the" were instructed to com'ain about
someone smoking. The reason for their difficut" was ob-ious! Ja'anese
'eo'e rare" com'ain. es'ecia" about something the" regard as a fair"
tri-ia matterR ,though unintended. such cutura bias affects the
reiabiit" of the test being administered.
>5erha's the most im'ortant criterion for communicati-e tests is that
the" shoud be based on 'recise and detaied s'ecifications of the needs of
the earners for whom the" are constructed! hence their 'articuar
suitabiit" for the testing of Engish for s'ecific 'ur'oses. Howe-er. it
woud be a mistake to assume that communicati-e testing is best imited to
E+5 or e-en to adut earners with 'articuar" ob-ious short8term goas.
,though the" ma" contain tota" different tasks. communicati-e tests for
"oung earners foowing genera Engish courses are based on e)act" the
same 'rinci'es as those for adut earners intending to enter on high"
s'ec aised courses of a 'rofessiona or academic nature.
;ina". communicati-e testing has introduced the conce't of
Buaitati-e modes of assessment in 'reference to Buantitati-e ones.
Language band s"stems are used to show the earners e-es of
<n
.'erformance in the different skis tested. 6etaied statements of each
'erformance e-e ser-e to increase the reiabiit" of the scoring b"
enabing the e)aminer to make decisions according to carefu" drawn8u'
and we8estabished criteria. Howe-er. an eBua" im'ortant ad-antage of
such an a''roach ies in the more humanistic attitude it brings to anguage
testing. Each student>s 'erformance is e-auated according co his or her
degree of success in 'erforming the anguage tasks set rather than soe" in
reation to the 'erformances of other students. Suaitati-e Jugements are
aso su'erior to Buantitati-e assessments from another 'oint of -iew. 7hen
'resented in the form of brief written descri'tions. the" are of considerabe use
in famiiarising testees and their teachers 9or s'onsors: with much8
needed guidance concerning 'erformance and 'robem areas. Aoreo-er. such
descri'tions are now reati-e" eas" for 'ubic e)amining bodies to 'roduce in
the form of com'uter 'rintouts.
The foowing contents of the 'reiminar" e-e of a we8known test
show how Buaitati-e modes of assessment. descri'tions of 'erformance
e-es@ etc. can he incor'orated in e)amination brochures and guides.2
7#(TTEN EN&L(+H
5a'er 1 8 ,mong the items to be tested are! writing of formaKinforma
etters@ initiating etters and res'onding to them@ writing connected 'rose.
on to'ics ree-ant to an" candidate>s situation. in the form of messages.
notices. signs. 'ostcards. ists. etc.
5a'er < 8 ,mong the items to be tested are! the use of a dictionar"@
abiit" to fi in forms@ abiit" to foow instructions. to read for the
genera meaning of a te)t. to read in order to seect s'ecific information.
+54$EN EN&L(+H
+ection 1 8 +ocia Engish
Candidates must be abe to!
9a: #ead and write numbers. etters. and common abbre-iations.
9b: 5artici'ate in short and sim'e cued con-ersation. 'ossib" using
-isua stimui.
9c: #es'ond a''ro'riate" to e-er"da" situations described in -er"
sim'e terms.
9d: ,nswer Buestions in a directed situation.
+ection < 8 Com'rehension
Candidates must be abe to.
9a: *nderstand the e)act meaning of a sim'e 'Tce of s'eech. and
indicate this com'rehension b"!
8 marking a ma'. 'an. or grid@
888888.881choosiagOthe8nost.a''ro.'riateOof.ae.G..".isuas.@.OOO
8 stating whether or not. or how. the aura stimuus reates to the
-isua@
8 answering sim'e Buestions.
9b: *nderstand the basic and essentia meaning of a 'iece of s'eech too
difficut to be understood com'ete".
+ection 3 8 E)tended +'eaking
Candidates wi be reBuired to s'eak for =28?3 seconds in a situation or
situations ike" to be a''ro'riate in rea ife for a s'eaker at this e-e.
This ma" incude e)'anation. ad-ice. reBuests. a'oogies. etc. but wi
not demand an" use of the anguage in other than mundane and
<1
i
(
'ressing circumstances. (t is assumed at this e-e that no candidate
woud s'eak at ength in rea ife uness it were rea" necessar". so that.
for e)am'e. narrati-e woud not be e)'ected e)ce't in the conte)t of
something ike an e)'anation or a'oog".
,fter isting these contents. the test handbook then describes brief" what a
successfu candidate shoud be abe to do both in the written and s'oken
anguage.
The foowing s'ecifications and format are taken from another wide"
used communicati-e test of Engish and iustrate the o'erations. te)t t"'es
and formats which form the basis of the test. ;or 'ur'oses of com'arison. the
e)am'es incuded here are confined to basic e-e tests of reading and
s'eaking. (t must be em'hasised. howe-er. that s'ecifications for a four
skis are incuded in the.S''ro'riate test handbook. together with other
ree-ant information for 'otentia testees.fi
TE+T+ 4; #E,6(N&
4'erations 8 Basic Le-e
a. +can te)t to ocate s'ecific information.
b. +earch through te)t to decide whether the whoe or 'art is ree-ant to
an estabished need.
c. +earch through te)t to estabish which 'art is ree-ant to an
estabished need.
rR. +earch through te)t to e-auate the content in terms of 're-ious"
recei-ed information.
Te)t T"'es and To'ics 8 Basic Le-e
;orm T"'e
Leafet ,nnouncement
.&uide 6esc i'tion
,d-ertisement Narration
Letter Comment
5ostcard ,necdoteKJoke
;orm #e'ortK+ummar"
+et of instructions
6iar" entr"
Timetabe
Aa'K5an
;ormat
a. 4ne 'a'er of 1 hour. (n addition. candidates are aowed ten minutes
before the start of the e)amination to famiiarise themse-es with the
contents of the source materia. The Buestion 'a'er must not be
ooked at during this time.
b. Candidates wi be 'ro-ided with source materia in the form of
authentic bookets. brochures. etc. This materia ma" be the same at
a e-es.
c. Suestions wi be of the foowing forms!
i: Auti'e choice
ii: True ;ase
iii : 7rite8in 9singe word or 'hrase:
d. Aonoingua or biingua dictionaries ma" be used free".
TE+T 4; 4#,L (NTE#,CT(4N
4'erations 8 Basic Le-e
E)'ressing! thanks
reBuirements
o'inions
comment
attitude
confirmation
a'oog"
wantKneed
information
Narrating!
seBuence of e-ents
Eiciting!
information
directions
ser-ice
9and a areas abo-e:
T"'es of Te)t
,t a e-es candidates ma" be e)'ected to take 'art in diaogue and
muti8'artici'ant interactions.
The interactions wi norma" be of a face8to8face nature but tee'hone
con-ersations are not e)cuded.
The candidate ma" be asked to take 'art in a simuation of an"
interaction deri-ed from the ist of genera areas of anguage use.
Howe-er. he wi not be asked to assume s'eciaised or fantas" roes.
;ormat
The format wi be the same at each e-e.
a. 8rests are di-ided into three 'arts. Each 'art is obser-ed b" an
assessor nominated b" the Board. The assessor e-auates and score
the candidate>s 'erformance but takes no 'art in the conduct of the
test.
b. 5art ( consists of an interaction between the candidate and an
interocutor who wi norma" be a re'resentati-e of the schoo or
centres where the test is hed and wi norma" be known to the
candidate. This interaction wi norma" be face8to8face but tee'ho8
formats are not e)cuded. Time a''ro)imate" 2 minutes.
c. 5art (( consists of an interaction between candidates in 'airs 9or
e)ce'tiona" in threes or with one of the 'air a non8e)amination
candidate:. ,gain this wi norma" be face8to8face but tee'hone
formats are not e)cuded. Time a''ro)imate" 2 minutes.
d. 5art ((( consists of a re'o
rt from the candidates to the interocutor
9who has been absent from the room: of the rnteraction rom 5art811
Time a''ro)imate" 2 minutes.
,s 'ointed out at the beginning of this cha'ter. a good test wi
freBuent" combine features of the communicati-e a''roach. the
integrati-e a''roach and e-en the structuraist a''roach 8 de'ending on the
'articuar 'ur'ose of the test and aso on the -arious test constraints. (f. for
instance. the 'rimar" 'ur'ose of the test is for genera 'acement 'ur'oses
and there is -er" itte time a-aiabe for its administration. it m. be
necessar" to administer sim'" a 238item coLe test.
1
Language testing constant" in-o-es making com'romises between
what is idea and what is 'racticabe in a certain situation. Ne-ertheess this
shoud not be used as an e)cuse for writing and administering 'oor tests!
whate-er the constraints of the situation. it is im'ortant to maintain
ideas and goas. constant" tr"ing to de-ise a test which is as -aid and
reiabe as 'ossibe 8 and which has a usefu backwash effect on the
teaching and earning eading to the test.
Notes and references
( 4er. J 7 1/7< 6ictation as a test of E+L 5roficienc". (n &eaching English as a
+econd (anguage. A Boo- o! 7eadings. Ac&raw8Hi
< Cohen. , 6 1/03 &esting (anguage A"ilit# in the Classroom. Newbur" House
3 7iddowson. H & 1/70 &esting (anguage as Communication. 4)ford
*ni-ersit"
5ress
= Carro.B T1/70 ,n English 8anguage testing service. speci5cations.
The
British
Counci
2 The 4)ford8,res E)aminations in Engish as a ;oreign Language! #eguations
and '#lla"uses
? #o"a +ociet" of ,rts! &he Communicative 9se o! Engish
as a 6oreign (anguage
/'peci5cations and 6ormat1
<=
4bJecti-e testing
9with s'ecia reference to muti'e8choice techniBues:
3.1 +ubJecti-e and
'u":ective and o":ective are terms used to refer to the scoring of tests. ,
obJecti-e testing
test items. no matter how the" are de-ised. reBuire candidates to e)ercise
a subJecti-e Judgement. (n an essa" test. for e)am'e. candidates must
think of what to sa" and then e)'ress their ideas as we as 'ossibe@ in a
muti'e8choice test the" ha-e to weigh u' carefu" a the aternati-es and
seect the best one. ;urthermore. a tests are constructed subJecti-e" b"
the tester. who decides which areas of anguage to test. how to test those
'articuar areas. and what kind of items to use for this 'ur'ose. Thus. it is
on" the scoring of a test that can be described as obJecti-e. This means
that a testee wi score the same mark no matter which e)aminer marks the
test.
+ince obJecti-e tests usua" ha-e on" one correct answer 9or. at east. a
imited number of correct answers:. the" can be scored mechanica".
The fact that obJecti-e tests can be marked b" com'uter is one im'ortant
reason for their e-ident 'o'uarit" among e)amining bodies res'onsibe for
testing arge numbers of candidates.
4bJecti-e tests need not be confined to an" one 'articuar ski or
eement. (n one or two we8known tests in the 'ast. attem'ts ha-e e-en
been made to measure writing abiit" b" a series of obJecti-e test items.
Howe-er. certain skis and areas of anguage ma" be tested far more
effecti-e" b" one method than b" another. #eading and -ocabuar". for
e)am'e. often end themse-es to obJecti-e methods of assessment.
Cear". the abiit" to.write can on" be satisfactori" tested b" a subJecti-e
e)amination reBuiring the student to 'erform.a writing task simiar to that
reBuired in rea ife. , test of ora fuenc" might 'resent students with the
foowing stimuus!
%8ouwenrto8tiEe inICairo tIwo "earsDagDo.8+omeoneaskU "oiiOho--ong"ou
ha-e i-ed there. 7hat woud "ou sa"G
This item is arge" subJecti-e since the res'onse ma" be whate-er students
wish to sa". +ome answers wi be better than others. thus 'erha's causing a
'robem in the scoring of the item. How. for instance. ought each of the
foowing answers to8be markedG
,N+7E# 7! (>-e been i-ing in Cairo since 1/0?.
,N+7E# <! ( didn>t ea-e Cairo since 1/0?.
,N+7E# 3! ( ha-e i-ed in the Cairo Cit" for abo-e two "ears.
,N+7E# =! ;rom 1/0?.
<2
(T
,N+7E# s! ( came to i-e here before 1/0? and ( sti i-e here.
,N+7E# ?! +ince 1/0? m" home is in Cairo.
,though the task itsef attem'ts to simuate to some degree the t"'e
of task students might ha-e to 'erform in rea ife. it is more difficut to
achie-e reiabiit" sim'" because there are so man" different degrees of
acce'tabiit" and wa"s of scoring a the 'ossibe res'onses. Carefu
guideines must be drawn u' to achie-e consistenc" in the treatment of the
-ariet" of res'onses which wi resut.
4n the other hand. reiabiit" wi not be difficut to achie-e in the
marking of the foowing obJecti-e item. The Buestion of how -aid such an
item is. howe-er. ma" now be of considerabe concern. How far do items ike
this refect the rea use of anguage in e-er"da" ifeG
Com'ete the sentences b" 'utting the best word in each bank. >(s
"our home sti in CairoG>
>%es. (>-e been i-ing here ...
1/0?.>
,. for
B. on C. in
6. at E. since
Language sim'" does not function in this wa" in rea8ife situations.
ConseBuent". the ast item tests grammar rather than communication! it is
concerned with students> knowedge of forms of anguage and how
anguage works rather than with their abiit" to res'ond a''ro'riate" to
rea Bu>stions.
4n the whoe. obJecti-e tests reBuire far more carefu 're'aration
than subJecti-e tests. E)aminers tend to s'end a reati-e" short time on
setting the Buestions but considerabe time on marking. (n an obJecti-e test
the tester s'ends a great dea of time constructing each test item as
carefu" as 'ossibe. attem'ting to antici'ate the -arious reactions of the
testees at each stage. The effort is rewarded. howe-er. in the ease of the
3.< 4bJecti-e tests 4bJecti-e tests are freBuent" criticised on the grounds that the" are
sim'er to answer than subJecti-e tests. (tems in an obJecti-e test.
howe-er. can be made Just as eas" or as difficut as the test constructor
wishes. The fact that obJecti-e tests ma" genera"
loo- easier is no
indication at a that the" are easier. The constructor of a standardised
achie-ement or 'roficienc" test not on" seects and constructs the items
carefu" but ana"ses student 'erformance on each item and rewrites the
items where necessar" so that the fina -ersion of his or her test
discriminates wide". +etting the 'ass8mark. or the cutting8off 'oint. ma"
de'end on the testers subJecti-e Judgement or on a 'articuar e)terna
situation. 4bJecti-e tests 9and. to a smaer degree. subJecti-e tests: can he
're8tested before being administered on a wider basis! i.e. the" are gi-en
to a sma but tru" re'resentati-e sam'e of the test 'o'uation and then
each item is e-auated in the ight of the testees> 'erformance. This
'rocedure enabes the test constructor to cacuate the a''ro)imate degree
of difficut" of the test. +tandards ma" then be com'ared not on" betN-een
students from different areas or schoos but aso between students taking
the test in different "ears.
,nother criticism is that obJecti-e tests of the muti'e8choice t"'e
encourage guessing. Howe-er. four or.fi-e aternati-es for each item are
sufficient to reduce the 'ossibiit" of guessing. ;urthermore. e)'erience
Or.
shows that candidates rare" make wid guesses! most base their guesses on
'artia knowedge.
, much wider sam'e of grammar. -ocabuar" and 'honoog" can
genera" be incuded in an obJecti-e test than in a subJecti-e test.
,though the 'ur'osi-e use of anguage is often sacrificed in an attem't to
test students> abiit" to mani'uate anguage. there are occasions
9'articuar" in cass 'rogress tests at certain e-es: when good obJecti-e
tests of grammar. -ocabuar" and 'honoog" ma" be usefu 8 'ro-ided that
such tests are ne-er regarded as measures of the students> abiit" to
communicate in the anguage. (t cannot be em'hasised too strong".
howe-er. that test obJecti-it" b" itsef 'ro-ides no guarantee that a test is
sound and reiabe. ,n obJecti-e test wi be a -er" 'oor test if!
8 the test items are 'oor" written@
8 irree-ant areas and skis are em'hasised in the test sim'" because the"
are >testabe>! and
8 it is confined to anguage8based usage and negects the communicati-e
skis in-o-ed.
(t shoud ne-er be caimed that obJecti-e tests can do those tasks
which the" are not intended to do. ,s aread" indicated. the" can ne-er
test the abiit" to communicate in the target anguage. nor can the"
e-auate actua 'erformance. , good cassroom test wi usua" contain
both subJecti-e and obJecti-e test items.
3.3 Auti'e8choice
(t is usefu at this stage to consider muti'e8choice items in some detai. as
items! genera
the" are undoubted" one of the most wide" used t"'es of items in
obJecti-e tests. Howe-er. it must be em'hasised at the outset that the
usefuness of this t"'e of item is imited. *nfortunate". muti'e8choice
testing has 'roiferated as a resut of attem'ts to use muti'e8choice items to
'erform tasks for D hich the" were ne-er intended Aoreo-er. since the
muti'e8choice item is one of the most difficut and time8consuming t"'es of
items to construct. numerous 'oor muti'e8choice tests now abound.
(ndeed. the ength of time reBuired to construct good muti'e8choice items
coud often ha-e been better s'ent b" teachers on other more usefu tasks
connected with teaching or testing.
The chief criticism of the muti'e8choice item. howe-er. is that
freBuent" it does not end itsef to the testing of anguage as
communication. The 'rocess in-o-ed in the actua seection of one out of
four or fi-e o'tions bears itte reation to the wa" anguage is used in most
rea8ife situations. ,''ro'riate res'onses to -arious stimui in e-er"da"
situations are produced rather than chosen from se-era o'tions.
Ne"etthe8tess.Omutti'eOchoiceOitemsOcan8'ra-ide aOusefu means of
teaching and testing in -arious earning situations 9'articuar" at the ower
e-es: 'ro-ided that it is awa"s recognised that such items test
-nowledge
of grammar. -ocabuar". etc. rather than the abiit" to use anguage.
,though the" rare" measure communication as such. the" can 'ro-e
usefu in measuring students> abiit" to recognise correct grammatica
forms. etc. and to make im'ortant discriminations in the target anguage.
(n doing this. muti'e8choice items can he' both student and teacher to
identif" areas of difficut".
;urthermore. muti'e8choice items offer a usefu introduction to the
construction of obJecti-e tests. 4n" through an a''reciation and master"
of the techniBues of muti'e8choice item wnting is the woud8be test
<7
(
(
constructor fu" abe to recognise the imitations im'osed b" such items
and then em'o" other more a''ro'riate techniBues of testing for certain
'ur'oses.
The o'timum number of aternati-es. or o'tions. for each muti'e8
choice item is fi-e in most 'ubic tests. ,though a arger number. sa"
se-en. woud reduce e-en further the eement of chance. it is e)treme"
difficut and often im'ossibe to construct as man" as se-en good o'tions.
(ndeed. since it is often -er" difficut to construct items with e-en fi-e
o'tions. four o'tions are recommended for most cassroom tests. Aan"
writers recommend using four o'tions for grammar items. but fi-e for
-ocabuar" and reading.
Before constructing an" test items. the test writer must first determine
the actua areas to be co-ered b" muti'e8choice items and the number of
items to be8incuded in the88test8The test must be ong enough to aow for
a reiabe assessment of a testee>s 'erformance and short enough to be
'racticabe. Too ong a test is undesirabe because of the administration
difficuties often created and because of the menta strain and tension
which ma" be caused among the students taking the test. The number of
items incuded in a test wi -ar" according to the e-e of difficut". the
nature of the areas being tested. and the 'ur'ose of the test. The teacher>s
own e)'erience wi genera" determine the ength of a test for cassroom
use. whie the ength of a 'ubic test wi be affected b" -arious factors. not
east of which wi be its reiabiit" measured statistica" from the resuts of
the tria test.
Note that conte)t is of the utmost im'ortance in a tests.
6econte)tuaised muti'e8choice items can do considerabe harm b"
con-e"ing the im'ression that anguage can be earnt and used free of an"
conte)t. Both inguistic conte)t and situationa conte)t are essentia in
using anguage. (soated sentences in a muti'e8choice test sim'" add to
the artificiait" of the test situation and gi-e rise to ambiguit" and
confusion. ,n awareness of the use of anguage in an a''ro'riate and
meaningfu wa" 8 so essentia a 'art of an" kind of communication 8 then
becomes irree-ant in the test. ConseBuent". it is im'ortant to remember
that the foowing muti'e8choice items are 'resented out of conte)t here
sim'" in order to sa-e s'ace and to draw attention to the saient 'oints
being made.
The initia 'art of each muti'e8choice item is known as the
stem the
choices from which the students seect their answers are referred to as
options2responses2alternatives.
4ne o'tion is the answer, correct
option or
ke". whi? the other o'tions are
distractors. The task of a distractor is
to distract the maJorit" of 'oor students 9i.e. those who do not know the
answer: from the correct o'tion.
+ta" here unti Ar +hort ... . "ou to come. 0 stem
,. tod
B. wi te
C. is teing
6. tes
options2
0
responses2
alternativ
es
0 distractors
0 answer2correct option2-e#
The foowing genera 'rinci'es shoud be obser-ed when
muti'echoice items are constructed!
1 Each muti'e8choice item shoud ha-e on"
one answer. This answer
must be absoute" correct uness the instruction s'ecifies choosing the "est
o'tion 9as in some -ocabuar" tests:. ,though this ma" seem an eas"
matter. it is sometimes e)treme" difficut to construct an item ha-ing on"
one correct answer. ,n e)am'e of an item with two answers is!
>( sta"ed there unti John ...
,. had come
C. came
B. woud come
6. has come
< 4n" one feature at a time shoud be tested! it is usua" ess confusing
for the testees and it he's to reinforce a 'articuar teaching 'oint.
4b-ious". few woud wish to test both grammar and -ocabuar" at the same
time. but sometimes word order and seBuence of tenses are tested
simutaneous". +uch items are caed
impure items!
(
ne-er knew where ...
,. had the bo"s gone
C. ha-e the bo"s gone
B. the bo"s ha-e gone 6. the bo"s had gone
9Note that it ma" sometimes be necessar" to construct such im'ure items at
the -er" eementar" e-es because of the se-ere" imited number of
distractors genera" a-aiabe.:
3 Each o'tion shoud be grammatica" correct when 'aced in the stem.
e)ce't of course in the case of s'ecific grammar test items. ;or e)am'e.
stems ending with the determiner a. foowed b" o'tions in the form of nouns
or noun 'hrases. sometimes tra' the unwar" test constructor. (n the item
beow. the correct answer C. when mo-ed u' to com'ete the stem.makes the
sentence grammatica" incorrect!
+omeone who designs houses is a .. O ...
,. designer B. buider
C. architect 6. 'umber
The item can be easi" recast as foows!
+omeone who designs houses is ...
,. a designer B. a buider C. an architect
6. a 'umber
+tems ending in are, were, etc. ma" ha-e the same weaknesses as the
foowing and wi reBuire com'ete rewriting!
The bo">s hobbies referred to in the first 'aragra'h of the 'assage were
,. cam'ing and fishing
B. tennis and gof
C. c"cing ong distances
6. fishing. rowing and swimming
E. coecting stam's
,n" fair" inteigent student woud soon be aware that o'tions C and E
were ob-ious" not in the tester>s mind when first constructing the item
abo-e because the" are ungrammatica answers. +uch a student woud.
therefore. reaise that the" had been added ater sim'" as distractors.
+tems ending in 're'ositions ma" aso create certain difficuties. (n the
foowing reading com'rehension item. o'tion C can be rued out
immediate"!
John soon returned to ...
,. work B. the 'rison
C. home 6. schoo
</
= , muti'e8choice items shoud be at a e-e a''ro'riate to the 8
'roficienc" e-e of the testees. The conte)t. itsef. shoud be at a ower
e-e than the actua 'robem which the item is testing! a grammar test item
shoud not contain other grammatica features as difficut as the area being
tested. and a -ocabuar" item shoud not contain more difficut semantic
features in the stem than the area being tested.
2 Auti'e8choice items shoud be as brief and as cear as 'ossibe 9though it
is desirabe to 'ro-ide short conte)ts for grammar items:.
? (n man" tests. items are arranged in rough order of increasing difficut". (t
is genera" considered im'ortant to ha-e one or two sim'e items to Dead in>
the testees. es'ecia" if the" are not too famiiar with the kind of test being
administered. Ne-ertheess. areas of anguage which are tri-ia and not worth
testing. shoud.beOe)cuded from the8test.
3.= Auti'e8choice
items! the stemKthe
correct o'tionKthe
distractors
&he stem
4
( The 'rimar" 'ur'ose of the stem is to 'resent the 'robem cear" and
concise". The testee shoud be abe to obtain from the stem a -er" genera
idea of the 'robem and the answer reBuired. ,t the same time. the stem
shoud not contain e)traneous information or irree-ant cues. thereb"
confusing the 'robem being tested. *ness students understand the 'robem
being tested. there is no wa" of knowing whether or not the" coud ha-e
handed the 'robem correct". ,though the stem shoud be short. it shoud
con-e" enough information to indicate the basis on which the correct o'tion
shoud be seected.
< The stem ma" take the foowing forms!
9a: an incomplete statement
He accused me of ... ies. 8 O
,. s'eaking B. sa"ing C. teing 6. taking
9b: a complete statement
E-er"thing we wanted was to hand.
,. under contro C. we cared for
B. within reach 6. being 're'ared
/c1 a ,uestion
,ccording to the writer. what did Tom immediate" doG
,. He ran home. C. He began to shout.
B. He met Bob. 6. He 'honed the 'oice.
3 The stem shoud usua" contain those words or 'hrases which Aoud
otherwise ha-e to he re'eated in each o'tion.
The word >astronauts> is used in the 'assage to refer to
,. tra-eers in an ocean iner
B. tra-eers in a s'ace8shi'
C. tra-eers in a submarine
6. tra-eers in a baoon
The stern here shoud be rewritten so that it reads!
The word >astronauts> is used in the 'assage to refer to tra-eers in
,. an ocean iner C. a submarine
B. a s'ace8shi' 6. a baoon
The same 'rinci'e a''ies to grammar items. The foowing item!
( enJo" ... . the chidren 'a"ing in the 'ark.
,. ooking to C. ooking at
B. ooking about 6. ooking on
shoud be rewritten in this wa"!
e=o" ooking ...
. the chidren 'a"ing in the 'ark.
to B. about C. at 6. on
(f. howe-er. one of the errors made b" students in their free written work
has been the conission of the 're'osition aft@ r
loo- 9a common error:. then
it wi be necessar" to incude
loo-
in the o'tions.
enJo" ... . the chidren 'a"ing in the 'ark.
,. ooking on C. ooking at
B. ooking 6. ooking to
= The stem shoud aow for the >number of choices which ha-e been
decided u'on. This is 'articuar" ree-ant. for e)am'e. when com'arisons
are in-o-ed in reading com'rehension. There is no 'ossibe fourth o'tion
which can be added in the foowing item!
Tom was ... . the other two bo"s.
,. taer than
B. smaer than
8 C. as ta as
&he correct ;or norma 'ur'oses of testing. this shoud be cear" the
correct or "est
o'tion! thus. it is most im'ortant that each item shoud be checked b"
another 'erson.
(t can be argued that a greater degree of subtet" is sometimes gained
b" ha-ing more than one correct o'tion in each item. The correct answers
in the foowing reading com'rehension and grammar items are circed!
,ccording to the writer. Jane wanted a new racBuet because ,
her od one was damaged sight"
E she had ost her od one
C. her father had gi-en her some mone" for one
4 Aar" had a new racBuet
E. ,nn often borrowed her od racBuet
7ho ... . "ou c"ce here to see usG
,. or8dere 88 B!88caused 8 C4 made 6 asked 4 et
(t is -er" im'ortant. howe-er. to a-oid confusing the students b" ha-ing a
different number of correct o'tions for each item. and this 'ractice is
not
recommended. Each of the two muti'e8choice test items abo-e actua"
com'rises a grou' of trueKfase 9i.e. rightKwrong: items and. therefore.
each aternati-e shoud be marked in this wa"! e.g. in the first item. the
testee scores ( mark for circing ,. 1 mark for not circing B. 1 mark for
not circing C. ( mark for circing 6. and ( mark for not circing E 9tota
score 1 2:.
The correct o'tion shoud be a''ro)imate" the same ength as the
distractors. This 'rinci'e a''ies es'ecia" to -ocabuar" tests and tests of
31
i
&he distractors
3<
i
reading and istening com'rehension. where there is a tendenc" to make
the correct o'tion onger than the distractors sim'" because it is so often
necessar" to Buaif" a statement or word in order to make it absoute"
correct. ,n e)am'e of such a Dgi-eawa"> item is!
He began to cho-e whie he was eating the fish.
,. die
B. cough and -omit
C. be unabe to breathe because of something in the wind'i'e
6. grow -er" angr"
Each distractor. or incorrect o'tion. shoud be reasonab" attracti-e and
'ausibe. (t shoud appear right to an" testee who is unsure of the correct.
o'tion. (tems shoud be constructed in such a wa" that students obtain the
correct o'tion8b88direct seection8rather than b" the eimination of
ob-ious" incorrect o'tions. Choice 6 in the foowing grammar item is
much beow the e-e being tested and wi be eiminated b" testees
immediate"! their chances of seecting the correct o'tion wi then be one
in three.
The 'resent ta) reforms ha-e benefited ... . 'oor.
,. that B. the C. a 6. an
;or most 'ur'oses. each distractor shoud be grammatica" correct when
it stands b" itsef! otherwise testees wi be e)'osed to incorrect forms. (n the
abo-e item 9and in a grammar items: it is on" the wrong choice. and its
im'ied insertion into the stem. which makes a 'articuar 'attern
ungrammatica. ;or e)am'e. o'tion , is grammatica" correct on its own
and on" becomes incorrect when inserted into the stem.
The foowing item 9which actua" a''eared in a cass 'rogress test of
reading com'rehension: contains two absurd items!
How did 5icard first tra-e in s'aceG
,. He tra-eed in a s'ace8shi'. C. He went in a submarine.
B. He used a arge baoon. 6. He Jum'ed from a ta buiding.
*ness a distractor is attracti-e to the student who is not sure of the
correct answer. its incusion in a test item is su'erfuous. 5ausibe
distractors are best based on 9a: mistakes in the students> own written
work. 9b: their answers in 're-ious tests. 9c: the teacher>s e)'erience. and
9d: a contrasti-e ana"sis between the nati-e and target anguages.
6istractors shoud not he too difficut nor demand a higher 'roficienc" in
the anguage than the correct o'tion. (f the" are too difficut. the" wi succeed
on" in distracting the good student. who wi be ed into
considering the correct o'tion too eas" 9and a tra':. There is a tendenc"
for this to ha''en. 'articuar" in -ocabuar" test items.
%ou need a ... . to enter that miitar" airfied.
,. 'ermutation B. 'erdition C. 'ermit 6. 'ers'icuit"
Note that ca'ita etters are on" used in o'tions which occur at the
beginning of a sentence. Com'are the foowing!
Has ... . of 'etro increasedG
,. the 'rice B. 'rice C. a 'rice
of 'etro has actua" faen.
,. The 'rice
B. 5rice
C. , 'rice
3.2 7riting the test 7here muti'e8choice items are used. the testees ma" be reBuired to
'erform an" of the foowing tasks!
1 7rite out the correct o'tion in fu in the bank.
He ma" not come. but we> get read" in case he ...
,. wi B. does C. is 6. ma"
< 7rite on" the etter of the correct o'tion in the bank or in a bo) 9which
ma" a''ear at the side of the Buestion. etc.:.
He ma" not come. but we> get read" in case he ...
,. wi B. does C. is 6. ma"
3 5ut a tick or a cross at the side of the correct o'tion or in a se'arate bo).
He ma" not come. but we> get read" in case he ...
,. wi ,.
B. does B.
C. is C.
6. ma" 6.
= *nderine the correct o'tion.
He ma" not come. but we> get read" in case he ...
,. wi
B. does C. is 6. ma"
2 5ut a circe round the etter at the side of the correct o'tion.
He ma" not come. but we> get read" in case he ...
,. wi
4.
Bdoes C. is 6. ma"
Auti'e8choice items are rare" o'tiona in a test. for the testees
woud then s'end considerabe time in unnecessar" reading before
choosing the items the" wished to answer. Aoreo-er. uness there are good
reasons for weighting different items 9using the a''ro'riate statistica
methods:. it is ad-isabe to award eBua marks for each item.
The correct o'tion shoud a''ear in each 'osition 9e.g. ,. B. C. 6 or
E: a''ro)imate" the same number of times in a test or sub8test. This can
usua" be achie-ed if it is 'aced at random in a certain 'osition or if a
the o'tions are 'aced in a'habetica order 9i.e. according to the first
etter of the first word in each o'tion:. Howe-er. if the o'tions ha-e a
natura order 9e.g. figures.8dates:.8it is ad-isabe to kee' to this order.
Backwe started his career as a aw"er in
,. 1/<1 B.
1/<2 C. 1/<? 6. 1/3<
Note that few tests consist entire" of muti'e8choice items. Aan"
we8known tests strike a ha''" baance between obJecti-e items 9incuding
muti'e8choice items: and subJecti-e items. The" test famiiarit" with the
grammatica and e)ica com'onents of the anguage as we as the abiit" to use
the anguage 'roducti-e".
33
(
Tests of grammar and usage
=.1 (ntroduction The foowing are some of the most common t"'es of obJecti-e items used
to test awareness of the grammatica features of the anguage. Each t"'e of
item wi be treated in greater detai in this cha'ter.
8 muti'e8choice items
8 error8recognition items
8 rearrangement items
8 com'etion items
8 transformation items
8 items in-o-ing the changing of words
8 >broken sentence> items
8 'airing and matching items
8 combination items
8 addition items
DO.i
, . ?8.I
(t shoud awa"s be remembered that such items as the abo-e test the
abiit" to recognise or 'roduce correct forms of anguage rather than the abiit"
to use anguage to e)'ress meaning. attitude. emotions. etc.
Ne-ertheess. it is essentia that students master the grammatica s"stem of the
anguage the" are earning. Thus. cassroom tests of grammar and
usage can 'a" a usefu 'art in a anguage 'rogramme.
=.< Auti'e8choice The t"'e of muti'e8choice item fa-oured b" man" constructors of
grammar items!item grammar tests is the incom'ete statement t"'e. with a choice8 of four or
t"'es i f-e o'tions. This item ma" be written in an" of the foowing wa"s!
T"'e 1 Tom ought not to ... 9,. te
B. ha-ing tod C. be
teing. 6. ha-e tod: me "our secret. but he did.
T"'e < Tom ought not to ... . me "our secret. but he did.
,te
B.ha-ing tod
1
C.be teing
6.ha-e tod
,. te
B. ha-ing tod
T"'e 3 Tom ought not to
3=
C. be teing
6. ha-e tod
me "our secret. but he did.
T"'e = Tom ought not to have told me #our secret. but he did.
A. ;o change
B. te
C. ha-ing tod
6. be teing
(tem t"'es < and 3 are 'referabe to 1 because the o'tions do not
interru't the fow of meaning in the sentence! these items 'resent the
entire sentence so that it can be read at a gance. *nfortunate". t"'e 1
confuses the reader because of the ong 'arenthesis 9i.e. the four o'tions
occurring between ought not to and me1. (tem t"'e = shows the correct
9or
an incorrect: form as 'art of the sentence in such a wa" that it cannot be
com'ared on eBua terms with the other o'tions! a correct o'tion. for
instance. is genera" easier to recognise when it a''ears in the framework
of the sentence than as 'art of a ist of distracters.
,nother item t"'e a''ears beow. but it is not recommended since it
reBuires the testees to s'end time on unnecessar" reading. Not on" is it
uneconomica but it does not 'resent the >'robem> 9i.e. the o'tions: as
cear" as item t"'e < does.
T"'e 2 ,. Tom ought not to te me "our secret. but he did.
B. Tom ought not to ha-ing tod me "our secret. but he did.
C. Tom ought not to be teing me "our secret. but he did.
6. Tom ought not to ha-e tod me "our secret. but he did.
The foowing method is usefu for testing short answers and
res'onses!
T"'e ? >Tom ought not to ha-e tod an"one the secret.>
,. >+o ought "ou.>
C. >Neither "ou oughtn>t.>
B. >Neither ought "ou.>
6. >+o oughtn>t "ou.>
(tem t"'e 7 reBuires the students to seect the aternati-e which is
true according to the information con-e"ed in each sentence. +uch an item
ma" he incuded either in a test of reading com'rehension or in a test of
grammar! a knowedge of the 'articuar s"nta) is necessar" for the
understanding of the sentence.
T"'e 7 >Tom ought not to ha-e tod me.
,. Tom did not te me but he shoud.
B. 5erha's Tom ma" not te me.
C. Tom tod me but it was wrong of him.
6. (t was necessar" for Tom not to te me.
(t ma" be argued that an understanding of s"ntactica 'atterning is Just as
necessar" for the foowing item!
. was #obert ate ast weekG>
Three times.>
,. How much
C. How often
B. How man"
6. How ong
32
11
1
(tems which a''ear in a test of grammar and structure shoud be made
to sound as natura as 'ossibe. The foowing mechanica test item!
This book beongs to 5eter. (t is ...
,. 5eter>s book
C. the book of 5eter
B. the book to 5eter 6. the book of 5eter>s
can be rewritten as foows!
This book beongs to 5eter. but that is ...
,. Aar">s book C. the book of Aar"
B. the book to Aar" 6. the book of Aar">s
Note that distractors shoud genera" be correct both in writing and in
.s'eech8 T8he..foio.wing8item8'roO-..edOunsuccessfu. when it was incuded in a
test because man" of the more abe students seected o'tion 6. the reason
being that the" 'ronounced used to Buite correct" as use toKJu!staK!
. to go to m" unce>s farm e-er" weekend.
,. am used C. was used
B. used 6. use
Note that the sam'e items discussed in this section ha-e so far taken
the form of short deconte)tuaised items. (n 'ractice. howe-er. such items
woud a form 'art of a 'aragra'h or series of 'aragra'hs of descrir.ti-e.
narrati-e or e)'ositor" 'rose. The 'ro-ision of a detaied conte)t in this
wa". howe-er. often imits the range of grammatica features being tested.
(t is usua" im'ossibe. for e)am'e. to test the future continuous tense in
a narrati-e set in the 'ast 9uness direct s'eech is used:. +imiar". a
'aragra'h describing a sim'e manufacturing 'rocess ma" not 'ro-ide the
test writer with the o''ortunit" to test a the -erb forms and tenses he or
she ma" wish to test. This is the 'rice to be 'aid for incuding more
natura. Corttc)tuah+eu test (LC(*+. <n the other hand. the ad-antage of
such items as that on 'age 37 ies in the interesting and fair" authentic
conte)t 9i.e. a news'a'er artice: which contains the items. This is rea
anguage used for a 'articuar 'ur'ose. ;urthermore. the 'ro-ision of
conte)t he's to ensure that there is on" one correct o'tion in each case.
+hort deconte)tuaised sentences can ead to ambiguit" as the" are usua"
o'en to se-era inter'retations when used as stems for muti'e8choice
items. ;or e)am'e. o'tion 6 in the foowing deconte)tuaised item might
be correct 9as we as o'tion B: if the student ha''ens to know of a
medica research estabishment which 'a"s -ounteers to assist with
research and deiberate" catch a cod so as to be abe to test -arious curesR
coudn>t take the test ast week because ( ... . a cod.
,. ha-e caught C. woud catch
B. had caught 6. was catching
Auch better8 for testing 'ur'oses is the foowing item. The 'assage is
taken straight from a news'a'er artice and thus the anguage is authentic
and unatered in an" wa". The conte)t 'ro-ides students with enough
background knowedge and detais to a-oid ambiguit" and aternati-e
inter'retations. and the news'a'er re'ort itsef is -er" interesting. 6oes it
rea" matter if it wi not aow us the o''ortunit" to test e-er" 'oint of
grammar which we ma" want to testG +tudents taking this test are being
gi-en a rea fee for the anguage the" are earning.
, ong wa" from home
, 7<8"ear8od +amoan who 91: ...
. no Engish at a s'ent
thirteen da"s 9<: ...
. on buses in the +an ;rancisco area after
had become se'arated 93: ...
. his fami". 'oice said.
said that ;aaitua Logo. 92: ...
mo-ed to the
*nited +tates two "ears ago. eft his son and daughter8in8aw
9
?: . afewminutes inamarket in+anJose 97:...
something at a nearb" sta. 7hen he tried to return to th
em. he coud
not remember where the" 90: ...
. for him.
90: .
O . first. he began to wak to their home in 5ao ,to. <3
kiometres 913:...
.butater he911: . onabus. H
changed from bus to bus 91<: .. .
bushes and trees. 'oice said.
91: ,. is s'eaking B. s'eaks
9<: ,. to ride B. was riding
93: ,. with B. from C. b"
9=: ,. He B. The" C. 4ne
. the da"time and se't unde
C. has s'oken 6. was s'eakir
C. ride 6. riding
6. off
6. (t
92: ,. which B. that
C. who 6. what
9?: ,. in B. for C. since
6. at
97: ,. to bu" B. for bu"ing C. and bu"
6. bu"ing
90: ,. waited
9/: ,. ;or
B. were waiting. C. ha-e waited 6. wait
B. 4n C. (n 6. ,t
=.3 Constructing
muti'e8choice items
913: ,. far B. from C. near 6. awa"
911: ,. woud Jum' B. Jum'ed C. has Jum'ed 6. woud h.
Jum'ed
91<: ,. on B. at C. for 6. during
,though it is not awa"s 'ossibe to use sam'es of students> own writte
work to 'ro-ide the basis for the test items. it shoud not be too difficut for
constructors of cassroom tests and schoo achie-ement tests to take
ad-antage of the t"'es of errors made b" students in their free
com'ositions and o'en8ended answers to Buestions.
The foowing e)tract from a student>s etter is used here and in ate
sections to show how test items can be constructed. The etter was actuLR
written b" a student in a countr" where Engish is earnt as a foreign
anguage. The errors ha-e not been Dmanufactured> for the 'ur'ose of
iustration. but the" do re'resent errors made b" students from on" on
'articuar anguage background. The mistakes. therefore. wi not he t"'ica
of mistakes made b" students from man" other anguage
backgrounds and thus the distractors a''earing ater ma" be useess for
such students.
There is -er" much time ( didn>t write "ou. and now ( ha-e a itte free
time. 7inter is behind us and therefore ( ho'e that "ou woudn>t min
on such a ong 'eriod between m" ast etter and this one. %ou know
how is it. +un is shining. trees become green and it>s difficut to sta" is -
cosed among was. +ometimes whenethe
nJo" ooking the chidren
to
through the 'ark near m" odging
'a"ing. %ou know. the da" before "esterda" whie ooking through r
window ( saw the wet street and 'eo'e with umbreas rushing for
mone" and 'restige. ( sudden" remembered
beor
me what ha''ened.
to us. ( su''ose that "ou were not angr"
that it is not good to discuss about 'assed feeings.
;
(tem 1
Let us ignore the error in the first sentence for the time being and
concentrate on the error of tense after
hope.
'tep =.
The first ste' is to reduce the ength of the sentence and to correct the
error 9and an" other errors in the origina sentence:. Thus.
( ho'e that "ou woudn>t mind on such a ong 'eriod between m" ast
etter and this one.
becomes ( ho'e "ou won>t mind waiting for so ong.
'tep 2. Ne)t we write out the sentence. substituting a bank for the area
being tested. 7e write in the correct o'tion and the distractor which the
student has 'ro-ided for us. Howe-er. we ha-e to add a sentence because. in
certain.89rare: conte)fs wouldn>t ma" be correct.
( ho'e "ou ... . mind waiting for so ong. ( 'romise to re'"
sooner in future.
,. won>t
B. woudn>t
'tep$3. 7e now add another two distractors. ,gain. we go to the written
work of our students to 'ro-ide these distractors. But if we cannot ocate
an" suitabe errors without too much difficut". we use our own e)'erience
and knowedge of the target and nati-e anguages. Thus. two usefu
distractors which woud aso baance the e)isting two o'tions might be
shouldn>t and shan>t.
(
ho'e "ou ... . mind waiting for so ong. ( 'romise to re'"
sooner in future.
,. won>t B. woudn>t
C. shoudn>t
6. shan>t
(t ma" be argued. howe-er. that shan>t is acce'tabe usage amongst certain
s'eakers. thus gi-ing us two correct o'tions instead of one. Though it is high"
im'robabe that 'eo'e in most areas woud use
shan>t, there is a sight shadow of
doubt. This is enough to make it desirabe to remo-e
shan>t from our ist of o'tions.
'tep +. 4ne suggestion ma" be that we re'ace
shan>t with can>t. (f
students from a 'articuar anguage background make such mistakes as
can>t mind,
can>t shoud be used as a distractor. and 'ossib"
shouldn>t
changed to
couldn>t. ,s can be seen at this ear" stage. the actua 'rocess of item
writing is e)treme" subJecti-e.
( ho'e "ou ...
. mind waiting for so ong. ( 'romise to re'"
sooner in future.
,. won>t B. woudn>t C. coudn>t 6. can>t
,n aternati-e suggestion for a fourth o'tion might be
don>t or didn>t.
(
ho'e "ou don>t mind waiting for so ong.
( ho'e "ou didn>t mind waiting for so ong.
*nfortunate". both don>t and didn>t are correct. Howe-er. in the
foowing conte)t. didn>t is not acce'tabe!
>How ong are "ou going to beG>
>,bout haf an hour. ( ho'e "ou ...
. mind waiting for so ong.
,. won>t B. woudn>t C. shoudn>t 6. didn>t
30
11
1
(t ma" be argued that didn>t stands out too much. (f so 8 and if it is eBua"
usefu to test the use of don>t 9instead of
won>t1 after hope $ the item couc be
rewritten as!
>How ong wi "ou beG>
>,bout haf an hour. ( ho'e "ou ... . mind waiting for so ong.
,. don>t B. woudn>t
C. shoudn>t 6. didn>t
4b-ious". there are -ar"ing degrees of refinement in the constructioi of
muti'e8choice items. ;urthermore. some items are much more difficu to
construct than others. The foowing two items based on errors in the
student>s etter are fair" sim'e to write.
(tem <
Error!
. and enJo" ooking the chidren 'a"ing.
(tem! 4d Ar Jones enJo"s ...
. the chidren 'a"ing.
,. ooking C. ooking on
B. ooking at 6. ooking to
+ome test constructors might be tem'ted to use
!or as a distractor. (t can be
argued. howe-er. that loo-ing !or is correct! od Ar Jones might enJo" ooking for the
chidren 'a"ing 9i.e. he might enJo" waking throug the 'ark. chatting to his friends. etc.
whie he is in the 'rocess of ooking for his grandchidren. who are 'a"ing:.
Note that the correct o'tion is now in the third 'osition. C. (t is
im'ortant to -ar" its 'osition. Note aso that the word loo-ing a''ears in each o'tion!
in some tests the item might a''ear as foows!
4d Ar Jones enJo"s ooking ...
. the chidren 'a"ing.
,. 8 B. on C. at 6. to
Howe-er. when this format incudes a dash 98:. it is unnatura and not
recommended since the insertion of a dash in the stem woud not be
norma 'ractice in rea ife.
(tem 3
Error! >( su''ose that "ou were not angr" to me.>
(tem!
( do ho'e "ou weren>t angr" ... . me.
,. to
B. with C. on 6. about
Note that at is aso incorrect and ma" be used as a 'ossibe distracto@
4n the other hand. it ma" be fet that a number of nati-e Engish s'eake do
sa" angr# at a 'erson. The decision whether or not to incude
at in the ist
of incorrect o'tions is .again a "er" subJecti-e one.
=.= Constructing error8
The fourth sentence of the etter on 'age 37 begins +un
is shining, trees
recognition muti'e8 "ecome green and ...
. The error caused b" the omission of t!
choice items
artice ma" be tested as foows. using a muti'e8choice item!
. is shining bright" toda".
,. +un B. The sun
C. , sun 6. +ome sun
(t ma" be argued. howe-er. that the choice here is strict" between
o'tions , and B at certain e-es where students ha-e earned to a-oid
using >a> and >some> with >sun>O (n such instances. one usefu de-ice 9sti
using the muti'e8choice format: is the error8recognition t"'e of item.
#
1
T"'e 1
Each sentence contains four words or 'hrases underined. marked ,. B. C
and 6. +eect the underined word or 'hrase which is incorrect or
unacce'tabe.
1. 1 do ho'e "ou woudn>t mind waiting for such a ong time.
, B C 6
<. (>m worried that "ou> be angr" to me.
, B C 6
3. ( didn>t see Bi since he went into hos'ita ast month.
, B C 6
=. A" car had broken down. so ( went there b" foot.
,88 OOE.. C. %.
T"'e <
There is a mistake in grammar in each of the foowing sentences. 7rite the
etter of that 'art of the sentence in which it occurs.
, B C 6
1. +unKis shiningKbright" toda"K. isn>t itG
, B C 6
<. 4d Ar JonesKenJo"sKooking the chidrenK'a"ing in the 'ark.
, B C 6
3. Ton">s fatherKwoud not et himKto sta" outKate at night.
, B C 6
=. 6idn>tK+usan te "ouKshe woudn>t mind to comeKwith us on the
'icnicG
(tem t"'e < aows the test writer to test errors caused b" omission!
e.g. 'un is shining and loo-ing the children. This t"'e of error
cannot be
tested b" the first item of the error8recognition t"'e. Howe-er. there are
different wa"s of correcting man" sentences. ;or e)am'e. students ma"
write B or C to denote the incorrect 'art of the third sentence abo-e.
according to which of these correct -ersions is in their mind!
Ton">s father woud not 'ermit him to sta" out ate. 91 B:
Ton">s father woud not et him sta" out ate. 91 C:
;or this reason. the test writer is strong" ad-ised to a-oid items of the
second t"'e.
+ometimes Vstudents are gi-en correct sentences together with the
incorrect ones! the" are then reBuired to write the etter E if the sentence
does not contain an" error. (n 'ractice. this method does not work too we
since man" students tend to regard e-er" sentence as containing an error.
(ndeed. another argument against this t"'e of item is that it em'hasises the
more negati-e as'ects of anguage earning. (t is cear" not sufficient for
students sim'" to recognise sources of error! the" ought to be encouraged
at a times to concentrate on recognising and 'roducing the correct forms.
This argument is su''orted b" man" 's"choogists and teachers who hod
that it is undesirabe for students to be e)'osed too much 8to incorrect
forms. 4n the other hand. this item t"'e is cose" reated to those skis
reBuired when students check. edit or 'roof8read an" re'ort. artice. 'a'er
or essa" the" ha-e Just written.
=.2 Constructing #earrangement items can take se-era forms. the first of which to consider
rearrangement items
here wi be the muti'e8choice t"'e.
The student who wrote the etter in =.3 ob-ious" e)'erienced
considerabe difficut" with word order in re'orted s'eech. es'ecia" after
the -erbs -now and wonder. Here are two of the errors he made!
>%ou know how is it.
93rd sentence:
>( wonder did "ou grow more fatter since summer.>
9ater in the same etter:
(f we attem't to test the first error b" means of an ordinar"
muti'echoice item. we are faced with the 'robem of being restricted to
on" two o'tions! the correct o'tion and the distractor 9i.e. the error:.
%ou know how ...
,. it is
B. is it
,s the item stands here. we cannot 'ossib" construct other o'tions. (t
becomes necessar". therefore. to engthen the origina statement to! ?ou
-now how warm it is toda#. The item woud then read!
>7on t ( need a coatG>
>7e. "ou know how ...
,. warm is it toda"
B. toda" it is warm
C. is it warm toda"
6. warm it is toda"
E. toda" is it warm
There seems to be a danger here of confusing the testees b" 'resenting
them with the 'robem in such a wa" that a certain amount of menta
Jugging becomes necessar" on their 'art. , 'referabe item t"'e> is the
foowing word8order item!
Com'ete each sentence. b" 'utting the words beow it in the right order. 5ut
in the bo)es on" the etters of the words.
>7on>t ( need a coatG>
>7e. "ou know how ...
,. it B. toda" C. warm
6. is
wonder if ...
. since summer.
,. grown B. "ou C. fatter
6. ha-e
7ord order items are usefu for testing other structures and features
in-o-ing in-ersion!
E-er"one>s forgotten ...
,. cu' B. he C. which
6. used
Not on" ...
. but he took me to his house.
,. me B. he C. did 6. meet
Howe-er ... . "ou> ne-er 'ass that test.
,. "ou B. tr" C. hard 6. ma"
Leeds *nited shoud ha-e won! Just think ...
,. unuck"
B. were C. how 6. the"
=1
1
8i don>t know how ong ...
,. going B. Jim C. is 6. to be
The order of adJecti-es and the 'osition of ad-erbs can be tested in
this wa". as indeed can se-era other grammatica areas!
The 'oice are ooking for ...
,. big B. two C. cars 6. back
7oud "ou ike to read 6a-id Brown>s ...
,. short B. new C. stor" 6. e)citing
Tom said ... . ceaning his car.
,. had B. finished C. he 6. Just
4n" ... . been rude to "ouR
,. e-er B. ( C. ha-e 6. once
Ars 7aker made ,nn ...
,. her new 'en B. to C. show 6. me
+omeone warned #ob ... D . thie-es.
,. for B. to C. out 6. ook
(n man" cases it wi be usefu to change from a muti'e8choice item
format to a format in-o-ing some actua writing. The rearrangement item
can be used to test the same features of word order. but the item format
becomes a itte ess artificia. The students are sim'" reBuired to
unscrambe sentences and to write out each sentence. 'utting the words or
constituent 'arts in their correct order!
1. Not on" ...
88
Kthe e)aminationK-er" difficutKunfairKwasKbutKitKwasKaso
<. (t is not ad-isabe
Kthe e)aminationKateKu'Kthe nightKto sta"Kbefore
3. The best wa" to 're'are ...
K i sK"oursefK'ast 'a'ersKtimed 'racticeKfor the 'a'erKto gi-eKin doing
Note that rearrangement items can be used for sentences as we as for
words and 'hrases. 7hen used for this 'ur'ose. such items can offer a
means of testing an understanding of connecti-es and reference de-ices.
+tudents ma" be reBuired to write out a the sentences in their correct
seBuence or sim'" to 'ut the etters of the -arious sentences in their
correct order!
,. ConseBuent". "ou shoud make e-er" effort to com'ete the 'a'er.
B. +cribbe them down as Buick" as "ou can. if necessar".
C. (f "ou find "oursef running out of time before "ou can com'ete it.
howe-er. don>t worr" about writing "our answers neat".
6. #emember that it is im'ossibe to score marks on Buestions which
"ou ha-e not attem'ted.
=.?
Constructing
Carefu" constructed com'etion items are a usefu means of testing a
com'etion items student>s abiit" to 'roduce acce'tabe and a''ro'riate forms of anguage.
The" are freBuent" 'referabe to muti'e8choice items since the" measure
=<
'roduction rather than recognition. testing the abiit" to insert the most
a''ro'riate words in seected banks in sentences. The words seected for
omission are grammatica or functiona words 9e.g. to, it, in, is, the1.
content words ma" be seected in a -ocabuar" or reading test.
The error 'un is shining in the e)tract from the student>s etter in
=.3
iustrates one 9minor: difficut" of constructing satisfactor" com'etion
items. ,though on" one answer is 'ossibe here. this com'etion item
woud ha-e to a''ear as!
. +un is shining toda".
or as! . sun is shining toda".
The former item suggests to the testees that no determiner is necessar"
9since 'un is written with a ca'ita etter: whie the atter item suggests that
a determiner is necessar" 9because sun is written without a ca'ita:.
The item can be sim'" rewritten as a Buestion to o-ercome this
'robem!
(s
. sun shining toda"G
Here are two more e)am'es of com'etion items based on the student>s
etter!
7rite the correct word in each bank.
1. The od man enJo"s ooking ... . the chidren 'a"ing.
<. That car beongs ... . Heen>s mother.
3. ( ho'e "ou>re not angr" ...
me.
5ut a. the, or some in each bank on" where necessar". (f "ou think that
no word shoud be 'aced in the bank. 'ut a cross 9): there.
1. Can "ou see ... . sun shining through the coudsG
<. ( saw "our unce ... . da" before "esterda".
3. 7hat ha-e "ou been doing since ( saw "ou ...
. ast summerG
Com'etion items cannot. of course. be machine8marked but the" are
-er" usefu for incusion in cassroom tests and for e)ercise 'ur'oses.
Howe-er. sometimes the most straightforward com'etion items can cause
'robems in the scoring. (n the foowing e)am'e was preparing and
prepared are eBua" correct. (t can be argued that had prepared is aso
correct if as is regarded as meaning "ecause. i.e. 'eo'e gas'ed because
the" didn>t e)'ect him to 're'are for the Journe" 8 the" thought he woud
go without 're'aration.
5#E5,#E 1. He heard a gas' behind him as he ... . to go.
*ne)'ected wa"s. of com'eting banks are shown "et again in the
foowing e)am'e!
,s soon as 'ossibe the ne)t da" ( sent m" stor" ... . the
editor ... . the magaLine ... . which m" best work
usua" a''eared.
(t is Buite 'ossibe to write a stor" a"out an editor. and send the stor" about
the editor to a magaLine. ,though such an inter'retation ma" sound
somewhat absurd. it iustrates the engths to which the test writer must
sometimes go to make certain that testees 'roduce on" the answer he or she
wants to be used in each bank. ;or cass tests. such a critica attitude might
we be harmfu if it took the teacher>s attention off more im'ortant and
urgent 'robems in teaching and testing.
=3
The foowing e)am'e< indicates the wide range of 'ossibiities for
one com'etion item!
( go to the cinema reguar". but ( ... . to the theatre for months.
The answer ob-ious" reBuired b" the tester is
haven>t "een howe-er.
'ossibe answers are!
ha-en>t been shan>t be going
hadn>t been
can>t go
9sometimes: don>t go ha-en>t been abe to go
ma" not go am not going
don>t know whether (>-e been didn>t go
shan>t go
ha-en>t gone
won>t go ha-en>t been going
(f the aim of this 'articuar item is to force the use of the 'resent 'erfect
tense. there are.three wa"s of restricting the choice a-aiabe to the testees
9athough the first two wa"s de'end hea-i" on reading com'rehension:!
/a1 "# providing a conte4t.
$im usua" goes to the cinema about once a week but she ...
four fims aread" this month and it>s on" the <3th toda". 95ossibe
answers! has seenKwi ha-e seenKmust ha-e seen:
/"1 "# providing data.
( go to the cinema reguar". but it>s ages since ( ast saw a 'a".
( go to the cinema reguar". but ( ...
. to the theatre for
months.
95ossibe answers! ha-en>t beenKha-en>t goneKha-en>t been
goingKha-en>t been abe to go:
/c1 "# using multiple$choice techni,ues.
( . .
. to the theatre three times since ( ast saw "ou.
,. go C. had gone
B. ha-e been 6. went
There are two maJor ad-antages in using a 'assage of continuous
'rose rather than se'arate sentences when gi-ing a com'etion t"'e test.
;irst". the use of conte)t often a-oids the kinds of ambiguit" referred to in
the 're-ious 'aragra'hs. +econd". the students e)'erience the use of
grammar in conte)t. being reBuired to use a the conte)t cues a-aiabe in
order to guess man" of the missing words. ,s a conseBuence. the" are
genera" ad-ised to read or gance at the whoe 'assage before starting to
f i in an" of the banks.
The foowing are e)am'es of two different t"'es of com'etion items
in conte)t. (n e)am'e 9a: the banks are indicated whie in e)am'e 9b:
9sometimes referred to as a modified coLe 'assage: the omissions
themse-es are not indicated. (n e)am'e 9a: on" one word shoud be
written in each bank. whie in e)am'e 9b: one word has been omitted
from each ine. (n this atter t"'e of e)am'e. the students are reBuired to
'ut an obiBue stroke 9K: at the 'ace where the word has been omitted and
then to write the missing word in the a''ro'riate s'ace.
9a: (t 91 :
. awa"s usefu
9<: . 'ractise answering
the t"'es of Buestions 93: ...
O O
"ou ma" 9=: ...
. asked. Howe-er.
92:
. is not enough 8
sim'" 9?: ...
. gance through
a 'ast 'a'er 97: ...
. answer
the Buestions 90: ...
. "our head.
This wi gi-e "ou 9/: ...
indication
at a 913:
. what "ou can do.
9b: Historians and anthro'oogists used think
91:
that i heath and disease 're-aent among
9<:
'rimiti-e tribes. #esuts recent in-estigations.
93:
howe-er. ha-e shown we much to earn
9=:
from 'rimiti-e 'eo'e. 7ithout benefit of
92:
immunisation or -accination. 'rimiti-e often
9?:
acBuired immunit" the diseases 'resent in
97:
their societ". Aoreo-er. the diseases now beset
90: .O.
modern societ" were uncommon 'rimiti-e
9/:
man. ,though he did i-e a ong ife. 913:
according to research. he ha-e been far
911:
heathier 're-ious" thought.
91<:
(n the foowing e)am'e. 'art of a sentence haO :een omitted.
,though sentence com'etion items of this t"'e mai concentrati orc
s'ecific grammatica 'oints more difficut. the" ne-e8heess offer a usefu
wa" of testing an abiit" to hande structures accordi1J to certain 'atterns.
7hether such items are regarded as tests of grammaO or of controed
writing is of minor im'ortance! the" demand an abii! to use a''ro'riate
grammatica forms for a 'articuar 'ur'ose in a 'articuar conte)t.
Aan" Loo officias are worried b" the increasing ii8treatment of animas
b" -isitors to the Loo. es'ecia" b" chidren. Aost risitors e)'ect to see
the animas -er" acti-e in their encosures. 7hen 8e animas fai to
'erform in the wa"s e)'ected ...
Crocodies seem to be the chief -ictims of such arscks because
the" ...
. The bot .s. cans. sticks and
stones that ...
. enc J' as unsight" itter
in the encosures. +ome animas. howe-er. swae.w
. Aost of the -isors wttu8
H H H H H H H H
H H . (
are not rea" 8aicious but
sim'" ..
. Qoo ofPcias are
constant" ...
. (ndiscriminate feeding of
the animas is not aowed because ...
The com'etion of diaogues can aso 'ro-ide a uOs.fu wa" of testing
the abiit" to mani'uate the grammar and 'atterns of the anguage in
conte)t. ,gain. meaning 'a"s a ke" roe in determinir> the students>
abiit" to 'ro-ide satisfactor" answers. *nfortunate". howe-er. if its 4
W
-er" difficut to write natura diaogues and at the same time 'rcre
students with usefu cues.
B! +o do (. ( genera" watch it for an hour or two e-er" e-ening.
,!
B! (mmediate" after (>-e finished m" homework. at about eight8thirt" or
nine.
B! +o are mine. The funnier. the better. ( ike ,merican ones best of a 8
"ou know. 'rogrammes ike >Benson> and >6ifferent +trokes .
, !
B! Neither do 1. ( can>t stand an"thing that>s too frightening.
B! ( agree about educationa 'rogrammes. but there are sti -er" few of
themO
=.7 Constructing K The transformation t"'e of item is e)treme" usefu for testing abiit" to
transformation items 'roduce structures in the target anguage and he's to 'ro-ide a baance
when incuded in tests containing muti'e8choice items. (t is the one
obJecti-e item t"'e which comes cosest to measuring some of the skis
tested in com'osition writing. athough transforming sentences is different
from 'roducing sentences. +ubJecti-e decisions. of course. ma" ha-e to be
made in the scoring 'rocess! e.g. how shoud s'eing errors be markedG
The foowing transformation items ha-e been based on errors which
occurred in the student>s etter. an e)tract of which was gi-en in +ection =.3.
#ewrite each of the foowing sentences in another wa". beginning each
new sentence with the words gi-en. Aake an" changes that are
necessar" but do not change the genera meaning of the sentence.
1. ( ha-en>t written to "ou for a ong time.
(t>s a ong time ..
<. (n sunn" weather ( often go for a wak.
7hen the weather ...
3. 4d Ar Jones ikes to ook at the chidren 'a"ing.
4d Ar Jones enJo"s ...
4ther transformation items gi-ing some idea of the range of areas that
can be co-ered in this wa" are!
1. (t was im'ossibe to work under those conditions.
7orking ...
<. ( don>t think it>s necessar" for "ou to sta" an" onger.
( don>t think "ou ...
3. ( was abe to ea-e the office ear" "esterda".
(t was 'ossibe ...
=. Joe can sing better than "ou.
%ou cannot ...
2. This book is too big to go on an" of the she-es.
This book is so big ...
?. ;rances is -er" good at tennis.
;rances 'a"s ...
7. 5oor 5eter was bitten b" a mosBuito.
, mosBuito ...
=?
0. >7hen wi "ou -isit LondonG> Ar +trong asked me.
Ar +trong asked me ...
,s with com'etion items. it is often difficut to restrict the number of
'ossibe answers. Howe-er. such restrictions are not essentia for
constructors of cassroom tests. 'ro-ided that the" are fu" aware of a the
'ossibe correct answers and of the s'ecific area the" are testing. The
foowing e)am'es indicate some of the aternati-es 'ossibe for four of
the 'receding items!
( ha-en>t written to "ou for a ong time.
(t>s a ong time since 2 /last1 wrote /to1 #ou
since #ou received a letter !rom me, etc.
( don>t think it>s necessar" for "ou to sta" an" onger.
( don>t think "ou need /to1 sta# an# longer 0
e)'ected answer
will 5nd it necessar# to sta# an# longer 0
'ossibe
answer
Joe can sing better than "ou.
%ou cannot sing as well as Joe 0 e)'ected
answer
sing "etter than Joe 0 'ossibe
answer
;rances is -er" good at tennis.
;rances 'a"s tennis ver# well2e4tremel#
well,
etc.
1 e)'ected answer
ver# good tennis 0 'ossibe answer
*nfortunate". the aternati-es in the ast three e)am'es defeat the
'ur'ose of the tests as the" stand at 'resent. since students can a-oid using
the actua grammatica 'atterns being tested. Howe-er. it is a sim'e
matter to rewrite them as foows!
(s it necessar" for us to sta" an" ongerG
Need ...
G
%ou cannot sing as we as Joe.
Joe can sing ...
;rances is -er" good at tennis.
;rances 'a"s tennis ...
+ometimes it is difficut to eicit the 'articuar form we wish to test.
;or e)am'e!
( fee miserabe e-en though ( shoudn>t.
( know ( shoudn>t fee miserabe but I do.
,though81 do is the answer8reBuired. we coud scarce" faut!
( know ( shoudn>t fee miserabe but I certainl# don>t !eel
happ#.
know ( shoudn>t fee miserabe but
= am misera"le.
(n some tests.3 students ma" be reBuired to rewrite a sentence
beginning with a certain word underined in the origina sentence. ;or
e)am'e!
The" beie-ed that the earth was fat.
The earth was beie-ed to be fat.
This item t"'e is a usefu -ariation of the 're-ious t"'e discussed. but
sometimes restricts the kind of transformation 'ossibe since the first word
=7
11
of the new sentence has to a''ear in the origina sentence. Thus it becomes
im'ossibe to test the reBuired transformation of a sentence ike Is it
necessar# !or us to sta# an# longer@ /0 ;eed we sta# an# longer@1
Transformation can aso be effected b" reBuiring students to substitute
a
gi-en -erb in a sentence.3 the new -erb necessitating a change in the
structura 'attern.
Ten essons ma-e up the course. 9C4N+(+T:
The course consists of ten essons.
couldn>t go swimming "esterda". 9,LL47:
( wasn>t aowed to go swimming "esterda".
,s with a the t"'es of items treated in the 're-ious sections. the
transformation t"'e of item is im'ro-ed if it can be 'ut into a conte)t 9i.e. if the
sentences for transformation can be written in seBuenee to form 'art of a
continuous 'iece of 'rose:. *nfortunate". howe-er. the 'ro-ision of a conte)t
does not aow for the range of sentence 'atterns the test writer
ma" wish to test. Aoreo-er. most students tend to treat each sentence in
isoation for the 'ur'oses of the test.
The foowing e)am'es iustrate how each of the sentences for
transformation can be made to form 'art of a continuous sentence.
= Changing sentences according to a given pattern
9a: Eer" few obJecti-e tests aow for. choice.
%ou ha-e ...
9b: Howe-er. the instructions shoud be carefu" checked.
Howe-er. "ou ...
9c: 6ifferent t"'es of Buestions on the same 'a'er wi necessitate
changes in the instructions.
The instructions
2 Changing sentences "# using selected words
9a: #emember that it is not necessar" to answer the Buestions in the
order set. 9NEE6:
9b: %ou are ad-ised to check "our answers carefu" after each Buestion.
9,6E(+,BLE:
9c: Aost teachers aso recommend "ou to ea-e fi-e minutes s'are at
the end of the e)amination in order to check "our 'a'er. 9+*&&E+T:
=.0
Constructing items This t"'e of item is usefu for testing the student>s abiit" to use correct
in-o-ing the changing
tenses and -erb forms. (t is a traditiona t"'e of test but the a"out is
of words
im'ro-ed in this 'articuar case b" 'ro-iding banks on the right of the te)t
for com'etion. The continuit" of the te)t is not im'aired more than
necessar" b" ha-ing both banks and underined words inserted in the
sentences. Thus the risk of obscuring the meaning of the te)t is reduced.
I Aer"s. tenses, etc.
#esearchers 91: to con-ince that a drug 91:
the" 9<: to test can im'ro-e the memor" and that 9<:
>+
it 93: to be the forerunner of other drugs which
93:
e-entua" 9=: to im'ro-e menta abiit"
9=:
2 )ord "uilding
+tudents who were gi-en the drug for a fortnight did
considerab" 91. we: in tests than others. The tests
91:
incuded the 9<. memorise: of ists of words as we 9<:
as of 93. inform: from two messages transmitted at 93:
the same time. 6uring the first week there was no
9=. notice: difference between the two grou's. but 9=:
after a fortnight the grou' on the drug was found to
ha-e increased its 92. abe: to earn b" amost 92:
twent" 'er cent.
=./ Constructing
This t"'e of item> tests the student>s abiit" to write fu sentences from a
>broken sentence> series of words and 'hrases. and thus does not aow the test writer to
items
concentrate e)cusi-e" on testing those 'articuar grammatica features
which ma" ha-e Just been 'ractised in cass. (t is ne-ertheess a usefu
de-ice for testing grammar 'ro-ided that the tester is aware that se-era
other areas of the anguage are being tested in addition to those on which
he or she wishes to focus attention. (ndeed. this -er" fact ma" be
considered an ad-antage. +o man" students are abe to score high marks
on grammar items when each item is set in isoation and concentrates on
on" one area of grammar. Errors are made. howe-er. when the attention
of the student is concerned with the meaning of the conte)t as a whoe and
with 'erforming a number of different grammatica tasks necessar" to
achie-e8that meaning.
7hen setting this item. make sure that the instructions are -er" cear
indeed and 'ro-ide one or two e)am'es. +tudents unfamiiar with this
'articuar item format freBuent" ha-e difficut" in knowing e)act" what to
do. es'ecia" as the 're-ious e)'erience of man" ma" ead them to think
that the 'resence of an obiBue stroke indicates the omission of a word
instead of signaing mere" a fragmented sentence or series of notes. (n the
rubric. students shoud be instructed to make whate-er changes are
necessar" to form good sentences. adding artices. 're'ositions. etc. where
reBuired and 'utting -erbs in their correct tense.
TakeK drugs and stimuantsK kee' awakeKwhie re-ise e)aminationK
often "e -er" harmfu.K (t "e far betterK
lead2 baanced ifeK and get
enough see'K e-er" night.KThereK beKimitKdegree and s'anK
concentration Kwhich "ou be ca'abeKe)ert.K BrainK
need restKas much
bod".K (ndeed.K it "e Buait"K than Buantit" workK that be im'ortant.
=.13 Constructing This t"'e of item usua" consists of a short con-ersation! e.g. a stimuus in
'airing and matching the form of a statement or Buestion foowed b" a res'onse often in the
items form of a statement. (t is used to test the abiit" to seect a''ro'riate
res'onses to stimui which woud be 'resented ora" in norma e-er"da"
situations. The item is more usefu for testing students> sensiti-it" to
a''ro'riac" and their awareness of the functions of anguage rather than
their knowedge of grammar 9athough grammatica cues ma" 'ro-e
im'ortant in com'eting this item satisfactori":. To 'erform the task
reBuired. students are sim'" reBuired to write the etter of the correct
res'onse in the s'ace 'ro-ided.
=/
:
Coumn 1
&oing to see a fim tonightG
How was the fimG.
can>t stand war fims. can "ouG
+o "ou went to the cinema.
6on>t "ou find war fims too -ioentG
Ha-e "ou e-er seen a Ja'anese war fimG
( ike war fims.
(s e-er"one going to see the fimG
7hat8a>.tout going to see a cowbo" fim insteadG
7h" didn>t "ou come with us to see the fimG is
that wh" "ou don>t ike war fimsG
Letter Coumn <
;. ,. No. ( didn>t.
B. Aost are. ( think.
C. (t>s one of the reasons.
6. ( had a ot of work to do.
E. ,ctua". ( Buite ike them.
;. %es. ( 'robab" wi.
&. No. ( ha-en>t.
H. 7hat a good ideaR ( 'refer them to war fims.
(. +o do 1.
J. , right. Nothing s'ecia.
$. Not rea". ( Buite ike them.
(t shoud be remembered. of course. that this is not authentic
discourse. Howe-er. athough the anguage and situation here are
ine-itab" artificia. the item does ser-e to he' students to associate the
anguage the" are earning with rea8ife situations. abeit to a imited
e)tent.
=.11 Constructing These obJecti-e8t"'e items ha-e ong been used in 'ast tests. The" shoud
combination and be. used s'aring". howe-er. as the" in-o-e arge" mechanica res'onses
addition items on the 'art of the student. Note that athough the se'arate sentences are
inked to one another b" theme. the items can hard" be described as being
conte)tuaised in an" rea wa".
= Com"ination items
9+tudents are instructed to Join each 'air of sentences. using the word in
brackets.:
9a: %ou finish the 'a'er. Then check "our answers carefu". 9,;TE#:
9b: +ome Buestions ma" be -er" difficut. The" shoud be eft unti ater.
97H(CH:
9c: %ou shoud usua" write answers in com'ete sentences for a the
Buestions on "our 'a'er. Howe-er. write notes for those Buestions
which "ou do not ha-e time to answer. 9,LTH4*&H:
2 Addition items
9+tudents are instructed to insert the word in ca'itas in the most
a''ro'riate 'ace in each sentence.:
9a: %ET Ha-e "ou answered a the BuestionsG
9b: +T(LL +ome students had not mastered the correct
techniBues for answering e)amination Buestions.
9c: 4CC,+(4N,LL% There ma" be itte choice of Buestions.
Notes and references
1 +imiar t"'es of test items a''eared in 'ast 'a'ers of the Joint Aatricuation
Board! &est in English /<verseas1.
< ( am indebted to Ar John Bright for this e)am'e and the 'ossibe soutions.
3 +imiar items a''eared in the 'ast in the *ni-ersit" of Cambridge Loca
E)aminations +"ndicate! Certi5cate o! %ro5cienc# in English.
= +imiar items ha-e a''eared in the *ni-ersit" of Cambridge Loca E)aminations
+"ndicate! 6irst Certi5cate in English.
so
Testing -ocabuar"
2.1 +eection of items
, carefu seection. or sam'ing. of e)ica items for incusion in a test is
genera" a most e)acting task. Aan" of the more traditiona t"'es of
-ocabuar" tests are designed in such a wa" that the" test a knowedge of
words which. though freBuent" found in man" Engish te)tbooks. are rare"
used in ordinar" s'eech.
The first task for the writer of a -ocabuar" test is to determine the
degree to which he or she wishes to concentrate on testing the students>
acti-e or 'assi-e -ocabuar". The ne)t task is to decide whether the e)ica
items in the test shoud be taken from the s'oken or the written anguage.
+eection of -ocabuar" can thus be thought of as faiing into the foowing
rough8di-isions according to the four maJor anguage skis!
Listening! 'assi-eKs'oken
#eading! 'assi-eKwritten
+'eaking! acti-eKs'oken
7riting! acti-eKwritten
, four di-isions can be incuded in a singe test. of course. but e-en
then carefu consideration shoud be gi-en to the different weighting each
di-ision wi carr" in the test! for e)am'e. shoud there be a greater
concentration on those e)ica items seected from the students> reading
materiaG &enera" s'eaking. the more eementar" the e-e of the test.
the greater the number of e)ica items associated with the s'oken
anguage.
The test constructor>s task is made much easier if a the students ha-e
foowed a 'articuar s"abus. Le)ica items can then be seected from!
8 the s"abus 9incuding a word freBuenc" ist if a-aiabe:@
8 the students> te)tbook 9'ro-ided the items a''ro)imate to those used in
natura s'eech situations:@
8 the students> reading materia 9e.g. sim'ified readers. iterar" te)ts:@ and
8 e)ica errors taken from students> free8written work 9or from students>
incorrect answers in a coLe test:.
The foowing error. howe-er. ma" be one of -erb 'atterning or sim'" the
wrong choice of -erb!
(s the go-ernment going to contribute the new industr"G
21
(
(f an error of -erb 'atterning. the correct -ersion woud be
(s the go-ernment going to
contri"ute to the new industr"G
(f caused b" the wrong choice of -erb. it woud be
(s the go-ernment going to
su"sidise the new industr"G
Aoreo-er. according to the findings of research> conducted into the
effecti-eness of distractors in muti'e8choice -ocabuar" tests. those
distractors based on students> incorrect answers in coLe tests 9though
moderate" usefu: were found to be ess 'owerfu than
9a: the use of fase s"non"ms 9i.e. words with eBui-aent meanings to the
ke" word underined or shown in itaics in the sentence but
ina''ro'riate in the 'articuar conte)t:!
(>d ike to book two ...
. in the circe. 'ease.
seats 1 correct word
chairs 0 fase s"non"m
9b: conte)tua" ree-ant items 9i.e. words reated to the conte)t but
different in meaning to the ke" word in the sentence:!
>How much is a ... . to Tok"o. 'easeG>
>Nine hundred "en. and a return is si)teen hundred "en.
single 0 correct word
tic-et 0 conte)tua" ree-ant
The test constructor is faced with a difficut 'robem if the testees
ha-e foowed different s"abu T>s. +uch a situation is genera" associated
with 'roficienc" tests. in which.student>s suitabiit" and 'otentia for a
certain task are tested 9e.g. uni-ersit" studies in the medium of Engish:.
(n these cases. the tester ma" wish to base the seection of e)ica items on
those used in the tasks for which the student is being tested. ,n aternati-e
method. a''ro'riate for a kinds of tests. is the seection of items from
such we8known word ists as
A Beneral 'ervice (ist o! English )ords
9Aichae 7est 8 Longman:.
Cam"ridge English (e4icon
9#oand
Hindmarsh 8 Cambridge: and
&he )right 6re,uenc# Count. These 9and
other: word ists. howe-er. are based entire" on the written anguage@
furthermore. no account is taken of difficut" e-es 9as o''osed to
freBuenc" e-es: and of areas where interference is encountered between
the -ernacuar and the target anguage.
But testing the e)tent of a student>s -ocabuar" is on" one as'ect of
the whoe 'robem! contro of the -ocabuar" at his or her dis'osa must
aso be measured. ,n abiit" to discriminate between words is of the
utmost im'ortance at a but the eementar" e-es. (n broader terms. this
abiit" to discriminate ma" be regarded as de-eo'ing a !eel for the
anguage.
Tests of -ocabuar" shoud a-oid grammatica structures which the
students ma" find difficut to com'rehend. +imiar". tests of grammar
shoud contain on" those e)ica items which 'resent no difficut" to the
students.
2.< Auti'e8choice (t is usefu to distinguish between the foowing two maJor kinds of
items 9,: muti'e8choice -ocabuar" items!
@.
&rou' , Choose the etter of the word which is the nearest in meaning to
the word in itaics.
He>s been -er" !ee"le since his iness.
,. unwe B. thin C. fooish 6. weak
&rou' B Choose the etter of the correct or best word to com'ete each
sentence.
Ha-e "ou heard the 'anning committee>s ...
. ror so-ing
the cit">s traffic 'robemsG
,. theor"
B. design C. 'ro'asa 6. 'ur'ose
This section concentrates on &rou' , items and the ne)t section on
&rou' B. The foowing item t"'es are e)am'es of four -ocabuar"
recognition items which fa within the first grou'.
T"'e 1 (n this t"'e of recognition item the stem is re'aced b" a 'icture.
The testees see the 'icture and ha-e to seect the most a''ro'riate word
reating to the 'icture from four or fi-e o'tions. This t"'e of item is cear
-er" a''ro'riate at the eementar" stages.
,. running
B. Jum'ing
C. standing
6. kicking
T"'e < Here the stem consists of a definition! the testees ha-e to seect th
correct o'tion to which the definition refers. .
a 'erson who recei-es and 'a"s out mone" in a bank
,. broker
B. accountant C. creditor 6. cashier
T"'e 3 The stem consists of a e)ica item! the testees ha-e to seect the
best s"non"m or definition.
ad-ocate
,. su''ort B. ad-ise
C. contradict 6. damage
diator"
,. growing gradua" arger C. showing care and effort
B. sow in getting things done
6. hea-" with dro's of water
T"'e = The stem here consists of a sentence. Hence. this t"'e of
recognition item is genera" to be 'referred to the 're-ious three t"'es ir. so
far as the >'robem> word a''ears in conte)t. Eocabuar" is much more
usefu" tested in conte)t since it is the conte)t that gi-es s'ecific meaning
and ree-ance to a word. thus creating a situation which is as inguistica".
-aid as 'ossibe in the circumstances.
(t>s rained continuousl# !or
two whoe da"s.
,. without sto''ing C. reguar"
B. hea-i" 6. at odd moments
((
+ince subte shades of meaning are often determined on" b" the
s'ecific conte)t in which a 'articuar word a''ears. it is genera" ad-isabe to
'ro-ide fair" fu conte)ts for -ocabuar" testing. es'ecia" at an.
ad-anced e-e. The fuer the conte)t. howe-er. the. more difficut it
sometimes can 'ro-e to find 'ausibe distractors. ;ew good distractors. for
e)am'e. can be found for the foowing item!
7e>-e had to put oC the meeting unti ne)t week.>9'ost'one:
+"non"ms are not awa"s interchangeabe in a conte)t 9without
atering the meaning:. Howe-er. where a word ma" be re'aced b" another inHa
'articuar conte)t. testees ma" easi" be mised into regarding
s"non"ms as being genera" interchangeabe.
Buidelines !or writing $ ( (f the8'robem8area8being tested.is ocated in the o'tions. 9as8in T"'e <:.
items the stem shoud be ke't sim'e. (f. howe-er. the 'robem area is incuded
in the stem 9as in T"'es 3 and =:. the o'tions themse-es shoud be sim'e in
so far as the" shoud contain on" those -ocabuar" items which the testees can
understand.
< Each o'tion shoud beong to the same word cass as the word in the
stem. 'articuar" when the word a''ears in the conte)t of a sentence. (f
this rue is obser-ed. there wi be ess danger of the conte)t 'ro-iding
im'ortant grammatica cues for the testees. ;or e)am'e. athough the
f i rst of the foowing test items is usabe. o'tions ,. B and C in the second
item woud be grammatica" incorrect when 'ut in the conte)t.
contem'tuous
,. dee' in thought C. sef8satisfied
B. without a sense of humour 6. scornfu
(an was contemptuous of the efforts of his friends to raise some mone"
for the charit".
,. dee' in thought C. sef8satisfied
B. without a sense of humour
6. scornfu
3 The correct o'tion and the distractors shoud be at a''ro)imate" the
same e-e of difficut".> (f the correct o'tion is more difficut than the
distractors. the testees wi arri-e at the correct answer b" 'rocess of
eimination. Thus. the test ma" ha-e a negati-e effect on the testees! i.e.
the" wi seect the correct o'tion not because the" know it is correct but
on" because the" know the other o'tions are wrong. The foowing item
measures the testees> knowedge of the distractors rather than their
famiiarit" with the correct o'tion!
theatrica
,. angr" B. histrionic C. 'roud 6. fooish
The con-erse aso hods good. (f the distractors are more difficut than the
correct o'tion. the item ma" be eBua" unreiabe. (n such a case.
there wi usua"8be a tendenc" for the more abe students to think that the
correct o'tion is too eas" and therefore wrong! the" are thus tricked into seect
g one of the more difficut o'tions!
suffice
,. be adeBuate
B. harass C. acBuiesce 6. be contrite
= There is some disagreement concerning the reationshi' of the o'tions tk
the 'robem area being tested. +ome test writers argue that the o'tions
shoud be reated to the same genera to'ic or area. whie others 'refer as
wide a range of associations as 'ossibe. *ness the -ocabuar" item being
tested has a -er" ow freBuenc" count 9i.e. is -er" rare" used:. howe-er.
the item writer is ad-ised to imit the o'tions to the same genera area of
acti-it" where 'ossibe.>8
Item = Item$2
a''arition a''arition
,.skeeton ,.scener"
B.ghost
B.ghost
C.nightmare C.magician
6.cor'se
6.caste
(f item < were set in a test. students who had read a few ghost stories
woud 'robab" seect o'tion B because the" woud associate
apparition
with the stories the" had read. (n item (. howe-er. students are reBuired t
show a much greater contro o-er -ocabuar".
2 , the o'tions shoud be a''ro)imate" the same ength.< There is a
tem'tation both in -ocabuar" and in reading com'rehension tests to mak!
the correct o'tion much onger than the distractors. This is 'articuar"
true in a -ocabuar" test item in which the o'tions take the form of
definitions! the item8writer tends to take great 'ains to ensure that the
o'tion is absoute" correct. Buaif"ing it at great ength. Howe-er. the
item8writer rare" takes such troube o-er the distractors. since the" are
deiberate" wrong and need not be Buaified in an" wa".
a hitch8hiker
,. a man who makes ro'es
B. a 'erson who tra-es about b" asking motorists to gi-e him free ride
C. an od8fashioned saior
6. a bo" who waks ong distances
,n" student who did not know Die meaning of
hitch$hi-er woud cear"
choose o'tion B 8 and woud be correct in doing so. ConseBuent". if it is
e-er necessar" to Buaif" a definition at some ength. either one distractor
or a three or four distractors shoud be made eBua" ong. (n this wa".
the correct o'tion wi be disguised a itte more effecti-e".
(t is ad-isabe to a-oid using a 'air of s"non"ms as distractors! if the testees
recognise the s"non"ms. the" ma" reaise immediate" that neither is the correct o'tion.
since there can be on" one correct answer.
The od woman was awa"s
courteous when an"one s'oke to her.
,. 'oite B. gad C. kind 6. 'eased
E-en such near s"non"ms as
glad and pleased are sufficient to indicat.
to inteigent students that the choice must be between polite and -ind,
since if glad were correct.
pleased woud 'robab" aso be correct.
(t is aso dangerous to D'air off> o'tions b" 'ro-iding an anton"m as !
distractor. 4'tions , and C in the foowing -ocabuar" item immediate.
stand out@ again. ce-er students wi be abe to narrow their choice down
to two o'tions once the" reaise that , means the o''osite of C.
ascend
,. go u' B. tak
C. come down 6. fetch
2.3 Auti'e8choice
The guideines gi-en in 2.< for constructing -ocabuar" items a''" eBua"
items 9B: for the &rou' B items now being treated. (n certain wa"s. the items shown
in this section are more difficut to construct than those in the 're-ious
section. The 'robem is chief" one of conte)t! too itte conte)t is
insufficient to estabish an" meaningfu situation. whie too much conte)t
ma" 'ro-ide too man" cues 9both grammatica and semantic:.
1. ( saw a nast" ... between two cars this morning.
,. ha''ening B. danger C. damage 6. accident
<. ( was s'eaking to Cath" on the 'hone when sudden" we
were ...
,. hung u' B. run out C. broken down 6. cut off
3. ( shoud ha-e returned this book ast Tuesda"! it is now fi-e
da"s ...
,. 'ost'oned B. e)cessi-e C. o-erdue 6. dea"ed
=. Nothing had been organised and confusion seemed ...
,. ine-ident
B. inefficient C. ineigibe 6. ine-itabe
2. Tom awa"s tries to he' 'eo'e. but recent" he has been
. kind and generous.
,. chief" B. es'ecia" C. 'rinci'a" 6. fundamenta"
Aan" muti'e8choice -ocabuar" test items of the t"'e being deat
with in this section re" on the conte)t itsef to 'ro-ide grammatica cues
which automatica" rue out at east one of the o'tions. These kinds of test
items are usefu in man" res'ects but ma" 'ossib" beong8more to tests of
grammar and structure rather than to -ocabuar". Ne-ertheess. there can
be itte obJection to introducing. sa". a few items on -erb 'atterning in a
test of -ocabuar".
?. (>m . . of getting a new Job! ( don>t ike m" 'resent one.
,. contem'ating B. thinking C. desiring 6. ho'ing
7. ,nn ...
. me of a gir ( used to know.
,. recas B. reminds C. remembers 6. recoects
(t is sometimes argued that mar.> iTiuiii' c Cu-iCc -ocabuar" tests
consist arge" of items such as the foowing and that these test on" a
knowedge of coocation.
0. The tee-ision station was ...
. with etters and 'hone cas
after the announcement.
,. drowned B. stormed C. deuged 6. absorbed
+ince this item ignores the abiit" to create une)'ected coocations. it can
aso be argued that an imaginati-e use of the anguage is discouraged.
,though there ma" be some truth in this argument. une)'ected
coocations resut from a creati-e and intuiti-e handing of anguage.
which in turn demands an im'icit understanding of e-er"da" coocations. (t
is usua" the writer>s -er" awareness of the degree of incongruit" which
makes a new coocation -igorous and meaningfu.
,though the coocations in such items as the foowing ma" be tested
eBua" we without a conte)t. it is usua" ad-isabe to test them in
sentences.
2?
/. 6r Heston charges a high ... . for his ser-ices.
,. fee B. 'rofit C. saar" 6. 'a"ment
9Coocations being tested here. for e)am'e. are! charge a feeKmake a
'rofitKrecei-e a saar"Kmake or recei-e a 'a"ment 8 athough it is
'ossibe to charge a 'a"ment to an account.:
13. ( don>t beie-e "ou! ( think "ou>re ...
ies.
,. sa"ing B. taking C. s'eaking
6. teing
11. (ron wi e-entua" ... ! . if grease is not a''ied.
,. wear B. corrode C. damage 6. corru't
1<. A" dri-ing icence ... . at the end of this month.
,. e)'ires B. 'asses out C. retires 6. concudes
(f se'arated from such conte)ts as the 'receding ones. these test items
woud read!
/. charge a !ee2pro5t2salar#2pa#ment
13. sa#ltal-lspea-ltell lies
11. iron wears2corrodes2damages2corrupts
1<. a icence e4pires2passes out2retires2concludes
(n this t"'e of item. howe-er. each conte)t reBuires a Dnorma>
reaction and takes no account of cutura differences. ;or e)am'e. in the
foowing item B or 6 woud be correct in certain societies since it is
im'oite to acce't a gift without first -ehement" refusing it.
Emma cried out with ... at the beautifu 'resent Ars 7hite
ga-e her.
,. deight B. horror C. disma"
6. anger
(n -iew of such ambiguit". it is e-en more im'ortant than usua to
'ro-ide a conte)t for this 'articuar kind of item. The foowing is a L"'ica
e)am'e of this t"'e of muti'e8choice item as it a''ears in se-era
tests. The diaogue takes 'ace in a doctor>s surger" which has a
'harmac".
5,*L LEE! Can "ou te me what time the doctor>s 91: ...
. o'ensG
A#+ $(N&! (t>s o'en now. The 9<: ... . wi he' "ou.
5,*L LEE! E)cuse me. ( Just want to coect a 93: ...
A#+ $(N&! (s it for some 9=: ... . for a headacheG
5,*L LEE! No. it>s for some cough 92: . . .
A#+ $(N&! Here it is. This shoud soon 9?: ...
. "our bad cough.
91: ,. office B. surger" C. hos'ita
6. ward
9<: ,. 'orter B. hostess C. waitress 6. rece'tionist
93: ,. 'rescri'tion
B. reci'e C. cure 6. direction
9=: ,. 're-ention B. iBuid C. medicine
6. soution
92: ,. mi)ture
B. drink C. wash 6. com'ound
9?: ,. 're-ent B. treat C. refresh 6 cure
This is undoubted" the most common t"'e of muti'e8choice -ocabuar"
item. The 'ro-ision of a conte)t. howe-er. imits the test constructor to
testing on" the -ocabuar" associated with a 'articuar to'ic. Hence. man"
we8known tests sti tend to incude singe sentence -ocabuar" items
rather than fu" conte)tuaised ones in order to co-er the range of
27
11
1
-ocabuar" considered>desirabe for sam'ing. The choice between the use
of singe sentences and the use of 'aragra'hs 'ro-iding a far fuer conte)t
wi be determined b" the 'ur'ose of the test and the test writer>s own
a''roach to the communicati-e as'ects of anguage earning. The Buestion
is sim'" whether the testing of anguage in conte)t is worth the sacrifices
demanded. and the answer must differ according to each 'articuar
situation.
2.= +ets 9associated Aan" of the difficuties arising from the testing of coocations are a-oided
words: b" the testing of word sets. (n such tests the students> famiiarit" with a
range of associations is measured.
T"'e 1! #ecognition
#ead each of the foowing ists of four words. 4ne word does not
beong in each ist8!85ut8a cTree round the odd word in each ist.
son ha''" arri-e
father married de'art
bo"
engaged go awa"
brother singe ea-e
T"'e <! 5roduction
Each grou' of words is reated to a 'articuar subJect. 7rite down the
'articuar subJect which is connected with each grou' of words.
hand theatre
-oume nurser"
wrist sister track ift
dia
bed head so'e
face ward s'oo snow
91 watch:
91 hos'ita:
91 ta'e recorder: 91 skiing:
2.2 Aatching items
T"'e ( of the foiuwing est items suffers from testing e)ica items from
different word casses. whie T"'e < tests a mi)ed hag. of tense forms. etc.
The resut is that for both t"'es of test items grammatica cues assume
great im'ortance. since the" are instrumenta in imiting the range of
choices facing the testees for each bank. ;or e)am'e. athough there ma"
a''ear to be <3 words for seection for bank 91: in T-'e t . in 'ractice
there are on" three which woud fit grammatica"!
turned /down1, "ro-en
/down1, -noc-ed /down1. +imiar". in the first sentence of T"'e < there are
on" two o'tions
/pull through, get wa#1,
since a the other o'tions are
either 'ast tense forms or 'artici'es. Both items need to he rewritten.
therefore. if a higher degree of reiabiit" is to he obtained.
T"'e (
7rite the correct word from the foowing ist at the side of each number on
"our answer sheet. *se each word once on".
road accident tra-eing turned side
broken know
knocked midde
ooked
orr" 'oiceman 'a-ement running hurt
"ing
crossed eft forgot tak
5oor Tom 7right was 91: down b" a 9<: ast week when he was crossing the
93:. He was Buite bad" 9=: and he8had to go into hos'ita for a few da"s.
His eft eg was 92: and both his arms were cut. 7hie he was 9?: in. bed in
the hos'ita. a 97: came to 90: to him.
20
>7as the orr" 9/: -er" Buick"G> he asked Tom.
Tom tod him a about the 913:.
>( was 911: home from schoo and 1 91<: the road. 1 913: right but 1 91=: to
ook 912:. (n the 91?: of the road ( sudden" saw a orr". ( didn>t 917: what
to do. so ( began to run to the other 910: of the road. The orr" 91/: but it
hit me when ( was near the 9<3:.>
T"'e <
Com'ete the foowing sentences with the most suitabe -erb 'hrase
from the ist.
came about 'u through
broken out faing out
running into brought u' get awa" 'ut off
1. >6id the 'risoner manage to ... G> >%es. the 'oice are sti
ooking for him.>
<. The doctor thought Ar Benson woud ... . after the
o'eration.
3. The cou'e are awa"s ... . and causing a disturbance.
=. ,nd so it ... . that we e-entua" 'arted.
9etc.:
(t is much more efficient to test words from the same word cass 9e.g.
nouns on" in T"'e 1:. or 'arae tense forms 9e.g. the 'ast sim'e tense in
T"'e <:. Thus the T"'e < item coud be rewritten as foows!
came about ran into 'ued through
got awa"
1. >( hear the 'risoner ... . "esterda" and the 'oice are sti
ooking for him.>
<. >7e were a reie-ed that Ar Benson ... . after the
o'eration.>
9etc.:
T"'e 3
;rom the ist of words gi-en. choose the one which is most suitabe for
each bank. 7rite on" the etter of the correct word after each number on
"our answer sheet. 9*se each word once on".:..
,. com'ete" C. busi" E. Buick"
B. 'oite" 6. carefu" ;. angri"
>7rite 91: . . >the teacher shouted 9<: ... . >but don>t
waste time. %ou must get used to working 93: ...
>5ease. sir.> a student said 9=: ... (>-e finished.>
>No. "ou ha-en>t.> answered the teacher. >%ou ha-en>t 92: ...
finished unti "ou>-e rued a ine at the end.> Aeanwhie. the bo" sitting
ne)t to him was 9?: ... . engaged in drawing a ma'.
This t"'e is satisfactor" in man" wa"s because a the e)ica items
tested are ad-erbs. Howe-er. ike the other two t"'es. this t"'e gi-es the
student8too itte choice. ;or instance. there wi be on" one word eft for
the ast number. Thus. it coud be im'ro-ed considerab" b" the addition of
a few other ad-erbs. The ist might then read as foows!
,.com'ete" E.deiberate" (. Buick"
B.hea-i" ;.busi" J.hard"
C.ab" &.hasti" $.angri"
6.'oite" H.carefu" L.suitab"
2/

The first attem't to construct this ist incuded the ad-erbs
silentl# and
alread#, but it was then found that either of these coud be used at 9?:
instead of the correct o'tion "usil#. This iustrates one of the dangers of
this 'articuar testing de-ice! cear" the more distractors there are. the
greater is the chance that one of the distractors might be a correct o'tion for
at east one of the other items.
T"'e =
The most usefu t"'e of matching item is undoubted" that based on a
reading com'rehension 'assage. The students are gi-en a ist of words at
the end of the 'assage and reBuired to find words of simiar meaning in the
'assage. +ince a detaied conte)t is 'ro-ided b" the 'assage and itte
additiona materia is reBuired. this is an economica method of testing
-ocabuar". The chief. risk. here. howe-er. is the du'ication of Buestions! if
one of the reading com'rehension Buestions de'ends for its answer on a
knowedge of the meaning of a 'articuar word. care must be e)ercised
not
to
test that word again in the -ocabuar" section.
(n most we8known tests in which this t"'e of item is incuded. a
different reading te)t from the com'rehension te)t is used as the basis for
the matching -ocabuar" test. Thus. the reading te)t contains on"
Buestions on -ocabuar" and does not incude com'rehension Buestions as
such. (n this wa" the test constructor can be sure that the abiit" to answer
the com'rehension Buestions does not de'end on a knowedge of the
indi-idua words seected for the -ocabuar" test.
(n the foowing e)am'e3 candidates in the test are instructed to
re'ace the words isted beow with the a''ro'riate words contained in the
'assage without changing the meaning.
grou's
owned
s'ecific H H
H H H H H H H H
made u' H H
H H
chief
knowedge
simiar"
H H H H H
cose to each other
were incined
work together
THE TEH*ELCHE+
The Tehueches i-ed in a band 8 usua" of between fift" and a hundred
'eo'e. Each band had e)cusi-e rights to a 'articuar hunting area and
no other band was abe to hunt there without 'ermission. Each band
was com'osed of famiies reated through the mae ine and the man
who ed them was the hunter who had the greatest e)'erience of the
hunting grou's. Each man married a woman from another band and his
sister woud aso marr" men outside his band. (n this
wa# bands in a
neighbourhood were inked b" ties of marriage and so tended to co8
o'erate with each other in hunting and other tasks.
2.? Aore obJecti-e
This section contains e)am'es of t"'es of -ocabuar" items which ha-e
items
a''eared in certain tests. 7hie T"'es 1 and < are usefu for cassroom
testing. T"'es 3 and = are rather artificia. and shoud be a-oided where
'ossibe.
11
1
T"'e 1! 7ord formation test items
9a: 7rite a word in each bank. The word "ou write must be the correct
form of the word on the eft.
9i:C,#E Be ... . when "ou cross the road.
9ii: C#*EL
To mistreat animas is a form of ...
9iii: (NTE#E+T6o "ou think this book is ... G
9i-:ENTE# Can "ou show me the ...
. to the ca-eG
9b: #ewrite the foowing 'aragra'h 'utting in each bank the correct
form of the word in ca'ita etters.
A4AENT
Can "ou s'are a ... G> 5eter asked his brother. He thought he
coud detect a ... ook of im'atience on his eder brother>s
face. but it was gone in an instant.
>(>m -er" bus" at the ...
.> his eder brother said.
7hat is it "ou want to s'eak to me aboutG> he asked 5eter.
5eter>s mind ... . went bank. >(>-e forgotten he said.
>7e. then it must ha-e been nothing of ... . im'ortance.> his
eder brother said rather sarcastica".
T"'e <! (tems in-o-ing s"non"ms
9a: 7rite in each s'ace the best word to re'ace the words underined in
each sentence.
9i: Tom went at once to the doctor>s.
immediate"
9ii: , of a sudden there was a oud cr".
9iii: ( came across an interesting book.
9i-: The boat is o-er fourteen feet in ength.
9b: (n each s'ace write one word that means amost the same as the
word on the eft. The word "ou write must rh"me with the word on
the right.
E)am'e! ear" soon moon
9i: 'urchase ... . die
9ii: miserabe ... . bad
, simiar item ma" be constructed so as to in-o-e anton"ms rather
than s"non"ms. The 'honoogica eement 9rh"ming: in <9b:. howe-er.
ma" on" confuse testees instead of he'ing them. 7ords are tested in
isoation. so. a'art from its sheer no-et". the item is of itte use and is not
to be recommended for most 'ur'oses. The acti-it" in-o-ed is more a
game than a test
T"'e 3! #earrangement items
#earrange the foowing etters to make words. Then use each word in a
sentence of "our own so as to show the meaning of the word.
5LE,5 #4L#% CEL5,
+*H4E (#,CH
E&*#,
,s can be seen. this item is itte more than a crossword 'uLLe. (t ma".
'erha's. be of some use in an inteigence test. but it is of doubtfu use in a
anguage test.
?1
((
2.7 Com'etion items
?<
(
T"'e =! 6efinitions
9a: *se each of the foowing words in a sentence so as to show the
meaning of the word.
econom" 'oitics industrious 9etc.:
9b: E)'ain the meaning of each of the underined words in the
foowing 'hrases.
an archaic word a fortuitous e-ent
These item t"'es are of -er" itte use. The" test writing abiit" in
addition to a knowedge of word meanings. ;urthermore. it is e)treme"
difficut e-en for nati-e s'eakers to 'roduce sentences Dto show the
meaning> of words 8 and it is certain" not a usefu task. , student ma"be
famiiar8with8the meaning8of aword and ma".usG..it.uorrect". without being
abe to e)'ress this meaning cear" in a sentence 9es'ecia" under test
conditions:.
The foowing t"'es of com'etion items can be used for the testing of
-ocabuar". Tests which 'resent such items in a conte)t are genera"
'referabe to those which re" on singe words or on definitions.
T"'e 1
#ead through the foowing 'assage containing a number of incom'ete
words. 7rite each com'eted word on "our answer sheet at the side of
the a''ro'riate number. 9Each dash re'resents one etter.:
+nakes are one of the 91: d8m8n88t grou's of 9<: r8't8888! there
are at east <.333 different 93: s'8c88s of snakes 9=: sc8t8888d o-er
a wide area of the earth. Not a snakes are 92: '88s8 n88s! in fact. the
9?: m8J8888 " are Buite harmess. Contrar" to 97: '8'8(88 beief. a
snake>s 90: f88k8d tongue is not 9/: d8ng 88888 to human beings! it
is mere" for touching and smeing 913: s8bs88n88s. +nakes
911: in88ct 'oison into their. 91<: -i8888>s bod" b" 913: b8t88g him
with their 91=: f88gs.
T"'e <
9a: Com'ete each bank with the most a''ro'riate word to re'ace each
number in the te)t.
#osN,H! 7hat>s the 91: toda"G 91:
A4H,AE6! (t>s the se-enth.
#osN,H! ,t what 9<: does the concert startG 9<:
A4H,AE6! +e-en o>cock. ( think. Just a moment.
( made a note of it in m" 93:. 93:
#osN,H! How ong do "ou think it> 9=:G 9=: H H H
A4H,AE6! (t finishes about ten.
#4+N,H! That>s Buite a ong 92:. isn>t itG 92:
A4H,AE6! ( su''ose so. (t>s three hours.
Note the range of 'ossibe answers. es'ecia" with 3. = and 2 9e.g. 3!
diar". notebook. e)ercise book@ =! ast. take@ 2! time. concert.
'erformance:.
9b: Com'ete the foowing 'aragra'h on 'robems caused b"
weightessness b" writing 4NE word in 'ace of each bank.
(ncreasing ...
. is now being ... . on the effects of
weightessness on man. ;or ... . scientists are ..
. the
8roe of gra-it" on the wa" ces function. E-en in the first manned s'ace8
fights doctors were arge" unaware of the -arious
'robems ... .. b" absence of gra-it". The" found that
weightessness ... . in the redistribution of bood and other
fuids from the egs to the to' of the bod". The" were abe
to . . how astronauts> egs actua" shrank and their faces
sweed during the first few da"s of s'ace fight. Aoreo-er. doctors had
a chance to ... . astronauts both at the time of the fight
b" ... . of tee-ision cameras and after the fight during
e)tensi-e medica ... . +cientists are now ...
. into the
effects of diet and e)ercise as a ...
. of reducing some of the
'robems ... . b" weightessness . . most of the
fundamenta scientific Buestions wi ne-er be satisfactori" ... b"
scientists working on the Earth.
These test items come cose to the kind of item often used to test
reading com'rehension 9described in Cha'ter 0:. Cear". a degree of
com'rehension is necessar" before each of the banks can be com'eted.
The items ha-e been incuded in this cha'ter because there is a deiberate
attem't to concentrate on the testing of -ocabuar" 8 in the first cas8 the
-ocabuar" associated with information about the time and date. and in the
second case with the anguage of research and inBuir".
Notes and references
( &oodrich. H C 1/77 6istractor Efficienc" in ;oreign Language Testing. &E'<(
Duarterl# 11
Cohen. , 6 1/03 &esting (anguage A"ilit# in the Classroom.
Newbur" House
< +ee Harris. 6 5 1/?/
&esting English as a 'econd (anguage.
Ac&raw8Hi.
''. 2=827.
3 North 7estern #egiona ,d-isor" Counci for ;urther Education. ,'ri 1/03.
English as a 'econd (anguage 5a'er 3 C=
?3
i
Listening com'rehension tests
?.1 &enera ,n effecti-e wa" of de-eo'ing the istening ski is through the 'ro-ision
of carefu" seected 'ractice materia. +uch materia is in man" wa"s
simiar to that used for testing istening com'rehension. ,though the
auditor" skis are cose" inked to the ora skis in norma s'eech
situations. it ma" sometimes be usefu to se'arate the two skis for
teaching and testing. since it is 'ossibe to de-eo' istening abiit" much
be"ond the range of s'eaking and writing abiit" if the 'ractice materia is
not de'endent on s'oken res'onses and written e)ercises.
,n awareness of the wa"s in which the s'oken anguage differs from
the written anguage is of crucia im'ortance in she testing of the istening
skis. ;or instance. the s'oken anguage is much more com'e) than the
written anguage in certain wa"s. as a resut of the arge eement of
Dredundanc"> that it contains. ,n e)am'e can be seen in the s'oken
Buestion DHa-e "ou got tc go nowG>. the Buestion being signaed b" the
rise in 'itch on go now and b" the in-ersion of the word order 9i.e. b" both
'honoogica and grammatica features@. Thus.8if the istener did not hear
the Buestion signa Have #ou, the rise in 'itch woud indicate that a
Buestion was being asked. (f the s'eaker surred o-er
got to, the Buestion
woud sti be inteigibe. (n addition. meaning might aso be con-e"ed.
em'hasised and >re'eated> b" means of gestures. e"e mo-ements. and
sight changes in breathing. +uch features of redundanc" as those described
make it 'ossibe for mutiated messages to be understood. e-en though the
fu message is on" 'artia" heard. ;urthermore. the human brain has a
imited ca'acit" for the rece'tion of information and. were there no such
features buit into the anguage. it woud often be im'ossibe to absorb
information at the s'eed at which it is con-e"ed through ordinar" s'eech.
+uch con-ersationa features as re'etition. hesitation and grammatica re8
'atterning are a e)am'es of this t"'e of redundanc". so essentia for the
understanding of s'oken messages.
7hat is the significance of these features for testing 'ur'osesG ;irst".
the abiit" to distinguish between 'honemes. howe-er im'ortant. does not
in itsef im'" an abiit" to understand -erba messages. Aoreo-er.
occasiona confusion o-er seected 'airs of 'honemes does not matter too
great" because in rea8ife situations isteners are abe to use conte)tua
cues to inter'ret what the" hear. ,though isteners re" on a the
'honoogica cues 'resent. the" can often afford to miss some of them.
?=
+econd". im'rom'tu s'eech is usua" easier to understand than
carefu" 're'ared 9written: materia when the atter is read aoud. 7ritten
tests genera" omit man" of the features of redundanc" and im'art
information at a much higher rate than norma s'eech does. ConseBuent".
it is essentia to make 'ro-ision for restating im'ortant 'oints. rewriting
8and re'hrasing them when writing materia for aura tests. The ength of
the segments in each breath grou' shoud be imited during dei-er". for
the onger the segment the greater the amount of information and the
greater the strain on the auditor" memor". The 'auses at the end of each
segment shoud aso be engthened to com'ensate for the ack of
redundant features.
,though not awa"s 'ossibe when istening com'rehension tests are
conducted on a wide scae. it is he'fu if the s'eaker can be seen b" the
isteners. Howe-er e)ceent the Buait" of a ta'e recorder. a disembodied
-oice is much more difficut for the foreign earner to foow. (n 'ractice.
most ta'e recorders are not of a high Buait" and are used in rooms where
the acoustics are unsatisfactor". (f the Buait" of the re'roduction is 'oor.
the test wi be unreiabe. es'ecia" when such discrete features as
'honeme discrimination. stress and intonation are being tested.
,'art from the use of -ideota'e. howe-er. the ta'e recorder is the on"
wa" of ensuring com'ete uniformit" of 'resentation and thus a high degree
of reiabiit". (t is aso 'ossibe to use recordings made b" nati-e s'eakers
and thus 'resent 'erfect modes of the s'oken anguage 8 an im'ortant
ad-antage in countries where nati-e s'eakers are not a-aiabe to
administer the test. Aoreo-er. ta'e recorders are essentia for the
'roduction and use of authentic materia.
;or 'ur'oses of con-enience. auditor" tests are di-ided here into two
broad categories! 9i: tests of 'honeme discrimination and of sensiti-it" to
stress and intonation. and 9ii: tests of istening com'rehension.
?.< 5honeme T"'e 1
discrimination tests
9a: This t"'e of discrimination test consists of a 'icture. accom'anied b"
three or four words s'oken b" the e)aminer in 'erson or on ta'e.
1 < 3
The testees hear!
1. ,. 'in
B. 'en
C. 'air 6.'ain
<. ,. shark B. sock C. sack 6.shock
3. ,. thin
B. tin C. fin 6.din
,fter each grou' of four words the testees write the etter of the most
a''ro'riate word for that 'icture. ;or e)am'e!
1. , <. B 3. B
?2
9b: Con-erse". four 'ictures ma" be shown and on" one word s'oken. (n
this case. it is usua" better if the word is s'oken twice.
, B
C 6
3
B
C
The testees hear!
1. 'ain 8 'ain 91 6:
<. sock 8sock91 B:
9etc.:
T"'e <
The testees hear three sentences and ha-e to indicate which sentences are
the same and which are different.
1. ,. There>s a bend in the midde of the road.
B. There>s a bend in the midde of the road.
C. There>s a band in the midde of the road.
<. ,. (s that sheet o-er there ceanG
B. (s that seat o-er there ceanG
C. (s that seat o-er there ceanG
3. ,. (>-e Just ocked the car in the garage.
B. (>-e Just knocked the car in the garage.
C. (>-e Just ocked the car in the garage.
9etc.:
T"'e 3
9a: (n each of these items one word is gi-en on ta'e whie three or four
words are 'rinted in the answer booket. The testees are reBuired to
choose the written word which corres'onds to the s'oken word.
=. 'po-en. den
)ritten. ,. ten
B. den C. Ben 6. 'en
2. 'po-en. win
)ritten. ,. when
B. one C. wane 6. win
3. 'po-en. 'a"s
)ritten. ,. 'a"s B. 'ra"s C. 'a"s 6. bra"s
9etc.:
??
9b: This t"'e of item is simiar to the 're-ious one@ this time. howe-er. the
words s'oken b" the tester occur in sentences. The four o'tions ma"
then be either written or s'oken.
=. 'po-en. (> thread it for "ou.
)ritten or spo-en. A.
thread B. tread C. threat 6. dread
2. 'po-en. 6id John manage to catch the trainG
)ritten or spo-en. A.
drain B. chain C. 'ane 6. train
3. 'po-en. 5ut the 'an in some hot water.
)ritten or spo-en. A. 'an
B. 'en C. 'in 6. 'ain
9c: This item t"'e> is simiiar to T"'e 39a:! one word is s'oken b" the
tester 9'referab" twice:. Howe-er. instead of a choice of four words.
testees ha-e in front of them a choice of four definitions. The" ha-e thus
to seect the correct definition for the word the" hear.
=. 'po-en. cot 8 cot
)ritten. ,. sto''ed and hed
B. a bab">s bed
C. 'ued b" horses
6. a sma 'et anima co-ered with fur
2. 'po-en. threw 8 threw
)ritten. A. made something mo-e through the air
B. not fase
C. some but not man"
6. made a 'icture or diagram on 'a'er
3. 'po-en. bud 8 bud
)ritten. ,. 'art of a tree or a fower
B. a creature with wings
C. something to see' on
6. not good
The test items described in this section are a of imited use for
diagnostic testing 'ur'oses. eiiabing the teacher to concentrate ater on
s'ecific 'ronunciation difficuties. The items are 'erha's more usefu when
testees ha-e the same first anguage background and when a contrasti-e
ana"sis of the mother tongue and the target anguage can be used. Aost of
the item t"'es described are short. enabing the tester to co-er a wide
range of sounds.
T"'e 39c:. howe-er. tests not on" the abiit" to discriminate between
the different sounds of a anguage but aso a knowedge of -ocabuar". ,
testee who ma" be abe to discriminate accurate" wi ne-ertheess find the
test -er" difficut if he or she cannot understand the definitions in the
o'tions. +imiar". T"'e 39a: is a test of 'honeme discrimination and
s'eing abiit". (n T"'e 39b: 'roficienc" in grammatica structure wi
fa-our the testee. Thus. for e)am'e. a testee who cannot discriminate
between thread, tread, threat
and dread ma" immediate" rue out the
distractors
threat and dread
since the" cannot be 'ut in the 'attern
I>ll
. it !or #ou.
Each indi-idua test item in a the t"'es described must be ke't fair"
sim'e. 4bscure e)ica items shoud be a-oided. This ma" seem to be a
sim'e enough 'rinci'e to obser-e. but the a-oidance of difficut e)ica
items freBuent" makes it im'ossibe to test a the sound contrasts that
need to be incuded in the test. ;or e)am'e. the contrasts shark.
soc-,
?7
(
sac-, shoc- woud not be suitabe for incusion in a test intended for
eementar" earners of Engish.
Auch of the materia in such tests is unfortunate" -er" artificia.
differing great" from s'ontaneous s'eech. ;reBuent" there is a tendenc"
for the tester to ado't a certain tone8'attern and rh"thm which ma" be a
source of irritation to the isteners or affect their concentration. Howe-er.
if the tester changes 'itch 9e.g. i-e. ea-e. i-e: this wi on" confuse the
isteners. Thus. the tester must attem't to 'ronounce e-er" s"abe using
the same stress and 'itch 'atterns.
The abiit" to discriminate between certain 'honemes ma".sometimes
'ro-e -er" difficut for nati-e s'eakers. Aan" Engish diaects fai to make
some of the -owe and consonant contrasts and thus. in addition to a the
other -ariabes 9e.g.8the acoustics of the room. the Buait" of the ta'e
recorder. etc.:. these tests are affected b" the 'ronunciation differences of
nati-e s'eakers.<
?.3 Tests of stress and ,though features of stress. intonation. rh"thm and Juncture are genera"
intonation
considered more im'ortant in ora communication skis than the abiit" to
discriminate between 'honemes. tests of stress and intonation are on the
whoe ess satisfactor" than the 'honeme discrimination tests treated in the
're-ious section. Aost tests are im'ure in so far as the" test other skis at
the same time@ man" are aso -er" artificia. testing the rarer 9but more
>testabe>: features.
T"'e ( The foowing item t"'e3 is designed to test the abiit" to recognise
word stress or sentence stress. The testees isten to a sentence 9usua"
s'oken on ta'e: and are reBuired to indicate the s"abe which carries the
main stress of the whoe structure. The" show the main stress b" 'utting a
cross in the brackets under the a''ro'riate s"abic.
'po-en. (>-e Just gi-en TH#EE books to Bi.
)ritten. (>-e Just gi-en three books to Bi.
9 : 9 :4h : 9M( 9 : 49 (
'po-en. A" ;,ther wi he' "ou do it.
)ritten. A" father wi he' "ou do it.
49M:4 : 9
: 4 44
*nfortunate". this test acks conte)t and is -er" artificia. (t tests on"
recognition of stress and is of imited use for ear8training 'ur'oses.
T"'e < The e)aminer makes an utterance and the testees ha-e to seect the
a''ro'riate descri'tion to indicate whether the" ha-e understood the
origina utterance. The utterance is s'oken once on". but the test is based
on the 'rinci'e that the same utterance ma" be s'oken in se-era different
tone8'atterns indicating a 'ain statement. a Buestion. sarcasm. sur'rise.
anno"ance. etc.
'po-en. Tom>s a fine goakee'er.
)ritten. Tom>s a fine goakee'er.
The s'eaker is
,. making a straightforward statement
B. being -er" sarcastic
C. asking a Buestion
?0
'po-en. %ou wi>send me a cou'e of tickets.
)ritten. %ou wi send me a cou'e of tickets.
This is 'robab"
,. a reBuest
B. a command>
C. an e)'ression of disbeief
'po-en. (> he' ,nn.
)ritten. (> he' ,nn.
The s'eaker is
,. reuctant to he' ,nn
B. eager to he' ,nn
C. making a 'ain statement
This t"'e of test item is sometimes difficut to construct. +ince the
conte)t must he neutra. it is often hard to a-oid ambiguit". There is aso a
danger of in-enting 8odd inter'retations or of concentrating on the rarer
meanings! e.g. sarcasm. iron". increduit". Aoreo-er. it can be argued that
the item tests -ocabuar" and reading com'rehension in addition to
sensiti-it" to stress and intonation.
?.= ia+tatements and
diaogues
These items designed
short sam'es a of s'eech 'and deaswith a -ariet" ofa gnas on theceL ca and
gram grammatica e-es of 'honoog". The" ..re
-er"
suitabe for use in tests
administered in the anguage aborator"
but the" do not resembe natura
discourse. The s'ontaneit". redundanc". hesitations. fase
starts and
ungrammatica forms. a of which constitute such an im'ortant 'art of
rea8ife s'eech. are genera" absent from these t"'es of items sim'"
because the" ha-e been 're'ared 'rimari" as written anguage to be
read
aoud.
Aoreo-er. the res'onses reBuired on the 'art of the isteners are not
communicati-e res'onses in an" sense at a. The isteners are not reBuired
to res'ond b" inter'reting what the" ha-e heard or b" adding further
information. as in rea ife. +uch communicati-e res'onses. athough idea
for man" teaching situations. woud be difficut to incor'orate in such
istening tests. es'ecia" those intended for 'articuar diagnostic 'ur'oses.
Ne-ertheess. the im'ortance of such res'onses in tests of istening shoud
be borne in mind when communicati-e 'roficienc" tests are being
constructed 8 in other words. when the test writer is interested in finding
out what students can
do with the anguage the" are earning.
T"'e ( This item t"'e ma" be incuded in a test of grammar. a test of
reading com'rehension or a test of istening com'rehension. de'ending on
whether the item is written or s'oken. (t tests the abiit" to understand
both e)ica
a
hear a statement 9usua" on t
a'e: and then choose the best o'tion from
four written 'ara'hrases.
'po-en. ( wish "ou>d done it when ( tod "ou.
)ritten. ,. ( tod "ou and "ou did it then.
B. ( didn>t te "ou but "ou did it then.
C. ( tod "ou but "ou didn>t do it then.
6. ( didn>t te "ou and "ou didn>t do it then.
11
1
'po-en. (t took ,an a ong time to find he coudn>t mend m" bic"ce.
)ritten. ,. ,fter a ong time. ,an reaised he was unabe to mend m"
bic"ce.
B. ,an s'ent a ong time mending m" bic"ce but he was at
ast successfu.
C. ,an was a ong time before he found m" bic"ce.
6. (n s'ite of searching for a ong time. ,an coudn>t find m"
bic"ce and. therefore. coudn>t mend it.
7hen constructing these items. it is ad-isabe to kee' the
grammatica. e)ica and 'honoogica difficuties in the stem. ea-ing the
written o'tions free of such 'robems and at a ower e-e of grammatica
and e)ica difficut" than the s'oken stimuus.
T"'e < These item t"'es are more8satisfactor" than T"'e 1 insofar as the"
are an attem't to simuate s'eech situations. The testees isten to a short
Buestion and ha-e to seect the correct res'onse from a choice of four
'rinted ones.
'po-en. 7h" are "ou going homeG
)ritten. ,. ,t si) o>cock.
B. %es. ( am.
C. To he' m" mother.
6. B" bus.
Each o'tion shoud be so constructed as to a'
'ear correct in some wa"
to the testee who has not recognised the correct signas in the Buestion.
Thus. in the 're-ious item. o'tion , woud a''ear correct if the testee had
confused
)h# with )hen, and o'tion 6 if he or she had >heard>
How
signa the Buestion. (f. on the other hand. a testee had faied com'ete" to
'ick u' the 7h8Buestion signa. he or she woud be tem'ted to seect
o'tion B. considering it the answer to a %esKNo Buestion.
The Buestion t"'es shoud be -aried as much as 'ossibe and %esKNo
Buestions incuded as we as 7h8Buestions.
'po-en.6oes ,ison mind "ou 'a"ing the 'ianoG
)ritten.,. %es. she>s awa"s thinking about it.
B. No. she rather ikes it.
C. No. she doesn>t 'a" the 'iano.
6. %es. she must be carefu.
(n this item two of the distractors 9, and 6: are based on confusion
reating to mind in order to tem't an" testee who has faied to understand
the Buestion accurate". 6istractor C has been incuded to attract an"
testee who has genera" misunderstood the Buestion and thinks it is about
,ison 'a"ing the 'iano.
(t can be argued that for certain tests the 'rimar" 'ur'ose of a
istening com'rehension item shoud be to test com'rehension aone and
not the abiit" to seect an a''ro'riate re'" to a stimuus. (t is 'ossibe
that a student who fais to answer this t"'e of item correct" ma" ha-e
actua" understood the statement or Buestion but faied to seect the
correct re'".
;urthermore. each8statement or Buestion which 'ro-ides the stimuus
in this t"'e of item usua" takes the form of an isoated item 'resented out
of conte)t and occurring with other unreated items. Conseniuent"
73
11
?.2 Testing
com'rehension
through -isua
materias
1
students who answer a these items in a test ha-e to accom'ish an
intricate seBuence of menta g"rations. sudden" Jum'ing from one
situation to another. Ne-ertheess. this item t"'e is usefu for se-era
'ur'oses in cass 'rogress tests 'ro-ided that the imitations indicated here are
recognised and the item t"'e is not o-er8used.
Aost of the item t"'es in this section are more a''ro'riate for the
eementar" stages of earning Engish. The" are. howe-er. 'referabe to
the discrimination items 're-ious" discussed as the" in-o-e the testing of
grammar and e)is through 'honoog". 5ictures. ma's and diagrams can he
used effecti-e" for testing such skis. thereb" making the testee>s
'erformance ess de'endent on other skis 9e.g. s'eaking. -ocabuar" and
reading:.
T"'e 1 (n this item t"'e a 'icture is used in conJunction with s'oken
statements. The statements are about the 'icture but some are correct and
others incorrect. The testees ha-e to 'ick o o ut the true 9i.e.
statements and write T 9or 'ut a tick -: a side a''ro'riate '5ro5
numbers. The" 7rite ; 9or 'ut a cross ".
fase 9i.e. incorrect: statements.
'po-en.
: at the side of the numbers of the
1. The orr">s on the eft of the motorc"cist.
<! The car>s tra-eing in the same direction.
3. , dog>s running in front of the car.
=. , itte gir>s running after her mother.
2. +he>s hoding a do.
?. Her mother>s carr"ing a bag.
7. The two bo"s are ooking in a sho' window.
0. , -er" sma bo">s he'ing the od woman.
/. The od woman>s going into a sho'.
13. , ta man>s 'osting some etters.
11. There are a ot of cars in the street.
1<. The two bo"s are on the same side of the street as the itte gir.
>i
i
(
)ritten.
1. <. 3. =. 2. ?.
7. 0. /. 13.
11.
1<.
T"'e < (n the foowing istening tests students ha-e fi-e 'ictures in front of
them. The" isten to four sentences. at the end of which the" are
reBuired to seect the a''ro'riate 'icture being described.
(
The testees hear!
1. Both car doors are o'en.
<. (t>s da"ight but both headam's are on.
3. The man who>s re'airing the car is "ing underneath it.
=. ,though the gir sees the man working hard. she doesn>t he' him.
Thus the testees are abe to narrow down the choice a-aiabe to them as
foows!
1 . B C 6 E
94n" , shows one door o'en:
<.B 6 E 94n" C shows the headam's off:
3.B 6 94n" E shows the man standing u':
=.6 94n" B shows the gir he'ing the man:
T"'e 3 The foowing t"'e of test item= is used in a number of istening
com'rehension tests. The testees see a set of three or four 'ictures and
hear a statement 9or a short series of statements:. on the basis of which
the" ha-e to seect the most a''ro'riate 'icture. (n the test the testees
often see a tota of ten or twe-e such sets of 'ictures.
1.
3.
=.
The testees hear!
1. The car>s going to crash into a tree.
<. 6ann" can>t run as fast as Caire.
3. Tom wishes his sister coud 'a" tennis with him.
=. The switch is so high that $atie can>t reach it.
T"'e = +im'e diagrams 9consisting of ines. sBuares. rectanges. circes
and trianges: can be drawn to function as o'tions in a test of eementar"
com'rehension.2 (ustrations of this nature end themse-es in 'articuar to
testing such grammatica features as com'arison. 're'ositions and
determir. rs.
Look carefu" at each of the four diagrams. %ou wi hear a series of
statements about each of the diagrams. 7rite down the a''ro'riate
etter for each statement.
, B C 6
4
4
3
'po-en.
1. ,! LookR 7hat>s that inside the sBuareG
B! (t>s a white circe.
<. ,! (s that a back circeG
B! 7hereaboutsG
,! ,bo-e the sBuare.
B! %es. it is. (t>s a back circe abo-e the sBuare.
73
3. ,! (s the white circe on the eft of the sBuareG
a! No. it>s on the right of the sBuare.
=. ,! (s there an"thing at a in the sBuareG
a! No. it>s com'ete" em't". There>s neither a white nor a back
circe in the sBuare.
2. ,! There>s nothing at a under the sBuare. is thereG
a! No. "ou>re wrong. There>s a back circe under the sBuare.
?. ,! 7hat are "ou ooking atG
a! (>m ooking at a sBuare.
c! 7hich sBuareG
6! The one under the back circe. of course.
7. ,! (s the circe on the eft of the sBuareG
888a!8No. it8isn>t8The8sBuare>s on the. eft of the circe.
0. ,! 7hat>s unusua about this drawingG
B! 7e. there>s a back circe inside the sBuare.
,! That>s not unusua. There are two sBuares with back circes inside.
a! But this one (>m ooking at has a white circe Just outside the
sBuare.
, kinds of sha es and forms can be used to test isteningt.
(
com'rehension. The foowing e)am'e iustrates how an understanding C4
com'e) structures can be tested in this wa". Howe-er. there is often a
tem'tation for the test writer to be too Dce-er> and set an item testing
inteigence 9or menta agiit": rather than anguage acBuisition 8 as in the
foowing e)am'e.
,
B C
6
'po-en. (f the thick ine had been on" a miimetre onger. it woud ha-e
been the ongest of the three ines.
+hort con-ersationa e)changes as in the first e)am'e of T"'e = are
far 'referabe to singe sentences 9as in T"'es 1. < and 3:. , four t"'es.
howe-er. are scarce" -aid tests of the abiit" to understand natura
discourse. Ne-ertheess. such items are of some use for certain 'ur'oses
and are usua" Buite reiabe guides to 'articuar as'ects of the istening
abiit".
T"'e 2 This test is designed to assess the testee>s abiit" to understand
sim'e instructions. ,n" street ma' can be used or ada'ted for this
'ur'ose 9see the e)am'e at the to' of the ne)t 'age:.
'po-en. %ou come out of schoo into Centra #oad and wak in the
direction of &reen Lane. Howe-er. "ou take the eft turning Just before
"ou reach &reen Lane. ,t the end of the street "ou turn right and
continue unti "ou come to the second turning right.. %ou cross this road
and "ou wi see on "our right ... 97hich buiding wi "ou
seeG:
7=
11 1
7est 6err#
East ;err"
1
+ea-iew &ardens
North +treet
5ost
4ffice
Centra #oad
+choo
Head ,-enue
Cinema
&o-ernment
4ffices
+'orts &round
+tadium
The foowing istening com'rehension test is in the form of a
diaogue. The idea on which it is based was suggested b" an actua robber"
and a 'oice8 chase. (n this wa". the istener is gi-en a greater sense of
reaism and an added interest in the diaogue.
&reen Lane
;ootba +tadium
Bramho'd ,-enue .6
co
3
#egents Lane P
Centra
(ndoor
Aarket
#itL Cinema
Cit" Ha
a7oo
4)ford +treet
7ison>s
Jeweer>s +ho'
( X n (
+outh +treet
Centra
Librar"
#aiwa"
+tation
,! Ha-e "ou heard about the raid on the Jeweer" sho' in Cifton
#oadG
B! %es. in fact. ( saw 'art of the chase. (t was e)traordinar".
,! (>-e on" heard a -er" -ague re'ort about it. 7hat e)act"
ha''enedG
B! 7e. the thie-es 'anned to rob the sho' 8 "ou know..7ison>s in
Cifton #oad 8 Just after it>d o'ened ear" "esterda" morning.
72
i
,! (n broad da"ightG ( didn>t know that.
B! The" 'anned to arri-e as the Jeweer" was being taken from the
safe into the big dis'a" window. The" arri-ed in a arge red car
which the" 'arked on the o''osite side of the road. Can "ou see
the 'ace on this ma' (>-e gotG
1. )! 7rite the etter , on "our ma' at the 'ace where the thie-es
'arked the car.
,! How man" robbers were thereG
B! Three. 4ne waited inside the car and the other two waked o-er
to the sho'. carr"ing arge briefcases. 4nce the" were in the
sho'. the# made the manager and his assistant ie down on the
foor whie the" fied the briefcases with Jeweer". 7hat the"
didn>t know. though. was that another assistant was in the room
at the back of the sho'. He had caught sight of the two thie-es
and had 'ressed a sma aarm be. ,t that 'recise moment. a
'oice 'atro car was at the Junction between Aain +treet and the
road that runs 'ast the ibrar" 8 "ou know. near the #itL.
<. )! 7rite the etter B at the Junction referred to b" the s'eaker.
,! +o things went wrong for the robbers from the startG
a! %es. B" the time the" were ea-ing the Jeweer>s. the 'oice car
was aread" turning into Cifton #oad. The two men hadn>t e-en
time to cose one of the car doors 'ro'er" as the" set off in the
direction of the footba stadium. , 'asser8b" heard one of the
men te the dri-er to take the first turning off Cifton #oad. Just
after that. one of the briefcases fe out as their car swung eft.
3. )! 7here did the thie-es ose one of the briefcasesG
7rite the etter C on the s'ot.
B! But that wasn>t the end of it a. 7hen the" ooked back. the" saw
the 'oice car gaining on them.
,! But the" didn>t gi-e u'G
a! No. the" acceerated. The" turned eft and then the" turned right.
Then the" swung into a narrow street and sto''ed a few "ards
down it at the side of a second car 8 most ike" their getawa"
car.
=. )! 7here was the getawa" car 'arkedG 7rite 6 on "our ma'.
,! Had the" managed to throw off the 'oice carG
B!
No. ,s the" were about to change cars. the" heard it coming u'
behind them. +o the" changed their minds and started off again
in the red car. ,t the end of the narrow street. the" turned eft
into Hiiard #oad again and s'ed off in the direction of the
stadium. ,t the ne)t but one Junction before the stadium 8 "ou
know. on the south side of it 8 a second 'oice car sudden" cut
across their 'ath and forced them to sto'.
2. )! 7rite E at the 'ace where the robbers were forced to sto'.
,! 7hat on earth did the" do thenG
B! 7e. b" this time. the" were rea" des'erate. The dri-er of the
red car got out and fired a 'isto at the 'oice car. But this didn>t
sto' the 'oice. 4ne of them scrambed o-er the bonnet of the
'oice car and chased the man with the 'isto down Hiiard #oad.
E*
11
1
The man ran in the direction of the Cit" Ha and then took the first
turning eft after #egent Lane. Then he made as if to turn right. but
Just at that moment. he tri''ed and fe. (n a matter of seconds.
two 'oicemen were on him.
?. )! 7here was the man caughtG 7rite the etter ; to show the 'ace.
,! 7here were the other two robbers whie a this was ha''eningG
a! 7e. b" this time one was hafwa" down #egent Lane with two
'assers8b" chasing him. He si''ed through the narrow ae" at
the end of the ane and turned right before dashing across the
road. He got most of the wa" down the road to the station before
one of the 'assers8b" fina" caught u' with him and
o-er'owered him.
7. )! 7here was this robber caughtG 7rite the etter & to show the
'ace.
,! ,maLingR ( su''ose the 'oice soon caught the third man.
a! No. the" didn>t. The third robber had a shotgun and he>d s'rinted
aong #egent Lane and into Aain +treet. He was about to set uff
running in thedirection of the market. Then he caught sight of a
butcher>s -an tra-eing towards him. He stood Buite sti in the
midde of the road. 'ointed his gun at the bewidered dri-er and
shouted to him to sto' and get out.
0. )! 7rite the etter H to show where the robber sto''ed the -a81.
a! Then he got into the -an. started off down Aain +treet and
turned eft on" to find himsef in the midde of the bus stationR
He Buick" turned round and headed u' Aain +treet. Ne)t he took
the road eading to the main entrance of the footba stadium.
Hafwa" down this road. howe-er. he saw two 'oicemen on
motorc"ces in front of him at the end of the road.
/. )! 7rite the etter ( where the two 'oice motorc"cists were.
,! 7e. he must ha-e been we and tru" cornered b" now.
a! %es. but he sti fired se-era shots at the motorc"cists. Then he
re-ersed and Jum'ed out of the -an at the end of the road. He
turned in the direction of the Cit" Ha. He hadn>t got more than a
"ard or two when he found himsef surrounded b" a doLen
'oicemen.
13. )! 7rite the etter J to show where>the third robber was caught.
,! ,nd so at ast he was caughtR
a! %es. and so was the manager of the Jeweer" sho'.
,! 7hat on earth do "ou meanG
a! 7e. the 'oice ha-e Just8found that it wasn>t rea Jeweer" at a.
(t was imitation stuff. +o !the Jeweer>s been arrested for fraudR
T"'e ? There are man" other wa"s>of e)'oiting -isua materias for testing
sim'e istening com'rehension. The foowing kind of item ma" be usefu in
the testing of the istening abiit" of students of mathematics.
1.
6raw a straight ine ,B three centimetres ong.
<. Continue the ine ,B to 'oint C so that ,C is twice as ong as ,B.
3. 6raw a 'er'endicuar from 'oint B.
=. Aeasure an ange =2P to be caed 6,C.
8
77
i
(
2. Now draw the ine ,6 unti it meets the 'er'endicuar at 'oint 6.
Duestion =. How ong is ,6G
?. Now Join 6C.
Duestion 2. 7hat does ange ,6C measureG
7. 6raw a ine from 'oint BT'arae to ,6 and mark the 'oint M --heTe it
bisects C6.
Duestion 3. How ong is BMG
0. Now draw a ine from M 'arae to ,C so that it bisects ,6 at %.
Duestion +. How ong is ,%G
Duestion F. How man" figures ha-e "ou drawnG
7hat are the"G
T"'e 7 ,nother usefu test item 9and e)ercise: which is inde'endent of the
s'eaking. reading and writing skis is that in which the testees are
'resented with an incom'ete 'icture 9usua" a sim'e ine drawing: and
are reBuired to add to it 'ieces of -isua information according to certain
ora instructions the" are gi-en. The foowing is an e)am'e of such an
item!
9The testees ook at the 'icture:
The testees hear!
1. 6raw a tabe and two chairs in front of the cafe.
<. 6raw two traffic ights on the o''osite side of the road.
70
3. 6raw a Lebra8crossing between the 4)ford +choo and the cinema.
=. 6raw a orr" tra-eing in the o''osite direction to the car Just before
the Junction.
2. , 'oiceman directing traffic is standing in the midde of the Junction.
6raw him.
?. ,though there>s on" one tree at the side of the office buiding. there
are two trees on the o''osite side of the road. 6raw them.
7. +ome 'eo'e ha-e com'ained about the danger of crossing the road
between the cafe and the cinema. , 'edestrian footbridge has now
been buit at this 'oint. 6raw it.
0. , man who has been ceaning the windows of the second foor of the
office buiding o''osite the cafe has forgotten to take his adder awa".
(t>s sti eaning against the window on the e)treme right of the front of
the buiding. 6raw it.
(t is cear" im'ortant to kee' an" such drawing sim'e so that too
much is not reBuired from the testees. Basic 'ractice in matchstick
drawings woud be a usefu 're'aration for such istening com'rehension
tasks in cass 'rogress tests. , sim'e countr" scene in-o-ing the drawing
of cows. trees. tents. 'ots and 'ans. ri-ers. bridges. fish. birds or an indoor
scene in-o-ing the 'ositions of furniture and sim'e obJects might form a
usefu basis for such a istening test. Aoreo-er. it is im'ortant to tr" out
this kind of acti-it" before gi-ing it as a test. item to students. This kind of
're8testing of items wi he' to a-oid such 'robems as ea-ing students
insufficient room to draw a the -arious obJects reBuired.
(t is aso usefu to buid u' an interesting stor" instead of imiting the
com'rehension test to instructions in singe sentences. The foowing
istening test. constructed b" a teacher some time ago. coud ha-e been
much more interesting had it been 'ut in the form of a sim'e stor" or
seBuence of e-ents.
8,a
Look at the drawing and isten to the sentences. %ou wi see Ar 5eanut
sitting at a tabe. 6o what ( te "ou and com'ete the 'icture. (t doesn>t
matter if "ou cannot draw -er" we. ,re "ou read"G Now begin.
1. Ars 5eanut is sitting on a chair o''osite Ar 5eanut.
%9& HE7 I;.
<. The door is behind Ar 5eanut>s back.
%9& I& I;.
3. Ar 5eanut can see Ars 5eanut but he can>t see Charie because
Charie>s standing behind him. %9& HIM I;...
9etc.:
7/.
The test was ater 'ut into the form of a short narrati-e and found to be far
more interesting!
4ne da" Ar and Ars 5eanut were sitting down to ha-e a mea when
something strange ha''ened. Ar 5eanut had his back to the door and
Ars 5eanut was sitting o''osite him. 6raw them in "our 'icture.
/%ause1
Their sma 'et dog was asee' under the tabe hafwa" between them 8
a itte nearer Ar 5eanut than Ars 5eanut. 6raw the 'et dog.
/%ause1
+udden" the door was fung o'en and a "oung man rushed into the
room and stood behind Ar 5eanut. hoding a ong knife. Can "ou draw
himG /%ause1 9etc.:
7hen 'ut into the form of a sim'e narrati-e. the test at once
becomes interesting. The foowing diagram is used for a istening test
based on the reconstruction of an actua batte on a warehouse roof
between two "ouths and the 'oice.
Ac 64N,t.4! n Ac
H,##L+oni! JH
mr(E s ! J,,
The test? consists of two 'eo'e discussing what ha''ened. minute b"
minute. in the e-ents eading u' to the shooting of a 'oiceman. +tudents
are asked to 'ot the 'ositions of the 'oice and the "ouths on the roof of
the warehouse at -arious times in this gun batte. according to the
information the" hear on the ta'e. +e-era co'ies of the 'an of the
warehouse roof are gi-en to each student so that the" can draw the
'ositions of the indi-iduas at the -arious times stated in the discussion.
(ndeed. it is on" through buiding u' a 'icture of the seBuence of e-ents in
this wa" that it becomes 'ossibe to gain an understanding of what rea"
ha''ened on the night of the shooting. ,s a resut of attem'ting to do this.
students e)'erience a dee' interest and a rea sense of 'ur'ose when
istening to the discussion.
03
T"'e 0 +e-era we8known e)amining bodies freBuent" use 'ictures based on
a istening com'rehension tak in order to test students> abiit" to
identif" and cassif" information which the" ha-e heard. The foowing is an
e)am'e> of such a t"'e of item.
(n this 'art "ou wi hear a tak about drawings made b" chim'anLees and
chidren. %ou wi hear information twice. Then "ou shoud!
1. identif" the nine sentences. from , to $ beow. which are about the
de-eo'ment of a chid. Number them in the order the" occur. 7rite
the numbers 18/ in the bo)es.
<. cassif" the diagrams b" writing abo-e each diagram an a''ro'riate
s"mbo from the ke".
$e"
C6 Chid on"
C5 Chim' on"
B Both
M Not in the two
stages described
,. sha'es inside circes
&. unconnected ines
B. singe ines H. e-er"da" obJects
11
C. human figures rows of sha'es
6. singe sha'es J. masses of connected ines
E. o-era''ing sha'es $. human faces

;. reguar 'atterns
T"'e / +im'e 'a'er8foding and drawing can aso be used to measure
abiit" to understand instructions. ;or e)am'e!
/'po-en1 5rint "our name in the to' eft8hand corner of "our 'a'er! draw a
one8inch ine si) inches under it. 6raw a sma circe on the ine and on the
right of it draw a sBuare rough" the same siLe as the circe. Now
take the to' right8hand corner and the bottom eft8hand corner and fod "our
'a'er so that the drawing a''ears on the outside.
5ractica considerations. howe-er. shoud not he ignored in the
administration of this t"'e of test. ,though usefu for ordinar" cassroom
'ur'oses. such tests are difficut to administer in 'ubic e)aminations.
Co'"ing is a sim'e matter and test reiabiit" ma" thus be great" affected.
01
(
Aoreo-er. there is a tendenc" for such tests to become tests of inteigence
rather than of anguage 'roficienc". The item writer must be carefu to test
on" the student>s abiit" to understand a s'oken message 8 not the abiit" to
inter'ret it and see hidden reationshi's.
?.? *nderstanding The abiit" to understand both informa taks and forma ectures is an
taks and ectures
im'ortant ski for students stud"ing subJects in the medium of Engish at
intermediate and ad-anced e-es.
T"'e 1 Testees isten to a short tak and seect the correct answer about
the tak.
'po-en. There>s a marked tendenc" for most de-eo'ed countries to
grow steadi" noisier each "ear. This continua" increasing amount8of
noise is uncomfortabd and. what is more im'ortant. can affect our
heath. The noise of machines. hea-" traffic and aero'anes constitutes
'erha's the most serious threat to 'ubic heath. +uch noise can
interfere with our abiit" to con-erse. it can disturb our see'. and it care
Buick" make us become ner-ous wrecks. , oud bast or an e)'oYion
ma" e-en cause damage to our hearing. But there>s another danger 8
Just as great. This is the gradua damage which ma" be caused if we>re
continua" e)'osed to noise o-er se-era "ears. +uch e)'osure to noise
can undermine our heath 8 as we as our 'erformance and efficienc".
;ortunate". technoog" is 'rogressing at a -er" ra'id rate. +ome
manufacturers are now designing new siencing mechanisms in their
'roducts. and 'anning e)'erts are e-en beginning to 'an cities
according to sound Lones.
)ritten. 4n" one of the foowing statements about the tak "ou ha-e
Just heard is correct. 5ut a circe round the etter ne)t to the correct
statement.
,. Aodern technoog" is now making towns in de-eo'ing countries
free of oud noise.
B. The increase in noise is a 'robem which cannot "et be so-ed b"
modern technoog".
C. &radua noise o-er a ong 'eriod ma" ha-e Just as harmfu an effect
as oud or sudden noise.
6. There is no rea soution to the 'robem of increasing noise in
modern ife.
T"'e < Like T"'e ( this test combines istening com'rehension with
reading co@..'reiieii i> n. The testees hear a short tak and then read a
summar" containing 'anks. The" must then com'ete the banks from the
tak the" ha-e heard. The danger here. howe-er. is that testees coud
successfu" com'ete the written summar" of the tak e-en if on" itte had
been understood.
'po-en. 7oud "ou ike a robot in "our houseG (t>s now genera"
acce'ted that in the future robots wi take o-er man" of our tasks.
es'ecia" Jobs of a re'etiti-e nature. But it>s high" doubtfu if robots
wi e-er be abe to do an" of the more creati-e t"'es of work 8 or
indeed if 'eo'e woud want them to. (n the home. robots woud
'robab" be used to do the ceaning. tabe8a"ing. scrubbing and
washing8u'. but
it>s considered unike" so far that the"> be used to do
the cooking 8 at east. not in the near future. ,ccording to engineers.
11
1
robots wi do nothing more origina or so'histicated than the" ha-e
been 'rogrammed to do b" human beings. ,nd so robots in the home
might not be creati-e enough to do the cooking 'an the meas. and so on.
The" woud be used as sa-es. thereb" freeing 'eo'e to do more of the
things the" wanted.
(n factories. mobie robots woud carr" out a the distribution and
assemb" tasks whie human beings carried out research and drew u'
'ans for new 'roducts. Human beings woud sti be res'onsibe for
diagnosing fauts and for re'airing and maintaining machiner". 4n the
farm. robots woud 'robab" dri-e tractors@ the">d be 'rogrammed to
kee' their e"es on the ground. in front to guide the tractor aong a
straight ine or between rows of -egetabes.
The robots themse-es woud 'robab" not ook at a ike human
beings because their design woud be chief" functiona. ;or instance. it
woud not be at a sur'rising to find a robot with an e"e in the 'am of its
hand and a brain in one of its feetR
)ritten. The foowing 'assage is a written summar" of the short tak
"ou ha-e88Just8heard. &i-e the correct word which can be used in 'ace of
each number.
(n future 91: wi do man" Jobs. 'articuar" those which are 9<: b"
nature..it is genera" doubted if the" coud do 93: work and in the home
the" woud 'robab" not do things ike 9=:. #obots wi do nothing more
92: than the" ha-e been 9?: to do b" human beings. , robot woud be a
kind of 97:. freeing human beings so that the" coud do whate-er the"
wanted. ,though robots woud be used in factories. human beings
woud 'robab" 90: the machiner". 4n farms. robots woud 'robab"
dri-e 9/:. The robots woud ook 913: human beings because the" woud
be 911: in design. (t woud e-en be 'ossibe for a robot to ha-e an e"e in
its hand or a 91<: in one of its feet.
T"'e 3 The testees hear a short tak or ecture and are reBuired to answer
Buestions on it. *ness the" are aowed to take notes on the tak. the test
ma" 'ut too hea-" a oad on the memor". (n certain instances. in fact. it
ma" be desirabe to gi-e them some guidance for note8taking. The
'ro-ision of a ist of 'oints on which Buestions wi be asked ma" im'ro-e
the test.
The foowing is an e)am'e of a test based on a 9fictitious: no-eist@ the
Buestions that foow reate to the no-eist>s 'ace of birth. ear"
infuences on his chidhood. the books he read at schoo. his first
'ubications. his tra-es. etc. The sheet gi-en to each testee a few minutes
"e!ore the ecture reads as foows!
;<&E %A%E7
%ou are going to hear a tak about Chares Edward Backwe. a writer of
chidren>s books. %ou are being tested on "our abiit" to isten and
understand. ,fter the tak "ou wi be asked <2 Buestions about Chares
Edward Backwe.
This sheet of 'a'er is for an" notes which "ou wish to take whie
"ou are istening to the tak. The notes wi not be marked in an" wa" b"
the e)aminer.
The Buestions "ou wi be asked after the tak wi be about the
'oints isted beow. , s'ace has been eft to enabe "ou to write notes for
each 'oint.
0G
((
1. 7hat Backwe enJo"s doing
<.. Backwe>s birth
3. His age at the time of the economic de'ression
=. The books Backwe read
9etc.:
The testees ma" take notes during the ecture. The" wi ater recei-e the
foowing answer sheet.
I;'&79C&I<;'
%ou ha-e Just heard a tak about Chares Edward Backwe. a writer of
chidren>s books. %ou are being tested on "our abiit" to isten and
understand. %ou now ha-e 12 minutes to answer the Buestions which
foow. The <2 Buestions foow the order of the tak and "ou shoud
com'ete each statement with the best answer. 7rite >,>. >B>. >C>. or >6> on
the ine 'ro-ided at the side of each Buestion.
G< ;<& )7I&E <9& &HE
69(( A;')E7.
Here is an e)am'e!
This tak is about
,. writers of chidren>s books.
B. chidren>s reading.
C. Chares Edward Backwe.
6. Leeds *ni-ersit".
1. Backwe enJo"s
,. writing books for chidren.
B. gi-ing ectures for writers.
C. reading books to chidren.
6. taking about himsef.
<. 7hen Backwe was born. his father was
,. a cricketer.
B. an inn8kee'er.
C. a writer.
6. a factor" worker.
3. ,t the time of the great economic
de'ression Backwe was
,. three "ears od.
B. fi-e "ears od.
C. twent"8fi-e "ears od.
6. thirt" "ears od.
7hen Backwe was a bo". he read
,. books about chid geniuses.
B. Tosto">s )ar and %eace.
C. stories written for bo"s of his age.
6. ad-ice about writing for chidren.
9etc.:
E). C
1.
<.
3.
=.
This t"'e of tCZt is genera" administered in one of the foowing wa"s! 1
The testees recei-e note 'a'er and take notes whie the" isten to the
ecture. The" are then gi-en the Buestion 'a'er 9usua" consisting of
muti'e8choice items:.
< The testees recei-e the Buestion 'a'er first and are gi-en a few minutes
to dance through it. The" then hear the ecture and work through the
r1
Buestions. The Buestions are genera" in the form of 9a: muti'e8choice
items. or 9b: trueKfase items. or 9c: incom'ete sentences! Com'etion.
howe-er. is not usua" to be recommended as the testees are faced with
the tasks of istening. reading and writing simutaneous" 8 an e)treme"
difficut o'eration e-en for nati-e s'eakers. E-en muti'e8choice items
ma" cause confusion 9es'ecia" if not carefu" s'aced out throughout
the ecture:. since the testees ha-e to isten whie reading carefu"
through a the o'tions and making their seection. (ndeed. if this
'articuar 'rocedure is to be ado'ted at a. it is 'erha's best to use
trueKfase t"'e items since this reduces the amount of reading and the
seection to be made.
3 The testees isten to the ecture and then recei-e the Buestion 'a'er.
The" read it through and then isten to the ecture gi-en a second time.
,though the testees wi be istening with a 'ur'ose during the second
reading of the ecture. the criticisms made 're-ious" sti a''".
Aoreo-er. this test does not a''ro)imate as cose" to a norma ecturinO
situation as does method 1.
(n writing such tests. it is ad-isabe to gi-e a tak from rough notes or to
record a tak and then to work from a transcri't of the tak in setting
suitabe Buestions. The im'ortance of 'resenting rea s'eech instead of
written 'rose s'oken aoud cannot be em'hasised too strong". This. of
course. ma" be -er" difficut for non8nati-e s'eaking teachers. but access to
radio broadcasts in Engish or recordings of taks gi-en b" nati-e
s'eakers wi be of rea he' in such cases.
Therewi ine-itab" he times when the non8nati-e teacher of Engish
wi be forced to use written te)ts as the basis of a istening com'rehensio
test. (n these cases. it is im'ortan. to kee' to the norma dei-er" rate.
increasing the ength of 'auses at the end of breath segments 9i.e. units of
meaning such as 'hrases. causes and short sentences:. (n addition. the
written te)t itsef shoud be ada'ted to assume the features of s'eech as fa
as 'ossibe! the im'ortant 'oints can be restated in -arious wa"s and
com'e) sentences can be rewritten in the form of short com'ound or
sim'e sentences.
(t is most inad-isabe to attem't to introduce other essentia features
of s'oken discourse when ada'ting written te)ts for the 'ur'oses of
reading aoud. The more the test writer tries to incor'orate such features
as hesitation. fase starts. and ungrammatica sentences. the more artificia
the tak wi become. (n other words. the deiberate introduction of those
s'ontaneous eements inherent in a s'eech wi on" increase the
artificiait" of the situation. (n an e)'eriment conducted se-era "ears ago.
a re'ort about the e-ents in an imaginar" East ,frican countr" was read
aoud and recorded for aPistt ring test b" a teacher 'a"ing the 'art of a
re'orter who had Just returned from the countr" in Buestion. The test
'ro-ed fair" successfu e-en though the students were reBuired at first to
isten to written discourse carefu" 're'ared and read aoud. The same
re'ort was then read aoud to another grou'. the s'eaker deiberate"
introducing hesitation features. making fase starts. etc. Though taking
more time to dei-er and im'arting information. at a sower rate. the re'o
was not on" e)treme" difficut to understand but aso -er" aboured and
irritating for the isteners. , third 'resentation of the same re'ort was th".
gi-en. but this time it took the form of an inter-iew. The >re'orter> began
in a sef8conscious wa". acting his 'art and s'eaking we8rehearsed ines.
Howe-er. the 'erson conducting the inter-iew became so interested in the
subJect that he 'ut the scri't aside and asked Buestions which the Dre'orter>
was not e)'ecting. +udden" the inter-iew came to ife and was far more
natura and s'ontaneous. The inter-iew itsef asted much onger than
either of the 're-ious taks and new Buestions were reBuired. but the
istening test had at once become -aid.
, number of istening tests in-o-e e)tra8inguistic factors 8 memor".
knowedge of a to'ic and interest in that to'ic. (t is thus im'ortant to a-oid
testing memorisation of unim'ortant and irree-ant 'oints in a tak 9e.g.
)hen was the writer>s grand!ather "orn@1. There is itte
Justification for
setting Buestions on such 'oints in a reading com'rehension test 8 and far
ess in a test of istening com'rehension. The taking of notes aso
minimises the memor" factor. but the test itsef ma" then become more a
8888testof8note taking8skisOO8[
,bo-e a. remember that it is the 'ro'ositiona meaning of sentences
which is retained b" the istener 9i.e. their general meaning and intention:
and not the actua words or grammatica forms used to e)'ress that
meaning80 ;or e)am'e. the genera meaning is sti the same whiche-er of the
foowing two sentences is used!
The wear" band of e)'orers managed to cross the wide ri-er in a -er"
sma "acht.
The tired 'art" of e)'orers succeeded in getting to the other bank of the
great ri-er in a tin" dingh".
7e are rare" caed u'on to remember the e)act words someone s'oke in
rea ife uness in -er" unusua circumstances. e.g. e-idence gi-en in a
court case. in which a s'eaker>s e)act words ma" ha-e great significance
E-en in such circumstances. indi-iduas usua" ha-e great difficut" in
recaing the actua words s'oken e-en though the" can remember
'erfect" the genera meaning of what the 'erson said. Therefore. a-oid
setting Buestions which in-o-e the memorisation of indi-idua words in
sentences. (f a summar" of a tak is gi-en for com'etion. the words
omitted in the summar" shoud be those words essentia to the meaning of
d
the whoe tak 9e.g. the word Drobots> in T"'e < in this section:.
T"'e = (t is im'ortant that the content of the te)t itsef shoud determine
the t"'e of Buestion or item used to test com'rehension. There is nothing
intrinsica" either right or wrong about the use of muti'e8choice items for
a istening com'rehension test. (t is essentia. howe-er. that the most
a''ro'riate t"'e of item shoud be used. Certain te)ts wi end themse-es
far more to muti'e8choice items than others@ other te)ts wi end
themse-es to Buestions set in a tabuar form@ others wi suggest -isuas.
whie "et others are better e)'oited if foowed b" o'en8ended Buestions.
The foowing e)am'e/ is incuded to show how trueKfase items can
be de-eo'ed to incude a third choice 9no information a-aiabe:. This is a
'articuar" usefu de-ice in istening and reading com'rehension tests
since freBuent" the information being sought is not contained in a te)t. (t
is Just as usefu to test the abiit" to be aware of im'ortant information
not
gi-en in a tak as it is to test information gi-en in the tak. ;urthermore.
the choice becomes no onger a two8wa" one but a three8wa" one. thereb"
reducing the effect of guessing.
0?
1
%es
No
7
Notes and references
(
<
The The 7e ha-e no
statement statement information
is true. is fase. about it.
The" had a 'robem
taking off because the"
were carr"ing so much
fue.
The" started on June
12th. 1/1/.
(t was fogg" when the"
took off.
The" had a 'robem
when something fe off
the 'ane during the
fight.
The" had difficut"
finding their wa".
The" had no heating in
the 'ane. , snowstorm
affected the engine.
The" were inJured when
the" anded.
There were a ot of 8
'robems on the fight.
+uch item t"'es ha-e been used in the *ni-ersit" of
E)aminations +"ndicate (ower Certi5cate in English
but arednow noaonger used
in the
6irst Certi5cate in English
e)amination.
Eisabeth (ngram re'orts that nati-e s'eakers of ,merican Engish made u' to (t: 'er cent
errors in the ELB, 'honeme sub8test as com'ared with an a-erage of < 'er cent errors made
b" nati-e British s'eakers /(anguage &esting
'#mposium,
4)ford *ni-ersit" 5ress 1/?0:.
3 This item t"'e was de-ised b" Eisabeth (ngram for use in the
E(BA &est
9Engish Language Batter":.
= #obert Lado made use of this techniBue in his
&est o! Aural Comprehension.
2
, -ersion of this test item used to be incuded in the
Braded Achievement &ests
in English
9&,TE: oW thWH.,merican Language (nstitute. &eorgetown *ni-ersit".
? The test item was written b" 6a-id Bonam" and John Be-ere".
7 Joint Aatricuation Board.
&est in English /<verseas1, 4ra 5a'er. June 1/03. 0 ;or
further detais. see #ichards. J C 1/02 &he Conte4t o! (anguage &eaching.
Cambridge *ni-ersit" 5ress
/ The #o"a +ociet" of ,rts!
E4aminations in the Communicative 9se o! English as
a 6oreign (anguage,
(ntermediate e-e. Test of Listening. Aa" 1/03.
07
rah 'roduction tests
7.1 +ome difficuties in
Testing the abiit" to s'eak is a most im'ortant as'ect of anguage testing.
testing the s'eaking Howe-er. at a stages be"ond the eementar" e-es of mimicr" and
skis
re'etition it is an e)treme" difficut ski to test. as it is far too>com'e) a
ski to 'ermit an" reiabe ana"sis to be made for the 'ur'ose of obJecti-e
testing. Suestions reating to the criteria for measuring the s'eaking skis
and to the weighting gi-en to such com'onents as correct 'ronunciation
remain arge" unanswered. it is 'ossibe for 'eo'e to 'roduce 'ractica"
a the correct sounds but sti be unabe to communicate t it ideas
a''ro'riate" and effecti-e". 4n the other hand. 'eo'e don make
numerous errors in both 'honoog" and s"nta) and "et succeed in
e)'ressing themse-es fair" cear". ;urthermore. success in
communication often de'ends as much on the istener as on the s'eaker! a
'articuar istener ma" ha-e a better abiit" to decode the foreign s'eaker>s
message or ma" share a common ne)us of ideas with him or her. thereb"
making communication sim'er. Two nati-e s'eakers wi not awa"s.
therefore. e)'erience the same degree of difficut" in understanding the
foreign s'eaker.
(n man" tests of ora 'roduction it is neither 'ossibe nor desirabe to
se'arate the s'eaking skis from the istening skis. Cear". in norma
s'eech situations the two skis are interde'endent. (t is im'ossibe to hod
an" meaningfu con-ersation without understanding what is being said and
without making onesef understood at the same time. Howe-er. this -er"
interde'endence of the s'eaking and istening skis increases the difficut"
of an - serious attem't to ana"se 'recise" what is being tested at an" one
time. Aoreo-er. since the s'oken anguage is transient. it is im'ossibe
without a ta'e recorder to a''" such 'rocedures as in the marking of
com'ositions. where e)aminers are abe to check hack and make an
assessment at eisure. The e)aminer of an ora 'roduction test is working
under great 'ressure a the time. making subJecti-e Judgements as Buick"
as 'ossibe. E-en though sam'es of s'eech can he recorded during a test.
the ta'e8recording. b" itsef. is inadeBuate to 'ro-ide an accurate means of
reassessing or checking a score. since it cannot reca'ture the fu conte)t of
the actua situation. a of which is so essentia to an" assessment of the
communication that takes 'ace.
%et another 9though not insu'erabe: difficut" in ora testing is that of
administration. (t is freBuent" im'ossibe to test arge numbers of students
because of the imited time in-o-ed. (t is not difficut to a''reciate the
huge 'robems reating to a test situation in which thousands of students
ha-e. to be e)amined b" a handfu of e)aminers. each student being tested for
a 'eriod of. Vsa". ten or fifteen minutes. ,though the use of anguage
aboratories for such tests has made it 'ossibe in some cases to administer
more reiabe ora 'roduction tests to arge numbers of students. the actua
scoring of the tests has not been so easi" so-ed.
E)cuding tests of reading aoud and one or two other simiar tests.
ora tests can ha-e an e)ceent backwash effect on the teaching that takes
'ace 'rior to the tests. ;or e)am'e. in one countr" the ora test was
retained as 'art of a schoo8ea-ing e)amination sim'" to ensure that at
east some Engish woud he s'oken in the ast two "ears of the secondar"
schoo 8 e-en though the test itsef was considered an unreiabe measuring
instrument as a resut of the arge number of unBuaified e)aminers who
had to administer it. ;or this reason. and indeed. because ora
communication is genera" rated so high" in anguage earning. the testing
of ora 'roduction usua" forms an im'ortant 'art of man" anguage
testing 'rogrammes.
The foowing sections in this cha'ter wi gi-e an idea of the range of
'ossibe t"'es of ora tests. +ome of the e)ercises 9e.g. 'icture descri'tions:
ha-e 'ro-ed -er" usefu in man" tests whie others 9e.g. 'enci8and8'a'er
tests: ha-e met with -ar"ing degrees of success. (n s'ite of its high
subJecti-it". an e)treme" good test is the ora inter-iew. (n man" cases.
one or two sub8tests 9or ora acti-ities: are used together with the ora
inter-iew to form a com'rehen.i-e test of ora 'roduction skis.
7.< #eading aoud
Aan" 'resent8da" ora tests incude a test of reading aoud in which the
student is gi-en a short time to gance through an e)tract before being
reBuired to read it aoud. The abiit" to read aoud differs great" from the
abiit" to con-erse with another 'erson in a fe)ibe. informa wa".
,though reading aoud ma" ha-e a certain usefuness. on" a few
newsreaders and teachers ma" e-er reBuire training and testing in this
'articuar ski. The maJorit" of students wi ne-er he caed on to read
aoud when the" ha-e eft schoo. (t is a 'it". therefore. that students are
reBuired to sacrifice their enJo"ment of sient reading in order to 'ractise
reading aoud. 7e read 'rimari" for information or enJo"ment. and the
sient reading skis so necessar" for this 'ur'ose differ great" from those
of reading aoud. The backwash effects of this kind of test ma" he -er"
harmfu. es'ecia" in areas where the reading skis are misguided"
'ractised through reading aoud. ;ina". how man" nati-e s'eakers can
read aoud without making an" errorsG
Tests in-o-ing reading aoud are genera" used when it is desired to
assess 'ronunciation as distinct from the tota s'eaking skis. (n order to
construct suitabe tests of reading aoud. it is he'fu to imagine actua
situations in rea ife in which the testees ma" be reBuired to read aoud.
5erha's one of the most common tasks is that of reading aoud directions or
instructions to a friend. coeague or feow8worker! e.g. how to wire a 'ug.
how to trace fauts in a car engine. how to.cook certain dishes. ;or e)am'e.
the foowing instructions reate to a situation in which a teacher or cass
monitor ma" be asked to read aoud!
;irst 'ut the headset on. Aake sure it is in its most comfortabe 'osition
with the headband o-er the centre of the head. The micro'hone shoud be
about it inches from the mouth.
0/
((
To record. 'ut the white switch to the 'osition marked
)or-. 5ut the red
switch to
'pea- and 'ress the red recording button. which wi now ight
u'.
9etc.:
,nother situation which might occur in rea ife is that in which the
student is asked to read aoud 9'art of: a etter he has recei-ed. ;or a the
different e)tracts. howe-er. it is ad-isabe to draw u' certain features
which must be incuded in each 'assage! e.g. one %esKNo Buestion. one 7h8
Buestion. two sentences each containing a subordinate cause. one Buestion
tag. the 'honeme contrasts is8t. '8b. o8Q:!. etc. (n this wa". some degree of
consistenc" can be achie-ed.
, test more usefu in man" wa"s than reading aoud is the reteing of
a short stor" or incident. (n this t"'e of e)amination. the students are
reBuired to rete8a8ster8"8the" ha-e.Justsead. (f carefu" constructed. such8
a test can assess most of the 'honoogica eements which are otherwise
tested b" reading aoud. *nfortunate". it often measures other skis such
as reading com'rehension. memor" and organisation. too.
7.3 Con-ersationa
These dris are es'ecia" suitabe for the anguage aborator" and can ser-e
e)changes to focus attention on certain as'ects of the s'oken anguage. es'ecia" in
those countries where Engish is taught as a foreign anguage and the
8
em'hasis is 'rimari" on the reading skis. Howe-er. se-era of the test
items themse-es are far from communicati-e in an" sense at a and do not
aow for authentic interaction of an" kind. The essentia.eement of
constructi-e inter'a" with un'redictabe stimui and re'onses is ahsent
from a these items as a resut of the attem't to contro the interaction
taking 'ace. The item t"'es range from items 'resenting the testees with
situations in which the" initiate con-ersations to incom'ete comersations
with the 'art of one s'eaker omitted 9i.e.a one8sided diaoDVue:. Tests
containing such item t"'es are on the whoe reiabe. but the" cannot h.O
described as being -aid tests of s'eaking. (f an o''curtunit. is 'ro-ided in
other 'arts of the test for rea ora interaction 9i.e. genuine con-ersation
and discussion:. howe-er. these controed test items can he of some use in
directing the attention of the students to s'ecific anguage areas and skits.
T"'e 1
The testees are gi-en a series of situations and are reBuired to
construct sentences on the ines of a certain 'attern or grou' of 'atterns.
,gain. it is essentia that two or three modes be gi-en to the testees sro
that the" know e)act" what is reBuired. 9The testees read or hear the
situation and then make the a''ro'riate res'onses. shown in the hraekets.(
E)am'es!
Ars &reen i-es in a fat. +he doesn>t ike i-ing in a fat and woud ike to
i-e in a sma house with a garden. 9+he wishes she
lived in a small
house with a arden.1
(t>s raining hea-i". Tom and ,nna are waiting im'atient" at home to set off
on their 'icnic. /&he# wish it would
sto' raining.1
1. Ar Back has a sma car but his neighbours a ha-e arge cars. He
woud ike a arBe car. too.
<. ,nna hasn>t earnt how to swim "et but most of her friends can swim.
3. Tom is waiting for Bi outside the cinema. The show is Just about to
start but Bi has not arri-ed "et.
/3
=. Ars #obinson doesn>t. ike i-ing in towns@ she wants to i-e in the
countr".
9etc.:
T"'e < This t"'e of test item is simiar to the 're-ious t"'e but not as
strict" controed.\ No mode8res'onses are gi-en b" the e)aminer and the
students are free to use whate-er 'atterns the" wish.
, friend of "ours has forgotten where he has 'ut his gasses. He cannot see
too we without them. 7hat wi "ou sa" to himG /(et me help #ou to
loo- !or them, etc.:
%ou are on "our wa" to schoo when it starts to rain hea-i".
*nfortunate". "ou and "our friend ha-e no raincoats. There is nowhere
to sheter but "our schoo is on" a hundred "ards awa". 7hat do "ou
sa" to "our friendG /'hall we ma-e a dash !or it@2(et>s run the rest
of the
wa#.1
1. %ou are tr"ing to get to the 'ubic ibrar" but "ou are ost. ,sk a
'oice officer the wa".
<. %our friend has Just returned from a hoida" abroad. 7hat do "ou sa" to
himG
3. , waitress has Just brought "ou the bi but has totaed it u'
incorrect". 7hat do "ou sa" to herG
=. , friend of "ours wants to see a fim about a murder. %ou ha-e
aread" arranged to see it another e-ening. but "ou know she woud be
hurt if she knew. Aake u' an e)cuse.
T"'e 3 The students hear a stimuus to which the" must res'ond in an"
a''ro'riate wa". 9This test often reies on con-entiona greetings.
a'oogies. acce'tabe wa"s of e)'ressing 'oite disagreement. etc.:
6o "ou mind if ( use "our 'enci for a momentG
/;ot at all2Certainl#2%lease do2Bo ahead, etc.:
7hat about a game of tennisG
/?es, I>d love a game2All right. I don>t mind2Gon>t #ou thin-
it>s a "it too
hot@, etc.:
1. 5ease don>t go to a ot of troube on m" behaf.
<. 4h dear. it>s raining again. ( ho'e it sto's soon.
3. 7e shan>t be ate. sha weG
=. >$aren asked m. e to sa" she>s sorr" she can>t come tonight.
T"'e = This is simiar to the 're-ious t"'e of item. but the stimui and
res'onses form 'art of a onger diaogue and the situation is thus
de-eo'ed. Because8e. its tota 'redictabiit". howe-er. this t"'e of item is
sometimes referred to as a diaogue of the deafR The man in the diaogue
beow continues regardess of what the testee8sa"s.
%ou>re on "our wa" to the su'ermarket. , man comes u' and s'eaks to
"ou.
A,N! E)cuse me. ( wonder if "ou can he' me at a. (>m ooking for a
chemist>s.
%A9'E 6<7 &E'&EE>' 7E%(?
A,N! Thank "ou. 6o "ou know what time it o'ensG
%A9'E 6<7. &E'&EE>' 7E%(?
A,N! Thanks a ot. 4h. er. b" the wa". is there a 'hone bo) near hereG
/1
%A9'E 6<7 &E'&EE>' 7E%(?
A,N! 4h dear. (> need some coins. 6o "ou ha-e an" change for a ]2
noteG
%A9'E 6<7 &E'&EE>' 7E%(?
A,N! 7e. thanks a ot. %ou>-e been most he'fu.
This diaogue cear" becomes absurd if. when asked where there is a
chemist>s. the testee re'ies. D(>m sorr". ( don>t know.> and the man
'rom't" thanks him and asks what time it o'ens. Ne-ertheess. the use of
're8recorded materia of this kind makes it 'ossibe to use the anguage
aborator" to test arge numbers of students in a -er" short time.
T"'e 2 This item>8 takes the form of an incom'ete diaogue with 'rom'ts
9shown in brackets in the foowing e)am'e: whis'ered in the students
%ou are at the rece'tion desk of a arge hote. The rece'tionist turns to
address "ou!
#ECE5T(4N(+T! Can ( he' "ouG
9%ou want to know if there is a singe8room a-aiabe.:
%ou!
#ECE5T(4N(+T! %es. we ha-e a singe room with an attached bathroom.
9,sk the 'rice.:
%ou!
#ECE5T(4N(+T! Thirt"8four 'ounds fift" a night.
9%ou want to know if this incudes breakfast.:
%ou!
#ECE5T(4N(+T! %es. that>s with continenta breakfast.
9%ou ha-e no idea what >continenta breakfast> is.:
%ou.
#EcE5TiorasT! it>s fruit Juice. coffee or tea and bread ros.
9The rece'tionist is s'eaking too Buick". 7hat do "ou sa"G:
%ou!
#ECE5T(4N(+T! ;ruit Juice. coffee or tea. and bread ros.
9Book the room for two nights.:
%ou.
#ECE5T(4N(+T! Certain". #oom <1?. The 'orter wi take "our bag and show "ou
where it is.
9Thank the rece'tionist.:
%ou!
7.= *sing 'ictures for 5ictures. ma's and diagrams can he used in ora 'roduction tests in simiar
assessing ora wa"s to those described in the 're-ious cha'ter on testing the istening
'roduction skis. 5ictures of singe obJects can be used for testing the 'roduction of
significant 'honeme contrasts. whie a 'icture of a scene or an incident can
be used for e)amining the tota ora skis. This section wi concentrate on
the use of 'ictures for descri'tion and narration.
The students are gi-en a 'icture to stud" for a few minutes! the" are
then reBuired to describe the 'icture in a gi-en time 9e.g. two or three
minutes:8. 4ccasiona". the number of words each student s'eaks is counted
b" one e)aminer in the room. whie the other e)aminer counts the number
of errors made. The score is thus obtained on the basis of the number of
words s'oken and the errors made 9hut this 'rocedure is -er"
unreiabe.: +e'arate scores for genera fuenc". grammar. -ocabuar".
L:
1
'honoog". and accurac" of descri'tionKnarration are far better.
,d-ertisements. 'osters and stri' cartoons ma" be used in this wa" for
cass tests. 'ro-ided that there are enough a-aiabe to 're-ent the students
from 're'aring one or two set 'ieces.
Carefu seection of the 'ictures used for the e)amination wi he' in
controing the basic -ocabuar" reBuired and ma". to some e)tent.
determine the t"'e of sentence structure that 'redominates. 6ifferent
st"es and registers can be tested b" incuding ma's and diagrams as we as
'ictures for com'arison. 'ictures for instructions and 'ictures for
descri'tion and narration. (f the 'ictures de'ict a stor" or seBuence of
e-ents. it is usefu to gi-e the testees one or two sentences as a >starter>.
thereb" famiiarising them with the tense seBuencing the" shoud em'o".
E)aminer! Last summer Luc" s'ent a few da"s with her unce and aunt in
the countr". 7hen it was time for her to return home. her unce and aunt
took her to the station. Luc" had made a ot of friends and she fet sad on
ea-ing them. +he got on the train and wa-ed goodb"e to
them... Now "ou continue to te this stor".
/3
((
((
The most effecti-e t"'e of ora e)amination using 'ictures reBuires not
on" narration or 'icture descri'tion on the 'art of the students but aso a
discussion about the 'icture9s: concerned. (f the e)aminer asks
Buestions and discusses the 'icture9s: with each student. the forma s'eech
situation is combined with the reci'roca s'eech situation and two different
t"'es of ora 'roduction skis can thus be measured. E-en if no discussion
is incuded in the e)amination. the e)aminer woud be we ad-ised to
'rom't the student whene-er he or she a''ears to need encouragement. (t
is awa"s im'ortant to find out what a student knows 8 not what he or she
doesn>t know! ong 'eriods of sience wi te the e)aminer -er" itte.
, simiar techniBue3 to that described in the 're-ious cha'ter can be
used to test ora 'roduction. The student and the e)aminer ha-e fi-e
'ictures in front of them. each 'icture differing in on" one res'ect from
the other four 'ictures. The student is gi-en a card bearing a etter 9,. B.
C. 6 or E:@ the e)aminer cannot see the etter. The student is reBuired to
describe the a''ro'riate 'icture 9according to the etter:. The e)aminer
then seects a 'icture according to the descri'tion. assessing the.student not
on" on the correctness and fuenc" of his or her s'eech but aso on the
ength of time taken before the student>s descri'tion resuts in the
identification of the a''ro'riate 'icture. The e)aminer then checks the
card.
A
/=
,nother effecti-e wa" of assessing a student>s abiit" to. s'eak.
howe-er. is to gi-e 'airs or grou's of students a sim'e task to 'erform.
7orking in 'airs. students can describe their own 'icture before istening
to their 'artner>s descri'tion of a simiar 9but not identica: 'icture. The
two students cannot see each other>s 'icture. The" can then discuss in
which wa"s the. two 'ictures are the same and in which wa"s the" differ.
The 'ictures used for such discussion ma" range from amost identica ones
to Buite different ones. The most successfu 'ictures for this acti-it".
howe-er. wi genera" de'ict fair" simiar subJects but differ considerab"
in their treatment of the subJect. The foowing two 'ictures of a harbour
ha-e been used -er" successfu" to generate discussion between two
grou's of students.
,
B
7hen scoring students> 'erformances. the e)aminer shoud
concentrate on what indi-idua students are doing with the target anguage
and how the" are using it to achie-e their 'ur'ose. Language errors which
interfere with successfu communication wi thus be 'enaised hea-i". 4n
the other hand. those minor errors which. though anno"ing in certain
res'ects. do not seem to im'ede communication to an" degree wi not be
'enaised in the same wa". Cear". the successfu outcome of the acti-it"
wi thus be im'ortant for both e)aminer and students.
(f students are e)amined in sma8grou's. one of the most usefu
acti-ities in-o-ing 'ictures is for them to be gi-en a seBuence of 'ictures
to rearrange. +tudents shoud begin b" describing their own 'icture
without showing it to the other members of the grou'. ,fter each 'icture
has been described and discussed in reation to the other 'ictures. the
grou' decides on an a''ro'riate seBuence. Each member of the grou'
then 'uts down his or her 'icture in the order decided u'on. Comments
are made and the order changed if the grou' considers it desirabe. ,gain.
students are using anguage to achie-e a certain 'ur'ose in this t"'e of test.
The foowing e)am'e> shows how sim'e it is to base such an acti-it"
on 'ictures aread" 'ubished. es'ecia" those foowing the comic stri'
'rinci'e.
/2
Two mues
7.2 The ora inter-iew Like man" other e)aminations of ora 'roduction. the scoring of the ora
inter-iew is high" subJecti-e and thus. sometimes has on" ow reiahiit".(n
addition. the 'erformance of a student in a 'articuar inter-iew ma" not
accurate" refect his or her true abiit".
/?
+u''orters of the ora inter-iew caim that the e)amination at east
a''ears to offer a reaistic means of assessing the tota ora ski in a
Dnatura> s'eech situation. 4thers. howe-er. argue that the e)amination
ne-ertheess is artificia and unreaistic! students are 'aced not in natura.
rea8ife s'eech situations but in e)amination situations. The" are thus
susce'tibe to 's"choogica tensions and aso to constraints of st"e and
register necessar" in such a situation. ;or e)am'e. man" students ado't a
Buiet and coouress tone in inter-iews@ some e-en de-eo' a guarded
attitude. whie others become o-er8friend".
4ne soution to this 'robem is to ha-e the cass teacher as the
inter-iewer! if an e)terna e)aminer is reBuired. he or she ma" sit at the
back of the room or in an" other obscure 'ace. The inter-iewer 9whether
teacher or e)aminer: shoud endea-our to 'ut the student at ease at the
beginning of the inter-iew. ado'ting a s"m'athetic attitude and tr"ing to
hod a genuine con-ersation 9i!onstant" making his or her own contribution
without. at the same time. taking too much:. The inter-iewer shoud
never
attem't to note down marks or comments whie the student. is sti engaged
in the inter-iew. The dua roe 9i.e. of both anguage 'artner and assessor:
which the e)aminer is reBuired to assume in the ora inter-iew is awa"s a
most difficut one.
,nother soution to 'robems caused b" tension and anguage
constraints is to inter-iew students in 'airs or e-en threes. thus not on"
'utting them more at ease through the 'resence of a friend or cassmate
but aso enabing them to s'eak to each other as members of the same 'eer
grou'. ConseBuent". the whoe atmos'here wi become more rea)ed and
the constraints of register wi disa''ear. resuting in ess artificia and
stited anguage being used. +tudents wi use the anguage which the"
norma" use in most s'eech situations in e-er"da" ife. No onger wi an
inferior 9i.e. the student: be reBuired to address a su'erior 9i.e. the
teacher: throughout the entire inter-iew. ,though the teacher. or
e)aminer. ma" interru't or direct the discussion whene-er necessar". he or
she wi aso be abe to ado't a more 'assi-e roe in the discussion. (n this
wa". students wi fee free to con-erse and use anguage in a more natura
and 'ur'osefu wa".
The chief danger in conducting inter-iews with 'airs of students is that
resuting from 'ersonait" conficts or the dominance of one of the
members of the 'air. (t is therefore -er" im'ortant for the teacher or
e)aminer to ensure that the two students forming a 'air ha-e simiar or
s"m'athetic 'ersonaities and ha-e simiar e-es of anguage abiit". The
anguage task itsef can aso he' since. if the two students are 'resented
with a rea 'robem to so-e. there wi be a greater chance of them co8
o'erating and working together in the target anguage. (n addition to the
use of 'ictures for com'arison and contrast. students can be gi-en sim'e
'uLLes and 'robem8so-ing tasks. , short BuiL ma" e-en be 're'ared b"
one of the students and gi-en to the other. (n a these cases. howe-er. the
e)aminer shoud be concerned more with the students> use of the target
anguage to achie-e their goas rather than with their knowedge or their
actua abiit" to accom'ish the task gi-en. Thus. a student>s inabiit" to
answer se-era Buestions in a BuiL shoud not weigh too hea-i" 'ro-ided
that he or she is abe to gi-e some answers 9right or wrong: and conduct a
reasonabe con-ersation. etc. with the student gi-ing the BuiL. (n short.
each student shoud be assessed not on" on such features of the s'oken
anguage as grammatica acce'tabiit" and 'ronunciation. but aso on
/7
a''ro'riac" of anguage and effecti-eness of communication 8 and. where
a''ro'riate. the time taken to accom'ish the task gi-en.
The ora inter-iew shoud be scored on" after the student has eft the
room 9uness two or more e)aminers are 'resent 8 in which case one of
them can sit behind the student and score:. ,though.this settes the
'robem of when the inter-iew shoud be scored. the Buestion of how it
shoud be scored sti remains. ;or e)am'e. how shoud the re'ies to this
Buestion be scoredG
9Tester: >7hat are "ou going to do this weekendG>
9+tudent ,: >(>m Buite we. thank "ou.>
9+tudent B: >( go to fish. ( fish in ri-er near the big wood.
,>s re'" is 'erfect" correct but it is ne-ertheess Buite ina''ro'riate! the
student sim'" hasn>t heard..n !.understood the Buestion correct". 4n the
other hand. B>s re'" shows a rea attem't to answer the Buestion but
unfortunate" contains se-era errors.
The scoring of the inter-iew can range from an im'ression mark to a
mark arri-ed at on the basis of a fair" detaied marking scheme 9showing
accurac" of 'ronunciation. grammar. -ocabuar" a''ro'riac". fuenc" and
ease of s'eech:. The foowing marking schemes 9using a ?8'oint scae: is
gi-en as Just one e)am'e of a number of such schemes in 'resent8da" use.
#ating ,biit" to communicate ora"
? E)ceent! on a 'ar with an educated nati-e s'eaker. Com'ete" at
ease in his use of Engish on a to'ics discussed.
2 Eer" good! athough he cannot be mistaken for a nati-e s'eaker. he
e)'resses himsef Buite cear". He e)'eriences itte difficut" in
understanding Engish. and there is no strain at a in
communicating with him.
= +atisfactor" -erba communication causing itte difficut" for nati-e
s'eakers. He makes a imited number of errors of grammar. e)is
and 'ronunciation but he is sti at ease in communicating on
e-er"da" subJects. He ma" ha-e to correct himsef and re'attern his
utterance on occasions. but there is itte difficut" in understanding
him.
3 ,though -erba communication is usua" fair" satisfactor". the
nati-e s'eaker ma" occasiona" e)'erience some difficut" in
communicating with him. #e'etition. re8'hrasing and re8'atterning
are sometimes necessar"@ ordinar" nati-e s'eakers might find it
difficut to communicate.
<
Auch difficut" e)'erienced b" nati-e s'eakers unaccustomed to
>foreign> Engish. His own understanding is se-ere" imited. but
communication on e-erda" to'ics is 'ossibe. Large number of
.errors of 'honoog". grammar and e)is.
1
E)treme difficut" in communication on an" subJect. ;aiure to
understand adeBuate" and to make himsef understood.
Note that an e-en numbered scae is often 'referred because it he's
e)aminers to a-oid awarding the midde mark 9a tendenc" in man" cases:.
Thus. athough marks ma" custer round the median 3 on a 28'oint scae.
e)aminers using a ?8'oint scae wi ha-e to decide whether to award 3 9Just
"elow the midde 'oint on the scae: or = 9Just
a"ove the midde 'oint on
the same scae:. (t is aso ad-isabe to a-oid a narrow scae 9i.e. a =8'oint
scae: as this wi not aow for the range of discriminations "ou ma" wish
to make. 4n the other hand. a wide scae 9e.g. a <38'oint scae: is not
recommended as few markers seem to make use of either the u''er or ower
ends of this scae. most scores tending to custer around /81<.
Aan" e)amining bodies 'refer fair" short descri'tions of grades in
order to enabe the e)aminer to gance Buick" through the marking
scheme. E)aminers who are faced with a ot of reading in the assessment
wi be tem'ted to re" soe" on the numerica grade itsef 9e.g. 2: 8 which
makes their scoring e)treme" subJecti-e and iabe to fuctuation.
7here-er 'ossibe. it is usefu in 'ubic e)aminations to ha-e two or more
e)aminers istening for 'articuar areas or features before ater 'ooing
their assessments. The im'ortance of e)aminers> meetings and D'ractice>
inter-iews 9using ta'e recorders and marking guides: cannot be too great"
em'hasised. +uch sessions. when conducted with a arge number of
e)aminers. are of considerabe he' in increasing the marker reiabiit" of
ora inter-iews. +am'e recordings and scores are discussed and some
degree of standardisation of marking is thus achie-ed.
The 're-ious 'Lragra'h arge" concerns 'ubic e)aminations or
achie-ement tests set outside the cassroom. ;or most cassroom and
schoo tests. howe-er. the teacher shoud de-ise his or her own rating scae
after ha-ing carefu" considered the e-e and kind of ora skis the
students shoud be e)'ected to achie-e. The o'timum 'erformance
e)'ected. therefore. wi not necessari" be near nati-e8s'eaker fuenc".
Cear". students who ha-e been s'ending a few hours a week earning
Engish o-er a 'eriod of two "ears cannot 'ossib" be e)'ected to achie-e
band ? on the scae incuded here! hence the scae itsef woud be arge"
ina''ro'riate for the 'ur'oses of most cassroom tests of ora Engish.
(n order to de-ise a suitabe scae. the teacher shoud first begin to
describe cear" the criteria for assessing ora abiit". The teacher ma". for
e)am'e. wish to consider each student>s achie-ement in terms of accurac".
a''ro'riac" and fuenc"@? accurac". a''ro'riac". range. fe)ibiit" and
siLe@7 or fuenc". com'rehensibiit". amount of communication. Buait" of
communication. and effort to communicate.> 7hate-er the criteria
seected. the teacher shoud begin b" describing in one or two sentences
e)act" what he or she e)'ects the a-erage successfu student to ha-e
8achie-ed under each8of the headings b" the time the test is taken. These
descri'tions wi then form band = 9if a ?8'oint scae is being used:. The
teacher shoud then re'eat the same 'rocedure for the student who is
sight" beow a-erage in his or her e)'ectations 9i.e. band 3:. Ne)t. the
teacher writes the descri'tions for the successfu student 9abo-e a-erage:.
assigning these descri'tions to band 2 on the scae. ,fter this. the teacher
describes the 'erformance of the unsuccessfu student who is beow
a-erage 9band <:. then the 'erformance of the -er" successfu student
9band ?:. and fina" that of the east abe and most unsuccessfu student
9band 1:.
The foowing is an e)am'e of a teacher>s rating scae for the ower
in
termediate e-e.
//
i
,
? 5ronunciation is on" -er"
sight" infuenced b" the
mother8tongue. Two or three
minor grammatica and e)ica
errors.
2 5ronunciatior is sight"
infuenced b" the mother8
tongue. , few minor
grammatica and e)ica errors
but most utterances are
correct.
= 5ronunciation is sti
moderate" infuenced b" the
mother8tongue but no serious
'honoogica errors. , few
grammatica and e)ica errors
but on" one or two maJor
errors causing confusion.
3 5ronunciation is infuenced b"
the mother8tongue but on" a
few serious 'honoogica
errors. +e-era grammatica
and e)ica errors. some of
which cause confusion.
< 5ronunciation serious"
infuenced b" the mother8
tongue with errors causing a
breakdown in communication.
Aan">basic> grammatica and
e)ica errors.
1 +erious 'ronunciation errors
as we as man" >basic>
grammatica and e)ica errors.
No e-idence of ha-ing
mastered an" of the anguage
skis and areas 'ractised in
the course.
;uenc"
+'eaks without too great an
effort with a fair" wide range of
e)'ression. +earches for words
occasiona" but on" one or two
unnatura 'auses.
Has to make an effort at times to
search for words. Ne-ertheess.
smooth dei-er" on the whoe
and on" a few unnatura 'auses.
,though he has to make an
effort and search for words.
there are not too man" unnatura
'auses. ;air" smooth dei-er"
most". 4ccasiona"
fragmentar" but succeeds in
con-e"ing the genera meaning.
;air range of e)'ression.
Has to make an effort for much
of the time. 4ften has to search
for the desired meaning. #ather
hating dei-er" and fragmentar"
#ange of e)'ression often
( nited.
Long 'auses whie he searches
for the desired meaning.
;reBuent" fragmentar" and
hating dei-er". ,most gi-es u'
making the effort at times.
Limited range of e)'ression.
;u of ong and unnatura
'auses. Eer" hating and
fragmentar" dei-er". ,t times
gi-es u' making the effort. Eer"
imited range of e)'ression.
1
Com'rehensibiit"
Eas" for the istener to
understand the s'eaker>s
intention and genera meaning.
Eer" few interru'tions or
carifications reBuired.
The s'eaker>s intention and
genera meaning are fair" cear. ,
few interru'tions b" the
istener for the sake of
carification are necessar".
Aost of what the s'eaker sa"s is
eas" to foow. His intention is
awa"s cear but se-era
interru'tions are necessar" to
he' him to con-e" the message
or to seek carification.
The istener can understand a ot
of what is said. but he must
constant" seek carification.
Cannot understand man" of the
s'eaker>s more com'e) or
onger sentences.
4n" sma bits 9usua" short
sentences and 'hrases: can be
understood 8 and then with
considerabe effort b" someone
who is used to istening to the
s'eaker.
Hard" an"thing of what is said
can be understood. E-en when
the istener makes a great effort
or interru'ts. the s'eaker is
unabe to carif" an"thing he
seems to ha-e said.
7hate-er the criteria chosen. the brief descri'tions can be made much
more s'ecific at each e-e in order to refect the contents of the course
being foowed. The most im'ortant 'oint to bear in mind. howe-er. is that
for most cassroom 'ur'oses the rating scae shoud not ha-e nati-e8
s'eaker 'erformance as the desired coa. (nstead. it shoud be based on
reaistic e)'ectations of what successfu earners can achie-e at a 'articuar
stare in their de-eo'ment.
;ina". ora inter-iews do not sim'" ha''en s'ontaneous".
,though each ora inter-iew shoud simuate as natura and reaistic a
s'eech situation as 'ossibe. it is essentia that a certain amount of materia
be 're'ared beforehand. The inter-iew ma" be carefu" structured or ma@ be
structured Buite oose"! in both cases. the e)aminer shoud ha-e 'ent" of
materia on which to fa back if necessar". He or she ma" ead in to the
inter-iew b" asking a %esKNo Buestion. foowed at some stage or other b"
certain 7h8Buestions and Buestion tags.
There are dangers. howe-er. in adhering to a -er" rigid structure or
'an. ;or e)am'e. a student ma" de-eo' Da certain to'ic and be
'roceeding ha''i" in one direction when the e)aminer interru'ts and
sto's the whoe fow of the inter-iew in order to incude
a How o!ten
Buestion. ,gain. the demand for certain set res'onses ma" reach an absurc
stage! in a certain ora e)amination. for e)am'e. it was agreed that
)herea"outs . . G ought to signa a different res'onse to
)here .
ConseBuent". the more he'fu students. who antici'ated the in
ter-iewer.
Dfaied> on this item!
e.g.
(NTE#E(E7E#! 7hereabouts do "ou i-eG
9#eBuiring the name of the
district where the testee i-ed .
+T*6ENT! 3=1 $ing>s #oad. North 5oint.
Had the student re'ied
;orth %oint in a natura s'eech situation. the othc 'erson
woud 'robab" ha-e asked! )here/a"outs1 in ;orth %oint@ (n the inter-iew the
student mere" antici'ated such a reaction 9conscious" or subconscious": and ga-e a fu
answer initia".
5ro-ided that fe)ibiit" can be retained. it is usefu to 're'are a serie of Buestions
on a wide range8 of. to'ics. The foowing ist of <3 to'ics is gi-en here to he'. not to
inhibit. the e)aminer!
fami". home. schoo. s'orts. hobbies. books. fims. trans'ort.
weekends. hoida"s. radio. heath. teeth. sho''ing. traffic. crime.
friends. mone". fines. careers. etc.
,t east ten or twe-e Bi estions can be asked on each to'ic. but the
e)aminer shoud ne-er attem't to >work through> ists of Buestions.
(ndeed. the e)aminer shoud contribute to the inter-iew from time to tim@O
E)am'es of the t"'es of Buestions which can be asked on one of the to'i.
are!
+'orts and games
6o "ou 'a" an" gamesG
7hat>s "our fa-ourite s'ortG
How often do "ou 'a"G
,re "ou in "our schoo teamG
6o "ouike watching s'ortG
7hich do "ou think is the most interesting s'ort to watchG
Can "ou swimG
How did "ou earn to swimG
7hereabouts do "ou go swimmingG
7hich game woud "ou ad-ise me to take u'G 7h"G
7hich is the most difficut game to 'a"G
7hich is better as an e)ercise! basketba or footbaG
7here considered necessar". current affairs and high" contro-ersia
issues ma" be introduced in an inter-iew to stimuate or 'ro-oke a stude!
'ro-ided that some aowance is made for the emoti-e content of the
discussion. (t is e-en more im'ortant here that the e)aminer shoud remain
fe)ibe and -ar" the range of Buestions for discussion.
(f the ora inter-iew is recorded on ta'e. the e)aminer can score the
inter-iew at eisure. 'a"ing and re'a"ing sections where necessar".
E)tensi-e notes can be taken from the recordings. com'arisons made and
confirmation sought where there is doubt concerning a 'articuar mark. (f
inter-iews are recorded. the e)aminer must of course take care to identif"
each student at the beginning of the inter-iew. es'ecia" when
com'arisons in-o-ing rewinding and re'a"ing are to be made.
7.? +ome other
techniBues for ora
e)amining
&he shor$t tal-H
Broup discussion
and role pla#ing
i S<
(n certain e)aminations students are reBuired to 're'are a short tak on a
gi-en to'ic. The" ma" be aowed se-era da"s or on" a few minutes in
which to 're'are the tak and. in some cases. the" ma" be 'ro-ided with
notes or reference materia. This is cear" a reaistic test of sustained
s'eech but it constitutes an e)treme" difficut e)amination for second8
anguage earners at a but the most ad-anced stages. (ndeed. this
'articuar t"'e of e)amination is genera" -er" difficut for first8anguage
s'eakers. The e)amination can be im'ro-ed sight" b" reducing the time
aotted for the tak and asking students Buestions based on their tak. thus
introducing a reci'roca s'eech situation. The Buestions might be asked either
b" the e)aminer or b" a grou' of students 9if the tak is gi-en in front of an
audience:. (n whate-er situation the tak is gi-en. howe-er. the e)aminer
must make e-er" attem't to 'ut the s udents at ease.
Care must be taken to 're-ent students from earning whoe sections of
their tak! +ubJects about which an indi-idua student knows -er" itte
shoud be a-oided. E)'erience of such e)aminations has shown that
candidates tak better when the" ha-e something worthwhie to sa" and can
bring into the tak a genuine interest in the subJect cou'ed with e)'erience
and imagination. , co8o'erati-e audience aso he's great".
Eague subJects are best a-oided@ man" to'ics are best 'resented as
Buestions!
+houd cJuntries s'end huge sums of mone" on s'ace e)'orationG
6o demonstrations ser-e an" usefu 'ur'oseG
6o 'eo'e e-er rea" earn an"thing from the mistakes the" makeG
&rou' discussion and roe 'a"ing are two other im'ortant techniBues for
assessing ora 'roduction. Through grou' discussion and roe 'a"ing the
teacher can disco-er how students are thinking and using the target
anguage. ;or e)am'e. are the" using the anguage the" are earning to
e)'ore conce'ts and ideasG 4r are the" sim'" using the foreign anguage
to 'resent ideas aread" we8formedG (n this wa". grou' acti-ities in both
teaching and testing can be used to 'ro-ide an o''ortunit" for meaningfu
and acti-e in-o-ement. +tudents are thus gi-en an o''ortunit" to use what
can be termed De)'orator" tak>!> i.e. the anguage 'eo'e use when tr"ing
to communicate rather than when the" are engaged in the mechanica
'roduction of -erba formuae or 'atterns. (n short. anguage becomes a
means to an end rather than an end in itsef. &rou' anguage acti-ities
then become ego8su''orting 9unike the ego8threatening e)'erience of too
man" students in traditiona tests:. as the other members of the grou' are
interested chief" in the message rather than in forma correctness.
+e-era usefu books ha-e been written on grou' work 9deaing with
the siLe and nature of -arious grou's: and on acti-ities for grou'
discussion. &enera" s'eaking. the t"'e of acti-it" most suitabe for grou'
work is that in which the e-e of difficut" in-o-ed makes the task a itte
too difficut for the indi-idua student to accom'ish aone. but not so
difficut as to discourage a grou' of students 9es'ecia" gi-en a the
constraints and tensions of a test:. (n addition to a the 'robem8so-ing
acti-ities and 'uLLes which abound in books and materias of -arious
kinds. the t"'e of task in-o-ing concensus8seeking is 'articuar" suitabe
for grou' discussion. (n this atter t"'e of acti-it". the members of the
grou' are gi-en a 'articuar situation and instructed to make -arious
decisions. (t is necessar" for them to use the target anguage to Justif" their
decisions and seek agreement from the other members of the grou'.
I#adioacti-it" from a nucear 'ower station accident wi reach "our area in
a few hours. There is a sma but -er" safe nucear faout sheter
nearb". but there is room for on" si) 'eo'e out of a tota of twe-e.
7hich si) 'eo'e from the foowing ist do "ou think it woud be most
usefu to sa-e in the interests of future generationsG List them in order of
'riorit". 9Note! A 1 mae@ ; 1 femae.:
8 a marine bioogist. aged 2? 9;:
8 a 'h"sicist. aged <2 9A:
8 a famous musician. aged 30 9;:
8 a farmer. aged 3< 9A:
8 an eectrician. aged =/ 9A:
8 a mathematics teacher. aged 3= 9;:
8 a we8known footbaer. aged << 9A:
8 a doctor. aged ?3 9;:
8 a uni-ersit" student of socioog". aged 1/ 9;:
8 a fireman. aged 33 9A: 8888888. 8 8
8 a factor" worker. aged <0 9;: 8
8 a garage mechanic. aged <7 9A:
#oe8'a" acti-ities can aso be used successfu" to test ora
communicati-e abiit". The students in-o-ed are assigned fictitious roes
and are reBuired to im'ro-ise in anguage and beha-iour. (t is ad-isabe for
the students to be gi-en fictitious names before the roe 'a". as these
usua" 'ro-e -er" he'fu in encouraging them to act out the roes assigned
to them. The roe 'a"s used for the test ma" -ar" from short sim'e roe
'a"s in-o-ing on" two or three students to far onger roe 'a"s in-o-ing
se-era students. The students needn>t be informed that the" are being
obser-ed and assessed in their use of Engish! in fact. it is usua"
im'ortant that the teacher or e)aminer does this as discreet" as 'ossibe.
The foowing is an e)am'e of an e)treme" sim'e roe 'a" suitabe
for use at the eementar" e-e.
4ne student acts the 'art of a 'oic
e officer. another a bus conductor. a
third a bus8dri-er. a fourth a 'assenger hurr"ing to -isit a sick friend in
hos'ita. and a fifth a b"stander who wants to be he'fu. The 'assenger
hurries to get on the bus and Jum's on as it is mo-ing off. The
conductor sto's the bus and tes him that the bus is fu and that he
must get off. The 'assenger can see an em't" seat and he begins to
argue. The bus is now in the midde of the road and is a danger to other
traffic. ,ct the roes gi-en.
(t is usua" ad-isabe for each >character> to be gi-en a card on which
there are a few sentences describing what kind of a 'erson he or sheds. etc
Aoreo-er. it is often he'fu if the e)aminer can take a minor roe in order
133
((
&enera concusions
Notes and references
w=
(
to be abe to contro or infuence the roe 'a" if necessar". (ndeed. e-en in
grou' discussions and acti-ities. the e)aminer shoud awa"s fee free to
interru't or contro the discussion in as di'omatic a wa" as 'ossibe in order to
ensure that each member of the grou' makes a contribution. Both grou'
discussions and roe 'a"s are best assessed if the" are incuded as 'art of the
anguage 'rogramme rather than as a forma test.
&enera" s'eaking. a reiabe method of obtaining measurements of ora
'roduction skis is that which in-o-es the students> cass teacher. The
tensions and artificiaities that inhibit the students in an ora e)amination
conducted b" an e)terna e)aminer wi now be a-oided since the teacher is a
famiiar figure and the cassroom a reaistic 'art of the students> ife.
Continuous assessment b" the teacher. with a his or her cassroom
e)'erience@i.8genera" 9but8not awa"s: a reiabe method of measuring the
ora skis. %et the ora inter-iew 9conducted b" a s"m'athetic
e)aminer: is sti a usefu e)amination to retain. 'articuar" for its
beneficia backwash effects on teaching. , com'rehensi-e and baanced
e)amination of ora 'roduction might thus consist of!
8 an ora inter-iew in-o-ing two students@
8 a short 'robem8so-ing acti-it" in-o-ing the com'arison or seBuencing
of 'ictures. etc.@
8 a onger acti-it" com'rising grou' discussion 9concensus8seeking acti-it":
or a roe 'a".
The first two com'onents isten here ma" be gi-en as 'art of a forma test
whie the atter com'onent is much better gi-en during the course itsef and
assessed b" the cass teacher in as informa a wa" as 'ossibe. The
e)aminer must freBuent" consider the effects of the e)amination on
teaching 8rid earning. howe-er. and if. for instance. the reading aoud
section is considered harmfu in its effect on teaching.
tier. it shoud be
omitted from the e)amination.
1 This t"'e of Buestion has been used effecti-e" in the A7E(' <ral E4amination
9,ssociation of #ecognised Engish Language +choos:.
< The e)am'e is based on an item t"'e in the *ni-ersit" of Cambridge Loca
E)aminations +"ndicate! %reliminar# English &est 9re-ised -ersion:.
3 This techniBue is deri-ed from a simiar one used b" ,. +. 5amer and described
in the artice DTesting Communication> in I7A( /International 7eview o!
Applied (inguistics in (anguage &eaching1, (ssue $K1. ;ebruar" 1/7<.
= #ichardson. # 9ed.: 1/7? (earning !or Change in )orld +ociet#. 4ne
7ord
Trust
2 Based on the marking scheme de-ised b" 6r ;rank Cha'en for use in assessing
the ora skis of o-erseas students at British uni-ersities. 9The scheme is. in
turn. deri-ed from the ,merican ;oreign +er-ice (nstitute rating scae.:
? Aorrow. $ 1/0< Testing +'oken Language. (n (anguage &esting 9ed. Heaton:.
Aodern Engish 5ubications
7 #o"a +ociet" of ,rts. Tests of 4ra (nteraction! 6egrees of +ki. &he
Communicative 9se o! English as a 6oreign (anguage.
0 BartL and +chutL. 1/7=
/ Barnes. 6 1/73 (anguage in the Classroom. 5enguin8
13 Based on an idea in #ichardson. # 9ed.: 1/7? (earning !or Change in )orld
'ociet#. 4ne 7ord Trust
Testing reading com'rehension
0.1 The nature of the *nti recent" the man" and di-erse reading skis and strategies for use in
reading skis e-er"da" situations ha-e been arge" subordinate to a narrower range of
skis reBuired for deaing with sim'ified readers. es'ecia" at the
eementar" e-es. ;urthermore. on a few anguage courses. efficient
reading skis ha-e been 'ushed into the background in an attem't to
de-eo' ora fuenc" skis. ,ttem'ts at deaing with the man" com'e)
reading skis freBuent" come too ate. at the tertiar" e-e 9i.e. at
uni-ersit". technica coege:. when students sudden" find themse-es
confronted with 'rofessiona and technica iterature in the foreign
anguage.
8
(n s'ite of the wide range of reading materia s'ecia" written or
ada'ted for Engish anguage earning 'ur'oses. there are few
com'rehensi-e s"stematic 'rogrammes which ha-e been. constructed from
a detaied ana"sis of the skis reBuired for efficient reading. Auch test
materia is sti imited to short reading e)tracts on which genera
Dcom'rehension> Buestions are based. ,s with istening com'rehension.
reading com'rehension test materia is -er" cose" reated to the t"'e of
'ractice materia used b" the teacher to de-eo' the reading skis. ;ew
anguage teachers woud argue against the im'ortance of reading! what is
sti urgent" reBuired in man" cassroom tests is a greater awareness of the
actua 'rocesses in-o-ed in reading and the 'roduction of a''ro'riate
e)ercise and test materias to assist in the master" of these 'rocesses.
Before reading tests in the second or foreign anguage can be
successfu" constructed. the first anguage reading skis of the students
must. be ascertained.OCear".there.is often itte 'ur'ose in testing in the
second anguage those basic reading skis which the students ha-e not "et
de-eo'ed in their own anguage. Howe-er. the mere fact that a student
has mastered some of the reBuired reading skis in the first anguage is no
guarantee at a that he or she wi be abe to transfer those skis to reading
another anguage.
,t this stage in our e)amination of reading difficuties. it woud be
he'fu to attem't to identif" some of the s'ecific skis in-o-ed in
reading.> Broad" s'eaking. these can be defined as the abiit" to!
8 recognise words and word grou's. associating sounds with their
corres'onding gra'hic s"mbos@
132
1
((
(
8 deduce the meaning of words b"
9a: understanding word formation 9roots. affi)ation. deri-ation and
com'ounding:@
9b: conte)tua cues 9e.g. <ne o! the mem"ers o! the group

e4posed the
plot, and the police were soon a"le to arrest the leaders.1
8 understand e)'icit" stated information 9e.g.
I wish Ann had come. H
Ann did not come $ hence m##
wish.:@
8 understand reations within the sentence. es'ecia"
9a: eements of sentence structure
9b: negation
9c: fronting and theme
9d: com'e) embedding@
8 understand reations between 'arts of a te)t through both e)ica de-ices
.9e.g..re'etition. s"non"ms..antithesis: and grammatica cohesi-e de-ices.
es'ecia" ana'horic and cata'horic reference 9e.g. he.
the#, it the
!ormer, the latter1
and connecti-es 9e.g.
since, a!ter, "ecause, although.
however, in addition1.
'ercei-e tem'ora and s'atia reationshi's. and aso seBuences of ideas@
8 understand conce'tua meaning. es'ecia"
9a: Buantit" and amount
9b: definiteness and indefiniteness
9c: com'arison and degree
9d: means and instrument
9e: cause. resut. 'ur'ose. reason. condition. addition. contrast.
concession@
8 antici'ate and 'redict what wi come ne)t in the te)t!
identif" the main idea and other saient features in a te)t@
generaise and draw concusions!
understand information not e)'icit" stated b"
@a: making inferences 9it8. reading between the ines:
9b: understanding figurati-e anguage!
8 skim and scan 9ooking for the genera meaning and reading for s'ecific
information:@
8 read critica"@
8 ado't a fe)ibe a''roach and -ar" reading strategies according to the
t"'e of materia being read and the 'ur'ose for which it is being read.
No mention has been made here of reading aoud. since this 'articuar
ski is uniBue in that it in-o-es different skis from sient reading.
Two different kinds of com'ementar" reading acti-ities to which
students are usua" e)'osed are genera" cassified as
intensive and
e4tensive reading. +hort reading e)tracts of a moderate degree of difficut"
and containing features which merit detaied stud" form a basis for
intensi-e reading 'ractice. 7hoe artices. cha'ters and books 9usua"
sim'ified readers: are used for e)tensi-e reading 'ractice@ in this case.
howe-er. the materia seected is genera" sight" beow the anguage
attainment e-e of the students using it. *nfortunate". most reading tests
concentrate on intensi-e reading to the e)cusion of e)tensi-e reading.
'robab" because it is more economica to ha-e a arge number of items
based on a short reading e)tract than a few items based on a much onger
one. Howe-er. these are insufficient grounds for negecting to test
e)tensi-e reading at certain e-es.
(n most tests. es'ecia" tests of genera 'roficienc". it is usefu to
incude a -ariet" of te)t t"'es for reading com'rehension in addition to the
(t7.
usua. more iterar" 'rose e)tracts! e.g. news'a'er artices. instructions for
using a''iances and machiner". director" e)tracts. 'ubic notices.
timetabes and ma's. ad-ertisements. etc. The incusion of such te)t t"'es
wi not on" 'ro-ide a more reaistic and reiabe means of assessment but
wi aso he' to moti-ate students b" demonstrating how the target
anguage is used in rea8ife situations. ConseBuent". it becomes im'ortant
that the actua 'resentation of the materia shoud be as authentic as
'ossibe. (n short. a news'a'er artice shoud a''ear in the actua form of
a news'a'er artice. thereb" gi-ing a genuine fee to the materia.
+e-era wa"s of testing reading com'rehension are treated in the
foowing sections of this cha'ter. Certain of the item t"'es wi be more
suited to testing com'rehension of a 'articuar te)t than other item t"'es.
(ndeed. there are numerous wa"s of testing reading com'rehension.
ranging from muti'e8choice items to o'en8ended Buestions 9i.e. Buestions
which reBuire students to write an answer in a sentence of their own:.
,though muti'e8choice items are sometimes the most suitabe
instruments for testing reading com'rehension. the" shoud not be o-er8
used. ;reBuent". other item t"'es are far more interesting and usefu. The
te)t itsef shoud awa"s determine the t"'es of Buestions which are
constructed. Certain te)ts ma" end themse-es to muti'e8choice items.
others to trueKfase items. others to matching items. others to re8
arrangement items. others to ordinar" com'etion items. others to the
com'etion of information in tabes. and "et others to o'en8ended
Buestions. (ndeed. sometimes the same te)t wi demand at east two or
three different t"'es of items.
0.< (nitia stages of The tests described in the first haf of this section are concerned 'ure"
reading! matching with word and sentence recognition. The" test students> abiit" to
tests discriminate -isua" between words which are s'et in fair" simiar wa"s.
if used in e)ercise materia and 'rogress tests. these test items wi assist in
de-eo'ing word recognition s'eed. Though not administered as s'eed
tests in the strict sense in the -er" ear" stages. word and sentence
matching items shoud be co-ered b" the students as Buick" as 'ossibe.
4nce the students ha-e gained famiiarit" and confidence with this t"'e of
test. their 'erformances shoud be timed so that the" are forced to read
under some 'ressure. ,t first. it is ad-isabe to confine the words used in
the items to those aread" encountered ora"@ ater a number of words not
encountered ora" shoud be introduced.
)ord matching The testees are reBuired to draw a ine under the word which is the same as
the word on the eft.
riow bowKnotKhowKnow mow
shee'
sho'Ksha'eKsee'Khea'Kshee'
e-er ne-erKo-erKe-erKfewerKe-en
to' to'Ksto'Kta'K'otKton
wonder wanderKwonderKwindowKfonderKwon
has gone is goneKhas wonKhas goneKhis gameKhad gone
cothes cothKcothingKcothsKcotsKcothes
most most 'easantKmore 'easantKmost 'resentKnot 'easantKmost
'easant 'easants
'entence matching This item is simiar to the word8matching item. The testees are reBuired to
recognise as Buick" as 'ossibe sentences which consist of the same words
137
i
%ictures and
sentence
matching
130
(
in the same order. The" read a sentence. foowed b" four simiar
sentences. on" one of which is e)act" the same as the 're-ious one.
1. Tom is not going to "our schoo.
,. Tom is not going to "our 'oo.
B. Tom is going to "our schoo.
C. Tom is not coming to "our schoo.
6. Tom is not going to "our schoo.
<. The thief can hide in the Junge.
,. The thief can die in the Junge.
B. The thie-es can hide in the Junge.
C. The thief can be hidden in the Junge.
8. OO..6.rQhe)bi.efOcanhide in the Junge.
(n the remainder of this section the items wi concentrate on word and
sentence com'rehension. using 'ictures to test this ski.
T"'e 1 This t"'e of item is simiar to that used to test istening
com'rehension and described under T"'e 3 in +ection ?.2. The testees ook
at four 'ictures and then read a sentence about one of the 'ictures. The"
are reBuired to identif" the correct 'icture.
The" are c"cing to work.
He is showing her the 'hotogra'h.
T"'e < This t"'e is simiar to the 're-ious one but is much more
economica in that on" one 'icture is reBuired for each item 9instead of
four:. The testees ook at a 'icture and read four sentences. on" one of
which is about the 'icture. The" then ha-e to seect the correct sentence.
,. Jenn" is throwing the ba to 5eter.
B. 5eter is kicking the ba to Jenn".
C. 5eter is throwing the ba to Jenn".
6. Jenn" is kicking the ba to 5eter.
,. The man under the tree is reading
his book.
B. The man resting under the tree is
ooking at his book.
C. The man with the book is see'ing
under the tree.
6. The man carr"ing the book is going
to sit down under the tree.
T"'e 3 ,though this item t"'e is referred to here as a matching item. it
coud eBua" we take the form of a trueKfase item 9in which the testees
write T or ; at the side of each sentence according to whether or not the
sentence agrees with the contents of the 'icture:. (n this 'articuar
instance. testees ha-e to seect the 9four: sentences which match the
'icture.
;our of the foowing sentences agree with the 'icture. 5ut a circe round
the etter of each of the. four sentences.
,. The dog on the foor is asee'.
B. The bab" is 'a"ing with the dog.
C. The bab" has Just broken a to" car.
6. The tee-ision set is on fire.
13/
11
E. The dog is behind the bab".
;. The woman has taken the fowers out of the bow.
&. 4ne of the two bo"s is he'ing the woman.
H. The woman is going to 'ick u' a book.
1. The two bo"s are istening to the radio.
J. The radio is on the tabe but the book is on the foor under the tabe.
0.3 (ntermediate and
ad-anced stages of
reading! matching
tests
C<rre
, ... B ...
T"'e 1 The foowing matching item< shows how -isuas can be used to test
the com'rehension of definitions of certain words. Testees are reBuired to
match the meaning of certain terms in a dictionar" with the a''ro'riate
sha'es which those terms denote. The e)am'e shows how the matching
techniBue can be used at a more ad-anced e-e and how it can end itsef to a
more communicati-e testing of reading. ,bo-e a. this 'articuar item
measures the abiit"8of.the testee to understand the kinds of definitions
usua" found in a dictionar" 8 an essentia ski reBuired in earning and
using a foreign anguage.
+ome of the sha'es are described in the dictionar" e)tracts. Name
on" those that8 are described in the e)tracts. This first one has been
done for "ou.
A
c ... 6 ... E ;
N,A
7
& ...
H ... ( J $ ... L
A ...
N ...
3 ... 5 ...
( 1 t:
coi T3iK
vt,vi wind or twist into a continuous
circuar or s'ira sha'e@ cur round and round!
&he sna-e $ed /itsel!1 round the "ranch.
a n ICJ =
something coied@ a singe turn of something
coied! the thic- $$$s o! a p#thon.
< ength of
wire
wound in a s'ira to conduct eectric current.
coneKkaunK n \CX 1 soid bod" which narrows to a
'oint from a round. fat base. < something of this
sha'e whether soid or hoow. 3 fruit of certain
e-ergreen trees 9fir. 'ine. cedar:.
cube KkJubK n
\CX 1 soid bod" ha-ing si) eBua
sBuare sides@ bock of something so sha'ed or
simiar" sha'ed. 2 /maths1. 'roduct of a
number
muti'ied b" itsef twice! &he $ o! F /F31 is 2 )
2
) 2 /=2F1. K -t muti'" a number b" itsef twice!
=0 $$d is = 000.
cu8bicK>kJubikK ad: ha-ing the sha'e of a cube@ of
a cube! one $$ metre, -oume of a cube whose
edge is one metre.
cu.bi8caK>kJubikaK ad: 0 cubic 9the usua word:.
c"(8inHderK>siinda9r:K n \CX 1 soid or hoow bod"
sha'ed ike a 'oe or og. < c"inder8sha'ed
chamber 9in an engine: in which gas or steam
works a 'iston! a si4 engine2motor$car.
c"indrica Ksi>indnkaK ad: c"inder8sha'ed.
ei'seKi>i'sK n \CX reguar o-a.
e(8(i'HticKi>(1'tikK. e8i'8tiHca K8kaK ad: sha'ed ike
an ei'se.
funne K>fnnaK n \CX 1 tube or 'i'e wide at the to'
and narrowing at the bottom. for 'ouring iBuids
or 'owders through sma o'enings. < outet for
smoke of a steamer. raiwa" engine. etc.

vt, vi
(( 8. *+ 8(8: 9cause to: mo-e 9as if: through a
funne.
at8ticeK>(aetisK n \CX framework of crossed aths or
meta stri's as a screen. fence or door. or for
cimbing 'ants to grow o-er! a $$ window.
at8ticed ad:
'"ramid K>'iramidK n \CX 1 structure with a
trianguar or sBuare base and so'ing sides
meeting at a 'oint. es' one of those buit of stone
in ancient Eg"'t. < 'ie of obJects in the sha'e of
a '"ramid.
s'hereKsfia9r:K n \CX 1 form of a gobe@ star@ 'anet.
music o! the spheres, /m#th1 music 'roduced
b"
the mo-ement of hea-en" bodies which men
cannot hear. < gobe re'resenting the earth or the
night sk".
s'herica K>sfenkaK adJ sha'ed ike a s'here.
wedgeKwedsK n \CX 1 E8sha'ed 'iece of wood or
meta. used to s'it wood or rock 9b" being
hammered:. to widen an o'ening or to kee' two
things se'arate. the thin end o! the wedge,
/5g1 a
sma change or demand ike" to ead to big
changes or demands. < something sha'ed ike or
used ike a wedge! $$ heels /on shoes:. -tfi)
tight" 9as: with a wedge! $$ a door open, b"
'acing a wedge under it. "e tightl#$$d
"etween
two !at women on the "us.
T"'e < The foowing item t"'e3 is incuded to 'ro-ide another e)am'e of
how reading com'rehension matching tests can be based on the dictionar".
,gain. the item is intended for use at a fair" ad-anced e-e.
'ettlements
Encosed hut grou's are characteristic>settements in the area and
remains8of more than a. hundred sti e)ist. The o'en settements. the
-iages of 'redominant" unencosed huts. are not numerous but on" a
doLen grou's ha-e sufficient numbers of co8ordinated huts to be
described as -iages. Though there ma" be some was in these -iages.
the" are on" fragmentar".
Aan" encosed settements ha-e disa''eared but one sti e)tant is
#ides #"s. (t consists of two encosures. one rough" sBuare and a arger
one rough" obong in 'an with a shared wa. ,n area of si) acres was
encosed containing more than thirt" buidings. The arge encosure. as
in the case of other muti'e settements. had been buit on to the
smaer and indicates an e)'anding communit".
111
The foowing dictionar" definitions are for words used in the 'assage
+ettements. 7rite the words from the 'assage ne)t to the a''ro'riate
definition.
ha-ing or in-o-ing more than one 'art as an indi-idua
ser-ing as re'resentati-e
becoming greater in siLe
H H H
enough to meet a need or 'ur'ose
H
ha-ing something in common
H H H H H
in most cases
is a sign of
-er" near to@ a''ro)imate
broken off or incom'ete
in e)istence@ sur-i-ing
T"'e 3 The foowing item= is simiar to the 're-ious t"'e but here testees are
reBuired to match a''ro'riate words or information in the te)t with the correct
'arts of a diagram.
1 ?
< 7
3 ...
0 ..
= .. /
2 ...
13
T"'e = This item t"'e in-o-es the matching of a 'ara'hrase of 'hrases
and sentences in a te)t with the origina words in the te)t. each item
starting with D(nstead of . .>. ,though of use occasiona" in achie-ement
and 'roficienc" tests. this t"'e of item is more suited to teaching than to
testing as it can be used to he' students to increase their understanding of
a te)t.
(n the foowing e)am'e. students are asked to read a te)t and then
com'ete each sentence. The item. howe-er. cas for the matching of
information and is thus not actua" a com'etion t"'e of item.
11<
(t is im'ortant for each student to obtain at reguar inter-as a rough
idea of his or her 'rogress. How are goas being achie-ed week b"
weekG Aethods of continuous assessment of students> work are
re'acing e)aminations 8 or 'arts of e)aminations 8 on certain courses.
There are sti doubts about the ad-antages of continuous assessment it
the earning 'rocess but. if a''ied with care and discretion. continuous
assessment can be a far more -auabe means of assessing standards
than an e)amination. 5ro-ided that methods of continuous assessment
do not im'art a feeing of tension and strain. the" can be used to guide
students in their work and to inform them of the 'rogress the" are
making. (f no means of continuous assessment is a-aiabe. students
shoud attem't to e-auate and summarise their 'rogress -er" brief"
week b" week. Cear". such an attem't is more difficut in a subJect
which teaches skis 9e.g. earning a anguage. 'a"ing a musica
instrument: than in a content subJect 9e.g. histor". chemistr":. E-en as
far as skis are concerned. howe-er. it is a sim'e matter for students to
go back to an od e)ercise and do it again. The ease with which the" can
do what 're-ious" seemed a difficut e)ercise is often Buite remarkabe.
1. (nstead of taking about using such methods carefu" and wise". the
writer taks about a''"ing them
<. (nstead of sa"ing that it is harder for the students to assess 'rogress
made in earning a anguage than in histor". the writer sa"s that
3. (nstead of referring to wa"s of measuring students> 'rogress at
reguar inter-as. the writer uses the 'hrase ...
.....
=. (nstead of sa"ing that students shoud tr" to assess and re'ort brief"
on their 'rogress e-er" week. the writer sa"s that
8 ...
2. (nstead of sa"ing that continuous assessment is usefu uness it
makes students fee u'set and worried. the writer sa"s that it is usefu
0.= TrueKfase reading
The trueKfase test is one of the most wide" used tests of reading
tests
com'rehension. Not on" is the scoring of such a test straightforward and
Buick. but the scores obtained b" the testees can be -er" reiabe indices of
reading com'rehension 'ro-ided that the items are we constructed and
that there are enough of them. TrueKfase tests are of considerabe use for
incusion in cass 'rogress tests chief" because. unike muti'e8choice Lest
items. the" can be constructed easi" and Buick". aowing the teacher
more time for other tasks.
The trueKfase test. howe-er. has two main disad-antages! first". it can
encourage guessing. since testees ha-e a 23 'er cent chance of gi-ing a
correct answer for each item. +econd". as the base score is 23 'er cent
91 3 'er cent: and thus the a-erage test difficut" genera" in the region of 72
'er cent 91 23 'er cent:. the test ma" fai to discriminate wide" enough
among the testees uness there are a ot of items.
(t is. of course. 'ossibe to 'enaise the testee for guessing. and
instructions on the ines of the foowing ma" be incuded in the rubric!
113
(
Each correct answer wi be awarded two marks. Howe-er. for each
wrong answer. one mark wi be deducted from "our score. (t is better.
therefore. not to guess bind" and to ea-e a bank if "ou do not know
the correct answer.
+uch 'enaties. howe-er. are of dubious -aue. and the whoe subJect
of guessing is treated in greater detai in Cha'ter 11.
,nother soution to the 'robem of guessing is to incude a third
Buestion in addition to the trueKfase o'tion! e.g. true. fase. not stated 9i.e.
the reBuired information is not gi-en in the 'assage:. Thus. a rubric for
this item t"'e woud read!
,ccording to the 'assage. are the foowing statements true or fase. or is
it im'ossibe to draw an" concusionG
(n addition to the ease and s'eed with which the items can be
constructed. the great merit of the trueKfase reading test ies in the ease
with which suitabe test 'assages can be seected! a short reading e)tract.
for e)am'e. can 'ro-ide a basis for numerous items. Aoreo-er. the
trueKfase test can be used as a -auabe teaching8de-ice with which the
students> attention is directed to the saient 'oints in the e)tract b" means
of the trueKfase items.
(f the students> com'rehension of the trueKfase reading e)tract 9and
not the trueKfase items themse-es: is being tested. each of the trueKfase
items shoud be as cear and concise as 'ossibe. (n such cases. it is
essentia that the 'robem 'osed b" each item is fu" understc ,. (n man"
trueKfase reading tests some indication of the number of correct and
incorrect statements is gi-en to the students. ,though this ma" make the
test sight" easier for them. it does at east 'resent them with a cear
statement of the 'robem.
TrueKfase reading tests fa into two genera categories! those which
are inde'endent of a reading te)t 9T"'e 1: and those which de'end on a
te)t 9T"'e <:.
T"'e 1 (t is 'ossibe to construct trueKfase items which are com'ete in
themse-es! a testee>s com'rehension of each trueKfase item is tested b"
means of a series of genera truths. ;or e)am'e!
5ut a circe round the etter T if the statement is true. (f it is not true. 'ut a
circe round the etter ;.
1. The sun rises in the west. T ;
<. ;ish can>t f". but birds can.
T ;
3. Engand is as arge as ,ustraia. T ;
=. 7hen ice mets. it turns into water. T ;
T"'e < The construction of trueKfase items based on a reading e)tract
forms one of the most wide" used t"'es of reading tests. (t is often used at
eementar" e-es of reading com'rehension. but it can be used eBua"
effecti-e" at more ad-anced e-es. The foowing e)am'e iustrates its use
at a fair" ad-anced e-e!2
E"e8gaLing and e"e8a-oidance ha-e meanings and 'atterns of
'rofound significance. &aLing at others> e#es genera" signas a reBuest
for information and 'erha's affection. but embarrassment can resut
from too ong a mutua gaLe. (n fact. in intimate situations. there seems
to be an eBuiibriumm in-o-ing 'ro)imit". e"e contact. intimac" of to'ic.
11=
and smiing. (f one com'onent is changed. the others tend to change in the
o''osite direction.
But the e)tended gaLe seems to ha-e a function much dee'er than
that of maintaining a baance or ensuring a smooth fow of con-ersation.
(t signas. not sur'rising". an intensification of reationshi'. not
necessari" aong amorous ines. (t ma" be a threat. or a chaenge for
dominance.
, definite 'ecking order of dominance and submission emerges
from the -er" first e"e contact of strangers. Curious". when
con-ersation is 'ossibe. it turns out that the one who ooks awa" first
tends to be dominant. The a-erted e"e is a signa that its owner is about
to take the foor. 7hen con-ersation is not 'ossibe. howe-er. the first to
ook awa" wi be the submissi-e one.
,bnorma use of e"e contact or a-ersion ma" we indicate an
abnorma 'ersonait". ,dut schiLo'hrenics tend to use their e"es at a
the wrong 'oints in a con-ersation. and the bod iar can hod a stead"
gaLe far onger than his truthfu coeague when both are caught in the
same misdemeanour.
,ccording to the 'assage. si) of the foowing statements are true and si)
are fase. (t is not 'ossibe to draw an" concusions from the
information in the 'assage about the remaining three statements. 5ut T ;
or G in each bo). as a''ro'riate.
1. Looking at someone ese>s e"es or ooking awa" from them
means a 'erson is thinking -er" dee'".

<. 7e genera" ook towards another 'erson>s e"es when we
want information or a sign of affection from that 'erson.
11
3. (f two 'eo'e ook too ong at each other>s e"es. the" wi
usua" become embarrassed.
=. 7e are usua" 'uLLed b" someone who freBuent" ooks into
our e"es during a con-ersation.
2. 7hen engaged in -er" friend" con-ersation. a cou'e wi
'robab" ook ess at each other>s e"es the more the" smie
and the coser the" sit.
?. (t is easier to tak to someone in a friend" wa" if the 'erson we
are taking to does not gaLe too ong at the s'eaker.
7. Looking for a ong time at the other 'erson>s e"es is on" a
means of continuing a con-ersation smooth".
0. 7hen two 'eo'e gaLe for a ong time at each other>s e"es. it
888is a sign that8the"8are going to argue.
/. ,n e)tended gaLe can signa a threat or a bid for authorit" o-er
the other 'erson.
13. 7hen two strangers meet. the" use their e"es to contro or
infuence the other or to show their surrender to the other>s
authorit".
11. 7hen it is 'ossibe to tak. the first 'erson who ooks awa" is
the one who submits to the other 'erson.
1<. , 'erson shows that he or she wants to tak b" ooking
towards the other 'erson>s e"es.
11
112
13. 5eo'e with 'oor e"esight genera" stand or sit -er" cose to

the 'erson the" are addressing.
1=. ,bnorma 'eo'e usua" turn their e"es awa" from the other
'erson>s e"es more often than norma 'eo'e do.

12. (t is 'ossibe to te the difference between a iar and an honest
'erson b" their e"e8gaLing 'atterns when both are tr"ing to te
a ie.
The reading te)t in the 'receding e)am'e contained anguage at a
higher difficut" e-e than that used in the statements which foowed. (t is
'ossibe 9though not common 'ractice:. howe-er. to construct a reati-e"
sim'e te)t foowed b" more difficut statements! in such cases. the
com'rehension 'robems wi be contained in the statements themse-es
rather than in the te)t. 8
0.2 Auti'e8choice T"'e 1 (t can be argued that the t"'e of item in this section is in man"
items 9,:! short te)ts
wa"s a test of -ocabuar" rather than of reading com'rehension. These
'articuar items. howe-er. ha-e been incuded here because it is fet that a
com'rehension of the te)t is genera" of at east as much im'ortance as an
understanding of the meaning of the words for seection. This. of course. is true
of an" -ocabuar" item 'resented in conte)t! howe-er. here the
em'hasis is more on the correct understanding of the conte)t. The
foowing three e)am'es show the use of this item t"'e at eementar".
intermediate and ad-anced e-es res'ecti-e".
1. The e"es are wonderfu teachers 8 e-en musicians. who dea with
sound. earn as much b"
/doing, pla#ing, watching, practising1 as b"
istening.
<. The housewife who coud not afford to bu" cothes woud s'end
hours at her s'inning whee. s'inning her woo into "arn 8 a Job
which took itte ski but reBuired a ot of /a"ilit#, patience, talent,
wisdom1
and was done b" the fireside during the ong winter
e-enings.
3. Two8thirds of the countr">s
/!uel, endeavour, industr#, energ#1 comes
from im'orted oi. whie the remaining one8third comes from coa.
Aoreo-er. soon the countr" wi ha-e its first nucear 'ower station.
T"'e < Just as the 're-ious item t"'e is cose" reated to the testing of
-ocabuar". so this t"'e is 'erha's more accurate" described as a test of
com'rehension of grammatica structure. The testees are reBuired to
identif" the correct 'ara'hrase of a statement from a choice of four or fi-e.
The" are tod in the rubric that the 9four: statements ma" refer to the
entire sentence or on" 'art of the sentence. ,gain. e)am'es are 'ro-ided for
each of the three genera e-es.
1.
John is not as ta as +a" but he>s a itte taer than #ick.
,. +a" is taer than John and #ick.
B. John is not as ta as #ick.
C. +a" is taer than John but not as ta as #ick.
6. #ick is taer than John and +a".
11?
<. (n s'ite of the oud music. ( soon managed to fa asee'.
,. The oud music soon he'ed me to fa asee'.
B. ( soon fe asee' as a resut of the oud music.
C. The oud music made me unabe to fa asee' soon.
6. ( soon fe asee' e-en though the music was oud.
3. (f "ou>d forgotten to 'ut out "our hand. "ou woudn>t ha-e 'assed
"our dri-ing test.
,. %ou didn>t forget to 'ut out "our hand and "ou 'assed "our
dri-ing test.
B. %ou forgot to 'ut out "our hand and "ou faied "our dri-ing test.
C. %ou forgot to 'ut out "our hand but "ou 'assed "our dri-ing test.
6. %ou didn>t forget to 'ut out "our hand but "ou didn>t 'ass "our
dri-ing test.
T"'e 3 This item t"'e consists of a -er" short reading e)tract of on" a few
sentences 9or sometimes of on" one sentence:. The testees are reBuired to
answer on" one com'rehension test item on each reading 'assage. The
actua construction of muti'e8choice reading com'rehension items based on
a reading e)tract wi be treated in greater detai in the ne)t section.
Aeanwhie. here are two e)am'es of the use of muti'e8choice items for
testing reading com'rehension. the first being at a fair" eementar" e-e and
the second at a more ad-anced e-e.
1.
The 'resident was taking to a "oung woman in the crowd when Tim
sudden" caught sight of a man standing se-era "ards behind her.
The man had something in his hand! it was a short stick.
7hat made Tim notice the man in the crowdG
,. He was -er" cose to Tim.
B. The 'resident was taking to him.
C. He was standing in front of the woman.
6. He was carr"ing a stick.
<. There were on" two wa"s of mo-ing aong the narrow edge! face
outwards or face to the wa. ( concuded that e-en the smaest of
bottoms woud 'ush a 'erson with his back to the wa far enough
out to o-erbaance him and so. with arms outstretched in the sha'e
of a cross and with chin 'ointed in the direction ( was heading. (
inched m" wa" aong.
The writer managed to cross the narrow edge b"
,. crawing aong on his knees with his arms stretched out in front c
him.
B. mo-ing sidewa"s inch b" inch with his back to the wa.
C. working his wa" forward on his stomach with his face amost
touching the edge.
6. waking sow" with his face and stomach cose to the wa.
0.? Auti'e8choice
The muti'e8choice test offers a usefu wa" of testing reading
items 9B:! onger te)ts
com'rehension. Howe-er. not a muti'e8choice reading tests are
necessari" good tests of reading com'rehension. ,s was cear" indicated
earier. the e)tent to which a test is successfu in measuring what it sets ou@
to measure de'ends arge" on the effecti-eness of each of the items used.
(ndeed. certain genera as'ects of man" reading tests ma" be sus'ect. ;or
instance. does the usua brief e)tract for reading com'rehension
concentrate too much on de-eo'ing on" those skis reBuired for intensi-e
reading. encouraging freBuent regressions and a word8b"8word a''roach to
readingG
The sam'ing of the reading 'assage is of the utmost im'ortance and
must be reated to the broader aims of the anguage teaching situation.
Aan" of the te)ts in both schoo and 'ubic e)aminations concentrate too
much on a iterar" kind of Engish. (f certain students are earning Engish
in order to read technica Journas. for e)am'e. then the sam'ing of the
reading e)tract shoud refect this aim. (dea". in a test of 'roficienc" the
te)t shoud contain the t"'e of reading task which wi be demanded of the
testees in ater rea8ife situations. (f the test is a cass 'rogress or
achie-ement test. the reading 'assage shoud be simiar to the t"'e of
reading materia with which the students ha-e been confronted in their
work at schoo. (n other words. if other subJects are being taught in the
medium of Engish 9as in man" second anguage situations:. the te)t shoud
freBuent" 9though not alwa#s1 refect the t"'e of reading the students are
reBuired to do in histor" or chemistr". etc.
(n this section. it is assumed that on" intensi-e reading skis are being
tested. Thus. the ength of the reading e)tract recommended might -ar"
from 23 to 133 words at the eementar" e-e. <33 to 333 words at the
intermediate e-e. and =33 to ?33 words at the ad-anced e-e. These
f i gures are. of course. e)treme" rough guides and ma" not be a''ro'riate
for man" reading situations. Aoreo-er. the e)tract seected shoud be
ca'abe of 'ro-iding the. basis for a sufficient number of muti'e8choice
com'rehension items. (t is not an eas" task to find an e)tract which wi
i
su''ort a number of muti'e8choice items 8 e-en though the same e)tract
ma" form a basis for a arge number of trueKfase items or o'en8ended
Buestions. &enera" s'eaking. 'assages deaing with a series of e-ents. a
coection of facts. or different o'inions and attitudes make the best t"'es
of te)ts for testing 'ur'oses@ those deaing with a si nge idea or main thane
are rare" suitabe.
The ength of the e)tract shoud aso be reated to its e-e of
difficut"! a 'articuar" difficut or com'e) 'assage woud 'robab" be
considerab" shorter than a more straightforward one. 4n the whoe. the
difficut" e-e of the te)t. howe-er. shoud coincide with the e-e of the
students> 'roficienc" in Engish. but we must remember that the reading
matter used outside the test situation 9e.g. sim'ified readers: shoud be
seected for enJo"ment and shoud thus be at a sight" ower e-e than the
actua standard of the reading skis acBuired. 9The difficut" e-e of a te)t
de'ends chief" on the degree of the structura and e)ica com'e)it" of
the anguage used.:
7hen writing test items based on a reading te)t. the tester shoud
attem't to construct more items than the number actua" reBuired. ,fter
the construction of the items. it is usefu to secure the ser-ices of one or
two coeagues so that a the items can be moderated. (n-ariab" this
'rocess brings to the attention of the item writer certain faws in some of
the items. ,though a number of the faws wi be easi" rectified. in certain
cases it wi be necessar" to dis'ense with entire items. (n tests of grammar
and -ocabuar". new items can awa"s be constructed in 'ace of the
discarded items. but this does not foow with reading com'rehension
items. The te)t itsef has to be rewritten. certain sections added and others
deeted in order to obtain the reBuired number of items. +uch 'rocesses
are difficut and time8consuming! thus. it is awa"s an ad-antage toH>b>
110
construct in the first instance more items than are actua" reBuired. (f the
te)t wi not aow for more items. another more suitabe te)t shoud be.
chosen to a-oid wasting time at a ater stage.
(t is usefu to incude items testing the students> abiit" to recognise8
8reference features in a te)t. no matter whether muti'e8choice. com'etion or
o'en8ended items are being constructed. , reference8word item can 'ro-ide the
e)aminer with s'ecific information about reading difficuties. (f a student fais
to 'ercei-e what the reference de-ice Dit> refers to in the .. te)t. for e)am'e. the
e)aminer immediate" knows the reason for his or her faiure to understand
that 'art of the te)t.
The griLL" bear roams some 1< miion acres in rugged 'arts of the
*nited +tates. ,nd this great bear sti roams our imagination at wi!.
it is 'art of its natura habitat.
The word it in ine 3 refers to
,. >the *nited +tates> 9ines 18<:
B. >this great bear> 9ine <:
C. >our imagination> 9ine <:
6. >its natura habitat> 9ine 3:
How man" muti'e8choice items shoud be set on one te)tG Cear".
the number of items wi de'end on the ength and com'e)it" of the te)t.
Howe-er. tests of reading com'rehension genera" contain fewer items
than other ski tests. ;urthermore. the testees reBuire much more time! to
work through a reading com'rehension test since the" first ha-e to read the
te)t carefu" once or twice before the" can begin to answer the items
based on it. 7hie as itte as ten or fifteen seconds for each item can be
aowed in muti'e8choice tests of. grammar and -ocabuar". at east one or
two minutes must be aowed for each item in the a-erage readir test. 9if
the time reBuired to read the te)t is taken into account:. Conseghent".
such tests. though ong in terms of time. must of necessit" be short in terms
of items and. therefore. ess reiabe.
The construction of items de'ending sim'" on a matching of wordsand
'hrases shoud be a-oided. (tems shoud test more than a su'erficia
understanding of the te)t and shoud reBuire the testees to digest and
inter'ret what the" ha-e read. The foowing e)am'es show how
ineffecti-e items can be if testees are sim'" reBuired to match the
words in the items with the words in the te)t.
,t four o>cock on +e'tember 33th two men armed with iron bars
attacked a sodier in 5rior" +treet.
7hat ha''ened at four o>cock on +e'tember 33thG
,. Two neminsi deraden with rinot babbers tackened a derisodt.
(magine that a testee did not understand much of the sentence to the 8e)t.
(n order to a''reciate this fu". it necessar"
to change
sight". and the te)t might a''ear to
u ike ti
,t four o>cock on +e'tember 33th two neminsi deraden with rinotH
babbers tackened a derisodt.
7hat ha''ened at four o>cock on +e'tember 33thG
,. Two neminsi deraden with rinot babbers tackened a derisodt8
, sight" better item stem woud be!
7hat ha''ened one afternoon at the end of +e'temberG
11/
i
(
Howe-er. to be com'ete" satisfactor". it woud be necessar" to rewrite
both the te)t and the item. as in the foowing e)am'e!
5au was sur'rised when he met +ue at the 'art". He was under the
im'ression she had gone awa" from the ocait". The ast time he saw
her was when Jane was teaching her to dri-e. , few da"s afterwards she
had sudden" become i.
9first -ersion: 5au was sur'rised when
,. +ue went awa".
B. he met +ue at the 'art".
C. Jane was teaching +ue to dri-e.
6. +ue sudden" became i.
9second -ersion: 5au did not e)'ect to see +ue because
,. he knew she was at the 'art".
B. he thought she had eft the district.
C. he had seen Jane teaching her to dri-e.
6. he had heard she was i.
There is often a tem'tation to concentrate too much on facts. figures
and dates when constructing test items based on a factua te)t. &enera"
s'eaking. figures and dates are incuded in a te)t chief" for the 'ur'ose of
iustration or to show the a''ication of a genera 'rinci'e. (t is usefu in
such cases to construct items which reBuire the testees to use the figures in
the te)t to state 9or restate: the genera 'rinci'e behind them. E.g.!
;rom Januar" to 6ecember ast "ear. </1 'eo'e were kied and ?.<=0
were inJured in road accidents on the cit">s roads. 127 of a the fata
accidents in-o-ed motorc"cists or their 'iion 'assengers. whie /2 O
in-o-ed 'edestrians and the remaining 3/ the dri-ers and 'assengers of
motor -ehices.
4-er haf of a the 'eo'e kied in road accidents ast "ear were
,. motorc"cists and 'iion 'assengers.
B. 'edestrians.
C. dri-ers of buses. cars and orries.
11
6. both 'edestrians and motorists.
Testees can aso be encouraged to use the figures the" are gi-en in a te)t
and to work out sim'e arithmetica sums and 'robems. Cear". there is a
imit to the tasks which the testees ma" be reBuired to 'erform! otherwise the
test writer wi be testing something other than anguage skis. The
foowing is an e)am'e of an item which tests students> abiit" to hande
sim'e facts and figures in Engish! the stem 'resents a usefu task provided
that this -ind o! reading e4ercise is not overdone.
Latest re'orts from the northeast 'ro-inces state that at east si)teen
'eo'e ost their i-es in +aturda">s foods. , further nine 'eo'e. most"
chidren. are re'orted missing. beie-ed dead. +e-en sma bo"s.
howe-er. had a miracuous esca'e when the" were swe't onto the
branches of some ta trees.
The tota number of 'eo'e re'orted dead or missing as a resut of
+aturda">s foods is
,. 7 B. / C. 1? 6. <2 E. 3<
1<3
(
The choice of the8correct o'tion in each muti'e8choice item must de'end
on a testee>s com'rehension of the reading te)t rather than on genera
8
knowedge or inteigence. The foowing item. for e)am'e. can be
answered without an" knowedge of the te)t on which it has been based.
Aemorising is easier when the materia to be earnt is
,. in a foreign anguage.
B. aread" 'art" known.
C. unfamiiar and not too eas".
6. of no s'ecia interest.
Care must be taken to a-oid setting distractors which ma" be true. e-en
though the" ma" not ha-e been e)'icit" stated b" the writer. (n the
foowing test item based on a reading te)t about the *nited Nations
and the dangers of war. C is the reBuired answer@ howe-er. a four o'tions
are correct 8 e-en though not stated in so man" words b" the writer.
7hat woud ha''en if there was a goba warG
,. Nations woud train men for war.
B. Lots of terribe wea'ons woud be made.
C. The whoe human race woud be com'ete" destro"ed.
6. 5eo'e woud grow -er" des'erate.
The correct o'tion must .be rough" the same ength as the distractors. (n the
foowing test item the correct o'tion has been modified to such a
degre. that it a''ears as the ob-ious answer without e-en necessitating an"
reference to the te)t.
The curricuum at the new coege is a good one in man" wa"s because
it
,. incudes man" science courses.
B. offers a we8baanced 'rogramme in both the humanities and the
sciences.
C. is reaistic.
6. consists of usefu technica subJects.
, the o'tions must be grammatica" correct! there is a tendenc"
es'ecia" in reading com'rehension to o-erook the grammatica
a''ro'riateness of some of the distractors used. 4'tion 6 in the foowing
item can be rued out immediate" because it is ungrammatica.
The writer sa"s that he had studied engineering for
,. a ong time.
B. on" a -er" short 'eriod.
C. se-era "ears.
6. ne-er.
6oube negati-es are on" confusing and such items as the foowing 9based on
the e)tract on 'age 1<3: are best a-oided!
5au did not e)'ect to see +ue because
,. he did not know she was at the 'art".
B. no one knew she had eft the district.
C. he hadn>t seen Jane teaching her to dri-e.
6. he didn>t reaise she was we.
, usefu de-ice in muti'e8choice tests of reading com'rehension is the
o'tion ,LL 4; THE+E or N4NE 4; THE+E!
1<1
1
,ccording to the 'assage. what do some 'eo'e think there shoud be
outside a modern cit"G
,. Buses
B. Car 'arks
C. 4ffice buidings
6. Ta)is
E. ,LL 4; THE+E
(f an o'tion ike E is used. it is ad-isabe to ha-e it as the correct answer in
at east one of the items. The testees shoud not be encouraged to think
that it has been incuded sim'" to make u' the reBuired number of
o'tions.
The foowing te)t and com'rehension items? iustrate some of the
guideines aid down. in.this section!
+tud" the foowing 'assage and then answer the Buestions set on it.
&he Captive is a strange but sincere and tender fim. as indeed one
woud e)'ect from a director of the caibre of Aarce L"me. (n addition
to his keen sensiti-it". L"me has a strong feeing for historica
atmos'here. so a''arent in his earier fim
9nder the 'hadow o! the
Buillotine,
in which the e-ents of the ;rench #e-oution are de'icted
2
with sur'rising reaism and -itait". (n
&he Captive L"me manages to
e-oke the atmos'here of an Engish town in the ear" 'art of the
nineteenth centur". not so much through the more ob-ious de-ices of
stage8coaches. od inns. and thatched cottages as through minute
attention to detais of s'eech. dress. customs. and mannerisms. +imiar 1
in theme to
Adam Brown, &he Captive
is distinguished b" a sincerit"
which the former acks and which he's to transform this fim from an
ordinar" ad-enture stor" into a memorabe and a -er" mo-ing traged".
Es'ecia" unforgettabe is the farewe scene at 5"mouth. when 8
Jonathan #obson sees Catherine 7insome on his wa" to the grim.
sBuaid shi' which is waiting to take him to ,ustraia. #obson breaks
oose from his ca'tors for a feeting moment to bid farewe to
Catherine. >(> 'ro-e m" innocence.> he cries -ehement" as he shakes
his fist at Catherine>s cousin.
,s the shi' sets sai. one enters a grotesBue nightmare word in
which e-i seems trium'hant. 4ur identification with #obson becomes
so 'ersona that we fee e-er" stroke of the fogging after he has been
caught steaing medicine for his sick com'anion. 7e share his s"m'ath"
for Joe Biggs as the od saior is haued under the shi'>s kee. (ndeed.
e-ents might we ha-e become unbearabe but for the ight reief
'ro-ided b" the comica antics of Bobo. the sma cabin bo" who ski's
about uncom'aining" doing whate-er task he is gi-en. 7e know. of
course. that utimate" e-i wi be -anBuished. and so we are gi-en
strength to endure the ad-ersities which confront the hero. The mutin"
and the conseBuent esca'e of Jonathan #obson. therefore. come as no
sur'rise.
Suestions
9a: ;or each of the foowing statements choose the word or 'hrase that
best com'etes the statement according to the informationO
contained in the 'assage. 7rite the number of the Buestion and the
answer "ou ha-e chosen in "our answer book.
1<<
/i1 &he Captive was directed b"
,. Jonathan Brown.
B. ,dam Brown.
C. Aarce L"me.
6. Catherine 7insome.
9ii:
(n &he Captive Aarce L"me con-e"s the atmos'here of the
nineteenth centur" chief" through
,. cose attention to sma detais.
B. the use of con-entiona scener".
C. stage8coaches. od inns. and thatched cottages.
6. de'icting dramatic e-ents of the time.
9iii: The 'assage im'ies that Adam Brown
was
,. a -er" mo-ing fim.
B. a reaistic and -ita fim.
C. an ordinar" ad-enture fim.
6. a sincere fim.
9i-: Jonathan #obson is angr" as a resut of
,. ha-ing to wait to go to ,ustraia.
B. being wrong" con-icted.
C. meeting Catherine.
6. being reca'tured.
9-: 4n the -o"age to ,ustraia #obson
,. becomes i.
B. begins to ha-e nightmares.
C. is haued under the shi'>s kee.
6. recei-es a fogging.
9-i: Bobo is introduced into the stor" to he' us to bear the grim e-ents
"#
,. beha-ing in a strange but interesting wa".
B. making us augh.
C. doing e-er"thing without com'aining.
6. acting kind" toward the hero.
9-ii: 7e can endure the hero>s sufferings because we know
,. things cannot get worse.
B. the crew wi mutin".
C. good wi win in the end.
6. the hero is -er" bra-e.
9-iii: The writer>s attitude to this fim is
OO
,. a''reciati-e
8 B. 'atronising.
C. scornfu.
6. critica.
9i): The word >his> in ine 3 refers to
A. >&he Captive> 9ine 1:
B. >one> 9ine 1:
C. >a director> 9ine <:
6. >L"me> 9ine 3:
9): The words >the former> in ine 1< refer to
,. >theme> 9ine 11:
B. >Adam Brown> 9ine 11:
1<3
C. >&he Captive> 9fine 11:
6. >a sincerit"> 9ine 11:
9)i: The word >his> in the 'hrase >7e share his s"m'ath"> in ine <3
refers to
,. >#obson> 9find <1:
B. >his sick com'anion> 9ine <3:
C. >Joe Biggs> 9fine <=:
6. >the od saior> 9ine <=:
9)ii: The word >he> in ine <7 refers to
,. >#obson> 9ine <1:
B. >Joe Biggs> 9ine <=:
C. >the comica antics> 9ine <?:
6. >Bobo> 9ine <.?LO
9b: Each of the foowing words and 'hrases can be used to re'ace one
word in the 'assage. ;ind the words and write them in "our answer
book. Number "our answers.
9i: dragged
9ii: conBuered
9iii: troubes and misfortunes
9i-: -er" brief
9-: fina"
0.7 Com'etion items Com'etion items measure reca rather than recognition. ,though such
items are simiar in man" wa"s to o'en8ended Buestions in tests of reading
com'rehension. the" are often regarded as beonging more to the obJecti-e
categor" of test items. There is -er" itte difference. howe-er. between the
foowing o'en8ended reading com'rehension Buestion!
7h" was the author sur'rised to meet 6r +hortG
and the eBui-aent com'etion item!
The author was sur'rised to meet 6r +hort because ...
*sua". com'etion items reBuire the testees to su''" a word or a
short 'hrase. *ness great care is taken to ensure that there is on" one
correct answer. the mar)ing wi 'ro-e -er" difficut when the tester is
confronted with a -ariet" of answers ranging from acce'tabe to
unacce'tabe. , -aid inter'retations. whether or not these were in the
test writer>s mind at the time of the construction of the test. must be
regarded as correct.
T"'es of com'etion items for testing reading com'rehension are
di-ided into two grou's for ease of treatment! T"'e 1 consisting of banks for
com'etion in the items foowing the te)t. and T"'e < consisting of banks in
the te)t itsef.
T"'e 1 *ness carefu" constructed. this t"'e of com'etion test can
become mere" a matching e)ercise in which the words and 'hrases
reBuired in the com'etion are determined after a 'rocess of matching the
whoe item with the a''ro'riate 'art of the te)t. Howe-er. ess em'hasis is
'aced on the writing skis in such a test than in a test consisting of o'en8
ended Buestions. The first e)am'e of this item t"'e iustrates how a short
informa etter can be used to 'ro-ide the basis for com'etion items.
1<=
E4ample =
<2? 7eeton #oad.
<nd ;oor.
Hong $ong.
7th June. 1/00.
6ear 6a-id.
( am -er" sorr" that ( coud not meet "ou ast night. ( ho'e that "ou did
not wait too ong outside the New %ork Theatre. ( had to ook after m"
sma brother unti m" mother returned home. +he was a ong time at
the doctor>s and she arri-ed home -er" ate. ( ran a the wa" to the bus
sto'. but ( had aread" missed the bus. ( decided to get on a tram and (
arri-ed at the New %ork Theatre at eight o>cock. ( did not think that "ou
woud sti be there because ( was three8Buarters of an hour ate. ( do
ho'e that "ou wi forgi-e me.
%our friend.
5eter
7rite one word or more in each bank.
1.
5eter i-es at ...
<. He wrote the etter on ...
3. 5eter coud not ea-e home because he had to wait for
. ( . to return.
=. His mother had been to the ...
2. 5eter went to the New %ork Theatre b" ...
8
?. He thought that 6a-id ...
7. The word ...
. means e4cuse.
0. 5eter had8arranged to meet 6a-id at ...
se-en on June ...
E4ample 2 The second e)am'e gi-es the stimuus in the form of a theatre
ad-ertisement. (t has been incuded here as a reminder of the im'ortance of
-ar"ing te)t t"'es in a reading com'rehension test and of using the t"'e of
reading materia which the student ma" meet in 'aces where Engish is used
as an e-er"da" means of communication.
#ead the foowing ad-ertisement and com'ete each sentence. 7rite
one word or more in each s'ace.
NE7T4N THE,T#E
;4#THC4A(N& ,TT#,CT(4N+
Monda#, L Januar# !or 2 wee-s $
A% ;,T ;#(EN6
Chares Laurence>s 'o'uar comed"
)ednesda#, 2+ Januar# to
'aturda#, 2E Januar#
+hanghai ;esti-a Baet
'resents
+7,N L,$E
Monda#, 2M Januar# !or one
wee-
#*N ,N6 $(LL
Tim 6anb">s thriing m"ster"
1. The +hanghai ;esti-a Baet wi 'erform
on ... e-enings.
< . . wi be the
O O most amusing 'a".
3. (f "ou ike dancing. "ou shoud see
=. The 'a" written b"
.is -er"
e)citing.
1<2
(
9
E4ample 3 (n this com'etion item the students are reBuired to fi in a
tabe based on a 'assage for reading com'rehension. The information
e)tracted from the te)t is best tested b" reBuiring the students to 'ut it in
tabuar form.
The cit" with the highest tem'erature "esterda" was +inga'ore. ,t noon
the tem'erature in +inga'ore was 33PC and at midnight the tem'erature
there was <2PC. Tok"o had the second highest tem'erature. (t was on"
<PC ower there than in +inga'ore at noon. The tem'erature in #ome at
noon was 33PC. the same as in New %ork. Howe-er. New %ork>s
tem'erature at midnight was one degree ower than #ome>s. The noon
tem'erature in Cairo was </PC. one degree higher than that in ,thens and
Hong $ong. The tem'erature at midnight in 5aris was 11PC ess than that at
noon in 5aris "esterda". ,though +tockhom>s tem'erature at midnight8w8
as8tire same as the tem'erature at midnight in 5aris. its noon tem'erature
was 1=PC. The cooest cit" was London. with a tem'erature of on" <<PC at
noon and 13PC at midnight.
The fi-e cities with the highest tem'eratures had a ot of sunshine
throughout the da". ,though the sun did not shine at a in ,thens and
5aris. it did not rain. (t was aso coud" and dr" in +tockhom. but it
rained hea-i" in both London and Hong $ong.
%esterda">s 7eather ,round the 7ord
+ 1 +unn" C 1 Coud" # 1 #ain
Noon Aidnight
<?PC
Cairo
<?PC +
Hong $ong <2PC
<<PC #
33PC
5aris <2PC
#ome <2PC
+
+inga'ore
33PC <2PC +
+tockhom
C
Tok"o <?PC
+
T"'e < (n this item t"'e the testees are reBuired to com'ete the bank
s'aces in a reading te)t. The banks ha-e been substituted for what the test
writer considers are the most significant content words. ConseBucntZ. a
'ossibe weakness of such a test ma" resut from the faiure to su''"
adeBuate guideines to the testees@ the foowing is an e)am'e of a 't)r item
because the framework is insufficient to guide them. The inguistic cues are
thus inadeBuate and the testees are faced with the task of has ing to guess
what was in the e)aminer>s mind.
7hen we 91: ...
. something aong the 9<: ...
.. it wi
cause93:...
1<?
The foowing te)t iustrates how banks shoud be inters'ersed@ the
testee>s degree of success in com'eting the banks de'ends amost entire" on
his or her com'rehension of the whoe te)t.
7hen we side something aong the foor. it wi cause
91:
. (f something is -er" 9<: ...
. there wi be a ot
of friction between it and the ground. Howe-er. friction is
93:
. when something ros instead of 9=: ...
. The
in-ention of the 92: ... . was rea" an attem't to reduce friction.
*ness there is snow or ice. it is much harder to 9?: ...
something on a bo) or sedge than in a cart. Ba8bearings are used a ot
in machiner" to 97: ... . friction. (t is friction which causes
90: . to machiner" as the -arious 'arts 9/: ...
against one another. ;riction is reduced if we 'ut oi onto the
913: . (t shoud not be forgotten. howe-er. that
91 1:
. is aso usefu to us! it is necessar". for instance. for car
whees to gri' the 91<: ...
E4ample = (n some tests certain etters of missing words are gi-en. (n these
cases. the testees are genera" informed that each dash in the bank
signifies a etter.
The might" ,maLon f888s into the ,tantic near the EBuator! its
es8888" is about 173 mies wide. The w88th88 is often so mist" that
the b888s of the ri-er cannot be seen from a shi'. e-en if it is
'8s888g Buite cose to them.
(t is aso 'ossibe to 'ro-ide on" the initia etter of the missing word. This
item is not to be recommended for most 'ur'oses as the incusion of etters
can often create menta bocks and on" confuse students if the" fai to
think of the e)act word reBuired. This is cear" disad-antageous for
reiabe assessment as a student might understand the 'assage and woud
otherwise ha-e been abe to com'ete the 'assage with a suitabe s"non"m.
E4ample 2 , -ariation of this t"'e of reading com'rehension ma"
incor'orate the muti'e8choice techniBue!
,stroog" is the ancient 91: ...
. of teing what wi
9<: . in the future b" stud"ing the 93: ... . of the stars
and 'anets. 9=: ...
.. astroogers thought that the stars and
'anets infuenced the 92: ... . of men. the" caimed the" coud
te 9?:
91: custom business magic knowedge art
9<: coincide ha''en chance come forete
93: 'aces sha'es times 'ositions ight
9=: Howe-er Because ,though ;or 7hie
92: affairs matters businesses chances times
9?: horosco'es future ad-ice fortunes uck
E4ample 3 The foowing t"'e of item is used to most ad-antage when the
item itsef is reated to the kinds of tasks the testees are reBuired to
'erform in their studies or in rea ife. ;or e)am'e. testees can be
'ro-ided with the dictionar" definitions of a number of words. 9This
materia is sim'" e)tracted from a good dictionar". together with
information concerning 'ronunciation. -erb 'atterns. 'arts of s'eech. etc.
1<7
i
and e)am'es of use8where a''ro'riate.: *nderneath the dictionar"
e)tract are 'rinted incom'ete sentences. which testees are reBuired to
com'ete with the most a''ro'riate word.
E4ample + (n the foowing t"'e of item the reading com'rehension te)t is
reated to information contained in a diagram or tabe.
Look at the foowing diagram! (t shows the 'aces to which trains tra-e
from London. (t aso shows the times taken to tra-e to these 'aces.
N
7 ZE
+
Ho"head
=hr </ Chester
<hr 3/
Hereford
<hr ==
+wansea
Edinburgh
Z=hr 3=
Hu
<hr 37
6oncaster
1 hr 3/
Norwich
1 hr 20
<hr =1
?
( N
Cardiff
1 hr =3 +outham'ton
7e"mouth 1 hr /
<hr =< 8>8Q
L4N64N
Cochester
23 min
6o-er
1 hr <7
(t takes two hours and se-en minutes to tra-e to 91: 91:
from London. haf an hour ess than it takes to
9<:. (t takes one hour and thirt"8nine minutes to 9<:
tra-e to 93:. which is aso in the north of Engand. (t 93:
takes the same time to reach 1=: as it does to reach
9=:
Aanchester. 92: is the furthest 'ace from London. 92:
9?:. a sma town in the west of the countr". is on" 9?:
fi-e minutes nearer London than Edinburgh is. Just
under two hours> tra-eing time from London is 97:. 97:
a cit" to the east of London. (t takes a itte under an
hour and a haf to reach 90:. a 'easant town to the 90:
southeast of London. To the southwest of London is
9/:. which is one hour and nine minutes from there 9/:
and is on the wa" to 913:. (f "ou tra-e to 911:. it wi 913:
take two hours and fort"8one minutes. %ou wi 'ass 911:
through 91<: amost an hour before "ou reach there. 91<:
913: is the second argest cit" in Engand. one hour 913:
and thirt"8four minutes from London. (f "ou i-e in
91=:. "ou can catch the Birmingham train. the 91=:
Aanchester train or the Ho"head train. , three
trains wi 'ass through this o-e" countr" town.
E4ample F Com'etion items are aso usefu for testing the abiit" to
deduce word meanings from the cues a-aiabe in the conte)t. +uch an
abiit" is of the utmost im'ortance in reading and shoud be encouraged in
both teaching and testing at a e-es. The foowing is an e)am'e of an
1<0
1
item designed to test this abiit" at the intermediate e-e. +tudents are
instructed to read the entire te)t before attem'ting to com'ete each
bank. The first e)am'e contains on" one bank whie the second e)am'e
contains three banks. each reBuiring the same word.
94ne bank! one word:
The &reat 5"ramid at &iLa in Eg"'t is without doubt sti one of the
greatest ...
. in the word. e-en though it has been studied -er"
thorough" o-er the ast centur" or so. (t ma". howe-er. be 'ossibe to
find out more about the '"ramid b" coser stud" o-er the ne)t few
"ears. 8
9Three banks! one word:
Ja'an. in the interests of both the Ja'anese and the word ...
needs to be assured that efforts wi be made to stabiise the "en. ;or its
'art. Britain is now taking acti-e ste's to im'ro-e its ...
. b"
de-eo'ing new industries and cutting down on im'orted goods of a
kinds. Taks now being hed in 5aris between ;rance and &erman" are
directed at wa"s of increasing the manufacturing ca'abiities of the two
countries in the ho'e that this wi ead to a genera im'ro-ement in the
Euro'ean ...
. and create more Jobs.
0.0 #earrangement These two item t"'es are 'articuar" usefu for testing the abiit" to
items
understand a seBuence of ste's in a 'rocess or e-ents in a narrati-e. 7hie
in an e)ercise for cassroom 'ractice the students wi often be reBuired to
rewrite the Jumbed sentences in their correct seBuence. it is ob-ious"
'referabe for testing 'ur'oses to instruct them to write sim'" the
numbers or etters of the Jumbed sentences. (t is aso ad-isabe to 'ro-ide
them with one or two answers! if students start off b" 'utting the first two
or three sentences in the wrong order. it ma" be im'ossibe for them to 'ut
the remaining sentences in the correct order. (n other words. one wrong
answer wi ine-itab" ead to a second wrong answer and 'ossib" a third.
and so on. This weakness must be borne in mind when marking! for
instance. shoud two incorrect answers 9i.e. one incorrect answer
automatica" resuting in another: be 'enaised b" the oss of two marks>
T"'e ( The first of these items takes the form of Jumbed sentences which
the students ha-e to >unscrambe> and arrange in the correct order. The
students are instructed to rearrange the etters of the sentences in their
correct order in the bo)es beow.
,. The dogs were se'arated from their human masters and were 'ut
into arge fieds.
B. ;or instance. the" wagged their tais. the" barked and growed. and
the" fawned on animas which 'ossessed food.
C. 5s"choogists obser-ing them found that the" reacted towards each
other in much the same wa" as the" used to res'ond to 'eo'e.
6. #ecent" an e)'eriment was conducted with a grou' of dogs to find
out how much their beha-iour was. in fact. sim'" a resut of human
en-ironment.
E. 5u''ies born to those dogs and ke't out of contact with 'eo'e
showed the same beha-iour 'atterns! the" were e)treme" wid and
afraid of human beings.
1</
T"'e < The Jumbed sentences in the second item are based on a reading
com'rehension te)t and shoud be unscrambed in the ight of the
information contained in the te)t. The students must write the etters of
the sentences in their correct order in the bo)es at the end of the item.
,gain. howe-er. the fact that one mistake ine-itab" resuts in another
reduces the item>s reiabiit".
7hen a customer waks into a tra-e agenc" to make a booking. the cerk
behind the counter turns immediate" to the sma com'uter unit on the
desk. The unit ooks ike a s"nthesis of a tee-ision set and a t"'ewriter.
The customer states the date and destination of the fight reBuired. and
the cerk t"'es this information on the ke"board. ,s each ke" is 'ressed.
a etter is formed on the screen of the unit. The cerk then checks the
information onthe screen before transmitting the data to a centra
com'uter. This com'uter contains a the information about current
bookings and destinations. and ra'id" estabishes whether the new
booking is 'ossibe. The com'uter immediate" sends a re'". indicating
the number of -acancies on the fight reBuested or showing that the
fight is fu" booked. 7hie these figures are being dis'a"ed on the
screen. the" ma" change to indicate that another booking has been
made in another 'art of the word.
The cerk now t"'es in the customer>s reser-ation. after which the
com'uter wi reBuest his or her name and address and then other
information. This information. incuding an indication of how the ticket
wi be 'urchased 9cash or credit card:. is then t"'ed onto the screen.
Ne)t the com'uter confirms the booking and reBuests that 'a"ment be
made. 7hen the customer has 'aid for the ticket. the cerk t"'es this
information into the com'uter as we. ;ina". if modern eBui'ment is
being used. it is 'ossibe for the com'uter to 'rint out a ticket on the
s'ot.
Now 'ut these sentences in the correct order. 7rite on" the
letter of
each sentence in each bo). 9Three bo)es ha-e been com'eted for "ou.:
,. 6etais about the seats a-aiabe are sent back.
B. The com'uter then wants to be informed about the method of
'a"ment.
C. The tickets are issued.
6. The com'uter asks for 'ersona detais.
E. The customer makes his or her reBuest to the tra-e agent.
;. The reBuest is sent to the main com'uter.
&. The customer goes to a tra-e agent.
H. The com'uter reBuests 'a"ment.
1. The tra-e agent feeds the initia reBuest into a sma com'uter unit.
J. The booking is t"'ed into the com'uter.
$. The booking is confirmed.
&
; J
,s in the case of trueKfase t"'e reading items. it is 'ossibe to im'ro-e
the reiabiit" of the item b" introducing an additiona feature! i.e. D6on>t
know>. D(nformation not gi-en>. DNot a''icabe>. The foowing item7 is based
on an artice contained in a se'arate 'u8out sheet using a
news'a'er format 9but not shown here:.
133
11
%ou woud ike to earn something about migraine and so read the
artice >4ne8sided Headache>.
(n what order does the writer do the foowing in her articeG To answer
this. 'ut the number 1
in the first answer coumn ne)t to the one that
a''ears first. and so on. (f an idea does not a''ear in the artice. write
NK, 9not a''icabe: in the answer coumn.
7rite in e-er" bo)
a. +he gi-es some of the histor" of migraine. a.
b. +he recommends s'ecific drugs.
b.
c. +he recommends a herba cure. C.
d. +he describes migraine attacks. d.
e. +he gi-es genera ad-ice to migraine sufferers. e.
0./
CoLe 'rocedure ,though simiar in a''earance to com'etion items. coLe tests shoud not
be confused with sim'e bank8fiing tests. (n ordinar" com'etion tests the
words for deetion are seected subJecti-e" 9consisting arge" of structura
words in tests of grammar and ke" content words in -ocabuar" or reading
tests:. (n coLe tests. howe-er. the words are deeted s"stematica". Thus.
once the actua te)t has been chosen. the construction of a coLe test is
'ure" mechanica! e-er" nth word is deeted b" the test writer. Certain
test writers argue that the bank substituted for the deeted word shoud
corres'ond to the ength of the missing word but in most coLe tests a the
banks are now of uniform ength. *ness a 'hotostat co'" of the actua
'rinted te)t is being used 9in which case the words are deeted before the
'hotoco'"ing 'rocess:. it is sim'e8 to insert banks of uniform ength.
The inter-a at which words are deeted is usua" between e-er" fifth
and e-er" tenth. word. Howe-er. if e-er" se-enth word has been deeted in
the first few sentences. then e-er" se-enth word must be deeted for the
rest of the te)t. The fifth. si)th and se-enth words are the most wide"
fa-oured for deetion in doLe tests. (f e-er" third or fourth word is deeted.
the student wi ha-e e)treme difficut" in understanding the te)t as
insufficient cues wi be a-aiabe. (f e-er" tenth or twefth word is deeted.
it wi be necessar" to ha-e a ong te)t. ;or e)am'e. if there are =3
deetions e-er" se-enth word. the te)t wi be <038333 words in ength 9i.e.
=3 ) 7:@ if. howe-er. there are =3 deetions e-er" twefth word. the ength
of the te)t wi be =038233 words 9i.e. =3 ) 1<:.
The choLe test was origina" intended to measure the reading difficut"
e-e of a te)t. *sed in this wa". it is a reiabe means> of determining
whether.or not certain te)ts are at an a''ro'riate e-e for 'articuar
grou's of students 9both nati-e s'eakers and non8nati-e s'eakers:. The
reading te)t being e-auated is gi-en to a grou' of students and the
a-erage score of a the students in the grou' is obtained. (f the mean score
of the grou' is o-er 23 'er cent. the materia can be used b" the students
for reading at Dthe inde'endent e-e>. the te)t being considered eas"
enough for students to read on their own without an" he'. (f the mean
score obtained is between == and 23 'er cent. howe-er. the materia is
suitabe for use at Dthe instructiona e-e> 8 i.e. with the he' of the
131
1
teacher. (f the mean score is beow == 'er cent. the te)t is described as
being at Dthe frustrationa e-e> and shoud not be used e-en with the he'
of a teacher. Later research. howe-er. has shown that this range of scores
ma" -ar" considerab". the instructiona e-e on some occasions ranging
from 30 to 23 'er cent and on other occasions from =7 to ?1 'er cent.
5erha's the most common 'ur'ose of the doLe test. howe-er. is to
measure reading com'rehension. (t has ong been argued that doLe
measures te)tua knowedge! i.e. an awareness of cohesion in a te)t.
in-o-ing the interde'endence of 'hrases. sentences and 'aragra'hs within
the te)t.I *nike the other t"'es of items treated here. howe-er. a true
doLe test 9i.e. the mechanica deetion of words in the te)t: is genera"
used to measure glo"al reading com'rehension 8 athough insights can
undoubted" be gained into 'articuar reading difficuties. Howe-er. it is
aso argued that doLe measures an under"ing goba inguistic abiit" 9aid88
indeed e-en knowedge of the word: rather than sim'" those skis
associated with reading com'rehension. ,s the doLe 'rocedure is now
such an im'ortant feature in anguage testing 9es'ecia" in the *nited
+tates:. the whoe subJect is treated at much greater ength in the foowing
cha'ter.
Aeanwhie. the foowing e)am'es ha-e been incuded in this section
to demonstrate how doLe 'rocedure can be a''ied to the testing of
reading com'rehension at both the eementar" and the more ad-anced
e-es.
E)am'e 1 9Eementar":
4nce u'on a time a farmer had three sons. The farmer was rich and
had man" fieds. but his sons were aL". 7hen the farmer was d"ing. he
caed his three sons to him. >( ha-e eft "ou ...
which wi
make "ou ... .. he tod them. >But .. must dig in
a . fieds to find the .
. where the treasure
is
,fter the od man ... his three aL" sons ...
out
into the fieds . . began to dig. >(> . the first to
find . 'ace where the treasure ...
buried.> cried the
edest .. . >That>s the fied where ... . father 'ut the
treasure.> . . another son. The three ...
. dug a the
fieds . se-era "ears. but the" ... no treasure.
Howe-er. man" ... . grew in the fieds ..
. the sons had
dug. . -egetabes made them -er" ...
E)am'e < 9,d-anced:
(t is estimated that in the ast two thousand "ears the word has ost
more than a hundred s'ecies of animas. , simiar number of s'ecies of
birds has aso become e)tinct. The rea significance of ...
figures. howe-er. ies in ...
. fact that amost three8
Buarters ... . a the osses occurred .. . the 'ast
hundred "ears ... . as a direct resut .
man>s
acti-ities on this ... (t is essentia for . . whoe
'rocess of e-oution ...
. the e)tinction of certain ...
shoud occur o-er a ... . of time. But e)tinction ..
occur b" nature>s design ...
. not as a resut ... 8. . the
acti-ities of man ...
. is b" no means ... . to the
'reser-ation of ... . s'ecies of anima and ...
ife.
1 3<
Conser-ation means the ... . of a heath"
en-ironment ... . a whoe. (f conser-ation ...
ignored.
then within a ...
. short time our water ... . wi be
found inadeBuate. .. . seas and ri-ers wi ..
. fewer
fish. our and ...
'roduce fewer cro's. and ... . air we
breathe wi ... . 'oisonous. (t becomes on" ... .O!
matter
of time before ...
. heath deteriorates and before ...
0.13 4'en8ended and
misceaneous items
0.11 Cursor" reading
together with e-er" other ... . thing. disa''ears from
the ... . of the earth.
The term Do'en8ended> is used to refer to those Buestions which eicit a
com'ete" subJecti-e res'onse on the 'art of the testees. The res'onse
reBuired ma" range from a one8word answer to one or two sentences!
94ne8word answer:
&i-e the name of the town where the writer had a bad accident.
9,nswer in a few words:
%ou ha-e a friend who is keen on cross8countr" running. 7hich e-ent
can he enter at the end of the monthG
9+entence answer:
,ccording to the artice. wh" do "ou think so few foreign cars ha-e bee n
im'orted into +inga'ore recent"G
7hen marking o'en8ended items which reBuire answers in sentences. it is
freBuent" ad-isabe to award at east two or three marks for each correct
answer. (f the ma)imum for a correct answer is three marks. for e)am'e. the
marking guide might be as foows!
Correct answer in a grammatica" correct sentence or a sentence
containing on" a minor error
Correct answer in a sentence containing one or two minor errors 9but
causing no difficut" in understanding: <
Correct answer but -er" difficut to understand because of one or more
maJor grammatica errors ... 1
(ncorrect answer in a sentence with or without errors ... ...3
(t is awa"s usefu to write down 'recise" how marks shoud be awarded. e-en
if on" one 'erson is marking the items. This marking scheme wi then ser-e as
a reminder at a times. (t is. of course. essentia to write brief guideines if
more than one e)aminer is s'arking the items.
;ina". when constructing a reading com'rehension test. the test
writer shoud remember to et the te)t itsef determine the t"'es of items
set. (ndeed. if this 'rinci'e is obser-ed. it foows that different 'arts of a
'articuar te)t wi freBuent" reBuire different item h"'es. (t is now
becoming common 'ractice for more than one t"'e of item to be used to
test com'rehension of the same te)t. Thus. a reading com'rehension
'assage ma" be foowed b" one or two muti'e8choice items. se-era
trueKfase items. a few com'etion items and one or two o'en8ended items.
The tite of this section ser-es as a genera term to denote the skis in-o-ed in
reading Buick". skimming and scanning. The term
s-imming is used to denote
the method of gancing through a te)t in order to become famiiar with the gist of the
content@ scanning refers to the skis used when reading in order to ocate s'ecific
information.
1383
(
(n tests of reading s'eed the students are genera" gi-en a imited
time in which to read the te)t. Care must be taken to a-oid constructing
Buestions on the ess ree-ant 'oints in the te)t. but the students shoud be
e)'ected to be famiiar with the successi-e stages in which the te)t is
de-eo'ed. The actua reading s'eed considered necessar" wi be arge"
determined b" the t"'e of te)t being read and wi -ar" according to the
'ur'ose for which it is being read. (t is sufficient to note here that 'oor
readers 9nati-e s'eakers: genera" read beow <33 words 'er minute@ a
s'eed of between <33 and 333 words 'er minute is considered to be an
a-erage s'eed@ and 333 to 233 words is considered fast. 4n the whoe. it is
reaistic to e)'ect no more than a reading s'eed of 333 words from man"
ad-anced earners of a second anguage. Aost 'eo'e tend moreo-er to
read at a sower rate under test conditions or in an" situation in which the"
are reBuired to answer Buestions8on a te)t.
(n tests of skimming. the rubrics genera" instruct the students to
gance through the te)t and to note the broad gist of the contents. The" are
then gi-en a sma number of Buestions concerning on" the maJor 'oints
and genera outine of the te)t. +ometimes at the end of the skimming the
students are aowed a few minutes to Jot down an" notes the" wish to
make. but the" are not usua" aowed to refer back to the te)t. (f the
students are aowed to retain the te)t. the time for answering the -arious
Buestions on it wi be imited to order to discourage them from referring
back too often to the te)t.
(n scanning tests. the Buestions are gi-en to the students before the"
begin to read the te)t. thus directing them to read the te)t for s'ecific
information. (n such cases. it is he'fu to set sim'e o'en8ended Buestions
9e.g. D7hat is the writer>s -iew of modern trans'ortationG>: rather than
muti'e8choice items. The atter t"'e of item tends on" to confuse the
students since the" then find it necessar" to kee' in mind four or fi-e
o'tions for each item whie the" are reading.
Tests of s'eed reading shoud be administered on" when students
ha-e been adeBuate" 're'ared for the tasks in-o-ed in such tests. (t is
gross" unfair to test those reading strategies which ha-e ne-er 're-ious"
been 'ractised.
Notes and references
1
Based in 'art on Aunb". J 1/70 Communicative '#lla"us Gesign. Cambridge
*ni-ersit" 5ress
< *ni-ersit" of 4)ford 6eegac" of Loca E)aminations! E4amination in
English
'tudied as a 6oreign (anguage, 5reiminar" Le-e. 1/01
3 Joint Aatricuation Board! &est in English /<verseas1, Aarch 1/0<
= Joint Aatricuation Board! &est in English /<verseas1, Aarch 1/03
2 The te)t is sight" ada'ted from an artice b" ,nthon" Tucker in &he Buardian,
+e'tember 2th 1/?/.
? This com'rehension test was first constructed b" the author for the Hong Nong
English 'chool Certi5cate E4amination 9Education 6e'artment. Hong
$ong: in
1/?0. but was ater e)'anded to test>awareness of reference de-ices.
7
#o"a +ociet" of ,rts! &he Communicative 9se o! English as a 6oreign
(anguage,
Test of #eading. ,d-anced Le-e. June 1/0=
0 +ee Cohen. , 6 1/03 &esting (anguage A"ilit# in the Classroom.
Newbur"
House
132
Testing the writing skis
/.1
The writing skis
The writing skis are com'e) and sometimes difficut to teach. reBuiring
master" not on" of grammatica and rhetorica de-ices but aso of
conce'tua and Judgementa eements. The foowing ana"sis attem'ts to
grou' the man" and -aried skis necessar" for writing good 'rose into fi-e
genera com'onents or main areas.
8 anguage use! the abiit" to write correct and a''ro'riate sentences@
8 mechanica skis! the abiit" to use correct" those con-entions 'ecuiar to
the written anguage 8 e.g. 'unctuation. s'eing@
8 treatment of content! the abiit" to think creati-e" and de-eo'
thoughts. e)cuding a irree-ant information@
8 st"istic skis! the abiit" to mani'uate sentences and 'aragra'hs. and
use anguage effecti-e"@
8 Judgement skis! the abiit" to write in an a''ro'riate manner for
a
'articuar 'ur'ose with a 'articuar audience in mind. together with an
abiit" to seect. organise and order ree-ant information.
The actua writing con-entions which it is necessar" for the student to
master reate chief" 9at the eementar" stages: to 'unctuation and s'eing.
Howe-er. in 'unctuation there are man" areas in which 'ersona
Judgements are reBuired. and tests of 'unctuation must guard against being
too rigid b" recognising that se-era answers ma" be correct. *nfortunate".
tests of 'unctuation and s'eing ha-e often tended to inhibit writing and
creati-it".
4f far greater im'ortance in the teaching and testing of writing are
those skis in-o-ing the use of Judgement. The abiit" to write for a
'articuar audience using the most a''ro'riate kind of anguage is essentia
for both nati-e8s'eaking and foreign student aike. The use of correct
registers becomes 8an8im'ortant ski at ad-anced e-es of writing. ;aiure
to use the correct register freBuent" resuts in incongruities and
embarrassment. 7hereas nati-e s'eakers earn to make distinctions of
register intuiti-e". students of foreign anguages usua" e)'erience
'robems in mastering this com'e) area of anguage. The -arious kinds of
register incude cooBuiaisms. sang. Jargon. archaic words. ega
anguage. standard Engish. business Engish. the anguage used b"
educated writers of Engish. etc. The 'ur'ose of writing wi aso he' to
estabish a 'articuar register! for e)am'e. is the student writing to
entertain. inform. or e)'ainG
13>
, 'iece of continuous writing ma" take the form of a narrati-e.
descri'tion. sur-e". record. re'ort. discussion. or argument. (n addition to
the subJect and the format. the actua audience 9e.g. an e)aminer. a
teacher. a student. a friend: wi aso determine which of the -arious
registers is to be used. ConseBuent". the use of a''ro'riate register in
writing im'ies an awareness not on" of a writing goa but aso of a
'articuar audience.
,fter the 'ur'ose of writing and the nature of the audience ha-e been
estabished. Judgement is again reBuired to determine the seection of the
materia which is most ree-ant to the task at hand 9bearing in mind the
time a-aiabe:. 4rganisation and ordering skis then foow seection.
/.<
Testing ,n attem't shoud be made to determine the t"'es of writing tasks with
com'osition writing
which the students are confronted e-er" da". +uch tasks wi 'robab" be
associated with the writing reBuirements im'osed b" the other subJects
being studied at schoo if the medium of instruction is Engish. +hort
artices. instructions and accounts of e)'eriments wi 'robab" form the
main bod" of writing. (f the medium of instruction is not Engish. the
written work wi often take the form of consoidation or e)tension of the
work done in the cassroom. (n both cases. the students ma" be reBuired to
kee' a diar". 'roduce a magaLine and to write both forma and informa
etters. The concern of students foowing a 'rofession or in business wi
be chief" with re'ort8writing and etter8writing. whie at coege or
uni-ersit" e-e the" wi usua" be reBuired to write 9technica: re'orts
and 'a'ers.
4ne arge 'ubic e)amining bod"> e)'icit" states the kinds of writing
tasks its e)aminations test and the standards of writing e)'ected in the
'erformance of those tasks!
, successfu candidate wi ha-e 'assed an e)amination designed to test
abiit" to 'roduce a seection of the foowing t"'es of writing!
Basic Le-e! Letter@ 5ostcard@ 6iar" entr"@ ;orms
(ntermediate Le-e! ,s Basic Le-e. 'us &uide@ +et of instructions
,d-anced Le-e! ,s (ntermediate Le-e. 'us News'a'er re'ort@ Notes
The candidate>s 'erformance wi ha-e met the foowing minimum
criteria!
Basic Le-e! No confusing errors of grammar or -ocabuar"@ a 'iece of
writing egibe and readi" inteigibe@ abe to 'roduce sim'e
unso'histicated sentences.
(ntermediate Le-e! ,ccurate grammar. -ocabuar" and s'eing. though
'ossib" with some mistakes which do not destro" communication@
handwriting genera" egibe@ e)'ression cear and a''ro'riate. using a
fair range of anguage@ abe to ink themes and 'oints coherent".
,d-anced Le-e! E)treme" high standards of grammar. -ocabuar" and
s'eing@ easi" egibe handwriting@ no ob-ious imitations on range of
anguage candidate is abe to use accurate" and a''ro'riate"@ abiit" to
'roduce organised. coherent writing. dis'a"ing considerabe
so'histication.
(n the construction of cass tests. it is im'ortant for the test writer to f i
nd out how com'osition is tested in the first anguage. ,though the
em'hasis in the teaching and testing of the skis in Engish as a
13?
foreignKsecond anguage wi of necessit" be Buite different to the
de-eo'ment of the skis in the first anguage. a com'arison of the abiities
acBuired and methods used is -er" he'fu. (t is cear" udicrous. for
instance. to e)'ect in a foreign anguage those skis which the students do not
'ossess in their own anguage.
(n the 'ast. test writers ha-e been too ambitious and unreaistic in
their e)'ectations of testees> 'erformances in com'osition writing! hence
the constant com'aint that reati-e" few foreign earners of Engish attain
a satisfactor" e-e in Engish com'osition. ;urthermore. the backwash
effect of e)aminations in-o-ing com'osition writing has been unfortunate!
teachers ha-e too often antici'ated e)amination reBuirements b" beginning
free com'osition work far too ear" in the course. The" ha-e >'rogressed>
from controed com'osition to free com'osition too ear". before the basic
writing skis ha-e been acBuired.
Howe-er. once the students are read" to write free com'ositions on
carefu" chosen reaistic to'ics. then com'osition writing can be a usefu
testing too. (t 'ro-ides the students with an o''ortunit" to demonstrate
their abiit" to organise anguage materia. using their own words and
ideas. and to communicate. (n this wa". com'osition tests 'ro-ide a
degree
of moti-ation which man" obJecti-e8t"'e tests fai to 'ro-ide.
(n the com'osition test the students shoud be 'resented with a cear"
defined 'robem which moti-ates them to write. The writing task shoud be
such that it ensures the" ha-e something to sa" and a 'ur'ose for sa"ing it.
The" shoud aso ha-e an audience in mind when the" write. How often in
rea8ife situations do 'eo'e begin to write when the" ha-e nothing to
write. no 'ur'ose in8writing and no audience in mindG Thus. whene-er
'ossibe. meaningfu situations shoud be gi-en in com'osition tests. ;or
e)am'e. a brief descri'tion of a rea8ife situation might he gi-en when
reBuiring the students to write a etter!
%our 'en8friend is going to -isit "our countr" for a few weeks with her
two brothers. %our house is big enough for her to sta" with "ou but
there is not enough room for her brothers. There are two hotes near
"our house but the" are -er" e)'ensi-e. The third hote is chea'er. but it
is at east fi-e mies awa". 7rite a etter to "our 'en8friend. e)'aining
the situation.
Com'osition tites which gi-e the students no guidance as to what is
e)'ected of them shoud be a-oided. E)am'es of 'oor tites which fai to
direct the students> ideas are A pleasant evening, M# "est !riend,
(oo-
"e!ore #ou leap. A good 5lm which
I have recentl#, seen.
7ith the em'hasis on communicati-e testing. there is a tendenc" for
test writers to set tasks asking8the students to write notes and etters in
their own roe 9i.e. without 'retending to be someone ese:. Tasks
reBuiring the students to act the 'art of another 'erson are often a-oided
as it is fet the" are ess reaistic and communicati-e. Howe-er. this is
usua" far from being the case. (t is usefu to 'ro-ide the students not on"
with detais about a s'ecific situation but aso with detais about the
'articuar 'erson the" are su''osed to be and the 'eo'e about 9or to:
whom the" are writing. Com'are. for e)am'e. the two foowing tasks!
9a: 7rite a etter. teing a friend about an" interesting schoo e)cursion on
which "ou ha-e been.
9b: %ou ha-e Just been on a schoo e)cursion to a nearb" seaside town.
137
Howe-er. "ou were not taken to the beach and "ou had no free time at
a to wander round the town. %ou are -er" keen on swimming and "ou
aso enJo" going to the cinema. %our teacher often tes "ou that "ou
shoud stud" more and not waste "our time. 4n the e)cursion "ou
-isited the aw courts. an art gaer" and a big museum. (t was a -er"
boring a'art from one room in the museum containing od8fashioned
armour and scenes of battes. %ou found this room far more interesting
than "ou thought it woud be but "ou didn>t tak to "our friends or
teacher about it. (n fact. "ou were so interested in it that "ou eft a sma
camera there. %our teacher tod "ou off because "ou ha-e a re'utation for
forgetting things. 4n" "our cousin seems to understand "ou. 7rite a etter
to him. teing him about the e)cursion.
,though the former task is one which students ma" concei-ab" ha-e
to 'erform in rea ife. the atter task wi resut in far more reaistic and...
natura etters from the students sim'" because the s'ecific detais make
the task more meaningfu and 'ur'osefu. The detaied descri'tion of both
the situation and the 'erson in-o-ed he's the students to sus'end their
disbeief and gi-es the task an immediac" and reaism which are essentia
for its successfu com'etion.
Two or more short com'ositions usua" 'ro-ide more reiabe guides
to writing abiit" than a singe com'osition. enabing the testing of different
registers and -arieties of anguage 9e.g. a brief. forma re'ort:. (f the
com'osition test is intended 'rimari" for assessment 'ur'oses. it is
ad-isabe not to aow students a choice of com'osition items to be
answered. E)amination scri'ts written on the same to'ic gi-e the marker a
common basis for com'arison and e-auation. ;urthermore. no time wi
be wasted b" the testees in deciding which com'osition items to answer. (f.
on the other hand. the com'osition test forms 'art of a cass 'rogress test
and actua assessment is thus of secondar" im'ortance. a choice of to@ ics
wi cater for the interests of each student.
;ina". the whoe Buestion of time shoud be considered when
administering tests of writing. 7hie it ma" be im'ortant to im'ose strict
time imits in tests of reading. such constraints ma" 'ro-e harmfu in tests
of writing. increasing the sense of artificiait" and unreait". Aoreo-er. the
fact that candidates are e)'ected to 'roduce a finished 'iece of writing at
their -er" first attem't adds to this sense of unreait". +tudents shoud be
encouraged to 'roduce 'reiminar" drafts of whate-er the" write. and this
means gi-ing them sufficient time in an e)amination to do this. 4n" in this
wa" can writing become a genuine communicati-e acti-it".
/.3 +etting the (n addition to 'ro-iding the necessar" stimuus and information reBuired
com'osition
for writing. a good to'ic for a com'osition determines the register and
st"e to be used in the writing task b" 'resenting the students with a
s'ecific situation and conte)t in which to write. +ince it is easier to
com'are different 'erformances when the writing task is determined more
e)act". it is 'ossibe to obtain a greater degree of reiabiit" in the scoring of
com'ositions based on s'ecific situations. ;urthermore. such
com'osition tests ha-e an e)ceent backwash effect on the teaching and
earning 're'arator" to the e)amination.
The difficut" in constructing such com'ositions arises in the8writing
of the rubrics. 4n the one hand. if the descri'tion of the situation on which
the com'osition is to be based is too ong@ then the te)t becomes more
130
of a reading com'rehension test and there wi be no common basis for
e-auation. 4n the other hand. howe-er. sufficient information must be
con-e"ed b" the rubric in order to 'ro-ide a reaistic. he'fu basis for the
com'osition. (t is im'ortant. therefore. that e)act" the right amount of
conte)t be 'ro-ided in sim'e anguage written in a concise and ucid
manner. The foowing rubric. for e)am'e. can be sim'ified considerab"!
%ou ha-e been directed b" "our su'erior to com'ose a etter to a
'otentia cient to ascertain whether he might contem'ate entering an
undertaking that woud concei-ab" be of mutua benefit ...
The foowing are 'ro-ided as e)am'es of situationa com'ositions
intended to be used in tests of writing!
T"'e (
(magine that this is "our diar" showing some of "our acti-ities on certain
da"s. ;irst. fi in "our acti-ities for those da"s which ha-e been eft
bank. Then. using the information in the diar". write a etter to a friend
teing him or her how "ou are s'ending "our time. 7rite about 133
words. The address is not necessar".<
T"'e <
9Suestion 1:
8 7hie "ou are awa" from home. some8,merican friends are coming to
sta" in "our house. %ou are ea-ing before the" are due to arri-e. so "ou
decide to ea-e them some notes to he' them with a the things the"
wi need to know whie sta"ing in the house. %our friends ha-e ne-er
been to "our countr" before so there is Buite a ot of ad-ice "ou need to
'ass on. 7rite "our message on the noteet 'ad sheet beow.
/A "lan- notelet !ollows.1
9Suestion <:
7hie "our ,merican friends are sta"ing in "our house. the" write to
sa" that the" are enJo"ing themse-es so much that the" woud ike to
s'end two weeks. -isiting some other 'arts of the countr". The" woud
ike "our ad-ice about what to go and see and where to sta".
13/
7rite to "our friends gi-ing the best 'ossibe ad-ice "ou can from "our
own knowedge and e)'erience. with whate-er s'ecia hints and
warnings ma" be necessar". Aake sure "our friends know who the" can
write to for further information of an >officia> kind to he' them to 'an the
best 'ossibe hoida".
7rite "our etter in the s'ace beow. (t shoud be between 123 and <33
words in ength.3
/A "lan- s'ace !ollows.1
T"'e 3
#ead the foowing etter carefu".
17? 7ood Lane
London N7<
12th Aa"
6ear Ar Johnson.
wish to com'ain about the noise which has come from "our home ate
e-er" night this week. 7hie ( reaise that "ou must 'ractise "our
trum'et some time. ( fee "ou ought to do it at a more suitabe time. Ten
o>cock in the e-ening is rather ate to start 'a"ing. E-en if "ou coud
'a" we. the noise woud sti be unbearabe at that time.
( ho'e that in future "ou wi b a itte more considerate of the
feeings of others.
%ours sincere".
7. #obinson
Now write a re'" to this etter. %ou do not 'a" the trum'et but on two
or three occasions recent" "ou ha-e 'a"ed some trum'et music on
"our record 'a"er. %ou did not 'a" the ecord -er" oud" 8 certain"
not as oud" as Ar #obinson>s tee-ision. %ou want to te him this but
"ou do not want to become enemies so "ou must be reasonab" 'oite in
"our etter.
Care must be taken in the construction of etter8writing tasks to imit
the amount of information to which the student must re'". (f this is not
done scoring can become e)treme" difficut.
T"'e = , diaogue can be -er" usefu in 'ro-iding a basis for com'osition
work. (n such a writing task. students must demonstrate their abiit" to
change a te)t from one register to another. as in the foowing e)am'e!
#ead the foowing con-ersation carefu".
A# BL,C$! 7hat was the weather ike whie "ou were cam'ingG
L(N6,! Not too bad. (t rained the ast cou'e of da"s. but most" it was
fine. 7e weren>t abe to -isit the &orge 7aterfas on the ne)t
to the ast da". but ...
A# BL,C$! 7hat a 'it"R
L(N6,! 7e. a'art from that we did e-er"thing we wanted to 8
waking. cirr bing and Just sitting in the sun. 7e e-en managed a
-isit to Hock Ca-e.
A# BL,C$! How on earth did "ou get that farG
L(N6,! 7e c"ced. 4h ... and we went to the beach Buite a few times.
A# BL,C$! 6id "ou take "our bikes with "ouG
1 =3
L(N6,!> No. we borrowed some from a 'ace in the -iage.
A# BL,C$! 7hereabouts were "ouG
L(N6,! 4h. in a o-e" -ae"8 ots of woods and about twent" mies
from the sea. Just north of Hison.
A# BL,c$! ( remember one time when ( went8cam'ing. 7e forgot to take a
tin8o'enerR
L(N6,! That>s nothing. , goat came into our tent in the midde of the
night8 it ate a the food we had with usR
A# BL,C$! 7e. "ou seem to ha-e had a good time.
Now write an account of Linda>s hoida". using the con-ersation abo-e as
a guide. (magine other things which ha''ened to her during the
cam'ing hoida".
T"'e 2 Tabes containing information are aso usefu for situationa
com'osition since the" can genera" be read b" the students without much
difficut". Aoreo-er. as on" a short written te)t is used. the students are
thus not encouraged to re'roduce 'art of the rubric for use in their
com'osition.
(magine that a oca news'a'er has asked "ou to write an artice of
a''ro)imate" <23 words about the information in the foowing tabe.
7rite down the concusions "ou draw from the figures about the -arious
wa"s in which 'eo'e s'ent their hoida"s in 1/?0 as com'ared with
1/00. ,ttem't to e)'ain the reasons for these differences.
1/?0 1/00
Tra-eing abroad
= 17
&oing to seaside 30 31
Cam'ing 0 31
Eisiting friendsKreati-es in another town 11 13
&oing to another town 9but not to -isit friendsKreati-es: 1? 3
+ta"ing at home <3 0
T4T,L 133 133
T"'e ? (nformation con-e"ed in the form of a sim'e gra'h or histogram
=3
ma" aso 'ro-ide a suitabe stimuus for writing. +uch writing tasks of this
nature. howe-er. are suitabe on" for more ad-anced students.
32
*se the chart together with the information
33
beow8 to8gi-e8a(bdef sur-e".of the causes of
accidents on Link #oad between 1/77 and
<2
1/07.
<3
1/77870 #oad not in great use
1/7087/ Nearb" road cosed! road now in great use
12 1/7/803 Bus sto' mo-ed 133 "ards
1/03801 No changes
13
1/0180< +ign! Beware animals
1/0<803 No 'arking signs
1/030= +ign! ;o right turn
1/0=802 6oube white ine!
;o overta-ing
77
70 7/
03 01
0<
03
0= 02 0? 07 9etc.:
1=1
T"'e 7 The stimuus for writing ma" e-en take the form of notes.
*se the foowing notes to write an account of an accident for a
news'a'er. %ou shoud write about <23 words.
C"cist about to turn right.
Not 'ut hand out.
Lorr" behind sows down.
+'orts car behind orr" o-ertakes.
+wer-es to a-oid bo".
$nocks o-er od man on 'a-ement.
T"'e 0 ,n e)ceent de-ice for 'ro-iding both a 'ur'ose and content for
writing is the use of 'ictures. , 'icture or series of 'ictures not on"
'ro-ides the students with the basic materia for their com'osition but
stimuates their imaginati-e. 'owers.
The 'icture beow shows a dangerous Junction where accidents often
ha''en. 7rite a etter to "our oca news'a'er. describing the Junction
and mentioning some of the dangers and causes of accidents.
Link #oad! ,n ,ccident>Back +'ot>
(f the stimuus in a situationa com'osition is 'ure" -erba. the testees
often tend to re'roduce the 'hrases and sentences contained in it. 5ictures
and diagrams fortunate" a-oid this weakness.
T"'e / 5ictures and diagrams ser-ing as stimui for descri'tions of obJects.
'eo'e. 'aces and 'rocesses ma" a be used to ad-antage in a test 8 as
we as those in-o-ing the writing of directions. instructions.
cassifications. com'arisons and narrati-es. The foowing e)am'e> shows
how the iustration of two aircraft can be used as a basis for a fair"
detaied com'arison in a test.
1=<
,ngo8;rench Concorde
ength! ?<.17 m
wing s'an! <2.?3 m
no. of 'assengers! 13381==
ma)imum cruising s'eed! <333 kmih
Boeing 7=7
ength! 73.21 m
wing s'an! 2/.?= m
width of cabin! ?.1 m
no. of 'assengers! 3338233
ma)imum cruising s'eed! /70 km.h
T"'e 13 The foowing e)am'es tests students> abiit" to describe sha'es
and inear forms. as it is assumed that none of them wi ha-e an"
knowedge of the technica terms reBuired for reference to each 'icture. (t
is a searching test of their descri'ti-e writing abiit" and is. of course.
suitabe on" for fair" ad-anced students. The rubric is gi-en for this
'articuar item to he' readers to obtain a cearer idea of what is reBuired.
The 'ictures beow are arranged from the odest 91: to the most recent
93:. *se them to comment on de-eo'ments in a warrior>s cothes and
eBui'ment.
1=3
1
/O= &rading the
The chief obJection to the incusion of the com'osition Buestion as 'art of
com'osition
an" test is genera" on grounds of unreiabiit". Considerabe research in
the 'ast has shown how e)treme" unreiabe markers are 8 both in their
own inconsistenc" and in their faiure to agree with coeagues on the
reati-e merits of a student>s com'osition.
Aarkers ma" award their marks on 9i: what a student has written!
9ii: what the" beie-e the student meant b" what he or she wrote@
9iii: handwriting and genera a''earance of what the student has written@
and 9i-: 're-ious knowedge of the student. Aoreo-er. two markers ma"
differ enormous" in res'ect of s'read of marks. strictness and rank order.
;or e)am'e. marker , ma" gi-e a wider range of marks than marker B
9i.e. ranging from a ow mark to a high mark:@ marker C ma" ha-e much
higher e)'ectations than marker , and thus mark much more strict".
awarding ower marks to a the com'ositions! and fina" marker 6 ma"
'ace the com'ositions in a different order of merit. ,n e)am'e of these
differences can be seen in the foowing tabe. 9The tota number of
'ossibe marks was <3.:
+'read +tandard
4rder
, B , C
, 6
#ick 1= 13 1= /
1=
/
,manda 11 / 11 ? 11 1<
6ebbie
13 0 13 2 13 13
Tina 7 7 7 < 7 11
6a-e 2 ? 2 1 2 ?
The whoe Buestion of unreiabiit". howe-er. does not sto' here.
E-en if a student were to take two com'osition e)aminations of
com'arabe u ^^ficuiti>. there woud be no guarantee whatsoe-er that he or she
woud score simiar marks on both e)aminations. This 'articuar t"'e of
unreiabiit" is more common to the com'osition 'a'er than to an"
other and is termed test2re$test relia"ilit#. , further com'ication resuts
from a ack of mar-2re$mar- reiabiit"! in other words. if the same
com'osition is marked b" the same e)aminer on two occasions there ma" be a
difference in the marks awarded.
(n s'ite of a such demonstrations of unreiabiit". com'osition is sti
wide" used as a means of measuring the writing skis. The -aue of
'ractice in continuous or e)tended writing cannot he stressed too great".
, student>s abiit" to organise ideas and e)'ress them in his or her own
words is a ski essentia for rea8ife communication. Thus. com'osition
can be used to 'ro-ide not on" high moti-ation for writing but aso an
e)ceent backwash effect on teaching. 'ro-ided that the teacher does not
antici'ate at too ear" a stage the com'e) skis reBuired for such a task.
Aoreo-er. if a more reiabe means of scoring the com'osition can he
used. sam'ing a student>s writing skis in this wa" wi a''ear a far more
-aid test than an" number of obJecti-e tests of grammar.
,s is cear" demonstrated at the end of this section. it is im'ossibe to
obtain an" high degree of reiabiit" b" dis'ensing with the subJecti-e
eement and attem'ting to score on an >obJecti-e> basis. according to a
carefu" constructed s"stem of 'enaties. Howe-er. com'osition marking
can be im'ro-ed considerab" once the subJecti-e cement is taken into
1==
account and once methods of reducing the unreiabiit" inherent in the
more traditiona methods of assessment are em'o"ed. To start with.
testees shoud be reBuired to 'erform the same writing task. ,though
there ma" sometimes be a case for a imited choice of com'osition to'ics in
the cassroom. attem'ts at accurate assessment of writing abiit" can on"
be successfu if the choice of to'ic is se-ere" restricted or aboished
com'ete". , we8defined task in terms of one or two situationa
com'ositions can he' enormous" to increase the reiabiit" of the
e)amination.
Because of the inherent unreiabiit" in com'osition marking. it is
essentia to com'ie a banding s"stem 8 or. at east. a brief descri'tion of
the -arious grades of achie-ement e)'ected to be attained b" the cass. The
foowing are two e)am'es of descri'tions of e-es of 'erformance used b" a
we8known e)amining bod" in Britain! tabe 9a: for intermediate8e-e
earners and tabe 9b: for more ad-anced8e-e earners.
,s with the scoring of ora 'roduction. banding s"stems de-ised for a
'articuar grou' of students at a 'articuar e-e are far 'referabe to scaes
drawn u' for 'roficienc" tests administered on a nationa or an
internationa basis.
Tabe 9a:>
108<3
1?817
1<812
E)ceent Natura Engish. minima errors. com'ete reaisation
of the task set.
Eer" good&ood -ocabuar" and structure. abo-e the sim'e
sentence e-e. Errors non8basic.
&ood +im'e but accurate reaisation of task. +ufficient
naturaness. not man" errors.
O
0811 5ass #easonab" correct if awkward 4# Natura treatment
of subJect with some serious errors.
287 7eak
Eocabuar" and grammar inadeBuate for the task set.
38= Eer" 'oor(ncoherent. Errors showing ack of basic knowedge
of Engish.
Tabe 9b:>
108<3 E)ceent Error8free. substantia and -aried materia.
resourcefu and controed in anguage and
e)'ression.
1?817
Eer" good&ood reaisation of task. ambitious and natura in
st"e.
1<812 &ood
+ufficient assurance and freedom from basic error to
maintain theme.
0811 5ass
Cear reaisation of task. reasonab" correct and
natura.
287 7eak
Near to 'ass e-e in genera sco'e. but with either
numerous errors or too eementar" or transated in
st"e.
38= Eer" 'oorBasic errors. narrowness of -ocabua
r".
1=2
11
1
The foowing rating scae is the resut of considerabe and carefu research
conducted8in the scoring of com'ositions in the *nited +tates.G 4n" a
summar" of the scae is shown here and it must be remembered that in its
origina form sight" fuer notes are gi-en after each item.
Content
338<7 EMCELLENT T4 EE#% &446! knowedgeabe 8 substanti-e 8 etc.
<?8<<
&446 T4 ,EE#,&E! some knowedge of subJect 8 adeBuate
range 8 etc.
<1817
;,(# T4 544#! imited knowedge of subJect 8 itte substance 8
etc.
1?813 EE#% 544#! does not show knowedge of subJect 8 non8
substanti-e 8 Etc.
4rganiLation
<3810 EMCELLENT T4 EE#% &446! fuent e)'ression 8 ideas cear"
stated 8 etc.
1781= &446 T4 ,EE#,&E! somewhat cho''" 8 oose" organiLed but
main ideas stand out 8 etc.
13813
;,(# T4 544#! non8fuent 8 ideas confused or disconnected 8 etc.
/87 EE#% 544#! does not communicate 8 no organiLation 8 etc.
Eocabuar"
<3810 EMCELLENT T4 EE#% &446! so'histicated range 8effecti-e
8
wordKidiom choice and usage 8 etc.
1781= &446 T4 ,EE#,&E! adeBuate range 8 occasiona errors of
wordKidiom form. choice. usage but meaning not obscured. 138
13 ;,(# T4 544#! imited range 8 freBuent errors of wordKidiom
form. choice. usage 8 etc.
/87
EE#% 544#! essentia" transation 8 itte knowedge of Engish
-ocabuar".
Language use
<28<< EMCELLENT T4 EE#% &446! effecti-e com'e) constructions 8
etc.
<181/ &446 T4 ,EE#,&E! effecti-e but sim'e constructions 8 etc.
17811
;,(# T4 544#! maJor 'robems in sim'eKcom'e) constructions
8 etc.
1382 EE#% 544#! -irtua" no master" of sentence construction rues 8
etc.
Aechanics
2 EMCELLENT T4 EE#% &446! demonstrates master" of
con-entions 8 etc.
= &446 T4 ,EE#,&E! occasiona errors of s'eing. 'unctuation 8
etc.
3 ;,(# T4 544#! freBuent errors of s'eing 'unctuation.
ca'itaiLation 8 etc.
< EE#% 544#! no master" of con-entions 8 dominated b" errors of
s'eing. 'unctuation. ca'itaiLation. 'aragra'hing 8 etc.
4ne 'oint worth noting in the scae is that in each cassification 9e.g.
Content. 4rganiLation. Eocabuar". etc.: the owest grade described as
1=?
>Eer" 5oor> ends with the 'hrase >4# not enough to e-auate.> 9This is not
incuded here.:
Com'ositions ma" be scored according to one of two methods! the
impression method or the anal#tic method. Note. howe-er. that the
former
method does not in-o-e the use of a rating scae to an" arge e)tent.
The im'ression method of marking entais one or more markers
awarding a singe mark 91 muti'e marking:. based on the tota
im'ression of the com'osition as a whoe. ,s it is 'ossibe for a
com'osition to a''ea to a certain reader but not to another. it is arge" a
matter of uck whether or not a singe e)aminer ikes a 'articuar scri't. ,s
has been demonstrated. the e)aminer>s mark is a high" subJecti-e one
based on a faibe Judgement. affected b" fatigue. careessness. 'reJudice.
etc. Howe-er. if assessment is based on se-era 9faibe: Judgements. the
net resut is far more reiabe than a mark based on a singe Judgement.
&enera". three of four markers score each 'a'er. marks being
combined or a-eraged out to re-ea the testee>s score. The foowing tabe
shows how four markers can score three com'ositions using a fi-e8'oint
scae for im'ression marking.
Com'.1 Com'.< Com'. 3
Aarker ,! 3 2 =
Aarker B!
< = <
Aarker C! < =
3
Aarker 6! 3 = 1
Tota! 13 17 13
9G:
,-erage! <.2
= <.29G:
(n those cases where there is a wide discre'anc" in the marks aocated
9e.g. Com'osition 3 in the 're-ious e)am'e:. the scri't is e)amined once
again b" a four markers and each mark discussed unti some agreement is
reached. ;ortunate". such discre'ancies occur on" rare" after the initia
stages in muti'e marking.
, the e)aminers 'artici'ating in a muti'e8marking 'rocedure are
reBuired to gance Buick" through their scri'ts and to award a score for
each one. The marking scae ado'ted ma" be as itte as from 3 to 2 or as
arge as from 3 to <3 9athough it has been the author>s e)'erience that
most markers 'refer to use a 28'oint scae or an" simiar scae with on" a
few categories in order to obtain a wide range of marks:. (t is most
im'ortant that a markers be encouraged to use the whoe range of an"
scae! cear". marks which bunch around / to 1< on a <38'oint scae are of
itte use in discriminating among candidates. (t is aso im'ortant that a
the markers read through a certain number of scri'ts in a gi-en time
9usua" about <3 'er hour: and time themse-es. (f the" find themse-es
sowing down. marking fewer scri'ts 'er hour. the" are ad-ised to rest and
resume work when the" fee abe to mark at the reBuired rate. (m'ression
marking is genera" found more e)hausting than mechanica methods of
marking@ thus. it is essentia that markers sto' work when their attention
begins to wander or when the" find themse-es aborious" reading through
the content of each com'osition. (m'ression marks must be based on
im'resssion onl#, and the whoe obJect is defeated if e)aminers start to
reconsider marks and ana"se com'ositions. Aost e)aminers find it more
1=7
enJo"abe than an" other method of scoring com'ositions. +ome argue in
fa-our of marking for one or two hours at a stretch in order to maintain
consistenc". but itte concusi-e research has been carried out in this area
and there a''ears to be no e-idence that marking>for a ong 'eriod
'roduces more consistent and reiabe marks than marking for short
'eriods. (m'ression marking is genera" found to be much faster than
ana"tic or mechanica marking. (f com'ositions are scored b" three or four
im'ression markers. the tota marks ha-e been found to be far more
reiabe than the marks awarded b" one ana"tic marker. 9The com'arison
is a fair one. since it takes as ong for one ana"tic marker to score a
com'osition as it does four im'ression markers.: 4n the other hand. the
marks awarded b" one im'ression marker are ess reiabe than those
awarded b"..one ana"tic marker.
+ince most teachers ha-e itte o''ortunit" to enist the ser-ices Iof8two
or three coeagues in marking cass com'ositions. the ana"tic method is
recommended for such 'ur'oses. This method de'ends on a marking
scheme which has been carefu" drawn u' b" the e)aminer or bod" of
e)aminers. (t consists of an attem't to se'arate the -arious features of a
com'osition for scoring 'ur'oses. +uch a 'rocedure is idea" suited to the
cassroom situation! because certain features ha-e been graded se'arate".
students are abe to see how their 'articuar grade has been obtained. The
foowing is re'roduced sim'" as one e)am'e of such an ana"tic scheme!
in this 'articuar case du'icate 9bank: co'ies of this scheme were
stencied b" the teacher and attached to the end of each com'osition.
&rammar
Eocabuar"
Aechanics M
;uenc"
#ee-ance A M
Note that Mechanics refers to 'unctuation and s'eing@
6luenc# to st"e
and ease of communication@ and 7elevance to the content in reation to the
task demanded of the student. , 28'oint scae has been used.
(f the ana"tic method of scoring is em'o"ed. it is essentia that
f e)ibiit" is maintained. ,t the -arious e-es it ma" become necessar" to
change either the di-isions themse-es or the weighting gi-en to them. ,t
the eementar" e-e. for e)am'e. the tester ma" be far more interested in
grammar and -ocabuar" than in fuenc". thus deciding to omit 6luenc#. ,t
the intermediate e-e. the tester ma" be 'articuar" interested in
ree-ance and ma". therefore. decide to award a ma)imum of 13 marks for
this feature whie awarding on" 2 marks for each of the others. ,t the
more ad-anced e-e. the tester ma" wish to incude se'arate di-isions for
organisation and register and to incude mechanics and fuenc" in one
di-ision.
, third method of scoring com'ositions is the mechanica accurac" or
error8count method. ,though this is the most obJecti-e of8a methods of
scoring. it is the east -aid and is not recommended. The 'rocedure
consists of counting the errors made b" each testee and deducting the
number from a gi-en tota! for e)am'e. a testee ma" ose u' to 13 marks
1=0
for grammatica errors..2 marks for misuse of words. 2 for miss'eings.
etc. +ince no decision can be reached about the reati-e im'ortance of
most errors. the whoe scheme is actua" high" subJecti-e. ;or e)am'e.
shoud errors of tense be regarded as more im'ort ant than certain
miss'eings or the wrong use of wordsG ;urthermore. as a resut of
intuition and e)'erience. it is fair" common for an8e)aminer to fee that a
com'osition is worth se-era marks more or ess than the score he or she
has awarded and to ater the assessment according". ,bo-e a. the
mechanica accurac" method unfortunate" ignores the rea 'ur'ose of
com'osition writing 8 communication@ it concentrates on" on the negati-e
as'ects of the writing task. 'acing the students in such a 'osition that the"
cannot write for fear of making mistakes. The conseBuent effect of such a
marking 'rocedure on the earning and teaching of the writing skis can be
disastrous.
/.2
Treatment of Before brief" re-iewing some of the attem'ts to identif" error gra-it". it i
written errors
interesting to note a re'ort of an e)'eriment> in which nati-e s'eakers whc.
were not teachers scored written work b" its degree of inteigibiit" rather
than b" the errors it contained. Nati-e8s'eaking teachers. on the other hand.
e-auated written work b" the number and t"'es of errors it
contained. Non8nati-e s'eaking teachers of the anguage. howe-er.
'enaised students for what the" consider Dbasic errors> 9e.g. DHe go>: and
were genera" far stricter in their attitude to errors.
,though resuting in a far more 'ositi-e a''roach. e-auating wetter
work according to the degree of inteigibiit" is not awa"s a reiabe
method of assessment. ;reBuent". the student>s 'erformance 8 and
success in accom'ishing the task 8 ma" be masked b" errors and a tired
marker ma" fai to make the necessar" effort to res'ond to the writing as
means of communication.
,n im'ortant/ distinction is now made between
glo"al and local
errors. Those errors which cause on" minor troube and confusion in a
'articuar cause or sentence without hindering the reader>s com'rehensio!
of the sentence are categorised as oca errors 9e.g. misuse of artices.
omission of 're'ositions. ack of agreement between subJect and -erb.
incorrect 'osition of ad-erbs. etc.! D( arri-ed Leeds.>:. &oba errors are
usua" those errors which in-o-e the o-era structure of a sentence and
resut in misunderstanding or e-en faiure to understand the message whit
is being con-e"ed 9e.g. the misuse of connecti-es! D,though the train
arri-ed ate. we missed the ast bus to the cit" centre>@ the omission. misus
and unnecessar" insertion of reati-e 'ronouns! D%ou shoud tr" to be as
heath" as the gir arri-ed on the bic"ce a short time ago>@ etc.:. This
usefu distinction. which 'ro-ides criteria for determining the
communicati-e8im'ortance of errorshas een further de-eo'ed recent"
so that it can be more readi" a''ied to the marking of free writing. .
11
(n addition. it is necessar" in cassroom tests to distinguish between
those errors which. though 'erha's not resuting in an" breakdown in
communication. indicate that the student has faied to earn something
which has Just been taught or which shoud ha-e been mastered '.Ee-ious.
(n most norma writing situations. howe-er. we can on" assess what
student writes and not
what he or she wants to write. ;or this reason.
'ictures and diagrams can 'a" a -er" usefu 'art. in testing writing. since
the" enabe the e)aminer to te immediate" what a student wishes to
write. 5ictures were recent" used b" researchersI in an e)'eriment to
18
show how L< earners 9i.e. ess fuent earners: used a-oidance
strategies or
reduction strategies,
a-oiding an actua to'ic. Lt earners 9and 'ossib"
more fuent L8. earners: tended to use paraphrase strategies or
achievement strategies.
;uent 'erformance seems to be characterised b"
the use of fewer communication strategies of both kinds.
The test writer>s attitude to error gra-it" and a''roach to treating
errors in marking free writing wi -ar" according to the students> e-e of
attainment. ,t the eementar" e-es. for e)am'e. the test writer wi
'robab" be far more toerant than at the intermediate and ad-anced
e-es. ,t the ower e-es he or she wi e)'ect more a-oidance strategies
and more goba t"'es of error. 7hat is im'ortant at a e-es is an
awareness of different t"'es of error and of communication strategies.
resuting in an increased sensiti-it" to them.
/.? 4bJecti-e tests!
mechanics
%unctuation
T"'e 1 The foowing t"'e of 'unctuation item is -er" 'o'uar and is
genera" used to co-er8a wide range of 'unctuation marks. (t is not tru"
obJecti-e. and the scoring of such an e)ercise woud take considerabe
time since 'unctuation is to a arge degree subJecti-e and one 'articuar
use of a 'unctuation mark ma" we determine the correctness of the
'unctuation mark foowing it.
(n the foowing 'assage there is no 'unctuation. 7rite out the 'assage.
'utting in a the 'unctuation and ca'ita etters.
end me "our 'en 'ease 'eter asked
i took m" 'en out of m" 'ocket
be carefu i said
i gi-e it back to "ou in a moment he 'romised
dont worr" i said "ou can kee' it as ong as ". o. .u-IPID
(t is ad-isabe. howe-er. to maintain some degree of contro o-er the task
which the testees are e)'ected to 'erform. 4ne method of doing this is b..
substituting ines or circes for those 'unctuation marks which are being
tested. thus aso faciitating scoring.
T"'e <
5ut the correct 'unctuation mark in each bo).
6 7hat do "ou want. 6 1 asked Henr"6
6 Aa" ( use "our tee'honeG 6 he asked.
et me
Certain" n 6 1 said. 6 7hen "ou 6 -e finished 6 'ease
know 6 6
ered.
6 1 sha on" be a moment n 6 Henr" answ
6 Has John Lee in-ited "ou to his 'art" 66 ( asked.
6 No. he hasn 6 t "et 66 Henr" re'ied.
\ He 6 s in-ited 5au 6 6a-id 6 Ton" and Aar" 4n (
c continued. o
6 He 6 s 'robab" forgotten about me 6 6 Henr" aughed.
6 How strange 6 6 1 answered. 6 (>m sure he wants "ou to go to>his
'art".
T"'e 3 , greater degree of obJecti-it" can be obtained b" using the
muti'e8choice techniBue. e.g.
5ut a circe round the etter 9,. B. C. or 6: of the correct" 'unctuated
sentence.
,. Tom asked me if ( was going to the meetingG
B. Tom asked me. if ( was going to the meeting.
C. Tom asked me. >(f ( was going to the meetingG>
6. Tom asked me if ( was going to the meeting.
'pelling
T"'e 1! 6ictation
,s with -ocabuar" testing. sam'ing is of 'rimar" im'ortance in the
construction of s'eing tests. 7ords used in connection with the students> free
com'osition work or e-er"da" writing form the most suitabe basis for tests
of s'eing. athough items ma" aso be drawn from the students>
reading 'ro-ided that the tester is aware of the im'ications of testing the
more 'assi-e items of the students> -ocabuar".
6ictation of ong 'rose 'assages is sti regarded as an essentia
method of testing s'eing. Howe-er. dictation measures a com'e) range of
integrated skis and shoud not be regarded as constituting sim'" a test of
s'eing. The dictation of singe words. ne-ertheess. can 'ro-e a fair"
reiabe test of s'eing. +e-era such tests consist of u' to fift" words and use
simiar 'rocedures to the foowing!
9i: Each word is dictated once b" the tester@
9ii: the word is then re'eated in a conte)t@ and fina".
9iii: the word is re'eated on its own.
T"'e <! Auti'e8choice items
,nother fair" wides'read method of testing s'eing is through the use of
muti'e8choice items usua" containing fi-e o'tions. four of which are
s'et correct". The students are reBuired to seect the word which is
incorrect" s'et. e.g.
1. ,. thief B. beief C. seiLe 6. ceiing
E. decie-e
<. ,. ha''ening
B. offering C. occuring
6. beginning
E. benefiting
3. ,. iega B. genera" C. summar" 6. beggar
E. neccessar"
=. ,. interru't
B. su''ort C. answering 6. ocasiona
E. command
(n some tests on" four words are gi-en as o'tions. the fifth o'tion
being ;o mista-es or All correct,
e.g.
,. e)hibition B. 'unctua" C. 'easure 6. oba"ed
E. All
correct
T"'e 3! Com'etion items
+uch items as the foowing differ from simiar ones used in tests of
-ocabuar" because sufficient cues are 'ro-ided both in the banks and in
the definitions to enabe the students to know e)act" which word is
reBuired. The banks occur on" in those 'arts of the word which gi-e rise
to a s'eing difficut" for man" students. 4ne ad-antage of such a test is
that it does not 'resent the students with incorrect forms. 9Aan" nati-e
s'eakers argue that the" freBuent" fai to recognise correct forms after
e)'osure to miss'eings.:
121
7rite one or more etters in each of the s'aces in the foowing words.
9The definitions wi he' "ou to recognise the correct word.:
1. om s n something le!t out
<. di uade persuade someone
not to do something
3. o u ing happening, ta-ing place
=. rec t a written statement
to show that a "ill "as "een paid
(t is he'fu in tests of this nature to 'ro-ide a conte)t for the word 9in
addition to gi-ing a s"non"m or definition:.
1. The horse ga ed 91 ran1 to the front of the race.
<. ( doubt if an"one e-er 'rof
ed 91 gained1 from that business dea.
3. The schoo has an enro ment 91
num"er on its register1 of o-er 233
students..
=. 6on>t worr"@ m" dog wi go into the water and retr -e 91 "ring
"ac-1
"our ba.
T"'e =! Error8recognition items
(n these items the students are reBuired to identif" 9according to its etter:
the 'art of the sentence in which a word has been miss'et.
,
B C
1. The dissatisfied womanKrefused to admitKthat there was sufficentK
6
coffee for e-er"one.
, B
C
<. 6on>t be decie-edKb" the new 'rocedure!Kit hard" differs fromK
6
the od corru't s"stem.
, B C
3. The man was eBui''edKwith a doube8barre shotgunKand his
C
6
coeagueKwith an innocent8ooking waking8stick.
,
B
C
=. Ar &rant>s e)aggeratedKaccount of the im'ortanceKof his new a'oint8
6
mentK was Buite unnecessar".
/.7 4bJecti-e tests! The muti'e8choice items beow are concerned chief" with measuring
st"e and register
students> sensiti-it" to st"e. +ome of the distractors in the two e)am'es
are incorrect on grammatica grounds whie others are grammatica"
't#le
correct but not re'resentati-e of the kind of Engish used b" an educated
nati-e s'eaker in the 'articuar conte)t in which the" a''ear. (ndeed.
some test writers distinguish tests of writing from tests of grammar and
usage in terms of the 'erformance of nati-e s'eakers! whereas a nati-e
s'eakers of a anguage woud be e)'ected to score high marks in a test of
grammar. on" certain educated nati-e s'eakers 'ossessing the reBuired
writing skis woud score high marks in an obJecti-e test of writing.
+ome of the re-iews were fa-ourabe to the new 'a"
,. and as man" were unfa-ourabe.
B. athough others of the same amount were unfa-ourabe.
VC. whie an eBua number were unfa-ourabe.
t2.
6. but the same number were unfa-ourabe.
E. in s'ite of haf being unfa-ourabe.
The weather has awa"s been an im'ortant factor in 'eo'e>s i-es
V,. because of its effects on a as'ects of farming.
B. for it has considerabe infuence o-er farming.
C. since farmers concern themse-es with it.
6. as weather constitutes the dominant worr" for farmers.
E. on account of its affecting farming affairs.
/O 0 Correct answer1
#egister
The use of the correct register denotes the abiit" to write for a s'ecific
'ur'ose with a s'ecific audience in mind. Confusion and embarrassment
resut from the use of ina''ro'riate registers. +uch tests as the foowing.
howe-er. are not too difficut to construct and 'resent the students with an
interesting task. 'ro-ided that the e)tract used is written in a fair"
distincti-e st"e.
T"'e 1 The foowing t"'e of 9ad-anced: register test reBuires the students to
identif" those words which are incongruous. re'acing each with a much more
suitabe word. The student is instructed to re'ace si)teen of the
words underined in the 'assage.
(t has now been made out
be"ond an" doubt whatsoe-er that the
nicotine
contained
in tobacco smoke is 'oisonous. 4ne minute dro' of
'ure nicotine
'unged
into the boodstream of a rat is sufficient to ki it.
(t has aso been 'ro-ed that the nicotine contained in tobacco smoke
sends u' the 'use rate and the bood 'ressure. There is aso strong
e-idence that the nicotine
content in fags is a 'rimar"
cause of oss of
weight and hungriness. (t is aso ike" that a few
hea-" smokers wi
ose contro of .heir heir finer musces and be unabe to 'a" around with
obJects with ease and 'recision. +uch a oss of musce acti-it" ma"
widen the e"es and
s'oi -ision. Aoreo-er. smoking 'uts back growth in
adoescents and
owers
athetic abiit".
Ho._-e-er. the most serious disease connected with smoking is
cancer of the ung! the direct connection between smoking and cancer
has
recent" been estabished so
assured" that cancer research fok and
'ubic heath authorities throughout the word ha-e begun intensi-e
cam'aigns against smoking. (n certain countries not on" are cigarette
ad-ertisements banished
from cinema and tee-ision screens but aso
makers are forced to 'rint on each 'acket a
warning concerning the
dangers of smoking.1<
T"'e < Aatching tests are we8suited to tests of register@ such tests can be
constructed both at word and sentence e-e.
9a: 7ord e-e! The students are instructed to match each word in List ,
with a word in List B. de'ending entire" on how forma or informa a
word is 8 not on its meaning8
13
(ist A (ist B Answers
1.
cr" a. boss
91e:
<. commence b. gee gee
9<c:
3. kid c. e)'ire
93a:
=. 'uss"
d. hos'itaise 9=b:
2. entrain e. draw
92d:
123
9b: +entence e-e! The students are instructed to 'ut the etter of the most
a''ro'riate sentence in List B with the number of each sentence in
List ,. The sentences ha-e been taken from instructions. ega
documents. scientific Engish. ad-ertisements. chidren>s comics and
news'a'ers.
(ist A
1. Buid the assemb" formers fat on the 'an and bend the
undercarriage down to the 'attern shown.
<. The Tenant sha kee' the interior of the 'remises in good order and
condition.
3. , bic"ce 'um' is a de-ice for mo-ing air agains
t a 'ressure
difference.
=. Because the Barceno has front whee dri-e. there>s
no 'ro' shaft. +o
"ou get big car roominess in on" thirteen feet.
2. But it>s too ateR The e-i 'an. cooked u' b" the monster Babo. has
ed Cato to Aadam Qena.
?. ,ce dri-er inJured in thriing race of "ear.
List B
a. There>s a new andmark for o-ers and others
at 7ateroo +tation.
The 6rum Bar and Buffet.
b. ,n obJect norma" becomes hot when it is 'aced in the
sun.
c. The mi)ture shoud be taken three times dai" after meas.
d. &ang fight death 8 "outh kied when 'ushed onto eectric ine.
e. ,n" amendment of this certificate or faiure to com'ete an" 'art of
it ma" render it in-aid.
f. &i-e o-er. (>m not a genius. The radio transmits a kind of buLL. ,
beam that can be 'icked u' for a cou'e of mies.
/.0 Controed writing There are se-era wa"s of controing students> freedom of e)'ression in
their written work and. as a conseBuence. increasing the reiabiit" of the
scoring. Howe-er usefu such methods are as teaching de-ices. the" wi
on" 'ro-e usefu for testing 'ur'oses if each student is com'ete" famiiar
with the 'articuar task to be 'erformed! hence the im'ortance of cear
instructions foowed b" at east one e)am'e. +ometimes there is e-en the
danger that certain students wi fee inhibited rather than he'ed b" such
contro. E)am'es of controed writing e)ercises are incuded in this
section.
T"'e 1 The students are gi-en a short reading e)tract and then reBuired to
write a simiar 'aragra'h. using the notes the" ha-e been gi-en. e.g.
,though dogs are on" animas. the" are -er" usefu and he' 'eo'e a
ot. ;or e)am'e. certain dogs he' farmers to ook after their shee'.
+ome dogs are used for hunting and others he' to rescue 'eo'e. E-en
now 'oice officers use dogs when the" are ooking for thie-es and
criminas. 5eo'e aso teach dogs to race. and dog racing is a s'ort
which man" 'eo'e ike. , dogs ike eating meat -er" much and ike
bones best of a.
,though 8 horses 8 animas. 8 usefu 8 a ot. ;or e)am'e. 8 horses 8
'eo'e 8 catte. +ome horses 8 hunting 8 'u things. (n the 'ast 8
sodiers 8 horses 8 fighting against the enem". 5eo'e 8 horses 8 horse
racing 8 s'ort 8 ike. , horses 8 ha" 8 oats.
12=
T"'e < The foowing item t"'e1= is set in a few wide"8used e)aminations
and can 'ro-e -er" usefu in controing writing once students are famiiar
with the con-entions obser-ed in the item. E-en the foowing rubric and
item ma" cause difficut" if a student has not 're-ious" been gi-en 'ractice
in com'eting such items. 4biBue strokes are used in the fragmented
sentences chief" in order to reinforce the im'ression that the sentences
ha-e been gi-en in note form.
*se the foowing notes to write com'ete sentences. 5a" carefu
attention to the -erbs underined and insert a missing words. The
obiBue ines 9K: are used to di-ide the notes into sections. 7ords ma" or
ma" not be missing in each of these sections. #ead the e)am'e
carefu" before "ou start.
E)am'e! 5arachute Jum' from aero'anes and baoonsKbe -er"
'o'uar s'or man" 'arts of word.
5arachute Jum'ing from aero'anes and baoons is a -er"
'o'uar s'ort in man" 'arts of the word.
&reatest heightKfrom which 'arachute Jum' e-er make be o-er 31.333
metres.K1/?3Kdoctor in *nited +tates ,ir ;orceK
Jum' from basket of
baoonKand
fa near" <?.333 metresKbefore
o'en 'arachute.K;a ast =
minutes and 30 seconds.Kand bod" reach s'eed /03 kiometres hour.KHe
and safe" in fiedK13 minutes and =2 secondsKafter he Jum'.K4n ste' of
basket of baoonKbe wor8>sK>This be highest ste' in word.
hen as if
he ike Jum' againKfrom such heightKdoctor
shake head.
T"'e 3 +e-era t"'es of writing tasks can be based on the foowing reading
e)tract.12 ,n" simiar te)t can aso be used for!
8 co'"ing with minor aterations! e.g. tenseK'erson changes
8 changing the 'oint of -iew! e.g.
)rite this stor# as seen "# .
changing the st"e and register! e.g.
)rite this stor# in the !orm o! a
newspaper report2a humorous account, etc.
adding further information.
, "oung man who refused to gi-e his name di-ed into the ri-er
"esterda" morning to sa-e a twe-e8"ear8od bo".
The bo". who ran awa" after being rescued. had been swimming in
the ri-er and had caught his foot between two concrete 'osts under the
bridge. He shouted out for he'.
,t the time. a "oung man was riding across the bridge on his
bic"ce. He Buick" dismounted and di-ed fu" cothed into the ri-er. He
then freed the bo">s foot8and he'ed him to the ri-er bank where a sma
crowd had coected. The bo" thanked his rescuer courteous" and
sincere". then ran off down the road. He was ast seen cimbing o-er a
gate before disa''earing o-er the to' of the hi.
The "oung man. who was about twent" "ears of age. said >( don>t
bame the bo" for not gi-ing his name. 7h" shoud heG (f he wants to
swim in the ri-er. that>s his business. ,nd if ( want to he' him. that>s
mine. %ou can>t ha-e m" name eitherR>
He then ran back to the bridge. mounted his bic"ce and rode awa".
Test 9i: #ewrite this stor" but imagine that "ou are actua" watching
e-er"thing that is ha''ening. Begin!
&here is a small "o#
swimming ...
122
Test 9ii: #ewrite this stor" as tod b" 9a: the "oung man who sa-ed the
bo" and 9b: the bo" who was sa-ed.
Test 9iii: 7rite this stor" as if "ou were gi-ing e-idence at a 'oice
station.
Test 9i-: (t was a sunn" da" but at the time of the rescue it began
to rain
hea-i". +e-era 'eo'e were 'assing nearb" on their wa" to a
footba match. 7hen the "oung un
thought that he had got
the rain. 7rite out the stor".
adding these facts.
T"'e = There are aso se-era methods of 'ractising or measuring the
abiit" to ink sentences. in-o-ing subordination and co8ordination
features. +ome tasks in
u'
9arge" determined8bN8) - onectr-es gi-en:. -en:. The foowing
is an e)am'e
of a controed writing ta
h e
skc'ractising subordination!
Join the short sentences in each of the grou's beow to form one
sentence. Then write each of the finished sentences so as to form a
'aragra'h. *se the Joining words gi-en. but note that sometimes no
Joining word is necessar"@ aso
$ing denotes the -erb ending on".before Each
4"m'ic &ames o'ens.
,n athete a''ears.
He hods a torch. 98ing:
(t has been carried from Aount 4"m'us in &reece. which
The ceremon" was started in Berin in 1/3?. which
(t inks the sites of the modern &ames with the first
4"m'ic &ames.
Howe-er. the actua torch ceremon" dates back to ,ncient
&reece. where
4ne of the most s'ectaci>>Har e-ents was the torch race. which
(t was awa"s run at night.
The athete entefs the stadium. 7hen
He is hoding the torch. who
He runs to the huge bow.
The sacred fame wi burn there. in which
Aan" such tests do not gi-e the reBuired inkers to the testees but
ea-e them free to Join the sentences in whiche-er wa" the" consider
a''ro'riate. (ndeed. since such tests are sti -er" subJecti-e and reBuire a
ot of time to score. it is often better not to 'ro-ide the testees with ink
ers
but to ea-e them free to so-e each 'robem in their own wa".
T"'e 29a: (n some tests of com'osition. es'ecia" at the eementar" and
intermediate e-es. sentences and causes are 'ro-ided at first in order to he'
the students to start writing. The" 7a" then reBuired to finish the incom'ete
sentences in an" a''ro'riate
#ead these sentences. ;inish each one and then com'ete the stor" in
"our own words.
4ne da" Hannah and Beck" got u' ear" to
go ...
The" caught a bus to the arge de'artm
ent store
where ...
12?
11
1
>Look. that>s 5ete +haw o-er there.> Beck" cried.
>Let>s .
The" shouted to 5ete but ...
>7h" doesn>t he ook at usG> Hannah asked. He>s beha-ing as
if .>
9b: This item is simiar to the 're-ious one. but here the testees are reBuired
to write a''ro'riate sentences father than causes. The foowing e)am'e
shows how the item t"'e can be used at the u''er8intermediate e-es.
1. +tudents who do not know a ot of Engish can take se-era ste's to
're'are for their stud" in a British uni-ersit". ;or e)am'e.
<. #ecent research shows that 'ubic o'inion is di-ided on the subJect of
s'ending mone" on defence. ,bout =3 'er cent of the countr"
beie-es we shoud increase such s'ending. 4n the other hand.
, we8seected series of such items can incude sentences eiciting an
abiit" to use e)em'ification. contrast. addition. cause. resut. 'ur'ose.
concusion and summar". ConseBuent". students can be tested on their
abiit" to use whate-er s'ecific functions and notions the test writer wishes.
T"'e ? 4ne of the most usefu de-ices for e)ercising contro o-er the kind
of written res'onse reBuired and "et. at the same time. gi-ing the testees
considerabe freedom of e)'ression is the two8sentence te)t designed to
measure the abiit" to form a coherent unit of anguage.12 ;or e)am'e.
testees ma" be instructed to write a sentence to
precede the statement!
Aoreo-er. it was im'ossibe to o'en the windows.
+am'e res'onses coud be!
(t was -er" hot in the sma room.
There was on" one fan in the room. but it was broken.
The door sammed behind John. and he reaised he was8ocked in the
room.
(n a cases. the students are reBuired to demonstrate an awareness of
the communicati-e nature of anguage in. genera and cohesi-e de-ices in
'articuar whie sti retaining a arge degree of freedom of res'onse.
4ther sam'e items are!
There was one outside the schoo entrance. too.
To do this. the water must first be boied.
These shoud then be carefu" sorted.
;or. widife. howe-er. there are e-en greater dangers in the 'oution
of ri-ers. akes and seas.
Howe-er. there is no reason to be 'essimistic.
127
11
Notes and references
1
The communicati-e nature of this item t"'e woud be great" reduced if
the first sentence were gi-en instead of the second sentence. The constraints
woud then be minima. the cohesi-e de-ices acking in ree-ance to a certain
degree. and the range of acce'tabe res'onses -er" wide indeed. ;or
e)am'e. after the sentence
There was a strange8ooking creature outside our door.
,n" of the foowing res'onses woud be acce'tabe!
( went u' to it and stroked it tender".
6o "ou ike itG
Ars Lee screamed.
The ne)t thing ( knew ( was "ing on m" back.
There were aso se-era cats and dogs.
,be8tee'hone8sudden"8rangt
(t was a hot da".
1 #o"a +ociet" of ,rts! &he Communicative 9se o! English as a 6oreign
(anguage
< *ni-ersit" of Cambridge Loca E)aminations +"ndicate! %reliminar# English
Test 9re-ised -ersion:
3 #o"a +ociet" of ,rts! &he Communicative 9se o! English as a
'econd
(anguage, Test of 7riting! ,d-anced Le-e. summer 1/0=
= Heaton J B 1/0? )riting through pictures. Longman
2 Joint Aatricuation Board! &est in English /<vers $as1, Aarch
1/03
? *ni-ersit" of Cambridge Loca E)aminations +"ndicate Cam"ridge
E4aminations in English 9Tabe 9a: 0 6irst Certificate in English Tabe
9b: O
Certi5cate o! %ro5cienc# in English1
7 Hartfie. ;a"e et. al. 1/02 E+L Com'osition %ro5le. (earning E'(
Composition. Newbur" House
0
Hughes. , and Lascaratou. C 1/0< Com'eting criteria for error gra-it". E(&
Journal 3*/31
/ Burt. A $ and $i'arsk" C
1/7< &he Boo5con. a repair manual !or English.
Newbur" House
Hendrickson. J 9ed.: 1/7/
Error ,na"sis and Error Correction in Language
Teaching. 7E(C <ccasional %apers ;o. 13
13 Tomi"ana. A 1/03
&rammatica errors and communication breakdown.
&E'<( Duarterl# 1=9:
11 Eis. #
1/0= Communication strategies and the e-auation of communicati-e
'erformance. E(&Journal3L/l1.
1<
made out 9'ro-ed:. 'unged 9inJected:. sends u' 9increases:. fags 9cigarettes:.
hungriness 9a''etite:. 'a" around with 9mani'uate:. widen 9e)tend:. s'oi
9im'air:. 'uts back 9retards:. owers 9reduces:. assured" 9concusi-e":. fok
9organisations:. begun 9aunched:. banished 9banned:. forced 9reBuired:.
makers 9manufacturers:
13
(n an e)ceent artice >+t"e and #egister Tests>. in <":e-tive &ests im
Englischunterricht der 'chule and 9niversitat, ,thenaum Eerag. #obert
5"nsent draws attention to the use of semantic and structura distractors in
matching
tests of register 8 i.e. incuding words with the same meaning 9or 'hrases with
identica structures: but in a different register.
1=
This t"'e of item has been used in the *ni-ersit" of Cambridge Loca
E)aminations +"ndicate! 6irst Certi5cate in English.
12 +amonte. , L 1/7?
TechniBues in Teaching 7riting. 7E(C Journal 191:
+harwood8+mith. 1/7? New 6irections in Teaching 7ritten Engish. 6orum
120 M(E9<:
Criteria and t"'es of tests
13.1 Eaidit" This section attem'ts to summarise much of what was contained in Cha'ter
1. Brief". the -aidit" of a test is the e)tent to which it measures what it is
su''osed to measure and nothing else.
E-er" test. whether it be a short.
informa cassroom test or a 'ubic e)amination. shoud be as -aid as the
constructor can make it. The test must aim to 'ro-ide a true measure of
the 'articuar ski which it is intended to measure! to the e)tent that it
measures e)terna knowedge and other skis at the same time. it wi not
be a -aid test. ;or e)am'e. the foowing test item is in-aid if we wish
soe" to measure writing abiit"! (s 'hotogra'h" an art or a scienceG
6iscuss.> (t is ike" to be in-aid sim'" because it demands some
knowedge of 'hotogra'h" and wi conseBuent" fa-our certain students.
+imiar". man" ora inter-iew tests run the risk of assessing
'ersonait" as we as ora 'roficienc". The atter case is an interesting one.
howe-er. as it can be strong" argued that success shoud be measured b"
concentrating on whether the task set has been achie-ed using whate-er
strategies the students ha-e at their dis'osa rather than sim'" on 'ure
inguistic abiit". (n other words. wh" shoud 'ersonait" and a -ariet" of
usefu non8-erba strategies be ignored in assessing ora 'erformance in a
anguageG The answer to this Buestion. howe-er. must awa"s de'end on
the 'ur'ose of the test! an achie-ement test or a cassroom 'rogress test
might we e)cude such factors as 'ersonait" whie a 'roficienc" test or
'ubic e)amination might consider such factors as im'ortant in e-auating
ora abiit" in the target anguage. Ne-ertheess. the Buestion ser-es to
iustrate a significant difference between the rather narrow
's"chometric8structura a''roach and the broader communicati-e
a''roach to testing.
;ace
validit# E-en a su'erficia ins'ection of the essa" item referred to in the first
'aragra'h in this section woud be sufficient to re-ea that it was not -aid.
This t"'e of -aidit". in fact. is often referred to as
!ace validit#. if a test
item ooks right to other testers. teachers. moderators. and testees. it can
be described as ha-ing at east face -aidit". (t is. therefore. often usefu to
show a test to coeagues and friends. ,s constructors of the test. we can
become so in-o-ed in the test that we sometimes fai to stand back and
ook at the indi-idua test items obJecti-e". 4n" if the test is e)amined b"
other 'eo'e can some of the absurdities and ambiguities then be
disco-ered.
12/
i

Language tests whicct ha-e been designed 'rimari" for one countr"
and are ado'ted b" another countr" ma" ack face -aidit". , -ocabuar"
or reading com'rehension test containing such words as >t"'hoon>.
>sam'an>. >abacus>. and >cho'sticks> wi ob-ious" not be -aid in East
,frica no matter how -aid and usefu a test it has 'ro-ed in Hong $ong.
The same argument a''ies to man" standardised tests designed for
immigrants in the *nited +tates and used ater in man" other 'arts of the
word.
,though no substitute for em'irica -aidit" in 'ubic e)aminations
and standardised tests. face -aidit" can 'ro-ide not on" a Buick and
reasonabe guide but aso a baance to too great a concern with statistica
ana"sis. Aoreo-er. the students> moti-ation is maintained if a test has
good face -aidit". for most students wi tr" harder if the test ooks sound.
(f. on the8!Tther hand..the.test a''ears to ha-e itte of ree-ance in the e"es
of the student. it wi cear" ack face -aidit". 5ossib" as a direct resut.
the student wi not 'ut ma)imum effort into 'erforming the tasks set in
the test! hence the reiabiit" of the test wi be affected.
(t is 'ossibe for a test to incude a the com'onents of a 'articuar
teaching 'rogramme being foowed and 8"et at the same time ack face
-aidit". ;or e)am'e. a reading test for engineers coud ha-e most of the
grammatica features of the anguage of engineering 9e.g. freBuent use of
the 'assi-e -oice. nomina strings. etc.: as we as most of the anguage
functions and notions associated with reading and writing engineering te)ts
9e.g. defining. cassif"ing. h"'othesising. drawing concusions. describing
'rocesses. e)'ressing notions of Buantit" and amount. etc.:. E-en so.
howe-er. the test wi ack face -aidit" if the subJect of the reading te)t
concerns. sa". 'ubic institutions in Britain.
The conce't of face -aidit" is far from new in anguage testing but the
em'8 3sis now 'aced on it is reati-e" new. (n the 'ast. face -aidit" was
regarded b" man" test writers sim'" as a 'ubic reations e)ercise. Toda".
howe-er. most designers of communicati-e tests regard face -aidit" as the
most im'ortant of a t"'es of test -aidit". (ndeed. man" argue that a test
must ook -aid e-en as far as the re'roduction of the materia itsef is
concerned! thus. a test of reading com'rehension using such authentic
tasks as reading and skimming news'a'ers must contain actua news'a'ers
8 or. at east. artices 'rinted in e)act" the same wa" as the" a''eared in
the news'a'er from which the" were taken.
Content validit# This kind of -aidit" de'ends on a carefu ana"sis of the anguage being
tested and of the 'articuar course obJecti-es. The test shoud he so
constructed as to contain a re'resentati-e sam'e of the course. the
reationshi' between the test items and the course obJecti-es awa"s being
a''arent. There is a strong tendenc". es'ecia" in muti'e8choice testing.
to test on" those areas of the anguage which end themse-es readi" to
tasting. Aan" tests of 'honoog". for instance. concentrated in the 'ast. on
testing 'honeme discrimination rather than the more im'ortant features of
stress and intonation! one cannot he' but sus'ect the reason for this was
sim'" that 'honeme tests were easier to construct than items testing stress
and intonation.
7hen embarking on the construction of a test. the test writer shoud
first draw u' a tabe of test s'ecifications. describing in -er" cear and
'recise terms the 'articuar anguage skis and areas to be incuded in the
test. (f the test or sub8test being constructed is a test of grammar. each of
the grammatica areas shoud then be gi-en a 'ercentage weighting 9e.g.
1?3
the future sim'e tense 13 'er cent. uncountabe nouns 12 'er cent. reati-e
'ronouns 13 'er cent. etc.: as touched u'on 're-ious". (f the test or sub8
test concerns reading. then each of the reading sub8skis shoud be gi-en a
weighting in a simiar wa" 9e.g. deducing word meanings from conte)tua
cues <3 'er cent. search8reading for s'ecific information 33 'er cent.
reading between the ines and inferring 1< 'er cent. intensi-e reading
com'rehension =3 'er cent. etc.:. (t scarce" matters what the tota
'ercentage is! the im'ortant 'oint is that the test writer has attem'ted to
Buantif" and baance the test com'onents. assigning a certain -aue to
indicate the im'ortance of each com'onent in reation to the other
com'onents in the test. (n this wa". the test shoud achie-e content -aidit"
and refect the com'onent skis and areas which the test writer wishes to
incude in the assessment.
Construct validit#
Empirical validit#
(f a test has construct validit#,
it is ca'abe of measuring certain s'ecific
characteristics in accordance with a theor" of anguage beha-iour and
earning. This t"'e of -aidit" assumes the e)istence of certain earning
theories or constructs under"ing the acBuisition of abiities and skis. ;or e)am'e. it can
be argued that a s'eed reading test based on a short
com'rehension 'assage is an inadeBuate measure of reading abiit" 9and
thus has ow construct -aidit": uness it is beie-ed that the s'eed reading of short 'assages
reates cose" to the abiit" to read a book Buick" and efficient" and is a 'ro-en factor in
reading abiit". (f the assum'tion is
hed that s"stematic anguage habits are best acBuired at the eementar"
e-e b" means of the structura a''roach. then a test which em'hasises the communicati-e
as'ects of the anguage wi ha-e ow construct -aidit".
Con-erse". if a communicati-e a''roach to anguage teaching and
earning has been ado'ted throughout a course. a test com'rising chief"
muti'e8choice items wi ack construct -aidit".
, fourth t"'e of -aidit" is usua" referred to as statistical or empirical
-aidit". This
-aidit" is obtained as a resut of com'aring the resuts of the test with the
resuts of some criterion measure such as!
8 an e)isting test. known or beie-ed to be -aid and
given at the same
time
or
8 the teacher>s ratings or an" other such form of inde'endent assessm ent
given at the same time or
8 the su"se,uent 'erformance of the testees on a certain task measured
b"
some -aid test@ or
8 the teacher>s ratings or an" other such form of inde'endent assessment
given later.
#esuts obtained b" either of the first two methods abo-e are
measures of the test>s concurrent validit#
in res'ect of the 'articuar
criterion used. The third and fourth methods estimate the predictive
validit#
of a test which is used to 'redict future success. 7e coud estimate
the 'redicti-e -aidit" of a anguage test administered to ci-i engineers
embarking on a ci-i engineering course gi-en in the medium of Engish. for
instance. if we measured their subseBuent 'erformances on their academic
courses and com'ared these resuts with the resuts of the anguage test gi-en
at the beginning of their course.
The test situation or the techniBue used is awa"s an im'ortant factor in
determining the o-era -aidit" of an" test. ,though an idea test
1?1
situation wi b" no means guarantee -aidit". a 'oor test situation wi
certain" detract from it. (s a istening com'rehension test -aid if the
students hear on" a disembodied -oice on. sa". a 'oor Buait" ta'e
recorderG
Aoreo-er. the fact that a new test com'ares fa-ourab" with another
supposedl# valid test wi not necessari" ensure that the new test is -aid.
'articuar" if the other test is not rea" a -aid measure. itsef. (n short. how
far can we trust the criteria we use for estabishing -aidit"G This is
one of the maJor criticisms of the whoe conce't of em'irica -aidit" made
b" the communicati-e schoo of test writers. The argument is sim'" that the
estabished criteria for measuring -aidit" are themse-es -er" sus'ect! two
in-aid tests do not make a -aid test.
13.< #eiabiit" #eiabiit" is a necessar" characteristic of an" good test! for it to be -aid at
a. a test must first. be reiabe as a measuring instrument. (f the test is
administered to the same candidates on different occasions 9with no
anguage 'ractice work taking 'ace between these occasions:. then. to the
e)tent that it 'roduces differing resuts. it is not reiabe. #eiabiit"
measured in this wa" is common" referred to as test2re$test relia"ilit#
to
distinguish it from
mar-2re$mar- relia"ilit# and the other kinds of
reiabiit" described ater in this section. This atter kind of reiabiit"
denotes the e)tent to which the same marks or grades are awarded if the
same test 'a'ers are marked b" 9i: two or more different e)aminers or 9ii: the
same e)aminer on different occasions. (n short. in order to be reiabe. a test
must be consistent in its measurements.
#eiabiit" is of 'rimar" im'ortance in the>use of both 'ubic
achie-ement and 'roficienc" tests and cassroom tests. Aethods of
estimating the reiabiit" of indi-idua items in a test wi be indicated in thc
ne)t cha'ter. Howe-er. an a''reciation of the -arious factors affecting
reiabiit" is im'ortant for the teacher at the -er" outset. since man"
teachers tend to regard tests as infaibe measuring instruments and fai to
reaise that e-en the best test is indeed a somewhat im'recise instrument with
which to measure anguage skis.
;actors affecting the reiabiit" of a test are!
8 the e)tent of the sam'e of materia seected for testing! whereas -aidit"
is concerned chief" with the content of the sam'e. reiabiit" is
concerned with the siLe. The arger the sam'e 9i.e. the more tasks the
testees ha-e to 'erform:. the greater the 'robabiit" that the test as a
whoe is reiabe 8 hence the fa-ouring of obJecti-e tests. which aow for
a wide fied to be co-ered.
8 the administration of the test! is the same test administered to different
grou's under different conditions or at different timesG Cear". this is an
im'ortant factor in deciding reiabiit". es'ecia" in tests of ora
'roduction and istening com'rehension.
The wa" in which this factor differs from test situation -aidit" can be
seen from the foowing e)am'e! if a recording for a istening
com'rehension test is initia" 'oor in Buait". then it is 'oor in Buait" for
a testees. This wi conseBuent" make for in-aidit" 9uness s'eech has
been deli"eratel# masked with noise. as a testing de-ice:. But if the Buait"
of the recording is good and if certain grou's hear it 'a"ed under good
acoustic conditions whie other grou's hear it under 'oor acoustic
conditions. this wi make for unreiabiit" and therefore in-aidit".
1?<
8 test instructions! are the -arious tasks e)'ected from the testees made
cear to all candidates in the rubricsG
8 'ersona factors such as moti-ation and iness.
8 scoring the test! one of the most im'ortant factors affecting reiabiit".
4bJecti-e tests o-ercome this 'robem of marker reiabiit". but
subJecti-e tests are sometimes faoed with it! hence the im'ortance of the
work carried out in the fieds of the muti'e8marking of com'ositions and
in the use of rating scaes.
Communicati-e anguage testing has recent" introduced another
dimension to the whoe conce't of reiabiit"! name". 'rofie re'orting. (n
order to obtain a fu 'rofie of a student>s abiit" in the target anguage. it
is necessar" to assess his or her 'erformance se'arate" for each of the
different areas of communication! e.g. istening com'rehension. s'eaking
and istening. reading. reading and writing 9summarising. etc.:. and
writing. ;urthermore. 'erformance is assessed according to the 'ur'ose for
which the anguage is to be used! e.g.> academic. occu'ationa. socia
sur-i-a. The obJect of the sub8tests through which 'erformance is assessed.
is to indicate the e)tent of the earner>s master" of the -arious anguage
skis which he or she wi reBuire for a 'articuar 'ur'ose. , score or
grade is gi-en for each of the skis or areas seected for testing. and an
a-erage mark is e-entua" obtained. This atter mark. howe-er. is on"
gi-en aongside the -arious scores which ha-e contributed to it. 5rofie
re'orting is thus -er" -auabe for 'acement 'ur'oses. and indeed it is an
essentia feature of one of the most wide" used 'roficienc" tests set in
Britain and administered in man" countries throughout the word.> ,
student>s 'erformance on the -arious 'arts of the test can be shown in the
form of a sim'e tabe or chart. in which the target score a''ears beside
the student>s score. (t is thus -er" eas" to com'are a student>s 'erformance
e-es in each area with the reBuired e-es.
4ne method of measuring the reiabiit" of a test is to re8administer
the same test after a a'se of time. (t is assumed that a candidates ha-e
been treated in the same wa" in the inter-a 8 that the" ha-e either a been
taught or that none of them ha-e. 5ro-ided that such assum'tions 9which
are freBuent" hard to Justif": can be made. com'arison of the two resuts
woud then show how reiabe the test has 'ro-ed. Cear". this method is
often im'racticabe and. in an" case. a freBuent use of it is not to be
recommended. since certain students wi benefit more than others b" a
famiiarit" with the t"'e and format of the test. Aoreo-er. in addition to
changes in 'erformance resuting from the memor" factor. 'ersona factors
such as moti-ation and differentia maturation wi aso account for
differences in the 'erformances of certain students.
,nother means of estimating the reiabiit" of a test is b"
administering 'arae forms of the test to the same grou'. This assumes
that two simiar -ersions of a 'articuar test can be constructed! such tests
must be identica in the nature of their sam'ing. difficut". ength. rubrics.
etc. 4n" after a fu statistica ana"sis of the tests and a the items
contained in them can the tests safe" be regarded as 'arae. (f the
.correation between the two tests is high 9i.e. if the resuts deri-ed from the
two tests corres'ond cose" to each other:. then the tests can be termed
reiabe.
The s'it8haf method is "et another means of measuring test
reiabiit". This method estimates a different kind of reiabiit" from that
1?3
11
estimated b" testKre8test 'rocedures. The s'it8haf method is based on the
'rinci'e that. if an accurate measuring instrument were broken into two
eBua 'arts. the measurements obtained with one 'art woud corres'ond
e)act" to those obtained with the other. The test is di-ided into two and
the corres'onding scores obtained. the e)tent to which the" correate with
each other go-erning the reiabiit" of the test as a whoe. 4ne 'rocedure
wide" used is to ascertain the correation between the scores on the odd
numbered items and those on the e-en numbered items. Howe-er. if the
items are graded according to increasing difficut". di-ision according to
odd and e-en numbers woud not be an accurate means of assessing
reiabiit".. since item < woud be sight" more difficut than item 1. item =
again more difficut than item 3. and so on. , more accurate 'rocedure is to
baance the items as foows!
item 1 = 2
0
/ 1<
against item <
3 ? 7 13 11
Howe-er. it woud be better. though ess con-enient. to aow chance to
decide which items go into one haf and which into the other.
The reiabiit" of the whoe test can be estimated b" using the formua!
O O
N 9 O m N8m :
ri i
N 8 ( Z 1
N)< K
where N 1 the number of items in the test@
m 1 the mean score on the test for a the testees 9see 'age
172:@
) 1 the standard de-iation of a the testees> scores 9see 'age
17?:. and
r1 1 reiabiit".
9Note that in this formua. ) has to be sBuared.:
(n +ections 11. 1 and 11.< the cacuation of the mean and standard
de-iation of scores on a anguage test containing =3 items is iustrated. The
mean is found to be <7 and the standard de-iation =.377. *sing these
i f gures with the abo-e formua. we obtain!
O O =3 9 8 <7 ) 13 Z
rI 3/ 1 =3 ) 1?.??<:
1 3.=0=
This formua is sim'e to use since 9i: it a-oids troubesome
correations and 9ii:. in addition to the number of items in the test. it
in-o-es on" the test mean and standard de-iation. both of which are
norma" cacuated an"how as a matter of routine.
;ina". it.shoud be noted that a test can be reiabe without
necessari" 'ossessing -aidit". Howe-er. reiabiit" is cear" inadeBuate b"
itsef if a test does not succeed in measuring what it is su''osed to
measure.
13.3
#eiabiit" -ersus ,s we ha-e seen. test -aidit" and reiabiit" constitute the two chief
-aidit"
criteria for e-auating an" test. whate-er the theoretica assum'tions
under"ing the test. The fundamenta 'robem. howe-er. ies in the confict
between reiabiit" and -aidit". The idea test shoud. of course. be both
reiabe and -aid. Howe-er. the greater the reiabiit" of a test. the ess
-aidit" it usua" has. Thus. the rea8ife tasks contained in such 'roducti-e
skis tests as the ora inter-iew. roe8'a". etter writing. etc. ma" ha-e
been gi-en high construct and face -aidit" at the e)'ense of reiabiit".
1 ?=
4bJecti-e tests are cear" not subJect to such a degree of unreiabiit".
But does this mean that a forms of subJecti-e testing shoud be
abandoned in fa-our of obJecti-e testingG ;or most 'ur'oses obJecti-e
tests are -ast" inferior to such more meaningfu and communicati-e tasks
as free8writing. roe 'a"ing and 'robem8so-ing. Language8earning
beha-iour cannot be demonstrated soe" b" means of an abiit" to seect
correct o'tions from gi-en aternati-es. Language use sim'" does not
function in this wa".
The choice facing the test writer is. therefore. whether to attem't to
increase the -aidit" of a test known to be reiabe or ese to increase the
reiabiit" of a test known to be -aid. (f the test writer tries to do the
former. he or she wi be faced with an im'ossibe task because the -er"
features which make the test reiabe 9usua" muti'e8choice and
com'etion items tested out of conte)t: are those -er" features which
render the test in-aid. ConseBuent". it is essentia to de-ise a -aid test
f i rst of a and then to estabish wa"s of increasing its reiabiit".
4ne effecti-e wa" of increasing test reiabiit" in such cases is b"
means of a carefu" drawn u' banding s"stem or rating scae. +uch a scae
9with a cear and concise descri'tion of the -arious characteristics of
'erformance at each e-e: enabes the marker to identif" 'recise" what.he
or she e)'ects for each band and then assign the most a''ro'riate grade to
the task being assessed. ;urthermore. markers are encouraged in this wa"
not on" to be consistent in their marking but aso to formuate Judgements
in Buaitati-e terms before ater con-erting such Judgements into
Buantitati-e assessments. ,n e)am'e of such a rating scae is gi-en in the
're-ious cha'ter! it is sufficient to em'hasise here that reiabiit" can be
increased b" means of 'rofie re'orting and Buaitati-e Judgements 9rather
than Buantitati-e ones:.
13.=
6iscrimination
+ometimes an im'ortant feature of a test is its ca'acit" to discriminate
among the different candidates and to refect the differences in the
'erformances of the indi-iduas in the grou'. ;or e)am'e. 73 'er cent
means nothing at a uness a the other scores obtained in the test are
known. ;urthermore. tests on which amost a the candidates score 73 'er
cent cear" fai to discriminate between the -arious students. Tests which
are designed for a arge test 'o'uation 9and which are to be standardised:
are first tried out on a re'resentati-e sam'e of students. This sma sam'e
mirrors the much arger grou' for whom the test is intended. The resuts of
the test are then e)amined to determine the e)tent to which it
discriminates between indi-iduas who are different. 7hen the fina -ersion
of the test is e-entua" used. therefore. its discriminator" 'owers ha-e
aread" been estabished.
ConseBuent". there wi then be itte need for
concern if. for e)am'e. it is found that the scores of indi-iduas in a grou'
custer around a centra 'oint. The test has been 'ro-ed ca'abe of
discriminating@ it does not do so in this case because there is nothing to
discriminate.
The e)tent of the need to discriminate wi -ar" de'ending on the
'ur'ose of the test! in man" cassroom tests. for e)am'e. the teacher wi
be much more concerned with finding out how we the students ha-e
mastered the s"abus and wi ho'e for a custer of marks around the 03
'er cent and /3 'er cent brackets. Ne-ertheess. there ma" be occurrences
in which the teacher ma" reBuire a test to discriminate to some degree in
order to assess reati-e abiities and ocate areas of difficut".
1?2
(
E-en the best test can ne-er be so 'recise as to determine the true
score of a testee to within < or 3 'er cent of a certain mark. This ack of
'recision is often referred to as the
margin o! error in a test. (t is im'ortant
to be aware of this gre". borderine area in test scores. es'ecia" when cut8
off 'oints 9i.e. 'assKfai e-es: are being fi)ed. (ronica". such cut8off
'oints are usua" fi)ed around a midde range of scores 9i.e. =3 'er cent or
23 'er cent:. affecting most of the students. (t is 'recise" in this range of
scores where the im'ortant decisions about 'ass or fai are usua" made.
There is a imit to the e)tent to which such scores can be refined in
order to enabe these 'assKfai decisions to be made with com'ete
certaint". (n other words. the gre" borderine area referred to in the
're-ious 'aragra'h wi awa"s remain. The situation can be im'ro-ed
consider .b" howe-erOiftheOscores themse-es can be s'read out o-er the
whoe range of the scae. (n this wa". fewer students wi be affected when
discriminations are made between scores on the critica 'art of the scae.
(n the foowing gra'h showing the usua 'attern of scores. the
e)aminer wi be uncertain about the arge number of students whose
scores fa within < 'er cent on either side of the 'assKfai score of 23 'er
cent.
<3
12
13
2
3
13 <3 33 =3 23 ?3 73
03
/3 133
(f the test can be constructed so as to discriminate as much as 'ossibe.
the scores wi then be s'read out o-er a far wider range as shown on the
foowing gra'h. (t wi sti be im'ossibe to be certain when distinguishing
between a student who has scored =/ 'er cent 9and thus faied: and a
student who has scored 21 'er cent 9and 'assed:. Howe-er. the number of
students in this borderine area wi now be much smaer.
The s'read of scores wi now a''ear in the gra'h ike this!
13 <3 33
=3
23 ?3 73 C3 /3
133
1 ??
How can test scores be s'read in this wa" and the test conseBuent"
make finer discriminations among the testeesG Brief". the items in the test
shoud be s'read o-er a wide difficut" e-e as foows!
8 e)treme" eas" items
8 -er" eas" items
8 eas" items
8 fair" eas" items
8 items beow a-erage difficut" e-e
8 items of a-erage difficut" e-e
8 items abo-e a-erage difficut" e-e
8 fair" difficut items
8 difficut items
8 -er" difficut items
8 e)treme" difficut items.
13.2 ,dministration , test must be 'racticabe! in other words. it must be fair" straight8
forward to administer. (t is on" too eas" to become so absorbed in the
actua construction of test items that the most ob-ious 'ractica
considerations concerning the test are o-erooked. The ength of time
a-aiabe for the administration of the test is freBuent" misJudged e-en b"
e)'erienced test writers. es'ecia" if the com'ete test consists of a
number of sub8tests. (n such cases sufficient time ma" not be aowed for
the administration of the test. the8coection of the answer sheets. the
reading of the test instructions. etc. (n the case of a arge8scae tests. the
time to be aowed shoud be decided on as a resut of a 'iot
administration of the test 9i.e. a tr"out of the test8 to a sma but
re'resentati-e grou' of testees:.
,nother 'ractica consideration concerns the answer sheets> and the
stationer" used. Aan" tests reBuire the testees to enter their answers on
the actua Buestion 'a'er 9e.g. circing the etter of the correct o'tion:.
thereb" unfortunate" reducing the s'eed of the scoring and 're-enting
the Buestion 'a'er from being used a second time. (n some tests. the
candidates are 'resented with a se'arate answer sheet. but too often
insufficient thought has been gi-en to 'ossibe errors arising from the
9menta: transfer of the answer from the conte)t of the item on the
Buestion 'a'er to the answer sheet itsef. Confusion ma" resut. for
e)am'e. if the items are numbered -ertica" on the Buestion 'a'ers and
horiLonta numbering is ado'ted for the corres'onding answer sheet!
1. %ou>d aread" eft8b" se-en o>cock...
. "ouG
,. didn>t
B. weren>t
C. hadn>t
6. ha-en>t
<. (f "ou take swimming essons. "ou ...
. soon.
,. wi be abe to swim
B. swim
C. can swim
6. sha ha-e swum
1?7
3. 6id an"one te Tim ...
,. off
B. o-er
C. on
6. about
. the careess error he madeG
5ut a cross 9M: in the bo) containing the etter of the correct answer.
1.
(,(B(C(6( <. (,(B(C(6( 3. 6
The use of se'arate answer sheets. howe-er. great" faciitates marking
9through the use of a mask or ke": and is strong" recommended when
arge numbers of students are being tested.
(t is of 'aramount im'ortance that e)aminers are fu" con-ersant with
the test situation. (f the test is to be administered b" se-era e)aminers
8working in different test8centres. cear directions s'ecif"ing e)act" what
each e)aminer shoud sa- and do shoud be issued in order to ensure that
e)act" the sarrie 'rocedure is foowed in each centre. ,though this
'rinci'e seems ob-ious enough. it is e)treme" difficut in man" cases for
test writers to see their own test from the 'oint of -iew of the 'eo'e
conducting the test. sim'" because the" are so cose" in-o-ed in their test
that the" are incined to take too man" things for granted. ConseBuent".
where-er 'ossibe. a test arrangements shoud be discussed in detai b"
the test writer and b" those conducting the test. the -arious ste's in the
administering of each sub8test being stated in sim'e anguage and cear"
numbered. This is 'articuar" essentia in tests of istening com'rehension
and ora 'roduction. where the administrator>s roe in the test is so
im'ortant. ,ccom'an"ing these instructions shoud be a cear statement of
aims together with a com'rehensi-e 9but sim'e: marking scheme.
Before beginning to construct a test. the test writer must make certain
that the necessar" eBui'ment wi be a-aiabe in each centre and that there
wi be a high degree of standardisation in the test administration Cear".
it is useess to record taks or diaogues on ta'e if certain test centres do
not ha-e a ta'e recorder. 7hat is not so ob-ious. howe-er. is the 'otentia
unreiabiit" of a istening test resuting from the different siLes of the
rooms where the test is administered and the different degrees of
interference caused b" e)traneous noise. The Buestion of 'racticabiit".
howe-er. is not confined soe" to auraEora tests! such written tests as
situationa com'osition and controed writing tests de'end not on" on the
a-aiabiit" of Buaified markers who can make -aid Judgements
concerning the use of anguage. etc.. but aso on the ength of time
a-aiabe for the scoring of the test.
, fina 'oint concerns the 'resentation of the test 'a'er itsef. 7here
'ossibe. it shoud be 'rinted or t"'ewritten and a''ear neat. tid" and
aesthetica" 'easing. Nothing is worse and more disconcerting to the
testee than an untid" test 'a'er. fu of miss'eings. omissions and
corrections.
13.? Test instructions
+ince most students taking an" test are working under certain menta
to the candidate 'ressures. it is essentia that a instructions are cear" written and that
e)am'es are gi-en. *ness a students are abe to foow the instructions.
the test wi be neither reiabe not -aid. &rammatica terminoog" shoud
be a-oided and such rubrics as the foowing rewtHritten!
1?0
5ut the correct 'ronoun in the banks.
Choose one of the foowing -erbs to go in each bank s'ace and 'ut it in
the correct tense.
+tudents ma" be abe to 'erform a the reBuired tasks without ha-ing an"
knowedge of forma grammar. (ndeed. since their knowedge of forma
grammar is not being tested. a reference to grammatica terms shoud be
a-oided. Thus. the first of the rubrics abo-e shoud be rewritten and the
'hrase Dwords ike the foowing> 9foowed b" e)am'es: be used to re'ace
D'ronouns>@ the second rubric shoud refer to Dwords> instead of -erbs. and
e)am'es shoud be gi-en so that students are shown the tense changes
the" are reBuired to make. This 'rinci'e does not a''" on" to
grammatica terms! if students are instructed to 'ut a tick o''osite the
correct answer. an e)am'e of what is meant b" the word Dtick> shoud be
gi-en 8 e.g. 9(K :. The same a''ies to crosses. circes. underining. etc.
+ometimes it is difficut to a-oid writing cums" rubrics or rubrics
consisting of com'e) sentences abo-e the difficut" e-e being tested. e.g.
,nswer each of the foowiig Buestions b" seecting the word or grou' of
words which best com'etes each sentence from the words or grou's of
words which are ettered ,. B. C. and 6.
;or each of the banks in thG foowing sentences choose one of the
words or grou's of words which best com'etes the sentence. 7rite in the
s'ace shown b" the dotted ine the etter corres'onding to the word. or
grou' of words. which best com'etes the sentence.
4ne 'ossibe soution is to write the rubric in short sentences 8 cear" and
concise"!
Choose the word or grou' of words which best com'etes each
sentence. Each is ettered ,. B. C. or 6. 5ut a circe round the etter of the
correct answer.
,nother soution is to use the students> first anguage when the test grou' is
monoingua. Howe-er. this 'rocedure is recommended on" at the
eementar" e-e where absoute" necessar".
The rubrics in too man" e)isting tests assume that the testees aread"
know what to do. 7hie this can be e)cused to a certain e)tent in cass
tests where the teacher is 'resent to e)'ain. it is -er" disturbing when it
occurs in more wide" used achie-ement and 'roficienc" tests where no
such he' is a-aiabe. Howe-er. it is often difficut to strike the right
baance between short. cear instructions and ong. in-o-ed rubrics. ,
rubric shoud ne-er. in itsef. become a test of reading com'rehension. The
foowing are further e)am'es of cear instructions!
Com'ete each of the foowing sentences. 7rite in the bank s'ace the
etter of the correct word or words. The first one is an e)am'e.
There are =3 Buestions in this section. Each Buestion consists of an
incom'ete sentence. This incom'ete sentence is foowed b" fi-e
'ossibe wa"s of com'eting it. Each wa" is abeed ,. B. C. 6. or E.
Choose the answer which "ou think best com'etes the sentence. The
first one is done for "ou.
Each word shoud be carefu" considered in the writing of a rubric. ;or
e)am'e. the word Dbest> is used in certain instances instead of Dcorrect>
because man" test items 9es'ecia" those in tests of -ocabuar": contain
1?/
se-era >correct> answers. athough on" one is rea" acce'tabe and cear"
the reBuired answer. ;ina". a rubrics shoud be tried out on 'iot grou's in
the same wa" in which items are tried out.
Because the writing of cear and concise rubrics is so difficut. it is
essentia that sim'e e)am'es are 'ro-ided for the testees. 4ne or two
e)am'es of the t"'e of. task set are recommended. uness the 'articuar
testing techniBue ado'ted is one with which testees are e)treme" famiiar.
(ndeed. if the testees are a unfamiiar with the t"'e of test being gi-en. it
is ad-isabe for the test su'er-isor to work through a few e)am'es with
them. (n one test of 'roficienc". the testees are gi-en fi-e >'ractice> items
to work through. their answers being subseBuent" checked before the test
is commenced. This sma 'recaution ensures that a testees are con-ersant
with the t"'e of test in which the" are about to 'artici'ate. (n certain other
tests@ a1'ractice>8testHisadministered beforehand to the testees. +uch a test
is s'ecia" constructed to incude e)am'es of a the t"'es of items in the
test 'a'er.
(f new testing techniBues are being used on a arge scae for the first
time. it is essentia for the test writers and test administrators concerned to
construct sam'e items and rubrics to send to schoos we in ad-ance.
together with sufficient detaied information about the new 'rocedures. No test
can 'ossib" be -aid if the techniBues ado'ted are so new and
unfamiiar as to bewider the testees.
13.7 Backwash effects
173
(n Cha'ter ( and throughout this book the im'ortance of the backwash
effects of testing on teaching has been em'hasised. (n Cha'ter 1 reference
was made to ora e)amining. where it was 'ointed out that. in s'ite of
'ossibe unreiabiit". ora tests shoud be continued as far as 'ossibe in
certain anguage earning situations if for no other reason than the
backwash effects the" ha-e on the teaching that takes 'ace before the test.
The 'ossibe conseBuences of man" reading com'rehension tests on the
de-eo'ment of the re ading skis were cited as another e)am'e of the
backwash effects of testing. Each eement and ski has been treated in the
book in reation to its 'otentia infuence on teaching. Ne-ertheess. the
im'ortance of the infuence of testing on teaching is worth em'hasising
again. since the test constructor can so easi" become too dee'" in-o-ed in
test statistics and in other test criteria.
, arger issue at stake is the effect of obJecti-e tests on anguage
earning in genera. (m'ortant Buestions are raised@ man" as "et remain
unanswered. ;or e)am'e. do obJecti-e tests freBuent" ead to a greater
em'hasis on accurac" than on fuenc"G (t is high" 'ossibe that. in s'ite of
a efforts. man" testing techniBues sti em'hasise the negati-e as'ects of
anguage earning. encouraging teachers and students to 'ace more
em'hasis on correctness than on the communicati-e as'ects of anguage
earning. , number of obJecti-e tests aso encourage the teaching of
anguage in artificia" constructed situations. thereb" reducing moti-ation
in anguage earning.
4ther issues are eBua" im'ortant in their im'ications. How much
infuence do certain tests e)ert on the com'iation of s"abuses and
anguage teaching 'rogrammesG How far is such an infuence harmfu or
actua" desirabe in certain situationsG ,gain. what 'art does coaching
'a" in the test situationG (s it 'ossibe to teach effecti-e" b" re"ing soe"
on some of the techniBues used for testingG Cear". the answers to these
and other Buestions remain in some doubt. , that can be done at 'resent is
to discourage as acti-e" as 'ossibe the use of testing techniBues as the chief
means of 'ractising certain skis. 7hie coaching undoubted" 'a"s a 'art in
increasing test scores. good teaching can do far more. Aoreo-er. 'ro-ided
that the students ha-e at east one o''ortunit" to 'artici'ate in a 'ractice test
'a'er before embarking on a 'articuar test. coaching b" itsef wi 'roduce
itte im'ro-ement in test resuts.
ConseBuent". whie we ma" de'ore. and must guard against. certain
backwash effects of testing on the one hand. it is fair to 'oint out on the
other hand that testing has been one of the greatest singe beneficia forces
in changing the direction of anguage teaching in man" areas and in
encouraging the more res'onsi-e teachers to e)amine not on" their own
teaching methods but aso the anguage the" are teaching.
13.0 T"'es of tests
There is some confusion regarding the terminoog" used to denote the
different t"'es of anguage tests in use. Aost test s'eciaists. howe-er.
agree on the foowing broad di-isions! achie-ementKattainment tests.
'roficienc" tests. a'titude tests and diagnostic tests.
AchievemenAattainment This grou' can be further subdi-ided into cass 'rogress tests and
tests
9standardised: achie-ement tests.
Cass 'rogress tests
This book has been concerned chief" with cass 'rogress tests. since these
are the most wide" used t"'es of tests. Aost teachers are. at some time or
other. reBuired to construct such tests. Each 'rogress test situation is
uniBue and can on" be e-auated fu" b" the cass teacher in the ight of
his or her knowedge of the students. the 'rogramme which the" ha-e been
foowing. and the cass teacher>s own 'articuar aims and goas. (t is
iogica to e)'ect genera 'ur'ose tests and books of tests to function as
effecti-e" as a test constructed s'ecia" for a 'articuar situation! hence
the 'ur'ose of this book in encouraging teachers to construct their own
tests.
The 'rogress test is designed to measure the e)tent to which the
students ha-e mastered the materia taught in the cassroom. (t is based on
the anguage 'rogramme which the cass has been foowing and is Just as
im'ortant as an assessment of the teacher>s own work as the student>s own
earning. #esuts obtained from the 'rogress test enabe the teacher to
become more famiiar with the work of each student and with the 'rogress
of the cass in genera. The cass 'rogress test is a teaching de-ice. its
backwash effect onteaching8and moti-ation being im'ortant features. ,
good 'rogress test shoud encourage the students.to 'erform we in the
target anguage and to gain additiona confidence. (ts aim is to stimuate
earning and to reinforce what has been taught. &ood 'erformances act as
a means of encouraging the students. and athough 'oor 'erformances ma"
act as an incenti-e to more work. the 'rogress test is chief" concerned with
aowing the students to show what the" ha-e mastered. +cores on it shoud
thus be high 9'ro-ided. of course. that 'rogress has indeed been made:.
7hereas in standardised achie-ement and 'roficienc" tests. a wide range of
'erformance shoud be indicated. the 'rogress test shoud show a custer of
scores around the to' of the scae. Com'are the foowing gra'h with the
two gra'hs in 13.=.
171
1
i
<3
12
13
2
13 <3 33 =3 23 ?3 73 03 /3 133
,chie-ement tests
,chie-ement 9or attainment: tests.. though simiar in a number of wa"s to
'rogress tests. are far more forma tests and are intended to measure
achie-ement on a arger scae. Aost annua schoo e)aminations take the
form of achie-ement tests@ a 'ubic tests which are intended to show
master" of a 'articuar s"abus are aso achie-ement tests. These tests are
based on what the students are 'resumed to ha-e earnt 8 not necessari"
on what the" ha-e actua" earnt nor on what has actua" been taught.
,chie-ement tests freBuent" take the form of secondar" schoo entrance
tests and schoo certificate e)aminations@ man" are based on a 'ubished
s"abus and e)ert a strong infuence on the teaching in schoos.
Constructors of such tests rare" teach an" of the students being tested
9often an ad-antage provided that the test constructors are -er" famiiar
with the teaching and earning 'robems of the testees:. (ndeed. this is
often a 'rereBuisite before an"one can be a''ointed to an" 'osition of
res'onsibiit" in connection with this t"'e of test. though this 'rinci'e
ob-ious" cannot awa"s be a''ied to schoo e)aminations.
+e-era aciiie-ement tests are standardised! the" are 're8tested. each
item is ana"sed and re-ised where necessar". norms are estabished and
com'arisons made between 'erformances of different students and
different schoos. +ince such tests are administered "ear after "ear. it is
'ossibe to com'are 'erformances of students one "ear with those of
students taking the test another "ear.
(f the students ha-e foowed a structura a''roach to anguage
earning. it is cear" unfair to administer a communicati-e achie-ement
test at the end of their course. (t is eBua" unfair to administer a
structura8based>test to those students who ha-e foowed a communicati-e
a''roach to earning the target anguage. , good achie-ement test shoud
refect the 'articuar a''roach to earning and teaching that has 're-ious"
been ado'ted.
ro5cienc# tests
7hereas an achie-ement test ooks back on what shoud ha-e been earnt.
the 'roficienc" test ooks forward. defining a student>s anguage 'roficienc"
with reference to a 'articuar task which he or she wi be reBuired to
'erform. 5roficienc" tests are in no wa" reated to an" s"abus or teaching
'rogramme@ indeed. man" 'roficienc" tests are intended for students from
se-era different schoos. countries and e-en anguage backgrounds. The
'roficienc" test is concerned sim'" with measuring a student>s888 . contro of
the anguage in the ight of what he or she wi be e)'ected to do with it in
the future 'erformance of a 'articuar task. 6oes the student know enough
Engish. for e)am'e. to foow a certain uni-ersit" or coege course gi-en
in the medium of EngishG 6oes the student know enough Engish in order
to function efficient" in a 'articuar t"'e of em'o"mentG The 'roficienc"
test is thus concerned with measuring not genera attainment but s'ecific
skis in the ight of the anguage demands made ater on the student b" a
future course of stud" or Job.
Aptitude tests
, anguage a'titude test 9or 'rognostic test: is designed to measure the
student>s pro"a"le 'erformance in a foreign anguage which he or she has
not started to earn! i.e. it assesses a'titude for earning a anguage.
Language earning a'titude is a com'e) matter. consisting of such factors
as inteigence. age. moti-ation. memor". 'honoogica sensiti-it" and
sensiti-it" to grammatica 'atterning. The reati-e weighting gi-en to these
eements must de'end on man" factors and thus -ar" considerab" from
one indi-idua to another. +ome s'eciaists in this fied maintain that it is
neither 'ossibe nor desirabe to take an o-era measurement of anguage
a'titude@ conseBuent" a'titude is sometimes di-ided into -arious as'ects
according to the s'ecific tasks for which a 'erson is being trained! e.g.
istening. inter'reting. transating. ,'titude tests genera" seek to 'redict
the student>s 'robabe strengths and weaknesses in earning a foreign
anguage b" measuring 'erformance in an artificia anguage. The abiit" to
earn new 'honemic distinctions and aso to use anguage 'atterns in an
unfamiiar but s"stematic manner is tested b" means of the artificia
anguage. +ince few teachers are concerned with the com'e) fied of
a'titude testing. it is not necessar" to go into further detai here.
Giagnostic tests ,though the term diagnostic test
is wide" used. few tests are constructed
soe" as diagnostic tests. ,chie-ement and 'roficienc" tests@8howe-er. are
freBuent" used for diagnostic 'ur'oses! areas of difficut" are diagnosed in
such tests so that a''ro'riate remedia action can be taken ater. +ections
of tests which end themse-es 'articuar" we to diagnostic 'ur'oses are
'honeme discrimination8 tests. grammar and usage tests. and certain
controed writing tests. Cear". weaknesses indicated in a test of
-ocabuar" are not high" significant in themse-es and can on" be
regarded as indicating genera weaknesses. +imiar". man" e)isting tests of
reading com'rehension are not -er" suitabe for diagnostic 'ur'oses. Tests
of writing and ora 'roduction can be used diagnostica" 'ro-ided that
there is an a''reciation of the imits to which such tests can be 'ut. +ince
diagnosing strengths and weaknesses is such an im'ortant feature of
'rogress tests and of teaching. the teacher shoud awa"s be aert to e-er"
facet of achie-ement re-eaed in a cass 'rogress test.
Note that diagnostic testing is freBuent" carried out for grou's of
students rather than for indi-iduas. (f on" one or two students make a 'articuar error.
the teacher wi not 'a" too much attention. Howe-er. if se-era students in the grou'
make a certain error. the teacher wi note the error and 'an a''ro'riate remedia
teaching.
Notes and references
( The Engish Language Testing +er-ice 9ELT+:. The British Counci
173
((
(nter'reting test scores
11.1
;reBuenc" Aarks awarded b" counting the number of correct answers on a test scri't
distribution are known as raw marks. D12 marks out of a tota of <3> ma" a''ear a high
mark to some. but in fact the statement is -irtua" meaningess on its own.
;or e)am'e. the tasks set in the test ma" ha-e been e)treme" sim'e and
12 ma" be the owest mark in a 'articuar grou' of scores.
T,BLE 1 T,BLE < T,BLE 3
TesteeAarkTesteeAark#ank
AarkTa";reBuenc"
, <3 6
32
1 =3
B <2 A 3= < 3/
C 33 C 33
3.2 9or
31: 30
6 32 7 33 3.2 9or 31:
37
E </ L 3< 2 3?
; <2 & 33 ?.2 9or ?1: 32 K 1
& 33 + 33 ?.2 9or ?1: 3= K
1
H <? E </ 0.2 9or
01: 33
<
1 1/ 5 </ 0.2 9or 01: 3< K
1
J <7
J <7 11 9or 131:
31
$ <? N <7 11
9or 131:
33
<
L
3<
3
<7 11 9or 131: </
<
A
3=
H <? 12 9or 131: <0
N <7
$
<?
12 9or 131: <7
3
3 <7 T <? 12
9or 131:
<? 2
5 </ M
<? 12
9or 131:
<2
3
S <2 Q <? 12 9or 131: <=
K 1
# <3
B
<2
1/
9or 101:
<3
<
+ 33 ; <2 1/ 9or 101: << K 1
T <?
S <2 1/ 9or 101: <1
* << % <= <1 <3 K
1
E
<3
# <3 <<.2 9or <<1: 1/ K 1
7 33
E
<3 <<.2 9or <<1: 10
M <?
* << <= 17
% <= ,
<3 <2
1?
Q
<? 1 1/ <? 12 Tota <?
17=
Con-erse". the test ma" ha-e been e)treme" difficut. in which case 12
ma" we be a -er" high mark. Numbers sti e)ert a strange and 'owerfu
infuence on our societ". but the shibboeth that =3 'er cent shoud awa"s
re'resent a 'ass mark is ne-ertheess both sur'rising and disturbing.
The tabes on the 're-ious 'age contain the imaginar" scores of a
grou' of <? students on a 'articuar test consisting of =3 items. Tabe 1
con-e"s -er" itte. but Tabe <. containing the students> scores in order of
merit. shows a itte more. Tabe 3 contains a freBuenc" distribution
showing the number of students who obtained each mark awarded@ the
strokes on the eft of the numbers 9e.g. KKKK: are caed
tallies and are
incuded sim'" to iustrate the method of counting the freBuenc" of
scores. Note that norma" the freBuenc" ist woud ha-e been com'ied
without the need for Tabes 1 and <@ conseBuent". as the range of highest
and owest marks woud then not be known. a the possi"le scores woud
be isted and a record made of the number of students obtaining each score
in the scae 9as shown in the e)am'e:.
Note that where ties occur in Tabe <. two wa"s of rendering these are
shown. The usua cassroom 'ractice is that shown in the 'arentheses.
7here statistica work is to be done on the ranks. it is essentia to record
the a-erage rank 9e.g. testees J. N and 3. each with the same mark.
occu'" 'aces 13. 11 and 1< in the ist. a-eraging 11:.
The foowing freBuenc" 'o"gon iustrates the distribution of the
scores!
2
=
38(
<7
11.< Aeasures of
centra tendenc"
Mode
Median
Mean
1/ <3 <1 << <3 <= <2 <? <7 <0 </ 33 31 3< 33 3= 32 3?
)
The mode refers to the score which most candidates obtained! in this case it
is <?. as fi-e testees ha-e scored this mark.
&he median refers to the score gained b" the midde candidate in the order
of merit! in the case of the <? students here 9as in a cases in-o-ing e-en
numbers of testees:. there can ob-ious" be no midde 'erson and thus the
score hafwa" between the owest score in the to' haf and the highest
score in the bottom haf is taken as the median. The median8score in this
case is aso <?.
The mean score of an" test is the arithmetica a-erage! i.e. the sum of the
se'arate scores di-ided b" the tota number of testees. The mode. median.
and mean are a measures of centra tendenc". The mean is the most
efficient measure of centra tendenc". but it is not awa"s a''ro'riate.
(n the foowing Tabe = and formua. note that the s"mbo ) is used
to denote the score. N the number of the testees. and m the mean. The
172
((
s"mbo f denotes the freBuenc" with which a score occurs. The s"mbo E
means the sum o!.
T,BLE =
) f f)
32)1 32
3= ) 1 3=
33 ) < ??
3< ) 1 3<
33 ) < ?3
</ ) < 20
<7)3 01 m O Mf) 1 73< 1
<?)2 133 N <?
<2)3 72
<= ) 1 <=
<3)< =?
<< ) 1 <<
<3 ) 1 <3
1/ ) 1 1/
Tota 1 73<
1 Ef)
<7
Note that ) 1 73< is the tota number of items which the grou' of <?
students got right between them. 6i-iding b" N 1 <? 9as the formua
states:. this ob-ious" gi-es the a-erage.
(t wi be obser-ed that in this 'articuar case there is a fair" cose
corres'ondence between the mean 9<7: and the median 9<?:. +uch a cose
corres'ondence is not awa"s common and has occurred in this case
because the scores tend to custer s"mmetrica" around a centra 'oint.
11.3 Aeasures of 7hereas the 're-ious section was concerned with measures of centra
dis'ersion
tendenc". this section is reated to the range or s'read of scores. The
mean b" itsef enabes us to describe an indi-idua student>s score b"
com'aring it with the a-erage set of scores obtained b" a grou'. but it tes
us nothing at a about the highest and owest scores and the s'read of
marks.
7ange 4ne sim'e wa" of measuring the s'read of marks is based on the
difference between the highest and owest scores. Thus. if the highest score on
a 238item test is =3 and the owest <1. the range is from <1 to =3! i.e. <<. (f the
highest score. howe-er. is on" 3/ and the owest </. the range is 13. 9Note
that in both cases. the mean ma" be 3<.: The range of the <? scores gi-en in
+ection 11.1 is! 32 8 1/ 1 1?.
'tandard deviation The standard de-iation 9s.d.: is another wa" of showing the s'read of
scores. (t measures the degree to which the grou' of scores de-iates from
the mean@ in other words. it shows how a the scores are s'read out and
thus gi-es a fuer descri'tion of test scores than the range. which sim'"
describes the ga' between the highest and owest marks and ignores the
information 'ro-ided b" a the remaining scores. ,bbre-iations used for
the standard de-iation are either s.d. or o 9the &reek etter sigma: or s.
17?
4ne sim'e method of cacuating s.d. is shown beow!
s.d. 1
1d<
N
N is the number of scores and d the de-iation of each score from the mean.
Thus. working from the <? 're-ious resuts. we 'roceed to!
1. i fnd out the amount b" which each score de-iates from the mean 9d:@
<. sBuare each resut 9d<:@
3. tota a the resuts 9<d<:@
=. di-ide the tota b" the number of testees 9Ed<KN:@ and
2. find the sBuare root of this resut 9EEd<KN:.
+core Aean 6e-iation 9d:+Buared 9d<:
9+te' 1: 32 de-iates from <7 b"
0
9+te' <:
?=
3=
7 =/
33 ? 3?
33 ?
3?
3< 2 <2
33 3
/
33
3 /
</ < =
</
< =
<7 3 3
<7
3 3
<7 3 3
<? 81
1
<? 81
1
<? 81 1
<? 81 1
<? 81 1
<2 8<
=
<2 8< =
<2
8< =
<= 83
/
<3 8= 1?
<3 8= 1?
<< 82
<2
<3 87 =/
1/ 80 ?=
73<
9+te' 3:Tota
1 =3<
9+te' =:s.d.
O E =3?<
9+te' 2:s.d. 1 ZK1?.?<
s.d. 1 =.377
=.30
Note! (f de-iations 9d: are taken from the mean. their sum 9taking account of
the minus sign: is Lero _ =< 8 =< 1 3. This affords a usefu check on the
cacuations in-o-ed here.
, standard de-iation of =.30. for e)am'e. shows a smaer s'read of
scores than. sa". a standard de-iation of 0./?. (f the aim of the test is
sim'" to determine which students ha-e mastered a 'articuar 'rogramme
177
of work or are ca'abe of carr"ing out certain tasks in the target anguage. a
standard de-iation of =.30 or an" other denoting a fair" narrow s'read wi be
Buite satisfactor" 'ro-ided it is associated with a high a-erage score. Howe-er.
if the test aims at measuring se-era e-es of attainment and
making fine distinctions within the grou' 9as 'erha's in a 'roficienc" test:.
then a broad s'read wi be reBuired.
+tandard de-iation is aso usefu for 'ro-iding information concerning
characteristics of different grou's. (f. for e)am'e. the standard de-iation
on a certain test is =.30 for one cass. but 0./? on the same test for another
cass. then it can be inferred that the atter cass is far more heterogeneous
than the former.
11.=
hem ana"sis Earier carefu consideration of obJecti-es and the com'iation of a tabe of
test s'ecifications were urged before the construction of an" test was
attem'ted. 7hat is reBuired now is a knowedge of how far those
obJecti-es ha-e been achie-ed b" a 'articuar test. *nfortunate". too
man" teachers think that the test is finished once the raw marks ha-e been
obtained. But this is far from the case. for the resuts obtained from
obJecti-e tests can be used to 'ro-ide -auabe information concerning!
8 the 'erformance of the students as a grou'. thus 9in the case of cass
'rogress tests: informing the teacher about the effecti-eness of the
teaching@
8 the 'erformance of indi-idua students@ and
8 the 'erformance of each of the items com'rising the test.
(nformation concerning the 'erformance of the students as a whoe
and of indi-idua students is -er" im'ortant for teaching 'ur'oses.
es'ecia" as man" test resuts can show not on" the t"'es of errors most
freBuent" made but aso the actua reasons for the errors being made. ,s
shown in earier cha'ters. the great merit of obJecti-e tests arises from the
faci that the" can 'ro-ide an insight into the menta 'rocesses of the
students b" showing -er" cear" what choices ha-e been made. thereb"
indicating definite ines on which remedia work can be gi-en.
The 'erformance of the test items. themse-es. is of ob-ious
im'ortance in com'iing future tests. +ince a great dea of time and effort
are usua" s'ent on the construction of good obJecti-e items. most
teachers and test constructors wi be desirous of either using them again
without further changes or ese ada'ting them for future use. (t is thus
usefu to identif" those items which were answered correct" b" the more
abe students taking the test and bad" b" the ess abe students. The
identification. of certain difficut items in the test. together with a
knowedge of the 'erformance of the indi-idua distractors in muti'e8
choice items. can 'ro-e Just as -auabe in its im'ications for teaching as
for testing.
, items shoud be e)amined from the 'oint of -iew of 91: their
difficut" e-e and 9<: their e-e of discrimination.
Item diPcult#
The inde4 o! diPcult# 9or !acilit# value1 of an item sim'" shows how eas"
or difficut the 'articuar item 'ro-ed in the test. The inde) of difficut"
9;E: is genera" e)'ressed as the fraction 9or 'ercentage: of the students
who answered the item correct". (t is cacuated b" using the formua! 88
;E1
N
17/
# re'resents the number of correct answers and N the number of students
taking the test. Thus. if< out of <? students tested obtained the correct
answer for one of the items. that item woud ha-e an inde) of difficut" 9or a
faciit" -aue: of .77 or 77 'er cent.
;E1
<3 <? 1 .77
(n this case. the 'articuar item is a fair" eas" one since 77 'er cent of the
students taking the test answered it correct". ,though an a-erage faciit"
-aue of .2 or 23 'er cent ma" be desirabe for man" 'ubic achie-ement
tests and for a few 'rogress tests 9de'ending on the 'ur'ose for which one
is testing:. the faciit" -aue of a arge number of indi-idua items wi -ar"
considerab". 7hie aiming for test items with faciit" -aues faing
between .= and .?. man" test constructors ma" be 're'ared in 'ractice to
acce't items with faciit" -aues between .3 and .7. Cear". howe-er. a
-er" eas" item. on which /3 'er cent of the testees obtain the correct
answer. wi not distinguish between abo-e8a-erage students and beow8
a-erage students as we as an item which on" ?3 'er cent of the testees
answer correct". 4n the other hand. the eas" item wi discriminate
amongst a grou' of beow8a-erage students@ in other words. one student
with a ow standard ma" show that he or she is better than another student
with a ow standard through being gi-en the o''ortunit" to answer an eas"
item. +imiar". a -er" difficut item. though faiing to discriminate among
most students. wi certain" se'arate the good student from the -er" good
student.
, further argument for incuding items co-ering a range of difficut"
e-es is that 'ro-ided b" moti-ation. 7hie the incusion of difficut items
ma" be necessar" in order to moti-ate the good student. the incusion of -er"
eas" items wi encourage and moti-ate the 'oor student. (n an" case. a few
eas" items can 'ro-ide a >ead8in> for the student 8 a de-ice which ma" be
necessar" if the test is at a new or unfamiiar or if there are
certain tensions surrounding the test situation.
Note that it is 'ossibe for a test consisting of items each with a faciit"
-aue of a''ro)imate" .2 to fai to discriminate at a between the good
and the 'oor students. (f. for e)am'e. haf the items are answered
correct" b" the good students and incorrect" b" the 'oor students whie
the remaining items are answered incorrect" b" the good students but
correct" b" the 'oor students. then the items wi work against one
another and no discrimination wi be 'ossibe. The chances of such an
e)treme situation occurring are -er" remote indeed@ it is high" 'robabe.
howe-er. that at east one or two items in a test wi work against one
another in this wa".
Item discrimination The discrimination inde) of an item indicates the e)tent to which the item
discriminates between the testees. se'arating the more abe testees from
the ess abe. The inde) of discrimination 96: tes us whether those
students who 'erformed we on the whoe test tended to do we or bad"
on each item in the test. (t is 'resu''osed that the tota score on the test is
a -aid measure of the student>s abiit" 9i.e the good student tends to do
we on the test as a whoe and the 'oor student bad":. 4n this basis. the
score on the whoe test is acce'ted as the criterion measure. and it thus
becomes 'ossibe to se'arate the >good> students from the >bad> ones in
'erformances on indi-idua items. (f the >good> students tend to do we on
17/
11
1
an item 9as shown b" man" of them doing so 8 a freBuenc" measure: and
the D'oor> students bad" on the same item. then the item is a good one
because it distinguishes the Dgood from the Dbad> in the same wa" as the
tota test score. This is the argument under"ing the inde) of
discrimination.
There are -arious methods of obtaining the inde) of discrimination! a
in-o-e a com'arison of those students who 'erformed we on the whoe
test and those who 'erformed 'oor" on the whoe test. Howe-er. whie it
is statistica" most efficient to com'are the to' <7< 'er cent with the
bottom <73 'er cent. it is enough for most 'ur'oses to di-ide sma sam'es
9e.g. cass scores on a 'rogress test: into ha-es or thirds. ;or most
cassroom 'ur'oses. the foowing 'rocedure is recommended.
1
,rrange the scri'ts in rank order of tota score and di-ide into two
grou's of eBua siLe 9i.e. the to' haf and the bottom haf:. (f there is an
odd number of scri'ts. dis'eosc with one scri't chosen at random.
< Count the number of those candidates in the u''er grou' answering the
f i rst item correct"@ then count the number of ower8grou' candidates
answering the item correct".
3 +ubtrwt the number of correct answers in the ower grou' from the
number of correct answers in the u''er grou'! i.e. find the difference in
the 'ro'ortion 'assing in the u''er grou' and the 'ro'ortion 'assing in
the ower grou'.
= 6i-ide this difference b" the tota number of candidates in one grou'!
6 1 Correct * 8 Correct L
n
96 1 6iscrimination inde)@ n 1 Number of candidates in one grou'V!
* 1 *''er haf and L 1 Lower haf. The inde) 6 is thus the difference
between the 'ro'ortion 'assing the item in * and L.:
2 5roceed in this manner for each item.
The foowing item. which was taken from a test administered to =3
students. 'roduced the resuts shown!
( eft Tok"o ... ;rida" morning.
,. in
s 9
on C. at 6. b"
61
128? /
8 2
<3 1
+uch an item with a discrimination inde) of .=2 functions fair" effecti-e".
athough cear" it does not discriminate as we as an item with an inde) of
.? or .7. 6iscrimination indices can range from _ ( 91 an item which
discriminates 'erfect" 8 i.e. it shows 'erfect correlation with the testees>
resuts on the whoe test: through 3 91 an item which does not discriminate
in an" wa" at a: to 81 91 an item which discriminates in entire" the
wrong wa":. Thus. for e)am'e. if a <3 students in the u''er grou'
answered a certain item correct" and a <3 students in the ower grou' got
the wrong answer. the item woud ha-e an inde) of discrimination of 1.3.
>The reader shoud carefu distineuish between n91 the nwnber of candidates in
either the * or L grou': and N 91 the number in the whoe grou': as used
're-ious". 4h-ious n 1 1.K<
N.
(f. on the other hand. on" 13 students in the u''er grou' answered it
correct" and furthermore 13 students in the ower grou' aso got correct
answers. the discrt inaton inde) woud be 3. Howe-er. if none of the <3
students in the u' 'er grou' got a correct answer and a the <3 students in
the ower grou' answered it correct". the item woud ha-e a negati-e
discrimination. shown b" 81.3. (t is high" inad-isabe to use again. or
e-en to attem't to amend. an" item showing negati-e discrimination.
(ns'ection of such an item usua" shows something radica" wrong with it.
,gain. working from actua test resuts. we sha now ook at the
'erformance of three items. The first of the foowing items has a high
inde) of discrimination@ the second is a 'oor item with a ow discrimination
inde)@ and the third e)am'e is gi-en as an iustration of a 'oor item with
negati-e discrimination.
1 High discrimination inde)!
NE,#L% 7hen ... . Jim .
. crossed ... . the
road. he D... ran into a car.
6 1
10 8 3 1 12 1 72 ;E 1
<11 3.2<2
<3
<3 =3
9The item is at the right e-e of difficut" and discriminates we.:
< Low discrimination inde)!
(f "ou .. . the be. the door woud ha-e been o'ened.
,. woud ring
C. woud ha-e rungOOO.OOO .
O
had rung
6. were ringing
;E
372
613<331.12 =3
9(ii this case. the item discriminates 'oor" because it is too difficut
for e-er"one. both Dgood> and Dbad>.:
3 Negati-e discrimination inde)!
( don>t think an"bod" has seen him.
,. %es. someone has.
B. . %es. no one has.
C. %es. none has.
6. %es. an"one has.
6 1
= ?
<3 8<3 18H13 ;E1 =3 13.<2
9This item is too difficut and discriminates in the wrong Lircction.:
7hat has gone wrong with the third item abo-eG E-en at this stage and
without counting the number of candidates who chose each of the o'tions.
it is e-ident that the item was a trick item! in other words. the item was far
too >ce-er>. (t is e-en concei-abe that man" nati-e s'eakers woud seect
o'tion B in 'reference to the correct o'tion ,. (tems ike this a too often
esca'e the attention of the test writer unti an item ana"sis actua" focuses
attention on them. 9This is one e)ceent reason for conducting an item
ana"sis.:
Note that items with a -er" high faciit" -aue fai to discriminate and
thus genera" show a ow discrimination inde). The 'articuar grou' of
101
((
Item diPcult# and
discrimination
(
students who were gi-en the foowing item had ob-ious" mastered the use of
!or and since foowing the 'resent 'erfect continuous tense!
He>s been i-ing in Berin ... 1/72.
6 1 1/<31/ 1 3 ;E 1 30 1 3./2
9The item is e)treme" eas" for the testees and has Lero
discrimination.:
;aciit" -aues and discrimination indices are usua" recorded together in
tabuar form and cacuated b" simiar 'rocedures. Note again the formuae
used!
;EOO Correct * _8Correct L
<n
6Correct * 8 Correct L n
or ;E 1 #
N
The foowing tabe. com'ied from the resuts of the test referred to
in the 'receding 'aragra'hs. shows how these measures are recorded.
(tem* L *_L;E *8L 6
1 1/ 1/ 30 ./2 3 3
< 13 1? </
.73 83 8.12
3 <3 1< 3< .03 0 .=3
= 10
3 <1 23 12 .72
2 12 ? <1 .23 / .=2
? 1? 12 31 .77 1
.32
7 17 0 <2 .?< / .=2
0 13
= 17 .=< / .=2
/ = ? 13 .<2 8< 8.13
13 13 = 1= .32 ? .33
11 10 13 31 .70 2 .<2
1< 1<
< 1= .32 13 .23
13 1= ? <3 .23 0 .=3
1= 2 1 ? 12 = .<3
12 7 1 0 .<3 ? .33
1? 3 3 3 .30 3 .12
Etc.
(tems showing a discrimination inde) of beow .33 are of doubtfu use
since the" fai to discriminate effecti-e". Thus. on the resuts isted in the
tabe abo-e. on" items 3. =. 2. 7. 0. 13. 1<. 13 and 12 coud be safe" used in
future tests without being rewritten. Howe-er. man" test writers woud
kee' item 1 sim'" as a ead8in to 'ut the students at ease.
E4tended answer
(t wi often be im'ortant to scrutinise items in greater detai. 'articuar"
anal#sis
in those cases where items ha-e not 'erformed as e)'ected. 7e sha want
to know not on" wh" these items ha-e not 'erformed according to
e)'ectations but aso wh" certain testees ha-e faied to answer a 'articuar
item correct". +uch tasks are reasonab" sim'e and straightforward to
'erform if the muti'e8choice techniBue has been used in the test.
1 3<
(n order to carr" out a fu item ana"sis. or an e)tended answer
ana"sis. a record shoud be made of the different o'tions chosen b" each
student in the u''er grou' and then the -arious o'tions seected b" the
ower grou'.
(f ( were rich. ( . work.
,. shan>t B. won>t woudn>t
* L *_L
,. 1 = 2
B. < 2 7 ;E 1
C. 1= = 10
6. 3 7 13
9<3: 9<3: 9=3:
6. didn>t
*_L O 10 1
<n =3 =2
*8L8 13
n <3 1.23
The item has a faciit" -aue of .=2 and a discrimination inde) of .23 and
a''ears to ha-e functioned efficient"! the distractors attract the 'oorer.
students but not the better ones.
The 'erformance of the foowing item with a ow discrimination inde) is
of 'articuar interest!
Ar 7atson wants to meet a friend in +inga'ore this "ear.
He ... . him for ten "ears.
,. knewB. had known8 8O C!8knows
F has known
*L*_L
8
,.
7 3 13
B. = 3 7 ;E1.3<2
C. 1 / 13 6 1 .12
6.
0
2 13
9<3: 9<3: 9=3:
7hie distractor C a''ears to be 'erforming we. it is cear that distractors ,
and B are attracting the wrong candidates 9i.e. the better ones:. 4n
coser scrutin". it wi be found that both of these o'tions ma" be correct in
certain conte)ts! for e)am'e. a student ma" en-isage a situation in which Ar
7atson is going to -isit a friend whom he had known for ten "ears in Engand
but who now i-es in +inga'ore. e.g.
He knew him 9we: for ten "ears 9whie he8i-ed in Engand:.
The same Justification a''ies for o'tion B.
The ne)t item shoud ha-e functioned efficient" but faied to do so! an
e)amination of the testees> answers eads us to guess that 'ossib" man" had
been taught to use the 'ast 'erfect tense to indicate an action in the 'ast
taking 'ace before another action in the 'ast. Thus. whie the resuts
obtained from the 're-ious item refect on the item itsef. the resuts
here pro"a"l# refect on the teaching!
John ;. $enned" ...
. born in 1/17 and died in 1/?3.
,. is B. has been F was 6. had been
103
*L*_L
,. 3 < <
B. 3
3 3 ;E1.?<2
C. 13 1< <2 6 1 .32
6. 7 3
13
9<3:
9<3: 9=3:
(n this case. the item might be used again with another grou' of students.
athough distractors , and B do not a''ear to be 'uing much weight.
6istractor 6 in the foowing e)am'e is ineffecti-e and cear" needs to
be re'aced b" a much stronger distractor!
.H8888.He com'ained that8he ...
O.
the same bad fim the night before.
had seen B. was seeing
C. has seen 6. woud see
*L*_L
,.
1=
0
<<
B. = 7 11 ;E1.22
C. < 2 7 6 1 .33
6.
3
3
3
9<3: 9<3: 9=3:
+imiar". the e-e of difficut" of distractors C and 6 in the foowing
item is far too ow! a fu item ana!.sis suggests on" too strong" that the"
ha-e been added sim'" to com'ete the number of o'tions reBuired.
7asn>t that "our father o-er thereG
,. %es. he was.
C. %es. was he.
%es. it was. 6. %es. was it.
*L*_L
,. 7 13 <3
B. 13 7 <3 ;E 1.23
C. 3 3 3 6 1.33
6. 3 3 3
9<3: 9<3:
9=3:
The item coud be made sight" more difficut a!.d thus im'ro-ed b"
re'acing distractor C b" ?es, he wasn>t and 6 b" ?es, it wasn>t. The
item
is sti im'erfect. but the difficut" e-e of the distractors wi 'robab"
corres'ond more cose" to the e-e of attainment being tested.
The 'ur'ose of obtaining test statistics is to assist inter'retation of
item and test resuts in a wa" which is meaningfu and significant. 5ro-ided
that such statistics ead the teacher or test constructor to focus once again
on the content of the test. then item ana"sis is an e)treme" -auabe
e)ercise. 4n" when test constructors misa''" statistics or become
uncritica" dominated b" statistica 'rocedures does item ana"sis begin to
e)ert a harmfu infuence on earning and teaching. (n the fina ana"sis.
the teacher shoud be 're'ared to sacrifice both reiabiit" and
discrimination to a imited e)tent in order to incude in the test certain
items which he or she regards as ha-ing a good Deducationa> infuence on
u
the students if. for e)am'e. their e)cusion might ead to negect in
teaching what such items test.
11.2 Aoderating The im'ortance of moderating cassroom tests as we as 'ubic
e)aminations cannot be stressed too great". No matter how e)'erienced test
writers are. the" are usua" so dee'" in-o-ed in their work that the"
become inca'abe of standing back and -iewing the items with an" rea
degree of obJecti-it". There are bound to be man" bind8s'ots in tests.
es'ecia" in the fied of obJecti-e testing. where the items sometimes
contain on" the minimum of conte)t.
(t is essentia. therefore. that the test writer submits the test for
moderation to a coeague or. 'referab". to a number of coeagues.
,chie-ement and 'roficienc" tests of Engish administered to a arge test
'o'uation are genera" moderated b" a board consisting of inguists.
anguage teachers. a 's"choogist. a statistician. etc. The 'ur'ose of such a
board is to scrutinise as cose" as 'ossibe not on" each item com'rising
the test but aso the test as a whoe. so that the most a''ro'riate and
efficient measuring instrument is 'roduced for the 'articuar 'ur'ose at
hand. (n these cases. moderation is aso freBuent" concerned with the
scoring of the test and with the e-auation of the test resuts.
The cass teacher does not ha-e at his or her dis'osa a the faciities
which the 'rofessiona test writer has. (ndeed. it is often a too tem'ting
for the teacher to construct a test without showing it to an"one. es'ecia"
if the teacher has had 're-ious training or e)'erience in constructing
e)aminations of the more traditiona t"'e. *nfortunate". few teachers
reaise the im'ortance of making a s"stematic ana"sis of the eements and
skis the" are tr"ing to test and. instead of com'iing a ist of test
s'ecifications. tend to seect testabe 'oints at random from coursebooks
and readers. 7eaknesses of tests constructed in this manner are brought to
igiH. in the 'rocess of moderation. Aoreo-er. because there is genera"
more than one wa" of ooking at something. it is incredib" eas" 9and
common: to construct muti'e8choice items containing more than one
correct o'tion. in addition. the short conte)ts of man" obJecti-e items
encourage ambiguit". a feature which can 'ass b" the indi-idua unnoticed.
To the moderator. some items in a test ma" a''ear far too difficut or ese
far too eas". containing im'ausibe distractors@ others ma" contain
unsus'ected cues. 4n" b" moderation can such fauts be brought to the
attention of the test writer.
(n those cases where the teacher of Engish is working on his or her
own in a schoo. assistance in moderation from a friend. a s'ouse. or an
oder student wi 'ro-e beneficia. (t is sim'" im'ossibe for an" singe
indi-idua to construct good test items without he' from another 'erson.
11.? item cards and
,s must be -er" cear at this stage. the construction of obJecti-e tests
banks necessitates taking a great dea of time and troube. ,though the
scoring of such tests is sim'e and straightforward. further effort is then s'ent
on the e-auation of each item and on im'ro-ing those items which do not
'erform satisfactori". (t seems somewhat iogica. therefore. to dis'ense
with test items once the" ha-e a''eared in a test.
The best wa" of recording and storing items 9together with an"
ree-ant information: is b" means of sma cards. 4n" one item is entered
on each card@ on the re-erse side of the card information deri-ed from an
item ana"sis is recorded! e.g. the faciit" -aue 9;E:. the (nde) of
102
6iscrimination 96:. and an e)tended answers ana"sis 9if carried out:.
,fter being arranged according to the eement or ski which the" are
intended to test. the items on the se'arate cards are grou'ed according to
difficut" e-e. the 'articuar area tested. etc. (t is an eas" task to arrange
them for Buick reference according to whate-er s"stem is desired.
;urthermore. the cards can be rearranged at an" ater date.
,though it wi ob-ious" take considerabie time to buid u' an item
bank consisting of a few hundred items. such an item bank wi 'ro-e of
enormous -aue and wi sa-e the teacher a great dea of time and troube
ater. The same items can be used man" times again. the order of the items
9or o'tions within each item: being changed each time. (f there is concern
about test securit" or if there is an" other reason indicating the need for
new items. man" of the e)isting items can be rewritten. (n such cases. the
same o'tions8are genera"8ke't. but the conte)t is changed so that one of
the distractors now becomes the correct o'tion. Auti'e8choice items
testing most areas of the -arious anguage eements and skis can be
rewritten in this wa". e.g.
9&rammar:
( ho'e "ou ... . us "our secret soon.
,. tod B. wi te C. ha-e tod
6. woud te
( wish "ou ... . us "our secret soon.
,. tod B. wi te C. ha-e tod 6. woud te
9Eocabuar": ,re "ou going to wear "our best ... . for the
'art"G
,. cothes B. cothing C. coths 6. cothings
8 7hat kind of ...
. is "our new suit made ofG
,. cothes B. cothing C. coth 6. cothings
95honeme
beat bit beat
discrimination: 88 beat beat bit
9Listening +tudent hears!
7h" are "ou going homeG
com'rehension:
+tudent reads! ,. ,t si) o>cock.
B. %es. ( am.
C. To he' m" mother.
6. B" bus.
+tudent hears! How are "ou going to 6a-id>sG
+tudent reads! ,. ,t si) o>cock.
B. %es. ( am.
C. To he' him.
6. B" bus.
10?
9#eading 88V8 Two8thirds of the countr">s 9fue. endea-our. industr".
com'rehensionK energ": comes from im'orted oi. whie the remaining
-ocabuar":
one8third comes from coa. Aoreo-er. soon the countr"
wi ha-e its first nucear 'ower station.
Two8thirds of the countr">s 9fue. endea-our. industr".
'ower: takes the form of im'orted oi. whie the
remaining one8third is coa. Howe-er. e-er"one in the
countr" was made to reaise the im'ortance of coa
during the recent miners> strike. when man" factories
were forced to cose down.
(tems rewritten in this wa" become new items. and thus it wi be
necessar" to coect faciit" -aues and discrimination indices again.
+uch e)am'es ser-e to show wa"s of making ma)imum use of the
-arious t"'es of test items which ha-e been constructed. administered and
e-auated. (n an" case. howe-er. the effort s'ent on constructing tests of
Engish as a second or foreign anguage is ne-er wasted since the insights
'ro-ided into anguage beha-iour as we as into anguage earning and
teaching wi awa"s be in-auabe in an" situation connected with either
teaching or testing.
107
i
+eected bibiogra'h"
Closed tests and e4aminations in English as
a
second2!oreign language
,ssociated E)amining Board 9,EB:
&est in English !or Educational %urposes /&EE%1
,ssociation of #ecogniLed Engish Language +choos
E)amination Trust 9,ET:
A7E(' <ral E4aminations. %reliminar#,$$Higher
Certi5cate, Giploma
British CounciK*ni-ersit" of Cambridge Loca E)aminations
+"ndicate
English (anguage &esting 'ervice
Cit" and &uids of London (nstitute
Communication in &echnical English
Educationa Testing +er-ice 95rinceton. New Jerse". *+,:
&est o! English as a 6oreign (anguage /&<E6(1
Engish Language Teaching 6e-eo'ment *nit 9ELT6*:
'tages o! Attainment 'cale and &est Batter#
Engish +'eaking Board 9(nternationa: Limited 9E+B:
<ral Assessments in 'po-en English as an Ac,uired
(anguage
&enera Aedica Counci 9&AC:
&he %(AB &est
(nstitute of Linguists Educationa Trust
E4aminations in English as a 6oreign (anguage.
(evels =,
2. 3and+
Certi5cates in English as a 6oreign (anguage.
%reliminar#,
Brade I and Brade ==
Giplomas in English as a 6oreign (anguage.
Intermediate
Giploma and 6inal Giploma
Joint Aatricuation Board 9JAB:
&est in English /<verseas1
London Chamber of Commerce and (ndustr" 9LCC(:
English !or Commerce. Elementar# 'tage,
Intermediate
'tage, and Higher +tage
'po-en English !or Industr# and Commerce
/'E6IC1
North 7est #egiona E)aminations BoardKNorth 7estern
#egiona ,d-i Counci for ;urther Education
English as a 'econd (anguage. 2 versions2levels
4)ford 6eegac" of Loca E)aminations 94)ford:
9niversit# o! <4!ord Gelegac#>s E4aminations in
English as
a 6oreign (anguage. %reliminar# (evel Certi5cate
and
Higher (evel Certi5cate
5itman E)aminations (nstitute 95E(:
English as a 6oreign (anguage /'#lla"us ( and
'#lla"us C1
and English (anguage /;CE1. Elementar#,
Intermediate,
Higher Intermediate and Advanced
#o"a +ociet" of ,rts E)aminations Board 9#+,:
English as a 6oreign (anguage. +tage 1. 'tage II and
'tage
III
Communicative 9se o! English as a 6oreign
(anguage.
Basic, Intermediate and Advanced
100
(
Trinit" Coege. London
'po-en English as a 6oreign or 'econd (anguage.
=2 grades )ritten English /Intermediate1
*ni-ersit" of Cambridge Loca E)aminations +"ndicate
%reliminar# English &est /%E&1
6irst Certi5cate in English /6CE1
.HCe.rti5cate o! %ro5cienc# in.English /C%E1
Giploma o! English 'tudies /GE'1
'chool Certi5cate and BCE /<verseas Centres1.
English (anguage
*ni-ersit" of London
0 (evel English (anguage '#lla"us B
Boo-s and articles
,itken. $ & 1/77 *sing cone 'rocedure as an o-era
anguage 'roficienc" test. &E'<( Duarterl# 1191:! 2/8?7
,itken. $ & 1/7/ TechniBues for assessing istening
com'rehension in second anguages. Audio$Aisual
(anguage Journal =E. 172801
,derson. J C 1/70 , stud" of the coLe 'rocedure with
nati-e and non8nati-e s'eakers of Engish. *ni-ersit" of
Edinburgh 5h6 thesis
,derson. J C and Hughes. , 9eds.: 1/01 (ssues in
Language Testing. E(& Gocuments III. British Counci
,en J 5 B and 6a-ies. , 9eds.: 1/77 Testing and
e)'erimenta methods. Edin"urgh Course in
Applied (inguistics -o =. 4)ford *ni-ersit" 5ress
,nderson. J 1/71 , techniBue for measuring reading
com'rehension and readabiit". English (anguage
&eaching Journal 2F/21. 17080<
,nderson. J 1/7? %s#cholinguistic e4periments in !oreign
language testing. *ni-ersit" of Sueensand 5ress.
Beardsmore. H B 1/7= Testing ora fuenc". I7.+(
1< 9=:! 3178<?
Bensoussan. A 1/03 6ictionaries and tests of E;L reading
com'rehension. English (anguage &eaching
Journal
37 9=:! 3=182
Brown. & 1/77 (istening to spo-en English. Longman
Brumfit. C J 1/0= Communicative Methodolog# in
(anguage
&eaching. Cambridge *ni-ersit" 5ress
Bursta. C 1/?/ The main stages in the de-eo'ment of
anguage tests. (n +tern. H H 9ed.: (anguages and
the
?oung +choo Child. 4)ford *ni-ersit" 5ress. 1/38/
Burt. A $ and $i'arsk". C 1/7< &he Boo5con. a repair
manual !or English. Newbur" House. #owe".
Aassachusetts
Canae. A and +wain. A 1/03 Theoretica Bases of 8
Communicati-e ,''roaches to +econd Language Teaching
and Testing. Applied (inguistics =. 18=7
3 O!1 O.. .O... @ .D
Carro. B J 1/03 &esting Communicative %er!ormance.
5ergamon
Chen. Q and Henning. & 1/02 Linguistic and cutura bias
in anguage 'roficienc" tests. (anguage
&esting
< 9<:! 10381/1
Cark J L 6 1/7< 6oreign (anguage &esting. &heor# and
%ractice. Centre for Curricuum 6e-eo'ment.
5hiade'hia. 5enns"-ania
Cark. J L 6 9ed.: 1/70 Girect &esting o! 'pea-ing
%ro5cienc#. &heor# and %ractice. Educationa
Testing +er-ice. 5rinceton
Cohen. , 6 1/03 &esting (anguage A"ilit# in the
Classroom. Newbur" House. #owe". Aassachusetts
Crocker. , C 1/?/ 'tatistics !or the &eacher /or How
to %ut
6igures in their %lace1. 5enguin
6a-ies. , 9ed.: 1/?0 (anguage &esting +"m'osium. 4)ford
*ni-ersit" 5ress
6a-ies. , 1/70 Language Testing 9+ur-e" ,rtices Not. 3
and =:. (n $insea. E 9ed.: (anguage &eaching and
(inguistics A"stracts -o. <! 1<782/. Cambridge
*ni-ersit"
5ress
6a-ies. + and 7est. # 1/0= &he %itman Buide to
English
(anguage E4aminations <nd edn. 5itman
6ougas. 6 1/70 &ain in reading 'roficienc" in Engish as a
;oreign Language measured b" three doLe scoring
methods. Journal o! 7esearch in 7eading
191:!?78
73O8O
Eis. # 1/0= Communication strategies and the e-auation
of communicati-e 'erformance.
English (anguage
&eaching Journa 3091:! 3/8==
;inocchiaro. A and +ako. + 1/03 6oreign (anguage &esting.
A %ractical Approach. #egents. New %ork
;ok. ,. Lord. #. Low. &. T>sou B $. and Lee.
% 5 1/01 )or-ing %apers in (inguistics and
(anguage
&eaching. +'ecia (ssue on Language Testing. No. =.
Language Centre. *ni-ersit" of Hong $ong
&annon. 5 1/02 Assessing )riting. principles and
practice
o! mar-ing written English. Edward ,rnod
&eoghegan. & 1/03 (anguage pro"lems o! non$native
spea-ers o! English at Cam"ridge
9niversit#.
Be Educationa Trust. Cambridge
&reen. J , 1/72
&eacher$Made &ests <nd edn. Har'er and
#ow. New %ork
Hae. & ,. +tansfied. C 7. and 6uran.
# 5 1/0= 'ummaries o! studies involving the
&est o! English as a 6oreign (anguage,
1/?381/0<.
Educationa Testing +er-ice. 5rinceton. New Jerse"
Hanania. E and +hikhani. A 1/0? (nterreationshi's ,mong
Three Tests of Language 5roficienc"! +tandardiLed E+L.
CoLe. and 7riting.
&E'<( Duarterl#
<391:! /7813/
Harris. 6 5 1/?/
&esting English as a 'econd (anguage.
Ac&raw8Hi. New %ork
Harrison. , 1/03 A (anguage &esting Hand"oo-.
Aacmian
Heaton. J B 9ed.: 1/0< (anguage &esting. Aodern Engish
5ubications
Hendrickson. 3 9ed.: Error ,na"sis and Error Correction in
Language Teaching. 7E(C <ccasional %apers
13.
+E,AE4 #egiona Language Centre. +inga'ore
8 8
Henning. & H et. a. 1/01 Com'rehensi-e ,ssessment of
Language 5roficienc" and ,chie-ement ,mong Learners
of Engish as a ;oreign Language.
&E'<( Duarterl#
12 9=:! =278?? 88
Hughes. , and Lascaratou. C 1/0< Com'eting criteria for
error gra-it". English (anguage
Teaching Journal
?3 93:! 17280<
Hughes. , and 5orter. 6 9eds.: 1/03 Current Gevelopments
in (anguage &esting. ,cademic 5ress
Hughes. , and 5orter. 6 9eds.: (anguage &esting.
9Journa
'ubished b" Edward ,rnod in June and 6ecember each
"ear.:
(be. A 6 1/72 , com'arison of doLe and muti'e8choice
tests for measuring the Engish reading com'rehension of
southeast ,sian teachers of Engish.
7E(C Journal
? 9<:! <=83<. +E,AE4 #egiona Language Centre.
+inga'ore
Jones. # L and +'osk". B 9eds.: 1/72 &esting (anguage
%ro5cienc#. Center for ,''ied Linguistics. ,rington.
Eirginia
Lado. # 1/?1. 1/?= (anguage &esting. the Construction and
9se o! 6oreign (anguage Tests. Longman
Lee. % 5 and Low. & 6 1/01 Classi!#ing tests o! language
use. 5a'er 'resented at ?th ,(L, 7ord Congress. Lund.
+weden
Lee. % 5. ;ok. C % %. Lord. #. and Low. & 1/0< ;ew
Girections in (anguage &esting. 5ergamon. Hong
$ong Lukmani. % 1!3< The communicationa testing of
reading.
English (anguage &eaching Journal
3?9=:! <178<2
Aoer. , 1/72 Eaidit" in 5roficienc" Testing. E(&
Gocuments 3.
2810. British Counci
Aorrow. $ E 1/77 &echni,ues o! Evaluation !or a
;otional
'#lla"us. Centre for ,''ied Language +tudies. *ni-ersit"
of #eading 9for the #o"a +ociet" of ,rts:
Aorrow. $ E 1/7/ Communicati-e Language Testing!
re-oution or e-oution. (n Brumfit. C J and Johnson. $ J
9eds.: &he Communicative Approach to (anguage
&eaching. 4)ford *ni-ersit" 5ress
Aunb". J L 1/70 Communicative '#lla"us
Gesign.
Cambridge *ni-ersit" 5ress
4ier. J 7 1/7< CoLe tests of second anguage 'roficienc"
and what the" measure. (anguage (earning
<391:! 132810
4ier. J 7 1/7/
(anguage &ests at 'chool. Longman
4((erT18E L8and 5erkins. $ 1/70 (anguage in Education.
testing the tests. Newbur" House. #owe". Aassachusetts
4ier. J 7 and +treiff. E 1/72 6ictation! a test of grammar8
based e)'ectancies. English (anguage &eaching
Journal
33 91:! <283?
5amer. , + 1/01 Testing communication. I7A( 13! 328=2
5amer. , + 1/01 Aeasures of achie-ement.
communication. incor'oration. and integration for two
casses of forma E;L earners. 7E(CJournal
1< 91:! 378?1
5amer. L and +'oisk". B 9eds.: =MEF.%apers on (anguage
&esting, =M*E$=ME+. TE+4L. 7ashington. 6. C.
5erkins. $ 1/03 *sing 4bJecti-e Aethods of ,ttained
7riting 5roficienc" to 6iscriminate ,mong Hoistic
E-auations. &E'<( Duarterl# 1=91:! ?18?/
10/
(( (
5erren. & E 9ed.:
1/77 6oreign (anguage &esting.
specialised "i"liograph#. Centre for (nformation on
Language Teaching and #esearch
5orta. A 9ed.: 1/0? Innovations in (anguage &esting.
N;E#8Neson
#ea. 5 A 1/70 ,ssessing anguage as communication! MA('
Journal. New series. No. 3. 6e'artment of Engish.
*ni-ersit" of Birmingham
#ead. J , + 9ed.: 1/01
Girections in (anguage &esting,
7E(C Antholog#, 'eries M. +E,AE4 #egiona Language
Centre. +inga'ore
#ichards. J C 1/02
&he Contest o! (anguage &eaching.
Cambridge *ni-ersit" 5ress
#i-era. C 1/0= Communicative Competence Approaches to
(anguage %ro5cienc# ,ssessment! 7esearch and
Application. Autiingua Aatters. Ce-edon O
#i-ers. 7 A 1/?0 &eaching !oreign$language s-ills.
*ni-ersit" of Chicago 5ress
#i-ers. 7 A and Tem'ere". A + 1/70 A %ractical Buide to
the &eaching o! English as a 'econd or 6oreign (anguage.
4)ford *ni-ersit" 5ress. New %ork
+chuL. # ,
1/77 6iscrete85oint -ersus +imuated
Communication Testing in ;oreign Languages. Modern
(anguage Journal ?193:! /18131
+immonds. 5 1/02 , sur-e" of Engish anguage
e)aminations. English (anguage &eaching Journal,
3/91:! 338=<
+'oisk". B
1/02 7hat does it mean to know how to use a
anguageG ,n essa" on the theoretica basis of anguage
testing. (anguage &esting 2/21. 10381/1
+'osk". B with Aur'h". 5. Hom. 7. and ;erret.
, 1/7< Three ;unctiona Tests of 4ra 5roficienc".
&E'<( Duarterl# ?93>! <<1832
+tubbs. J B and Tucker. & # 1/7= The CoLe Test as a
Aeasure of Engish 5roficienc". Modern (anguage Journal
20
92K?:! <3/8=1
Tomi"ana. A 1/03 &rammatica errors and communication
breakdown. &E'<( Duarterl# 1=91:! 718/
*'shur. J , 1/71
4bJecti-e e-auation of ora 'roficienc" in
the E+4L cassroom. &E'<( Duarterl# F. =78?3
*'shur. J , and ;ata. J 1/?0
5robems in foreign anguage
testing. (anguage (earning +'ecia (ssue. No. 3
Eaette. # A
1/77 Modern (anguage &esting <nd edn.
Harcourt Brace Jo-ano-ich. New %ork
Eaette. # A and 6isick. # +
1/7< Modern (anguage
%er!ormance <":ectives and Individualisation. Harcourt
Brace Jo-ano-ich. New %ork
1/3
(nde)
,chie-ement tests 17<
,ddition items 23
,dministration 1?781?0
,na"tic marking 1=0
,'titude items 173
,ttainment tests 171
Backwash 1738171
Banding
8 com'ositions 1=281=?
8 ora inter-iews /08133
Broken sentence items =/
Centra tendenc" 172817?
Chan g in g words =08=/
12
Error recognition items 3/8=3. 12<
Error8count marking 1=081=/
Errors 1=/8123
Essa"8transation a''roach 12
E-auation 7
E)tended answer ana"sis 10<8102
E)tensi-e reading 13?
;ace -aidit" 12/81?3
;aciit" -aue 170817/
;ragmented sentences 122
;reBuenc" distribution 17=8172
&rading com'ositions 1==81=/
&rammar /. 3=823
8 error8count method 1=081=/
8 im'ression method 1=781=0 8
mechanica accurac"
method 1=081=/
8 treatment of errors 1=/8123
Aatchin g items
8 reading 1378113
8 -ocabuar" 208?3
Aean 172817?
Aechanica accurac" marking 1=081=/
Aedian 172
Aistakes 7. 1=/8123
Aode 172
Aode 'aragra'hs 12=
Aoderating 10?8107
Aoti-ation 7
Cassification of tests
Cassroom tests ?
CoLe 'rocedure 1?817. 1318133
Combination items 23
Communicati-e a''roach 1/8<=
Com'etion items
8g rammar =<8=?
8 reading com'rehension 1<=81</ 8
sentences. te)ts 12?8127
8 s'eing 121812<
8 -ocabuar" ?<8?3
Com'osition
8 genera 13?8123
8 setting 13081=3
8 tites 1378130
Concurrent -aidit" 1?181?<
Connecti-es 12?
Construct -aidit" 1?1
Content -aidit" 1?381?1
Controed writing 12=8120
Con-ersations /38/<
Cursor" reading 133813=
6efinitions ?<
6iagnostic testing ?. 173
6iaogues ?/871. /38/<>
6ictation 17810. 121
6ifficut" inde) 170817/
6iscrimination 1?2. 17/810<
6is'ersion 17?8170
Em'irica -aidit" 1?181?<
Error. test margin of 1??
&rou' discussion 13<813=
(m'ression marking 1=781=0
(m'ure items </
(nde) of difficut" 170817/
(nstructions 1?08173
(ntegrati-e a''roach 1?
(ntensi-e reading 13?
(nter'retation of scores 17=8100
(ntonation ?08?/
(tems
8 ana"sis 1708102
8 cards and banks 1028107
8 difficut" 170817/
8 discrimination 17/810<
/6or t#pes o! items, see
individual entries, e.g. add t i
ion, com"ination,
multiple$choice1
Judgement skis 132
Language areas /
Language eements 13811
Language skis 0. 13811
Lectures 0<807
Length of te)ts 110
Linking sentences 12
Listening com'rehension ?=807
AarkKre8mark reiabiit" 1?<
Aarking
8 ana"tic method 1=0
8 com'ositions 1==81=/
Auti'e marking 1=781=0
Auti'e8choice items
8 construction <78=3
8 correct o'tion 37830
8 distractors.3<833
8 error recognition 3/8=3
8 grammar /
8 istening ??871
8 reading 11?81<=
8 s'eing 121
8stem 3?837
8 -ocabuar" 2<8<0
8 writing 12<8123
4bJecti-e testing <28<7
4'en8ended items 133
4ra 'roduction 00813=
4ra inter-iew /?813<
5airing and matching items =/823
5arae test forms 1?3
5erfomn race e-es <18<3 888888
5honeme discrimination ?28?0
5honoog" /
5ictures
8 istening com'rehension 7180< 8
matching 113. 11<
8 s'eaking /<8/?
8 writing 1=<81=3
5racticabiit" 1?781?0
5redicti-e -aidit" 1?181?<
5robem so-ing 13<813=
5roduction 11
1/1
i
5roficienc" tests 17<8173
5rofie re'orting 1/8<3. 1 ?3
5rogress tests 171817<
5s"chometric a''roach 1?
5unctuation 132. 1238121
Suaitati-e Judgements <38<<
#ange 17?
#ating scae
8 com'ositions 1=281=?
8 ora inter-iews /08133
#eading
8 e)tracts 9for writing: 122812?
8reading aoud801/388 8
8 skis 132813?
8 test s'ecifications <<8<3
#earrangement items
8 grammar =18=<
8 reading 1</8131
8 -ocabuar" ?1
#ecognition ((
#edundanc" ?=
#egister 123812=
#eiabiit" 1<. 1?<81?2
8 istening com'rehension ?2
8 markKre8mark 1?<
8 'arae test forms 1?3
8 'rofie re'orting 1?3
8 s'it haf8method 1?381?=
8 testKre8test 1?<
1/<
8- -aidit" 1?=81?2
#oe 'a"ing 13<813=
#ubrics 1 ?08173
+am'ing 1<. 2182<. 110
+canning 133813=.
+cores 17=8100
+coring
8 ora inter-iews /081338
+entence matching 1378113
+ets 20
+kimming 133813=
+'eaking <18<3. 00813=
+'ecifications 13
+'eing 132. 121812<
+'it8haf method 1?381?=
+'oken anguage ?=8?2
+'read of scores 1??81?7
+tandard de-iation 17?8170
+tandards 7
+tatements ?/871
+tress ?08?/
+tructuraist a''roach 12
+t"e 12<8123
+ubJecti-e testing <28<?
+"non"ms ?1
Taks 0<807. 13<
i
Teaching 2.173817(
Test t"'es 13?8137. 1718173
TestKre8test reiabiit" 1?<
Transformation items =?8=0
Transation 1081/
Tra's 1=
TrueKfase tests 113811?
*sae 3=823 g
Eaidit" 12/81?<
8concurrent 1?1
8 construct 1?1
8 content 1?381?1
8 em'irica 1?181?<
8 face 12/81?3
8 'redicti-e 1?1
8 - reiabiit" 1?=81?2
Eisuas
8 istening com'rehension 7180< 8
matching 113. 11<
8 s'eaking /<8/?
8 writing 1=<81=3
Eocabuar" / . 218?3
7ord formation ?1
7ord matching 137
7riting 1328120
8 e-es 13?
8 tasks 13?