Contents

Evaluation and assessment within the domain of

Language(s) of Education
Waldemar Martyniuk (ed.), Mike Fleming, Jos Noijons
Languages of Education
Language Policy Diision, !tras"ourg
###.coe.int$lang
Language Policy Division
Division des Politiques linguistiques
Evaluation and assessment within the domain of Language(s)
of Education
Waldemar Martyniuk (ed.), Jagiellonian %niersity, Poland
Mike Fleming, %niersity o& Dur'am, %nited (ingdom
Jos Noijons, )*+,, +'e Net'erlands
*ntergoernmental )on&erence
Languages of schooling within a European framework for Languages of
Education: learning, teaching, assessment
Prague -./0 Noem"er 1002
,rganised "y t'e
Language Policy Diision, )ouncil o& 3uro4e, !tras"ourg
in co.o4eration #it' t'e
Ministry o& 3ducation, 5out' and !4orts, )6ec' 7e4u"lic
Language Policy Division
D8 *9 : Directorate o& !c'ool, ,ut.o&.!c'ool and ;ig'er 3ducation
< )ouncil o& 3uro4e
The opinions expressed in this work are those of the authors and do not necessarily
reflect the official policy of the Council of Europe.
All correspondence concerning this publication or the reproduction or translation of all
or part of the document should be addressed to the Director of School, ut of School
and !igher Education of the Council of Europe "#$%&'&( Strasbourg Cedex).
The reproduction of extracts is authorised, except for commercial purposes, on
condition that the source is *uoted.
Contents
Contents......................................................................................................................................5
The Challenge of Assessment within Language(s) of Education...............................................9
Mike Fleming, ni!e"sit# of $u"ham, nited %ingdom............................................................9
The &ele!ance of 'nte"national Assessment fo" the de!elo(ment of a F"amewo"k fo" the
Languages of Education............................................................................................................)*
+os, -oi.ons, C'T/, The -ethe"lands......................................................................................)*
Assessing com(etences at the end of com(ulso"# schooling 0 the 1olish case........................23
4aldema" Ma"t#niuk, +agiellonian ni!e"sit#, %"ak5w, 1oland.............................................23
5
6
Evaluation and assessment within the domain of Language(s) of Education
Introduction
3aluation and assessment o& com4etences related to language(s) o& education is an area
o& s4ecial interest #it'in t'e )ouncil o& 3uro4e=s 4roject e>amining t'e &easi"ility o&
deelo4ing a common 3uro4ean re&erence document &or t'e L3 domain. +'e discussions
carried out so &ar #it'in t'e Working 8rou4 ?3aluation and @ssessment=
/
can "e
summarised as &ollo#sA
/. +'e &undamental Buestion to "e ans#ered #'en making decisions regarding
assessment is ?WH do #e #ant to 'ae an assessment sc'emeC= +'e most
4ro"lematic area is usually not t'e assessment itsel& "ut t'e use made o& t'e
results, t'e claims made on t'e "asis &or t'e assessment, and its im4act.
1. !uita"le assessment solutions and a44roac'es can "e o&&ered only once attainment
targets (com4etence standards) 'ae "een clearly identi&ied.
D. @ssessment designers s'ould ?negotiate= #it' curriculum deelo4ers #'en
attainment targets (com4etence standards) are "eing turned into assessment tasks
and items.
E. ,nly t'ose attainment targets (com4etence standards) t'at may "e turned into
o"sera"le "e'aiour can "ecome t'e su"ject o& assessment 4rocedures (testing
tasks$items).
F. *ssues suc' as alidity and relia"ility can "e &irst addressed at t'e leel o& items and
tasks.
G. *tem leel testing may not "e ca4a"le o&$suita"le &or 'andling all o& t'e com4le>ity
o& L3 com4etences.
2. !am4ling may "e t'e only &easi"le o4tion i& educational systems (not only indiidual
learners) need to "e ealuated.
-. *ntroducing standards and assessment sc'emes s'ould "e a "ottom.u4 4rocess
(re&lecting t'e needs o& learners and teac'ers) rat'er t'an a to4.do#n 4rocedure
(re&lecting only t'e needs o& decision makers).
H. !tandards and assessment sc'emes s'ould "e 4romoted as assistance tools rat'er
t'an control measures.
/0. +'e enisaged L3 &rame#ork document s'ould sere as an a#areness.raising and
re&lectie tool, "roadening t'e notion o& ealuation and assessment "eyond testing
and leels.
Ielo# #e 4resent a set o& studies dealing #it' some o& t'ese issues. +'e o4ening 4a4er "y
Mike Fleming is an oerie# article listing and addressing c'allenges related to ealuation
and assessment #it'in t'e domain o& L3 t'at may sere as a "asis &or a &uture c'a4ter in
t'e enisaged L3 &rame#ork document. +'e 4a4er concludes #it' a 4ro4osal called
?Languages o& 3ducation Port&olio= : an integrated a44roac' to ealuation and assessment
#it'in t'e domain o& L3.
+'e t#o case studies t'at &ollo# illustrate 'o# t'e issues raised in t'e oerie# article are
"eing turned into 4ractice on an international and a national leel. +'e 4oint made "y Jos
Noijons #'en 4resenting t'e t#o international assessment &rame#orks : P*!@ and P*7L! : is
t'at een i& 4eo4le, s4eci&ically teac'ers, may not "e in &aour o& internationally 4roduced
standards, t'ey are already #it' us and national 4olicy makers #is' to ad'ere to t'ese
)
8rou4 mem"ersA )'risto4' @rnold, Mike Fleming, Waldemar Martyniuk (co.ordinator), Jos Noijons
*
standards. ;is suggestion &or t'e #ork on an L3 &rame#ork document is to ackno#ledge
t'e already esta"lis'ed conce4tual &rame#orks o& international sureys and t'e standards
t'at go #it' t'em.
+'e 4resentation o& a national.leel system o& com4ulsory e>ternal e>aminations #'ere
com4etences in a language o& sc'ooling : Polis' : is assessed, is an attem4t to e>amine
'o# t'e c'allenges to standardise and assess com4etence in L! and L@) are "eing met in
4ractice. 9arious &orms o& e>ternal assessment sc'emes are in use in most o& t'e mem"er
states o& t'e )ouncil o& 3uro4e. 1H out o& t'e EE national res4ondents (including - &rom t'e
8erman Lnder and F &rom t'e %nited (ingdom) to a Preliminary surey on curricula &or
teac'ing national$o&&icial$sc'ool languages in com4ulsory education, carried out "y t'e
Language Policy Diision in @4ril 100F, claimed to 'ae e>ternal e>aminations in 4lace. @
ne# surey, targeting s4eci&ically t'e use, t'e sco4e and t'e signi&icance o& t'ese
e>aminations #as sent out "y t'e Language Policy Diision in mid July 1002. +'e results o&
t'e ne# surey #ill certainly sere as ery alua"le in4ut &or t'e #ork on an L3 &rame#ork
document.
Waldemar Martyniuk
7
!he Challenge of "ssessment within Language(s) of Education
#i$e %leming& 'niversity of Durham& 'nited (ingdom
Introduction
+'e 4ur4ose o& t'is 4a4er is to 4roide an oerie# o& some o& t'e key issues and
c'allenges related to ealuation and assessment #it'in t'e domain o& language(s) o&
education. *t incor4orates elements o& t'e 4reious 4a4er distri"uted at t'e !tras"ourg
con&erence in 100G
1
"ut is also addresses ne# more 4ractical issues related to (a) t'e
4ossi"ility o& situating assessment &or language as sc'ool su"ject (L!) #it'in t'e #ider
construct o& language(s) o& education (L3)J (") t'e need to reconcile 4ort&olio a44roac'es
#it' more &ormal testing. @ssessment is im4ortant "ut it is not #it'out controersy and it
can easily lead to 4olarised ie#s and un'el4&ul tensions. +'is is 4articularly t'e case in
t'e conte>t o& language as sc'ool su"ject (L!) "ecause o& t'e diersity and com4le>ity o&
its aims. )onstructie de"ate around di&&erences o& o4inion is al#ays 'el4&ul, "ut too o&ten
disagreements a"out assessment "ecome entrenc'ed and un4roductie. +'is 'a44ens &or a
num"er o& reasons, includingA
a &ailure to recognise t'at assessment needs to &ul&il a #ide range o& legitimate
4ur4osesJ
an assum4tion t'at a single assessment tool #ill "e a"le to sere all needsJ
a lack o& a#areness t'at it is t'e use made o& assessment, not necessarily t'e
assessment 4rocess itsel&, t'at #ill largely determine its im4actJ
a tendency to searc' &or uniersal solutions to assessment issues and neglect t'e
signi&icance o& conte>t.
+eac'ers o& language as sc'ool su"ject are sometimes 'ostile to t'e idea o& large.scale or
&ormal testing on t'e grounds t'at it diminis'es t'e su"ject and ignores t'e signi&icance o&
conte>t. +'is ie# needs to "e considered.
!ection ,ne #ill consider a range o& di&&erent 4ur4oses o& assessment "ased on di&&erent
4otential audiences. @n ideal assessment strategy #ould meet t'e needs o& all t'ose
interest grou4s. !ection +#o #ill e>amine di&&erent a44roac'es to assessment in relation
to language as sc'ool su"ject (L!), including t'e alue o& 4ort&olio assessment. !ection
+'ree #ill consider t'e im4lications o& situating t'e assessment o& L! #it'in a "road
strategy o& assessment #it'in languages o& education (L3). !ection Four #ill deelo4
&urt'er t'e conce4t o& an integrated a44roac' to assessment #'ic' seeks to reconcile its
di&&erent 4ur4oses #it'in an assessment strategy. *t #ill consider t'e ie# t'at a 4ort&olio
a44roac' is not necessarily in con&lict #it' more &ormal testing, as is o&ten assumed.
)* Pur+oses of assessment
*n its sim4lest &ormulation, assessment 4roides in&ormation on #'et'er teac'ing$learning
'as "een success&ul. ;o#eer t'e in&ormation it 4roides 'as a num"er o& 4otential
di&&erent audiences #'ose 4recise reBuirements may ary. )lassroom teac'ers need
regular in&ormation on 'o# 4u4ils= kno#ledge, skills and understanding are deelo4ing,
"ot' to in&orm 'o# t'ey s'ould adjust t'eir teac'ing and to determine #'at kind o&
&eed"ack is needed to im4roe 4u4ils= learning. ,n t'e ot'er 'and, sc'ool 4rinci4als and
4olicy makers need additional, "roader in&ormation on t'e Buality o& education in a sc'ool
or country. +'e sort o& com4aratie data reBuired &or t'is 4ur4ose needs a 'ig' leel o&
relia"ility and uni&ormity. *n t'e case o& language as sc'ool su"ject t'is reBuirement is
2
*ntergoernmental )on&erence on KLanguages of Schooling: towards a Framework for Europe, !tras"ourg,
)ouncil o& 3uro4e, /G./- ,cto"er 100G. !ee t'e 7e4ort onA ###.coe.int$lang
9
c'allenging "ecause it is di&&icult to create tests #'ic' are managea"le "ut at t'e same
time &ait'&ul to t'e aims o& t'e su"ject. 3m4loyers and society at large also need relia"le
in&ormation #'ic' can 'el4 certi&y ac'ieement and 4roide a "asis &or selection. Parents
too reBuire in&ormation #'ic' can 'el4 t'em understand t'eir c'ildren=s ac'ieements and
limitations. Learners t'emseles need to kno# 'o# t'ey are 4rogressing and 'o# to
im4roe t'eir 4er&ormance "ut t'ey may need to "e 4rotected &rom t'e 4otentially
demotiating e&&ects o& negatie assessment.
+'e conce4t o& ?accounta"ility= #'en used in relation to assessment usually re&ers to t'e
im4osition o& systems o& assessment e>ternal to t'e learning 4rocess as a &orm o& ?4olicing=
o& standards to ensure t'at t'e education system is &unctioning e&&ectiely. Iut t'e term
may "e em4loyed more "roadly and more 4ositiely t'an t'is, re&erring to t'e di&&erent
o"ligations t'at are releant to all. +eac'ers 'ae a res4onsi"ility to t'e learner "ut also
to t'e needs o& t'e #ider society. Policy makers clearly 'ae a duty to t'e 4u"lic and need
to ensure t'at t'e education system is deliering results "ut t'ey also 'ae res4onsi"ilities
to t'e indiidual learners and need to consider conseBuences o& 4olicies in t'ose terms.
+'e conce4t o& accounta"ility inter4reted in t'is #ay #ill take 4eo4le outside o& ested
interests in order to see t'e larger conte>t. @ccounta"ility needs to "e linked #it' a
4rocess o& s'aring 4erce4tions and &ostering understanding. *t is im4ortant t'ere&ore not to
e>aggerate di&&erences "et#een di&&erent 4otential ?stake'olders=J #'at all 4arties 'ae in
common is a &undamental concern t'at assessment s'ould 'el4 raise ac'ieement and
im4roe learning. @ starting 4oint &or resoling tensions related to matters o& assessment is
to deelo4 understanding o& ot'er 4oints o& ie#. @ key c'allenge is to deelo4 a system o&
assessment t'at ackno#ledges t'e di&&erent &unctions o& assessment and it 'el4s to see
t'ese as com4lementary rat'er t'an "eing in o44osition to eac' ot'er.
,* "++roaches to assessment
+'e di&&erent 4ur4oses o& assessment lead to di&&erent a44roac'es to assessment.
+raditionally assessment o& language as sc'ool su"ject took a ery sim4le &ormA 4u4ils
#ere gien a narro# #ritten task #'ic' #as t'en a#arded a grade or mark. +'is allo#ed
t'em (and t'e teac'er) to make a judgement o& 'o# t'ey ranked in relation t'e rest o& t'e
grou4 (normatie assessment). ;o#eer t'e a"sence o& clear criteria meant t'at t'e
in&ormation rarely gae an indication o& 'o# t'ey could make 4rogress in t'eir learning.
@lso t'e test itsel& o&ten em"odied a ery narro# conce4tion o& #'at com4etence in
language entailedJ it o&ten centred on kno#ledge o& language &orm and structures (synta>
and grammar) and a narro# range o& language uses (o&ten only a #ritten &orm o& essay).
+'e im4licit understanding o& #'at reading literacy inoled #as also ery narro#, o&ten
inoling just decoding and literal understanding. *n traditional a44roac'es o& t'at kind
t'ere #as unlikely to "e any attention to oral #ork, to a range o& #riting 4ur4oses, to a
#ide range o& reading and res4onse to reading. ,n t'e ot'er 'and, t'e adantage o& a
&airly narro# a44roac' to assessment #as t'at is #as easier to 4roide relia"le outcomesJ
t'e more com4le> t'e system o& assessment "ecomes, t'e more di&&icult it is to ensure
t'at t'e a#ard o& grades or marks &or 4articular outcomes are consistent.
*n a44roac'es to assessment, t#o central tendencies emerge #'ic' are releant to
language as su"ject. ,ne 4laces em4'asis on t'e assessment of learning #'ere relia"le,
o"jectie measures are a 'ig' 4riority. +'e &ocus 'ere is on making summatie judgements
#'ic' in 4ractice is likely to inole more &ormal e>aminations and tests #it' marks
sc'emes to ensure t'at t'e 4rocess is sound. @n alternatie a44roac' is to c'ange t'e
em4'asis &rom assessment of learning to assessment for learning, im4lying a more
&ormatie a44roac' #'ere t'ere is muc' more em4'asis on &eed"ack to im4roe
4er&ormance. +'e a44roac' 'ere mig't "e t'roug' course #ork and 4ort&olio assessment in
#'ic' dierse in&ormation can "e gat'ered #'ic' re&lects t'e true "road nature o& t'e
su"ject.
)8
Portfolio assessment 'as a num"er o& adantages &or language as sc'ool su"jectJ &or
e>am4le, it can motiate and em4o#er t'e learner, it can 4roide sam4les o& 4er&ormance
collected oer time, eidence o& use and a#areness o& 4rocess. Port&olio assessment
incor4orates eidence deried &rom more realistic tasks in meaning&ul conte>ts, rat'er
t'an relying on arti&icial, deconte>tualised tasks undertaken in timed conditions. @ &urt'er
adantage o& t'is a44roac' is t'at it can em"ody di&&erent &orms o& sel&.assessment #'ic'
can also "e 'el4&ul #ays o& motiating learners and 'aing t'em re&lect on t'eir 4rogress.
-elf.assessment encourages 4u4ils to take res4onsi"ility in t'e learning 4rocess alt'oug' it
is adisa"le &or t'em to "e trained in sel&.assessment tec'niBues &or t'is to #ork
e&&ectiely. +'e di&&iculty #it' 4ort&olio assessment i& it is conceied only as t'e
accumulation o& eidence 4roduced in in&ormal settings is t'at is does not easily satis&y
demands &or relia"ility. Work #'ic' 'as "een 4roduced oer an e>tended 4eriod o& time,
#it' &ormatie guidance &rom t'e teac'er and colla"oration #it' classroom 4eers is not
al#ays conincing eidence o& com4etence.
+'e "roader t'e a44roac' to assessment (incor4orating t'e judgement o& a range o&
di&&erent 4er&ormances in di&&erent conte>ts), t'e more it can "e said to constitute a
meaning&ul assessment o& 4er&ormance in t'e su"ject. ;o#eer, as suggested, tension
emerges "ecause it is sometimes di&&icult to com4are, #it' any degree o& accuracy, t'e
results dra#n &rom "road a44roac'es to assessment. +'e Buest &or ?o"jectie= and relia"le
met'ods o& assessment drien "y narro# ideas o& accounta"ility "rings #it' it a num"er o&
dangers. !o called ?teac'ing to t'e test= may not "e a 4ro"lem i& t'e tests are
so4'isticated and #ide.ranging "ut t'ere may "e 4ractical di&&iculties in administering
t'ose t'at are too com4le>. *& t'e tests are too narro# and sim4listic t'en t'is may 'ae
an aderse e&&ect on t'e teac'ing. ;ere t'en is one source o& 4olarised o4inions, one
stressing t'e im4ortance o& o"jectiity, relia"ility and summatie judgement, t'e ot'er
more tolerant o& su"jectiity in order to ensure t'at t'e assessment a44roac' is &ait'&ul to
t'e com4le>ity o& t'e aims.
@ key conce4t is em"odied in t'e notion o& /trans+arency0, t'e ie# t'at t'ose "eing
assessed are a#are o& t'e criteria #'ic' are "eing used to make judgements a"out t'em
and 'o# t'ose judgements are made. (no#ledge o& criteria can 'el4 4er&ormance and
im4roe motiation "ut once again, in t'e conte>t o& language as su"ject, t'e issues are
more com4le> t'an t'ey &irst seem. @ common assum4tion is t'at 4u4ils learn "est #'en
t'ey kno# #'at t'ey are trying to ac'iee and #'y. W'ile t'is ie# is largely true, t'ere
are e>ce4tions. Iecause t'e deelo4ment o& language can in some #ays "e descri"ed as a
?natural= 4rocess learners do not al#ays need to "e &ully &ocused on s4eci&ic as4ects o&
t'eir 4er&ormance in order to im4roe. *n &act too muc' &ocal a#areness on 4er&ormance
can make t'em too sel&.consciousA s4eakers can a44ear too groomed and arti&icialJ t'e
#riter #'o 'as "een told to strie &or e&&ect "y using more adjecties may deelo4 a 'ig'ly
arti&icial and a#k#ard style. +'ese insig'ts do not negate t'e im4ortance o& trans4arency
as a 4rinci4le "ut 'ig'lig't t'e &act t'at in 4edagogical 4ractice t'e 4rinci4le needs to "e
inter4reted and im4lemented #it' care.
+'e use o& com4etences to descri"e outcomes #'ic' can "e assessed 'as deelo4ed
considera"ly in recent years. *t is #ort' noting t'at t'e term /com+etence0 is used "y
#riters in di&&erent #ays #'ic' can "e a source o& con&usion. !ometimes it is used in a ery
general #ay as a synonym &or ?a"ility= or ca4a"ility as in ?language com4etence=. ,t'er
#riters use t'e term to descri"e "road language modes or domains suc' as reading,
#riting, s4eaking and listening. More commonly 'o#eer ?com4etences= re&ers to t'e
s4eci&ic actions #'ic' a learner must 4er&orm and #'ic' in turn can "e assessed to
demonstrate ac'ieement in a su"ject. @docates o& using com4etence statements &or
assessment 4ur4oses and sylla"us design see t'eir alue largely in "ringing clarity and
trans4arency to t'e s4eci&ication o& learning outcomes. )ritics o& a com4etence a44roac'
take t'e ie# t'at 4er&ormance statements are too narro# and s4eci&ic, and do not re&lect
t'e range and su"tlety o& #'at is inoled in language deelo4ment. +'ere are 4arallels
))
'ere #it' di&&erences o& o4inion oer t'e relatie merits o& 4ort&olio and &ormal testing.
+'e "alance o& adantages and disadantages needs to "e considered.
,ne o& t'e c'allenges 4osed "y assessing language as sc'ool su"ject (L!) is t'at t'e
content is so aried and com4le> (see t'e 4a4er "y Florentina !Lmi'aiMn
D
) #'ic' is in turn
a re&lection o& t'e com4le>ity o& t'e aims. 3ac' mode o& ?#riting=, ?reading=, ?s4eaking
?and ?listening= can "e "roken do#n into &urt'er areas. @ su"ject t'at is so
multidimensional raises t'e Buestion as to #'et'er an assessment task in one area is
re4resentatie o& ac'ieement in t'e su"ject as a #'ole. For e>am4le, it is &airly sa&e to
assume t'at 4er&ormance in s4eaking is not necessarily indicatie o& reading com4etence.
,n t'e ot'er 'and it is less clear #'et'er it is necessary to assume t'at reading a"ility
aries in relation to te>ts o& di&&erent ty4es (&iction, non.&iction, media). +'e assessment
o& reading can easily remain at a sur&ace leel only addressing recall or literal
com4re'ension rat'er t'an dee4er understanding. Multi4le c'oice Buestions on a te>t are
easy to mark and may yield 'ig' relia"ility (in t'e tec'nical sense) "ut are less eBui44ed to
assess t'e learner=s dee4 and indiidual res4onse to a te>t. 3en #it' ery young c'ildren
t'e reading 4rocess is more t'an sim4ly decoding te>t and any system o& assessment needs
to re&lect t'at &act. *t is t'e com4le>ity o& t'e su"ject t'at accounts &or t'e 'ostility
teac'ers o& languages as sc'ool su"ject sometimes e>4ress to#ards large.scale and &ormal
testing "ecause only a &airly narro# range o& com4etences can "e assessed in a single test.
@ similar Buestion arises in relation to #riting. Do 4u4ils need to "e assessed on a range o&
di&&erent tasks re&lecting t'e &act t'at #riting e>ists &or di&&erent 4ur4oses and &or
di&&erent audiencesC +'e assessment o& #riting a44ears to "e more straig't&or#ard t'an
t'at o& reading and s4eaking "ecause at least t'ere is al#ays a 4roduct #'ic' can "e
re&erred to a&ter t'e eent. Iut t'ere is a major c'allenge in determining #'at criteria
s'ould dominate in making a judgement (&or e>am4le t'e accuracy o& t'e #riting as
o44osed to t'e im4act o& t'e content) and #'et'er t'e criteria s'ould c'ange in relation
to di&&erent ty4es o& #riting. *t is also #rong to assume t'at t'e only res4onse to 4u4ils=
#riting comes #'en it is com4leted. Formatie assessment in t'e &orm o& a dialogue a"out
t'e #ork in 4rocess is an im4ortant #ay o& im4roing standards. @ssessment is al#ays a
selection and t'ere&ore can un#ittingly alue some as4ects o& t'e su"ject more t'an
ot'ers.
!4eaking and listening is e>tremely di&&icult to assess "ecause, een more t'an ot'er
as4ects o& L!, 4er&ormance aries #it' t'e t'eme, conte>t and leel o& motiation. Poor
4er&ormance in oral #ork is o&ten to do #it' t'e nature o& t'e task #'ic' 'as "een
deised, t'e atmos4'ere o& t'e classroom and t'e dynamics o& t'e grou4 rat'er t'an t'e
com4etence o& t'e 4u4ilsJ to 4roide a alid assessment o& s4eaking and listening,
eidence needs to "e dra#n &rom a ariety o& situations. !ome mig't argue t'at t'e
assessment o& s4eaking and listening is so com4le> and conte>t s4eci&ic t'at it s'ould not
"e assessed &ormally. ;o#eer "ecause assessment so o&ten determines t'e curriculum and
t'e #ay it is taug't t'ere are arguments to suggest t'at s4eaking and listening s'ould "e
assessed des4ite t'e di&&icultiesJ t'e a"ility to articulate a 4oint o& ie# orally and to
argue a case are essential skills &or meaning&ul 4artici4ation in a democracy.
"n ideal assessment system would reflect the full com+le1ity of language as school
su23ect (L-)& and would motivate learners 2y giving useful feed2ac$& while also
+roviding other sta$e.holders (e*g* +olicy.ma$ers and em+loyers) with the information
they need* "n integrated a++roach to assessment would ensure that the different
+ur+oses and a++roaches are 2alanced so that no one +riority has adverse and undue
influence on the system as a whole*
3
!Lmi'aiMn F. ontent considerations for a framework of reference for Language!s" of School Education#
###.coe.int$lang

)2
4* Language(s) of Education (LE)
!o &ar consideration 'as "een gien to t'e c'allenges 4resented "y assessment o& ?language
as sc'ool su"ject= (L!). *s t'ere any adantage in considering t'e assessment o& L! #it'in a
"roader &rame#ork o& language(s) o& education (L3) #'ic' incor4orates ?language across
t'e curriculum= (L@)) and &oreign language learning (FL)C ?Languages(s) o& education
conceied in t'at #ay is not a su"ject "ut an um"rella constructJ it is &airly clear t'ere&ore
t'at t'e notion o& e>amining or testing language(s) o& education is entirely ina44ro4riate.
;o#eer it is conceia"le to 4ro&ile a 4u4il=s com4etence in language(s) o& education "y
assem"ling t'eir com4etences and ac'ieements in a range o& domains. +'e 4resent
)ouncil o& 3uro4e=s 3uro4ean Language Port&olio (3LP) is a use&ul model to demonstrate
t'e ty4e o& instrument #'ic' could "e considered. +'is a44roac' 'as a num"er o& 4otential
adantages. *t takes seriously t'e conce4t o& 4lurilingualism in t'e sc'ool conte>tA a 4u4il
may "e a lo# ac'ieer in language o& sc'ooling taug't as sc'ool su"ject "ut "e &luent in
t#o or more ot'er languages. +'us t'e de&icit model #'ic' is o&ten a44lied to assessment
o& L! is re4laced "y a more 4ositie recognition o& ac'ieement. *t #ould still "e im4ortant
to "e a"le to identi&y an indiidual=s com4etence in as4ects o& L! as a com4onent #it'in a
"roader 4ro&ile, "ut "eing a"le to situate t'at descri4tion o& com4etence in a larger
conte>t could 'ae a 4ositie im4act on motiation and sel&.esteem.
)ould 4ort&olio assessment e>tend to em"race language com4etence in ot'er su"jects, to
include t'e dimension o& language across t'e curriculumC +'is is a 4ractical c'allenge "ut
4er'a4s not insurmounta"le. !ome uses o& language (e.g. giing 4resentations, #riting
&ormal re4orts, reading &or in&ormation) are clearly reBuired and demonstrated #it'in
di&&erent su"jects and it is not inconceia"le t'at t'ose su"jects s'ould make a
contri"ution to a 4u4il=s oerall language 4ro&ile. +'ere is an argument to suggest t'at i&
deelo4ing language com4etences across t'e curriculum is to "e taken seriously it must
'ae some im4act on 'o# language use is assessed. %nderlying Buestions 'ae to do #it'
su"ject "oundaries and t'e degree to #'ic' com4etence in language use can "e easily
se4arated &rom t'e conte>t in #'ic' it is usedJ learners o& 'istory could also in some sense
"e said to "e learning t'e language o& 'istory.
5* Integrated assessment
@s descri"ed earlier a key c'allenge and source o& tension in relation to assessment is to
satis&y t'e di&&erent 4ur4oses assessment is e>4ected to &ul&il and to 4roide a44ro4riate
in&ormation &or t'e di&&erent interest grou4s. @s suggested, teac'ers o& language as sc'ool
su"ject are sometimes 'ostile to large.scale, &ormal testing "ecause o& t'e 4erceied
narro#ing o& t'e su"ject. +'ere are a num"er o& 4oints to consider in relation to t'is ie#.
+'e testing ?industry= 'as "ecome increasingly so4'isticated in recent years and designers
o& assessment tasks and items are more ade4t at addressing issues o& alidity and
relia"ility. +'at does not mean t'at &ormal tests can address all language com4etences "ut
once t'at &act is recognised t'en t'e results o& suc' tests may make a use&ul contri$ution
to t'e oerall 4ro&ile o& a 4u4ils= com4etence in language use. Large.scale testing "ot' at
national and international leel is a &act o& modern li&e. Policy makers need in&ormation on
leels o& 4ro&iciency ac'ieed "y grou4s in sc'ools, education aut'orities or countries. *t is
4ossi"le to disc'arge t'is ?ealuatie= &unction o& assessment "y a 4rocess o& sam4ling
rat'er t'an "y using summatie data deried &rom an entire 4o4ulation, "ut it is unlikely
t'at t'e Buest &or in&ormation o& t'is kind #ill diminis'. W'en assessment data is used to
com4are t'e 4rogress o& di&&erent co'orts o& 4u4ils, t'e use o& a ?alue.added= a44roac'
#'ic' takes account o& t'e di&&erent "ase.lines &rom #'ic' t'e 4u4ils are 4rogressing is
"ecoming more common. @ll o& t'is in&ormation can "e in&ormatie &or teac'ers as long as
t'e limitations o& t'e data are recognised.
+'e conce4t o& an integrated a++roach to assessment is intended to counter t'e tendency
to#ards 4olarised ie#s. @s suggested a"oe, 4ort&olio assessment 'as many adantages
)3
"ut #it'out in&ormation "ased on some &orm o& ?controlled= 4er&ormance it #ill struggle to
make conincing claims to relia"ility. +'e incor4oration o& s4eci&ic results deried &rom
a44ro4riate tests designed to assess s4eci&ic (not all) language com4etences into a
4ort&olio can 'el4 strengt'en "ot' t'e alidity and relia"ility o& t'e assessment a44roac'.
+'e key may "e to t'ink "roadly in terms o& an assessment strategy #'ic' makes use o& a
ariety o& assessment tools, rat'er t'an assuming t'at one assessment tool #ill &ul&il all
4ur4oses.
@ sam4le outline o& a Languages o& 3ducation Port&olio "ased on t'e 4resent )ouncil o&
3uro4e model is gien "elo#A
)9
Languages of Education Portfolio
Aims:
to ("o!ide a "eco"d of achie!ement in all languages of education
to ("o!ide e!idence of de!elo(ing language com(etences needed fo" democ"atic
citi;enshi(
to moti!ate lea"ne"s to e<tend thei" "ange of language com(etences
to ensu"e that the language needs of all (u(ils a"e =eing add"essed
'nt"oducto"# ("ofile of de!elo(ing
com(etence in a "ange of languages
0 linked to othe" assessment
outcomes.
Language
across
Curriculum
(LAC)
E!idence of
language
com(etence
f"om othe"
su=.ects: e.g.
!ideo cli( of
("esentation in
geog"a(h#>
discu"si!e
w"iting in
histo"#>
technical "e(o"t
f"om science.
Language as School Subject (LS)
E<am(les:
"eading log inco"(o"ating lite"atu"e and
non?fiction> e!idence of "eading fo"
diffe"ent (u"(oses> w"iting sam(les> !ideo
cli( of discussion of lite"a"# te<ts> self?
assessment.
Other
Languages
(FL etc.)
A !a"iet# of
s(ecific test
"esults =oth local
and national.
E!idence of
achie!ing
th"eshold
com(etences
th"ough
"ecognised
"e(o"ting
mechanisms.
$etails of
ce"tificates,
di(lomas etc.
Language Biograh!
1e"sonal language auto=iog"a(h#
inco"(o"ating meta?cognition of language
("ocesses 0 dialect and accent, language
and identit#, etc.
Additional E"idence
)5
)6
!he 6elevance of International "ssessment for the develo+ment of a %ramewor$
for the Languages of Education
7os8 9oi3ons& CI!:& !he 9etherlands
Introduction
*n an ideal educational system educationists #it' di&&erent "ackgrounds sit toget'er #'en
ne# learning 4rogrammes, curricula, e>amination sylla"i and suc' like are "eing
deelo4ed. *n 4ractice it may 'a44en t'at testing e>4erts are called in rat'er late, #'en
descri4tors, targets, aims, leels etc 'ae already "een &ormulated. *n t'is 4a4er #e 'o4e
to illustrate 'o# use&ul it can "e to look at t'e ac'ieements in *nternational @ssessment
#'en deelo4ing a )ouncil o& 3uro4e=s Frame#ork &or t'e Languages o& 3ducation
(re&erred to 'ere as an LE Framework). We #ill discuss #'at it is and does, #'at its salient
c'aracteristics are, "ut #e #ill also d#ell on its limitations. *t is muc' a44reciated t'at
t'e )ouncil o& 3uro4e 'as inited testing e>4erts to take 4art in t'e deelo4ment o& an L3
Frame#ork. We 'o4e t'at t'e testing e>4ertise can "e o& use 'ere, &or e>am4le t'roug'
s'o#ing e>am4les o& good and releant 4ractices in language testing.
)* What is International "ssessment;
*nternational @ssessment (*@) is assessment t'at allo#s &or educational ac'ieement to "e
com4ared across countries. *t is concerned #it' measuring trends or 4rogress. *@ may
identi&y 4ercentages o& students in countries #'o meet standards &or a gien su"ject. *t
'el4s stake'olders in education : t'e teac'ing 4ro&ession and 4olicy makers : to measure
t'e success o& t'eir country=s educational 4olicies. *@ 'el4s to make sure t'at c'ildren #ill
'ae t'e kno#ledge and skills necessary to "e 4roductie mem"ers o& an im4roing
economy and e&&ectie citi6ens o& a deelo4ing democracy
E
. *@ is cyclicalA de4ending on
t'e 4rogramme measurements are re4eated eery t'ree or &ie years. ,n t'e "asis o& t'e
outcomes it can "e s'o#n i& countries 'ae 4rogressed com4ared to t'eir 4er&ormances
during earlier cycles, in a"solute terms and also in relation to ot'er countries.
,* Who is tested;
*n t'e t#o *@ 4rogrammes #e #ill re&er to 'ere, t'e Programme &or *nternational !tudent
@ssessment (P*!@) and t'e Progress in *nternational 7eading Literacy !tudy (P*7L!),
samples o& students are tested. +'ese sam4les can "e eit'er grade. or age."ased. *n t'e
case o& P*7L!, t'ey are &ourt'.grade students, roug'ly eBuialent to nine and ten.year old
students. *n t'e case o& P*!@, t'ey are /F.year.old students. +'ese grades and age.grou4s
'ae "een c'osen &or a 4articular reason. *n many countries grade.&our students are a"out
to make t'e c'ange &rom learning to read to reading to learn
F
. ,&ten t'ere #ill "e a
c'ange in t'e teac'ing circumstances. *n grades one to &our one teac'er teac'es all or
most su"jects, &rom grade &ie on#ards in many systems t'ere #ill "e su"ject s4eci&ic
teac'ers. *n t'e case o& P*!@, /F.year.old students are a"out to end com4ulsory education
and t'ey are no# su44osed to 'ae mastered t'ose skills t'at #ill ena"le t'em to &unction
inde4endently in society.
+'ere are more di&&erences "et#een t'e t#o *@ 4rogrammesA P*!@ 'as a t'ree.year cycle
and P*7L! 'as a &ie.year cycle. P*7L! only assesses reading literacy, P*!@ assesses reading
literacy, mat'ematical literacy and science literacy. @lt'oug' "ot' 4rogrammes s'are t'e
assessment o& reading literacy, t'eir construct o& #'at reading constitutes di&&ers. We s'all
discuss t'ese di&&erences in more detail later.
9
!ee t'e N@3P 4rogrammeA National @ssessment o& 3ducational Progress,
###.ed.go$4rograms$nae4$inde>.'tml
5
!ee t'e P*7L! @ssessment Frame#orkA ###.timss."c.edu$4irls100G$&rame#ork.'tml
)*
4* !he use of international assessment
@s #e indicated, *@ 'el4s t'e teac'ing 4ro&ession and 4olicy makers to ealuate t'e
success o& a country=s educational 4rogrammes. *t is not used to assess indiidual students.
!am4les o& students t'roug'out t'e years are assessed and een t'oug' data are collected
&rom indiidual students, not all students are administered t'e same test. +'us
4er&ormances o& indiidual students are not usually com4ared to eac' ot'er. +'e si6e o&
t'e sam4le usually does not een allo# indiidual schools to "e com4ared to eac' ot'er.
5* !he collection of 2ac$ground varia2les
Iot' *@ 4rogrammes do not only collect data t'roug' t'e assessment o& cognitie skills (in
t'is 4articular caseA reading literacy), t'ey also collect data on "ackground aria"les.
+'ese "ackground aria"les may "e student aria"les suc' as gender, age (in P*7L!),
academic success, motiation, &amily circumstances (languages are s4oken at 'ome,
social.economic status), e>4osure to reading te>ts, attitudes to reading etcJ school
aria"lesA location, si6e, eBui4ment, num"er o& sta&&, num"er o& students etc. Iackground
aria"les may also relate to educational 4olicy or to teac'ing met'ods, to mention some
more. Data are collected t'roug' Buestionnaires t'at may "e administered to students,
4arents, teac'ers, sc'ool 4rinci4als, educational 4olicy makers or ot'ers.
,nce data 'ae "een collected "ot' on t'e cognitie tests and on t'e Buestionnaires,
countries may like to relate student 4er&ormance on t'e tests to t'e "ackground aria"les.
3s4ecially mallea"le &actors, "ackground aria"les t'at can "e c'anged in t'e lig't o& t'e
outcomes on t'e cognitie tests, #ould seem to "e o& s4ecial releance in t'e conte>t o&
t'e )ouncil o& 3uro4e=s "rie& and an L3 &rame#ork in 4articular. *& re4resentatie sam4les
o& identi&ia"le grou4s o& students under.4er&orm on cognitie tests t'ere may "e reasons to
look at t'e "ackground aria"les o& t'ese grou4s and &ind out i& t'ere are signi&icant
correlations. *& a t'eory can "e deelo4ed t'at #ould 4redict an im4roement in
4er&ormance t'roug' a c'ange in "ackground &actors, a country mig't consider to c'ange
releant elements in its educational 4olicy. +'is latter ty4e o& researc' can "e ery
interesting and reealing &or countries. ;o#eer, &rom *@ 4rogrammes in 4rogress #e kno#
t'at researc' into t'ese areas can "e Buite sensitie.
<* Constructs of Language Proficiency
Iot' t'e P*7L! and t'e P*!@ 4rogrammes 'ae deelo4ed @ssessment Frame#orks. +'ese
contain de&initions o& t'e construct o& #'at is tested (reading literacy), among ot'ers. ,&
course t'e t#o 4rogrammes di&&er in t'eir constructs &or reasons gien a"oe.
For t'e P*7L! assessment, reading a"ility &or &ourt'.grade students is de&ined asA
%the a$ility to understand and use those written language forms re&uired $y
society and'or valued $y the individual( )oung readers can construct meaning from
a variety of te*ts( +hey read to earn# to participate in communities of readers in
school and every day life# and for en,oyment(
For t'e P*!@ assessment, t'e construct o& reading literacy &or /F.year.old students is
de&ined as &ollo#sA
-eading literacy is understanding# using# and reflecting on written tests# in order
to achieve one.s goals# to develop one.s knowledge and potential# and to
participate in society.
W'at is im4ortant 'ere is t'at "ot' *@ 4rogrammes define t'e construct to "e assessed,
t'ey de&ine #'at t'ey consider reading literacy to "e. +'is is releant in t'e conte>t o& t'e
deelo4ment o& an L3 Frame#ork. +'ose #'o are &amiliar #it' t'e )ouncil o& 3uro4e=s
&rame#ork &or &oreign language learning, teac'ing and assessment, t'e )ommon 3uro4ean
Frame#ork o& 7e&erence ()3F7), #ill kno# t'at t'at &rame#ork does not o&&er a construct
de&inition o& &oreign language com4etence. 7at'er, it de&ines leels o& com4etence
)7
t'roug' descri4tors, t'roug' t'e so.called can.do statements. )ritics considered t'is to "e
one o& t'e serious #eaknesses o& t'e )3F7.
=* !o what language(s) does reading literacy in the PI6L- and the PI-" +rogrammes
refer to;
*n &act in neit'er o& t'e *@ 4rogrammes= constructs e>4licit re&erence is made to t'e
language in #'ic' reading literacy is assessed. *t is le&t to t'e countries 4artici4ating to
c'oose #'ic' language(s) t'is$t'ese #ould 'ae to "e. *t #ould seem t'at in "ot'
4rogrammes language as a sc'ool su"ject (L!) may "e im4lied, i& in t'at language t'e skill
o& reading is taug't. ;o#eer, strictly s4eaking t'e language itsel& is not 4art o& t'e
constructJ it is a"out reading per se( *n most countries t'is means t'at t'e tests are
administered in t'e language(s) o& instruction. *n t'e case o& P*!@, attem4ts 'ae "een
made to assess reading in t'e conte>t o& language across t'e curriculum, as 4art o& an
um"rella construct, not so muc' #it' re&erence to &oreign languages "ut #it' re&erence to
mat' and science. +'e reason &or assessing reading in t'is #ay is t'at cross.curricular skills
are t'oug't to "e life/skills t'at modern education systems o&ten strie to train t'eir
students in.
>* "ssessment %ramewor$s
Iot' 4rogrammes &urt'er de&ine t'e construct t'ey are testing. We gie e>am4les &rom
P*!@ 'ere, "ut P*7L! 'as com4ara"le de&initions. *n t'e 4rocess o& reading, P*!@
distinguis'es a num"er o& com4onents t'at can "e mani4ulated in t'e testingA t'e reading
situation, t'e structure o& t'e te>t and t'e c'aracteristics o& t'e Buestions (items) t'at are
asked a"out t'e te>t. 3ac' o& t'ese &actors is &urt'er s4eci&ied.
G
+e>t Format
*n t'e P*!@ tests, a distinction is made "et#een t#o te>t ty4esA
)ontinuous te>ts, ty4ically com4osed o& sentences t'at may "e organised in turn into
and 4aragra4's, sections, c'a4ters and $or "ooksJ
Non.continuous te>ts, suc' as c'arts and gra4's, ta"les and matrices, diagrams, ma4s,
&orms, in&ormation s'eets, calls and adertisements, ouc'ers and certi&icates.
7eading 4rocesses
*n t'e P*!@ tests &ie reading 4rocesses are distinguis'edA
7etrieing in&ormationJ
Forming a "road general understandingJ
Deelo4ing an inter4retationJ
7e&lecting on and ealuating t'e content o& a te>tJ
7e&lecting on and ealuating t'e &orm o& a te>t.
*tem ty4es
*n t'e P*!@ 1000 and 100D studies, around ED 4er cent o& t'e reading literacy tasks #ere
o4en constructed.res4onse items #'ic' reBuired judgment on t'e 4art o& t'e marker. +'e
remaining tasks consist o& closed constructed.res4onse items t'at reBuire little judgment
on t'e 4art o& t'e marker, as #ell as sim4le multi4le.c'oice items, &or #'ic' students
c'oose one o& seeral alternatie ans#ers, and com4le> multi4le.c'oice items, &or #'ic'
students c'oose more t'an one res4onse.
6
!ee t'e P*!@ 7eading Frame#orkA ###.4isa.oecd.org
)9
!ituations
*n t'e P*!@ tests t'e &our situation aria"les as identi&ied in t'e )3F7 as domains of
language use 'ae "een taken oerA
7eading &or 4riate use (4ersonal)J
7eading &or 4u"lic useJ
7eading &or #ork (occu4ational)J
7eading &or education.
?* Levels in 6eading Literacy
P*!@ also distinguis'es leels : or standards : o& reading 4ro&iciency. *tems 'ae "een
scaled (on t'e "asis o& t'e res4onse data) &rom less di&&icult to more di&&icult. +'e
mem"ers o& t'e P*!@ reading e>4ert grou4 and test deelo4ers identi&ied a num"er o&
aria"les t'at can in&luence t'e di&&iculty o& any reading task. +'e 4rogression o& di&&iculty
in t'e set o& test items #as ca4tured in a com4osite reading literacy scale #it' t'ree
reading 4rocesses (retrieving information# interpreting te*ts and reflecting and
evaluating). For eac' o& t'e t'ree su"scales &ie leels #ere identi&ied. 3>4ert 4anels
judged t'at t'e tasks #it'in eac' leel o& reading literacy s'ared many task &eatures and
reBuirements, and di&&ered systematically &rom tasks at 'ig'er and lo#er leels. *n t'is
#ay a reading literacy leel ma4 could "e dra#n u4.
+'is ma4 some#'at resem"les t'e leels as distinguis'ed in t'e )3F7. ;o#eer, t'e P*!@
leels are "ased on response data &rom students ans#ering t'e Buestions. Judges in P*!@
grou4ed tasks #it'in leels and c'aracteri6ed t'e leels on t'e "asis o& t'e content o& t'e
items. *n t'e case o& t'e )3F7 t'e descriptors 'ae "een scaled and leels 'ae "een
&ormulated in terms o& t'ese descri4tors. 9alidation #as done t'roug' com4arison #it'
student 4er&ormance. *tems in )3F7.related tests 'ae "een deried &rom t'ese
descri4tors. *n t'at #ay t'e )3F7 a44roac' di&&ers considera"ly &rom 4rocedures in *@.
@* Aias in I"
@s 4art o& *@, researc' 'as "een carried out into t'e 4'enomenon t'at t'e same item
seems to measure di&&erent t'ings in di&&erent countries. +'us it 'a44ens t'at t'is same
item is &ound to "e easy &or #eak students in one country and more di&&icult &or more a"le
students in anot'er country. +'is 4'enomenon is called Di&&erential *tem Functioning (D*F).
For t'e sake o& alidity suc' items tend to "e remoed &rom a testing system. 5et t'ey may
teac' us somet'ing a"out t'e di&&erences in education 4olicies and testing 4ractices
"et#een countries, muc' in t'e same #ay as data &rom "ackground Buestionnaires may tell
us t'at. Iot' a44roac'es may 'el4 us to deelo4 t'eories #'y t'ere are di&&erences
"et#een 4er&ormances "et#een countries. +'is in turn can 'el4 countries to c'ange
elements in t'eir educational systems i& t'ey so #is'.
)B* !he limitations of I" with res+ect to the develo+ment of the LE %ramewor$
We must "e a#are t'at most *@ is limited in its aims. +'ese are limitations as to t'e skills
t'at are testedA rece4tie skills, rat'er t'an 4roductie skills, and to t'e age grou4s
tested. +'ere are some national assessment systems t'at do more, "ut t'eir limitation is
t'at only one educational system is sureyed. For an international L3 Frame#ork t'e
&ormat o& national assessments and t'eir data may "e too s4eci&ic.
28
))* !he im+ortance of I" with res+ect to the develo+ment of an LE %ramewor$
+'e most releant &eature o& *@ &or an L3 Frame#ork #ould seem to "e t'at it is "ased on
an assessment &rame#ork t'at incor4orates a de&inition o& t'e construct its tests are "ased
on. +'e ne>t im4ortant 4'enomenon in t'is conte>t is t'at t'is &rame#ork and construct is
acce4ted "y a great many countries (commitment is essential), ot'er#ise countries #ould
not take 4art in t'e surey. 3en i& t'ese sureys contain lo#.stakes tests &rom a student=s
4oint o& ie#, &or t'e countries= educationalists t'ese sureys are 'ig'.stakes. +'ey #ould
not acce4t data &rom 4rogrammes #'ose &undamentals t'ey did not agree #it'. *n t'e case
o& P*!@ 100H, some GF countries are 4artici4ating. *t t'ere&ore #ould seem to "e 4ossi"le
to deelo4 an L3 Frame#ork t'at is acce4ta"le to a large num"er o& mem"er states o& t'e
)ouncil o& 3uro4e, certainly i& it is "ased on t'e 4rinci4les o& t'e )ouncil o& 3uro4e t'at all
mem"er states 'ae signed u4 &or.
*t goes #it'out saying t'at t'e international language testing community is 'a44y to "e
inoled in t'e deelo4ment o& an L3 Frame#ork at an early stage. *& t'at &rame#ork is to
"e an in&luential 4olicy instrument it #ill "e use&ul to deelo4 instruments t'at can test
t'e e>tent to #'ic' countries ac'iee t'e aims (deried &rom t'e Frame#ork) t'ey 'ae
set t'emseles. Iut #e must remem"er t'at #e cannot 4roduce tests on t'e "asis o& aims,
goals, standards t'at are not translata"le into assessment targets.
),* 6ecommendations for the develo+ment of an LE %ramewor$
%sing t'e e>4eriences o& some 4rogrammes o& *nternational @ssessment, it #ould "e
adisa"le to 4ay attention to t'e &ollo#ing #'ile deelo4ing an L3 Frame#orkA
De&ine t'e construct o& com4etence in Languages o& 3ducation.
De&ine #'at conce4ts in t'e )ouncil o& 3uro4e=s )'arter t'is construct is "ased u4on.
!eek su44ort &rom mem"er states in acce4ting t'e construct o& com4etence in
Languages o& 3ducation.
De&ine #'at skills learners need to acBuire to "e com4etent in Languages o& 3ducation.
*denti&y key stages in t'e acBuisition o& com4etence in Languages o& 3ducation.
!4eci&y skills and su".skills at key stages in t'e acBuisition o& com4etence in Languages
o& 3ducation.
De&ine standards t'at ty4ically s'ould 'ae "een reac'ed at key stages in t'e
acBuisition o& com4etence in Languages o& 3ducation.
@ctiities suc' as t'e a"oe may result in an L3 Frame#ork t'at #ill "e c'allenging &or
language testers to 4roduce alid tests &or. +'ese tests can "e used "y mem"er states to
gauge to #'at e>tent t'ey 'ae ac'ieed t'e goals o& t'e )ouncil o& 3uro4e in t'e area o&
Languages o& 3ducation.
2)
22
"ssessing com+etences at the end of com+ulsory schooling C the Polish case
Waldemar #artyniu$& 7agiellonian 'niversity& (ra$Dw& Poland
Introduction
+'e aim o& t'is 4a4er is to 4resent a system o& com4ulsory e>ternal e>aminations #'ere
com4etences in a language o& sc'ooling : Polis' : are assessed. *t is an illustration o& 'o#
t'e c'allenges o& assessing com4etence in L! and L@) are "eing met in 4ractice. +'e 4a4er
starts #it' a "rie& introduction to t'e Polis' system o& education, continues #it' an
oerie# on t'e national system o& e>ternal e>aminations and concludes #it' a "rie&
4resentation o& one o& t'e e>aminations : t'e Lo#er.!econdary !c'ool 3>amination
(Eg0amin gimna0,alny).
)* Education in Poland
*n accordance #it' t'e !c'ool 3ducation @ct o& 2 !e4tem"er /HH/ t'e Polis' education
system com4rises 4re.sc'ool institutions, 4rimary sc'ools, lo#er.secondary and u44er.
secondary sc'ools. *nstitutions o& 'ig'er education &orm a se4arate 'ig'er education
system or sector.
@ c'ild aged D to F may receie 4re.sc'ool education, #'ic' is not com4ulsory. *n t'e
sc'ool year 100E$0F a one.year o"ligatory 4re.sc'ool 4re4aration &or G.year.olds #as
introduced, to "e 4roided eit'er "y kindergartens or 4rimary sc'ools.
)'ildren aged 2./D attend a G.grade 4rimary sc'ool (S0ko1a podstawowa). 3ducation in
4rimary sc'ools is diided into t#o stagesA t'e &irst stage (grades / to D) o&&ering su"ject.
integrated learning and t'e second stage (grades E to G) at #'ic' su"ject teac'ing is
4roided. @t t'e end o& t'e G.year 4rimary sc'ool 4u4ils take t'eir &irst e>ternal
interdisci4linary com4etence test. +'is test #as introduced in @4ril 1001.
@ll 4rimary.sc'ool leaers continue t'eir education in a D.year 2imna0,um, a com4ulsory
lo#er.secondary sc'ool. @t t'e end o& t'is cycle 4u4ils take t'eir second com4ulsory
e>ternal e>amination organised "y a regional e>amination commission. %4on t'e
com4letion o& t'e 2imna0,um, t'ey may c'oose "et#een t#o o4tionsA
Liceum profilowane (D.year s4ecialised u44er.secondary sc'ool #it' t'e e>ternal
3atura e>amination at t'e end)J
4asadnic0a s0ko1a 0awodowa (1.year ocational u44er.secondary sc'ool).
8raduates &rom ocational sc'ools may also take t'e 3atura e>amination u4on success&ul
com4letion o& a 1.year su44lementary general secondary sc'ool (50upe1nia,6ce liceum
og7lnoks0ta1c6ce) or a D.year su44lementary tec'nical secondary sc'ool (+echnikum
u0upe1nia,6ce). +'e ne# e>ternal 3atura e>amination introduced in t'e s4ring o& 100F 'as
re4laced entrance e>aminations to uniersities.
+'e @ct o& 1F July /HH- amending t'e !c'ool 3ducation @ct o& /HH/, @rticle /F, de&ines
com4ulsory education in Poland in t'e &ollo#ing #ayA
education is com4ulsory until t'e age o& /-J
&ull.time com4ulsory education starts during t'e calendar year in #'ic' t'e 4u4il
reac'es 2 years o& age and lasts until t'e end o& 2imna0,um (lo#er.secondary
sc'ool), yet no longer t'an #'en 'e$s'e reac'es /- years o& age.
%4on com4letion o& a 2imna0,um# com4ulsory education can "e im4lemented t'roug'A
attendance o& a 4u"lic or 4riate 4ost.2imna0,um sc'oolJ
23
4artici4ation in classes organised in out.o&.sc'ool &orms in 4u"lic or non.4u"lic
institutions (#it' accreditation)J
4artici4ation in classes organised "y legal "odies or indiidual 4ersons #it'
accreditation &or running educational actiitiesJ
ocational training o&&ered "y an em4loyer.
,* Lower.secondary education
+eac'ing at t'is stage is arranged in su"jects taug't "y s4ecialist teac'ers. !ince 1001
t'ere 'ae "een national )ore )urricula &or all su"jects taug't. +'e )ore )urriculum 'as to
"e res4ected "y eery sc'ool, alt'oug' teac'ers are &ree to &ollo# one o& t'e aut'oring
curricula "ased on t'e )ore and a44roed &or use in sc'ools "y t'e Minister, as #ell as use
arious te>t"ooks selected &rom a list a44roed "y t'e Minister.
Iesides se4arate su"jects, t'e &ollo#ing Neducational 4at'sN 'ae "een introduced at t'is
stageA
+ 4'iloso4'ical educationJ
+ reading and media educationJ
+ 'ealt' educationJ
+ ecological educationJ
+ regional education : cultural 'eritage o& t'e regionJ
+ ciil de&enceJ
+ 3uro4ean educationJ
+ Polis' culture in t'e conte>t o& Mediterranean ciilisation.
3ac' !c'ool ;ead is res4onsi"le &or t'e inclusion o& t'ese 4at's in t'e sc'ool curriculum.
*m4lementation o& t'ese 4at's is assured "y su"ject teac'ers t'roug' t'e integration o&
t'eir content in t'e su"ject curricula.
4* "ssessment
+'e education system re&orm, started in /HH/, encom4asses t'e introduction o& a ne#
co'erent ealuation system including internal and e>ternal assessment.
*nternal ealuation is carried out "y sc'ool teac'ers on t'e "asis o& reBuirements de&ined
in relation to t'e curricula im4lemented in t'e gien sc'ool. +'is ty4e o& ealuation aims
mainly at su44orting t'e 4u4ilOs deelo4ment.
3>ternal assessment is carried out "y institutions e>ternal to t'e sc'ools and it is "ased on
national e>amination standards. +'ese are de&ined and announced in t'e regulations issued
"y t'e releant minister o& education. *n order to 4re4are and im4lement t'is ty4e o&
assessment, a )entral 3>amination )ommission and eig't 7egional 3>amination
)ommissions 'ae "een esta"lis'ed.
4*)* Internal Evaluation
Pu4ils are assessed se4arately in eac' su"ject "y t'eir teac'ers. +'e results o& t'e
&ormatie assessment carried out during a sc'ool year are taken into account in t'e end.
o&.year summatie assessment.
*n grades /.D o& 4rimary sc'ool t'e ealuation is only descri4tie and a44lies to
educational attainments and "e'aiour (conduct) se4arately. !tarting #it' grade E, eac'
su"ject teac'er 'as t'e &ollo#ing scale o& marks at 'is$'er dis4osalA G . e>cellent, F . ery
good, E . good, D . satis&actory, 1 . acce4ta"le, / . unsatis&actory. !c'ool marks as #ell as
t'e criteria on #'ic' t'ey are "ased s'ould "e o4en to t'e 4u4il and 'is$'er 4arents.
29
Pu4ils also receie marks &or t'eir "e'aiour (conduct) according to t'e &ollo#ing scaleA
e>cellent, ery good, good, acce4ta"le, unacce4ta"le, and inadmissi"le. +'e mark &or
"e'aiour (conduct) s'ould not in&luence su"ject marks or 4romotion to a 'ig'er grade.
Pu4ils may take a eri&ying e>amination i& t'e mid.year or end.o&.year summatie mark,
gien "y t'e teac'er, is too lo# in t'eir or t'eir 4arentsO o4inion. +'is kind o& e>am is also
commissioned &or 4u4ils #'ose results cannot "e assessed &or ot'er reasons, suc' as an
indiidual study 4rogramme, out.o&.sc'ool study or a long 4eriod o& illness.
4*,* E1ternal Evaluation
+'e ne# system o& e>ternal assessment, im4lemented gradually &rom 1001 on, aims at
measuring t'e ac'ieements and identi&ying s'ortcomings o& students= education, assessing
t'e e&&ectieness o& teac'ing, and com4aring o"jectiely current certi&icates and di4lomas
irres4ectie o& t'e 4lace #'ere t'ey 'ae "een issued.
+'e e>ternal e>amination system in Poland encom4assesA
+'e )om4etence +est (+est kompetency,ny) taken at t'e end o& t'e G.grade
4rimary sc'oolJ
+'e Lo#er.!econdary !c'ool 3>amination (Eg0amin gimna0,alny) conducted at t'e
end o& lo#er.secondary sc'ool (H
t'
grade)J
+'e 3atura 3>am ination &or graduates o& general secondary sc'ools, s4ecialised
secondary sc'ools, tec'nical secondary sc'ools, su44lementary secondary sc'ools
or 4ost.secondary sc'oolsJ
+'e 9ocational 3>amination (e>amination con&irming ocational Buali&ications) &or
graduates o& ocational sc'ools, tec'nical sc'ools and su44lementary tec'nical
sc'ools.
4*,*)* !he Com+etence !est (Test kompetencyjny)
+'e )om4etence +est, introduced in 1001, is administered at t'e end o& 4rimary sc'ool
(8rade G) and is t'e &irst, com4ulsory e>ternal e>amination a 4u4il e>4eriences in 'is$'er
sc'ool education. *t is usually conducted at t'e 4u4il=s o#n sc'ool. +'e results do not 'ae
any in&luence on a 4u4il=s 4romotion and s'ould not "e t'e "asis o& any selection 4rocess.
@ny 4u4il #'o 'as graduated &rom 4rimary sc'ool, inde4endent o& test results, must "e
admitted to a lo#er.secondary sc'ool in t'e a44ro4riate area (until t'e age o& /G).
+'e test content integrates all sc'ool su"jects and tests t'e skills and a"ilities s4eci&ied in
t'e attainment standards related toA reading& writing& reasoning& using information&
a++lying $nowledge. Pu4ils 'ae G0 minutes to com4lete t'e test and can get a ma>imum
score o& E0 4oints. 3>ternal e>aminers mark t'e e>am 4a4ers.
4*,*,* !he Lower.-econdary -chool E1amination (Egzamin gimnazjalny)
(!ee t'e detailed descri4tion "elo#, section E.1.)
4*,*4* !he Matura E1amination
+'e ne# 3atura 3>amination, im4lemented &ully in 100F, is an e>ternal e>amination,
uni&orm t'roug'out t'e country, "ut not com4ulsory. *t #as introduced to ensure t'e
uni&ormity o& e>aminations and criteria &or assessment, com4ara"ility o& results and
o"jectiity o& assessment (t'e coded ans#er s'eets are marked "y e>ternal e>aminers).
+'e 3atura 3>amination is t'e "asis &or entry into uniersities and colleges o& tertiary
education.
25
4*,*5* !he Eocational E1amination
+'e 9ocational 3>amination (e>amination con&irming ocational Buali&ications) is an
e>ternal e>amination, not o"ligatory, conducted "y 7egional 3>amination Ioards. *ts task
is to assess and certi&y ocational Buali&ications &or a 4articular 4ro&ession. *t is conducted
&or graduates o&A "asic ocational sc'ools, tec'nical secondary sc'ools, su44lementary
tec'nical secondary sc'ools and 4ost secondary sc'ools.
5* Polish Language as school su23ect (L-) at Lower.-econdary Level (Gimnazjum)
+'e &ollo#ing documents, 4rocedures and standards create t'e "asis on #'ic' t'e Polis'
language is taug't as sc'ool su"jectA
)ore )urriculum, issued "y t'e Ministry o& 3ducationJ
local curricula, #ritten "y local aut'oring teams, alidated "y e>4erts a44ointed "y
t'e MinistryJ
te>t"ooks, #ritten "y aut'oring teams and alidated "y t'e Ministry, edited in
se4arate sets &or )ulture P Literature and &or LanguageJ
3>amination standards, issued "y t'e )entral 3>amination )ommission (incl.
se4arate ersions &or 'earing.im4aired candidates, and &or Qmot'er tongues o&R
national and et'nic minorities)J
3>ternal e>aminationA Lo#er.!econdary !c'ool 3>amination (Eg0amin gimna0,alny),
conducted "y 7egional 3>amination )ommissions.
5*)* Core Curriculum for the su23ect /Polish Language0 in Lower.-econdary -chools
()@@@)
+'e document starts #it' a Pream2le t'at underlines t'e general educational alue o&
mot'er tongue trainingA
K(no#ledge is acBuired most o& all t'roug' language. Mot'er.tongue teac'ing creates a
"asis &or 4u4ils= general deelo4ment, contri"utes to t'e s'a4ing o& t'eir 4ersonal
identity, and t'us &unctions as t'e main re&erence 4oint &or t'e #'ole o& sc'ool
education. Deelo4ing language a"ilities in #riting and s4eaking (t'e re4ertoire o&
language terms, ort'ogra4'y, and t'e aest'etics o& t'e #ritten code included) is a
s'ared res4onsi"ility o& all teac'ers, #'ateer t'e su"ject taug't.S
+'e &ollo#ing c'a4ter s4eci&ies t'e general Educational aims &or t'is stage (t'e same set
o& aims is listed &or u44er.4rimary and &or lo#er.secondary leel)A
/. Deelo4ing skills in s4eaking, listening, reading and #riting in di&&erent 4riate and
4u"lic communicatie situations : es4ecially t'ose o& s4ecial im4ortance &or li&e in
a democratic, ciic state.
1. 7aising a#areness o& language as 4art o& cultural 'eritage.
D. %neiling 4u4ils= interests, ca4acities, needs, and linguistic skills "y a44ro4riate
goal setting using suita"le tasks, te>ts and actiities.
E. !timulating 4u4ils= motiation to read and inter4ret literary #orks and te>ts o&
culture (including audio.isual ones) as a #ay to "etter understand t'e #orld and
our 4lace in it.
F. Presenting national and 3uro4ean cultural tradition.
G. +eac'ing to take 4art in and identi&y #it' culture, es4ecially in its sym"olic and
a>iological as4ects.
26
Deried &rom t'ese general aims, a set o& s4eci&ic !as$s for the school is de&ined (/.F in
common #it' t'e u44er.4rimary leel)A
/. Deelo4ing motiation to read literary #orks and ot'er te>ts o& culture t'at
contri"ute to gro#ing kno#ledge o& 'uman li&e in t'e #orld : contem4orary and
4astJ
1. 7e&ining communicatie com4etence, i.e. a"ility to s4eak, listen, read and #rite,
and to inter4ret a ariety o& te>ts o& cultureJ
D. )reating situations in #'ic' language learning 'a44ens "y conscious and t'oug't&ul
language use (#it'out unnecessary "urden o& t'eoretical, a"stract kno#ledge o&
t'e language system)J
E. !u44orting 4u4ils= creatiity in deelo4ing skills and accumulating kno#ledgeJ
F. *ntegrating cultural e>4eriencesJ
G. Presenting te>ts o& culture #it' a ie# to acBuiring general kno#ledge,
4artici4ating in contem4orary culture and social communication, and to 4ractise
cultural traditionsJ
2. !u44orting 4u4ils in t'eir 4rocess o& "uilding integrated systems o& 4ersonal aluesJ
-. Deelo4ing 4u4ils= learning strategies and eBui44ing t'em #it' tools &or
inde4endent intellectual #ork.
+'e !eaching content is s4eci&ied in t'e )ore )urriculum /HHH in t'e &ollo#ing, Buite
tec'nical #ayA
/. Formal and in&ormal s4eaking situations.
1. @danced &orms o& structured #riting, e.g. essay, re4ortage, reie#, interie#.
D. 3lements o& r'etoric and logic in s4eec'A t'esis, argument, conclusion, etc.
E. !ynta> : in&lected utterances.
F. Mor4'ology and Fle>ion : t'eir &unction in making meaning.
G. Iasic terms o& !emantics.
2. Iasic terms o& !tylistics.
-. @ssimilation mec'anisms in 4ronunciation : s4oken ersus #ritten code.
H. +erms o& literary analysisA irony, synt'esis, allegory, etc.
/0. Literary genres.
+'e e>4ected learning outcomes : t'e "ttainments : are 4resented in relation to linguistic
skills, learning skills and cultural com4etencesA
/. !4eaking, listening, reading, #riting.
@t t'e end o& t'e 2imna0,um t'e student s'ould "e a"le toA
a. 4roduce intentionally and situationally a44ro4riate s4oken
and #ritten te>tsJ
". recognise and 4roduce utterances t'at in&orm, descri"e,
ealuate, e>4ress o4inions, conince, "uild an argumentJ
c. use arious situationally a44ro4riate s4eec' modes
(es4ecially t'e standard one : s4oken and #ritten)J
2*
d. recognise and understand communicational 4rinci4les
("ased on situations, emotions, judgments) and im4lications
o& language useJ discuss t'e et'ical as4ect o& language useJ
e. 4rocess te>ts (summarise, deelo4, adjust style, etc.)J
&. trans&orm 4ictures and sounds to te>ts (s4oken and
#ritten)J
g. use situationally and conte>tually a44ro4riate grammatical
structures 4roducing clear and co'erent te>tsJ
'. #ork on sentence leel : deelo4 and condense content
using sim4le and in&lected structures, Passie oice, etc.
1. @utonomous learning
@t t'e end o& t'e 2imna0,um t'e student s'ould "e a"le toA
i. retriee in&ormation &rom arious sourcesJ
j. 4re4are documentation, take and make notes, select and
store in&ormationJ
k. make #ise use o& mass media.
D. )ultural rece4tion o& te>ts
@t t'e end o& t'e 2imna0,um t'e student s'ould "e a"le toA
l. recognise and comment on s4eci&ic &eatures (e.g. means o&
e>4ression) o& literary and ot'er te>ts o& cultureJ
m. use literary and cultural terminology in a natural and
&unctional #ayJ
n. relate e>4eriences, &eelings and am"itions 4resent in #orks
&rom di&&erent 4eriods in time, es4ecially #it' re&erence to
uniersal cultural traditionsJ
o. com4are literary #orks #it' t'e #orks in ot'er disci4lines
o& artJ
4. inter4ret literary #orks in t'eir di&&erent conte>tual
settingsJ
B. 4roduce creatie inter4retations o& literary #orks in
t'eatrical &orms (e.g. staging)J
r. recognise alues in literary #orks and relate t'em to t'e
o#n system o& aluesJ
s. ealuate cultural 4'enomena inde4endently and criticallyA
"uild and de&end an argument, o4inion, inter4retation : in
#riting and in s4eec'J
t. relate literary #orks to 4ersonal "iogra4'ies o& t'eir aut'ors
and to t'e 'istorical conte>t.
+'e )ore )urriculum ends #it' a 6eading list #it' a selection o& recommended "ooks &rom
Polis' and international literature, as #ell as ot'er te>t ty4es recommended &or use in t'e
classroom (ne#s4a4er articles, instructions, adertisements, moies, teleision
4rogrammes, internet sources, etc.).
5*,* E1ternal assessment of com+etences C the Lower.-econdary -chool E1amination
(Egzamin gimnazjalny)
27
+'e com4ulsory Lo#er.!econdary !c'ool 3>am is an interdisci4linary e>amination
consisting o& t#o 4artsA @rts and !ciences. +'e &irst 4art tests a"ilities and kno#ledge in
t'e ;umanitiesA t'e Polis' Language, ;istory, )iic 3ducation, @rt, Music and educational
4at's (see a"oe) suc' as P'iloso4'y, 7egional !tudies, 7eading and Media, Polis' )ulture
and t'e Mediterranean )iilisation.
+'e second 4art tests a"ilities and kno#ledge in science su"jectsA Mat's, Iiology,
8eogra4'y, )'emistry, P'ysics and @stronomy and educational 4at's suc' as ;ealt'
3ducation or 3cology.
*n eac' 4art o& t'e e>amination t'e ma>imum score is F0 4oints. +'e result is not
signi&icant &or t'e graduation &rom lo#er.secondary sc'ool "ut it is noted on t'e certi&icate
and taken seriously into account "y u44er.secondary sc'ools during t'e admission 4rocess.
From 100- on, it is 4lanned to im4lement a t'ird 4art o& t'e e>aminationA assessing
com4etences in a modern &oreign language.
5*,*)* E1amination standards (requirements) and their a++lication in the tests
+'e &ollo#ing seen reBuirements 'ae "een s4eci&ied to assess t'e 6eading skillA
/. 7eading and inter4reting ?+e>ts o& )ulture= (+o)) on t'eir literal, meta4'oric, and
sym"olic leels.
1. *denti&ying t'e aut'or=s intention, &acts and o4inions, 'istorical trut' and &iction,
4ersuasieness, mani4ulation, alorisation.
D. 7etrieing in&ormation &rom di&&erent ty4es o& +o), suc' as literary, 4o4ular
scienti&ic and normatie (legal) te>ts, ne#s4a4er articles, illustrations, ta"les,
gra4's, ma4s, &igures, sc'emata.
E. *denti&ying ty4es and &unctions o& linguistic means &or making meaning in literary
te>ts, ne#s4a4er articles, 4ieces o& art and music.
F. *denti&ying and inter4reting logical relations in t'e domains o& 4olitics, economy,
culture, and social li&e, ciilisation deelo4ment in Poland and in t'e #orld.
G. Perce4tion and analysis o& conte>tual as4ects necessary &or a44ro4riate
inter4retation o& +o), suc' as 'istorical, "iogra4'ical, 4'iloso4'ical, religious,
literary, artistic, musical, and regionalJ a"ility to e>4lain relations'i4s "et#een
di&&erent ty4es o& +o) (arts, music, and literature).
2. Perce4tion o& alues em"edded in +o).
+'e a44lication o& t'e reBuirements in real tests conducted oer t'e last t'ree years is
s'o#n in +a"le /A
6equirement
num2er C 6eading
9um2er of test items related to the requirement
,BB< ,BB= ,BB>
) ? > ?
1 1 2 1
4 < 5 =
E D 1 D
F 1 1 /
= < 4 <
2 0 0 0
29
+a"le /. @44lication o& reBuirements in tests. 7eading.
+'e analysis s'o#s t'at most test items used in t'e years 100F.1002 #ere related to
7eBuirement No. /A -eading and interpreting 8+e*ts of ulture. !+o" on their literal#
metaphoric# and sym$olic levels. +'e ne>t most &reBuently used #ere standards No. D
(-etrieving information from different types of +o# such as literary# popular scientific#
and normative 9legal: te*ts# newspaper articles# illustrations# ta$les# graphs# maps#
figures# schemata) and G (Perception and analysis of conte*tual aspects necessary for
appropriate interpretation of +o# such as historical# $iographical# philosophical#
religious# literary# artistic# musical# regional; a$ility to e*plain relationships $etween
different types of +o 9arts# music# literature:). +'ese t'ree reBuirements generated
toget'er "et#een FGT (100G) and 2GT (1002) o& t'e score 4oints aaila"le in t'is 4art o&
t'e e>amination. @t t'e o44osite end is 7eBuirement No. 2 (Perception of values
em$edded in +o ) not used at all in t'ese years : in &act it generated only one single test
item (in 100E) in t'e si> years &or #'ic' t'e system 'as "een in use.
+'e assessment o& t'e Writing skill 'as "een standardised "y a set o& nine reBuirementsA
/. Producing linguistically and stylistically correct te>ts, suc' as descri4tion,
narratie, re4ort, reie#, essay, note, 4lan, ne#s4a4er article, interie#,
announcement, adertisement, initation, dedication, reBuest, letter, memoirs.
1. %sing terms and categories s4eci&ic to t'e domain o& 'umanistic su"jects and
disci4lines.
D. Producing situationally a44ro4riate te>ts o& in&ormational or 4ersuasie
c'aracter.
E. (no#ledge and use o& organisational 4rinci4les to 4roduce logically and
syntactically co'erent te>ts a44ro4riate to t'e to4ic gien.
F. Putting &or#ard, organising and alorising arguments in &aour o& one=s o#n or
some"ody else=s o4inion.
G. @nalysing, com4aring, organising, and synt'esising in&ormation em"edded in
+o).
2. Processing te>tsA summarising, deelo4ing, trans&orming (stylistically).
-. 3>4ressing o#n o4inion regarding relations'i4s "et#een natie culture and
ot'er culturesJ commenting on 'istorical and contem4orary links "et#een t'e
Polis' and t'e Mediterranean cultures in suc' domains as 4olitics, economy, and
eeryday li&e.
H. +ackling 4ro"lematic issues 4resent in +o), suggesting solutions and dra#ing
conclusions.
38
+'e o4erationalisation o& t'ese standards in real tests is s'o#n in +a"le 1A
6equirement
num2er C Writing
9um2er of test items related to the requirement
,BB< ,BB= ,BB>
) ? @ ?
1 / 0 /
4 > < >
5 < = <
F D 1 /
G / / 1
2 0 1 /
- 0 0 0
H 0 0 0
+a"le 1. @44lication o& reBuirements in tests. Writing.
We can see t'at it is 7eBuirement No. / (Producing linguistically and stylistically correct
te*ts# such as description# narrative# report# review# essay# note# plan# newspaper article#
interview# announcement# advertisement# invitation# dedication# re&uest# letter#
memoirs) t'at 'as "een used most &reBuently, &ollo#ed "y No. D (Producing situationally
appropriate te*ts of informational or persuasive character) and E (<nowledge and use of
organisational principles to produce logically and syntactically coherent te*ts appropriate
to the topic given). *t is striking t'at no use at all 'as "een made o& 7eBuirements Nos. -
and H (E*pressing own opinion regarding relationships $etween native culture and other
cultures; commenting on historical and contemporary links $etween the Polish and the
3editerranean cultures in such domains as politics# economy# everyday life and +ackling
pro$lematic issues present in +o# suggesting solutions and drawing conclusions).
*n t'e 4art o& t'e e>amination t'at assesses com4etences in -ciences, t'e &ollo#ing &our
areas 'ae "een identi&ied to sere &or setting reBuirementsA
/. %sing terms and a44lying 4rocedures
1. 7etrieing and using in&ormation
D. *denti&ying and descri"ing &acts and relations'i4s (logical, &unctional, s4atial,
and tem4oral)
E. @44lying integrated kno#ledge and skills to 4ro"lem soling
*t is interesting to note t'at most o& t'ese reBuirements (/, 1, and D) are clearly related to
t'e use o& language across curriculum (L@)) : at least to a similar e>tent as t'ey are
related to t'e su"ject matter. W'at is tested 'ere is t'us as muc' language com4etence as
su"ject.matter com4etence.
3)
+'e 4ro4ortion o& test items 4roduced #it' re&erence to t'ese reBuirement areas is "eing
ke4t sta"le in all testsA
6equirement area
num2er C -ciences
9um2er of test items related to the requirement
(,BB<& ,BB=& ,BB>)
/ /F (D0T)
1 /1 (1ET)
D /F (D0T)
E - (/GT)
+a"le D. @44lication o& reBuirement areas in tests. !ciences.
5*,*,* !he results (,BB=)
+'e &ollo#ing t#o ta"les : "ased on t'e results o& t'e 100G test : s'o# re4resentatie
tendencies &or t'e #'ole set o& e>ams (results o& tests conducted in ot'er years are Buite
similar).
"rts -ciences
7eading Writing %sing terms
and
a44lying
4rocedures
7etrieing
and using
in&ormation
*denti&ying
and descri"ing
&acts and
relations'i4s
@44lying
integrated
kno#ledge
and skills to
4ro"lem
soling
"verage
facility of
test
items
.21 .FE .EE .G2 .EG .D0
"verage
for each
+art
.GD .E-
+a"le E. @erage &acility o& test items (100G)
+'e aerage test item &acility inde> indicates t'at testing com4etences in t'e !ciences is
"ased on items t'at are signi&icantly more di&&icult &or t'e candidates t'an t'ose assessing
t'e reading and #riting skills tested under t'e @rts. +est items assessing t'e reading skill
(most o& t'em closed multi4le.c'oice items) 4roe to "e less di&&icult to t'e candidates
t'an t'ose ealuating #riting a"ility, and &ar less di&&icult t'an t'ose testing t'e a"ility to
a44ly integrated kno#ledge and skills to 4ro"lem soling (in "ot' cases mostly o4en.ended
tasks).
32
Feneral findingsG "rts (6eading and Writing& ,BB=)
@"ilities con&irmed 7etrieing in&ormation
*denti&ying t'e aut'or=s intention
,rganising te>t
Producing co'erent, communicatiely a44ro4riate te>t
De&icits identi&ied 7eading instructions to test items
*gnoring &acts clearly 4resent in te>ts
Iuilding an argument
@ccuracy, ort'ogra4'y, 4unctuation
Lack o& 'istorical conte>t kno#ledge
%sing terms related to literary and language sciences
+a"le F. 8eneral &indingsA @rts (7eading and Writing, 100G)
*n general, t'e results o& t'e test 4arts assessing t'e com4etences in @rts indicate t'at
#'ile t'e reBuirements related to t'e communicatie use o& language are "eing largely
met "y t'e graduates o& t'e lo#er.secondary sc'ool in Poland, t'e standards, "y #'ic'
linguistic accuracy, t'e use o& (scienti&ic) terms and t'e a44lication o& 'istorical
kno#ledge are assessed, are muc' less satis&actorily &ul&illed.
Conclusions
+'e aim #it' t'is 4resentation o& a concrete case o& com4ulsory e>ternal e>amination
sc'eme #'ere com4etences in a language o& sc'ooling are assessed #as to illustrate some
o& t'e issues releant &or t'e deelo4ment o& an L3 &rame#ork. +'e &ollo#ing t'ree
general 4oints con&irming t'e initial o"serations noted in t'e introduction to t'is set o&
studies may sere as a conclusion 'ereA
/. ,nly t'ose attainment targets (com4etence standards) t'at may "e turned into
o"sera"le "e'aiour can "ecome real su"jects &or assessment 4rocedures (testing
tasks$items)J
1. *tem leel testing may not "e ca4a"le$suita"le o& 'andling all o& t'e com4le>ity o&
L3 com4etencesJ
D. Most (i& not all) com4etences t'at are su"ject to assessment in education are
language related (L@) P L! P FL).
33