The CEFR for teaching and

assessing young learners

LTF, 21 Nov 2009
CRELLA, University of Bedfordshire, Luton

Szilvia Papp, Hanan Khalifa, Nick Charge

University of Cambridge ESOL Examinations
 Aim of Talk

2
 Questions
1. the construct behind YLE
a) linguistic progression
b) cognitive development
c) educational, social, psychological aspects
2. the link between YLE and the CEFR
3. CEFR proficiency levels reflected in YLE
4. salient features of YLE performance in CEFR
5. additional salient features of YLE performance not
in CEFR
3
 Tests for YLLs need to be

supporting but challenging enough to cater for all
cognitive and emotional needs on both ends of the
spectrum

build on and reflect previous experience of language
learning

test language use rather than language knowledge

provide children with

a range of social situations and

sufficient breadth of experience with language

cover a wide enough sample of language
4
(Simon Smith 2009, University of York)
 Question 1
What is the construct behind Young Learners of
English tests?
a) What linguistic progression do the YLE tests
reflect?
b) How do the tasks reflect the cognitive
development of YLE candidature?
c) What educational, social, psychological
aspects of young language learners are catered for
in YLE tests?

5 (Papp 2007, 2008a)

 Linguistic progression

Limitation to lexical, structural, pragmatic
development due to age

Similar route, but different mechanisms, rate and
ultimate attainment among child L2 learners,
depending on age of onset of learning

Communication strategies

Language learning strategies

Basic interpersonal communication skills (BICS)

Cognitive academic language proficiency (CALP)
6
(Cummins 2001)
 Linguistic progression
Differences in language development based on age
can manifest themselves in variations in:

accuracy, fluency, complexity

use of formulaic sequences

interactional patterns and interactional moves

accent

content, topic, range of notions that the candidate is able to,
wants to or needs to encode

different goals

the presence of specific grammatical features in the ILG
7 (Nicholas and Lightbown 2008, Philp, Oliver and Mackey 2008)
8
 Cognitive development

Strategies for working memory use

Abstract concept formation

Reasoning abilities

Metalinguistic awareness

Executive control

Mind reading abilities

9
(Meadows 2006)
10
 Educational, social, psychological
variables

Amount & quality of input, exposure to, experience
with L2

Social, cultural and educational context of interaction

Development of literacy in L1 (different writing
systems)

Theory of mind (use of complex discourse &
narrative skills)

Individual differences

Scaffolding
11 (Cameron 2003)
 Task demands and scaffolding

12
 Summary
Age-related differences between young learners which
have an impact on their L2 development:

improved cognitive abilities related to growing attentional and
memory capacities

changing social and psychological make-up, such as
fluctuating self-confidence, self-generated and peer-
influenced interests, motivation, related affective schemata

unique experiential characteristics, such as differential
exposure to the target language, differential acceptance and
participation in activities and task types, etc.

13 (McKay 2006)
 Question 2
What is the demonstrable link between YLE test and
task specifications, YLE candidate performance and
the relevant CEFR scales and descriptors in terms of
a) proficiency levels
b) activities, strategies and competences YLE candidates
can be expected to do or have?

14
 Link between YLE and CEFR
YLE linked to CEFR by design through test
and task specifications (1992-1997 test
development, 2002-2007 revision):

Flyers at A2 (KET)

Movers at A1

Starters below A1

The link has been demonstrated empirically

and qualitatively:

equivalence and equating studies

formal alignment process in May 2008

next equivalence study in March 2010

15 (Gardiner 2000, Flux 2001, Jones 2001, Jones 2002a,b,c,

Burrow 2004, Barker and Shaw 2007, Papp and Salamoura 2009)
 Logits Flyers KET grade Logits Flyers KET
Listening Reading grade
shields & Writing
Shields
1.65 5 Merit
2.09 5 Merit
0.77 4
1.32 4
0.22
0.88
-0.33
0.55
-0.66 3 Pass
0.33
-0.99
0.00
-1.32
-0.11
Narrow
-1.65 Fail -0.33
-1.87 Fail -0.44 3 Pass
-2.09 2 -0.66
-2.31 -0.77
-2.53 -0.88
-2.75 -1.10
-2.97 -1.21
-3.19 -1.32
-3.41 1 -1.43 Narrow
Fail
-3.74 -1.54

16 (Gardiner 2000, Flux 2001)

 Logits Flyers Logits Flyers Movers
Movers
Reading & Reading &
Listening Listening
Writing shields Writing shields
shields shields
5.16 5
3.96 4 5
4.51 4
3.19
4.18 5
2.75
4.07
2.31 3 4
3.74
2.09
3.41
1.76
3.19
1.54
2.97
1.32
2.86 3 4
1.10 2 3
2.64
0.88
2.53
0.66
2.42
0.44
2.31
0.33
2.20
0.11
2.09
-0.11 1 2
1.98
-0.33
1.87
-0.55
1.76
-0.77
1.65
-0.99
1.54 2
-1.21
1.43
1.32
17 (Gardiner 2000, Flux 2001)
 Activities, strategies and competences
of YLE candidates
The relationship between Starters and Movers and
Breakthrough level investigated by a consultant in
2003.
In this project, the lexical content and topic areas of YLE tests
were mapped against the preliminary Breakthrough
objectives.
Results:

substantial areas of overlap between Starters and Movers &
Breakthrough in terms of lexis and topic areas,

however, YLE topic areas are expressed in more concrete
terms such as School & the Classroom rather than Education.

18
(Burrow 2004)
 Verifying YLE alignment to the CEFR
Formal alignment/linking exercise May 2008: CEFR ratings for levels& skills
Skill YLE Starters YLE Movers YLE Flyers
Speaking 1.06 1.85 2.94
Listening 2.17 2.46 3.13
Reading 1.79 2.63 3.10
Average 1.67 2.31 3.06
(all skills)
CEFR level Pre-A1/A1 A1+ A2
19 (Papp and Salamoura 2009)
 Question 3
How do the YLE tests embody and reflect the CEFR?

20
 Describing YLE exams using CEFR?

trade-off between control of candidate output and
authenticity of tasks, not reflected in CEFR

scaffolding of tasks absent in CEFR

learning to learn, enabling skills very sparse in CEFR

limits to childrens cognition, linguistic progress,
psychological, emotional, social development not
acknowledged in CEFR:

no task should tax childrens cognition by requiring them to
deal with multiple perspectives other than their own

effect of writing on the reader should be taken into account
in marking criteria
21 (Papp and Salamoura 2009)
 Question 4
What are the salient features of YLE candidate
performance that reflect the CEFR in terms of
a) proficiency levels
b) activities, strategies and competences YLE candidates
can be expected to do or have?

22
 Salient features of YLE candidates
performance: listening and speaking
Starters (pre-A1) candidates performance:
Ability to rely purely on a very finite, rehearsed repertoire of phrases, frequently used routines and patterns
limited to the performance of isolated tasks in specific situations, i.e. a list of pedagogic tasks in a primary
school setting.

Movers (A1) candidates performance:

Ability to interact in a simple way, initiate and respond to simple statements in areas of immediate need or on
very familiar topics (ask and answer simple questions about themselves, where they live, people they
know, and things they have, etc.).

Flyers (A2) candidates performance:

Ability to handle social functions (greet people, ask how they are and react to news; handle very short social
exchanges; ask and answer questions about what they do at school and in free time; make and respond to
invitations; discuss what to do, where to go and make arrangements to meet; make and accept offers).
Ability to perform simple classroom tasks in English.

Strong Flyers (A2+) candidates performance:

Ability to more actively participate in conversation.
Ability to sustain monologues.

23 (CEFR 2001)
 Question 5
Which additional salient features can be identified in
YLE candidate performances that are not currently
covered in the CEFR?

24
 Childrens literacy: reading and writing
There is a need to

list childrens L2 reading abilities, activities, strategies and
competences

investigate emerging L2 writing abilities (e.g. copying,
handwriting, spelling and other enabling skills)

list childrens specific L2 writing activities, strategies and
competences

in order to identify salient features of typical/likely candidate
performance at each level of YLE not currently in CEFR
This requires additional work

on sample performances from YLE reading & writing tests

with children in the classroom (ECML)
25

review of materials used in classrooms across the world
 Recommendations for future research
For Cambridge ESOL

further explore salient features of YLE performance, esp for
childrens reading and writing typical in & outside classroom

develop Can Do statements for YLE candidates (aged 712)
to complement those for 1114 year-old school learners
taking KET/PET for Schools examinations

further work on link between summative and formative
assessment (assessment for learning)
For other YLL test providers

carry out a similar linking exercise in order to

define and explicitly state construct tested

generate evidence of CEFR relatedness

identify qualities of YLL tests claiming links to the CEFR

raise awareness of good testing practices
26
 Recommendations for future research
For all YLL test providers

provide feedback to CoE on linking process for relating
exams such as the YLE tests to the framework of reference

provide calibrated samples to CoE of childrens performance
in tests thus related to the CEFR

answer the question that remains whether it is necessary to

carry out a whole-scale adaptation of current CEFR scales to reflect
young learner needs and contexts or

start developing scales from scratch to reflect the nature of second
language learning and use among children?

27
 Selected references
Barker, Fiona and Stuart Shaw (2006) YLE Vertical Linking Project. Validation Project No 171. Cambridge ESOL internal
document.
Barker, Fiona and Stuart Shaw (2007) Linking language assessments for younger learners across proficiency levels (Phase 1).
Research Notes 28: 14-18. UCLES: Cambridge ESOL.
Burrow, Trish (2004) Exploring the relationship between YLE Starters and Movers and Breakthrough level. Research Notes 16:
6-7. UCLES: Cambridge ESOL.
Gardiner, Kerri (2000) KET / Flyers Research Project: Analysis of data. Research and Validation Report No 106/806. Cambridge
ESOL internal document.
Flux, Tracy (2001) KET / YLE Link Project 2001. Research and Validation Report No 246. Cambridge ESOL internal document.
Jones, Neil (2001) Reliability of YLE: Dependability of grades for a high-facility exam. Research and Validation Report No 316.
Cambridge ESOL internal document.
Jones, Neil (2002a) Equating YLE levels: A study based on candidates taking two exams. Research and Validation Report No
318. Cambridge ESOL internal document.
Jones, Neil (2002b) Linking YLE levels into a single framework. Research Notes 10: 14-15. UCLES: Cambridge ESOL.
Jones, Neil (2002c) A Framework for YLE: Proposals for an integrated grade system and implications for calibration and grading.
Research and EFL Validation Report No 439. Cambridge ESOL internal document.
Papp, Szilvia (2007) The Cambridge YLE tests in the light of cognitive, linguistic, educational, social-psychological and cultural
aspects of childrens L2 development and the CEFR, internal Cambridge ESOL Research and Validation report.
Papp, Szilvia (2008a) Factors influencing L2 development and use in the 814 age group towards defining the construct,
Cambridge ESOL internal Research and Validation report.
Papp, Szilvia (2008b) Skills profiles by age in KET and PET, Cambridge ESOL internal Research and Validation report.
Papp, Szilvia (2008c) Coverage of Waystage and Threshold learning objectives within KET and PET and KET and PET for
Schools, Cambridge ESOL internal Research and Validation report.
Papp, Szilvia (2008) Quantitative linking YLE to the CEFR summary of empirical studies to date, internal Cambridge ESOL
Research and Validation report.
Papp, Szilvia (2009) Development of Can-do statements for KET and PET for Schools, Research Notes 36, 812, Cambridge:
Cambridge ESOL.
Papp, Szilvia and Salamoura, Angeliki (2009) An exploratory study linking young learners examinations to the CEFR, Research
Notes 37, 1522,Cambridge: Cambridge ESOL.
Trim, J L M (2001b) Breakthrough, unpublished manuscript, now available on the English Profile website.
van Ek, J and Trim, J L M (1990a/1998a) Threshold 1990, Cambridge: Cambridge University Press.
van28Ek, J and Trim, J L M (1990b/1998b) Waystage 1990, Cambridge: Cambridge University Press.