The use of young learner can-do

statements as part of formative

assessment

Szilvia Papp and Neil Jones

Cambridge ESOL
EALTA, Athens, Greece, 9 May 2008
Definitions and structure of talk

Definitions

Formative assessment = assessment to inform subsequent teaching in support

of learning

Young learners = here aged 14 and under in state education

Can-do statements = empirically validated age- and context-appropriate

performance indicators

Structure of discussion

Why should Cambridge ESOL get involved in formative assessment?

How should Cambridge ESOL get involved in formative assessment?

2
Developing YL can-do statements for formative assessment:
why? the context
Context: two recent developments:

in education: a general, continuing movement towards criterion-based goals

and interpretations
in the US: standards-based education
in Europe: the rise of CEFR to prominence
in the UK: the Assessment Reform Group (1999) and the Nuffield enquiry (2002)
in assessment: the evolution of the concept of validity
Does this test support inference to a candidates performance and abilities in
some real world domain of language use? (outcomes)
What happens in classrooms during the process of learning? (the emerging
abilities and competences as a result of inputs)
We should focus on language learning as a continuous process of acquiring
skills and abilities.

These two contexts make it essential for large examination boards to

engage in formative assessment.

The challenge: How to relate the two aspects: outcomes and inputs
3
(summative and formative aspects)?
Developing YL can-do statements for formative assessment:
why? the aims
For examination boards such as Cambridge ESOL:

test development: e.g. to inform item writing at task level in exams for young
learners
research: e.g. to act as a research tool to link young learner exams to the CEFR
and to other exams within Cambridge ESOLs main suite
validation: e.g. to assist in providing validity evidence for young learner exams, for
construct differentiation purposes
educational impact: to create a link between summative and formative assessment

For learners, teachers and other stakeholders:

information and interpretation: to offer parents, schools, local education

authorities, ministries and other stakeholders transparent descriptions of level of
likely proficiency of candidates
guidance: to support learners and teachers in the ongoing monitoring of progress
of learning towards summative exams

Main aim of present project: to inform and guide teachers work towards
preparing their learners for large-scale Cambridge ESOL exams.
4
 Key principles of assessing young learners
(Cameron 2001)

Cameron (2001) suggests the following key principles which should guide children's
language assessment:

assessment should be seen from a learning-centred perspective

assessment should support learning and teaching
assessment should be congruent with learning
young people and their parents and carers should understand assessment issues
assessment is more than testing.

These principles emphasise the important link between learning and assessment
as well as the affective impact of assessment which add an important dimension to
the usual considerations of test design, delivery and overall impact.

Washback by design and, by extension, impact by design.

Developing a conceptual framework in which formative and summative assessment

are seen as complementary aspects in service of learning, i.e. formative assessment
being part of and feeding into summative assessment, and in which language
learning and language use can be seen as different aspects of a single, continual
process.
5
6
 10 Principles of Assessment for Learning
(Assessment Reform Group 2002)
Assessment for learning should be part of effective planning of teaching and learning

Assessment for learning should focus on how students learn

Assessment for learning should be recognised as central to classroom practice

Assessment for learning should be regarded as a key professional skill for teachers

Assessment for learning should be sensitive and constructive because any assessment
has an emotional impact

Assessment for learning should take account of the importance of learner motivation

Assessment for learning should promote commitment to learning goals and a shared
understanding of the criteria by which they are assessed

Learners should receive constructive guidance about how to improve

Assessment for learning develops learners' capacity for self-assessment so that they can
become reflective and self-managing

Assessment for learning should recognise the full range of achievements of all learners
7
 Issues in assessing young learners

identifying plausible target language use situations for

young learners: where the domain is predominantly educational
restricted to language use within the classroom, context
and possibly some domains of language use for dimension
social and leisure activities;

the difference between natural development and typical

formal learning routes

the relevance of can-do statements which focus more

on the process of learning and/or on intermediate
outcomes formative
dimension
teachers views, needs and involvement in formative
assessment.
8
 The CEFR scales as basis of formative assessment

A functionally-based action-oriented approach to language teaching, learning

and assessment.

advantage:
using the relevant CEFR scales (A1-B2) ensures identifying salient features of
performance shared between young learners and adults

disadvantage:
need to be complemented with salient features of young learners performance
currently not in the CEFR
lacking a theory of learning and language development (a cognitive dimension),
only a taxonomy of behaviours
being too imprecise for test construction either summative or formative purposes
imbalanced focus: stressing the outcomes over the inputs to or process of learning
narrow scope and applicability: developed with not large cohorts in mainstream
educational settings in mind
potential for misuse: should not be used essentially as a needs analysis exercise to
define the learning task

9
The original CoE learning objectives as basis of formative
assessment

The notional-functional model based on contexts of use, i.e. an extension of

the original Waystage and Threshold levels: a detailed description for
English, setting the objectives for learning at, e.g. A2 and B1 levels

advantage:
more extensive lists of exemplars than in the CEFR scales
graded objectives

disadvantage:
no agreed-upon and commonly accepted Breakthough (A1) level yet published
Threshold (B1) developed by CoE originally to specify what kind of language an
immigrant or visitor needed to operate effectively in society (North 2006: 8)
also, there are minimal differences between Waystage (A2) and Threshold (B1), as
Alderson et al. (2004) found, in terms of
grammar
communicative activities
texts
functions
10
 Using existing scales for formative assessment?

North (2000) on using existing scales and descriptors:

Interpreted negatively one could say that in this way conventions and
clichs get copied from scale to scale without an empirical basis

[i]nterpreted positively one could say that this is a reflection of the

consensus over achievement in foreign language learning at different levels
of competence

(North, 2000: 182)

11
 Back to the CEFR

The CEFRs model of language use:

the actions performed by persons who as individuals and as social

agents develop a range of competences, both general and in
particular communicative language competences. They draw on
the competences at their disposal in various contexts under various
conditions and constraints to engage in language activities
involving language processes to produce and/or receive texts in
relation to themes in specific domains, activating those strategies
which seem most appropriate for carrying out the tasks to be
accomplished.

(CEFR: 9, emphasis in original)

12
 The CEFRs model of language use

TLU
domain

Strategies
The
Topic Language language
(situation, Task Processes
activity learner/
theme)
Knowledge
user

Monitoring,
assessment

13
 Relating the CEFRs model to a socio-cognitive model
of language learning in a classroom setting

TLU
domain

Strategies
The
Topic Language language
(situation, Task Processes
activity learner/
theme)
Knowledge
user

Monitoring,
assessment

14
 A socio-cognitive model of language learning in a
classroom setting

The classroom
Assessment
for learning
Method Learning
strategies,
Strategies
The
Topic Language language
Syllabus (situation, Task Processes
activity learner/
content theme)
Knowledge
user
Language
point

Lexis

Monitoring,
assessment

15
 Formative and summative focus

The classroom
Assessment
for learning
Method Learning
strategies,
Strategies
The
Topic Language language
Syllabus (situation, Task Processes
activity learner/
content theme)
Knowledge
user
Language
point

Lexis

Monitoring,
assessment

16
 Formative and summative focus

The classroom
Assessment
for learning
Method Learning
strategies,
Strategies
The
Topic Language language
Syllabus (situation, Task Processes
activity learner/
content theme)
Knowledge
user
Language
point

Lexis

Monitoring,
assessment

Formative focus
17
 Formative and summative focus

The classroom
Assessment
for learning
Method Learning
strategies,
Strategies
The
Topic Language language
Syllabus (situation, Task Processes
activity learner/
content theme)
Knowledge
user
Language
point

Lexis

Monitoring,
assessment

Formative focus Summative focus

18
 Model for FA: performance level, difficulty and support

Performance
Too hard for this learner
level depends

Lower Level Higher

on degree of
support

..
More support,
higher
performance
level
Less support,
lower
performance Easy for this learner
level

Less Support More

19
 Formative and summative focus
Learner
cannot
manage this Too hard for this learner

Lower Level Higher

task.

8 ?
..

Easy for this learner

Less Support More

20
 Formative and summative focus
Learner
cannot
manage this Too hard for this learner

Lower Level Higher

task. Formative
focus:

8 ?
..
Give more
support on this
teaching point
so learner can
grasp it.

Easy for this learner

Less Support More

21
 Formative and summative focus
Learner
cannot
manage this Too hard for this learner

Lower Level Higher

task.

8 ?
..

Summative
focus:
Select more
suitable
tasks for Easy for this learner
finding this
learners
level Less Support More
22
 Formative and summative focus
Learner
cannot
manage this Too hard for this learner

Lower Level Higher

task. The link:

8
Formative assessment feeding
? into summative assessment:
support mechanisms, products
..
and services for teachers:
a set of validated can-do
statements,
a set of graded formative tasks,
further teacher development,
support and resources, apart from
existing teacher qualifications,
Easy for this learner teacher seminars,
certification,
teacher portfolio, exam handbooks,
exam reports, teaching and exam
preparation tips, lesson plans, etc.
Less Support More
23
Developing YL can-do statements for formative assessment:
for learners and teachers
Different types of can-do statements relating to:

Curricular objectives, course content, instructional content areas: content to

be learnt/taught

Learning how to learn, i.e. assessment for learning objectives for

developing learning skills

Language awareness, knowledge about language

Cultural issues, inter-cultural awareness

Question: Which ones are the most useful in the monitoring and improving of
learning?

24
 European Language Portfolios as formative assessment

Some of the aspects of formative assessment have been taken account of in

the European Language Portfolios

The different kinds of can-do descriptors used in ELPs:

1. Scaled CEFR descriptors;

2. Adaptations of CEFR scaled descriptors for particular learner groups, e.g.
young learners;
3. Context-specific achievement-oriented descriptors, e.g. related to syllabus
content;
4. Descriptors which do not relate directly to a language proficiency scale but
rather to independent categories such as learning strategies or descriptions of
cultural or intercultural experiences.

(Lenz & Schneider 2004)

i.e. the content to be learned, the skills to deal with learning, and the targeted
outcomes in terms of capacity for language use.

25
Developing YL can-do statements for formative assessment:
for learners and teachers

On context-specific, achievement-, syllabus-oriented descriptors from ELPs:

Being so closely related to actual classroom practice, these

descriptors may be tremendously useful in improving learners self-
assessment skills and, if linked to an ELP in some way, enhance
the perceived usefulness of that ELP. It is not surprising that
descriptors of this type were produced in large numbers for use
within or in combination with ELPs for children

(Lenz and Schneider 2004)

26
Developing YL can-do statements for formative assessment:
for learners and teachers
Summary of survey carried out among teachers of young learners (11-14 year-olds):

what teachers want from an examination board

they would like to see the development of can-do statements that relate to
strategies of language learning and use as well as can do statements based on
syllabus or course content
they see these as most useful in monitoring learning and learner self-
assessment
they would like these in the format of printed photocopiable packs or learner
workbooks, not in electronic formats such as webpages or CDs/DVDs
they see learners disinterest or even resistance as well as the insufficient
number of can-do statements as the greatest hindrance
how they would like to get involved
getting familiar with the CEFR descriptors and competencies described at each
level
receiving detailed information to help understand the different abilities that
distinguish one CEFR level from another
receiving detailed guidelines for planning courses for young learners

27
Cambridge ESOLs methodology of using can-do
statements for formative assessment, so far
1. start from the relevant CEFR scales at A1-B2
2. ensure coverage of Waystage and Threshold learning objectives
3. build on existing work on user-oriented can-do statements
the qualitatively developed Bergen can-dos and
the empirically validated ALTE and EAQUALS/ALTE can-dos
4. complement those with can-do statements from selected
European Language Portfolios developed for young learners in relevant age group
5. supplement them with new performance indicators, especially ones related to syllabus
content, learning skills, and language strategies, derived and adapted for users from
benchmarking and standards frameworks such as
the English Language Proficiency Benchmarks for primary learners (Ireland) and
WIDA consortium (US)
6. categorise the collated 2380 can-dos in the 4 skills databases into domains of language
use for young learners: classroom, leisure, general/social
7. build a framework consisting of domain, age, and level; ask teachers to fill in possible gaps
8. validate a selection of can-do statements on learners and teachers, i.e. get them endorsed
by users, similar to a process that was carried out for
Asset languages
9. scale those that are scaleable for common language proficiency scales, such as
the CEFR and
the Cambridge Common Scale of ability
10.distribute them for teachers and learners use in the classroom
28
online and
in hard copy
References
Alderson, J. C., Figueras, N., Kuijper, H., Nold, G., Takala, S. & Tardieu, C. (2004) The development of specifications for item development and classification
within the common European framework of reference for languages: learning, teaching, assessment. Reading and listening. The final report of the
Dutch CEF Construct Project. Project Report, Lancaster University, Lancaster, UK. Accessed online at: http://eprints.lancs.ac.uk/44/ (accessed 7 May
2008)
Alderson, J. C. (2007) The CEFR and the need for more research. The Modern Language Journal, 91.
Assessment Reform Group (1999) Assessment for learning: beyond the black box. Available online at: www.qca.org.uk/7444.html (accessed 7 May 2008)
Cameron, L. (2001) Teaching Languages to Young Learners. Cambridge University Press.
CILT (online) The Languages Ladder. Available online at: www.cilt.org.uk/commlangs/curricul.htm#LangLadder (accessed 7 May 2008)
Council of Europe (2001) Common European Framework of Reference for Languages. Cambridge, Cambridge University Press.
Council of Europe (2002) Common European Framework of Reference for Languages: learning, teaching, assessment:. Case studies. Strasbourg: Council of
Europe Publishing. Available online at: http://www.coe.int/T/DG4/Portfolio/documents/case_studies_CEF.doc (accessed 7 May 2008)
Council of Europe (2003) Relating language examinations to the CEFR. Manual Preliminary pilot version. Available online at:
www.coe.int/t/dg4/linguistic/Manuel1_EN.asp (accessed 7 May 2008)
Hasselgren, A. (2003) Bergen Can do Project. Strasbourg, France: Council of Europe.
Jones, N. (2002) Relating the ALTE Framework to the Common European Framework of Reference, in Council of Europe (Eds) Case Studies on the Use of
the Common European Framework of Reference, Cambridge: Cambridge University Press, 16783.
Jones, N. (2000) Background to the validation of the ALTE Can Do Project and the revised Common European Framework, Research Notes, 2, 1113.
Available online at: http://www.cambridgeesol.org/rs_notes/offprints/pdfs/RN2p11-13.pdf (accessed 7 May 2008)
Jones, N. (2002) Relating the ALTE Framework to the Common European Framework of Reference, in: Council of Europe (Eds) Common European
Framework of Reference for Languages: learning, teaching, assessment: case studies. Strasbourg, Council of Europe Publishing, 167183.
Lenz, P. and Schneider, G. (2002) Developing the Swiss model of the European Language Portfolio. In Alderson, J.C.A. (ed.) Case Studies in applying the
Common European Framework, Strasbourg, Council of Europe, 68-86.
Lenz, P. and Schneider, G. (2004) Introduction to the bank of descriptors for self-assessment in European Language Portfolios. Available online at:
http://www.coe.int/T/DG4/Portfolio/?L=E&M=/documents_intro/Data_bank_descriptors.html (accessed 7 May 2008)
Leung, C. (2004) Developing formative teacher assessment, Language Assessment Quarterly, 1(1), 1941.
Little, D. (2007) The Common European Framework of Reference for Languages: Perspectives on the making of supranational language education policy. The
Modern Language Journal, 91, 641685.
Little, D. and Lazenby-Simpson, B. (2003) English language proficiency benchmarks for non-English-speaking pupils at primary level. Integrate Ireland
Language and Training.
North, B. (2000). The development of a common framework scale of language proficiency. New York, Peter Lang.
North, B. and Schneider, G. (1998). Scaling descriptors for language proficiency scales. Language Testing 15, 2, 217262.
North, B. (2002). Developing descriptor scales of language proficiency for the CEF common reference levels. In Alderson, J.C.A. (ed.) Case Studies in
applying the Common European Framework, Strasbourg, Council of Europe, 87-105.
North, B. (2006) The CEF levels & descriptor scales. Cambridge ESOL staff seminar talk. June 2006.
Nuffield Languages Inquiry (2000) Languages: the next generation. Available online at:
http://languages.nuffieldfoundation.org/filelibrary/pdf/languages_finalreport.pdf (accessed 7 May 2008)
Nuffield Languages Programme (2002) A Learning Ladder for Languages: possibilities, risks and benefits. Available online at:
http://languages.nuffieldfoundation.org/filelibrary/pdf/learning_ladder.pdf (accessed 7 May 2008)
Taylor, L. & Jones, N. (2006) Cambridge ESOL exams and the Common European Framework of Reference (CEFR), Research Notes, 24, 25. Available
online at: http://www.cambridgeesol.org/rs_notes/rs_nts24.pdf (accessed 7 May 2008)
Van Ek, J. A. & Trim, J. L. M. (1990) Threshold 1990, Cambridge, Cambridge University Press.
Van Ek, J. A. & Trim, J. L. M. (1990) Waystage 1990, Cambridge, Cambridge University Press.
Weir, C. (2004) Language testing and validation, Basingstoke, Palgrave Macmillan.
Weir, C. (2004) Limitations of the Common European Framework of Reference (CEFR) in developing comparable examinations and tests. Internal document.
30 Cambridge ESOL. June 2004.
The end

Thank you for your attention.

We welcome any observations, comments, suggestions, or

any other form of feedback.

Szilvia Papp
Papp.S@cambridgeesol.org

Neil Jones
Jones.N@cambridgeesol.org
31