Hill McNamara (2011) - Framework For Classroom-Based Assessment

428317 LTJ
/$1*8$*(
Article 7(67,1*
Language Testing
Developing a comprehensive,
29(3) 395–420
© The Author(s) 2011
Reprints and permission:
empirically based sagepub.co.uk/journalsPermissions.nav
DOI: 10.1177/0265532211428317
research framework for ltj.sagepub.com
classroom-based assessment
Kathryn Hill and Tim McNamara

University of Melbourne, Australia
Abstract
This paper presents a comprehensive framework for researching classroom-based assessment
(CBA) processes, and is based on a detailed empirical study of two Australian school classrooms
where students aged 11 to 13 were studying Indonesian as a foreign language. The framework can
be considered innovative in several respects. It goes beyond the scope of earlier models in addressing
a number of gaps in previous research, including consideration of the epistemological bases for
observed assessment practices and a specific learner and learning focus. Moreover, by adopting
the broadest possible definition of CBA, the framework allows for the inclusion of a diverse range
of data, including the more intuitive forms of teacher decision-making found in CBA (Torrance &
Pryor, 1998). Finally, in contrast to previous studies the research motivating the development of
the framework took place in a school-based foreign language setting. We anticipate that the
framework will be of interest to both researchers and classroom practitioners.
Keywords
classroom-based assessment, continuity in language learning, language policy, language testing,
school foreign language learning, transition in language programs
Introduction
The aim of the study reported in this paper was to develop from the ground up a compre-
hensive framework for conducting research on classroom-based assessment. The context
for this research is an increasing trend to devolve responsibility for assessment to classroom
teachers (e.g. Cumming & Maxwell, 2004; Davison & Leung, 2009), together with a grow-
ing awareness of the impact of assessment on learning (Black & Wiliam, 1998). However,
we would argue there is a lack of coherence in terms of focus and approach amongst
existing CBA studies. Previous studies have focused on issues such as validity and reliability
Corresponding author:
Kathryn Hill, Medical Education Unit, School of Medicine, University of Melbourne, Victoria 3010, Australia
Email: kmhill@unimelb.edu.au
396 Language Testing 29(3)
(e.g. Gipps, 1994; Heurta-Macias, 1995), criteria and standards (e.g. Leung, 2007; Leung
& Teasdale, 1997b) and the influence of external assessment and reporting regimes on
classroom practices (e.g. Brindley, 1998, 2001; Clarke & Gipps, 2000; Davison, 2004).
Relatively fewer studies, however, have focused on the actual processes of classroom-based
assessment (research on which this framework is intended to guide) and even then only
focus on aspects of the assessment process rather than assessment as a comprehensive whole.
The framework we propose is based on a detailed empirical study of two Australian
school classrooms where students aged 11 to 13 were studying Indonesian as a foreign
language. Although essentially a bottom-up study, an initial orientation to the research
reported on here was developed from the existing literature. In this case, two themes from
the literature guided the investigation: the dimensions and scope of CBA, as well as the
way these two aspects intersected with each other.
Dimensions
McNamara (2001) sets out three critical dimensions of assessment: evidence, interpreta-
tion and use. According to McNamara, CBA is ‘[a]ny deliberate, sustained and explicit
reflection by teachers (and by learners) on the qualities of a learner’s work’ and the use
of this information, for example, ‘as an aid to the formulation of learning goals…’ (2001,
p. 343). However, the literature reveals significant diversity in how each of these dimen-
sions is understood (Table 1).
The definition adopted for the study is designed to reflect this diversity and, in line with
ethnographic principles, to admit all possible evidence. We thus propose the following
amended definition of CBA:
any reflection by teachers (and/or learners) on the qualities of a learner’s (or group of learners’)
work and the use of that information by teachers (and/or learners) for teaching, learning (feedback),
reporting, management or socialization purposes.
Note this definition of CBA incorporates both formative (or assessment for/as learning)
and summative assessment (assessment of learning).
There are also important differences in how the unit of analysis in CBA research is
defined. Whereas Leung and Mohan (2004), for example, focused on a planned assessment
activity, the unit of analysis in Rea-Dickins’ (2001) study, the ‘assessment opportunity’,
also included activities identified post hoc (i.e. on reflection) by the participating teachers.
Torrance and Pryor (1998) go even further, however, to include the forms of assessment
occurring ‘within the largely taken-for-granted discourse structure of teacher questions
and pupil responses’ (p. 131). The difficulty here is identifying when (or whether) assess-
ment is actually taking place, given the often intuitive nature of teacher decision-making
in CBA (Rea-Dickins, 2006). The challenge for the researcher is thus how to understand
CBA when it takes place ‘without conscious planning, as part of ordinary interaction with
students, that is, when [teachers] consider themselves to be teaching’ (Leung, 2005, p. 877).
This phenomenon is nicely captured in the following comment from a teacher in our study:
Hill and McNamara 397
Table 1. Dimensions of classroom-based assessment
Evidence Data What is assessed? • Valued enterprises

Approach How is evidence • Planned (e.g. McNamara, 2001); Incidental
collected? (e.g. Torrance & Pryor, 1998)
• Visible (e.g. McNamara, 2001); Embedded
(e.g. Rea-Dickins, 2006)
Target Who is assessed? • Individual (e.g. McNamara, 2001); Group/
Class (e.g. Torrance & Pryor, 1998)
Agent By whom? • Teacher/Learner (e.g. McNamara, 2001)
Interpretation Reflection Level of attention • Sustained (e.g. McNamara, 2001); Fleeting
(e.g. Rea-Dickins, 2006)
Criteria Values guiding • Explicit (e.g. McNamara, 2001);
assessment Unconscious (e.g. Wiliam, 2001)
• External/indigenous (e.g. Leung &
Teasdale, 1997)
Use Purpose How is evidence • Assign level (reporting)
used? • Teaching (plan/modify)
• Learning
• Management (of behaviour or teaching)
• Socialization (to culture of assessment)
(e.g. Tunstall & Gipps, 1996)
Agent By whom? • Teacher/Learner (e.g. Black & Wiliam, 1998)
• School (e.g. Rea-Dickins, 2001)
It’s all like you’ve got antennae sticking out of your ears and it all comes in… You’re constantly
processing it, you’re constantly building up, I mean, I just know, just sitting in class, you know,
you become aware of who’s got the answer or who’s gonna have a go at it. Like Arthur will
keep trying till the cows come home. You know he won’t get it straight away but, you know?
So but, there’s that but there’s also, there’s their identity in the class and there’s all sorts of
things. (Year 6)
In order to allow for these more intuitive forms of assessment, we have extended Rea-Dickins’
(2001) notion of the ‘assessment opportunity’ to include the following:
any actions, interactions or artifacts (planned or unplanned, deliberate or unconscious, explicit

or embedded) which have the potential to provide information on the qualities of a learner’s (or
group of learners’) performance.
Once again this definition is deliberately broad. In particular, it allows the observer
to consider the nature of the ‘affordances’ (van Lier, 2004) or evidence available for
assessment in the classroom, in recognition of the possibility that incidental forms of
assessment are taking place.
Table 2. Scope and dimensions of CBA
Scope Dimensions
1. What do teachers do? Evidence What activities or behaviours are
assessed? Is it planned/incidental,
explicit/embedded? Does it target
individuals, groups, the whole class?
Interpretation Is reflection sustained or fleeting?
Use How is assessment used?
2. What do they look for? Interpretation What criteria do they apply?
3. What theory or Interpretation What are the values guiding
‘standards’ do they use? assessment?
4. Do learners share the Evidence What are learners’ beliefs about
same understandings? Interpretation how assessment is conducted,
Use interpreted and used?
Scope
In terms of scope, the following focal research questions, reflecting issues identified by
Leung (2005) and Rea-Dickins (2006) respectively, were used to guide the empirical study:
1. What do language teachers do when they carry out classroom-based assessment?

2. What do they look for when they are assessing learners?
3. What theory or ‘standards’ do they use?
4. Do learners share the same understandings?
The relationship between scope and dimensions is set out in Table 2. Questions 2 and 3
primarily relate to ‘interpretation’, while Questions 1 and 4 investigate all three dimen-
sions (‘evidence’, ‘interpretation’ and ‘use’).
In summary, to date there is no comprehensive framework for conceptualizing and
guiding research into processes of CBA in its broadest definition. Here we present such
a framework, based on a detailed empirical study of two Australian classrooms where
students aged 11 to 13 were studying Indonesian as a foreign language. Hence, following
Wiliam (2001), ‘instead of building theoretical models and then trying to apply them to
[teachers’] assessment practices, we try to theorise what is actually being done’ (p. 172).
Design of the study

As the focus is on understanding processes rather than describing outcomes, existing CBA
research is essentially qualitative in nature, typically drawing on discourse-oriented social-
constructivist approaches to cognition and language use. Torrance and Pryor (1998), for
example, drew on theories of classroom interaction, constructivist theories of learning
and theories of motivation and attribution to interpret their data (classroom observation
and interviews). Our study used ethnographic methods, in particular, participant observa-
tion and case studies, and a ‘grounded’ approach to analysis (Rea-Dickins, 2001) to explore
and extend the range and type of questions that should be asked in CBA research.
Research context
The study took place in two Indonesian language classrooms in Victoria, Australia, and
coincided with the introduction of a new curriculum and standards framework, VELS
(VCAA, 2008).1 Schooling in Victoria commences at age 5 and is divided into two levels,
primary (Preparatory to Year 6) and secondary (Years 7 to 12). Mainstream foreign language
programs in Australia involve ‘language arts’ (i.e. in contrast to, e.g., the Content-Language
Integrated Learning approach currently favoured in Europe). Furthermore, while govern-
ment schools are obliged to use the official framework for reporting, assessment at the
year levels involved in this study is not considered ‘high stakes’ (as is the case, for example,
in the UK National Curriculum).
There were a number of reasons for choosing Indonesian. Firstly, as this was an ethno-
graphic study involving participant observation, it was important to choose a language that
the primary researcher (Hill) speaks and understands. A second consideration was the avail-
ability of suitable programs. Since the 1990s there has been a particular emphasis on the
study of Asian languages in Australian language education policy. Indonesian is perceived
to be the easiest of the Asian languages promoted under the policy (e.g. in contrast to
Chinese, Japanese and Korean, there is a direct and transparent relationship between written
and spoken forms of the language). As a result, at the time of data collection Indonesian
was the most widely studied language in Victorian schools after Italian (DOE, 2006).
Participants and programs

One high school (Years 7 to 12) and one primary school (Preparatory to Year 6), both
government schools, were recruited for the study. The selected programs were both stable
and well established and could be considered ‘typical’ in terms of contact hours, resources,
curriculum and student demographic. One Indonesian class was recruited from each year
level. The Year 6 class received one hour of Indonesian instruction per week as compared
to five 55-minute classes per fortnight (or 125 minutes a week) in Year 7.
One Year 6 (primary) teacher and two Year 7 (high school) teachers volunteered for
the study. Each had a high level of competence in language teaching (Duff & Uchida,
1997) as well as a genuine interest in the research. The Year 6 teacher taught Indonesian
P-6 and had taught the participating students for four consecutive years. The Year 7 teach-
ers, both part-time, shared teaching for a single Year 7 Indonesian class.
Data collection and data analysis

Data collection took place in the final 10 weeks of Year 6 (end of primary school) and the
first 10 weeks of Year 72 (beginning of secondary school) (Figure 1) with the result that
a number of participants appear in both the Year 6 and Year 7 data sets.
Data comprised approximately 80 hours of audio-recordings (including classroom
interactions and teacher and focus group interviews) as well as participant observation,
field notes and documents (reports, worksheets, etc.). Table 3 summarizes the range of
data identified as potential sources of assessment-related information.
A summary of the data set from a single unit of instruction in Year 7 has been provided
as an example (Table 4).
Primary Year 6 (n=26) Term 4, 2005 (10 weeks)
Secondary Year 7 (n=25) Terms 1 & 2, 2006 (10 weeks)
Figure 1. Design for data collection
Table 3. Potential sources of assessment-related information
Interactions Documents
Teacher–Teacher/Researcher
planning sessions assessment task sheets
reporting sessions assessment rubrics (criteria)
discussions with researcher course outlines, work requirements
Teacher–Student
written (on board) or verbal instructions text books
explanation/clarification requests student workbooks, worksheets & assignments
oral or written feedback statements of aims (policy) at school & system level
Student–Student
discussion of task requirements teachers’ notes & ‘running records’
discussion of feedback/results written communications between teachers
self- and peer evaluations summative reports
student focus group interviews
Table 4. Data audit – ‘Introducing Yourself’ (Year 7)
Data Pair 1 Pair 2 Pair 3 Pair 4

Student ID Tam Jess* Dan Adam Jad* Sim Way Kar
Whole-class interactions 5 lessons
Pair-work interactions 4 lessons 2 lessons 2 lessons 1 lesson
Extended feedback on draft script 1 lesson – – –
Work-sheets & work samples √ √ √ √ √ √ √ √
Presentations & verbal feedback √ √ √ √ √ √ √ √
Focus Group discussion √ √ √ √ √ √ √ √
Teacher’s notes on presentations √ √ √ √ √ √ √ √
Teacher’s ledger entries √ √ √ √ √ √ √ √
Scored rubric (oral performance) √ √ √ √ √ √ √ √
Co-authored role-play script √ √ √ √ √ √ √ √
Comments & corrections on script √ √ √ √ √ √ √ √
Follow-up lesson: ‘common mistakes’ √
Note: *students who were focus group participants in both Years 6 and 7.
Table 4 shows that data for this unit of instruction (‘Introducing Yourself’) was col-
lected over the course of five lessons and includes recordings and transcriptions of whole-
class and paired interactions for each lesson. It shows, for example, that data for Pair 1
(Tamara and Jess) includes recordings of pair-work interactions from four lessons, an
extended individual feedback session, an oral presentation (including teacher feedback)
and related focus group interview discussions. Data also includes copies of the role-play
scripts they produced, the printed assessment rubric, the teacher’s notes about the pair’s
oral presentation, ledger entries, and other relevant documents (e.g. notes from the board,
completed homework exercises, etc.).
The coding process was grounded and iterative (Strauss, 1987; Strauss & Corbin, 1998).
A set of preliminary codes which had been established based on a combination of issues
suggested by theory and/or previous research were refined on the basis of a continuing
‘dialogue‘ between the research questions, the literature and the data (Ritchie & Spencer,
2002). The aim was to produce a set of analytic categories, grounded in the data, which
provided a framework ‘that leaves nothing unaccounted for and that reveals the interre-
latedness of all the component parts’ (Hornberger, 1994, p. 688). The validity of the
categories was tested through comparison of similarities and differences within as well
as across the two classroom contexts. Hence successive iterations of the scheme were
applied to a small subset of the data, starting with a transcript from a single lesson expand-
ing to additional transcripts from the same unit of instruction. Revisions were informed
by the need to provide a faithful representation of the data and to account for emerging
insights with relevance to the main research questions (Lynch, 2003).
Other measures to ensure the validity of the categories included the large quantity of
data collected and analysed (Morgan, 2002), participant checks of transcripts and analyses
as well as checking by an ‘external’ coder (a colleague with knowledge of Indonesian).
Findings
In what follows, we will report the findings in relation to each of the four focal questions
using brief illustrations from the classroom-based study and with reference to existing
CBA research.
RQ1 What do teachers do?

Four key processes were identified in the data. These were ‘planning’, ‘framing’, ‘conducting’
and ‘using’ assessment respectively.
The following section will elaborate each of these four assessment processes, using
examples from the respective year levels (Years 6 and 7).
Planning assessment
The first category, ‘Planning’, arose from an analysis of internal syllabus documents as
well as from discussions about teaching and assessment with, and between, the partici-
pating teachers. This category captures information about the type and nature of planned
Learning Activities Intercultural Understanding Key elements of

(ICU)/Language Focus (LF) standards [VELS]
Jigsaw reading – students ICU: History: Indonesia’s path to • organizes and

read short passages of democracy applies grammatical
Indonesian history leading to LF: Orang Belanda, sejarah, information and
Kemerdekaan [Independence] orang jepang, perang dunia knowledge of words
and recombine to create a kedua, kemerdekaan, pahlawan,
group timeline using teacher pada tahun [the Dutch, history,
generated word bank. the Japanese, World War II,
independence, hero, in the year]
Figure 2. Extract from Planned Learning Activities (Year 6 ‘Government’)
Jigsaw reading – students read short passages of Indonesian history leading to Kemerdekaan
[Independence] and recombine to create a group timeline using teacher generated word bank.
Figure 3. Extract from Planned Assessment Tasks (Year 6 ‘Government’)
assessment tasks and the relationship of assessment to instruction as well as to the relevant
external frameworks (this last aspect will be revisited under RQ3).
Examination of the planning documents from Year 6 showed teaching and assessment
was planned in some detail. Extracts from the documents, reproduced as Figures 2 and 3,
indicate that the same task, in this case ‘jigsaw reading’ (present in both documents), was
specified as both a ‘learning’ and an ‘assessment’ activity. Note that the format and ter-
minology (e.g. ‘intercultural understanding’) explicitly reference the mandated curriculum
and assessment guidelines.
Framing assessment for students

This category investigates how (or whether) learners in the respective classrooms came
to know that a given activity was ‘for assessment’, which Rea-Dickins (2006) argues is
important if learners are to adopt an appropriate orientation to the task. This category
became significant precisely because assessment activities in Year 6 were embedded in
teaching in a way that made the assessment dimension essentially ‘covert’ from the learners’
perspective.
In Year 7 the assessable nature of an activity was communicated verbally, as in the first
example, or in writing, as in the second.
Your task, we’re doing a writing task. On your report you have marks for reading comprehension,
writing tasks, speaking, listening and tests. This is going on your report. This is your writing
task for Term 1. (T2, Year 7)
The second example (Figure 4), which appears on the first page of the Year 7 student
workbook, informs students that each unit of instruction will be associated with an assess-
ment activity.
Work requirement Assessment
1. Introducing Yourself Oral
2. Numbers Assignment Written
Figure 4. Extract from Work Requirements (Year 7, Semester 1)
Other examples of how the assessable nature of the activity was flagged to learners
included printed assessment task specifications and scoring rubrics (see RQ 2).
Conducting assessment
The processes captured by this category range from explicit, planned, formal assessment
activities to less visible, unplanned, instruction-embedded assessment activities. Table 5
sets out the terms used to distinguish between these types.
Table 5. Terminology used to distinguish types of assessment
Term Description Example

Formal assessment Planned and evident Tests, assignments
Planned assessment Instruction-embedded Teaching activities also
opportunity (planned) used for assessment
Incidental assessment Instruction-embedded Unstructured
opportunity (unplanned) observation
A ‘planned assessment opportunity’ occurs when an assessment task is embedded in

regular classroom activities but not framed as ‘for assessment’ to learners. This was the
only type of assessment observed in the Year 6 classroom.
In this example of an ‘incidental assessment opportunity’ from Year 7, the class had
been reading through a model paragraph in preparation for a writing activity. Selecting
students in turn to read aloud afforded an opportunity for the teacher to assess individual
reading and pronunciation skills.
Um Dan can you read the next paragraph and then we’ll go um, Violet sorry Violet, Violet, um
Margie, sorry we’ll just go around, ok? (T1, Year 7)
This category was also used to code whether the assessment involved individual students,
pairs or small groups of students, or the whole class. In the following example of group-
level assessment in Year 7, asking for a show of hands provided the teacher with an indica-
tion of how the class, as a whole, had performed on a peer-corrected vocabulary test.
T: Hands up if you got them all right

[two or three students raise their hands]
T: Hands up if you only got one or two or three wrong

[about half the class raise their hands]
S: Oh, ok.
T: Very good. Excellent
(T1, Year 7)
Torrance and Pryor (1998) concluded from their study of CBA practices that assess-
ment more often occurred at the whole group level (as in this example), than at the level
of individual students.
Finally, this category investigates the teachers’ approach to CBA at a more general,
philosophical, level. Torrance and Pryor (1998) contrast convergent and divergent
approaches to assessment. Convergent assessment is concerned with ‘whether the learner
knows, understands or can do a predetermined thing’ (p. 153). It is characterized by the
use of closed questions, intended to elicit a single ‘expected’ answer. This type of ques-
tioning limits the amount and quality of information afforded for assessment purposes. It
also tends to encourage guessing as demonstrated in the following example.
T: What do you think ‘kapan’ [when] means? ‘Kapan’ [when]

S?: Oh I know. That’s in Turkish ‘close, close the door’?
T: No, it’s a question word.
S?: Um, ‘where’?
T: That’s ‘di mana’ [where]
S?: ‘Why’
S?: ‘Who’
S?: ‘How’
S?: ‘Which’
S?: ‘When’
S?: ‘Who’
S?: ‘Who’, ‘who’
S?: ‘What’
S?: ‘What’
S?: ‘When’
S?: ‘When’
S?: ‘How’
S?: ‘Why’
S?: ‘How’
S?: ‘How’
S?: ‘Who’, ‘what’, ‘when’, ‘where’, or ‘why’
S?: [.] too close
T: ‘When’
(Year 6)
Divergent assessment, on the other hand, involves asking ‘genuine’ questions (i.e. where
the answer is not predetermined in advance) and ‘require[s] learners to attend to the
principles at stake rather than the ritual of question and answer’ (Torrance & Pryor, 1998,
p. 129). Contrast the type of response elicited in the previous example (i.e. discrete ques-
tion words and guessing) with the level of learner engagement and type of knowledge
elicited using the more open, or exploratory, type of questioning used in the next example
(also from Year 6).
S1: Oh. ((translating)) ‘THE DUTCH DEFEATED THE PORTUGUESE.’

T: Ok. Why were they fighting?
S2: For their land?
S3: Spices
S4: Spices
T: For the right to control the spice trade, yeah.
S5: Why would they want to have control of them?
S6: They’re so expensive man.
S5: Expensive?
T: Remember, remember that last year we talked a lot about why (*)
S7: Is it hard to find? Is it like, hard to find (spices)?
T: Remember originally, remember originally nutmeg was only ever found on the
one island and that was such a tiny island, the Bandar islands (*)
S?: (*)
T: No
S8: Is that, you know the tree in front of the little library with the little things, are
they nutmeg?
T: No
S5: Um, um well like salt used to preserve meat are spices kind of like kind of like
the same reason?
T: No, the spices were used to mask the taste of food. So, there was no
refrigeration=
S5: Yeah that’s why they had salt.
T: or storage facility so the stuff might have been off anyway (*). But spices were
expensive so who were the people who could afford them?
S6: Rich, rich people.
T: People with lots of money.
(Year 6)
It is perhaps worth noting that the focus of this discussion (which was the only example
of divergent assessment occurring in the data) was cultural, rather than linguistic,
knowledge.
Using assessment data

This category classifies assessment as for ‘teaching’, ‘learning’, ‘reporting’, ‘manage-
ment’ and ‘socialization’ purposes respectively. Again, while these are presented as
distinct categories, analytically it is not always possible to separate one purpose from
another.
Teaching. The following example shows how the teachers’ informal observations of how
the Year 7 class was progressing (‘incidental’, ‘group-level’ assessment) was used to
inform the pace of teaching.
T1: (I’m) trying to think how long [to spend on this topic]. When I first taught it I spent
about five lessons on it but sometimes I feel like I tend to go a bit slow and maybe
I need to speed it up a bit. But sometimes I feel that [going slow] also works better
as well. But that class, I don’t know.
T2: I only had them period 6.
T1: But they’ve really pulled up their socks.
T2: They have. I reckon we can crank it up a notch.
(Year 7)
Learning (feedback). Assessment-related information was also used to promote immediate

learning, most commonly through the provision of feedback on performance. This section
will consider the type of feedback provided to learners, including whether it was verbal
or written, and provided to individuals, groups of learners or the whole class. (The actual
content of feedback will be considered under RQ2.)
Following Tunstall and Gipps (1996), feedback has been classified according to
whether it is primarily ‘person-(or ‘ego-’) referenced’ or ‘task-referenced’ in nature. We
have used the qualifier ‘primarily’ to acknowledge that these represent tendencies rather
than absolute categories and in practice the two types often coincide. The main target of
‘person-referenced’ feedback is the student’s ego. Feedback is non-specific, taking the
form of reward or punishment, approval or disapproval (as in the next example) and
often involves comparison with peers.
You’re very smart. That was very good. Very, very good. (T1, Year 7)
In contrast, the focus of ‘task-referenced’ feedback is the learner’s performance in relation

to specific aspects of the task requirements and/or qualities and standard of performance
(Tunstall & Gipps, 1996, p. 398). This feedback may, in turn, be classified as confirmatory,
explanatory or corrective.
Confirmatory feedback typically occurs where a single, ‘correct’, response is required.
It may take the form of a ‘tick’, a nod, or repetition of a correctly provided response. In
this example simply moving on to the next item for translation (dokter gigi [dentist]) can
be interpreted as confirmation that the first task (i.e. translation of ‘guru’ [teacher]) had
been completed successfully.
T: What’s a guru? [teacher]

S: ‘Teacher’
T: What’s a doktor gigi? [dentist]
(T1, Year 7)
Feedback is explanatory when it is used to highlight and/or explain the successful aspects
of performance.
Very good, um, good body language too. I liked the way you shook hands, said, then you waved
goodbye to each other at the end. (T1, Year 7)
The third type of task-related feedback, corrective feedback, is used to draw attention to
the gap between what the student has done and what was expected.
Corrective feedback may vary according to degree of explicitness, using Aljaafreh
and Lantolf’s (1994) ‘Regulatory Scale’. Based on an analysis of the level of assistance,
or ‘regulation’, provided by tutors, the scale specifies increasingly explicit forms of
corrective feedback, from ‘0’, where the learner is completely self-regulating, to ‘12’,
where the teacher provides additional examples to illustrate the point. In the following
example, we can see that the feedback became increasingly explicit over the course of
the interaction. The task was to translate the number, ‘1,979’ into Indonesian.
S1: Yep ok seribu sembilan ratus tujuh sembilan should be tujuh puluh
[one thousand nine hundred seven_ nine] [seventy]
T: Say it again Type 3 indicates something
is wrong
S1: Seribu sembilan ratus tujuh sembilan
[one thousand nine hundred seven^ nine]
T: Missing one word Type 6 identifies nature
of the error
S1: Ok
T: What’s the word please [*] over here
S2: Jake
T: What’s your name?
S2: Jake
T: Jake
S2: Tujuh puluh satu [seventy one]
T: Tujuh puluh? [seventy?] Type 5 identifies the location
(rising intonation) of the error.
S2: Satu [one]
T: Sembilan [nine] Type 10 T provides correct form.
S2: Is that a nine?
T: That’s a nine.
(T1, Year 7)
Table 6 summarizes the relationship of the feedback types as they have been defined here
to terminology used by Tunstall and Gipps (1996) and others.
Reporting. In the following example the Year 7 teachers are using assessment-related
information to inform decisions about students’ end-of-year reports.
Table 6. Summary of feedback types
Feedback type Description Related terminology

Person-referenced Non-specific; affective or Evaluative feedback
conative; comparison with (Tunstall & Gipps, 1996)
peers Ego-involving feedback
(Nicholls, 1989)
Task-referenced Descriptive feedback
(Tunstall & Gipps, 1996)
Confirmatory Ticks; repetition of correct
response; ‘moving on’
without comment
Explanatory Highlights successful Achievement-related feedback
aspects of performance (Tunstall & Gipps, 1996)
Corrective Identifies gap between Improvement-related feedback
performance and (Tunstall & Gipps, 1996)
expectation Negative feedback
(Nassaji & Swain, 2000)
Other regulation
(Aljaafreh & Lantolf, 1994)
She got a 17 for her writing task, her personal writing task. She only got 11 out of 20 for her
[first] oral and then 19 out of 24 for her [second] oral.
(T1, Year 7)
Management. A number of researchers have noted the use of assessment for classroom
management, such as controlling or reinforcing behaviour, for encouragement or for creat-
ing a positive atmosphere (Mavrommatis, 1997; Tunstall & Gipps, 1996). Torrance and
Pryor (1998) argue the ubiquitous initiation-response-feedback (IRF) sequence (Mehan,
1979), featured in the next example, functions as much to manage teaching or ‘accomplish
the lesson’ as to assess group-level knowledge.
T: Ok. What’s the word we use when we want to get a number in the answer?
S1: Ke- [the prefix used for ordinals]
T: Now I’ve worn myself out for four years teaching you this one
S2: Berapa? [how many?]
T: Hey! Who said that? Well done!
(Year 6)
This example demonstrates the tension between the pedagogic and managerial func-
tions of classroom-based assessment. While ‘Ke-’ was not the expected response, it provided
an opportunity to explore understanding of the use of ‘ke-’ [prefix used for ordinals] as
opposed to ‘berapa’ [how many]. However, instead the teacher gave priority to moving
through the lesson.
Socialization. Finally researchers have highlighted a role for assessment in the ‘socializa-
tion’ of learners into the local conventions of teaching and assessment (Moni, 1999;
Torrance & Pryor, 1998; Tunstall & Gipps, 1996). In this interaction, from Year 7, the
teacher introduces learners to the (at this point novel) concept of using descriptive feedback
from successive oral assessment tasks to monitor their own progress.
T1: Remember the rubrics, remember these from your last oral presentation?
S?: All right
T1: What did I ask you to do with these last time?
S1: Keep them
T1: Ok. Today when I give you your [new] rubric I want you to look at it very closely. I
want you to stick it in your books. The [rubric from the] last lesson, last oral. Look
at it very closely. Have a look at where you’ve improved.
S2: The conversation one?
T1: Yes the conversation.
S3: um (*)
T1: Where you’ve improved.
(T1, Year 7)
Again, the same interaction could also be seen as an example of using assessment for the
purpose of learning (self-assessment).
RQ2 What do teachers look for?

Sources of information about the valued enterprises and qualities and standards operating
in the respective classrooms included syllabus documents, written or verbal instructions,
assessment rubrics, written or verbal feedback, report-writing meetings and written reports.
This category has been divided into information provided ‘in advance’ of task performance,
information provided in written or spoken ‘feedback’ and information provided through
‘reporting’ respectively.
In advance
Figure 5 provides a segment of the instructions to students for a Year 6 speaking and
writing task. It provides advance information about the obligatory components of the task
(Parts A, B and C), the response format (writing and speaking), performance conditions
(‘with a partner’) and weighting (15 from a total of 60 points). However, the specified
criteria (‘organization’, ‘persistence’, ‘getting along’ and ‘confidence’) refer to personal
qualities rather than any feature of written and spoken language.
Activity 2: Interview an alien (15 points)

A. With a partner devise questions to ask an alien. Write your questions.
B. Using these sentences interview a class member to find out which planet they are from.
C. Perform a role-play with your chosen alien.
…
Your goal is a total of 60 points.
Bonus points will be awarded for organization, persistence, getting along and confidence.
Figure 5. Extract from instructions – ‘Space’ Unit (Year 6, Term 3)
In feedback
A number of researchers have highlighted the importance of feedback in communicating
criteria and standards as well as the strategies for achieving them (e.g. Sadler, 1989;
Torrance & Pryor, 1998; Tunstall & Gipps, 1996).
In the following example the Year 7 class had been given an aural discrimination task,
where they had to note how many times they heard a specific vocabulary item. Here the
teacher provides information about the acceptable standard (or range of performance) for
this task, that is, exactly seven times (the correct answer) or close to that number (e.g.
five times).
Ok, whether you heard it five times or seven times as long as you’re in the vicinity you should
be pretty happy with yourselves that you’ve heard it that many times. If you’ve written down
‘once’ or if you’ve written down ‘24’ then there’s a bit of a problem. (T2, Year 7)
In reporting
Another source of evidence about what teachers look for was provided during the report-
ing process. The following example occurred during the end of year reporting meeting in
Year 7.
T1: And Jessica, she’s still going great guns.

T2: Jessica’s good.
T1: Yeah, she’s really good and her pronunciation has improved and she’s, not that she
lacked confidence, but she’s now able to draw it all together a bit more. Do you
know what I mean? Bring it all together I think.
(Year 7)
A number of researchers have found that teachers draw on a number of factors outside
the official assessment criteria (e.g. Harlen, 1994; Leung, 2007; Leung & Teasdale, 1997;
Mavrommatis, 1997). Torrance and Pryor (1998), for example, noted a widespread
Topics Weeks Resources Work requirements
Introducing Yourself 2 Flashcards, Puzzles, Games Role-play
Numbers, Colours 2 Coursebook, Computers Numbers assignment

Numbers test
Figure 6. Extract from Year 7 syllabus document
perception amongst teachers that ‘knowledge of the ‘whole child’ is important in interpret-
ing performance and achievement’ (p. 36).
In the next example, the Year 7 teachers appeared to rely on a shared, but largely
unarticulated, understanding of the ‘expected’ level on VELS for Year 7 (4.5).
He’s been away, a lot of problems with absenteeism and that’s one of the reasons he’s down a
bit whereas everyone else, everyone else in my eyes is performing at 4.5. (T1, Year 7)
Tunstall and Gipps (1996) suggest teachers have a ‘notion of excellence’ which they
characterize as part of the teachers’ ‘guild knowledge’. This ‘guild knowledge’ informs
what Wiliam (2001) has termed ‘construct-referenced’ assessment, which ‘relies on the
existence of a construct (of what it means to be competent in a particular domain) being
shared by a community of practitioners’ (pp. 172–173).
RQ3 What theory or standards do teachers use?

A number of researchers have postulated a close relationship between teachers’ representa-
tions of the subject or content area, their pedagogic principles, and their assessment
practices (e.g. James, 2006; Leung, 2005, 2007; Thomas & Oldfather, 1997; van Lier,
2004; Wiliam, 2001). Hence this category explores articulated views regarding the subject
(Indonesian), language and language learning, and assessment, which may underlie class-
room practice.
Views of the subject or content area (Indonesian)

Syllabus documents provided an important source of evidence regarding how the disci-
pline (i.e. Indonesian) was constructed in the respective classrooms. For example, the
Year 7 syllabus document (Figure 6) comprises a series of generic (culturally neutral)
topics.
In contrast, the Year 6 document (Figure 2) includes a specific focus on ‘Intercultural
Understanding’ (i.e. ‘History: Indonesia’s path to democracy’), which then frames the
‘Language Focus’ (a list of vocabulary items: ‘the Dutch, history, the Japanese…’).
Beliefs about language and language learning

This category investigates the theories of language and language learning and models of
progression that appear to underpin teachers’ beliefs and practice. For example Torrance
and Pryor (1998) suggest that a convergent approach to assessment, which predominated
in both years 6 and 7, reflects a behaviourist view of learning while divergent assessment
reflects a social constructivist view.
The following example provides evidence about the teacher’s attributions (or beliefs
about the student’s potential for learning) used to explain a learner’s relative success or
failure. The student, Karim, was the only absolute beginner in the class.
R: Karim was good yesterday

T: Yeah he’s a little trooper
R: He’s got a great attitude to learning. His approach is ‘well it’s just
another subject’.
T: You can tell he speaks another language at home
R: No, no, no, but some of them [say] ‘I can’t learn a language. I’m not good at it’
T: Even that wouldn’t work for everyone. I reckon he speaks a second language at
home.
(Year 7)
The teacher here attributes Karim’s performance to his presumed bilingualism, a personal
background variable, rather than to effort alone. This in turn suggests a belief that knowl-
edge of a second language makes it easier to learn a third.
Beliefs about assessment

The following comment from the Year 6 teacher (who had taught the same students for
four years) suggests a belief in the importance of intuition and knowledge of the students
over time rather than a reliance on formal assessment.
I’ve had these for four years and I know where they’re at and I can probably pull a kid out at
the end and say, ‘Yes well he’s not too good at this, yeah, but he’s improving in that’ and whatever.
But yeah if you’ve had them for only one year it’s very, very hard to track. Yeah well it would
be very difficult. (Year 6)
In the next example, which occurred during a Year 7 planning meeting, the teachers
express a belief in the importance of assessing each unit of study.
T1: I did ‘Families and friends’ – do you feel you finished ‘Physical and emotional
descriptions’?
T2: Pretty much but I didn’t finish it off how I would have liked. I would have liked to
do some sort of test or something at the end.
T1: Do you want to finish that off?

T2: I think it would be a waste of time given they haven’t had anything for four weeks.
(Year 7)
The final comment suggests a view (supported by research) that assessment needs to
take place reasonably soon after instruction to be effective in reinforcing learning
(Black & Wiliam, 1998). It is also consistent with a mastery orientation to assessment
and learning which, according to Thomas and Oldfather (1997), reflects a transmission
model of learning.
The study found teachers’ articulated views were broadly reflected in classroom prac-
tice. For example, it concluded that the Year 6 teacher’s focus was exposing learners to a
rich variety of culturally embedded language and informal assessment whereas in Year 7
the focus was on mastery (as evidenced by regular, formal assessment) of a relatively
narrow repertoire of linguistic items (Hill, 2010).
RQ4 Learner understandings

Rea-Dickins (2006) and others have highlighted the need for further research on assess-
ment from a learner perspective. Hence this category explored learner understandings
regarding the nature of the subject (Indonesian, and language and foreign language learning
more broadly), as well as their notions of assessment (criteria and standards).
Opportunities to investigate students’ understandings arose in teacher-to-learner inter-
actions (e.g. when learners are called on to repeat instructions), learner-to-teacher interac-
tions (e.g. clarification requests) and learner-to-learner interactions (e.g. discussion of
task requirements). These interactions provide access to learners’ existing understandings
of assessment and criteria as well as to how they identify and resolve conflicting perspec-
tives (Torrance & Pryor, 1998). They also highlight any disparities between teacher inten-
tions and learner understandings.
Understandings of language learning

As with their teacher (cited earlier), these Year 7 students attribute their friend’s com-
petence in Indonesian to ‘innate’ variables (ethnicity and general intelligence), rather
than effort.
R: On the tape (you said) you thought she was good at Indonesian, so
S1: Yeah she is.
R: Why do you think that is?
S1: Cause she’s Vietnamese.
R: That would explain being good at Vietnamese but
S1: Yeah well isn’t it similar sort of?
R: Not really.
S1: Well see that just goes to show how much I don’t know.
S2: I think she’s really smart like she’s like smart at everything.
R: Is she? She does well in all of her subjects does she?

S2: Yeah she’s got a big brain
(Year 7)
Understandings of assessment
Previous research has found that learners often draw on their own, possibly incongruent,
understandings of task, criteria and standards (Coughlan & Duff, 1994; Moni, 1999;
Torrance & Pryor, 1998). In the following example, the Year 7 teacher anticipates and
expressly discourages a known propensity for students to focus on presentation at the
expense of content in their written work.
T: Please remember I’m collecting these today and I’m not marking them on how
pretty your border is. I’m marking you on your Indonesian…
T: Ok girls. Writing more than pictures.
S1: But I like pictures.
T: I know but I would like some writing, ok? Girls in this row in particular. Violet,
girls, Jessica, you need to do your writing first. Do your heading later, ok?
S2: (*)
T: Do it later. I’d rather you did it at the end.
S2: (*)
T: No. Write, write, write, write!
(T2, Year 7)
A number of researchers have emphasized the importance of transparency with regard

to assessment and criteria (e.g. Clarke, 1998; Gipps, 1994; Rea-Dickins, 2001, 2006;
Torrance & Pryor, 1998; Tunstall & Gipps, 1996). Given the predominantly embedded
nature of assessment in the Year 6 classroom we were particularly interested to compare
teacher intentions with learner understandings. In the following example, the Year 6 teacher
was going through a student workbook with the researcher.
R: All right and so that’s just, what does the eight mean?
T: Depending, they’re not very huge errors so I just figured that that would get an
eight. A person who had incorrect use of a verb or whatever, made a major mistake
might have got seven and a half or something like that
A subsequent discussion with a student focus group revealed a close correspondence

between their understandings and the Year 6 teacher’s explanation of her scoring, that is,
marking is based on the number and type of errors.
R: What, what sort of things do you think [the teacher’s] looking at when she gives you
an eight?
S: Um your language, your spelling, how you put them [words?] in order like if you put
them in the wrong order she’d take a point off or, something um
Table 7. A framework for research on CBA processes
1. What do language teachers do?

1.1. Planning Assessment Is there planning for assessment? How detailed is planning?
What is its intended relationship to instruction? How does
it relate to external standards and frameworks?
1.2. Framing Assessment Is assessment made explicit to learners? How is this done?
1.3. Conducting What opportunities does the classroom provide for assessment?
Assessment Does assessment tend to focus on the class, group/pairs of
students or individuals?
1.4. Using Assessment Data How is assessment-related information used?
Teaching
Learning (feedback) Person-referenced
Task-referenced
• Confirmatory
• Explanatory
• Corrective
Reporting
Management
Socialization
2. What do teachers look for?

What information about valued enterprises, qualities and standards is available?
2.1. In Advance in written/verbal instructions and/or assessment rubrics?
2.2. In Feedback in written and/or verbal feedback?
2.3. In Reporting in reporting deliberations and/or in written reports?
3. What theory or ‘standards’ do they use?

3.1 Teacher Theories What does the data reveal about teachers’ beliefs about
& Beliefs • the subject or content area
• second language learning and teaching, and
• the nature of assessment?

4.1. Learner Theories What does the data reveal about learners’ beliefs about
& Beliefs • second language learning, and
• the nature of assessment?
Discussion: A framework for CBA research

The findings from the study can now be summarized as a framework for CBA research,
as presented in Table 7.
How does this framework compare with the findings of previous research, and what
claim does it have to be more comprehensive or adequate? Previous research into teacher
assessment practices can be divided into studies which identify ‘stages’ of the assessment
process and those concerned with creating inventories of classroom-based assessment
practices (or ‘strategies’). Clarke (1998) attempted to combine both approaches by pro-
posing a model of classroom-based assessment which grouped different assessment
strategies according to function. These were ‘planning’, ‘sharing learning intentions’

(learning targets, criteria and standards), ‘marking and feedback’, ‘target setting’ and
‘reporting achievement’. Torrance and Pryor (1998), however, were more cautious about
postulating a direct link between strategy and function, preferring to relate an observed
behaviour (e.g. ‘T observes P at work’) to a possible teacher intention (as well as a pos-
sible effect on the learner).
Rea-Dickins (2001) used an ethnographic study of three elementary school classrooms
to devise a model for analyzing teacher assessment decision-making processes. The result-
ing model comprised four phases: a planning phase (identifying the purpose, choosing a
focus, choosing an activity), an implementation phase (introducing the assessment, teacher
observation and response strategies and learner strategies), a monitoring phase and a
recording phase. Mavrommatis (1997) used an investigation of classroom-based assess-
ment practices of primary school teachers in Greece to develop an ‘assessment episode
framework’, comprising four interrelated stages: ‘evidence collection’, ‘evidence inter-
pretation’, ‘teacher response’ (feedback, grading) and ‘impact on pupils’.
The relation of the framework proposed in this study to previous research on the pro-
cesses of classroom-based assessment is summarized in Table 8.
What Table 8 attempts to demonstrate is that these earlier studies essentially address
only the first part of the proposed framework, ‘What do teachers do?’ While other studies
have considered what teachers look for (e.g. Leung & Teasdale, 1997), the epistemological
beliefs underpinning assessment practices (e.g. James, 2006) and learner perspectives
(e.g. Rea-Dickins, 2006) to date there have been none, it is argued, which address all
aspects included in the proposed framework.
Conclusion
This paper proposes a comprehensive framework for researching classroom based assess-
ment processes. The framework can be considered innovative in several respects. First,
as argued in the previous section, the framework goes beyond the scope of earlier models
and addresses a number of gaps in previous research, including consideration of the
epistemological bases for observed assessment practices and a specific learner and learn-
ing focus.
Second, by adopting the broadest possible definition of CBA (‘any reflection by teach-
ers (and/or learners) on the qualities of a learner’s (or group of learners’) work and the
use of that information by teachers (and/or learners) for teaching, learning, reporting,
management or socialization purposes’), the framework allows for the inclusion of a
diverse range of data.
Furthermore, the chosen unit of analysis, the ‘assessment opportunity’ (‘any actions,
interactions or artefacts (planned or unplanned, deliberate or unconscious, explicit or
embedded) which have the potential to provide information on the qualities of a learner’s
(or group of learners’) performance’), enables consideration of the more intuitive forms
of teacher decision-making in CBA.
Finally, whereas previous studies have been conducted in the context of general educa-
tion (e.g. Mavrommatis, 1997), English literacy (e.g. Torrance & Pryor, 1998) or English
as an Additional Language EAL classes (e.g. Rea-Dickins, 2001) the research motivating
Table 8. Comparison with existing models
Clarke (1998) Rea-Dickins (2001) Mavrommatis (1997)

functions phases episodes
1. What do language teachers do?
Planning Assessment Planning Planning phase Evidence collection
(identifying the purpose, Evidence
choosing a focus, interpretation
choosing an activity)
Framing Assessment Sharing learning Implementation
intentions (learning phase (introducing the
targets, criteria & assessment, teacher
standards) observation & response
strategies & learner
strategies)
Monitoring phase
Conducting
Assessment
Using Assessment Marking & Recording phase Teacher response
Data feedback, Target (feedback, grading)
setting, Reporting
achievement
2. What do teachers look for?

In Advance Sharing learning – –
intentions (learning
targets, criteria &
standards)
In Feedback – – –
In Reporting – – –
3. What theory or ’standards’ do they use?

Teacher Theories & – –
Beliefs
Learner Theories & – – ‘Impact on pupils’
Beliefs
the development of the framework took place in a school-based foreign language setting.
It is hoped that other researchers will investigate the utility of the framework for investi-
gating CBA at different levels of education (e.g. tertiary classrooms), program types
(especially content-based language programs) and policy contexts (especially in high
stakes assessment regimes).
The aim of the empirical study was to understand rather than evaluate CBA practices
in the respective classrooms with the aim of expanding, rather than answering, the ques-
tions that should be asked in CBA research. However, in the words of Newman, Griffin
and Cole (1989), ‘descriptions of how a system works are never far removed
from questions about how to make it work better’. There is already a volume of research
evidence regarding the effects of different CBA practices on learning, not least of all that
found in Black and Wiliam’s (1998) influential meta-analysis. However, there is clearly
a place for experimental studies of how the different CBA processes outlined in this paper
might impact on learning outcomes.
In conclusion, we anticipate that the framework will be useful for researchers interested
in understanding classroom-based assessment and for teachers wishing to gain greater
insight into the integration of assessment in their everyday teaching practices and the
impact of their assessment practices on learning. As classroom-based assessment is increas-
ingly a focus of policy and research, the need for such a comprehensive framework for
both researchers and practitioners has never been more urgent.
Transcription symbols
(*) Indicates a section of the transcript was not audible.

two days? Underlining is used to indicate emphasis
(ask) (Brackets) are used to indicate uncertainty about the transcription
((laughs)) ((Double brackets)) are used to provide contextual information
anda Italics are used for Indonesian transcription
anda [you] [Square brackets] are used to gloss Indonesian transcription
ANDA CAPITALS are used to indicate text which is read aloud
A.N.D.A. C.A.P.I.T.A.L.S. with full stops indicate where text is being spelled out
R: Bolding is used to highlight teacher/researcher contribution
S? Identity of student not known
Notes
1. The Victorian Essential Learnings & Standards (VELS) (VCAA, 2008) contain a particular
emphasis on ‘intercultural understanding’ as well as the ‘interdisciplinary’ and ‘interpersonal’
aspects of language learning. It is linked to year levels (or ‘stages of learning’) and specifies
separate trajectories (or ‘Pathways’) for ‘beginning’ and ‘continuing’ students post-primary.
2. This design reflects the focus of the larger study from which the data for this paper is drawn;
that is, the issue of continuity between primary and high school language programs.
References
Aljaafreh, A., & Lantolf, J. P. (1994). Negative feedback as regulation and second language learn-
ing in the Zone of Proximal Development. The Modern Language Journal, 78(4), 465–483.
Black, P., & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education,
5(1), 7–74.
Brindley, G. (1998). Outcomes-based assessment and reporting in language learning programmes:
A review of the issues. Language Testing, 15, 45–85.
Brindley, G. (2001). Outcomes-based assessment in practice: Some examples and emerging insights.
Language Testing, 18(4), 393–407.
Clarke, S. (1998). Targeting assessment in the primary classroom. Strategies for planning assess-
ment, pupil feedback and target setting. London: Hodder & Stoughton.
Clarke, S., & Gipps, C. (2000). The role of teachers in teacher assessment in England 1996–1998.
Evaluation and Research in Education, 4, 38–52.
Coughlan, P., & Duff, P. (1994). Same task, different activities: Analysis of SLA from an activity
theory perspective. In J. Lantolf & G. Appel (Eds.), Vygotskian approaches to second language
research (pp. 173–194). Norwood, NJ: Ablex.
Cumming, J. J., & Maxwell, G. S. (2004). Assessment in Australian Schools: Current practices and
trends. Assessment in Education, 11, 89–108.
Davison, C. (2004). The contradictory culture of teacher-based assessment: ESL teacher assess-
ment practices in Australian and Hong Kong secondary schools. Language Testing, 21(3),
305–334.
Davison, C., & Leung, C. (2009). Current issues in English language teacher-based assessment.
TESOL Quarterly, 43(3), 393–415.
Department of Education, Victoria, Australia. 2006. Languages other than English in government
schools, 2006. Retrieved August, 2009 from www.education.vic.gov.au/studentlearning/teach-
ingresources/lote.html
Duff, P. A., & Uchida, Y. (1997). The Negotiation of teachers’ Sociocultural Identities and Practices
in Postsecondary EFL Classrooms. TESOL Quarterly, 31(3), 451–486.
Gipps, C. (1994). Quality in teacher assessment. In W. Harlen (Ed.), Enhancing quality in assess-
ment (pp. 71–86). London: Paul Chapman.
Harlen, W. (1994). Issues and approaches to quality assurance and quality control in assessment.
In W. Harlen (Ed.), Enhancing quality in assessment (pp. 11–25). London: Paul Chapman.
Heurta-Macias, A. (1995). Alternative assessment: Responses to commonly asked questions. TESOL
Journal, 5, 8–11.
Hill, K. (2010). Classroom-based assessment and the issue of continuity between primary and
secondary school languages programs. Babel, 45(1), 5–12.
Hornberger, N. H. (1994). In A. Cumming (Ed.), Alternatives in TESOL research: Descriptive,
interpretive and ideological orientations. TESOL Quarterly, 28(4), 673–703.
James, M. (2006). Assessment, teaching and theories of learning. In J. Gardner (Ed.), Assessment
and learning (pp. 47–60). London: Sage Publications.
Leung, C. (2005). Classroom teacher assessment of second language development. In E. Hinkel
(Ed.), Handbook of research in second language teaching and learning (pp. 869–888). Mahwah,
NJ: Lawrence Erlbaum.
Leung, C. (2007). Dynamic assessment: Assessment for and as teaching? Language Assessment
Quarterly, 4(3), 257–278.
Leung, C., & Mohan, B. (2004). Teacher formative assessment and talk in classroom contexts:
Assessment as discourse and assessment of discourse. Language Testing, 21(3), 335–359.
Leung, C., & Teasdale, A. (1997). What do teachers mean by speaking and listening? A contextu-
alised study of assessment in multilingual classrooms in the English National Curriculum. In
A. Huhta, V. Kohonen, L. Kurki-Suonio, & S. Luoma (Eds.), Current developments and alter-
natives in language assessment: Proceedings of LTRC 96 (pp. 291–326). Jyväskylä: University
of Jyväskylä.
Lynch, B. K. (2003). Language assessment and programme evaluation. Edinburgh: Edinburgh
University Press.
Mavrommatis, Y. (1997). Understanding assessment in the classroom: Phases of the assessment
process – The assessment episode. Assessment in Education, 4, 381–400.
McNamara, T. (2001). Language assessment as social practice: Challenges for research. Language
Testing, 18(4), 333–350.
Mehan, H. (1979). Learning lessons: Social organization in the classroom. Cambridge, MA:
Harvard University Press.
Moni, K. B. (1999). Constructions of literacy assessment in two year 8 English classrooms.
Unpublished PhD thesis, Education, University of Queensland.
Morgan, D. L. (2002). Focus group interviewing. In J. F. Gubrium & J. A. Holstein (Eds.), Hand-
book of interview research: Context and method (pp. 141–159). London: Sage Publications.
Nassaji, H., & Swain, M. (2000). A Vygotskian perspective on corrective feedback: The effect of
random versus negotiated help on the learning of English articles. Language Awareness, 9,
34–51.
Newman, D., Griffin, P., & Cole, M. (1989). The construction zone: Working for cognitive change
in schools. Cambridge: Cambridge University Press.
Nicholls, J. G. (1989). The competitive ethos and democratic education. Cambridge, MA: Harvard
University Press.
Rea-Dickins, P. (2001). Mirror, mirror on the wall: Identifying processes of classroom assessment.
Language Testing, 18(4), 429–462.
Rea-Dickins, P. (2006). Currents and eddies in the discourse of assessment: A learning focused
interpretation. International Journal of Applied Linguistics, 16(2), 164–188.
Ritchie, J., & Spencer, L. (2002). Qualitative data analysis for applied policy research. In
A. M. Huberman & M. B. Miles (Eds.), The qualitative researcher’s companion (pp. 305–330).
Thousand Oaks, CA: Sage Publications.
Sadler, D. R. (1989). Formative assessment and the design of instructional systems. Instructional
Science, 18, 119–144.
Strauss, A. L. (1987). Qualitative analysis for social scientists. Cambridge: Cambridge University
Press.
Strauss, A., & Corbin, J. (1998). Basics of qualitative research: Techniques and procedures for
developing grounded theory (2nd ed.). Newbury Park, CA: Sage Publications.
Thomas, S., & Oldfather, P. (1997). Intrinsic motivations, literacy and assessment practices: ‘That’s
my grade. That’s me.’ Educational Psychologist, 29, 107–123.
Torrance, H., & Pryor, J. (1998). Investigating formative assessment: Teaching, learning and
assessment in the classroom. Buckingham: Open University Press.
Tunstall, P., & Gipps, C. (1996). Teacher feedback to young children in formative assessment: A
typology. British Educational Research Journal, 22(4), 389–404.
Van Lier, L. (2004). The ecology and semiotics of language learning. A sociocultural perspective.
Boston: Kluwer Academic.
Victorian Curriculum & Assessment Authority (2008). Framework of essential learnings: Languages
other than English. Melbourne: Victorian Curriculum and Assessment Authority.
Wiliam, D. (2001). An overview of the relationship between assessment and the curriculum. In
D. Scott (Ed.), Curriculum and assessment (pp. 165–181). Westport, CT: Ablex.
Yin, M. (2010). Understanding classroom language assessment through teacher thinking research.
Language Assessment Quarterly, 7(2): 175–194.

Hill McNamara (2011) - Framework For Classroom-Based Assessment

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Hill McNamara (2011) - Framework For Classroom-Based Assessment

Uploaded by

Copyright:

Available Formats

428317 LTJ

Kathryn Hill and Tim McNamara

Table 1. Dimensions of classroom-based assessment

Evidence Data What is assessed? • Valued enterprises

any actions, interactions or artifacts (planned or unplanned, deliberate or unconscious, explicit

Table 2. Scope and dimensions of CBA

1. What do language teachers do when they carry out classroom-based assessment?

Design of the study

Participants and programs

Data collection and data analysis

Primary Year 6 (n=26) Term 4, 2005 (10 weeks)

Secondary Year 7 (n=25) Terms 1 & 2, 2006 (10 weeks)

Figure 1. Design for data collection

Table 3. Potential sources of assessment-related information

Table 4. Data audit – ‘Introducing Yourself’ (Year 7)

Data Pair 1 Pair 2 Pair 3 Pair 4

RQ1 What do teachers do?

Learning Activities Intercultural Understanding Key elements of

Jigsaw reading – students ICU: History: Indonesia’s path to • organizes and

Figure 2. Extract from Planned Learning Activities (Year 6 ‘Government’)

Figure 3. Extract from Planned Assessment Tasks (Year 6 ‘Government’)

Framing assessment for students

Work requirement Assessment

1. Introducing Yourself Oral

2. Numbers Assignment Written

Figure 4. Extract from Work Requirements (Year 7, Semester 1)

Table 5. Terminology used to distinguish types of assessment

Term Description Example

A ‘planned assessment opportunity’ occurs when an assessment task is embedded in

T: Hands up if you got them all right

T: Hands up if you only got one or two or three wrong

T: What do you think ‘kapan’ [when] means? ‘Kapan’ [when]

S1: Oh. ((translating)) ‘THE DUTCH DEFEATED THE PORTUGUESE.’

Using assessment data

Learning (feedback). Assessment-related information was also used to promote immediate

In contrast, the focus of ‘task-referenced’ feedback is the learner’s performance in relation

T:  What’s a guru? [teacher]

Table 6. Summary of feedback types

Feedback type Description Related terminology

RQ2 What do teachers look for?

Activity 2: Interview an alien (15 points)

Your goal is a total of 60 points.

Figure 5. Extract from instructions – ‘Space’ Unit (Year 6, Term 3)

T1: And Jessica, she’s still going great guns.

Topics Weeks Resources Work requirements

Introducing Yourself 2 Flashcards, Puzzles, Games Role-play

Numbers, Colours 2 Coursebook, Computers Numbers assignment

Figure 6. Extract from Year 7 syllabus document

RQ3 What theory or standards do teachers use?

Views of the subject or content area (Indonesian)

Beliefs about language and language learning

R: Karim was good yesterday

Beliefs about assessment

T1: Do you want to finish that off?

RQ4 Learner understandings

Understandings of language learning

R: Is she? She does well in all of her subjects does she?

A number of researchers have emphasized the importance of transparency with regard

A subsequent discussion with a student focus group revealed a close correspondence

Table 7. A framework for research on CBA processes

1. What do language teachers do?

2. What do teachers look for?

3. What theory or ‘standards’ do they use?

T: What’s a guru? [teacher]

T1: Do you want to finish that off?

2. What do teachers look for?

3. What theory or ‘standards’ do they use?

4. Do learners share the same understandings?

2. What do teachers look for?

3. What theory or ’standards’ do they use?