EDF5643 Second

Language
Assessment &
Testing
Assessing Intended
Consequences and
Decisions

Dr Gary Bonar
gary.bonar@monash.edu
Monash University
Key Objectives

Section Understanding the

EvaluatingAintended Section B
01 consequences and 02 decisions teachers
need to make in the
outcomes in CBLA.
assessment process.

Providing C
Section evidence
03 to support the
claims of the AUA
framework.
2
The Assessment Use Argument (AUA)
● AUA as a conceptual
framework for assessment
justification.
● Four critical claims of an AUA
1) Intended consequences
2) Intended decisions
3) Intended interpretations
4) Intended records
● This week we focuses on
claims 1 & 2 3
 Why Begin with Intended Consequences?
Where to begin the assessment
process?
● Often teachers will start with the
details of the assessment task
● Using an AUA framework means
you begin with the question “Why
do I need to assess my students?”
● This helps define the purpose and
leads to a better assessment
outcome for everyone
4
Claim 1 : INTENDED CONSEQUENCES

Intended consequences are the effects on

stakeholders that the test developer/user wants
to help bring about.
In stating Claim 1, there are three questions
you need to answer:
1. What beneficial consequences do I want to
help bring about?
2. Who will be affected by these consequences?
3. What is the intended quality of these
consequences?
5
Washback as a consequence
Washback is the effect or impact of using an
assessment on instruction and learning.
● Positive washback - an assessment leads
to teaching and learning that are in
accordance with their beliefs about
teaching and learning & what is
important for students to learn
● Negative washback - assessment leads
to teaching and learning activities that
do not match beliefs about teaching and
learning and about what is important for
students to learn
6
Enhancing positive washback
Teachers should ensure that the assessments
they create and use will provide positive
washback on learning.
● the content of the assessments should
adequately covers the content of the
course – the syllabus, textbook, and
teaching materials
● the kinds of assessment tasks you use
should correspond to the kinds of
teaching tasks you use in the classroom

7
Quality of the intended consequences

Beneficence is the degree to which the

consequences of using an assessment and of
the decisions that are made promote good and
are not detrimental to stakeholders. Questions
to ask include:
● In what specific ways will my students
benefit from the use of the assessment?
● In what specific ways will I benefit from
using the assessment?
● In what specific ways will others benefit
from using the assessment?
● How might using an assessment be
detrimental to my students and others? 8
Stating Claim 1: An example
A teacher in an ESL/EFL classroom in a primary/elementary school needs to
develop a classroom assessment for his beginning level students. This
assessment will be used for two different purposes. One purpose is to make
sure that the students who pass the grade are prepared for instruction at the
next level. In order to do this, the teacher will use the results of this
assessment, along with other assessments given during the course of the
school year, to make summative decisions about which students will pass to
the next grade. The second purpose is to help his students improve their
speaking and to improve his teaching. In order to do this, the teacher will also
use the results of this assessment to make formative decisions about his
teaching and to provide feedback to his students. The assessment will be
based on a unit of instruction in the course, and this is one task from the
assessment, in which students describe animals from pictures by speaking.
9
Claim 1: Intended Consequences & Stakeholders

10
Claim 1: Other Consequences & Stakeholders

11
 Providing backing for Claim 1
Students receiving instruction at the appropriate level:
● teacher talks with ESL/EFL teachers in the next course/grade about how well students who moved on
to the next course/grade are performing in speaking
● teacher talks with ESL/EFL teachers in the same course/grade about how well students who did not
move to the next grade are performing in speaking.
Students’ improvement in speaking:
● teacher observes students’ speaking performance in class
● teacher talks with students about their use of feedback from the assessment to improve their
speaking
● teacher asks the students to answer a questionnaire on how they have improved their speaking
● teacher compares students’ performance on the next speaking assessment with their performance on
this one.
Teacher’s improvement in teaching:
● teacher makes notes on how he changed his instruction for speaking activities.
● teacher gets feedback from students through a survey or questionnaire
● teacher asks one or two fellow ESL/EFL teachers to observe his classroom teaching.
12
CLAIM 2: DECISIONS ABOUT WHEN WE NEED
TO ASSESS OUR STUDENTS

● What decisions need to be made?

● Who will be affected by these
decisions?
● Who will make these decisions?
● When do these decisions need to be
made?
● What are the qualities of the
intended decisions?

13
Claim 2 : INTENDED DECISIONS

Intended decisions are specific actions that the test

developer or test user takes in order to help promote the
beneficial consequences that are specified in Claim 1
Formative & Summative decisions in terms of their
purpose, or the kinds of beneficial consequences they
are intended to help bring about.
Relative importance—low-stakes, medium-stakes, and
high-stakes—of the decisions that we need to make and
that others may make on the basis of our assessments
Most of the formative decisions from classroom-based
assessments will be low-stakes, while the summative
decisions you make will be medium- to high-stakes

14
Claim 2 : INTENDED DECISIONS - additional
questions

1. Who will be affected by

the intended decisions?
2. Who will make the
intended decisions?
3. When do the intended
decisions need to be
made?

15
The qualities of the intended decisions

Values-sensitivity is the degree to which the use of an

assessment and the decisions that are made take into
consideration existing educational and societal values
and relevant laws, rules, and regulations.
Equitability is the degree to which different test takers
who are at equivalent levels on the ability to be
assessed have equivalent chances of being classified
into the same group.
16
Relating intended consequences to decisions in classroom-based assessments

17
Stating Claim 2: An example
A teacher in an ESL/EFL classroom in a
primary/elementary school needs to develop a
classroom assessment for his beginning level
students. This assessment will be used for two
different purposes. One purpose is to make sure that
the students who pass the grade are prepared for
instruction at the next level. In order to do this, the
teacher will use the results of this assessment, along
with other assessments given during the course of
the school year, to make summative decisions about
which students will pass to the next grade. The
second purpose is to help his students improve their
speaking and to improve his teaching. In order to do
this, the teacher will also use the results of this
assessment to make formative decisions about his
teaching and to provide feedback to his students.
The assessment will be based on a unit of instruction
in the course, and this is one task from the
assessment, in which students describe animals from
pictures by speaking.
The high-stakes summative
decisions are made by the teacher at
the end of the school year. The
low-stakes formative decisions are
made by the teacher and the
students before the next unit of
instruction. These decisions affect
the stakeholders. The decisions take
into consideration the educational
values of the school and the societal
values of the community, and follow
the rules and regulations of the
school. They are equitable for the
stakeholders.
18
Intended decisions to be made for the example of Claim 2

19
Providing backing for Claim 2
Values-sensitivity: High-stakes summative decisions:
● the teacher meets with students, fellow teachers, school administrators, and parents/guardians to
discuss the relevant values that need to be considered in the decisions to be made
● the teacher reviews school rules and regulations regarding the use of assessments for decisions about
which students will pass to the next grade.
Values-sensitivity: Low-stakes formative decisions:
● the teacher considers how consistent the decisions to be made are with the their own values and beliefs
about effective instructional practice.
Equitability - High-stakes summative decisions:
● the teacher documents procedures for 1) setting standards and cut scores based on all of the
assessments given during the course of the school year; 2) monitoring how these are implemented in
practice; and 3) informing students and other stakeholders about these.
Values-sensitivity: Low-stakes formative decisions:
● equitability is not a concern in low-stakes formative decisions because these are not made for the
purpose of classifying students into groups.
20
Tutorial Focus

In the week 3 and 4

tutorials you will explores
the four claims in detail
with examples from the
textbook and from your
own experiences. This
will help you with your
AT1 and AT2.

21