ASSESSMENT AND TESTING

Javier Valdivia

CONTENTS
ASSESSMENT AND TESTING-------------------------------------------------------------------------1
INTRODUCTION-----------------------------------------------------------------------------------------------3
1 TARGET STUDENTS AND SELECTED SKILLS-----------------------------------------------------------3
2- OBJECTIVES OF THE TEST----------------------------------------------------------------------------- -3
3- TEST TYPOLOGY------------------------------------------------------------------------------------------4
3.1- GENERATION OF TESTING-------------------------------------------------------------------------4
3.2- PRINCIPLES OF LANGUAGE TESTING----------------------------------------------------------- -6
4- CRITERIA AND SCALES FOR GRADING--------------------------------------------------------------- -7
5- CONCLUDING REMARKS---------------------------------------------------------------------------------8
BIBLIOGRAPHY-----------------------------------------------------------------------------------------------9
APPENDIX----------------------------------------------------------------------------------------------------10
INTRODUCTION

Testing and assessment are only two of the myriad of aspects that make up that complex process
we call “teaching”, and two which are usually taken for granted, despite the enormous
signification attached to their results in the development of instruction. It is not rare the case of
teachers and institutions making use of tests without conscientiously analyzing the implications
they carry or the models of the learning of language they presuppose. The literature on teaching
foreign languages offers a good deal of materials devoted to the presentation and examination of
diverse approaches and methods for measuring the ability of students with respect to different
aspects of the language. The aim of this work is to propose an instrument for the assessment of
reading comprehension in a hypothetical group of students. Section 1 will introduce the
characteristics of the target group of students and the target language skill. Section 2 will
present and discuss the objectives of the proposed test. The typology of the test will be
discussed in section 3 and the grading scales and criteria will be offered in section 4. Finally, the
concluding remarks will be presented in 5.

1 TARGET STUDENTS AND SELECTED SKILLS

The proposed test has been devised so as to provide assessment on the progress of a group of
students with an intermediate level of English, as regards the reading skill. The hypothetical
group of students may consist of adolescents or adults. The age requirement is based on the
content of the text, in which topics such as “death”, “corpses” and “war” are treated. The
students are expected to have been dealing with the passive voice in class, with or without the
subject of the verb. Not only are the students expected to be able to cope with the grammatical
traits of passive voice, but also to understand the discursive reasons why those data appear in
the passive.

2- OBJECTIVES OF THE TEST

The test aims at checking the students’ progress concerning the following concepts:

- Passive voice: interpretation of sentences that contain passive structures.

- Comprehension of information transmitted in passivized structures.

- Understanding of reasons why agency may be backgrounded in some passivized sentences,
instead of being foregrounded as it typically occurs in active sentences.

- Identification of global meaning of a reading piece and of its sections.

- Extraction of specific data from the text.

The proposed test will then aim at eliciting the required information so as to assess the students’
abilities both for reading comprehension, in general, and for the correct interpretation of passive
constructions. Since the objective in the design of the test was to present language in an
integrative and contextualized manner, no explicit activity is proposed in which the students
need to overtly signal or analyze passive structures, but rather the analysis will be required from
them implicitly in the course of understanding and extracting the information transmitted by the
text.

3- TEST TYPOLOGY

In this section we will discuss the characteristics of the proposed test concerning the generation
of assessment devices it belongs to and the features of testing that can be identified in the test, in
the light of West’s (1990) “principles of language testing”.

3.1- GENERATION OF TESTING

As it’s claimed by Lennon on page 18 in FUNIBER’s material for the subject “Assessment and
Testing”, tests do not necessarily fall as a whole into any of the categories that make up the
three generations of testing. In accordance with that postulate, the test presented in this piece of
work can be said to display traits that can be considered as belonging to both 2 nd and 3rd
generation. This claim is rooted in the analysis of some of its characteristics: for instance, even
though the activities are devised so as to elicit answers that can be marked objectively, and so
showing a typical characteristic of 2 nd generation tests, the inclusion of a reading piece that
resembles an authentic text (Melvin & Stout, 1987; and Nunan, 1991 & 1999) so as to
contextualize the entire set of activities makes the test more similar to a 3 rd generation format.
We could summarize the traits of the two generations of tests mentioned above that can be
identified in the proposed test:

2nd generation aspects of the proposed test:

- The use of objectively graded activities. All of the multiple-choice questions and the
matching activity expect only one correct answer, thus they can be objectively graded.
- The activities, though requiring global comprehension of the text, are not exactly authentic,
in the sense that no one goes around, in everyday life, matching main ideas of the things she
has read with the number of paragraphs in which they appear.

3rd generation aspects of the proposed test:

- The use of a reading piece that provides a context for the comprehension activities favors an
integrative approach, since, as it is claimed by Lennon in the FUNIBER materials (p.18)
“techniques for listening and reading demand global (integrative) comprehension as well as
comprehension of discrete items.”

- The content of the text and the way it is redacted resemble an authentic newspaper article,
and its comprehension on the part of the students will result in what Ellis (2003) claims
should be a goal of Communicative Language Teaching (Canale & Swain, 1980, among
others), namely “language use that bears resemblance, direct or indirect, to the way language
is used in the real world”

- The test can also be analyzed in the light of West’s “Communicative Checklist” so as to
evaluate how communicative it is (West, 1990). The following questions out of West’s 12-
point checklist can be answered affirmatively and thus show the degree of
“communicativeness” of the proposed test:

1. Does the item assess meaning/content as well as form? Yes, because despite the focus on
the grammatical content “passive voice”, the students are required to understand
information and so to extract meaning from the text, rather than focusing overtly on its
grammatical traits.

2. Does the item test integrative rather than discrete-point language? Yes, the tested
objectives are presented in a contextualized manner, rather than being isolated.

3. Are the input texts authentic, or authentic-like? Yes. In this case, the text is authentic-like.

5. Does the task replicate a real-world task? Yes, if we focus on the “reading for meaning”
part of the test, we may say that it does resemble a real-life activity.

7. Do tasks involve information/opinion/reasoning gaps? Yes, the students are expected to

extract information from the text to complete the activities. Also, the answers to some of
the activities can be said to involve reasoning gaps, because they require students to make
connections and infer information not overtly presented in the text. They belong to a type
of activity that, as Prabhu (1987) says, “(…) necessarily involves comprehending and
conveying information, as in information-gap activity, but the information to be conveyed
is not identical with that initially comprehended. There is a piece of reasoning which
connects the two.”
 12. Is the situation plausible or necessary in the real world? Yes. Even though the types of
responses required from the students are not authentic in themselves (matching elements
and ticking correct answers), the situation of reading a text similar to the one that appears
in the test is likely to occur in their real lives.

As we can see, the proposed test can be considered a hybrid between second and third
generation testing instruments, yet it’s nevertheless possible to claim that the characteristics of
3rd generation seem to prevail.

3.2- PRINCIPLES OF LANGUAGE TESTING

In this section, we will discuss the characteristics of the proposed test following the “principles
of language testing” proposed by West (1990):

- Competence vs. Performance: This test can be said to assess the students’ performance in
using the language, particularly in the comprehension of its written form, since the students’
understanding of the test may be affected by factors such as lack of concentration,
incomplete previous knowledge about the topics at hand, nervousness, etc., and so the way
they handle the test will be affected. On the other hand, it’s widely acknowledged that
competence, as the idealization it is, is not distressed by any of those factors.

- Usage vs. Use: As regards the distinction first made by Widdowson (1978) between the real
use of language as “meaningful communicative behavior” (ibidem: 3) and its pedagogical
utilization to display specific linguistic items or structures, what he terms “usage”, we can
say that the contextualization of the structures targeted for testing and their communicative
function allow us to classify them as instances of “use”.

- Direct vs. Indirect assessment: It can be said that the fact that the activities in the test require
a response that reflects the students’ ability to extract and process information of the text
makes it possible for those activities to be considered as direct, since it is not the students’
competence that is assessed, but rather their performance of the communicative task of
decoding the information transmitted by the text. In that way, the language is more of a
medium for obtaining the required information, than a goal in itself.

- Discrete point vs. Integrative assessment: The proposed test clearly assesses language in an
integrative manner, since the objective structures are not isolated, but embedded in a
significant context.

- Objective vs. Subjective assessment: The type of activities devised for the proposed test can
be marked objectively, since all of them have been designed to require only one correct
answer.
- Backward-looking vs. Forward-looking assessment: It is difficult to determine whether the
results of this test will only provide assessment of previously introduced topics or if they will
help the teacher to look ahead and be able to select what lines to follow in teaching. Rather it
might be said that the test is both backward- and forward-looking, since it may provide the
teacher not only with feedback on the students’ mastery of previously taught items, but also
it could serve the function of supplying data on future teaching to be undertaken.

- Contextualized vs. Disembodied language: As has been said before, the proposed test
presents language in a contextualized manner, rather than in isolation.

- Criterion-referenced vs. Norm-referenced assessment: As will be discussed in section 4, the

grading of the test aims at providing a criterion-based assessment of the students, taking into
consideration their degree of success in understanding the content of the text.

- Reliability vs. Validity: Given the type of activities it contains, the proposed test is expected
to have an adequate level of reliability, for its answers can be consistently graded both by
different teachers and by the same teacher in different occasions, showing thus both inter-
and intra-rater reliability (Lennon, FUNIBER materials, p.16). As regards the validity of the
test, we can say that it may be considered to have a relatively high content validity, for the
language used in the test can be deemed as appropriate for the communicative situation it
aims at representing (i.e., extracting information out of a reading piece), adequate construct
validity, since the test reasonably reflects the communicative situation presupposed by a
communicative approach to the teaching of language, as far as the comprehension of the
information goes, while we lack information so as to judge its predictive and concurrent
validity.

4- CRITERIA AND SCALES FOR GRADING

The following marking scheme is proposed for the planned test:

Activity 1 (total points: 8)

Expected answer: c

Mark: award 8 points when option “c” is ticked. Other options ticked: award 0 points.

Activity 2 (total points: 28)

Expected answers:
- Where the victims of the war are buried Paragraph 1
 - Information about the cemetery provided by a neighbor Paragraph 2
- Approximate number of victims Paragraph 3
- Lack of information about some of the bodies in the cemetery Paragraph 4
- The need for new graves Paragraph 5
- Fallujah before the invasion Paragraph 6
- The unfairness of the invasion Paragraph 7
Mark: award 4 points to each correct match. Other options matched: award 0 points.

Activity 3 (total points: 24)

Expected answers:

1. In which place are the dead bodies put? b. In the former soccer stadium
2. Who killed the people who are buried there? c. The American attacks.
c. There’s no information in the text on
3. Who put the dead bodies in holes?
this topic.
4. What is the exact number of people killed in
a. The exact number is unknown.
Fallujah since April 2004?
5. Why was it necessary to dig more graves? c. Because more people have been killed.
6. When did people start to hear the sounds of
a. When the invasion began.
explosions?
Mark: award 4 points to each correct option. Other options selected: award 0 points

Total Mark: 60 points

Passing Mark: The students need to obtain at least 35 points in the test to get a pass. However,
since it’s fundamental for the students to achieve not only a global understanding of the content
of the text, but also to be able to extract specific facts, it’s indispensable to obtain at least the
40% of correct answers in both activity 2 and activity 3. That is to say, for instance, that a
student scoring 8/8 points in activity 1, 28/28 points in activity 2, but failing to attain over the
40% of correct answers in activity 3 will get a fail mark. When this situation occurs, remedial
work in either global or specific comprehension will be administered to the students in question.

5- CONCLUDING REMARKS

Testing has long been appreciated as a core aspect of teaching, rather than as a mere tool for
telling apart “passes” from “fails”. Assessment, not only of students’ abilities, but of many other
aspects such as teaching method, syllabus design, teacher’s performance, among several other
factors, is nowadays crucial for the development of teaching as a whole.
In this piece of work a particular example of a possible testing instrument has been presented
and analyzed. The thorough and careful design of tests is one of the abilities that teachers cannot
overlook, for its intricacies and the influence they have on the process of teaching and learning
languages are far from trivial.

BIBLIOGRAPHY

Canale, M. and M. Swain (1980). “Theoretical bases of communicative approaches to second

language teaching and testing”. Applied Linguistics 1, 1-47.

Ellis, R. (2003). Task-based Language Teaching and Learning. Oxford: OUP

Leaver, B. and J. Willis (eds) (2004). Task-based Instruction in Foreign Language Education:
Practices and Programs. Washington, DC: Georgetown University Press

Lennon, A. Material for the subject “Assessment and Testing in the Classroom”. Máster en
Lingüística Aplicada a la Enseñanza del Inglés como Lengua Extranjera, Universidad de Jaén,
FUNIBER.

Melvin, B.S. and Stout, D.S. (1987). “Motivating language learners through authentic
materials”. In W. Rivers (ed.) Interactive Language Teaching. New York: Cambridge
University Press, 44-56

Nunan, D. (1991). “Communicative tasks and the language curriculum”. TESOL Quarterly, 25
(2), 279-295.

Nunan, D. (1999). Second Language Teaching and Learning. Boston: Heinle and Heinle
Publishers.

Prabhu N. S. (1987). Second Language Pedagogy. Oxford: Oxford University Press

West, R. (1990). Introduction and Principles of Language Testing. University of Manchester

SEDE.

Widdowson, H. G. (1978). Teaching Language as Communication. Oxford: OUP.

APPE
Universidad de Jaén

i
Adapted from information extracted from the “Testimonies of Crimes Against Humanity in Fallujah: Towards a Fair
International Criminal Trial”. Proceedings of the 15th session of the UN Human Rights Council, Geneva 13
September - 1 October 2010.

10