Testing reading

comprehension
 The reading process

Reading is a dynamic process in which the

reader interacts with the text to construct
meaning.  Inherent in constructing meaning
is the reader's ability to activate prior
knowledge, use reading strategies and adapt
to the reading situation.
 What can a skilled reader do?

A skilled reader rapidly and accurately decodes the

words, attaches the meaning to words and
sentences, connects text information to relevant
background knowledge, maintains a mental
representation of what he or she has already
read, forms hypotheses about upcoming
information and makes decisions based on his
or her purpose for reading – all at the same
time. Carlisle and Rice, 2002
 What influences reading
difficulty?
• Reader variables
The reader’s background and subject/topic knowledge,
their cultural knowledge and their knowledge of the
language in which the target texts are written. The
reader’s ability to process printed information is clearly
crucial.
• Text variables
Linguistic features of text clearly affect readability of text
and reader’s comprehension, and text type, organization
and so on as well as text topic influence how well
readers can process meaning. Many features of text
need to be considered in the design of tests of reading.
 What should assessment of
reading focus on?
• Alderson argues that we should focus our assessment of
reading on the target skills we want out students to
develop.
Major reading subskills include
• Reading quickly to skim for gist, scan for specific details,
and establish overall organisation of the text
• Reading carefully for main ideas, supporting details,
author’s argument and purpose, relationship of
paragraphs, and fact vs opinion
• Drawing inferences from both stated and implied content
 Enabling reading skills include

• Understanding at the sentence level (lexis,

syntax, cohesive markers)
• Understanding at inter-sentence level (e.g.
anaphoric/catophoric reference,
recognizing discourse markers)
• Understanding components of non-linear
texts (meaning of graph, chart labels etc)
 Choosing texts

• Many sources for reading texts: purpose written,

taken directly from authentic material or adapted.
• Use a variety of text types; do not select texts of a
particular kind just because they are readily
available
• Choose texts of appropriate length; detailed reading
can be assessed with texts consisting of a few
sentences. Texts of up to 2000 words may be used
for extensive reading at higher levels
•
 Choosing texts
• Choose texts with many discrete pieces of information
• Choose texts that will interest the candidates but will not disturb or
excite them. Avoid texts with controversial or biased material (e.g.
abortion, international disputes, religion). These are not suitable for
assessment because they can upset candidates and affect the
reliability of results.
• You should check the language of your reading texts. Ninety
percent (90%) of the words in a text should be known to students for
good comprehension (Nation 1990).
• Choose topics within the experience of the test takers. However,
avoid texts that contain information which is part of the test takers’
general background knowledge.
Developing test items/questions
• The wording of the reading test questions should
not cause comprehension difficulties to learners.
It should always be within their capabilities and
less demanding than the text itself.
• Responses to test items should make minimal
demands on writing ability.
• The items should be in the same order as the
information in the text. Mixing up the order of
questions increases the difficulty level.
 Assessing reading
comprehension: Three levels
• The first level, literal comprehension, is
the most obvious. Comprehension at this
level involves surface meanings. At this
level, teachers can ask students to find
information and ideas that are explicitly
stated in the text.
 Levels of comprehension

• The second level is interpretive or referential

comprehension. At this level, students go beyond what
is said and read for deeper meanings. They must be
able to read critically and analyse carefully what they
have read. Students need to be able to see relationships
among ideas, for example, how ideas go together and
also see the implied meanings of these ideas. It is also
obvious that before our students can do this, they have
to first understand the ideas that are stated (literal
comprehension).
 Referential Level of
comprehension
• At this level, teachers can ask more challenging
questions such as asking students to do the following:
– Re-arrange the ideas or topics discussed in the text.
– Explain the author's purpose of writing the text.
– Summarize the main idea when this is not explicitly
stated in the text.
– Select conclusions which can be deduced from the
text they have read.
 Levels of comprehension

• Finally, the third level of comprehension is critical

reading whereby ideas and information are evaluated.
Critical evaluation occurs only after our students have
understood the ideas and information that the writer has
presented. At this level, students can be tested on the
following skills:
– The ability to differentiate between facts and opinions.
– The ability to recognize persuasive statements .
– The ability to judge the accuracy of the information
given in the text.
 Some tips on item writing

• Do not write items for which the correct response can be

found without understanding the text.
• Do not write items that some candidates are likely to
answer from general knowledge without reading the text.
• Make the items independent of each other; do not make
the correct response on one item depend on another
item being responded to correctly.
 Possible testing formats for reading tests:
Multiple choice

• Multiple-choice questions are a common

device for testing students’ text
comprehension. They allow testers to
control the range of possible answers to
comprehension questions, and to some
extent to control students’ thought
processes when responding.
 An example
1.1 Read the first part of the text on p. 2 (the left side column), and choose the best
answers (A, B, or C) for items 1-4.
1. This text
A.
is written in an academic register.
B.
has an impersonal tone.
C.
is written as a narrative.

2. One aim of this text is to

A.
argue for better housing for poor people.
B.
describe the writer’s family life.
C.
explain what is needed to become a writer.
8. What does the writer say about North Africa?

A Publishing there is of a high standard.

B Writers have achieved more freedom there.

C Publishing still has a long way to go there.

 Matching
• One alternative objective technique for the
testing of reading is multiple matching.
Here two sets of stimuli have to be
matched against each other as, for
example, matching headings for
paragraphs to their corresponding
paragraph, titles of books against extracts
from each book, and so on
 Example
Read the following extracts (21-25) and decide in which publication they might appear. Use
each of the options below (A-H) only once. There is one option you do not need.
A. Rules for playing a game B. Written safety instructions
C. Instructions for paying tax D. Reverse side of a book’s title page
E. University brochure F. Travel guide

21. Exception for children under age 18. If you are planning to claim a return for your child, who was
under 18 at the end of 2009, and certain other conditions apply, you can include your child’s income
on your form.
22. All rights reserved. No part of this publication may be reprinted or reproduced or utilised in any
form or by any electronic, mechanical, or other means, now known or hereafter invented, including
photocopying and recording, or any other information storage or retrieval system, without permis­
sion in writing from the publishers.

23. The inn is a little oasis amidst the narrow and bustling cobbled streets at the heart of the
historic city.
 Ordering tasks (Strip stories)
• Candidates are given a scrambled set of
words, sentences, paragraphs or texts and
have to put them into their correct order.
The ordering tasks can offer the possibility
of testing the ability to detect cohesion,
overall text organization.
 Short-answer tests

• Test-takers are simply asked a question which

requires a brief response, in a few words, not
just Yes/No or True/False. The justification for
this format is that it is possible to interpret
students’ responses to see if they have really
understood, whereas on multiple-choice items
students give no justification for the answer they
have selected and may have chosen one by
eliminating others.
 The free-recall test

• In free-recall tests (sometimes called

immediate-recall tests), students are asked
to read a text, to put it to one side, and then
to write down everything they can
remember from the text. The free-recall test
is often held to provide a purer measure of
comprehension, since test questions do not
intervene between the reader and the text.
 The summary test

• Students read a text and then are required to summarize

the main ideas, either of the whole text or of a part, or
those ideas in the text that deal with a given topic. It is
believed that students need to understand the main
ideas of the texts, to separate relevant from irrelevant
ideas, to organize their thoughts about the text and so
on, in order to be able to do the task satisfactorily.
(Example: Gapped Summary)
• Scoring is problematic and can be subjective
 Information-transfer test

• The students’ task is to identify in the

target text the required information and
then to transfer it, often in some
transposed form, on to a table, map, figure
etc.
 Cloze test

• Cloze Tests are reading passages with the

blanks representing words that are deleted
from the original text; the blanks are to be
filled in by the reader
 Types of cloze test
a. Fixed Ratio Cloze or Nth word deletion
• Words are deleted systematically by counting off,
regardless of the part of speech.
b. Rational Deletion Cloze
• Words are deleted by part of speech or content area
vocabulary.
c. Maze Technique
• Three word choices are provided at each missing word
interval.
d. Limited Cloze
• Word choices (one per blank) are provided all together in
a word bank at the top or bottom of the page.
 Tips for developing reading
comprehension tests
• Make sure your assessment matches your reading program. Test
the skills you have taught
• Sample a range of reading subskills with different task types. Allow
4-10 items per task type
• Choose a range of text types appropriate to your program.
Consider students’ background knowledge and interests in selecting
texts. Familiarity with the topic aids comprehension
• Use authentic or adapted texts whenever possible
• Exploit the entire text. Questions should cover all sections of a text
• Assess inferencing and critical thinking. Include questions that
require students to think beyond what they see in print