www.soharuni.edu.

om/tlc5/

Guidelines for Developing a Reading Test in EFL Context

Junifer Abatayo, Manal Al Abri

Jabatayo@su.edu.om

MAAbri@su.edu.om

Faculty of Language Studies, Sohar University

Sultanate of Oman

Abstract
Reading as a skill and activity involves a number of complexities such as processing information
and performing specific tasks that ensue in some specific contexts. In EFL context, reading
extends to some important issues particularly to the classroom setting, learning process and
sociocultural aspects where EFL learners live and learn the skill of reading. This article aims to
share practical guidelines in developing reading tests in EFL context. The guidelines are
anchored on a number of testing research and experimentation highlighting test design,
practicalities of integration and contextualization. Adding to the guidelines, some strategies for
reading are included to establish the notion that understanding important concepts of the
strategies can also support teachers develop effective reading tests. Furthermore, sample
reading tests and exercises as well are provided to offer new perspectives on effective reading
comprehension routines. Additionally, embedded in the reading tests are set of practices
relating to question types that are hoped to be useful in helping EFL learners increase their
control over reading strategies and hopefully become independent learners in EFL reading
classroom.

Introduction
In foreign language teaching context, reading is considered as a skill that teachers expect
learners to acquire and develop. In fact, it is a skill that provides students access to knowledge,
information, and to the world. Therefore, reading is considered the most important skill in
education (Bedle, 2017; Alexander, Argent, & Spencer, 2008; Chen & Intaraprasert, 2014;
Chirimbu & Barbu-Chirimbu, 2015; Ghahari & Basanjideh, 2017; Meniado, 2016; Nachmani,
2015). As described by Bedle (2017), as a pedagogical implication, students who are studying
in ESL/EFL classroom should be placed in an interactive reading environment, so that they can
construct meaning and eventually become aware of the process, and develop their proficiency
as active participants in the classroom. Interactive reading coupled with extensive practice when
integrated in ESL/EFL programmes can benefit students greatly in developing their language
competence, reading ability and cultural knowledge (Hedge, 2003). To ensure universal
success, teachers should be able to understand concepts on reading skills and strategies. Not

Electronic copy available at: https://ssrn.com/abstract=3543269

only reading strategies can help students develop comprehension, the inclusion of sociocultural
aspects expressing confidence in communicative practices can also safeguard reading
comprehension (Snow, 2002).
The purpose of this article is to provide EFL teachers with practical tips and guidelines in writing
and developing reading tests in EFL context. The sample reading tests provided in this section
were developed and evaluated during test validation sessions, focus group discussions,
workshops in MA TESOL class, and training sessions at the General Foundation Program
(GFP) at Sohar University.

The Importance of Reading

Reading comprehension is one of the most important skills in higher education. According to
Meniado (2016), reading as a skill, particularly in EFL context, requires substantial activities so
that students will succeed in their academic life and beyond. As a receptive skill, it is considered
a crucial skill for EFL learners who are going to study in an academic setting. If teachers can
identify the different skills and strategies that can possibly contribute to the development of the
reading process, there is a possibility that these skills can be tested and use the results to
outline reading proficiency of the students (Urquhart and Weir, 1998; Weir, Yang, & Jin, 2000;
Shiotsu, 2003; Kavlu, 2015).

Skills and Strategies

The figure below outlines the skills and strategies for reading that offer and represent the
spectrum of possibilities in defining objectives in assessing reading comprehension (Brown, D.
& Abeywickrama, 2010).
Figure 1
Microskills, Macroskills, and Strategies for Reading

Microskills

1. Discriminate distinctive graphemes such as letters or letters combinations, and

orthographic patterns of English
2. Retain chunks of language of different lengths in short-term memory
3. Process writing at an efficient rate of speed to fit the purpose
4. Recognize core words and interpret word order patterns
5. Recognize grammatical word classes such as verbs, nouns, etc.
6. Recognize particular meanings which are expressed in different grammatical
structures
7. Recognize cohesive devices in written discourse

Macroskills

1. Recognize the rethorical forms of written discourse and their significance for
interpretation

Electronic copy available at: https://ssrn.com/abstract=3543269

 2. Recognize the communicative functions of written texts, according to form and
purpose
3. Infer context that is not explicit by using background knowledge.
4. From described events, ideas, etc., infer links and connections between events,
deduce causes and effects, and detect such relations as main idea, supporting idea,
new information, given information, generalization, and exemplification
5. Infer context that is not explicit by using background knowledge.
6. Distinguish between literal and implied meanings
7. Learn to determine culturally specific references and interpret them in a context of
appropriate schemata
8. Develop and use a battery of reading strategies such as scanning, skimming and
guessing the meanings of words from context

Note. Adapted from Microskills, Macroskills, and Strategies for Reading (Brown, D., &
Abeywickrama, 2010)
Further to the suggested skills and strategies, Brown and Abeywickrama (2010) also shared
important strategies for reading comprehension. The strategies highlight the identification of the
test’s purpose, application of spelling rules, bottom-up decoding, guessing of meanings,
skimming and scanning techniques and the use of discourse markers. All these will serve as
guide to test writers and teachers who want to develop reading tests that will ensure fairness in
terms of evaluating students’ reading skills. This is helpful particularly in EFL classrooms.
Emphasizing the same strands of skills and strategies, Urquhart and Weir (1998) also offer the
following descriptions of their exposition of the different reading skills and strategies:

 Skimming. They call this “reading for gist” where reading is selective.
 Search reading. A strategy where learners can locate information, words and ideas on
predetermined topics.
 Scanning. This strategy involves searching for specific information, words or phrases
and other details in the reading test.
 Careful reading. The reader in this type of reading reads the whole text, adopts and
accepts the writer’s viewpoints, hence involve in reading interactively using all
information and prior knowledge.

The demonstration of these skills and strategies establishes the foundation in writing the
guidelines of developing a reading test.

Practical Guidelines in Developing a Reading Test

Step 1: Prepare a Test Blueprint

Test blueprint or test specifications are guidelines that help teachers in a reading class prepare
a good test. It highlights the most important aspects of reading skill that needs to be tested to
show ideal achievement of learning outcomes. To construct a test blueprint, Borich (2014)
offered the following steps:

Electronic copy available at: https://ssrn.com/abstract=3543269

 a. Classify each instructional objective by focusing on the content to be tested based on
the behaviors such as knowledge, comprehension, application, etc.
b. Determine the number of items to be constructed for each objective to establish fairness
and balance in terms of question distribution
c. Total the items for each instructional objective to present the correct distribution of
questions
d. Total the number of items that fall into each behavior
e. Examine both the content area and instructional objectives to ensure the questions are
correctly represented

Figure 2
Sample Test Blueprint of a Reading Test (20 marks)

Test or Knowledge Comprehension Application Analysis Synthesis Evaluation Total Percent

content
outline
(sample
tasks from
a reading
syllabus)
The student 5 5 25 %
will identify
meanings of
simple
terms,
words and
expressions
The student 5 5 25 %
will
infer/predict
meanings
based on
the material
given
The student 5 5 25 %
will
distinguish
similarities
and
differences
of the given
object/idea
The student 5 5 25 %
will
summarize

Electronic copy available at: https://ssrn.com/abstract=3543269

 main events
in the story
The student
will develop
a semantic
web based
on the
information
provided
Total 5 5 5 5 20
25 % 25 % 25 % 25 % 100 %
Percent

By constructing a test blueprint, it will surely help teachers categorize the test questions
according to the necessary behaviors that are observable and measurable. The figure shows
that the questions reflected in the test are systematically categorized with balanced distribution
of marks. This will also help teachers see the right fit of questions when matched to the skills
and capabilities.
Step 2: Design the test
To design a test is not easy. It requires more time particularly in writing and designing test
specifications, objectives and value of the tasks to be performed by the students. As a guide,
Brown and Abeywickrama (2010) discuss in detail the following steps that will surely help
teachers develop clear foundations in the process of designing classroom tests:
• Determine the purpose and usefulness of the test
• Write and state clear objectives
• Draw up specifications
• Select tasks and item tasks and arrange them systematically
• In terms of test administration, provide students the opportunity to achieve their optimal
performance
• Develop and construct a system of grading
• Provide feedback

Step 3: Choose appropriate and relevant reading material

The selection of the reading material is very important in order for the teacher to adapt in terms
of its content, length, context and level of difficulty. As Weir (1998) puts it, the selection of the
reading text should also consider the possibility of how students can expose themselves to
reading contexts and situations that can help them prepare in the future. Students’ knowledge of
information or “schemata’ can facilitate in understanding concepts and ideas (Urquart & Weir,
1998). However, choosing a text can sometimes be complex if it is not substantiated with

Electronic copy available at: https://ssrn.com/abstract=3543269

contents and topics that are not driven from the curriculum. Lloyd and Davidson (2009) maintain
that to help students’ understand reading texts, the selection should somehow relate to a
general topic to allow them to integrate their prior knowledge and experience.

Step 4: Write variety of test questions

It is vital that the content of the reading test offers variety of question types. By doing this, it will
provide students the opportunities to explore their potential, therefore provide a dynamic picture
of their comprehension skills.
Reading the works of Pearson and Johnson (1972) and Nuttall (1996), we would also like to
suggest considering several types of comprehension questions to guide teachers in developing
reading tests. Here are some basic and important descriptions adapted from Day and Park
(2005) on designing different types of reading comprehension questions:

 Literal level. This level of comprehension allows students to answer questions directly
and indirectly. This means that answers can be derived from the text such as facts,
vocabulary items, dates, people, places and events. This stage of questioning can be of
great help especially to EFL learners as a preparation stage before engaging to more
questions that are complex.

 Reorganization. This stage of questioning pertains to students’ knowledge and skills in

reorganizing or combining ideas and information based on the reading text. While it is
true that this type of question is considered difficult compared to the literal level,
reorganization as a task provides students a more challenging engagement because
they need to read and examine the whole article, thus understanding the text with
different perspectives.

 Inferences. This is a combination of literal understanding and students’ own knowledge

and experience. This could be difficult for students because not only the entirety of the
text is involved, but also generating ideas that are not explicitly mentioned in the text. To
answer inference questions, students need more time because the level of complexity is
high; hence, a much-needed integration of their understanding is required.

 Evaluation. This type of question requires students to analyze and interpret information
based on global views and understanding. They need to reflect their own judgement on
particular issues, discussions, and argument that can lead to a better understanding of
the whole text. Their very own experience when integrated in this process can help
facilitate in achieving ideal concepts and notions.

 Personal response. When students’ emotional appeal is highly valued, they too can write
and share effective relatable ideas that are foundations to developing effective reading
skills. Integrating different skills in reading from literal to comprehension, and inferences
to evaluation can certainly support in developing reading skill. However, it is highly

Electronic copy available at: https://ssrn.com/abstract=3543269

 recommended that teachers should provide model tasks and responses to help students
provide effective responses to questions.

Step 5: Fairness in terms of mark distribution, scoring, and test selection

In the context of validity argument, one important view of fairness includes test selection,
administration and scoring. It should be made clear for students that as test takers, they must
be aware of testing conditions and test taking process. Multiple perspectives on assessments
must be explicitly shared in order to attend to the most relevant competencies. But in this
particular context, fairness is treated as a supplementary quality in developing a good test
where students are given opportunities to communicate correct understanding on how they will
be evaluated and what instruments will be used in assessment.

Step 6: Ensure reliable rating

Reliable rating is very important in a classroom-based assessment. In the case of reading
comprehension test, questions should be distributed suitably in terms of difficulty, accuracy of
choices and clarity of ideas. Lloyd and Davidson (2009) suggested that test should be double
rated to ensure reliability. In addition, to support this characteristic of a good test, teachers are
encouraged to participate in moderation and calibration sessions to achieve consistency in
marking. Answer key and sample answers should also be evaluated to make sure teachers and
students share the same expectations in assessment. The works of Hughes (2003), Brown
(2005), Douglas (2010) and Green (2014), suggested that when clear, unambiguous and
comprehensive answer keys and guides are provided in marking, it can help teachers build
reliable rating to students’ tasks. Moreover, the training for raters or scorers will help achieve a
more consistent result.
Test moderation and validation can also facilitate in ensuring that tests developed by teachers
are representative samples of highly validated assessments. On a statistical point of view,
reliable rating supports complexity of different skills that are relevant when used in real-life
settings.

Sample reading texts and exercises

A. Sample Reading Test (Elementary level)

This section provides a sample reading text with questions for elementary level students.

Electronic copy available at: https://ssrn.com/abstract=3543269

Text One: Read the text and answer the questions that follow.

Travel Oman: Omani Architecture of Mirbat Castle

Gautam Viswanathan

Located less than an hour’s drive from the Dhofari capital of Salalah, Mirbat Castle is a popular
tourist attraction for people who visit Oman, particularly during the regular tourism periods of the
southern region, which vary from the rest of the country.
The Dhofar Municipality describes Mirbat as “ a tourist attraction famous for its fascinating
archaeological sites such as Mirbat Castle. The castle is distinguishable by its Omani style of
architecture. While Mirbat Castle and its ancient complex is distinguishable by Islamic
architecture, the tourists can also find here the tombs and graves of scholars.”
Mirbat Castle was temporarily closed to the public and reopened on the 4th of July 2018, after
extensive renovation works were undertaken on the building.
“East of Salalah, the coastal town of Mirbat was once the capital of Dhofar, trading in
frankincense and Arabian horses,” said the Ministry of Tourism. “Today, visitors can spot dhows
heading out to fish, or explore the old merchant houses near Mirbat Fort. Mirbat is a coastal
town in the Dhofar governorate, in southwestern Oman. Formerly known as Moscha, Mirbat was
involved in the export of frankincense in ancient times, to places as far as China.”
“One of the fort’s facilities was allocated to be an office for the management and staff of the fort
who oversee it and organise visits,” according to the Ministry of Information and Oman News
Agency. “They also brief the visitors on the fort’s history and its facilities, as well as the
distribution of publications about the tourist attractions in the Wilayat of Mirbat in particular and
the Governorate of Dhofar in general.
“It receives many visitors, including school students and tourists from most countries of the
world. It hosts many heritage events, such as poetry readings and folklore festivals, in addition
to national holidays, celebrations and other occasions. Given the importance of Mirbat’s historic
fort and to develop archaeological sites and promote tourism, the Ministry of Tourism also
launched a project to incorporate a permanent exhibition in the fort.”

Questions:
1. What is the capital of Dhofar? ____________________________ (literal)

2. Describe Mirbat Fort in your own words. ____________________ (comprehension)

3. How did the Ministry of Tourism support tourism in that region? ______________
(evaluation)

Electronic copy available at: https://ssrn.com/abstract=3543269

 4. Why do you think the castle was temporarily closed to the public? ____________
(inference)

5. What makes “Mirbat” an interesting attraction for tourists? _____________

(reorganization)

6. Do you think Mirbat Castle reflects Omani culture and tradition? _____________
(personal response)

B. Enrichment exercise : For teachers

Instruction for task 1 and 2: The text below is an actual reading test (unedited) developed for
elementary level students. Read the text carefully then determine the levels of comprehension
questions provided. You can also briefly explain whether the types of questions are relevant in
helping students become interactive readers in this context.

The First Omani Man to Cycle over 42 Countries in 90 days

Suleiman bin Saleh bin Salim Al Mawali from Izki is the first Omani who has cycled in 42
countries and is back after cycling for over 7,000 kilometres for some 90 days. Sulaiman, 45,
who retired from the Ministry of Agriculture and Fisheries, is very interested in adventure sport,
and cycling is his greatest passion.
Using his Chinese KLLM bicycle that can carry many items along with charging units for mobiles
and laptops when compared to other bicycles, Sulaiman has travelled to most countries of the
European Union and the Scandinavian countries as well. He said his trip started from Muscat by
air to Paris and from there to Belgium, the Netherlands, Germany, Denmark, Norway, and
Sweden. Then, by ferry, he went to Estonia to cycle across Latvia, Lithuania, Czech Republic,
Slovakia, Slovenia, Hungary, Romania, Serbia, Croatia, Austria, and Italy to finish in Madrid in
Spain and back home.
“During the trip, I did not visit some countries such as Portugal as I found that it would take a
long time owing to the weather conditions there. Instead, I added other countries such as
Croatia, which have recently joined the EU and which were on the way of my cycling trip. In
some countries, it has been difficult for me to go camping as some European countries prohibit
camping in the cities,” he said.
Suleiman made sure to visit the famous landmarks of the cities he visited during his journey.
“This made me familiar with the landmarks of some countries,” he said. Sulaiman had cycled for
800 kilometres in Poland in two weeks and for half an hour in Monaco. During the trip, he mainly
focused on getting familiar with human nature and the landmarks of such countries. He also
visited the villages of the countries where he had cycled.
“I purchased food items and prepared my meals and sometimes stayed in small hotels. During
the trip, I carried items such as a tent, shoes and sports costumes, gloves, water bottles,

Electronic copy available at: https://ssrn.com/abstract=3543269

medicines, a mobile phone and camera, which together weighed 30kg. The trip cost me some
OMR 800 including the visa charges,” he said.

Task 1 (Question 1-5)

With the use of true and false test structure, determine the types and levels of the questions
provided. Also, consider test construction as part of the validation.
____________________________________________________________________________
____________________________________________________________________________
____________________________________________________________________________

Questions 1-5. Read the statements carefully and determine whether the statements are true or
false. Tick the box for your answers.

TRUE FALSE

1. Suleiman likes adventure sports.

2. Suleiman started his trip by cycling from Muscat
to Paris.
3. His last cycling destination was Italy.
4. He spent more time in Poland than in Monaco.
5. He ate his meals at restaurants.

Questions 6-10: Answer the questions. Write no more than 10 words.

6. How is a KLLM bicycle different from other bicycles? _____________________

__________________________________________________________________

7. Why Suleiman did not visit Portugal? __________________________________

__________________________________________________________________

8. Why he could not go camping in some countries? ________________________

__________________________________________________________________

Electronic copy available at: https://ssrn.com/abstract=3543269

9. What did he give more importance to when he travelled to Poland and Monaco?
__________________________________________________________________

10. Name any two things that he took with him during the trip. ________________

Task 2 (Question 6-10)

Are the questions representative sample of the different comprehension question types?
____________________________________________________________________________
____________________________________________________________________________

Final Thoughts

As teachers in EFL context, we have the responsibility of providing our students with
opportunities where they can maximize their skills and abilities. The integration of the different
skills and strategies hopes to facilitate in the design of an effective reading text in EFL context.
When students are given the chance to practice through interactive and extensive teaching and
learning environment, they can understand and develop the process where their involvement is
central in achieving effective learning- “reading comprehension”. We would like to emphasize
that in EFL reading class, it is important that the teaching and assessment of reading skills take
place within the given context and must extend to the many facets of teaching and learning.

Acknowledgment: The writers would like to thank the General Foundation Program (GFP) at
Sohar University, Oman for the use of the reading test as a sample in outlining different levels of
questions. In addition, special thanks and appreciation to the Times of Oman for the reading text
to fit students’ context. This article is totally different compared to journal articles because this is
the output of the workshop conducted relating to reading assessment and test validation.

Electronic copy available at: https://ssrn.com/abstract=3543269

References

Alexander, O., Argent, S., and Spencer, J. (2008). EAP Essentials: A Teacher’s Guide to
Principles and Practice. Reading: Garnet Publishing
Bedle, S. (2017). Reading Strategies in an EFL Context: A Mixed Methods Research Synthesis.
Borich, G. D. (2014). Effective teaching methods: Research-based practice (8th edit).
Brown, H. D., & Abeywickrama, P. (2010). Language assessment: principles and classroom
practice (2nd).
Brown, J. D. (2005). Testing in Language Programs: A Comprehensive Guide to English
Language Assessment. McGraw-Hill College.
Chen, J., and Intaraprasert, C. (2014). Reading Strategies Empployed by University Business
English Majors with Different Levels of Reading Proficiency, English Language Teaching,
7(4), 25-37.
Chirimbu, S., and Barbi-Chirimbu, A. (2015). Reading in a Foreign Language and the
Development of Communicative Competences at the Level of Secondary School Pupils,
Professional Communication and Translation Studies, 8, 177- 185
Day, R. R., & Park, J. S. (2005). Developing Reading Comprehension Questions. Reading in a
foreign language, 17(1), 60-73.
Douglas, D. (2010) Understanding Language Testing. London: Hodder-Arnold.
Duke, N. K., & Pearson, P. D. (2009). Effective practices for developing reading comprehension.
Journal of education, 189(1-2), 107-122.
Ghahari, S., and Basanjideh, M. (2017). Psycho-linguistic Model of Reading Strategies
Awareness in EFL Contexts, Reading Psychology, 38(2), 125-153.
Green, A. (2014). Exploring language testing and assessment. Routledge.
Hedge, Tricia. (2003). Teaching & learning in the language classroom. UK: OUP
Hughes, A. (2003) Testing for Language Teachers (2nd edition). Cambridge, UK: Cambridge
University Press.
Kavlu, A. (2015). The Place of Reading in EFL Context. International Journal of Social Sciences
& Educational Studies, 81.
Lloyd, D., & Davidson, P. (2009). Guidelines for developing a reading test. In C. Coombe, P.
Davidson, & D. Lloyd (Eds.), The fundamentals of language assessment: A practical guide
for teachers (2nd ed.). Dubai, UAE: TESOL Arabia Publications

Electronic copy available at: https://ssrn.com/abstract=3543269

Meniado, J.C. (2016). Metacognitive Reading Strategies, and Reading Comprehension
Performance of Saudi EFL Students, English Language Teaching, 9(3), 117-129.
Nachmani, L. (2015). Cultural Aspects of EFL Reading Acquisition, Procedia, 209, 351-357.
Nuttall, C. (1996). Teaching reading skills in a foreign language. (2nd ed.) Oxford: Heinemann.
Orabi, K. (2018, October 16). First Omani man who cycled in 42 countries in 90 days. Times of
Oman. Retrieved from https://timesofoman.com/article/267632
Pearson, P. D. & Johnson, D. D. (1972). Teaching reading comprehension. New York: Holt,
Rinehart & Winston
Shiotsu, T. (2003). Linguistic knowledge and processing efficiency as predictors of L2 reading
ability: a component skills analysis (Doctoral dissertation, University of Reading).
Snow, C. (2002). Reading for understanding: Toward an R&D program in reading
comprehension. Rand Corporation.
Urquhart, S., & Weir, C. (1998). Reading in a second language: Process, Product, and Practice.
Harlow: Longman.
Urquhart, A. H., & Weir, C. J. (2014). Reading in a second language: Process, product and
practice. Routledge.
Viswanathan, G. (2019, October 20). Travel Oman: Omani architecture of Mirbat Castle. Times
of Oman. Retrieved from https://timesofoman.com/article/2097243/Oman/Tourism/Travel-
Oman-Omani-architecture-of-Mirbat-Castle
Weir, C. (1998). The selection of texts and tasks for testing and teaching academic reading
ability in English. Quality in Learning in English Medium Higher Education, 95-105.
Weir, C.J., Yang, H., & Jin, Y. (2000). An empirical investigation of the componentiality of L2
reading in English for academic purposes (Vol. 12). Cambridge University Press.

Electronic copy available at: https://ssrn.com/abstract=3543269