You are on page 1of 18

Topic  Planning

Classroom
3 Tests
LEARNING OUTCOMES

By the end of the topic, you should be able to:


1. Describe the process of planning a classroom test;
2. Explain the purposes of a test and their impotance in test planning;
3. Describe how learning outcomes to be assessed affect test planning;
4. Select the best item types for a test in line with learning outcomes;
5. Develop a table of specifications for a test;
6. Identify appropriate marking schemes for an essay test; and
7. Explain the general principles of constructing relevant test items.

 INTRODUCTION
In this topic, we will focus on the process of planning classroom tests. Testing is
part of the teaching and learning process. The importance of planning and writing
a reliable, valid and fair test cannot be underestimated. Designing tests is an
important part of assessing studentsÊ understanding of course content and their
level of competency in applying what they have learnt. Whether you use low-
stakes quizzes or high-stakes mid-semester and final examination tests, careful
design will help provide more calibrated results. Assessments should reveal how
well students have learnt what teachers want them to learn while instruction
ensures that they learn it.

Copyright © Open University Malaysia (OUM)


46  TOPIC 3 PLANNING CLASSROOM TESTS

Thus, thinking about summative assessment at the end of a programme of teaching


is not enough. It is also helpful to think about assessment at every stage of the
planning process, because identifying the ways in which teachers will assess their
students will help clarify what it is that teachers want them to learn, and this in
turn will help determine the most suitable learning activities.

This topic will discuss the general guidelines applicable to most assessment tools
when planning a test. Topics 4 and 5 will discuss in detail the objectives of essay
tests. The authentic assessment tools such as projects and portfolios will be
discussed in the respective topics.

3.1 PURPOSES OF CLASSROOM TESTING


Tests can refer to traditional paper-and-pencil or computer-based tests, such as
multiple choice, short answer and essay tests. Tests provide teachers with objective
feedback as to how much students are learning and how much they have
understood what they have learnt. Commercially published achievement tests to
some extent can provide evaluation of the knowledge levels of individual students,
but provide only limited instructional guidance in assessing the wide range of
skills taught in any given classroom.

Teachers know their students and they are the best assessors of their students.
Tests developed by the individual teachers for use with their own class are most
instructionally relevant. Teachers can tailor tests to emphasise the information
they consider important and to match the ability levels of their students. If
carefully constructed, classroom tests can provide teachers with accurate and
useful information about the knowledge retained by their students.

The key to this process is the test questions that are used to elicit evidence of
learning. Test questions and tasks are not just planning tools; they also form an
essential part of the teaching sequence. Incorporating the tasks into teaching and
using the evidence about the student learning to determine what happens next in
the lesson is truly an embedded formative assessment.

Copyright © Open University Malaysia (OUM)


TOPIC 3 PLANNING CLASSROOM TESTS  47

3.2 PLANNING A CLASSROOM TEST


A well constructed test must have high quality items. The well constructed test is
an instrument that provides accurate measure of test takerÊs ability within a
particular domain. It is worth spending time writing high quality items for the
tests. In order to produce high quality questions, the test construction has to be
properly planned. Let us look at the following steps of planning a test (refer to
Figure 3.1).

Figure 3.1: Planning a test

3.2.1 Deciding Its Purposes


The first step in test planning is to decide on the purpose of the test. Tests can be
used for many different purposes. If a test is to be used formatively, it should
indicate precisely what the student needs to study, and to what level. The purpose
of formative tests is to assess progress and to direct the learning process. These
tests will have limited sample of content and learning outcomes. Teachers must
prepare sufficient mix of easy and difficult items. These items are used to make
corrective prescriptions such as practice exercises for some students who do not
perform satisfactorily in the tests.

Copyright © Open University Malaysia (OUM)


48  TOPIC 3 PLANNING CLASSROOM TESTS

If a test is to be used summatively, the coverage of content and learning outcomes


would be different from that of formative tests. Summative tests are normally
conducted at the end of a teaching and learning phase, for example, at the end of
a course. They are used to determine the studentsÊ mastery level of the course and
help teachers to decide whether a particular student can proceed to the next level
of his or her studies. The summative tests should therefore cover the whole content
areas and learning outcomes of the course, or should at least cover a representative
sample of the contents and learning outcomes of the course. The test items are also
varied in their levels of difficulty and complexity as defined by the learning
outcomes.

Tests can also serve a diagnostic purpose. Diagnostic tests are used to find out
what students know and do not know, and their strengths and weaknesses. They
typically happen at the start of a new phase of education, like when they start
learning a new course. The tests normally cover topics (content as well as learning
outcomes) that students will study in the upcoming course. The test items included
in the test are usually simple. Besides, diagnostic tests are also used to „diagnose‰
the learning difficulties encountered by students. When used for this purpose, the
tests will cover specific content areas and learning outcomes and hope to unravel
the causes of the learning problems so that remediation can be implemented.

3.2.2 Specifying the Intended Learning Outcomes


The focus of instruction in a course of study is not mere acquisition of knowledge
by students but more importantly on how they can use and apply the acquired
knowledge in different and meaningful situations. The latter has been referred to
as course learning outcomes (CLOs), which should cover the cognitive, affective
and psychomotor domains as explained in Topic 2. In other words, the emphasis
in instruction should be on the mastery of CLOs when teachers deliver the content
covered in the topics of the course. The syllabus of a course should therefore
present not only the relevant content areas in the form of topics but also indicate
the CLOs to be achieved. A course of study might have a number of topics but only
three to five CLOs. For instance, for an Educational Assessment course, there may
be 10 topics to be covered with four CLOs, which are spread across the 10 topics
as shown in Table 3.1.

Copyright © Open University Malaysia (OUM)


TOPIC 3 PLANNING CLASSROOM TESTS  49

Table 3.1: Mapping of Course Learning Outcomes Across Topics

CLO Explain the Compare the


Develop Critically Evaluate
Different Different
Different the Suitability of
Principles and Methods of
Assessment Different
Theories of Educational
Methods for Assessment
Educational Testing and
Use in the Methods for Use in
Testing and Assessment
Classroom (C3) the Classroom (C6)
Topic Assessment (C2) (C4)
1 x
2 x
3 x x x
4 x x x
5 x
6 x x
7 x x
8 x
9 x
10 x

Note: The parentheses indicate the levels of complexity according to the BloomÊs
taxonomy

In line with the principle of constructive alignment, assessment of a course should


also focus on the mastery of CLOs. CLOs are normally written in general terms.
Under each topic, the learning outcome is more specific and is often referred to as
an intended learning outcome (ILO). In assessing a topic of a course, it is
imperative that its ILO is clearly specified. Table 3.2 states examples of CLO and
its related ILO for a specific topic in the Educational Assessment course (i.e.
Portfolio Assessment)

Table 3.2: Example of Course Learning Outcome (CLO) and Intended


Learning Outcome (ILO)

Critically evaluate the suitability of different assessment methods for use in the
CLO
classroom (C6)
ILO Critically evaluate the usefulness of portfolios as an assessment tool (C6)

Copyright © Open University Malaysia (OUM)


50  TOPIC 3 PLANNING CLASSROOM TESTS

A word of caution. Remember, not all ILOs can be assessed by tests. Tests are only
appropriate in assessing cognitive learning outcomes. For example, of the
following three intended learning outcomes (ILO), only ILO 1 can be assessed by
a test using an essay question. On the other hand, ILO 2, which belongs to the
pyschomotor domain is more appropriately assessed by practical work via teacher
observation, while ILO 3 which belongs to the affective domain, may be assessed
during the implementation of the class project via peer evaluation.

ILO 1 Explain the differences among the cognitive, affective and psychomotor
domains of learning outcomes.
ILO 2 Demonstrate the proper technique of executing a table tennis top-spin
in service.
ILO 3 Work collaboratively with other students in the team to complete the
class project.

SELF-CHECK 3.1

1. What type of learning outcome in BloomÊs taxonomy can be


assessed by tests? Why?

2. How is the intended learning outcome (ILO) different from course


learning outcome (CLO)?

3.2.3 Selecting Best Item Types


Once the intended learning outcomes (ILOs) for the topics to be assessed have been
specified, the next step in planning a test is to select the best item types. Different
item types have different purposes and are different in their usefulness. Table 3.3
shows two common item types used in a test ă multiple-choice and essay questions
and their respective purposes and usefulness. Refer to Topics 4 and 5 for more
details.

Copyright © Open University Malaysia (OUM)


TOPIC 3 PLANNING CLASSROOM TESTS  51

Table 3.3: Item Types and Their Respective Purposes and Usefulness

No. Item Type Purpose and Usefuness


1 Multiple-choice  Test for factual knowledge
 Assess a large number of items
 Score rapidly, accurately and objectively
2 Essay  Require candidates to write an extended piece on a
certain topic
 Assess higher-order thinking skills such as analysing,
sythesising and evaluating in BloomÊs taxonomy

It is thus imperative that the item types selected in assessment should be relevant
to the ILO to be assessed. There must be a close match between the ILOs and the
types of items to be used. For example, if the ILO is to develop the ability to
organise ideas, the use of multiple-choice test would be a poor choice. The best
item type would be an essay question. The following are two intended learning
outcomes (refer to Table 3.4). Can you select the best item types to assess them?

Table 3.4: Examples of Intended Learning Outcomes

ILO 1 Discuss the usefulness of portfolios as an assessment tool in education.


ILO 2 Define what a portfolio is.

ILO 1 requires students to present a discussion. They need to thoroughly review,


examine, debate or argue the pros and cons of a subject. To do this, they need to
write an extended response. ILO 1 can only be assessed by an essay test. However,
ILO 2 merely requires students to identify a definition. A multiple-choice question
(MCQ) is good enough to perform the assessment task.

Copyright © Open University Malaysia (OUM)


52  TOPIC 3 PLANNING CLASSROOM TESTS

ACTIVITY 3.1

The following is a list of learning outcomes. Identify the best item type
to assess each of them.

No. Learning Outcome MCQ/Essay

1 Name the levels of BloomÊs taxonomy and identify


the intellectual behaviour each refers to.

2 Devise a table of specification, complete with


information on what to assess and how to assess.

3 Discuss the strengths and weaknesses of using


essay questions as an assessment tool.
4 Defend the use of portfolios for classroom
assessment.

5 Define norm-referenced and criterion-referenced


assessments.

6 Explain the purposes of assessment in education.

7 Describe the process involved in planning a test.

8 Illustrate the use of item analysis in assessing the


quality of a MCQ.
9 Differentiate between formative and summative
assessments.

10 Develop appropriate scoring rubrics as marking


schemes for essay questions.

11 State the advantages and disadvantages of


multiple-choice items as an assessment tool.
12 Examine the usefulness of project work as an
assessment tool.

Copyright © Open University Malaysia (OUM)


TOPIC 3 PLANNING CLASSROOM TESTS  53

3.2.4 Developing a Table of Specifications


Making a test blueprint or table of specifications is the next important step that
teachers should do. The table presents the topics of the course, the cognitive
complexity levels of the test items according to BloomÊs taxonomy, the number of
test items corresponding to the number of hours devoted to the topics and course
learning outcomes in class. In fact, the decision of exactly how many test items to
include in a test is based on the importance of the topics and learning outcomes as
indicated by student learning time, the item types used, and also the amount of
time available for testing.

A table of specifications is a two-way table with the cognitive complexity levels


across the top, and the topics and course learning outcomes to be covered by a test
and hours of interaction down one side. The item numbers associated with each
topic are presented under the complexity level as determined by the CLO.
Table 3.5 presents an example of a table of specifications with MCQs as the item
type. For ease of understanding, let us assume that the test will only cover the
first three complexity levels of BloomÊs taxonomy, namely Knowledge (C1),
Comprehension (C2) and Application (C3).

Copyright © Open University Malaysia (OUM)


54  TOPIC 3 PLANNING CLASSROOM TESTS

Table 3.5: Table of Specifications: MCQ Item Type

Copyright © Open University Malaysia (OUM)


TOPIC 3 PLANNING CLASSROOM TESTS  55

In this example, the vertical columns on the left of the two-way table show a list of
the topics covered in class and the amount of time spent on those topics. The
amount of time spent on the topics as shown in the column „Hours of Interaction‰
is used as a basis to compute the weightage or percentage (% hours) and the marks
allocated. For a test with MCQs, the marks allocated also indicate the number of
test items for each topic.

In this hypothetical case, the teacher has spent 20 hours teaching the three topics
of which 4 hours are alloted to Topic 1. Thus, 4 hours from a total of 20 hours
amount to 20 per cent or six items from the total of 30 items as planned by the
teacher. Likewise, the weightage or percentage and the marks alloted for Topics 2
and 3 are computed in the same manner. The weightage and number of items for
Topic 2 are 30 per cent and nine items respectively. For Topic 3, they are 50 per
cent and 15 items respectively.

Based on the cognitive complexity level of the CLO for each topic, the teacher will
decide on the number of items to be included under each level. This information
is presented in the cells of the column on Item No. For example, the cognitive
complexity level of the CLO1 for Topic 1 is C2, the teacher has decided to have two
items at C1 (i.e. items 1 and 2) and four items at C2 (i.e. items 10, 11, 12 and 13). Of
course, he or she can decide to have all the six items framed at C2, but not at C3.
For Topic 2, the number of items required is nine at C2. Again, the teacher has
decided to have some items at C1 (i.e. four items) and the rest at C2 (i.e. five items)
to make up the required number of items. Topic 3 seems to be the most important
topic and it requires 15 items, i.e. half of the total in the test, and the teacher has
decided to have three items at C1, six items at C2 and another six items at C3.
Overall, of the total 30 items in the test, 30 per cent of them are at C1, 50 per cent
at C2 and 20 per cent at C3. The teacher, of course, might have a reason for such a
distribution. Perhaps, he or she feels that this is the beginning of the course, and
he or she wants to focus on the understanding of the key concepts of the course.
Whatever it is, the decision is the prerogative of the teacher who knows best on
what and how he or she wants to assess the students.

Table 3.6 is another example of a table of specifications. The table focuses on essay
items.

Copyright © Open University Malaysia (OUM)


56  TOPIC 3 PLANNING CLASSROOM TESTS

Table 3.6: Table of Specifications: Essay Questions

Copyright © Open University Malaysia (OUM)


TOPIC 3 PLANNING CLASSROOM TESTS  57

The first vertical column on the left presents the five topics identified for
assessment, followed by hours of interaction for each topic in the second column.
Based on this formation, the teacher can work out the weightage in terms of %
hours and marks for each topic. In this hypothetical case, the teacher has spent
50 hours teaching the five topics of which 5 hours each are alloted to Topics 1 and
2. Thus, 5 hours from a total of 50 hours amount to 10 per cent or 10 marks from
the total of 100 total marks as planned by the teacher. Likewise, the weightage or
percentage and the marks alloted for Topic 2, Topic 3, Topic 4 and Topic 5 are
computed in the same manner. The weightage and marks alloted for Topic 2,
Topic 3, Topic 4 and Topic 5 are 10, 20, 20 and 40 respectively.

Based on the marks alloted and the cognitive complexity levels of the CLOs, the
teacher then decides how he or she is going to distribute the marks according to
the levels of complexity. For example, for Topic 1, he or she can have one essay
item carrying 10 marks at C3 or two essay items, one at C2 and the other at C3 and
each carries 5 marks. In this hypothetical case, the teacher has decided to have two
items for Topic 1 and another two for Topic 2, each carrying 5 marks. The four
items make up the Section A of the test. For Topic 3, he or she has decided to
distribute the 20 marks between two items, each carrying 10 marks. He or she has
done the same for Topic 4. This makes up the Section B. Section C is alloted to
Topic 5 with two items, each carrying 20 marks. This is just an example of how the
marks for each topic are distributed and the number of items decided. There can
be many other variations, of course.

So far, we have looked at the table of specifications in the form of a two-way table.
A table of specifications can be in the form of a three-way table with item types as
an additional level. Whatever the format, the table of specifications is a very useful
piece of document in assessment. This kind of table ensures that a fair and
representative sample of items or questions appear in the test. Teachers cannot
measure every piece of content in the syllabus and cannot ask every question they
might wish to ask. A table of specifications allows the teacher to construct a test
which focuses on the key contents as defined by the weights in percentages given
to them. A table of specifications provides the teacher with evidence that a test
has content validity, that it covers what should be covered. This table also allows
teacher to view the test as a whole.

The teacher, especially a newly trained one, is advised to have this table of
specifications together with the subject syllabus reviewed by the subject expert or
the subject head of department whether the test plan has included what it is
supposed to measure. In other words, it is important that the table of specifications
must have content validity. To ensure this, the students should ideally not be given
choices in a test. Without choices, all students are thus assessed equally.

Copyright © Open University Malaysia (OUM)


58  TOPIC 3 PLANNING CLASSROOM TESTS

SELF-CHECK 3.2

What is a table of specifications?

ACTIVITY 3.2

1. Have you used a table of specifications?

2. Identify a course of your choice, prepare a table of specifications for


a test.

Share your answers with your coursemates in the myINSPIRE online


forum.

3.2.5 Constructing Test Items


Once a valid table of specifications has been prepared, the next step is constructing
the test items. While the different item types such as multiple choice, short answer,
true-false, matching and essay items are constructed differently, the following
principles apply to constructing test items in general.

(a) Make the instructions for each type of item simple and brief;

(b) Use simple and clear language in the questions;

(c) Write items that are appropriate for the learning outcomes to be measured;

(d) Do not provide clue or suggest the answer to one question in the body of
another question;

(e) Avoid writing questions in the negative. If you must use negatives, highlight
them, as they may mislead students into answering incorrectly;

(f) Specify the precision of answers;

(g) Try as far as possible to write your own questions. Check to make sure the
questions fit the learning objectives and requirements in the table of
specifications if you need to use questions from other sources; and

(h) If the item was revised, recheck its relevance.

Copyright © Open University Malaysia (OUM)


TOPIC 3 PLANNING CLASSROOM TESTS  59

In writing test items, you must also consider the length of the test as well as the
reading level of your students. You do not want students to feel rushed and
frustrated because they are not able to demonstrate their knowledge of the material
in the allotted time. Some general guidelines regarding time requirements for
secondary school student test takers are shown in Table 3.7.

Table 3.7: Allotment of Time for Each Type of Question

Task Approximate Time per Item


True-false items 20ă30 seconds
Multiple choice (factual) 40ă60 seconds
Multiple choice (complex) 70ă90 seconds
Matching (five stems/six choices) 2ă4 minutes
Short answer 2ă4 minutes
Multiple choice (with calculations) 2ă5 minutes
Word problems (simple math) 5ă10 minutes
Short essays 15ă20 minutes
Data analysis/graphing 15ă25 minutes
Extended essays 35ă50 minutes

If you are combining multiple choice and essay items, these estimates may help
you decide how many of each type of items to include. One mistake often made
by many educators is having too many questions for the time allowed.

Once your items are developed, make sure that you include clear directions to the
students. For the objective items, specify that they should select one answer for
each item and indicate the point value of each question, especially if you are
weighting sections of the test differently. For essay items, indicate the point value
and suggested time to be spent on the item (we will discuss essay questions in
more detail in Topic 5). If you are teaching a large class with close seating
arrangements and are giving an objective test, you may want to consider
administering several versions of your test to decrease the opportunities for
cheating. You can create versions of your test with different arrangements of the
items.

More detailed guidelines to prepare and write multiple choice, short answer, true-
false, matching, essay, portfolios and projects will be discussed in Topics 4, 5, 6
and 7 respectively.

Copyright © Open University Malaysia (OUM)


60  TOPIC 3 PLANNING CLASSROOM TESTS

ACTIVITY 3.3

To what extent do you agree with the allotment of time for each type of
question shown in Table 3.7?

Justify your answer by discussing it with your coursemates in the


myINSPIRE online forum.

3.2.6 Preparing Marking Schemes


Preparing a marking scheme well in advance of testing date will give teachers
ample time to review their questions and make changes to answers when
necessary.

The teacher should make it a habit to write a model answer which can be easily
understood by others. This model answer can be used by other teachers who act
as external examiners, if need be. For objective test items, the model answers are
simple. The marking scheme is just a list of answers with the marks alloted for
each. However, for essay items, the marking schemes can be a bit complicated and
require special skills and knowledge to prepare. The marking schemes may take
the form of a checklist, a rubric or a combination of both. Refer to Topic 5 for a
detailed explanation on the marking scheme.

Coordination on the use of marking schemes should be done once the test answer
scripts are collected. Teachers should try to read answers from some scripts and
review the correct answers in the marking scheme. Teachers may sometimes find
that students have interpreted the test question in a way that is different from what
is intended. Students may come up with excellent answers that may fall slightly
outside what has been asked. Consider giving these students marks accordingly.
Likewise, teachers should make a note in the marking scheme for any error made
earlier but carried through the answer; marks should be deducted if the rest of the
response is sound.

SELF-CHECK 3.3

Why is it neccessary for a test to be accompanied by a marking scheme?

Copyright © Open University Malaysia (OUM)


TOPIC 3 PLANNING CLASSROOM TESTS  61

3.3 ASSESSING TEACHER’S OWN TEST


Regardless of the kind of tests teachers use, they can assess their effectiveness by
asking the following questions:

(a) Did I Test for What I Thought I Was Testing for?


If you wanted to know whether students could apply a concept to a new
situation, but mostly asked questions determining whether they could label
parts or define terms, then you tested for recall rather than application.

(b) Did I Test What I Taught?


For example, your questions may have tested the studentsÊ understanding of
surface features or procedures, while you had been lecturing on causation or
relation ă not so much what the names of the bones of the foot are, but how
they work together when we walk.

(c) Did I Test for What I Emphasised in Class?


Make sure that you have asked most of the questions about the material you
feel is the most important, especially if you have emphasised it in class.
Avoid questions on obscure material that are weighted the same as questions
on crucial material.

(d) Is the Material I Tested for Really What I Wanted Students to Learn?
For example, if you wanted students to use analytical skills such as the ability
to recognise patterns or draw inferences, but only used true-false questions
requiring non-inferential recall, you might try writing more complex true-
false or MCQs.

Students should know what is expected of them. They should be able to identify
the characteristics of a satisfactory answer and understand the relative importance
of those characteristics. This can be achieved in many ways: you can provide
feedback on tests, describe your expectations in class, or post model solutions on
a class blog. Teachers are encouraged to make notes on the scripts. When exams
are returned to the students, the notes will help them understand their mistakes
and correct them.

SELF-CHECK 3.4

Describe the steps involved in planning a test.

Copyright © Open University Malaysia (OUM)


62  TOPIC 3 PLANNING CLASSROOM TESTS

 The first step in test planning is to decide on the purpose of the test. Tests can
be used for many different purposes.

 The next step is to consider the learning outcomes and their complexity levels
as defined by BloomÊs taxonomy. The teachers will have to select the
appropriate knowledge and skills to be assessed and include more questions
about the most important learning outcomes.

 The learning outcomes that the teachers want to emphasise will determine not
only what material to include on the test, but also the specific form the test will
take.

 Making a test blueprint or table of specifications is the next important step that
teachers should do.

 The table describes the topics, the behaviour of the students, the number of
questions on the test corresponding to the number of hours devoted to the
topics in class.

 The table of specifications helps to ensure that there is a match between what
is taught and what is tested.

 Classroom assessment is driven by classroom teaching which itself is driven


by learning outcomes.

 The test format used is one of the main driving factors in the studentsÊ learning
behaviour.

Checklist Intended learning outcome (ILO)


Complexity levels of BloomÊs Marking schemes
taxonomy
Rubrics
Course learning outcome (CLO)
Table of specifications
Hours of interaction

Copyright © Open University Malaysia (OUM)

You might also like