Construction of

Paper and pencil

Tests

Jan/2024
Writing Objective Test Items

Characteristics of objective tests

 if different individuals score the same paper ;they come up
with the same result
 It is a highly structured
 It limits the type of response the students can make
 It is not appropriate for measuring the ability to supply ,

organize and integrate ideas

Objective test is classified into :
 A. supply type – short answer & completion
 B. selection – True/False , matching and multiple choice
Writing True/False Test items
Strength
 Useful for assessing knowledge of
categorical, factual information like
correctness of facts, definitions of terms,
statements of principles etc.
 Relatively easy to write
 Allows wide sampling of course material
 Limitation
 It measures simple learning outcomes
 It is susceptible to guessing i.e. 50% chance

for guessing
 Truth or false word are relative terms and

can vary from situation to situation

 Do not discriminate students of various

ability
Guidelines for Writing True/False Items
 Incorporate only a single idea in each
statement.
 Write sharp, clear, and grammatically

correct statements as briefly as possible.

 Select precise wording for the item so that

it is not ambiguous. The item should be

either absolutely true or false; avoid
statements that are partly true and partly
false.
Cont…
 Underline or stress a word if there is an adverb or
adjective which is key to marking a statement
correctly,
 Avoid double negatives. Negatively worded

statements tend to confuse examinees so use them

rarely.
 Attribute opinion statements to their source.

However, if testing examinees’ ability to

distinguish fact from opinion, don’t attribute.
 Use true statements in measuring cause and

effect relationship
Cont..
 Avoid absolute or relative terms (e.g., always, never,
sometimes, probably, etc) since they tend to give clue to
the correct response.
 Make the length of true/false items approximately equal.
 Make the number of true and false statements nearly

equal.
 Avoid overlapping statements
 Cont…
 Randomize the occurrence of affirmative and negative
statements
 Avoid directly copying from text books because it

encourages rote memory

 Avoid presenting answers in a manner that form a pattern
 Do not use highly technical terms because it diverts

students attention
 Some Examples of Poor and Better
True/False Items
 Poor: Water will boil at a higher temperature if the
atmospheric pressure on its surface is increased and more
heat is applied to the container. (Do not express the single
idea)
 Better: Water will boil at a higher temperature if the

atmospheric pressure on its surface is increased.

 Poor: A body cannot produce sound unless it is vibrating.

(Double negative)
 Better: A body can produce sound when vibrating.
Cont…
 Poor: When sodium is put in water, it takes the water
molecules apart and joins with part of the broken water
molecules making a new substance called sodium
hydroxide. (Not clear, sharp and simple)
 Better: When sodium is combined with water the new

substance sodium hydroxide is produced.

 Poor: Equivalent sets are equal sets. (Not absolutely

true/false)
 Better: Equal sets are equivalent sets.
 Poor: ማንኛውም ዘር አየርና ሙቀት ብቻ ሲያገኝ ሊበቅል ይችላል፡፡

(use of modifiers)
 Better: ዘር አየርና ሙቀት ሲያገኝ ሊበቅል ይችላል ፡፡
Writing Matching Items
Strength
 It is useful for testing students’ knowledge of terms,

definitions dates, events names, principles and so on.

 It is efficient and useful in emphasizing relationships.

Limitation
 Difficulty of getting plausible distracters.
 It is restricted to the measurement of factual

information based on rote memory

 It is highly susceptible to the presence of irrelevant

clues
 Difficulty of getting homogenous distracters
Guidelines for writing Matching Items

 Give full directions which include the logical basis for

matching and indicate how often a response may be
used.
 Employ only homogeneous content for a set of

premises and responses.

 Keep each list reasonably short, but ensure that each is

complete. Mostly, seven premises and 10-12 options

are recommended as the maximum for each set of
matching items.
Cont…
 To reduce the impact of guessing, list more responses than
premises. The teacher can let a response be used more than
once. This helps prevent examinees from gaining points
through the process of elimination.
 Put responses in alphabetical or numerical order.
 Don’t break matching items across pages
Cont…
 Follow applicable grammatical rules.
 Keep responses as short as possible by using key precise

words.
 Put longer statement in the premise and shorter

statements in the response list.

 Avoid using incomplete sentences as premises
 Randomize the position of correct answers and avoid

making patterns of any kind.

Examples of relationships
 Persons------------------------------------------- Achievement
 Dates --------------------------------------------- Historical Events
 Terms -------------------------------------------- Definitions
 Rules ------------------------------------------- Examples
 Symbols ------------------------------------------ Concepts
 Authors ------------------------------------------ Titles of Books
 Foreign words ---------------------------------- English Equivalent
 Machines ------------------------------------------- Uses
 Plants and Animals ------------------------------ Classification
 Principles ------------------------------------------ Illustrations
 Objects --------------------------------------------- Names of Objects
 Parts ------------------------------------------------ Functions etc
Examples
 Poor- Directions: Match the following.
A B
1. Water A. NaCl
2. Discovered Radium B. Fermi
3. Salt C. NH3
4. Ammonia D. 1942
5. Year of the first Nuclear Fission E. H20
F. Curie
G. 1957
Better - Directions: On the line to the
left of each compound in Column A,
write the letter of the compound’s
formula presented in Column B in the
space provided.
Column A Column B
____1. Water A. H2S04
____2. Salt B. HCl
____3. Ammonia C. NaCl
____4. Sulfuric Acid D. H20
E. H2HCl
Cont….
Column A Column B
---1.ከፕሮቲን እጥረት የሚመጣ በሽታ ሀ. አኒሚያ
---2.ከቫይታሚን ሲ እጥረት የሚመጣ በሽታ ለ. እስከርቪ
---3.ከአዮዲን እጥረት የሚመጣ በሽታ ሐ.እንቅርት
--4.ከብረት ማዕድን እጥረት የሚመጣ በሽታ መ. ኩዋሻርኮር
ሠ. ፔላግራ
Writing Multiple-Choice Test Items
Types of Multiple Choice Items
1. Closed Stem: The stem poses a question
 Ends with a question mark
 Spells out clearly what is being asked
 Directs readers to the options
 In items in which the options are figures, diagrams,

tables, etc., the stem must be closed. Example: Which of

the following is the capital city of Ethiopia?
Cont…
2. Open stem: An incomplete sentence to be completed
by one of the options
 Shorter and more direct
 Could be too unfocused
 Options should be grammatically compatible with

stem

Example: The capital city of Ethiopia is_________.

Strength
 It provides a wide sampling of course content.
 Scoring is easy and objective
 Used in item analysis
 It provides valuable diagnostic information
 It is used in all level of teaching.

Limitation
 It is limited to learning out comes at verbal level
 It does not measure the ability to organize and present idea
 Difficulty to find incorrect but plausible distracters
 Guidelines for Writing Multiple-Choice
Items
 Every item should reflect specific content

and a single specific learning outcome

 Base each item on important content to

learn; avoid trivial content.

 Use novel material to test higher level

learning. Paraphrase textbook language

or language used during instruction when
used in a test item to avoid testing for
simply recall
Cont….
 Keep the content of each item independent
from content of other items on the test.
 Avoid over specific and over general

content when writing multiple choice

items.
 Avoid opinion-based items.
 Keep vocabularies simple for the group of

students being tested

Cont…
 Edit and proof items.
 Use correct grammar, punctuation,

capitalization, and spelling.

 Avoid lengthy item so as to

minimize the amount of reading in

each item
 Writing Stems
 State the stem in question form or completion
form (note: recent research findings favor
question form over completion).
 When using the completion format, don’t
leave a blank for completion in the beginning
or middle of the stem of the question
 Ensure that the directions in the stem are
clear, and that wording lets the examinee
know exactly what is being asked
Cont…
 Base each item to measure a specific learning
outcome.
 Avoid window dressing (excessive verbiage) in the
stem.
 Word the stem positively; avoid negative phrasing.
 Include the central idea and most of the phrasing in
the stem.
 The stem should fully state the problem and all
qualifications. To make sure that the stem presents
a problem, always include a verb in the statement.
 Cont….
 Use multiple-choice to measure higher level
thinking.
 Multiple-choice items should be independent.
That is, an answer to one question should not
depend on the answer to another question.
 Test for important or significant material;
avoid trivial material.
 Concentrate on writing items that measure
students’ ability to comprehend, apply,
analyze, and evaluate as well as recall.
 Cont…
 Include words in the stem that would otherwise be
repeated in each option. Following this guideline not only
saves time for the typist but also saves reading time for
the student.
 Don't ask a question that begins, "Which of the following
is true [or false]?" followed by a collection of unrelated
options. Each test question should focus on some specific
aspect of the course. Therefore, it's OK to use items that
begin, "Which of the following is true [or false]
concerning X?" followed by options all pertaining to X.
 Eliminate excessive wording and irrelevant information
in the stem.
 Punctuate complete sentences correctly
 Cont…
 Minimize the amount of reading by putting as
much of the word(s) as possible in the stem
 Avoid negative stems. Because a negative

stem causes confusion, its use should be

avoided. If the stem can be expressed only
negatively, make the font weight of the word
“not” bold, if a negative stem is unavoidable.
 Keep the vocabulary consistent with the

examinees’ level of understanding

 Cont…
 Avoid over specific knowledge when
developing the stem.
 Avoid direct copying from textbook, verbatim

phrasing when developing the stem.

 Use appropriate grammar, punctuation, and

spelling consistently.
 Avoid tricky items, those which mislead or

deceive examinees into answering incorrectly

 Writing Options
 Options are of similar length and are written in a similar
style to the key. The key must not stand out among the
distractors because of its length, wording, or other
superficial quality.
 Randomly distribute the correct response among the
alternative positions throughout the test. That is, have
approximately the same proportion of A’s, B’s, C’s, and
D’s, as the correct response
Cont…

 Make sure that there is only one

correct/best answer
 Options do not give a clue to the answer to

another item.
 Develop as many effective choices as you

can, but research suggests three to five

options. Each item should have exactly
one key, and three plausible distractors
(four options in all).
Cont…
 Options do not mislead or confuse through
lack of clarity or ambiguity.
 Options do not overlap in meaning. The

distractors must have different meanings

from one another. Distractors should not
be synonyms. A particular meaning in one
distractor should not be included in the
general meaning of another distractor
Cont…
 Avoid the use of the all-of-the-above option.
The problem with “all of the above” as an option is that it
makes the item too easy.
If students can recognize at least one incorrect option, they
can eliminate “all of the above” as a viable option.
On the other hand, if they can recognize at least two correct
options, then they know that “all of the above” is the correct
answer. Students are quick to pick up on this clue.
 Cont…
 Do not use “None of the above” as an option. It can
be used only when absolute standards of
correctness can be applied, such as in math,
grammar, spelling, geography, historical dates, and
so on. Otherwise, students can often argue about
the correctness of one of the other options.
 Avoid the complex multiple-choice format /pairs or

triplets of options. (e.g., A and D, A and C, A, B and

C, etc.). Options do not include partially correct
distractors, such as paired options, where each
distractor contains one incorrect and one correct
option.
 Cont…
 Avoid for specific determiners/absolutes such as
“all”, “always”, “never” which are more likely to be
in incorrect options, avoid also using vague terms
such as “usually” , “frequently” and “sometimes”.
 Format the options vertically, not horizontally if
possible.
 Place options in logical order, if there is a logical
sequence in which the alternatives can be arranged
(alphabetical if a single word, in order of magnitude
if numerals, in temporal sequence, or by length of
response), use that sequence
Cont…
 Keep options independent; options should
not be overlapping.
 Keep all options in an item homogeneous in

content.
 Try to make all responses similar in length.

A very short response or a very long

response often is clue to the correct answer.
 Avoid giving clues through the use of faulty

grammatical construction and

inconsistencies.
Cont…
 Use plausible distractors (equally attractive as the key but
wrong); avoid illogical distracters rather incorporate
common errors of students in distractors.
 Use true statements that do not correctly answer the item for

distractor.
 Avoid conspicuous /obvious correct choice.
 Avoid verbal association between the stem and the correct

answer.
 An item should contain only one correct or clearly best

answer.
 Examples Poor and Better Multiple
Choice Items
Poor: The cell is lets of the pancreas ____. (not a definite,
explicit and singular question in the stem)

A. are located around the edge of the pancreas

B. contain ducts
C. disappear as one grows older
*D. produce insulin
Better: The cell is lets of the
pancreas secrete the substance
called______ .
A. trypsin
B. insulin
C. tryptophan
D. adrenaline
Poor: If the pressure of a certain
amount of gas is held constant, what
will happen if its volume is increased?
(Repeated words in alternatives are not
included in the stem)
A. The temperature of the gas will decrease.
*B. The temperature of the gas will increase.
C. The temperature of the gas will remain the same
Better: If you increase the volume
of a certain amount of gas while
holding its pressure constant, its
temperature will ____.
 A. Decrease
 *B. Increase
 C. Remain the same
Poor : A word used to describe a
noun is called an____. (It has a clue,
the answer will begin with vowel letter)
*A. Adjective
B. Conjunction
C. Pronoun
D. Verb
Better: A word used to describe a
noun is called_____ .
*A. an adjective
B. a conjunction
C. a pronoun
D. a verb
Poor: What age of range represents
the physical "peak” of life? (Alternatives
are not mutually exclusive)

A. 11 to 15 years
B. 13 to 19 years
C. 18 to 25 years
D. 24 to 32 years
Better : What age of range
represents the physical "peak" of life?
A. 11 to 15 years
B. 16 to 20 years
C. 21 to 25 years
D. 26 to 30 years

Poor: 8032- 5743 =________(alternatives are

not plausible
* A. 2289
B. 2288
C. 2378
D. 3378
Better: 8032- 5743 =________
* A. 2289
B. 2389 [failing to change 0 to 9]
C. 3399 [failing to decrease two digits borrowed from]
D. 3711 [subtracting the big number from the small one]
Poor: Penicillin is obtained from a: (choice B
is grammatically inconsistent with the stem)
A. bacteria.
B. coal-tars.
* C. mold.
D. tropical tree
Better: Penicillin is obtained from
A. bacteria.
B. coal-tars.
* C. mold.
D. tropical tree.
Poor: Which of the following structures of the ear
is not concerned with hearing? (Negative stem)
A. cochlea
B. eardrum
C. oval window
* D. semicircular canals
Better: Which one of the following
structures of the ear helps to
maintain balance?
A. cochlea
B. eardrum
C. oval window
* D. semicircular canals
Poor: Which of the following units
is a fundamental unit? (Terms in the
alternatives are not in alphabetical
order)
A. Temperature C. Coulomb
B. Ampere D. Newton
Better: Which of the following units is a
fundamental unit?
A. Ampere C. Newton
B. Coulomb D. Temperature
Poor: What is the cost of an item that normally
sells birr 9.99 that is discounted 25%? (The
numbers in the alternatives are not in the natural
order)
A. 5.0
*B. 7.50
C. 2.50
D. 6.66
Better: What is the cost of an item
that normally sells for birr 9.99 that
is discounted 25%?

A. 2.50
B. 5.00
C. 6.66
D. 7.50
Writing Supply Items
 The two major supply items are completion and short
answers. Short answer and completion items are
equivalent test formats: they differ only in the phrasing
of the question.
 Completion Items are incomplete statements. They

require the examiner to supply a word, a number, a

symbol or a phrase, as a response to a question to
complete a statement.
 Short Answer items are usually given in a direct

question where the student is expected to write names,

places, dates, processes, terms etc.
Advantages of Supply Items
 Supply items are the easiest to construct.
 Guessing is minimized as compared to other
objective tests because students are required to
produce their own responses rather than
select.
 Efficiently measure simple (factual)
knowledge or lowest levels of cognitive
domain; knowledge of terminology, facts,
principles etc…
Limitations
 Does not measure complex learning
outcomes.
 They are relatively time consuming

to score than other objective tests.

 Although the answers are very short ,

they are not completely free from

subjectivity.
Guidelines for writing Completion and Short
Answer Items
 Focus the statement or question so that there is only one
concise answer of one or two words or phrases. Use
precise statement or question wording to avoid ambiguous
items.
 A complete question is recommended over an incomplete

statement. You should use one, unbroken blank of

sufficient length for the answer.
 If an incomplete statement is used, avoid using broken

lines which correlate to the number of letters in each word

of the desired answer.
 Cont…
 Omit only significant words from the statement, but do not
omit so many words that the statement becomes
ambiguous.
 Use blanks of the same length throughout the test so that

the length is not a clue

 Place the blank at the end of the statement or question.
 For incomplete statements, select a key word or words as

the missing element(s) of the statement.

Examples poor and Better Items
Poor: Two lines perpendicular to the same line in the
same plane are _____ to each other
Better: If two lines are drawn perpendicular to the same
line on a sheet of paper, then they are _______.
Poor: Ethiopia defeated Italy in _____. (Has no one
concise answer)
Better: Ethiopia defeated Italy at the battle ____. Or In
what battle did Ethiopia defeat Italy?
Cont…
Poor: ________ = y , if 4y+8=3y-10 (blank space is not at
the end of the statement)
Better: If 4y+8=3y-10, Then y = __________ .
Poor: Every atom has a central _____ called a nucleus.
(Omitted word is not significant)
Better: Every atom has a central core called ____.
Poor: If a room measures 7 meters by 4 meters, the
perimeter is _____ . (unit of measurement is not
indicated)
Better: If a room measures 7 meters by 4 meters, the
perimeter is _____ meters (or m).
 Writing Essay Test Items
 Essay test has two forms: Extended and Restricted
Response forms.
Extended response
The following are some examples of extended
response essay type items :
 Compare and contrast the use of land and air

transportation systems (Extended),

 Describe what you think the role of federal

government should be in maintaining a stable

economy in Ethiopia(extended)
 Discuss about the importance of the African Union
Restricted Response
The following are some examples of
restricted response essay type items:
 State two advantages and disadvantages of maintaining
high tariffs on goods from other countries(restricted)
 Name two types of clouds and describe the characteristics

that distinguish them (restricted)

 Explain the difference between Assessment of learning

and assessment for learning in a paragraph (restricted

Strengths
 Measure complex learning outcomes that cannot
be measured by other means.
 Emphasize the integration and application of

thinking and problem solving skills

 They have desirable influence on students study

habit. Students tend to direct their attention

toward the integration and application of larger
units of subject matter when essay questions are
included in classroom test.
 Are easier and quicker to construct
 Minimize guessing to a greater extent
Limitation
 Scoring is unreliable
 The amount of time required for scoring

is too long
 They limit the number of questions that

can be asked and thus limit the content

coverage of the examination
 The influence of bluffing due to writing

ability of the student

THANK YOU!