Topic: Constructing Objective Test Items

Topic
 Constructing
Objective Test
4 Items
LEARNING OUTCOMES
By the end of this topic, you should be able to:
1. Define objective tests;
2. Differentiate between different types of objective tests;
3. Discuss the advantages of using different types of objective tests;
4. Identify the weaknesses of using the different types of objective tests;
and
5. Explain the techniques for constructing the different types of
objective test items.
 INTRODUCTION
In this topic we will focus on using objective tests to assess various kinds of
behaviour in the classroom. Firstly, the discussion will be limited to the simple
forms of objective test items, namely short-answer item, true-false item and
matching item. Three types of objective tests are examined and guidelines for the
construction of each type of the tests are discussed. The advantages and limitations
of each of these types of objective tests are explained. Secondly, we will discuss the
multiple-choice item, a more complex form of objective test items. The discussion
will focus on the characteristics and uses of multiple-choice items, their advantages
and limitations and some suggestions for the construction of such items.
Copyright © Open University Malaysia (OUM)

56  TOPIC 4 CONSTRUCTING OBJECTIVE TEST ITEMS
4.1 WHAT IS AN OBJECTIVE TEST?

When objective tests were first used in 1845 by George Fisher in the United States,
it was not well received by the society. However, over the years it gained
acceptance and was widely used. Today it is perhaps the most popular format of
assessing various types of human abilities, competencies and socioemotional
attributes. Objective tests are extensively used in schools, industries, businesses,
professional organisations, universities and colleges. Can you guess why?
What is an objective test? An objective test is a written test consisting of items or

questions which require the respondent to answer them by supplying a word,
phrase or symbol or by selecting from a list of possible answers. The former test
items are often referred to as supply-type test items while the latter is referred to
as selection-type test item. The word objective means „accurate‰. An objective item
or question is „accurate‰ because it cannot be influenced by the personal
preferences and prejudices of the marker. In other words, it is not „subjective‰ and
not open to varying interpretations. This is one of the reasons why the objective
test is popular in measuring human abilities, competencies and many other
psychological attributes such as personality, interests and attitudes. Among the
common objective test items are as shown in Figure 4.1.
Figure 4.1: Common objective test items

TOPIC 4 CONSTRUCTING OBJECTIVE TEST ITEMS  57
4.2 SHORT-ANSWER QUESTIONS

A short-answer question is basically a supply-type item. It exists in two formats,
namely direct question and completion question formats. Figure 4.2 shows
examples of short-answer questions:
Figure 4.2: Examples of short-answer questions
You can refer to Linn and Gronlund (1995) for more examples.
Now, let us look at the advantages and the limitations of this type of question.
The short-answer question is generally used to measure simple learning outcomes.

It is used almost exclusively to measure memorised information (except for the
problem-solving outcomes measured in Mathematics and Science). This has partly
made the short-answer question one of the easiest to construct.
Another advantage of the short-answer question is that the possibility of guessing

which often occur in the supply-type item can be reduced. In this case, learners
must supply the correct answer when they respond to the short-answer question.
They must either recall the information asked for or make the necessary
computations to obtain the answer. They cannot rely on their partial knowledge to
choose the correct answer from the list of alternatives.
Many short-answer questions can be set for a specific period of time. A test paper
consisting of short-answer questions is thus able to cover a fairly wide content of
the course to be assessed. A wide content coverage enhances the content validity
of the test.

One major limitation of the short-answer question is that it cannot be used to

measure complex learning outcomes such as organising ideas, presenting an
argument or evaluating information. What is required of learners is providing a
word, phrase, number or symbol.
Scoring of answers to the short-answer question can also pose a problem. Unless
the question is carefully phrased, learners can provide answers of varying degree
of correctness. For example, answer to a question such as „When was Malaysia
formed?‰ could either be „In 1963‰ or „On 16 September 1963‰. The teacher has to
decide whether learners who gave the partial answer have the same level of
knowledge as those who provided the complete answer. Besides, learnersÊ
answers can also be contaminated by spelling errors. If spelling is taken into
consideration, the test scores of learners will reflect their level of knowledge of the
content assessed as well as their spelling ability. If spelling is not considered in the
scoring, the teacher has to decide whether the misspelled word actually represents
the correct answer.
Although the construction of short-answer questions is comparatively easier than

other types of questions, they have variety defects which should be avoided to
ensure that they will function as intended. The following are some guidelines for
the construction of short-answer questions:
(a) Word the question so that the intended answer is brief and specific
As far as possible, the question should be phrased in such a way that only
one answer is correct.
For example:
Poor item: An animal that eats the flesh of other animals is

_____________.
(Possible answer: a wolf, a lion, hungry, ⁄)
Better item: An animal that eats the flesh of other animals is classified as
___________.
(One specific answer: carnivorous)

(b) Use direct questions instead of incomplete statements

Direct questions are more natural and usually better structured.
For example:
Poor item: Columbus discovered America in ________.
Better item: In what year did Columbus discover America? (1492)
Possible answers for 1st item: a boat, the 15th century, a search for India
(c) If the problem requires a numerical answer, indicate the units in which the
answer is to be expressed
For example:
(d) Do not include too many blanks for the completion item
Blanks for answers should be equal in length. For the completion item, place
the blank near the end of the sentence.
For example:
SELF-CHECK 4.1
1. What are some advantages of short-answer questions?
2. Describe some limitations of short-answer questions.

ACTIVITY 4.1
Suggest other advantages and weaknesses of using short-answer

questions.
Share your answer with your coursemates in the myINSPIRE online

forum.
4.3 TRUE-FALSE QUESTIONS

In the most basic format, true-false questions are those in which a statement is
presented and the learner indicates in some manner whether the statement is true
or false. In other words, there are only two possible responses for each item, and
the learner chooses between them. A true-false question is a specialised form of
the multiple-choice format in which there are only two possible alternatives. These
questions can be used when the test designer wishes to measure a learnerÊs ability
to identify whether statements of fact are accurate or not. True-false questions can
be used for testing knowledge and judgement in many subjects. When grouped
together, a series of true-false questions on a specific topic or scenario can test a
more complex understanding of an issue. They can be structured to lead a learner
through a logical pathway (Brown, 1997) and can reveal part of the thinking
process employed by the learner in order to solve a given problem.
For example:
True False
A whale is a mammal because it gives birth to its young.
True-false questions can be quickly written and can cover a lot of content. True-
false questions are well suited for testing learner recall or comprehension. Learners
can generally respond to many questions, covering a lot of content in a fairly short
amount of time. From the teacherÊs perspective, these questions can be written
quickly and are easy to score. Because they can be objectively scored, the scores
are more reliable than for items that are at least partially dependent on the
teacherÊs judgment. Generally, they are easier to construct compared to multiple-
choice questions because there is no need to develop distractors. Hence, they are
less time consuming compared to constructing multiple-choice questions.

4.3.1 Limitations of True-false Questions

However, true-false questions do have a number of limitations:
(a) Guessing ă A learner has a one in two chance of guessing the correct answer
of a question. Scores on true-false items tend to be high because of the ease
of guessing the correct answers when the answer is not known. With only
two choices (true or false) the learner could expect to guess correctly on
half of the items for which correct answers are not known. Thus, if a learner
knows the correct answers to 10 questions out of 20 and guesses on the other
10, the learner could expect a score of 15. The teacher can anticipate scores
ranging from approximately 50 per cent for a learner who did nothing but
guess on all items to 100 per cent for a learner who knows the material.
(b) Because these items are in the form of statements, there is sometimes a
tendency to take quotations from the text, expecting the learner to recognise
a correct quotation or note a change (sometimes minor) in wording. There
may also be a tendency to include trivial or inconsequential material from
the text. Both of these practices are discouraged.
(c) It can be difficult to write a statement which is unambiguously true or false,

particularly for complex materials
(d) True-false items provide little diagnostic information. Teachers can often get
useful information about learner errors and misconceptions by examining
learnersÊ incorrect answers but true-false items do not provide such
diagnostic information.
(e) True-false items may produce a negative suggestion effect. Some testing
experts feel that exposing false statements might promote learning false
information.
(f) False statements do not provide evidence that learners know the correct
answer.

4.3.2 Suggestions for Construction of True-false

Questions
The following are suggestions for construction of true-false questions:
(a) Include only one main idea in each question.
For example:
Poor item: The study of biology helps us understand living organisms

and predict the weather.
Better item: The study of biology helps us understand living organisms.
(b) Use negative statements sparingly but avoid double negatives. Double
negatives tend to contribute to the ambiguity of the statement. Statements
with words like none, no and not should be avoided as far as possible.
For example:
Poor item: None of the steps in the experiment were unnecessary.
Better item: All the steps in the experiment were necessary.
(c) Avoid broad, general statements. Most broad generalisations are false unless
qualified.
For example:
Poor item: Short-answer questions are more favourable than essay

questions in testing.
Better item: Short-answer questions are more favourable than essay

questions in testing factual information.
(d) Avoid long complex sentences. Such sentences also test reading
comprehension besides the achievements to be measured.

For example:
Poor item: Despite the theoretical and experimental difficulties of

determining the exact pH value of a solution, it is possible
to determine whether a solution is acidic by the red colour
formed on litmus paper when inserted into the solution.
Better item: Litmus paper turns red in an acidic solution.
(e) Try using in combination with other materials such as graphs, maps and
written material. This combination allows for the testing of more advanced
learning.
(f) Avoid lifting statements directly from assigned reading, notes or other
course materials so that recall alone will not permit a correct answer.
(g) In general, avoid the use of words which would signal the correct response
to the test-wise learner. Absolutes such as „none‰, „never‰, „always‰, „all‰
and „impossible‰ tend to be false while qualifiers such as „usually‰,
„generally‰, „sometimes‰ and „often‰ are likely to be true.
(h) A similar situation occurs with the use of "can" in a true-false statement. If
the learner knows of a single case in which something „can‰ be done, it
would be true.
(i) Ambiguous or vague statements and terms such as "large", "long time",
"regularly", "some" and "usually" are best avoided in the interest of clarity.
(j) Some terms have more than one meaning and may be interpreted differently
by individuals.
(k) True statements should be about the same length as false statements (there is
a tendency to add details in true statements to make them more precise).
(l) Word the statement so precisely that it can be judged unequivocally true or
false.
(m) Statements of opinion should be attributed to some source.
(n) Avoid verbal clues (specific determiners) that indicate the answer.
(o) Test important ideas rather than trivial information.
(p) Do not present items in easily learned pattern.

SELF-CHECK 4.2
1. What are some advantages of true-false questions?
2. Describe some limitations of true-false questions.
ACTIVITY 4.2
Suggest other advantages and weaknesses of using true-false questions.

forum.
4.4 MATCHING QUESTIONS

What are matching questions?
Matching questions are used in measuring a learnerÊs ability to identify the

relationship between two lists of terms, phrases, statements, definitions, dates,
events, people and so forth.
For example:
Directions: Column A contains statement describing selected Asian cities.
For each description find the appropriate city in Column B. Each city in Column
B can be used only once.
Column A Column B
1. The ancient capital of Thailand A. Ayuthia
2. The largest city in Sumatera B. Ho Chi Minh City
3. The capital of Myanmar C. Karachi
4. Formerly known as Saigon D. Medan

One matching question can replace several true-false questions. In developing

matching questions, you have to identify two columns of materials listed
vertically. The items in Column A (or I) are usually called premises and assigned
numbers (1, 2, 3, ⁄) while items in Column B (or II) are called responses and
designated capital letters (A, B, C, ....) Have more responses than premises or allow
responses to be used more than once.
The learner reads a premise (Column A) and finds the correct response from
among those in Column B. The learner then prints the letter of the correct response
in the blank beside the premise in Column A. An alternative is to have the learner
draw a line from the correct response to the premise but this is more time
consuming to score. One of the ways to reduce the possibility of guessing the
correct answers is to list a larger number of responses (Column B) than premises
(Column A) as is done in the example. Another way to decrease the possibility of
guessing is to allow responses to be used more than once. Instructions to the
learners should be very clear about the use of responses.
Some psychometricians suggest there should be no more than five to eight

premises (Column A) in one set. For each premise, the learner has to read through
the entire list of responses (or those still unused) to find the matching response.
For this reason, the shorter elements should be in Column B rather than Column
A to minimise the amount of reading needed for each item (Alabama Department
of Education, 2001). Responses (Column B) should be listed in logical order, if there
is one (for example, chronological order, by size). If there is no apparent order, the
responses should be listed alphabetically. Premises (Column A) should not be
listed in the same order as the responses. Care must be taken to ensure that the
association keyed as the correct response is unquestionably correct and that the
numbered item could not be rightly associated with any other choice.
4.4.1 Advantages of Matching Questions

Now, let us look at the advantages of matching questions:
(a) Matching questions are particularly good at assessing a learnerÊs

understanding of relationships. They can test recall by requiring a learner to
match the following elements (McBeath, 1992):
(i) Definitions ă terms;
(ii) Historical events ă dates;

(iii) Achievements ă people;
(iv) Statements ă postulates; and
(v) Descriptions ă principles.
(b) They can also assess a learnerÊs ability to apply knowledge by requiring a
test-taker to match the following:
(i) Examples ă terms;
(ii) Functions ă parts;
(iii) Classifications ă structures;
(iv) Applications ă postulates; and
(v) Problems ă principles.
(c) Matching questions are really a variation of the multiple-choice format. If

you find that you are writing multiple-choice questions which share the same
answer choices, you may consider grouping the questions into a matching
item.
(d) Matching questions are generally easy to write and score when the content
tested and objectives are suitable for matching question.
(e) Highly efficient as large amount of knowledge can be sampled in a short

period of time.
4.4.2 Limitations of Matching Questions

How about the limitations? Let us look at the following:
(a) Matching questions are limited to the materials that can be listed into two
columns and there may not be much material that lends itself to such a
format;
(b) If there are four items in a matching question and the learners know the
answers for three of them, the fourth item is a give-away through
elimination;

(c) Difficult to differentiate between effective and ineffective items;
(d) Often leads to testing of trivial facts or bits of information; and
(e) Often criticised for encouraging rote memorisation.
4.4.3 Suggestions for Writing Good Matching

Questions
Here are suggestions for writing good matching questions:
(a) Provide clear instructions. They should explain how many times responses
can be used;
(b) Keep the information in each column as homogeneous as possible;
(c) Include more responses than premises or allow the responses to be used
more than once;
(d) List the items with more words in Column A;
(e) Correct answers should not be obvious to those who do not know the content
being taught;
(f) There should not be keywords appearing in both a premise and response,
providing a clue to the correct answer; and
(g) All of the responses and premises for a matching item should appear on the
same page.
SELF-CHECK 4.3
1. What are some advantages of matching questions?
2. Describe some limitations of matching questions.

ACTIVITY 4.3
1. Suggest other advantages and weaknesses of using matching
questions.
2. Select five true-false questions in your subject area and analyse each
item using the guidelines discussed.
3. Select five matching questions in your subject area and analyse each
4. Suggest how you would improve the weak items for each type of
questions that you have identified.
Discuss your answers with your coursemates in the myINSPIRE online

forum.
4.5 MULTIPLE-CHOICE QUESTIONS

Multiple-choice questions or MCQs are widely used in many different settings
because it can be used to measure low level cognitive outcomes as well as more
complex cognitive outcomes. It is challenging to write test items that can tap into
higher-order thinking. Good item writing requires a well-trained test writer.
Above all, test writers need to have expertise in the subject area being tested so
that they can gauge the difficulty and content coverage of test items.
MCQs are the most difficult to prepare. These questions have two parts, namely a
stem which contains the question and four or five options which contains the
correct answer. The correct answer is called the keyed response and the incorrect
options are called distractors. The stem may be presented as a question, direction
or a statement while the options could be a word, phrase, numbers, symbols and
so forth. Cruel as it may seem, the role of the distractor is to attract the attention of
respondents who are not sure of the correct answer.

4.5.1 Parts of a Multiple-choice Question

A traditional multiple-choice question (MCQ) or item is one in which a learner
chooses one answer from a number of choices supplied (illustrated in Figure 3.4).
Figure 4.3: Example of a multiple-choice question (MCQ)
Now, let us look at what an MCQ consists of. It has the stem and also the options.
(a) Stem
The stem should:
(i) Be in the form of a question or a statement to be completed;
(ii) Be expressed clearly and concisely, avoiding poor grammar, complex

syntax, ambiguity and double negatives;
(iii) Generally present a positive question. (If a negative is used it should be

emphasised with italics or underlining);
(iv) Generally ask for one answer only (the correct or the best answer); and
(v) Include as many of the words common to all alternatives as possible.

(b) Options (or Alternatives)

The options should:
(i) Have either four or five alternatives, all of which should be mutually
exclusive and not too long;
(ii) Follow grammatically from the stem and be parallel in grammatical

form;
(iii) Be expressed simply enough to make clear the essential differences

between them and must be unambiguous; and
(iv) Contain the intended answer or the keyed response and it should
appear to be clearly correct to the informed but it should be definitely
incorrect but plausible to the distractors.
SELF-CHECK 4.4
1. Why is MCQ a popular form of objective test?
2. Describe some weaknesses of using MCQs.
4.5.2 Construction of Multiple-choice Questions

Test writing is a profession. By that we mean that good test writers are
professionally trained in designing test items. Test writers have knowledge of the
rules of constructing items and at the same time they have the creativity to
construct items that capture learnersÊ attention. Test items need to be succinct but
yet clear in meaning.
As stated earlier, MCQs are the most difficult to prepare. We need to focus on
writing the stem as well as providing the options or alternatives. All the options in
multiple-choice items need to be plausible but they also need to separate learners
of different ability levels. Table 4.1 shows some considerations that need to be
taken into account when constructing MCQs, particularly the stems.

Table 4.1: Considerations When Constructing MCQ Stems
Considerations When Example

Writing a Stem Poor Item Better Item
(a) Present a single, World War II was: In which of these time
definite statement A. The result of the failure of period was World War II
to be completed or the League of Nations. fought?
answered by one B. Horrible. A. 1914ă1917
of the several given B. 1929ă1934
C. Fought in Europe, Asia and
choices.
Africa. C. 1939ă1945
D. Fought during the period of D. 1951ă1955
1939ă1945.
Note: In the poor item, there is no sense of what the question is
asking from the stem. The better item more clearly identifies the
question and offers the learner a set of homogeneous choices.
(b) Avoid unnecessary For almost a century, the Rhine Which of the following
and irrelevant River has been used by would be the most dramatic
material. Europeans for a variety of result if, because of diesel
purposes. However, in recent pollution from ships, the
years, the increased river traffic Rhine River was closed to
has resulted in increased levels all shipping?
of diesel pollution in the A. Increased prices for
waterway. Which of the Ruhr products.
following would be the most B. Shortage of water for
dramatic result if, because of the Italian industries.
pollution, the Council of
C. Reduced
Ministers of the European
competitiveness of the
Community decided to close the
French Aerospace
Rhine River to all shipping?
Industry.
A. Closure of the busy Rhine
D. Closure of the busy
River ports of Rotterdam,
river Rhine ports of
Marseilles and Genoa.
Rotterdam, Marseilles
B. Increased prices for Ruhr and Genoa.
products.
C. Reduced competitiveness of
the French Aerospace
Industry.
D. Shortage of water for Italian
industries.
Note: The poor item was too wordy and contained unnecessary
material.
Source: McKenna and Bull (1999). Designing effective objective
questions. Loughborough University.

(c) Use clear, straight As the level of fertility A major decline in fertility
forward language approaches its nadir, what is the in a developing nation is
in the stem of the most likely ramification for the likely to produce
item. Questions citizenry of a developing nation? A. a decrease in the labour
that are A. A decrease in the labour force participation rate
constructed using force participation rate of of women.
complex wordings women. B. a downward trend in
may become a test B. A downward trend in the the youth dependency
of reading youth dependency ratio. ratio.
comprehension
C. A broader base in the C. a broader base in the
rather than an
population pyramid. population pyramid.
assessment of
whether the D. An increased infant D. an increased infant
learner knows the mortality rate. mortality rate.
subject matter. Note: In the improved item, the word „nadir‰ is replaced with
„decline‰ and „ramification‰ is replaced with „produce‰ which
are more straight forward words.
(d) Use negatives Which of the following is not a Which of the following is a
sparingly. If symptom of osteoporosis? symptom of osteoporosis?
negatives must be A. Decreased bone density. A. Hair loss.
used, capitalise, B. Frequent bone fractures. B. Painful joints.
underscore or
C. Raised body temperature. C. Raised body
bold.
D. Lower back pain. temperature.
D. Decreased bone density.
Note: The better item is stated in the positive so as to avoid use
of the negative „not‰.
(e) Put as much of the Theorists of pluralism have Theorists of pluralism have
question in the asserted which of the following? asserted that the
stem as possible, A. The maintenance of maintenance of democracy
rather than democracy requires a large requires
duplicating middle class. A. a large middle class.
material in each of B. The maintenance of B. the separation of
the options. democracy requires governmental powers.
autonomous centres of C. autonomous centres of
countervailing power. countervailing power.
C. The maintenance of D. the existence of a
democracy requires the multiplicity of religious
existence of a multiplicity of groups.
religious groups.
D. The maintenance of
democracy requires the
separation of governmental
powers.

Note: In the better item, the phrase „maintenance of

democracy‰ is included in the stem so as not to duplicate
material in each option.
Source: McKenna and Bull (1999). Designing effective objective
questions. Loughborough University.
(f) Avoid giving away A fertile area in the desert in A fertile area in the desert in
the answer because which the water table reaches which the water table reaches
of grammatical the ground surface is called the ground surface is called
cues. an a/an
A. mirage. A. lake.
B. oasis. B. mirage.
C. water hole. C. oasis.
D. polder. D. polder.
Note: The poor item uses the article „an‰ which identifies
choice C as the correct response. Ending the stem with „a/an‰
improves the item.
(g) As far as possible Which of the following men
avoid asking an contributed most towards the
opinion. defeat of HitlerÊs Germany in
World War II?
A. F. D. Roosevelt
B. George Patton
C. Josef Stalin
D. Winston Churchill
(h) Avoid using ALWAYS and NEVER in the stem as test-wise learners are likely to rule
such universal statements out of consideration.
ACTIVITY 4.4
1. Select ten MCQs in your subject area and analyse the stem of each
2. Suggest how you would improve the poor item.
Discuss your answers with your coursemates in the myINSPIRE online

forum.

Now, let us look at Table 4.2 which shows some considerations when constructing
the distractors for MCQs.
Table 4.2: Considerations When Constructing Distractors for MCQs
Considerations When Example

Writing Distractors Poor Item Better Item
(a) For single response What is the main source of What is the main source of
MCQs, ensure that pollution of Malaysian rivers? pollution of Malaysian
there is only one A. Land clearing rivers?
correct response. A. Open burning
B. Open burning
C. Solid waste dumping B. Coastal erosion
D. Coastal erosion C. Solid waste dumping
D. Carbon dioxide
emission
Note: In the poor item, both options A and C could be
considered to be correct.
(b) Use only plausible Who was the third Prime Who was the third Prime
and attractive Minister of Malaysia? Minister of Malaysia?
alternatives as A. Hussein Onn A. Hussein Onn
distractors.
B. Ghafar Baba B. Abdullah Badawi
C. Mahathir Mohamad C. Mahathir Mohamad
D. Musa Hitam D. Abdul Razak Hussein
Note: In the poor item, B and D are not serious distractors.
(c) When writing distractors, if possible, avoid the choices „All of the above‰ and „None
of the above‰. If you do include them, make sure that they appear as correct answers
some of the time.
It is tempting to resort to these alternatives but their use can be flawed. To begin with,
they often appear as an alternative that is not the correct response. If you do use them,
be sure that they constitute the correct answer some of the time. An „all of the above‰
alternative could be exploited by a test-wise learner who will recognise it as the correct
choice by identifying only two correct alternatives. Similarly, a learner who can
identify one wrong alternative can then also rule out this response. Clearly, the
learnerÊs chance of guessing the correct answer improves as they employ these
techniques. Although a similar process of elimination is not possible with „none of the
above‰, it is the case that when this option is used as the correct answer, the question
is only testing the learnersÊ ability to rule out wrong answers but this does not
guarantee that they know the correct one. (Gronlund, 1988).

(d) Distractors based on common learner errors or misconceptions are very effective.
One technique for compiling distractors is to ask learners to respond to open-ended
short-answer questions, perhaps as formative assessments. Identify which incorrect
responses appear most frequently and use them as distractors for a multiple-choice
version of the question.
(e) Do not create distractors that are so close to the correct answer that they may confuse
learners who really know the answer to the question. „Distractors should differ from
the key in a substantial way, not just in some minor nuances of phrasing or
emphasis.‰ (Isaacs, 1994)
(f) Provide a sufficient number of distractors.
You will probably choose to use three, four or five alternatives in a MCQ. Until
recently, it was thought that three or four distractors were necessary for the item
to be suitably difficult. However a study by Owen and Freeman suggested that
three choices are sufficient (Brown, 1987). Clearly the higher the number of
distractors, the less likely it is for the correct answer to be chosen through guessing
(provided all alternatives are of equal difficulty).
ACTIVITY 4.5
1. Do you agree that teachers should not use negatives in the stems of
MCQs? Justify your answer.
2. Do you agree that teachers should avoid using distractors such as

„All of the above‰ and „None of the above‰? Justify your answer.
3. Select ten MCQs in your subject area and analyse the distractors of
each item using the guidelines discussed. Suggest how you would
improve the weak items.
Share your answers with your coursemates in the myINSPIRE online
forum.
4.5.3 Advantages and Limitations of MCQs

MCQs are widely used to measure knowledge outcomes and various types of
learning outcomes. They are popular because of the following reasons:
(a) Learning outcomes from simple to complex can be measured;
(b) Highly structured and clear tasks are provided;

(c) A broad sample of achievement can be measured;
(d) Incorrect alternatives or options provide diagnostic information;
(e) Scores are less influenced by guessing than true-false items;
(f) Scores are more reliable than subjectively scored items (such as essays);
(g) Scoring is easy, objective and reliable;
(h) Item analysis can reveal how difficult each item was and how well it
discriminates between strong and weaker learners in the class;
(i) Performance can be compared from class to class and year to year;
(j) Can cover a lot of material very efficiently (about one item per minute of
testing time); and
(k) Items can be written so that learners must discriminate among options that
vary in degree of correctness.
While there are many advantages of using MCQs, there are also many limitations
in using such items, namely:
(a) Constructing good items is time consuming;
(b) It is frequently difficult to find plausible distractors;
(c) MCQs are not effective for measuring problem-solving skills as well as the
ability to organise and express ideas;
(d) Scores can be influenced by reading ability;
(e) There is a lack of feedback on individual thought processes. It is difficult to

determine why individual learners select incorrect responses;
(f) Learners can sometimes read more into the question than was intended;
(g) It often focuses on testing factual information and fails to test higher levels
of cognitive thinking;
(h) Sometimes there is more than one defensible „correct‰ answer;
(i) It places a high degree of independence on learnersÊ reading ability and the
constructorÊs writing ability;
(j) Does not provide a measure of writing ability; and
(k) May encourage guessing.

Now, let us look at Table 4.3 which summarises the procedural rules for the
construction of MCQs.
Table 4.3: Procedural Rules for Constructing Multiple-choice Questions (MCQs)
Some Procedural Rules for the Construction of Multiple-choice Questions
 Test for important or significant  Avoid giving clues through the use of
information. faulty grammatical construction.
 Avoid trick items.  Avoid the use of humour when
 Keep the vocabulary consistent developing options.
with the learnersÊ level of  Present practical or real-world
understanding. situations to learners.
 Avoid overly specific knowledge  Use pictorial materials that require
when constructing items. learners to apply principles and
 Avoid items based on opinions. concepts.
 Be sensitive to cultural, religious  Avoid textbook, verbatim phrasing
and gender issues. when developing items.
 Keep options or alternatives  Use charts, tables or figure that
independent and not overlapping. require interpretation.
 Avoid distractors that can provide
clue to test-wiseness.
4.5.4 Constructing MCQs to Measure Higher-order

Thinking (HOT)
Although it is not easy to construct higher-order thinking (HOT) MCQs, learners
can try the following techniques to write HOT questions:
(a) Scenario-based Questions

One of the best ways to promote and assess HOT skill is to use scenario-
based questions, particularly those that simulate real work experiences. A
scenario is a snapshot of an event, generally providing a brief overall
description of a problem that you need to solve or give advice on.
For example:
Read the following comment that a teacher made about testing and then
answer the question.
„Learners go to school to learn, not to take tests. In addition, tests cannot be

used to indicate a learnerÊs absolute learning. All tests can do is to rank
learners in order of achievement and this relative ranking is influenced by

guessing, bluffing and the subjective opinions of the teacher who is scoring.
The teaching and learning process would be beneficial if we do away with
tests and depend on learner self-evaluation.‰
Which of the following types of test is this teacher primarily talking about?
A. Aptitude test
B. Diagnostic test
C. Formative test
D. Summative test*
Adapted from:
https://www.learningsolutionsmag.com/articles/804/writing-
multiple-choice-questions-for-higher-level-thinking
(b) Interpret and Analyse Visuals

You can also assess critical thinking skills by asking learners to analyse or
interpret information from visuals, which are provided as part of the
question stem or the answer choices. In many cases, visuals such as job aids,
diagrams and graphs simulate workplace tasks.
For example:
Table 4.1: Changes in Weight of P#1 and P#2

Table 4.2: Changes in Glucose Level of P#1 and P#2
Two 60-year-old male patients (P#1 and P#2) have Type 2 diabetes. Each has
a BMI of 27. The primary treatment for each patient is a diet to reduce blood
glucose levels.
What is the most likely reason why P#2 did not show a decline in glucose
level after three months?
A. P#1 may have exercised more than P#2.
B. P#2 may have a more resistant form of diabetes.
C. P#2 probably leads a more sedentary life than P#1.*
D. P#1 lost more weight on the glucose reduction diet.
Adapted from:
http://digitalcommons.hsc.unt.edu/cgi/viewcontent.cgi?article=100
9&context=test_items
(c) Distractors and Key Context Reasons in Complex MCQs

Another approach to measuring critical or creative thinking is to ask learners
to synthesise what they have learned into an explanation. The possible
responses include the answer and a variety of reasons that support the
answer. Of course, only one reason is logical and correct, based on the
knowledge and skills being assessed. A context is also provided when this
technique is used.

For example:
Both Su and Ramya want to lose weight. Su goes on a low carbohydrate diet
while Ramya goes on a vegan diet. After six months, Su lost 30 pounds and
Ramya lost 15 pounds.
In relation to losing weight, which of the following conclusions is supported?
A. A low carbohydrate diet is more effective than a vegan diet.*
B. Additional information is needed before making any conclusions.
C. A low carbohydrate diet is easier to maintain than a vegan diet.
D. A vegan diet contains more calories than a low carbohydrate diet.
Adapted from:
http://digitalcommons.hsc.unt.edu/cgi/viewcontent.cgi?article=100
9&context=test_items
Nitko (2004) refers to this type of questions as context-dependent items. The

context is presented in some form of introductory material which may
include graphs, charts, drawings, formulae, extracts from reading materials,
maps and list of words. In crafting this type of questions, it is important
that learners are required to think about and use the information in the
introductory material to answer the questions to solve the problems or
complete the assessment tasks. The examples given previously refer to
multiple-choice items. However, context-dependent items may be used with
any type of item format such as essay, true-false and matching items.
SELF-CHECK 4.5
1. What are some advantages of MCQs?
2. List some limitations of MCQs.
ACTIVITY 4.6
Suggest other advantages and weaknesses of using MCQs.

forum.

 An objective test is a written test consisting of items or questions which

requires the respondent to select the answer from a list of possible answers.
 An objective item or question is „accurate‰ because it cannot be influenced by

the personal preferences and prejudices of the marker.
 Objective tests vary depending on how the questions are presented. The four
common types of questions used in most objective tests are short-answer
questions, matching questions, true-false questions and multiple-choice
questions (MCQs).
 The two forms of short-answer questions are direct questions and completion
questions.
 True-false questions are those in which a statement is presented and the learner
indicates in some manner whether the statement is true or false.
 True-false questions can be written quickly and are easy to score. Because they
can be objectively scored, the scores are more reliable than for items that are at
least partially dependent on the teacherÊs judgment.
 Avoid lifting statements directly from assigned reading, notes or other course
materials so that recall alone will not permit a correct answer.
 Matching questions are used to measure a learnerÊs ability to identify the

relationship between two lists of terms, phrases, statements, definitions, dates,
events, people and so forth.
 To reduce the possibility of guessing correct answers in matching questions,

list a larger number of responses than premises and allow responses to be used
more than once.
 MCQs have two parts: a stem that contains the question and four or five
options that contains the correct answer called the keyed response and
incorrect options called distractors.

 MCQs are widely used because they can be used to measure learning
outcomes, from simple to complex. They are highly structured with clear tasks
provided and able to test a broad sample of achievement.
 MCQs are difficult to construct, tend to measure low level learning outcomes,
lend themselves to guessing and do not measure writing ability.
 It is not impossible to construct higher-order thinking skills (HOTS) MCQs.
Alternatives Objective tests

Content coverage Options
Distractors Premises
Guessing Responses
Higher-order thinking skills (HOTS) Short-answer questions
Matching questions Stem
Multiple-choice questions (MCQs) True-false questions

Topic: Constructing Objective Test Items

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Topic: Constructing Objective Test Items

Uploaded by

Copyright:

Available Formats

Topic

Copyright © Open University Malaysia (OUM)

4.1 WHAT IS AN OBJECTIVE TEST?

What is an objective test? An objective test is a written test consisting of items or

Figure 4.1: Common objective test items

Copyright © Open University Malaysia (OUM)

4.2 SHORT-ANSWER QUESTIONS

Figure 4.2: Examples of short-answer questions

The short-answer question is generally used to measure simple learning outcomes.

Another advantage of the short-answer question is that the possibility of guessing

Copyright © Open University Malaysia (OUM)

One major limitation of the short-answer question is that it cannot be used to

Although the construction of short-answer questions is comparatively easier than

Poor item: An animal that eats the flesh of other animals is

(Possible answer: a wolf, a lion, hungry, ⁄)

(One specific answer: carnivorous)

Copyright © Open University Malaysia (OUM)

(b) Use direct questions instead of incomplete statements

Poor item: Columbus discovered America in ________.

Better item: In what year did Columbus discover America? (1492)

Poor item: Columbus discovered America in ________.

Better item: In what year did Columbus discover America? (1492)

Poor item: Columbus discovered America in ________.

Better item: In what year did Columbus discover America? (1492)

1. What are some advantages of short-answer questions?

2. Describe some limitations of short-answer questions.

Copyright © Open University Malaysia (OUM)

Suggest other advantages and weaknesses of using short-answer

Share your answer with your coursemates in the myINSPIRE online

4.3 TRUE-FALSE QUESTIONS

A whale is a mammal because it gives birth to its young.

Copyright © Open University Malaysia (OUM)

4.3.1 Limitations of True-false Questions

(c) It can be difficult to write a statement which is unambiguously true or false,

Copyright © Open University Malaysia (OUM)

4.3.2 Suggestions for Construction of True-false

(a) Include only one main idea in each question.

Poor item: The study of biology helps us understand living organisms

Better item: The study of biology helps us understand living organisms.

Poor item: None of the steps in the experiment were unnecessary.

Better item: All the steps in the experiment were necessary.

Poor item: Short-answer questions are more favourable than essay

Better item: Short-answer questions are more favourable than essay

Copyright © Open University Malaysia (OUM)

Poor item: Despite the theoretical and experimental difficulties of

Better item: Litmus paper turns red in an acidic solution.

(m) Statements of opinion should be attributed to some source.

(o) Test important ideas rather than trivial information.

(p) Do not present items in easily learned pattern.

Copyright © Open University Malaysia (OUM)

1. What are some advantages of true-false questions?

2. Describe some limitations of true-false questions.

Suggest other advantages and weaknesses of using true-false questions.

Share your answer with your coursemates in the myINSPIRE online

4.4 MATCHING QUESTIONS

Matching questions are used in measuring a learnerÊs ability to identify the

Copyright © Open University Malaysia (OUM)

One matching question can replace several true-false questions. In developing

Some psychometricians suggest there should be no more than five to eight

4.4.1 Advantages of Matching Questions

(a) Matching questions are particularly good at assessing a learnerÊs

(i) Definitions ă terms;

(ii) Historical events ă dates;