Professional Documents
Culture Documents
Multiple Choice Item Construction: D. Michele Jacobsen Assistant Professor Dmjacobs@acs - Ucalgary.ca
Multiple Choice Item Construction: D. Michele Jacobsen Assistant Professor Dmjacobs@acs - Ucalgary.ca
Construction
D. Michele Jacobsen
Assistant Professor
dmjacobs@acs.ucalgary.ca
Test Development Process
1. Statement of Goals
2. Content Outline
3. Table of Specifications
4. Item Selection
5. Item Construction
6. Composition of Instructions
7. Development of Answer Sheets
8. Construction of Answer Keys
9. Test Administration
10. Test Revision
Today’s Focus
3. Table of Specifications
4. Item Selection
5. Item Construction
Two Important Characteristics of Tests
Reliability
– consistency
• free from extraneous sources of error
Validity
– how well a test measures what it is
supposed to measure
Formative vs. Summative Tests
Formative
– monitor progress toward goals within a
course of study
Summative
– assess overall achievement of course
goals
A. Table of Specifications
Blue print for test
Purpose
– ensure proper emphasis given to all
elements of a course of study
– content validity
Guide for writing items
Table of Specifications (2)
Start with Instructional Objectives
Levels Of Understanding
Parts
Function
Relation
to other
systems
Need to Consider and Decide:
Length of Test
Weight to be given to each objective
Weight to be given to each level of
taxonomy
Estimate number of items in each
cell
Biology 30: Circulation
Levels Of Understanding
Parts 10 5 0 15
Functio 5 5 5 15
n
Relation 0 5 10 15
to other
systems
15 15 15 45
B. Item Selection
Types of Items
– Objectively Scored (Selection)
• true/false
• completion
• matching
• multiple choice
– Subjectively Scored (Supply)
• interpretive exercises
• essay
C. Item Construction
Selected Response Test Items
– Item construction skills for valid and
reliable measures of student
achievement.
Guidelines which apply to all types
Specific Suggestions for writing each type
Advantages and Disadvantages of Each
Guidelines for Writing Objective Items
– Unacceptable:
• In King Lear, Regan ordered Gloucester’s eyes to be
plucked out and Gloucester died when he jumped off
the cliff of Dover.
– Acceptable:
• In King Lear, Regan ordered Gloucester’s eyes to be
plucked out. (T)
• In King Lear, Gloucester died when he jumped off
the cliff at Dover. (F)
True / False
Avoid using universal descriptors such as
“never”, “none”, “always”, and “all”.
– Testwise students will recognize that there are few
absolutes.
– a column of responses
Column I Column II
1. Name of the answer in addition A. Difference
problems. B. Dividend
2. Name of the answer in C. Multiplicand
subtraction problems. D. Product
3. Name of the answer in E. Quotient
multiplication problems. F. Subtrahend
4. Name of the answer in division G. Sum
problems.
Short Answer Test Items
Typically, the student is asked to reply with a
word, phrase, name, or sentence, rather than a
more extended response.
– Direct Questions / Short Answer
• Who is the current Prime Minister of Canada?
– Incomplete Sentences / Fill In the Blanks
• The current Prime Minister of Canada is _____?
– Consider:
In the year _____ , Prime Minister _________ signed the
__________ , which led to a ___________ which was
__________ .
Multiple Choice Items
Interpretive Exercise
Guidelines for Writing
Advantages & Disadvantages
Terminology: Multiple Choice
1 1. The capital city of Canada is
a. Vancouver
3 b. Montreal
4
c. Toronto
2 *d. Ottawa
Extrinsic Intrinsic
– Desirable quality in – Undesirable
multiple choice – Should be avoided
items – Inside the item
– Outside the item • poor wording
– Allows • more than one answer
discrimination
– Even those who know
between those who
know material and content have difficulty
those who do not. choosing correct answer
Types of Multiple Choice Items
Correct Answer*
– Only one correct response
Best Answer
– requires examinee to select alternative
closest to being correct
– fine distinctions
Multiple Answer
– More than one correct or best answer
Interpretive Exercise
Usually begins with verbal, tabular or graphic
information which is the basis for 1 or more
multiple choice questions.
– map, passage from a story, a poem, a cartoon
Can challenge students at various levels of
understanding
– application, analysis, synthesis, evaluation
Exercise contains all information needed to
answer questions
Readily adaptive to the more important
outcomes of disciplines.
Interpretive Exercises (con’t)
Examples
If student answers incorrectly it is because
they have not mastered the thinking or
reasoning required by the question, NOT
because they failed to memorize background
information.
Math questions: give students the formulas,
test ability to apply concepts, rather than
ability to memorize formulas.
Guidelines for Writing:
Multiple Choice Items
State stem in the form of a question.
– Weak
– Canada is
• a) a country
• b) where you live
• c) between the Pacific and Atlantic oceans
• d) all of the above
– Better
– Between which two oceans is Canada located?
• a) Atlantic and Indian
• b) Atlantic and Pacific
• c) Pacific and Indian
Guidelines for Writing: Stems
Place most of the subject matter in the Stem
– ensures full statement of problem
Eliminate extraneous material from the Stem
– goal is to measure student achievement, not to present
new material
– maximize use of time for demonstrating understanding,
not reading ability
Avoid Negatively phrased Stems
– students may miss the qualifier
– use only when learning outcome requires this type of
differentiation
I. Guidelines for Writing: MC
Ensure similarity among alternatives
with regard to:
– grammatical structure
– length
– mode of expression
Grammatical errors provide
unintentional clues to the answer
When in doubt, students will select the
longest alternative as the correct answer
Example: Length of Alternatives
a. be dangerous to society
b. have delusional symptoms
c. be dangerous to themselves
*d. have insight into their own inappropriate
behavior but nevertheless feel rather
helpless in terms of dealing with their
difficulties
II. Guidelines for Writing: MC
Make one of the alternatives the most clearly
correct or best answer
– exception: multiple answer form
– reduces intrinsic ambiguity
– reduces frustration during test