Reviewer

GUIDELINES IN WRITING TRUE-FALSE ITEMS

★ Desired method of marking true or false should be clearly explained before students begin
★ Conduct statements that are definitely true or definitely false, without additional qualifications
★ Use relatively short statements and eliminate extraneous material
★ Keep true and false statements at approximately the same length
★ Be sure that there are approximately equal numbers of true and false items
★ Avoid using double-negative statements
★ AVOID THE FOLLOWING:
○ Verbal cues, absolutes and incomplete sentences
○ Broad general statements that are usually not true or false without further qualifications
○ Terms denoting indefinite degree
○ Placing items in a systematic order
○ Taking statements directly from the text and presenting them out of context

GUIDELINES IN WRITING MATCHING ITEMS

★ Title the lists to ensure homogeneity
★ Arrange the descriptions and options in some logical order
★ Make sure that all the options are plausible distractors
★ List of descriptions should contain the longer phrases
★ Options should consist of short phrases
★ Each description in the list should be numbered
★ List of options should be identified by letter
★ Include more options than descriptions
★ Specify the basis for matching and whether options can be used more than once

GUIDELINES IN WRITING MULTIPLE-CHOICE ITEMS

★ Include as much of the item as possible, keeping the response options as short as possible
★ Be sure that there is one and only correct or clearly best answer
★ Be sure wrong answer choices are plausible
★ Use negative questions or statements only if the knowledge being tested requires it
★ Include from 3-5 options (2-4 distractors plus 1 correct answer)
★ To increase the difficulty of a multiple-choice item, increase the similarity of content among the
options
★ Use the option “none of the above” sparingly and only when the keyed answer can be classified
unequivocally as right or wrong
★ Avoid using “all of the above”

HIGHER LEVEL MULTIPLE-CHOICE QUESTIONS

★ Use pictorial, graphical, or tabular stimuli
★ Use analogies that demonstrate relationships among terms
★ Require the application of previously learned principles or procedures to novel situations

COMPLETION ITEMS
★ Items should require a single-word answer, or a brief and definite statement
★ Be sure the question poses a problem to the examinee
★ Be sure the answer is that the student is required to produce is factually correct
★ Be sure the language used in the questions is precise and accurate in relation to the subject
★ Omit only key words
★ Word the statement such that the blank is near the end of the sentence rather than near the
beginning
★ If the problem requires a numerical answer, indicate the units which it is to be expressed
WRITING ESSAY TEST ITEMS
★ An essay item is one for which the student applies, rather than selects, the correct answer
★ The student must compose a response to a question for which no single response can be cited as
correct to the exclusion of all answers
★ Essay items may be well constructed or poorly constructed
○ The well-constructed essay item aims to test complex cognitive skills by requiring the
student to organize, integrate, and synthesize knowledge
○ The poorly constructed essay item may require the student to do no more than recall
information as it was presented in the lecture or textbook
★ The potential of the essay item as an evaluation device depends not only upon writing appropriate
questions that elicit complex cognitive skills, but also upon being able to structure the situation

TWO TYPES OF ESSAY ITEMS

★ Extended Response Essay Item
○ An essay item that allows the student to determine the length and complexity of response
○ Most useful at the evaluation levels of the cognitive taxonomy
○ Because of the length of this type, it is sometimes better as a term paper assignment or a
take-home test
○ Often better for assessing communication ability than for assessing achievement
★ Restricted Response Essay Item
○ An essay item that poses a specific problem for which the student must recall proper
information, organize it in a suitable manner, derive a defensible conclusion, and express it
within the limits of the posed problem
○ The statement of the problem specifies response limitations that guide the student in
responding and provide evaluation criteria for scoring

ITEM ANALYSIS TERMINOLOGIES

★ Item Analysis - used to identify items that are deficient in some ways
★ Quantitative Item Analysis - a numerical method for analyzing test items employing student
response alternative or options
★ Qualitative Item Analysis - a non-numerical method for analyzing test items not employing student
responses
★ Key - corrector option in a multiple-choice item
★ Distractor - incorrect option in a multiple-choice item
★ Difficulty Index (p) - proportion of students who answered the item correctly
★ Discrimination Index (D) - measure of the extent to which a test item discriminates or
differentiates between students who do well on the overall test and those who do not

3 TYPES OF DISCRIMINATION INDEXES

★ Positive discrimination index - those who did well on the overall test chose the correct answer for
a particular item more often than those who did well on the overall test
★ Negative discrimination index - those who did poorly on the overall test chose the correct answer
for a particular item more often than those who did well on the overall test
★ Zero discrimination index - those who did well and those who did poorly on the overall test chose
the correct answer for a particular item with equal frequency

FORMULA
★ p = Number of students selecting correct answer
Total number of students attempting the item

★ D = (Numbers who got item correct in upper group) - (Number who got item correct in lower group)
Number of students in either group (if group sizes are unequal, choose the higher number)
★ When p levels are less than about .25, the item is considered relatively difficult
★ When p levels are above .75, the item is considered relatively easy
★ Test construction experts try to build tests that have most items between p levels of .20 and .80,
with an average p level of about .50
QUANTITATIVE ITEM ANALYSIS
 ★ Quantitative item analysis also enables us to make other decisions. We can use quantitative item
analysis to decide whether an item is miskeyed, whether responses to the item are characterized
by guessing, or whether the item is ambiguous
★ Miskeying - most students who did well on the test will likely select an option that is a distractor,
rather than the option that is keyed
★ Guessing - students in the upper half of the the class respond in more or less random fashion
★ Ambiguous - among the upper group, one of the distractors is chosen with about the same
frequency as the correct answer
★ Guessing Formula
Score = Total Right - Total Wrong .
No. of answer choices - 1

DEBRIEFING
Before handing back answer sheets or grades:
1. Discuss problem items
2. Listen to student reactions
3. Avoid on-the-spot decisions
4. Be equitable with changes
After handing back answer sheets or grades:
1. Ask students to double-check
2. Ask students to identify problems
In trying to remain non defensively and emotionally detached from your test, it is useful to keep a few
points in mind:
1. Your test will include at least some items that can be improved
2. Students are criticizing your skill as a test constructor, not you as a person
3. The quality of an item is not necessarily related to the loudness with which complaints are made.
At times, items that students loudly protest are indeed poor items