The Process of Becoming an

Expert Tester
 Assembly
 - Collecting or developing test items
 - Reviewing test items
 - Formatting the test
 - Preparing directions for the test
 Administration and scoring
 Analysis
Analysis:Using it to Improve Test Items
 Item Difficulty

 Item Discrimination

 Distractor Analysis
 Item Difficulty
 Refers to the percentage of students who
answered the item correctly. Can also be
expressed as a proportion

 Example: if 15 out of 25 students correctly

answered the question, the item difficulty
will be 15/25=60%
 Item Discrimination
 Ability of an item to distinguish between
more and less knowledgeable students.

 A common method to compute it is to take

two subgroups upper scoring 25% and lower
scoring 25% and subtract the number of
students who got it right in those two
subgroups, then divide by the total number
of students who took the test.
 Item Discrimination Ctd.
 Discrimination index ranges from -1 to +1.
 - .40 and up Very good item
 - .30 to .39 Good item
 - .20 to .29 Reasonably good item
 - .10 to .19 Marginal item, usually
subject to improvement
 - Below .10 Poor item, to be rejected or
revised
 A positive index of discrimination suggests
that getting that particular item correct
correlates positively with higher total test
scores.

 A negative index suggests that students

who get that item generally score lower on
the test.
 Distractor Analysis
 Distractors should be worded so that you
not only know which students are missing
which items but you can also make a guess
as to why.

 Distractor analysis is the third component

of an item analysis.
 Distractor Analysis Ctd.
 Example:
A *B C D E Omit
Upper 7 86 0 7 0 0
Lower 13 60 7 13 7 0
All 12 70 5 10 3 0

* Correct answer.
 All students answered the item (0 omitted).
 Item performing satisfactorily: higher
scoring students should answer correctly.
That’s the case with 86% upper and 60%
lower
 Conversely, expect a smaller number of
students in upper to select wrong answers.
This has happened.
 If this does not happen, review the item to
see why a greater percentage of the more
versus less capable students are selecting
the particular incorrect option.

 If a distractor is not selected at all, you

might want to delete it.
 If a distractor does not distinguish between
the two groups, i.e. the same percentage,
the instructor might want to determine why
it performed inappropriately.
 To avoid responding to an item provides
the instructor with no information as to the
misconceptions students are having
encourage them to attempt all questions.
 Recognize that number values associated
with an item analysis can be unstable,
especially when based on less than 50
students.
 Detecting ambiguities
 If discrimination is high, indicates students
are having difficulty with item or material
tested.
 Low discrimination does not distinguish.
 Low discrimination combined with
moderate difficulty means that the test item
or instruction should be revised.
 Measurement specialists commonly
discourage the inclusion of very easy or
very difficult items on the test.

 Students input should be used to interpret

an item analysis, very sensitive to wording
and explanation in class that confused
classmates.