Item Analysis

Item Analysis is the group of statistics that can be computed for each item on a test.

Purpose of Item Analysis

It increases our understanding of tests, especially as it relates to reliability and

validity. Poorly constructed test items affect the test’s ability to really measure
what it should and to measure individual differences among test takers.

Three measures used to analyze test items

 Distractor Analysis
 How many persons chose each distractor?
 Were the distractors effective?
 Examine the total pattern of responses to each item

 Item Difficulty
 How many persons answered the item correctly?

 Item Discrimination
 To what extent does the item discriminate between these who are
knowledgeable about the content and those who are not? i.e.
Discrimination between those who do well and those who do poorly on
the test.

Distractor Analysis

 Based on the number of persons who chose each incorrect response or distractor
on a multiple choice item.
 Those who know will choose correct answer.
 Those who don’t know should choose equally among distractors if all
distractors are equally effective.
 No. expected to choose each distractor: no. who answered incorrectly
no. of distractors

 Helps identify the distractors that are not effectively distracting

 When one or more distractors are not functioning effectively, this narrows
down guessing probability to 1 out of 3, or 1 out of 2, rather than 1 out of 4.
 This also lowers difficulty and affects reliability – internal consistency

Item Difficulty

This is the proportion of persons who are able to get the item right. A test item
should not be so easy that everyone gets it right, neither should it be so difficult
that no one chooses the correct response.

Simplest method of calculating difficulty:

P = no. who answered correctly

no. of persons taking the test

This gives the item difficulty index, which is a value ranging between 0 and 1.

0 – Very difficult (no one got it right) 1.00 – Very easy (everyone got it
right)

An item difficulty of 0 or 1.00 means that the test does not discriminate.

Ideally, an item should have an item difficulty index of about 0.5.

Item Discrimination

If an item is discriminating well, more persons in the top group/high performing

group should answer it correctly than those in the low performing group. The
rationale is that if a test item is good at discriminating, students who were well
prepared overall should do better at the item than those who were generally not
well prepared.

Item discrimination is simply the difference between the percentage of persons in

the high performing group who got the item right and the percentage in the low
performing group who got it right.

D = Up - Lp
U- L
Because U and L are the same, the formula changes to:

D = Up-Lp
U

Up – No. of persons in the upper 27% or 33% (the high performing group) who got
the item correct.
Lp – Persons in the lower 27% or 33% (the low performing group) who got the
item correct
U – Total number in the high performing group
L – Total number in the low performing group

NB. Note that you may choose any percentage within the range of 27% to 33% to
determine how many test-takers will comprise each group

The item discrimination index is a value ranging between 0 and 1. 0 means that the
item does not discriminate at all (the upper and lower performing group did equally
well on the item), while 1 means that it discriminates extremely well (everyone in
the high performing group got the item right, while no one in the low group got it
right). The higher the value of D, the more effective is the item at discriminating
between high and low scorers/performers.

An item is usually considered acceptable if the D value is .30 or higher.

Conclusion
Those items that do not have good difficulty or discriminatory power can be
discarded or reconstructed.