Ayesha Irfan

class no. 08
Subject: Measurement and Evaluation
Topic: The value of item analysis
procedure/process of item analysis
MPhil 2nd semester
contents
• Background
• Definition of item analysis
• Introduction of item analysis
• Purposes of item analysis
• Procedure/process of item analysis
• Item difficulty
• Item discrimination
• Inspection of distractors
• Item review
• Importance of item analysis
Background
Lets have a look on our understanding
about evaluation in Education first. shall
we?
What is teaching?
Teaching is basically a process of changing the behavior of students in desired directions.
 Fig.1.1 A simplified teaching model

Instructional Entering Instructional

Evaluation
objectives behavior procedures
My topic is concerned with the 4th stage

What is evaluation?
In educational setting, evaluation is defined as a process of
determining the extent to which educational objectives are achieved
by the students.
 Fig 1.2 Simple Evaluation
evaluation model

descriptive Value judgment

Quantitative
Qualitative
description
description
(Measurement)
 A wide verity of evaluative procedures is available for teachers.
Fig. 1.3 types of evaluation procedures

Evaluation procedures

Non-measurement
tools Measurement tools

interviews Psychodramas others Ranking

tests others
scales

Anecdotal
biographies Rating Socioma-
records
scales tric scales
What is a test?
A test is a systematic procedure for measuring a sample
of students’ behavior under specified conditions.
Educational testing procedure
• There are 7 major steps of educational testing procedure.
1. Planning the test
2. Constructing the test
3. Assembling, administrating and scoring the test
4. Analyzing the test items
5. Describing test result
6. Interpreting the test scores
7. Judging the quality of the test
Analyzing the test
item
What does it mean by “test item” ?
The questions, problems, and tasks in a Test
are called test items.
Item
analysis
• The analysis of the responses of students to each item on the
test is known as the item analysis.
• In an item analysis statistical methods are used to identify
any test items that are not working well.

• If an item is too easy, too difficult, failing to show a difference

between skilled and unskilled examinees, or even scored
incorrectly, an item analysis will reveal it.
The two most common statistics reported in an item analysis are:
• (1) the item difficulty, which is a measure of the proportion of
examinees who responded to an item correctly.

• (2)The item discrimination, which is a measure of how well the item

discriminates between examinees who are knowledgeable in the
content area and those who are not.
• An additional analysis that is often reported is the distractor analysis.
The distractor analysis provides a measure of how well each of the
incorrect options contributes to the quality of a multiple choice item.

• Once the item analysis information is available, an item review is

often conducted.
Purposes of item analysis
• Item analysis serves four important purposes.
1. It yields information that serves as a basis for improving the test
items.

2. It provides data for class discussion of the test.

3. It helps in diagnosing the strengths and weaknesses of the students.

4. It develops insight and skill for preparing better tests in future.

Item analysis raises and answers at least three
questions about each test items. These questions are:

1. How difficult is the item?

2. Does the item discriminate between the good (high-scoring) and

the poor (low-scoring) students?

3. How effective is each distractor in the item?(in the case of a MCQ’S

items only)
Procedure/process of item
analysis
1. Arrange the test papers in order from the highest score to the lowest
score.
2. Select the 27% or about one-fourth of the papers with the highest score
and call it the high scoring group.
3. Select the same number of papers with the lowest score and call it the
low-scoring group.
4. The middle group of paper is not needed in item analysis.
5. Calculation of the item difficulty.
6. Calculation of item discrimination.
7. Distractors inspection (MCQ’s)
8. Evaluation of items to be retain, revised or rejected.
For example: If we had 40 students in the class we would select 10
papers in the high-scoring group and 10 papers in the low-scoring
group.

pass= 1
High-scoring group Low-scoring group
Fail= 0
Item analysis working sheet
Item difficulty
• The difficulty of an item is define as the percentage of students who got
the item right.
• We can write the formula of item difficulty as:

• H= number of correct answers in the high group

• L=number of correct answers in the low group
• N=total number of students in both groups.
For example: item no.1
• Applying the formula we have
• p= H+L/N (100)
= 8/8 (100)
= 100%

Note; The answer would be reverse

difficulty value.. i.e. the item was
so easy that 100% students got right
answer. It was 0% difficult.
Item discrimination
• The discriminating power of an item is its ability to differentiate
between good ( high-scoring) and poor (low-scoring) students.
• We can write the formula of item discrimination as:
• D = H – L /N/2

• H=number of correct answers in the high group,

• L=number of correct answers in the low group.
• N=total number of students in both group.
Example of item no.1 (table 2)
• D= H – L/N/2
• D= 4 - 4/8/2
• D= 0/4
• D= 0 ( no discrimination)
Variation in index of item discrimination
• It may vary from +1.00 to 0.00 to –1.00
• +1.00
maximum index when all the high scoring student gets item right
and low scoring students gets wrong.
• Example: Item #5 of table 2
• D = 4 – 0/8/2
• D = 4/4
• D = +1
• -1.00
minimum size of index shows the wrong direction. All the HS get item wrong
and all LS get the item right.
For example: Item #6 of table #2
D= 0 -4/8/2
D = -4 /4
D = -1 ( shows all HSG get item
wrong and all LSG get
the item right.)
• 0.00
shows whether the both group get the item right/wrong.
Inspection of distractors

• Distractor analysis is an extension of item analysis, using techniques

that are similar to item difficulty and item discrimination.

• In distractor analysis, however, we are no longer interested in how

test takers select the correct answer, but how the distractors were
able to function effectively by drawing the test takers away from the
correct answer.
 For example
In the story, he was unhappy because…………
A. It rained all day
B. he was scolded
C. he hurt himself
D. the weather was hot
Let us assume that 100 students took the test. If we assume that A is the answer
and the item difficulty is 70%, then 70 students answered correctly. What about
the remaining 30 students and the effectiveness of the three distractors? If all 30
selected D, the distractors B and C are useless in their role as distractors. Similarly,
if 15 students selected D and another 15 selected B, then C is not an effective
distractor and should be replaced. The ideal situation would be for each of the
three distractors to be selected by 10 students.
DISTRACTORS in multiple choice test
by Inspectional method
• “Suspect” distractor when there is 5-6 score difference of high scoring
and lowest scoring group.

• “Danger signal” distractor when most responses are by H group than L

group.

• “Dead wood” distractor when there is no response from either

groups It should be improved urgently.
Item
review

• Once the item analysis data are available, it is useful to hold a

meeting of test developers, psychometricians, and subject matter
experts.
• During this meeting the items can be reviewed using the information
provided by the item analysis statistics.
• Decisions can then be made about item changes that are needed or
even items that ought to be dropped from the exam.
Importance of analysis of test
item

• Item Analysis is an important (probably the most important) tool to

increase test effectiveness.

• To write effective items, it is necessary to examine whether they are

measuring the fact, idea, or concept for which they were intended.

• It is a scientific way of improving the quality of tests.

Thankyo
u
References

Rizvi, A. (1973). Classroom testing in theory and practice.

Pakistan, Karachi: Tower book agency.
•
https://www.scribd.com/doc/288467672/Distractor-Analysis-Test-Ite
m-Analysis
• https://www.slideshare.net/energeticarun/item-analysis-16082143