CHAPTER 4

ADMINISTERING,
ANALYZING, AND
IMPROVING TEST
KEY TERMS
KEY TERMS
•Item analysis
•Difficulty index
•Discrimination index
•Measures of attractiveness
•Positive discrimination
KEY TERMS

•Negative discrimination
•Zero discrimination
•Miskeyed item
•Guessing item
•Ambiguous item
LEARNING OBJECTIVES
LEARNING OBJECTIVES
At the end of the chapter, the students should be able
to:
1. Define the basic concepts regarding item and
analysis;
2. Identify the steps in improving test items;
3. Solve difficulty index and discrimination index;
LEARNING OBJECTIVES

4. Identify the level of difficulty of an item;

5. Perform item analysis properly and correctly;
6. Identify the item to be rejected, revised, or
retained; and
7. Interpret the results of item analysis
INTRODUCTION
INTRODUCTION
One of the most important functions of a
teacher is to assess the performance of the
students. This is a very complicated task because
you will consider many activities such aa the
timing of the assessment process, the format of
the assessment tools and the duration of the
assessment procedures.
INTRODUCTION
After designing the assessment tools,
package the test, administer the test to the
students, check the test papers, score and
then the record them. Return the test papers
and the give feedback to the students
regarding the result of the test.
PACKAGING AND
REPRODUCING TEST ITEMS
PACKAGING AND REPRODUCING TEST ITEM
Assuming that you have already assembled
the test, you write the instructional objectives,
prepare the table of specification, and write the
test items that match with the instructional
objectives, the next thing to do is to package the
test and reproduce it as discussed in the previous
chapter.
PACKAGING AND REPRODUCING TEST
ITEMS
1. Put the items the same format together.
2. Arrange the test items from easy to difficult.
3. Give proper spacing for each item for easy
reading.
4. Keep questions and options in the same page.
5. Place the illustrations near the options.
PACKAGING AND REPRODUCING TEST
ITEMS
6. Check the key answer.
7. Check the direction of the test.
8. Provide space for name, date and score.
9. Proofread the test.
10. Reproduce the test.
ADMINISTERING THE
EXAMINATION
ADMINISTERING THE EXAMINATION
After constructing the test items and putting them in
order, the next step is to administer the test to the students.
The administration procedures greatly affect the
performance of the students in the test. The test
administration does not simply mean giving the test question
to the students and collecting the test papers after the given
time. Below are the guidelines in administering the test
before, during and after the test.
GUIDELINES BEFORE
ADMINISTERING
EXAMINATIONS
GUIDELINES BEFORE ADMINISTERING
EXAMINATIONS
1. Try to include positive test-taking attitude.
2. Inform the students about the purpose of the test.
3. Give oral directions as early as possible before
distributing the tests.
4. Give test-taking hints about guessing, skipping, and
the like, are strictly prohibited.
GUIDELINES BEFORE ADMINISTERING
EXAMINATIONS
5. Inform the students about the length of time allowed for
the test. If possible, write on the board the time in which they
must be finished with answering the test. Give the students a
warning before the end of the time limit.
6. Tell the students how to signal or call your attention if
they have a question.
7. Tell the students what to do with their papers when they
are done answering the test (how papers are to be collected).
GUIDELINES BEFORE ADMINISTERING
EXAMINATIONS
8. Tell the students what to do when they are done
with the test, particularly if they are to go on to
another activity (also write these directions on the
chalkboard so hey can refer to them).
9. Rotate the method of distributing papers so you
don’t always start from the left or the front row.
GUIDELINES BEFORE ADMINISTERING
EXAMINATIONS
10. Make sure the room is well lighted and has a
comfortable temperature.
11. Remind students to put their names on their papers
(and where to do so).
12. If the test has more than one page, have each
student checked to see that all pages are there.
GUIDELINES DURING
EXAMINATION
GUIDELINES DURING EXAMINATION
1. Do not give instructions or avoid talking while
examination is going on to minimize interruptions and
distractions.
2. Avoid giving hints.
3. Monitor to check student progress and discourage
cheating.
4. Give time warnings if students are not pacing their work
appropriately.
GUIDELINES DURING EXAMINATION

5. Make a note of any questions students ask

during the test so that items can be revised for
future use.
6. Test papers must be collected uniformly to
save time and to avoid test papers to be
misplaced.
GUIDELINES AFTER THE
EXAMINATION
GUIDELINES AFTER THE EXAMINATION

After the examination, the next activity that

the teacher needs to do is to score the test
papers, record the result of the examination,
return the test papers and last to discuss the test
items in the class so that you can analyze and
improve the test items for future use.
GUIDELINES AFTER THE EXAMINATION
1. Grade the papers (and add comments if you can);
do test analysis (see the module on test analysis) after
scoring and before returning papers to students if at all
possible. If it is impossible to do your test analysis
before returning the papers, be sure to do it another
time. It is important to do both the evaluation of your
students and the improvement of your tests.
GUIDELINES AFTER THE EXAMINATION
2. If you are recording grades or scores, record them
in pencil in your class record before returning the
papers. If there are errors/adjustments in grading
they (grades) are easier to change when recorded in
pencil.
3. Return papers in timely manner.
GUIDELINES AFTER THE EXAMINATION
4. Discuss test items with the students. If students
have questions, agree to look over their papers again,
as well as the papers of others who have the same
question. It is usually better not to agree to make
changes in grades on the spur of the moment while
discussing the tests with the students but to give
yourself time to consider what action you want to
GUIDELINES AFTER THE EXAMINATION
The test analysis may have already alerted you to a
problem with a particular question that is common to
several students, and you may already have made a
decision regarding that question (to disregard the
question and reduce the highest possible score
accordingly, to give all students credit for that
question, among others).
ANALYZING THE TEST
ANALYZING THE TEST
After administering and scoring the test, the teacher
should also analyze the quality of each item in the test.
Through this you can identify the item that is good, item that
needs improvement or items to be removed from the test.
But when do we consider that the test is good? How do we
evaluate the quality of each item in the test? Why is it
necessary to evaluate each item in the test?
ANALYZING THE TEST
Lewis Aiken (1997) an author of psychological
and educational measurement pointed out that a
“postmortem” is just as necessary in classroom
assessment as it is in medicine.
ANALYZING THE TEST
In this section, we shall introduce the technique to
help teachers determine the quality of a test item
known as item analysis. One of the purposes of item
analysis is to improve the quality of the assessment
tools. Through this process, we can identify the item
that is to be retained, revised or rejected and also the
content of the lesson that is mastered or not.
ANALYZING THE TEST
There are two kinds of item analysis,
quantitative item analysis and qualitative
item analysis (Kubiszyn and Borich, 2007).
ITEM ANALYSIS
ITEM ANALYSIS
Item analysis is a process of examining the student’s
response to individual item in the test. It consists of
different procedures for assessing the quality of the
test items given to the students. Through the use of
item analysis we can identify which of the given are
good and defective test items. Good items are to be
retained and defective items are to be improved, to be
revised or to be rejected.
ITEM ANALYSIS
Uses Of Item Analysis:
1. Item analysis data provide a basis for efficient class
discussion of the test results.
2. Item analysis data provide a basis for remedial
work.
3. Item analysis data provide a basis for general
improvement of classroom instruction.
ITEM ANALYSIS

4. Item analysis data provide a basis for

increased skills in test construction.
5. Item analysis procedures provide a basis for
constructing test bank.
ITEM ANALYSIS

Types of Quantitative Item Analysis:

There are three common types of quantitative item
analysis which provide teachers with three different
types of information about individual test items.
These are difficulty index, discrimination index, and
response options analysis.
ITEM ANALYSIS

1. Difficulty Index
It refers to the proportion of the number of students
in the upper and lower groups who answered an item
correctly. The larger the proportion, the more
students, who have learned the subject is measured by
the item. To compute the difficulty index of an item,
use the formula:
ITEM ANALYSIS

= , where
= difficulty index
n = number of the students selecting item correctly in
the upper group and in the lower group
N = total number of students who answered the test
ITEM ANALYSIS

Level of Difficulty
To determine the level of difficulty of an item,
find first the difficulty index using the formula
and identify the level of difficulty using the
range given below.
ITEM ANALYSIS

Level of Difficulty of an Item

Index Range Difficulty Level
0.00-0.20 Very Difficult
0.21-0.40 Difficult
0.41-0.60 Average/Moderately
Difficult
0.61-0.80 Easy
0.81-1.00 Very Easy
ITEM ANALYSIS

The higher the value of the index of

difficulty, the easier the item is. Hence,
more students got the correct answer and
more students mastered the content
measured by that item.
ITEM ANALYSIS

2. Discrimination Index
The power of the item to discriminate the students
between those who scored high and those who scored
low in the overall test. In other words, it is the power
of the item to discriminate the students who know the
lesson and those who do not know the lesson.
ITEM ANALYSIS

It is also refers to the number of students in the

upper group who got an item correctly minus the
number of students in the lower group who got an
item correctly. Divide the difference by either the
number of the students in the upper group or number
of students in the lower group or get the higher
number if they are not equal.
ITEM ANALYSIS

Discrimination index is the basis of measuring

the validity of an item. This index can be
interpreted as an indication of the extent to
which overall knowledge of the content area or
mastery of the skills is related to the response on
an item.
ITEM ANALYSIS

Types of Discrimination Index

There are three kinds of discrimination
index: positive discrimination, negative
discrimination and zero discrimination.
ITEM ANALYSIS

1.Positive discrimination
Happens when more students in the upper
group got the item correctly than those students
in the lower group.
ITEM ANALYSIS

2. Negative discrimination
occurs when more students in the lower
group got the item correctly than the
students in the upper group.
ITEM ANALYSIS

3. Zero discrimination
happens when a number of students in the upper
group and lower group who answer the test correctly
are equal, hence, the test item cannot distinguish the
students who performed in the overall test and the
students whose performance are very poor.
ITEM ANALYSIS

Level of Discrimination
Ebel and Frisbie (1986) as cited by Hetzel
(1997) recommended the use of Level of
Discrimination of an item for easier
interpretation.
ITEM ANALYSIS

Index Range Discrimination Level

0.19 and below Poor item, should be eliminated or need to be

revised
0.20-0.29 Marginal item, needs some revision

0.30-0.39 Reasonably good item but possibly for

improvement
0.40 and above Very good item
ITEM ANALYSIS
Discrimination Index Formula
= , where
= discrimination index value
= number of the students selecting the correct answer in the upper
group
= number of the students selecting the correct answer in the lower
group
D= number of students in either the lower group or upper group
ITEM ANALYSIS

Note:
Consider the higher number in case the sizes in
upper and lower group are not equal.
ITEM ANALYSIS

Steps in Solving Difficulty Index and Discrimination

Index:
1. Arrange the scores from highest to lowest.
2. Separate the scores into upper group and lower group.
There are different methods to do this: (a) if a class
consists 30 students who takes an exam, arrange their
scores from highest to lowest, then divide them into two
ITEM ANALYSIS

The highest score is belongs to the upper group. The lowest

score belongs to the lower group. (b) other literatures
suggested to use 27%, 30%, or 33% of the students for the
upper group and lower group. However, in the Licensure
Examination for Teachers (LET) the test developers always
used 27% of the students who participated in the
examination for the upper and lower groups.
ITEM ANALYSIS

3. Count the number of those who chose the alternatives in the upper
and lower group of each item and record the information using the
template: Options A B C D E
Upper Group

Lower Group

Note: Put asterisk for the correct answer.

ITEM ANALYSIS

4. Compute the value of the difficulty index and the

discrimination index and also the analysis of each
response in the distracters.
5. Make an analysis for each item.
ITEM ANALYSIS

Checklist for Discrimination Index:

It is very important to determine whether the test
item will be retained, revised or rejected. Using the
discrimination index we can identify the non
performing question items; just always remember
that they seldom indicate what is the problem. Use the
checklist below:
ITEM ANALYSIS

Yes No

1. Does the key discriminate positively?

2. Does the incorrect options discriminate negatively?

If the answers to questions 1 and 2 are both YES, retain the item.
If the answers to questions 1 and 2 are either YES or NO, revise the item.
If the answers to questions 1 and 2 are both NO, eliminate or reject the
item.
ITEM ANALYSIS

3. Analysis of Response Options

Aside from identifying the difficulty index and
discrimination index, another way to evaluate the
performance of the entire test item is through the
analysis of the response options. It is very important
to examine the performance of each option in a
multiple-choice item.
ITEM ANALYSIS

Through this, you can determine whether the distracters or

incorrect options are effective or attractive to those who do
not know the correct answer. The attractiveness of the
incorrect options is determined when more students in the
lower group than in the upper group choose it. Analyzing the
incorrect options allows the teachers to improve the test
items so that it can be used again in the future.
DISTRACTER ANALYSIS
DISTRACTER ANALYSIS

1. Distracter
Distracter is the term used for the incorrect options in the
Multiple-choice test while the correct answer represents the
key. It is very important for the test writer to know if the
distracters are effective or good distracters. Using
quantitative item analysis we can determine if the options
are good or if the distracters are effective.
DISTRACTER ANALYSIS

Item analysis can identify nonperforming test

items, but this item seldom indicates the error or
the problem in the given item. There are factors
to be considered why students failed to get the
correct answer in the given question.
DISTRACTER ANALYSIS

a.It is not taught in the class properly.

b.It is ambiguous.
c.The correct answer is not in the given options.
d.It has more than one correct answer.
DISTRACTER ANALYSIS

e. It contains grammatical clues to mislead the

students.
f. The student is not aware of the content.
g. The students were confused by the logic of the
question because it has double negatives.
h. The student failed to study the lesson.
DISTRACTER ANALYSIS

2. Miskeyed Item
The test item is a potential miskey if there are
more students from the upper group who choose
the incorrect options than the key.
DISTRACTER ANALYSIS

3. Guessing Item
Students from the upper group have equal
spread of choices among the given alternatives.
Students from the upper group guess their
answers because of the following reasons:
DISTRACTER ANALYSIS

a.The content of the test is not discussed in the

class or in the test.
b.The test item is very difficult.
c.The question is trivial.
DISTRACTER ANALYSIS

4. Ambiguous Item
This happens when more students from the
upper group choose equally an incorrect option
and the keyed answer.
QUALITATIVE ITEM
ANALYSIS
QUALITATIVE ITEM ANALYSIS
Qualitative item analysis (Zurawski, R. M) is a process in
which the teacher or expert carefully proofreads the test
before it is administered, to check if there are typographical
errors, to avoid grammatical clues that may lead to giving
away the correct answer, and to ensure that the level of
reading materials is appropriate. These procedures can also
include small group discussions on the quality of the
examination and its items, with examinees that have already
took the test
QUALITATIVE ITEM ANALYSIS

According to Cohen, Swerdlik, and Smith (1992) as cited

by Zurawski, students who took the examination are asked to
express verbally their experience in answering each item in
the examination. This procedure can help the teacher in
determining whether the test takers misunderstood a certain
item, and it can help also in determining why they
misunderstood a certain item.
IMPROVING TEST ITEMS
IMPROVING TEST ITEMS

As presented in the introduction of this

chapter, item analysis enable the teachers to
improve and enhance their skills in writing test
items. To improve multiple-choice test item we
shall consider the stem we shall consider the
stem, the distracters and the key answer.
IMPROVING TEST ITEMS

How to Improve the Test Item:

Consider the following examples in analyzing
the test item and some notes on how to improve
the item based from the results of item analysis.
IMPROVING TEST ITEMS

Example 1. A class is composed of 40

students. Divide the group into two. Option B.
is the correct answer. Based from the given
data on the table, as a teacher, what would you
do with the test item?
IMPROVING TEST ITEMS

Options A B* C D E

Upper Group 3 10 4 0 3

Lower Group 4 4 8 0 4
IMPROVING TEST ITEMS

1. Compute the difficulty index.

n=10+4=14
N=40
=
=
=0.35 or 35%
IMPROVING TEST ITEMS

2. Compute the discrimination index.

=10 =
=4 =
D=20 = 0.30 or 30%
=-
D
IMPROVING TEST ITEMS

3. Make an analysis about the level of difficulty,

discrimination and distracters.
a. Only 35% of the examinees got the answer
correctly, hence, the item is difficult.
b. More students from the upper group got the answer
correctly, hence, it has a positive discrimination.
IMPROVING TEST ITEMS

c. Retain options A, C, and E because most of

the students who did not perform well in the
overall examination selected it. Those options
attract most students from the lower group.
IMPROVING TEST ITEMS

4. Conclusion: retain the test item but

change option D, make it more realistic to
make it effective for the upper and lower
groups. At least 5% of the examinees
choose the incorrect option.
IMPROVING TEST ITEMS

Example 2. A class is composed of 50

students. Use 27% to get the upper and the lower
groups. Analyze the item given the following
results. Option D is the correct answer. What
will you do with the test item?
IMPROVING TEST ITEMS

Options A B C D* E
Upper Group (27%) 3 1 2 6 2
Lower Group (27%) 5 0 4 4 1
IMPROVING TEST ITEMS

1. Compute the difficulty index.

n=6+4=10
N=28
=
=
=0.36 or 36%
IMPROVING TEST ITEMS

2. Compute the discrimination index.

=6 =
=4 =
D=4 = 0.14 or 14%
=-
D
IMPROVING TEST ITEMS

3. Make an analysis.
a. Only 36% of the examinees got the answer
correctly, hence, the item is difficult.
b. More students from the upper group got the answer
correctly, hence, it has a positive discrimination.
IMPROVING TEST ITEMS

c. Modify options B and E because more

students from the upper group chose them
compare with the lower group, hence, they are
not effective distracters because most of the
students who performed well in the overall
examination selected them as their answers.
IMPROVING TEST ITEMS

d.Retain options A and C because most of

the students who did not perform well in the
overall examination selected them as the
correct answers. Hence, options A and C are
effective distracters.
IMPROVING TEST ITEMS

4. Conclusion: Revised the item by modifying

options B and E.
Example 3. A class is composed of 50 students.
Use 27% to get the upper and the lower groups.
Analyze the item given the following results. Options
E is the correct answer. What will you do with the test
item?
IMPROVING TEST ITEMS

Options A B C D E*
Upper Group (27%) 2 3 2 2 5
Lower Group (27%) 2 2 1 1 8
IMPROVING TEST ITEMS

1. Compute the difficulty index:

n=5+8=13
N=28
=
=
=0.46 or 46%
IMPROVING TEST ITEMS

2. Compute the discrimination index:

=5 =
=8 =
D=4 = 0.21 or -21%
=-
D
IMPROVING TEST ITEMS

3. Make an analysis.
a. 46% of the students got the answer to test item
correctly, hence. The test item is moderately difficult.
b. More students from the lower group got the item
correctly, therefore, it is a negative discrimination.
The discrimination index is -21%.
IMPROVING TEST ITEMS

c. No need to analyze the distracters because the item

discriminates negatively.
d. Modify all the distracters because they are not
effective. Most of the students in the upper group
chose the incorrect options. The options are
effective if most of the students in the lower group
chose the incorrect options.
IMPROVING TEST ITEMS

4.Conclusion: Reject the item because it has a

negative discrimination index.
Example 4. Potential Miskeyed Item. Make
an item analysis about the table below. What
will you do with the test that is potential
miskeyed item?
IMPROVING TEST ITEMS

Options A* B C D E
Upper Group 1 2 3 10 4
Lower Group 3 4 4 4 5
IMPROVING TEST ITEMS

1. Compute the difficulty index:

n=1+3=4
N=40
=
=
=0.10 or 10%
IMPROVING TEST ITEMS

2. Compute the discrimination index:

=1 =
=3 =
D=20 = 0.10 or -10%
=-
D
IMPROVING TEST ITEMS

3. Make an analysis:
a. More students from the upper group choose option
D than option A, even though option A is
supposedly the correct answer.
b. Most likely the teacher has written the wrong
answer key.
IMPROVING TEST ITEMS

c. The teacher checks and finds out that he/she did not
miskey the answer that he/she thought is the correct
answer.
d. If the teacher miskeyed it, he/she must check and
retally the scores of the students’ test papers before
giving them back.
IMPROVING TEST ITEMS

e. If option A is really the correct answer, revise to

weaken option D, distracters are not supposed to
draw more attention than the keyed answer.
f. Only 10% of the students got the answer to the test
item, correctly, hence, the test item is very difficult.
IMPROVING TEST ITEMS

g. More students from the lower group got the item

correctly, therefore a negative discrimination
resulted. The discrimination index is -10%.
h. No need to analyze the distracters because the test
item is very difficult and discriminates negatively.
IMPROVING TEST ITEMS

4. Conclusion: Reject the item because it is very

difficult and has a negative discrimination index.
Example 5: Ambiguous item. Below is the result of
item analysis of a test with an ambiguous test item.
What can you say the item? Are you going to retain,
revise or reject it?
IMPROVING TEST ITEMS

Options A B C D E*
Upper Group 7 1 1 2 8
Lower Group 6 2 3 3 6
IMPROVING TEST ITEMS

1. Compute the difficulty index:

n=8+6=14
N=39
=
=
=0.36 or 36%
IMPROVING TEST ITEMS

2. Compute the discrimination index:

=8 =
=6 = 0.10 or 10%
D=20
=-=
D
IMPROVING TEST ITEMS

3. Make an analysis:
a. Only 36% of the students got the answer to the test
item correctly, hence, the test item is difficult.
b. More students from the upper group got the item
correctly, hence, it discriminates positively. The
discrimination index is 10%.
IMPROVING TEST ITEMS

c. About equal numbers of top students went for

option A and option E, this implies that they
could not tell which is the correct answer. The
students do not know the content of the test,
thus, a reteach is needed.
IMPROVING TEST ITEMS

4. Conclusion: Revise the test item because it is

ambiguous.
Example 6. Guessing Item. Below is the result of
an item analysis for a test item with students’ answers
mostly based on a guess. Are you going to reject,
revise or retrain the test item?
IMPROVING TEST ITEMS

Options A B C* D E
Upper Group 4 3 4 3 6
Lower Group 3 4 3 4 5
IMPROVING TEST ITEMS

1. Compute the difficulty index:

n=4+3=7
N=39
=
=
=0.18 or 18%
IMPROVING TEST ITEMS

2. Compute the discrimination index:

=4 =
=3 =
D=20 = 0.05 or 5%
=-
D
IMPROVING TEST ITEMS

3. Make an analysis.
a. Only 18% of the students got the answer to the test item
correctly, hence, the test item is very difficult.
b. More students from the upper group got the correct
answer to the test item; therefore, the test item is a
positive discrimination. The discrimination index is 5%.
IMPROVING TEST ITEMS

c. Students respond about equally to all alternatives,

an indication that they are guessing.
Three possibilities why students guesses the
answer on a test item:
• the content of the test item has not yetbeen
discussed in the class because the test is designed in
advance.
IMPROVING TEST ITEMS

•test items were badly written that students have

no idea what the question is really all about;
and
•test items were very difficult as shown from the
difficult index and low discrimination index.
IMPROVING TEST ITEMS

d.If the test item is well-written but too

difficult, reteach the material to the class.
4. Conclusion: Reject the item because it is
very difficult and the discrimination index is
very poor, and options A and B are not effective
distracters.
IMPROVING TEST ITEMS

Example 7. The table below shows an item

analysis of a test item with ineffective
distracters. What can you conclude about the
test item?
IMPROVING TEST ITEMS

Options A B C* D E
Upper Group 5 3 9 0 3
Lower Group 6 4 6 0 4
IMPROVING TEST ITEMS

1. Compute the difficulty index:

n=9+6=15
N=40
=
=
=0.38 or 38%
IMPROVING TEST ITEMS

2. Compute the discrimination index:

=9 =
=6 =
D=20 = 0.15 or 15%
=-
D
IMPROVING TEST ITEMS

3. Make an analysis.
a. Only 38% of the students got the answer to the test
item correctly, hence, the test item is difficult.
b. More students from the upper group answered the
test item correctly, as a result, the test got a positive
discrimination. The discrimination index is 15%.
IMPROVING TEST ITEMS

c. Options A, B, and E are attractive and effective

distracters.
d. Option D is ineffective; therefore, change it with
more realistic one.

4. Conclusion: Revise the item by changing option D.