ADMINISTERING, ANALYZING, AND

IMPROVING THE TEST OR ASSESSMENT

NEMA GRACE B. MEDILLO

GOAL IN THIS CHAPTER:

To provide suggestion on how to

avoid common pitfalls in test assembly,
administration, and scoring.
Start

Assembling the Test Administering the Test Scoring the Test

Process of Evaluating Classroom

Analysing the Test Debriefing Achievement

Nema Grace B. Medillo

 ASSEMBLING THE TEST

Packaging the Test Reproducing the Test

 Packaging the Test
Group together all items of similar format
Arrange test items from easy to hard
Space the items for easy reading
Keep the items and options on the same page
Position illustration near descriptions and above the item
Check your answer
Determine how the students record their answer
Provide space for Name and Date
Check test directions
Proofread the test
Reproducing the Test
Know the Photocopying Machine

Specify Copying Instructions

File Original Test

 ADMINISTERING THE TEST

Maintain a Positive Attitude

Maximize Achievement Motivation
Equalize Advantages
Avoid Surprises
Clarify the Rules
Rotate Distribution
Remind Students to Check Their Copies
Monitor Students
Minimize Distractions
Give Time Warnings
Collect Test Uniformly
SCORING THE TEST

Prepare the Answer Key

Check the Answer Key
Score Blindly
Check Machine – Scored Answer Sheets
Check Scoring
Record Scores
 ANALYZING THE TEST
Item Analysis
Terminology
Quantitative Item Analysis

Qualitative Item Analysis

Item Analysis Modifications for
the Criterion – Referenced Test
 Item Analysis Terminology
Quantitative Item Analysis
• A numerical method for analyzing test items employing
students response alternatives or options.
Qualitative Item Analysis
• A non – numerical method for analyzing test items not
employing students responses, but considering test
objectives, content validity, and technical item quality
Key
• Correct option in a multiple – choice item
 Item Analysis Terminology
Distractor
• Incorrect option in a multiple – choice item

Difficulty Index
• Proportion of students who answered the item correctly.

Discrimination Index D
• Measure of the extent to which a test item discriminates or
differentiates between students who do well on the overall
test and those who do not do well on the overall test.
 Discrimination Index (D)
Positive Discrimination Index
• Those who did well on the overall test chose the
correct answer for a particular item more often than
those who did poorly on the overall test.
Negative Discrimination Index
• Those who did poorly on the overall test chose the
correct answer for a particular item more often than
those who did well on the overall test.
Zero Discriminanation Index
• Those who did well and those who did poorly on the
overall test chose the correct answer for a particular
item with equal frequency
 Quantitative Item Analysis

Case Sample and Difficulty Discrimination

Guide Questions Level Index

Miskeying Guessing Ambiguity

 Consider the case below
Suppose your students chose the options to a
four – alternative multiple – choice item.

Let C as the correct answer.

Item X
A B C* D
3 0 18 9
How does this information help us?
Is the item too difficult/easy for the students?
Are the distractors of the items effective?
Guide questions in quantitative item analysis

1.What is the difficulty level?

2.What is the discrimination index?
3.Should this item be eliminated?
4.Should any distractor(s) be modified?
To compute the difficulty level of an item;

𝑵𝒖𝒎𝒃𝒆𝒓 𝒔𝒆𝒍𝒆𝒄𝒕𝒊𝒏𝒈 𝒄𝒐𝒓𝒓𝒆𝒄𝒕 𝒂𝒏𝒔𝒘𝒆𝒓

𝒑=
𝑻𝒐𝒕𝒂𝒍 𝑵𝒖𝒎𝒃𝒆𝒓 𝒕𝒂𝒌𝒊𝒏𝒈 𝒕𝒉𝒆 𝒕𝒆𝒔𝒕

Item X
A B C* D
3 0 18 9

What is the difficulty level of the item?

Do you consider the item difficult or easy? Why?
Solving the difficulty index
𝑵𝒖𝒎𝒃𝒆𝒓 𝒔𝒆𝒍𝒆𝒄𝒕𝒊𝒏𝒈 𝒄𝒐𝒓𝒓𝒆𝒄𝒕 𝒂𝒏𝒔𝒘𝒆𝒓
𝒑=
𝑻𝒐𝒕𝒂𝒍 𝑵𝒖𝒎𝒃𝒆𝒓 𝒕𝒂𝒌𝒊𝒏𝒈 𝒕𝒉𝒆 𝒕𝒆𝒔𝒕
Item X 𝟏𝟖
A B C* D 𝒑=
𝟑𝟎
3 0 18 9
𝒑 = 𝟎. 𝟔𝟎

Note:
If P level > 0.75, the item is considered relatively easy.
If P level < 0. 25, the item is considered relatively difficult.

Since the difficulty level of the item is 0. 60 (60%), the item is

moderately difficult.
 Discrimination Index
Steps in determining Discrimination Index

1. Arrange the papers from highest to lowest score.

2. Separate the papers into an upper group and lower
3. For each item, count the number in the upper group
and the number in the lower group that chose each
alternatives
4. Record your information for each item
Example for item X ( Class Size = 30)

Options A B C* D
Upper 1 0 11 3
Lower 2 0 7 6
 5. Compute D, by plugging the appropriate
numbers in the formula
𝑵𝒖𝒎𝒃𝒆𝒓 𝒘𝒉𝒐 𝑵𝒖𝒎𝒃𝒆𝒓 𝒘𝒉𝒐
𝒈𝒐𝒕 𝒊𝒕𝒆𝒎 𝒈𝒐𝒕 𝒊𝒕𝒆𝒎
−
𝒄𝒐𝒓𝒓𝒆𝒄𝒕 𝒊𝒏 𝒄𝒐𝒓𝒓𝒆𝒄𝒕 𝒊𝒏
𝒖𝒑𝒑𝒆𝒓 𝒈𝒓𝒐𝒖𝒑 𝒍𝒐𝒘𝒆𝒓 𝒈𝒓𝒐𝒖𝒑
𝑫=
𝑵𝒖𝒎𝒃𝒆𝒓 𝒐𝒇𝒔𝒕𝒖𝒅𝒆𝒏𝒕𝒔
𝒊𝒏 𝒆𝒊𝒕𝒉𝒆𝒓 𝒈𝒓𝒐𝒖𝒑

(if group sizes are unequal, choose the higher number)

What is the discrimination index of item X?

Is the discrimination index positive or negative?
Which of the groups frequently get the item correctly?
 Plugging the data

𝟏𝟏 − 𝟕
𝑫= = 𝟎. 𝟐𝟔𝟕
𝟏𝟓
Since the discrimination index of
item X is 0. 267, which is positive. More
students who did well on the overall test
answered the item correctly than students
who did poorly on the overall test.
Implication
Difficulty Level (p) = 0. 60
Discrimination Index (D) = 0.267

Should this item be eliminated? NO

The item is considered a moderately difficult
item that has positive (desirable) discrimination ability.
Should any distractor(s) be modified?

Item X YES
A B C* D
3 0 18 9

Option B is ought to be modified or replaced.

(No one chose it)
 Let’s look at the responses for another item
Item Y ( Class size = 28) Item Z ( Class size = 30)

Options A* B C D Options A B* C D
Upper 4 1 5 4 Upper 3 4 3 5
Lower 1 7 3 3 Lower 0 10 2 3

1. What is the difficulty level?

2. What is the discrimination index?
3. Should this item be eliminated?
4. Should any distractor(s) be modified?
 Item Y
Difficulty Level = 0.18
Discrimination Index = 0. 214
Should this item be eliminated?
No, since it is positively discriminating. However, it is
difficult item; only 18% of the class got it right.
Should any distractor(s) be eliminated?
Yes; C and D attracted more students who did well on the
test overall.

Fewer students who do well on the test should choose

Remember: each distractor than students who do poorly.
More students who do well on the test should choose
the correct answer than students who do poorly
Item Z Difficulty Level = 0.467
Discrimination Index = - 0.40

Should this item be eliminated?

Yes! The item is moderately difficult but it
discriminate negatively.
Should any distractor(s) be eliminated?
Since we already decided to eliminate the item, this is
moot question.

Remember:
Testing is to discriminate between those students
who know their stuff and those who do not.
 MISKEYING
Most students who did well on the
test will likely select an option that is
a distractor, rather than the option that
is keyed.
 Consider the miskeyed item
Who was the first astronaut
to set foot on the moon?

a. John Glenn
Responses
b. Scott Carpenter
c. Neil Armstrong A B C D*
*d. Alan Sheppard Upper 1 1 9 2

Most students in the upper half of the class fail to

select the keyed option.
Remember, just as you are bound to make scoring
errors, you are bound to miskey an item occasionally.
 GUESSING
Likely to occur when the item measures content
that is,
• Not covered in class or the text
• So difficult that even the upper – half students
have no idea what the correct answer is
• So trivial that students are unable to choose from
among the options provided.
The choice distribution suggest that guessing occurred:

A B C* D
Upper Half 4 3 3 3

Each alternative is about equally attractive to

students in the upper half.
 Ambiguity
Among the upper group, one of the distractors is
chosen with about the same frequency as the correct
answer.

The distribution suggests that an item is ambiguous:

A B C D*
Upper Half 7 0 1 7
 QUALITATIVE ITEM ANALYSIS

• Matching items and objectives

• Editing poorly written items
• Improving content of the validity of the test
• Analyzing grammatical cues, specific determiners,
double negatives, multiple defensible answers, and
items that fail to match instructional objectives
 ITEM ANALYSIS MODIFICATIONS FOR
THE CRITERION – REFERENCED TEST
Using Pre- and
Comparing the
Posttest as
Percentage
Upper and
Answering Each
Lower Group
Item Correctly Determining the
on Both Pre- Percentage of
and Posttest Items Answered in Limitations of
the Expected Modifications
Directions
 Using Pre- and Posttests as
Upper and Lower Groups

Pretest prior instructions Post test after instructions

Most of the test items Most items were

answered incorrectly answered correctly
Results for Lower Results for Upper
Group Group
P level = 0. 30 or lower P = 0.70 or higher
 Analyzing sample Data
Example 1:

Number of students choosing option (n = 25)

Option At pretest (L) At posttest(U)
A 9 1
B 7 1
C 3 2
D* (Key) 6 21
Steps
1. Compute p Levels for both tests.

Pretest Posttest
𝑁𝑜.𝑐ℎ𝑜𝑜𝑠𝑖𝑛𝑔 𝐶𝑜𝑟𝑟𝑒𝑐𝑡𝑙𝑦 6 21
𝑃=
𝑇𝑜𝑡𝑙 𝑁𝑢𝑚𝑏𝑒𝑟 25 25

= 0.24 =0.84

It was an improvement from 24% to 84%.

2. Determine the discrimination index (D) for
the key

𝑁𝑢𝑚𝑏𝑒𝑟 𝑐𝑜𝑟𝑟𝑒𝑐𝑡 𝑝𝑜𝑠𝑡 −𝑁𝑢𝑚𝑏𝑒𝑟 𝑐𝑜𝑟𝑟𝑒𝑐𝑡 (𝑝𝑟𝑒)

𝐷=
𝑁𝑢𝑚𝑏𝑒𝑟 𝑖𝑛 𝑒𝑖𝑡ℎ𝑒𝑟 𝑔𝑟𝑜𝑢𝑝

21 −6 15
𝐷= = = 0.60
25 25
The key has a positive discrimination
index.
3. Determine whether each option separately
discriminates negatively.

1−9 −8
Option A: 𝐷 = = = −0.32
25 25
1−7 −6
Option B: 𝐷 = = = −0.24
25 25
2−3 −1
Option C: 𝐷 = = = −0.04
25 25
 Summary
1. There was sizeable increase in p value from
pretest to posttest.
2. The D index for the key was positive.
3. The distractors are all discriminated negatively.

If a criterion – reference test item manifests these

features, it has passed our “test” and a probably is a good
item with little or no need for modification.

Contrast with, the test item failed. Rather than

modify, it is probably more efficient to replace it with
another.
 Comparing the Percentage Answering Each
Item Correctly on Both Pre- and Posttest
If your test is sensitive to your objectives.

What to do?
Percentage Passing Posttest – Percentage of Passing Pretest

The more positive the difference, the more the item tapping
the content you are teaching.
 Analyzing Sample Data
Consider the following percentages for five test
items:
Item Percentage Percentage Difference
passing pretest passing posttest
1 16 79 +63%
2 10 82 +72%
3* 75 75 0%
4 27 91 +64%
5* 67 53 -14%

Item 3 may be eliminated. (Students already know the content it

represents.)
Item 5 may be revised. (Instruction was not related to the item or
confused some students.)
Determining the Percentage of Items
Answered in the Expected Directions
for the Entire Test
Steps
1. Find the number of items each student failed on the pretest
but passed on the posttest.

Result of Mary
Item Pretest Posttest
1* Incorrect Correct
2 Correct Correct
3* Incorrect Correct
4 Correct Incorrect
5 Incorrect Incorrect

Do same things to other students.

Example
Mary 18 Carlos 15
Sharon 22 Amanda 20
Charles 13

2. Add the counts and divide by the number of

students.

18+15+22+20+13 88
= = 17.6
5 5
3. Divide by number
of test items.

17.6 4. Multiply by 100.

= 0.70
25
0.70 x 100 = 70%

The greater the overall positive percentage of

change, the more your test is likely to match your
instruction and to be a content – valid test
Limitations of Modification
Difficult

Unit of instruction is brief

From norm – referenced test to criterion –

referenced test

Time devote to instruction (pre – post)

 DEBRIEFING
Discuss Problem Items
Listen to Students Reactions
Avoid – on – the Spot Decisions
Be Equitable with Changes
Ask Students to Double – Check
Ask Students to Identify Problems
 PROCESS OF EVALUATING
CLASSROOM ACHIEVEMENTS
 THANK YOU…

THANK YOU… THANK YOU…

THANK YOU…

THANK YOU… THANK YOU…

Shutting down