Professional Documents
Culture Documents
Basics of
Item
Analysis
QUESTIONS?
Contact Me…!!!
©James L. Paglinawan TM
® e-mail me @:
JLPaglinawan@cmu.edu.ph
JLPaglinawan@teacher.deped.gov.ph
Click to end
Item Analysis
Techniques to
improve test items
and instruction
Test Development Process
1. Review 2. Convene
13. Standard Setting
Study
National National
Advisory
14. Set Passing and Committee
Standard Profession 3. Develop Domain,
al Knowledge and
11. Administer Skills Statements
Tests
Standards 4. Conduct Job
Analysis
12. Conduct
Item 5. Establish Test
Analysis Specifications
6. Develop Test
9. Assemble Operational Design
Test Forms
10. Produce Printed Tests 7. Develop New Test
Mat. Questions
8. Review Test Questions
What is Item Analysis ?
– Individual level:
• isolate specific errors this student
made
2. Build future tests,
revise test items to make
them better
• know how much work in writing good
questions
• SHOULD NOT REUSE WHOLE TESTS -->
diagnostic teaching means responding
to needs of students, so after a few years
a test bank is build up and choose a
tests for the class
• can spread difficulty levels across your
blueprint (TOS)
3. Part of continuing
professional development
– doing occasional item analysis will
help become a better test writer
– documenting just how good your
evaluation is
– useful for dealing with parents or
administrators if there's ever a dispute
– once you start bringing out all these
impressive looking stats, parents and
administrators will believe why some
students failed.
Classical ITEM Analysis
Statistics
– Validity
Truthfulness of response
Characteristics:
1. The intercorrelations among the items --
the greater/stronger the relative number
of positive relationships are, the greater
the reliability.
4. Heterogeneous groups of
test takers
Types of reliability
• Stability
1. Test – Retest
• Stability
2. Inter – rater / Observer/ Scorer
• applicable for mostly essay questions
• Use Cohen’s Kappa Statistic
• Equivalence
3. Parallel-Forms/ Equivalent
Used to assess the consistency of
the results of two tests
constructed in the same way from
the same content domain.
• Internal Consistency
• Used to assess the consistency of
results across items within a test.
4. Split – Half
• 5. Kuder-Richardson
Formula 20 / 21
Correlation is
determined from a
single administration of
a test through a study
of score variances
• 6. Cronbach's Alpha (a)
Reliability
Indices
Interpretation
Excellent reliability; at the level of the best standardized
.91 and above
tests
.81 - .90 Very good for a classroom test
Item Difficulty
Percent answering correctly
Item Discrimination
How well the item "functions“
How “valid” the item is based
on the total test score criterion
WHAT IS A WELL-
FUNCTIONING
TEST ITEM?
>=80
<= 30% > 30% AND < 80%
%
0 10 20 30 40 50 60 70 80 90 100
Item Difficulty Level:
Sample
Number of students who answered each item = 50
Section 4 6 10 20
A (1,3,7,9)
Section 5 5 4 14
B (2,5,8,11,15)
Section 3 7 6 16
C (6,17,21)
Total 12 18 20 50
Steps in Item
analysis
1. Code the test items:
- 1 for correct and 0 for
incorrect
- Vertical – columns (item
numbers)
- Horizontal – rows
(respondents/students)
TEST ITEMS
No. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 . . . . 50
1 1 0 1 1 1 0 0 0 0 1 1 1 0 0 0 0 0 1 1
2 1 1 0 1 1 1 0 1 1 1 0 1 1 1 1 0 1 1 1
3 0 0 0 1 0 0 0 1 0 0 0 1 1 1 1 1 1 1 0
4 0 1 0 0 0 1 0 0 0 1 0 0 1 0 0 0 1 0 0
5 1 0 1 1 1 0 1 1 1 0 1 1 0 1 1 1 0 1 0
6 1 1 1 1 1 1 1 1 1 1 1 1 1 0 1 1 1 0 1
7 0 0 1 0 0 0 1 0 0 0 1 0 0 0 1 0 0 0 1
8 1 1 0 1 1 1 0 1 1 1 0 1 1 0 0 0 1 0 0
2. IN SPSS:
Analyze Scale
Reliability analysis
(drag/place variables to
Item box) Statistics
Scale if item deleted
ok.
• ****** Method 1 (space saver) will be used for this analysis ******
• R E L I A B I L I T Y A N A L Y S I S - S C A L E (A L P H A)
• Item-total Statistics
• Scale Scale Corrected
• Mean Variance Item- Alpha
• if Item if Item Total if Item
• Deleted Deleted Correlation Deleted
• VAR00001 14.4211 127.1053 .9401 .9502
• VAR00002 14.6316 136.8440 .7332 .9542
• VAR00003 14.4211 141.5695 .4774 .9574
• VAR00004 14.4737 128.6109 .6511 .9508
• VAR00005 14.4737 128.8252 .8274 .9509
• VAR00006 14.0526 130.6579 .2236 .9525
• VAR00007 14.2105 127.8835 .2533 .9511
• VAR00008 14.1053 128.6673 .1906 .9515
• VAR00009 14.4211 129.1410 .7311 .9513
• .....................
• VAR00022 14.4211 129.1410 .7311 .9513
• VAR00023 14.4211 127.1053 .4401 .9502
• VAR00024 14.6316 136.8440 -.0332 .9542
• VAR00047 14.4737 128.6109 .8511 .9508
• VAR00048 14.4737 128.8252 .8274 .9509
• VAR00049 14.0526 130.6579 .5236 .9525
• VAR00050 14.2105 127.8835 .7533 .9511
• Reliability Coefficients
• N of Cases = 57.0 N of Items = 50
• Alpha = .9533
3. In the output dialog box:
A 4 1 6 3 10 3
B 5 3 5 3 4 2
C 3 2 7 4 6 3
Total 12 6 18 10 20 8
% 50% 56% 40%
Over 24/50 = 48%
all
• Realistically: Do item analysis
to your most important tests