The Case of De La Salle-

Santiago Zobel School (DLSZ)

Shift from Classical Test Theory
approach to Item Response Theory
(IRT)

IRT was introduced to DLSZ in

School Year 2008-2009 by
Dr. Carlo Magno.
Curriculum-based Achievement tests

• Grade/Year Levels
Senior Prep
Grades 1 to Grade 7
First year to Fourth year high school

• Subject Areas
English Math
Filipino Science
Christian Living Social Studies
Two (2) forms of the
achievement tests were created
Experts in the subject areas
content validated and externally
reviewed the test items and the
curriculum
Subject Area Teachers’ workshop were
conducted

Teachers /Subject Area Coordinators

revised some items

Consultants reviewed revisions via

email
Test item pool and TOS to IPA

•Orientation of Teachers for the pilot testing

•Pilot testing for all levels

•Gathering of test data from the pilot tests

•Statistical analysis of test items

 Descriptive statistics

 Mean and standard deviations were

obtained to determine the central tendency
and variability of the scores for each form
of the tests.
Two ways:
 Internal consistency – Cronbach’s alpha

value of .70

 Parallel forms of the test

-Pearson r was used to correlate the
scores of each examinee for both forms.
-All correlation coefficients obtained
were significant (p<.05)
 Content validity
The items were reviewed by experts in
the field based on its intended skills and
content.

 Concurrent validity
Each form of the test was correlated with
each other. Majority of the correlation
coefficients were significant.
 The one-parameter Rasch model was used
to determine the difficulty and fit of the
items.

Item difficulty
 Positive logit values indicate that the
item is difficult and negative logit values
indicate that the item is easy.
 ITEM DIFFICULTY RESULTS
SY ‘08-’09 SY ’09-’10
Item Fit

Means Square Values (MNSQ) within 0.8 to 1.2

are said to have good fit.

Bad items are determined if the ability of the

persons do not match difficulty of the items.
 ITEM FIT RESULTS
SENIOR PREP
No. of No. of
No. of
Form Good Fit Bad Fit N MEAN STDEV PROPORTION
Items CORRECT
Item/s Item/s

SP 30 28 28 2 2 192 233 19.16 19.48 4.67 4.98 0.64 0.65

Legend:   - SY 2008-2009
  - SY 2009-2010

SY2008-2009 taken by the

FORM SY2008-2009 SY2009-2010
Upper Level (PILOT TESTING)

Senior Prep 7, 11, 14, 17, 18, 19, 20 18, 6 18, 22

FORM DIFFICULT to EASY items EASY to DIFFICULT items

SP 23, 28 2, 5

SY2008-2009 SY2009-2010
FORM No. of Items No. of Difficult No. of Easy No. of Difficult No. of Easy
Items Items Items Items
SP 30 16 14 16 14
 ENGLISH
No. No. of No. of
PROPORTION
Form of Good Fit Bad Fit N MEAN STDEV CORRECT
Items Item/s Item/s

Engl 1 45 42 45 3 - 215 241 31.51 26.60 6.52 7.45 0.70 0.59

Engl 2 44 43 43 1 1 210 291 29.07 26.53 6.66 6.66 0.66 0.60

Engl 3 60 56 58 4 2 215 328 32.53 31.44 9.55 8.91 0.54 0.52

Engl 4 70 65 69 5 1 265 313 37.48 40.23 11.18 8.02 0.54 0.57

Engl 5 70 62 68 8 2 198 290 39.73 37.77 11.22 10.14 0.57 0.54

Engl 6 75 75 74 - 1 269 285 47.19 42.73 9.34 10.07 0.63 0.57

Engl 7 80 79 80 1 - 292 316 51.42 45.82 10.72 10.66 0.64 0.57

Legend:   - SY 2008-2009
  - SY 2009-2010
 SY2008-2009 taken by the
FORM Upper Level SY2009-2010
(PILOT TESTING)
Engl 1 19, 27, 31 NO REVISIONS NEEDED
Engl 2 46 23
Engl 3 9, 10, 21, 40 39, 50
Engl 4 20, 22, 34, 35, 53 34
Engl 5 28, 32, 33, 37, 44, 46, 50, 53 12, 44
Engl 6 NO REVISIONS NEEDED 31
Engl 7 78 NO REVISIONS NEEDED

FORM DIFFICULT to EASY items EASY to DIFFICULT items

Engl 1 19 40, 41
Engl 2 NONE NONE
Engl 3 9, 10, 18, 33, 42, 55 5, 16, 40, 43, 47, 52
Engl 4 20, 25,50, 61, 70 27, 52, 53, 60
Engl 5 24, 32, 37, 39, 40, 48, 59, 65, 68 28, 46, 53, 54
Engl 6 29, 42, 53 21, 34, 35, 57, 64
Engl 7 3, 8, 11, 20 37, 55, 74
 SY2008-2009 SY2009-2010
No. of No. of
Form No. of No. of Easy No. of Easy
Items Difficult
Difficult Items Items Items
Items
Engl 1 45 21 24 22 23
Engl 2 44 21 23 23 21
Engl 3 60 32 28 32 28
Engl 4 70 36 34 35 35
Engl 5 70 36 34 31 39
Engl 6 75 37 38 39 36
Engl 7 80 39 41 39 41
 CHRISTIAN LIVING
No. of No. of
No. PROPORTI
Bad
Form of N MEAN STDEV ON
Good Fit Fit
Items CORRECT
Item/s Item/s
CL 1 30 28 29 2 1 198 241 25.69 25.00 2.82 3.17 0.86 0.83

CL 2 30 27 29 3 1 179 289 21.16 20.73 4.12 3.84 0.71 0.69

CL 3 40 39 39 1 1 255 323 25.58 24.20 5.28 5.24 0.64 0.61

CL 4 50 50 49 - 1 283 318 27.25 27.66 6.57 6.31 0.55 0.55

CL 5 50 50 48 - 2 282 301 36.15 35.04 5.86 7.29 0.72 0.70

CL 6 60 60 59 - 1 285 310 39.56 37.76 6.68 7.76 0.66 0.63

CL 7 60 60 58 - 2 233 320 42.14 43.06 6.29 8.05 0.70 0.72

Legend:   - SY 2008-2009
  - SY 2009-2010
 SY2008-2009 taken by the
FORM SY2009-2010
Upper Level (PILOT TESTING)
CL 1 22, 23 10
CL 2 2, 13, 25 30
CL 3 12 34
CL 4 NO REVISIONS NEEDED 40
CL 5 NO REVISIONS NEEDED 33, 44
CL 6 NO REVISIONS NEEDED 7
CL 7 NO REVISIONS NEEDED 2, 60

FORM DIFFICULT to EASY items EASY to DIFFICULT items

CL 1 5 28, 13, 14,
CL 2 14, 24 13, 26
CL 3 8, 21, 35, 36 1, 13, 22, 23, 33, 40
CL 4 6, 15, 17, 27, 38, 43, 47, 50 9, 12, 14, 44, 46
CL 5 5, 9, 21, 23, 35 14, 17, 24, 33
CL 6 30, 37 4, 25, 40, 58
CL 7 3, 6, 9, 29, 35, 51 4, 5, 14, 16, 47, 57
 SY2008-2009 SY2009-2010
No. of No. of
Form No. of No. of Easy No. of
Items Difficult
Difficult Items Items Easy Items
Items
CL 1 30 14 16 16 14
CL 2 30 16 14 16 14
CL 3 40 18 22 21 19
CL 4 50 26 24 22 28
CL 5 50 26 24 27 23
CL 6 60 28 32 31 29
CL 7 60 28 32 28 32
 MATH
No. of No. of
No. of PROPORTION
FORM Good Fit Bad Fit N MEAN STDEV
Items CORRECT
Item/s Item/s

Math 1 30 27 28 3 2 190 239 24.08 19.81 4.42 4.81 0.80 0.66

Math 2 40 35 36 5 4 212 292 26.68 21.69 6.62 5.35 0.67 0.54

Math 3 29 23 27 6 2 244 318 21.81 19.43 4.79 4.39 0.75 0.65

Math 4 49 47 49 3 - 216 312 30.25 32.24 7.72 8.49 0.62 0.66

Math 5 50 48 48 2 2 273 287 30.05 23.83 7.80 7.48 0.60 0.48

Math 6 58 59 55 1 3 260 281 31.92 28.59 8.16 9.01 0.55 0.48

Math 7 60 59 59 1 1 251 332 30.68 29.41 9.00 8.10 0.51 0.49

Legend:   - SY 2008-2009
  - SY 2009-2010
 SY2008-2009 taken by the
FORM Upper Level (PILOT SY2009-2010
TESTING)
Math 1 9, 16, 20 16, 25
Math 2 5, 19, 20, 24, 32 23, 26, 31, 32
Math 3 2, 3, 7, 18, 21, 25 18, 25
Math 4 31, 36, 39 NO REVISIONS NEEDED
Math 5 25, 41 25, 41
Math 6 14 8, 14, 50
Math 7 6 50

FORM DIFFICULT to EASY items EASY to DIFFICULT items

Math 1 5 12, 26, 20
Math 2 38, 25, 5 20, 32, 23, 18, 31, 19
Math 3 21, 27, 24, 14 7
Math 4 42, 36, 38 6, 27, 1
Math 5 39, 46, 44 8, 28, 4
Math 6 44, 37, 20, 42, 15, 34, 26, 18, 31, 28 45, 6, 25
Math 7 30, 31, 49 57, 52, 47, 46, 17, 36, 18, 11
 SY2008-2009 SY2009-2010
No. of No. of
Form No. of Difficult No. of Easy No. of Easy
Items Difficult
Items Items Items
Items
Math 1 30 16 14 18 12
Math 2 40 16 24 19 21
Math 3 29 15 14 12 17
Math 4 49 25 24 25 24
Math 5 50 27 23 27 23
Math 6 58 32 26 26 32
Math 7 60 26 34 32 28
 SCIENCE
No. of No. of
No. of PROPORTION
FORM Good Bad N MEAN STDEV
Items CORRECT
Fit Item/s Fit Item/s

Sci 1 30 28 30 2 1 264 241 24.22 20.65 3.65 4.05 0.81 0.69

Sci 2 30 25 30 5 - 265 293 24.46 22.02 3.33 3.44 0.82 0.73

Sci 3 40 39 38 1 2 233 317 23.06 23.30 6.06 6.27 0.58 0.58

Sci 4 50 49 50 1 - 224 319 28.65 30.42 7.42 6.86 0.57 0.61

Sci 5 50 43 49 7 1 272 300 32.97 29.69 7.49 7.06 0.66 0.59

Sci 6 60 60 60 - - 271 309 35.71 29.20 8.39 7.94 0.60 0.49

Sci 7 60 59 60 1 - 244 332 29.00 27.11 7.04 7.62 0.48 0.45

Legend:   - SY 2008-2009
  - SY 2009-2010
 SY2008-2009 taken by the
FORM Upper Level (PILOT SY2009-2010
ESTING)
Sci 1 10, 27 12
Sci 2 10, 22, 24, 25, 27 NO REVISIONS NEEDED
Sci 3 17 15, 32
Sci 4 30 NO REVISIONS NEEDED
Sci 5 6, 7, 8, 10, 11, 22, 44 10
Sci 6 NO REVISIONS NEEDED NO REVISIONS NEEDED
Sci 7 27 NO REVISIONS NEEDED
FORM DIFFICULT to EASY items EASY to DIFFICULT items

Sci 1 6, 7, 26 4, 1, 10
Sci 2 11, 26, 28 22, 25
Sci 3 24, 27 2, 17
Sci 4 11, 12, 23, 26,45 2, 10, 15, 20, 25, 27, 33, 37, 46
Sci 5 2, 13, 15, 16, 33, 36, 43, 47, 49 8, 11, 22, 24
Sci 6 11, 15, 18, 21, 23, 26, 33, 37, 48, 53, 54 8, 31, 32, 35, 41, 42, 56
Sci 7 31, 32, 36, 39, 51, 52, 54, 58 1, 3, 7, 12, 13, 20, 26, 28, 30, 41
 SY2008-2009 SY2009-2010
No. of No. of
Form No. of Difficult No. of Easy No. of
Items Difficult
Items Items Easy Items
Items
Sci 1 30 13 17 13 17
Sci 2 30 14 16 13 17
Sci 3 40 22 18 22 18
Sci 4 50 22 28 26 24
Sci 5 50 26 24 21 29
Sci 6 60 31 29 27 33
Sci 7 60 29 31 31 29
 FILIPINO
No. of No. of
No. of PROPORTION
FORM Good Fit Bad Fit N MEAN STDEV CORRECT
Items
Item/s Item/s

Fil 1 30 27 27 3 3 229 239 21.40 16.85 5.22 5.25 0.71 0.56

Fil 2 35 32 34 3 1 174 292 23.92 21.40 6.15 5.75 0.68 0.61

Fil 3 40 37 40 3 - 204 328 24.53 22.42 8.64 7.40 0.61 0.56

Fil 4 50 45 49 5 1 258 318 27.55 26.42 9.03 8.38 0.55 0.53

Fil 5 50 54 47 6 3 266 287 27.81 26.52 8.90 7.95 0.56 0.53

Fil 6 60 55 55 5 5 276 305 39.24 32.42 9.43 10.25 0.65 0.54

Fil 7 60 56 59 4 1 240 319 37.46 31.83 9.58 8.80 0.62 0.53

Legend:   - SY 2008-2009
  - SY 2009-2010
 SY2008-2009 taken by the
FORM SY2009-2010
Upper Level (PILOT TESTING)
Fil 1 4, 9, 26 9, 20, 26
Fil 2 24, 28, 29 8
Fil 3 2, 5, 24, 27 NO REVISIONS NEEDED
Fil 4 15, 19, 21, 39, 43 38
Fil 5 5, 16, 22, 26, 36, 38 5, 25, 26
Fil 6 5, 15, 20, 37, 54 5, 10, 11, 19, 20
Fil 7 4, 30, 39, 48 9

FORM DIFFICULT to EASY items EASY to DIFFICULT items

Fil 1    
Fil 2 33 21
Fil 3 4, 9, 14, 29, 38 10, 16, 30
Fil 4 2, 11, 40 16, 35, 49
Fil 5 31, 40, 42 3, 5, 9, 30, 34, 36
Fil 6   1, 31, 40, 44, 49, 56
Fil 7 6, 8, 12, 22, 23, 42, 43, 45, 57 2, 13, 27, 32, 34
 SY2008-2009 SY2009-2010
No. of No. of No. of No. of
Form No. of Easy
Items Difficult Difficult Easy
Items
Items Items Items
Fil 1 30 13 17 14 16
Fil 2 35 17 18 17 18
Fil 3 40 22 18 20 20
Fil 4 50 27 23 27 23
Fil 5 50 22 28 25 25
Fil 6 60 28 32 33 27
Fil 7 60 29 31 25 35
 Which of the tests obtained the same
results when administered to 2 different
groups for…

Item Difficulty?
English 2 and Filipino 1 only

Item Fit?
Science 6 only (no bad fit items)
 Batch 1 – IPA Personnel and Coordinators

What did you like in the activity?

- workshop is very useful and
very relevant.

- Everything about the latest

technology that we can use for
Item Analysis

My suggestions to improve the activity are:

- more workshops, it was an interesting

topic but then we need more time so we
can absorb all the important details

Batch 2 – Math and Science Teachers

 Batch 3 – Social Studies & CL teachers
What did you like in the activity?

- It is helpful in determining items difficulty and

good fitting items in tests.It will further help us
teachers in creating reliable and valid
assessments.
- The topic is interesting because it's ''new''
- New insights.
- Actual/hands on.
What did you dislike in the activity?

- It was just too fast.

My suggestions to improve the activity

are:
- Study further its advantage before bringing
this down to us, teachers.

Batch 4 – English & Filipino teachers

 What did you like in the activity?
- The topic is very applicable in test construction and testing.
- it's a different way to analyze items (basically multiple choice) although the
concern about why do it tediously needs to be addressed (aside from the fact
that the teacher gets to use the improvements needed in the test)
- I really like the lecture part. the speaker explained it very well in very
interesting manner. congratulations!
Batch 4 – English & Filipino teachers

What did you dislike in the activity?

- the IRT is ok but would need more time

to familiarize with it.

My suggestions to improve the activity are:

- Longer time for practice/hands on.
- use of varied data so we can practice using the program more.
 SUMMARY
Best Practices in the conduct of IRT:

1. The school sought the assistance of consultants

from De La Salle University , Manila, Dr. Alicia
Bustos-Orosa, to take the lead in the test
development project and Dr. Carlo Magno, for the
statistical treatment and analysis of test data.

2. The Instructional and Performance Assessment

Office (IP A) in collaboration with the Principals and
subject area coordinators, worked closely with the
consultants in relation to planning the in-service
programs for the faculty.
3. Two of the IPA personnel were trained to
prepare the test data needed by Dr. Magno,
which was followed by consultative meetings
on the technical analysis of test data.

4. The administration provided the financial

support needed for the conduct of the IRT
by allotting a budget for the purchase of a
scanner, software for making answer
sheets, software for IRT, training the faculty
and hiring additional personnel to perform
the tasks.
5. The IPA technical support staff did the IRT for
the achievement tests administered in February
and March 2009 under the supervision of Dr.
Magno.

6. The training for the IPA personnel, principals,

and subject area coordinators was held on
November 7, 2009. This was followed by training
for Science and Math teachers on December 5,
2009. The CL and Social Studies teachers had their
turn on June 26, 2010. The last group, the English
and Filipino teachers, had their training on July 24,
2010.
7. The items in the first end term
exams (September 15-17) will be
analyzed using the IRT approach
and this will be done by the
teachers with the assistance of the
IPA Office.
 IMPLICATIONS
1.As a whole, the academic community is
receptive and open to trying out new
approaches to improve their craft. It is therefore
easier for DLSZ to introduce innovations.

2.The IRT results can be further validated by

looking into other factors that have influenced
the results.
 IMPLICATIONS

3. Since there are still revisions required to

improve the test items, seminar- workshops
specific for the subject areas identified are
recommended.

4. To further practice using the IRT,

opportunities should be provided to the
teachers, not just limited to end-term
examinations.
 “Item writing is an art. It requires an uncommon
combination of special abilities. It is mastered
only through extensive and critically supervised
practice. It demands and tends to develop high
standards of quality and a sense of pride in
craftsmanship.”

Robert Edel

Thank you. God bless.