Module 1 A

ENHANCEMENT OF TEACHER ASSESSMENT COMPETENCIES

Dr. Marilyn U. Balagtas

University Professor, Philippine Normal University
President, PEMEA

Dr. Adonis P. David

Professor, Philippine Normal University
Board of Trustee, PEMEA

Dr. Carlo P. Magno

President and CEO, Center for Learning & Assessment Development-Asia
Ex-Officio President, PEMEA
 Session Target

The participant is expected to plan

one’s actions on identified strengths and
weaknesses among assessment
competencies required of teachers.
 Session Activities
Duration Activity In-charge
October 27-29 Pre-training Survey of the Teacher Assessment Literacy Inventory (TALI) Asynchronous at the
with feedback given in the MS Form LMS
Synchronous - October 30 - (Morning)
60 mins Discussion of the Assessment Literacy Inventory Framework and Its Dr. Marilyn Balagtas
Design in Online MS Form
1.5 hours Discussion of the Items in the Assessment Literacy Inventory Dr. Adonis David

1 hour Planning the Enhancement of Assessment Literacy and the Task for the Dr. Carlo Magno
Asynchronous Session
15 mins Planning the Professional ePortfolio Dr. Marilyn Balagtas

15 mins Q&A ALL

1:00 – 5:00 Asynchronous Session in the LMS
 Specific Session Target
The participant is expected to share
reflections on the process in the design,
development, evaluation, and
communication of the results of the Teacher
Assessment Literacy Inventory to illustrate
the use of assessment in informing teaching
and learning.
What is this Teacher
Assessment Literacy
Inventory (TALI)?
 Assessment Standards for Teachers
Source: National Council on Measurement in Education,
National Education Association & American Federation of Teachers (1990)

What assessment standards should a teacher possess when assessing

students’ learning?

v A teacher should be skilled in:

1. choosing assessment methods appropriate for instructional decisions

2. developing assessment methods appropriate for instructional decisions

3. administering, scoring, and interpreting the results of both externally-

produced and teacher-produced assessment methods
 Assessment Standards for Teachers
Source: National Council on Measurement in Education,
National Education Association & American Federation of Teachers (1990)

4. using assessment results when making decisions about individual

students, planning teaching, developing curriculum, and school
improvement.
5. developing valid student grading procedures which use student
assessments.
6. communicating assessment results to students, parents, other lay
audiences, and other educators.
7. recognizing unethical, illegal, and otherwise inappropriate
assessment methods and uses of assessment information.
 Development and Evaluation of the Teacher Assessment Literacy Inventory (TALI)
D E V E L O P M E N T
Adapt an Design the TOS Review the TOS Construct
Define International Consolidate
Review the based on the based on the Items based
the Items
Teacher Framework Adapted on the TOS
Framework Adapted
from
Assessment for Teacher Framework Framework with SOLO
Assessment vis-à-vis PPST Experts
Literacy and SOLO And SOLO Framework
Literacy

E V A L U A T I O N
Revise based Final Review of
Create the Validate Analyze the on the TALI in the
Content Language Pilot Test the
Online the TALI in Psychometric Psychometric Online
Validate Validate the TALI in the
Platform for the Properties of Properties of Platform for
the Items Items Platform
the TALI Platform TALI TALI Administration

Use the TALI to inform Professional Development Programs on Student Assessment

 1-Way Table of Specifications for the Teacher Assessment Literacy Inventory (TALI)
International Standards for Teacher Philippine Professional Standards for Teachers (PPST) Placement of
No. of Items
Competence in Assessing Students for Proficient Teachers % of Items Items
(AFT, NCME, & NAE, 1990) Domain: Assessment and Reporting
Choose assessment methods Design, select, organize and use diagnostic,
1
appropriate for instructional decisions formative and summative assessment
4 16% 1-4
5.1.2
Develop assessment methods strategies consistent with curriculum
2
appropriate for instructional decisions requirements.
4 16% 5-8
Administer, score, and interpret the
Monitor and evaluate learner progress and
3 results of both externally-produced and 5.2.2
achievement using learner attainment data.
4 16% 9-12
teacher-produced assessment methods
Use assessment results when making
Utilize assessment data to inform the
decisions about individual students,
4
planning teaching, developing curriculum,
5.5.2 modification of teaching practices and 4 16% 13-16
programs
and school improvement
Develop valid student grading procedures Use strategies for providing timely, accurate
5 which use student assessments 5.3.2 and constructive feedback to improve learner 3 12% 17-19
performance
Communicate assessment results to
6 students, parents, educators, and other
Communicate promptly and clearly the
3 12% 20-22
lay audiences
learners' needs, progress and achievement to
Recognize unethical, illegal, and 5.4.2
key stakeholders, including
otherwise inappropriate assessment
7
methods and uses of assessment
parents/guardians. 3 12% 23-25
information
Total 25 Items 100% 25 items
 1-Way Table of Specifications for the Teacher Assessment Literacy Inventory (TALI)
International Standards for Teacher Philippine Professional Standards for Teachers (PPST) Placement of
No. of Items
Competence in Assessing Students for Proficient Teachers % of Items Items
(AFT, NCME, & NAE, 1990) Domain: Assessment and Reporting
1 Choose assessment methods appropriate for 4 16% 1-4
Design, select, organize and use diagnostic, formative
instructional decisions
5.1.2 and summative assessment strategies consistent with
2 Develop assessment methods appropriate for curriculum requirements. 4 16% 5-8
instructional decisions
Administer, score, and interpret the results of both Monitor and evaluate learner progress and achievement
3 externally-produced and teacher-produced 5.2.2 4 16% 9-12
using learner attainment data.
assessment methods

Use assessment results when making decisions Utilize assessment data to inform the modification of
4 5.5.2 4 16% 13-16
about individual students, planning teaching, teaching practices and programs
developing curriculum, and school improvement
Develop valid student grading procedures which Use strategies for providing timely, accurate and
5 5.3.2 3 12% 17-19
use student assessments constructive feedback to improve learner performance
6 Communicate assessment results to students,
parents, educators, and other lay audiences Communicate promptly and clearly the learners' needs, 3 12% 20-22
Recognize unethical, illegal, and otherwise 5.4.2 progress and achievement to key stakeholders, including
7 inappropriate assessment methods and uses of parents/guardians.
3 12% 23-25
assessment information
Total 25 Items 100% 25 items
Structure of Observed Learning Outcomes (SOLO) Framework
Relational 3 Points
Multi-structural 2 Points
100%
Uni-structural 1 Point 75 Points maximum
Pre-Structural 0 Point
 1-Way Table of Specifications for the Teacher Assessment Literacy Inventory (TALI)
International Standards for Teacher Competence in Philippine Professional Standards for Teachers (PPST) for Proficient
No. of Items Placement of Items
Assessing Students Teachers % of Items
(AFT, NCME, & NAE, 1990) Domain: Assessment and Reporting
1 Choose assessment methods appropriate for
Design, select, organize and use diagnostic, formative 4 16% 1-4
instructional decisions
5.1.2 and summative assessment strategies consistent with
2 Develop assessment methods appropriate for
curriculum requirements. 4 16% 5-8
instructional decisions
3 Administer, score, and interpret the results of Monitor and evaluate learner progress and
both externally-produced and teacher-produced 5.2.2 4 16% 9-12
achievement using learner attainment data.
assessment methods
4
Use assessment results when making decisions Utilize assessment data to inform the modification of
about individual students, planning teaching,
5.5.2 4 16% 13-16
teaching practices and programs
developing curriculum, and school improvement
5 Develop valid student grading procedures which Use strategies for providing timely, accurate and
use student assessments
5.3.2 3 12% 17-19
constructive feedback to improve learner performance
6
Communicate assessment results to students, 3 12% 20-22
parents, educators, and other lay audiences Communicate promptly and clearly the learners'
7 Recognize unethical, illegal, and otherwise 5.4.2 needs, progress and achievement to key
inappropriate assessment methods and uses of stakeholders, including parents/guardians. 3 12% 23-25
assessment information
Total 25 Items 100% 25 items
SOLO Scoring
Relational 3 Points
Multi-structural 0
100%
Uni-structural 0 75 Points maximum
Pre-Structural 0
 What is SOLO?
vSOLO to mean Structure of Observed Learning Outcomes
was popularized by Kevin F. Collis and John B. Biggs in
1982 in their book “Evaluating the Quality of Learning: The
SOLO Taxonomy.

v It is a framework for understanding with five levels, where

each level embraces the previous level but adds
something more.
 What is SOLO?
Understanding/Competence/Proficiency

Source of Illustration: Doug Belshaw - Own work, CC0, https://commons.wikimedia.org/w/index.php?curid=60807631

What is SOLO?
v Use SOLO in structuring responses to questions in a
multiple-choice test format.
v Use SOLO in structuring rubrics in describing responses to
questions in an open-ended or constructed-response type
of test.
v Use SOLO to describe the degree of correctness of
responses given to questions in a test.
v Use SOLO in scoring, interpreting and communicating test
results.
 How is SOLO used in test development?
Levels of Level of Level of
Code Meaning
Proficiency Understanding Understanding

0 One does not know any idea about the lesson

Not Proficient Pre-structural assessed.
Somewhat 1 One knows an idea about the lesson assessed. Surface
Proficient Uni-structural Understanding
Nearly 2 One knows several ideas about the lesson
Proficient Multi-Structural assessed but could not link them together.
3 One knows several ideas about the lesson
Proficient Relational assessed and could link them together.
Deep
Extended 4 One could link several ideas about the lesson Understanding
Highly
assessed and could represent them in a new or
Proficient
Abstract a different way/situation.
 How is SOLO used in test development?
Levels of Level of Level of
Proficiency Understanding
Code Meaning Understanding Test Format
Not 0 One does not know any idea Multiple Constructed-
Proficient Pre-structural about the lesson assessed. Choice Response
Somewhat 1 One knows an idea about the
Proficient Uni-structural lesson assessed. Surface
2 One knows several ideas Understanding
Nearly Multi- about the lesson assessed
Proficient Structural but could not link them
together.
3 One knows several ideas
about the lesson assessed
Proficient Relational and could link them
together.
Deep
4 One could link several ideas Understanding
Extended about the lesson assessed
Highly
and could represent them in a
Proficient Abstract new or a different
way/situation.
 How is SOLO used in test development?
Levels of Level of Level of
Proficiency Understanding
Code Meaning Understanding Test Format Scoring in TALI
One does not know any idea
Not 0
Proficient Pre-structural about the lesson assessed. 0 0
One knows an idea about
Somewhat 1 the lesson assessed.
Proficient Uni-structural Surface 0 1
Understanding
2
One knows several ideas Multiple
Nearly Multi- about the lesson assessed Choice
Proficient but could not link them 0 2
Structural
together.
One knows several ideas
3 about the lesson assessed
Proficient Relational and could link them 3 3
together.
Deep
One could link several ideas
4 about the lesson assessed
Understanding
Highly Extended Constructed
Proficient
and could represent them in a X X
Abstract new or a different Response
way/situation.
 How is SOLO used in test development?
Competency: Choose assessment methods appropriate for instructional decisions

Which of the following is an appropriate assessment method for a teacher who aims to
measure if learners are able to demonstrate a product-type of learning target?
SOLO
Options Explanation/Feedback
Code/Score
A Demonstration with 1 When learners demonstrate a performance task, then skill-type
performance rubric targets are measured. The verb “demonstrate” connotes showing of
a skill that does not necessitate any product.
B Essay with scoring 2 While an essay is an output produced by learners, it is more often
rubric used to measure knowledge and reasoning rather than product-type
learning.
C Multiple-choice test 1 Multiple-choice tests are typically used for measuring knowledge and
reasoning targets and not for measuring product-type learning.
D Feasibility study 3 Feasibility study is an example of product-type learning as it could
produce knowledge and information as a result of learners’ use of
knowledge, reasoning, and skills.
What is
good in
MS Form
used in
TALI?
What is
good in
MS Form
used in
TALI?
What is
good in
MS Form
used in
TALI?
What is
good in
MS Form
used in
TALI?
What is
good in
MS Form The MS Form
in TALI? allows timely,
accurate,
relevant, and
differentiated
feedback to
those being
assessed.
How did we develop the
Teacher Assessment
Literacy Inventory (TALI)?
 What is the profile of the
pilot testing respondents
of the Teacher Assessment
Literacy Inventory (TALI)?
 Sector Age
3% 1%
13%
23%

20 to 29 years old
30 to 39 years old
Public
40 to 49 years old
Private
26% 50 to 59 years old
60 or older

37%
97% Total Number of Total Number of
Respondents: 304 Respondents: 304
 Ki

10
20
30
40
50
60
70

0
nd
er
ga
rt e
n

19
Gr
ad
e
1

28
Gr
ad
e
2

21
Gr
ad
e
3

22
Gr
ad
e
4

25
Gr
ad
e
5

51
Gr
ad
e
6

60
Gr
ad
e
7

41
Gr
ad
e
8
40

Gr
ad
Grade Level/s Handled

e
9
59

Gr
ad
e
10
43

Gr
ad
W e
ith 11
28

Ad
m Gr

30.92% (94) of the respondents handled multiple grade levels.

in ad
ist e
ra 12
tiv
22

e
Po
si
tio
n
9

M
20
40
60
80

0
100
120
140

at
he
m
at
ics
125

Sc
ie
n ce
90

En
gli
sh
130

Fi
lip
Ar in
Ed o
al M
109

uk in
as g Pa
ot
he
yo
n n lip rT
sa u on
na gu
Pa n/ e
62

gp So
a pa cia
h al Sc l
ag ie
a/ nc
Va e
98

lu
es
Ed
uc
at
io
n
Subject Area Handled

129

EP
P/
HE
LE
/T
LE
41

M
AP
Ot EH
he
112

rs
59.87% (182) of the respondents handled multiple subjects.

ub
je
ct
a re
as
37
Highest Educational Attainment
0% 2%

6%
Post doctorate degree
19%

Doctorate degree
17%
Master’s degree with
Doctorate units
Master’s degree

Bachelor’s degree with

Master’s units
Bachelor’s degree
56%
Total Number of
Respondents: 304
 Years in the Teaching Profession Community Type
100 160

90
140
80
120
70

60 100

50
80
89
40 139
60 124
67
30
54
40
20 37
26
10 19 20
12
22 19
0
0 to 2 years 3 to 5 years 6 to 10 years 11 to 15 years 16 to 20 years 25 to 30 years 31 or more 0
years Highly Urban Urban Partially Urban Rural
 No. of Hours of Training in Assessment in the Last 3 Resources at Home for Assessment
Years 300

120

250
100

200
80

150
60
248
236
99
100
40
143
50
20 45
36 39
34
24 28
18 17
9 0
0 Printed books and Digital/electronic Subscription to Subscription to Internet connectiv ity
none 1 to 8 hours 9 to 16 hours 17 to 24 25 to 32 33 to 40 41 to 48 49 or more other readi ng books and other online/electronic assessment and (e.g. WiFi , LAN)
hours hours hours hours hours materials reading materials library other related journal s

75% (228) of the respondents can access multiple resources at home for assessment.
 I am confident that I can I am confident that I can effectively
design/develop online assessment administer/conduct assessment of
tasks and tools for the subjects I learning in an online learning
handle. environment.
0% 0%
5% 9%
22% 18%

Strongly Agree Strongly Agree

Agree Agree
Disagree Disagree
Strongly disagree Strongly disagree

73% 73%
Total Number of Total Number of
Respondents: 304 Respondents: 304
Pilot testing results
were analyzed using
Item Response
Theory to determine:
(1) person and item reliability;
(2) item difficulties;
(3) item fit;
(4) fit of the responses categories
using the SOLO; and
(5) category threshold of the
responses for each item using the
SOLO.
 Thank you!

MARILYN U. BALAGTAS, PhD

University Professor, Philippine Normal University
President, Philippine Educational Measurement and Evaluation Association, Inc.