You are on page 1of 50

www.newyorkprincipals.

org / @scfeeney
Accountability in the Age of
Pearson: Advocating for
What works in Schools
Robert and Augusta P. Finkelstein Memorial Lecture
Adelphi University
Ruth S. Ammon School of Education
30 October 2013

Sean C. Feeney, Ph.D.
Principal
The Wheatley School
Old Westbury, NY 11568

www.newyorkprincipals.org / @scfeeney
Spackenkill HS Town Hall
www.newyorkprincipals.org / @scfeeney
Accountability in
Schools
www.newyorkprincipals.org / @scfeeney
NYS APPR Legislation
! Passed by Legislature in May 2010
! Required student test scores to be
used in evaluating teachers and
principals
! NY SED Guidelines distributed in
April 2011
! Effective in September 2011
www.newyorkprincipals.org / @scfeeney
New York Principals
APPR Position Paper
www.newyorkprincipals.org / @scfeeney
Concern 1:
Research does not support it
1. Value-added models (VAM) of teacher effectiveness do not
produce stable ratings of teachers
a) Measurement errors
b) Inter-temporal stability concerns
c) Year to year correlations for elementary school
teachers are in the 0.2-0.4 range
2. No evidence that evaluation systems incorporating
student test scores produce gains in student achievement
3. NYS exams are designed to evaluate student learning, not
teacher effectiveness or student learning growth
www.newyorkprincipals.org / @scfeeney
Concern 2:
Students will be adversely affected
1) Narrowing of curriculum to focus on end of year
examinations
2) System will be incentivize to place struggling
students in safer classes
3) Teachers will be incentivized to avoid students
with issues — health issues, ELL, emotional
4) Dynamic between students and teacher will
change. Teacher and student versus the exam is
replaced by teacher versus students
performance
5) Collaboration among teachers replaced by
competition
www.newyorkprincipals.org / @scfeeney
Concern 3:
Costs to districts excessive
1) 10 days of APPR training for building principals; 17
days for Superintendents
2) Tax dollars redirected to testing companies for
correction, security and analysis of exams
www.newyorkprincipals.org / @scfeeney
What have we seen since
APPR was instituted?
We have become
Test Crazy!
www.newyorkprincipals.org / @scfeeney
Student Testing
Commissioner John King:
! writes “..the number and frequency of state
assessments has remained relatively constant
over the last decade…”
! and explains that “…a maximum of 540 of
those [64,800 instructional] minutes are
devoted to State tests in Grades 3-8 ELA and
math…”
! while boasting of a decrease in testing in 2013
from 2012
(October 24, 2013 “Teaching is the Core” memo)
www.newyorkprincipals.org / @scfeeney
NYS Testing Minutes in Grades 3-8
(Math and ELA)
Yes…thats 9 hours
of testing!
www.newyorkprincipals.org / @scfeeney
NYS Testing Minutes in Grades 3-8
(Math and ELA)
Total Minutes of Testing (ELA and Math) for NYS Grades 3-8 Examinations
Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8
2008 160 300 165 255 190 320
2009 160 300 165 265 190 320
2010 160 300 170 265 200 320
2011 250 300 260 290 295 305
2012 540 540 540 540 540 540
2013 420 440 540 540 540 540
% Change 163% 47% 227% 112% 184% 69%
RTTT
www.newyorkprincipals.org / @scfeeney
9 Hours of NYS ELA and math Testing!
Test When taken Length
AP exams
Grades 10-12, for college credit 3 hours
SAT exam
Grades 11-12, for college admission 3.3 hours
ACT exam
Grades 11-12, for college admission 3.5 hours
GRE
for graduate school admissions 3.75 hours
LSAT
for law school admissions 3.5 hours
MCAT
for medical school admissions 4.3 hours
CPA
to become a certified public accountant 14 hours
www.newyorkprincipals.org / @scfeeney
But there are more
tests…
www.newyorkprincipals.org / @scfeeney
SLO tests
! In order to evaluate teachers based on
student test scores, more tests are needed!
! Teachers are administering exams based on
Student Learning Objectives (SLOs)
! ALL subjects without a NYS test must have
one of these SLOs
! PE, art, technology, family and consumer
science, kindergarten
! The pre-test / post-test comparison will be
used to evaluate teachers
www.newyorkprincipals.org / @scfeeney
Field tests
! Just after completing two weeks of
testing (for Math and ELA), 3,600
schools were required to administer field
tests to these same students
! SED is very quiet about the
administration of these tests
! The results of the field tests are used to
help Pearson fulfill the terms of their
$32 million NYS contract
www.newyorkprincipals.org / @scfeeney
Lots and lots of tests…
www.newyorkprincipals.org / @scfeeney
What are we doing with
all these tests?
www.newyorkprincipals.org / @scfeeney
Evaluating teachers
! 25% to 40% of a teachers end of year
evaluation will depend on how his or
her students performed on these
tests
! Passing is no longer sufficient;
students must show growth every
year

www.newyorkprincipals.org / @scfeeney
Evaluating principals
! 25% to 40% of a principals end of
year evaluation depend on how
students perform on these tests

www.newyorkprincipals.org / @scfeeney
Improving schools?
Pearson Stock Price, 2009-present
Washington, DC Performance Index (under IMPACT)
www.newyorkprincipals.org / @scfeeney
We are losing our way
! Dylan was in second
grade
! One of five children
! Mom completed 7th
grade

www.newyorkprincipals.org / @scfeeney
Dylan’s Reading Teacher
! Dylan’s report card

www.newyorkprincipals.org / @scfeeney
Dylan’s Reading Test
" Florida Assessment for Instruction in Reading
(FAIR), a computerized assessment
" After three tests in second grade:
" Accuracy: 95%
" Vocabulary percentile rank: 88th
" Spelling percentile rank: 19
th

" Determined that he would need to work on
multisyllabic words
" Probability of Reading Success (PRS) score of
44% (probability of reading at or above grade
level)


www.newyorkprincipals.org / @scfeeney
Test trumps Teacher
! Because he did not “pass” the FAIR
test, Dylan was retained in 2nd Grade
" Despite teacher comments regarding
Dylan’s progress and achievement
as a reader
" Despite the overwhelming evidence
that grade retention simply does not
help students “catch up”


www.newyorkprincipals.org / @scfeeney
Test trumps Reality
! Sept 2009: Dylan’s grandfather dies
! Feb 2010: Dylan’s father is killed
! Jan 2012: Dylan’s Grandmother dies
of cancer
! June 2012: Dylan does not “pass”
the FAIR test


www.newyorkprincipals.org / @scfeeney
What are standardized
tests telling us about
schools and students?
!"#
!%&
!%' !%' !%(
!'!
!() !(#
!(" !(' !(( !(( !((
&*+ &*# &*! &*&
&)*
&)# &)#
&)" &)'
&+)
&+&
&+'
&&*
&%!
!
#!!
$!!
%!!
&!!
'!!
(!!
)
*
+
,
-
+
.

/
-
+
-
.
-

0
+
*
1
2
.

3
*
+
4
.
5
6

7
2
8

9
2
-
,
-
+
.

:
-
;
-
+

<
=
>
1
?
-
,
*
-

7
2
1
@
2
?
,
-
+
.
>

A
2
,
4
*
=
6

7
5
?
8
-
B

C
8
*
1
D
2
?
,
-
+
.

E
5
,
-
+
.

0
+
*
1
2
.

C
1
-
1
2
>

F
G
2
,
-
+
.

H
2
?
6
-
+
B

)
?
-
+
G
2

I
2
+
6
-
?
J

K
=
+
4
-
?
B

E
5
?
1
=
4
-
,

F
?
2
,
-
+
.

F
1
-
,
B

H
?
2
2
G
2

C
;
-
*
+

/
D
2
G
@

L
2
;
=
M
,
*
G

F
>
?
-
2
,

<
=
>
1
?
*
-

/
@
*
,
2

N
2
O
*
G
5

,-./ .0123 45 6178925
EFC< CG5?2
3.4%
16.3%
6.7%
7.3%
12.4%
13.3%
21.7%
Pov. Rate
http://nces.ed.gov/surveys/pisa/
http://nasspblogs.org/principaldifference/2010/12/pisa_its_poverty_not_stupid_1.html
!!"
!$%
!&'
!&(
!&" !&" !&)
!"!
!)* !)%
!)$ !)& !)" !)) !)) !))
(+' (+% (+! (+(
(*+
(*% (*%
(*$ (*"
('*
('(
('" ('"
((+
((%
(&!
0
100
200
300
400
300
600
u
.
S
.

(
<
1
0
°
)

l
l
n
l
a
n
d

u
.
S
.

(
1
0
-
2
4
.
9
°
)

C
a
n
a
d
a

u
n
l
L
e
d

k
l
n
g
d
o
m

n
e
w

Z
e
a
l
a
n
d

!
a
p
a
n

A
u
s
L
r
a
l
l
a

n
e
L
h
e
r
l
a
n
d
s

8
e
l
g
l
u
m

n
o
r
w
a
y

u
.
S
.

(
2
3
-
4
9
.
9
°
)

S
w
l
L
z
e
r
l
a
n
d

Þ
o
l
a
n
d

u
.
S
.

(
A
v
g
.
)

l
c
e
l
a
n
d

C
e
r
m
a
n
y

l
r
a
n
c
e

u
e
n
m
a
r
k

P
u
n
g
a
r
y

Þ
o
r
L
u
g
a
l

l
r
e
l
a
n
d

l
L
a
l
y

C
r
e
e
c
e

S
p
a
l
n

C
z
e
c
h

8
e
p
u
b
l
l
c

l
s
r
a
e
l

A
u
s
L
r
l
a

u
.
S
.

(
3
0
-
7
4
.
9
°
)

C
h
l
l
e

u
.
S
.

(
>
7
3
°
)

M
e
x
l
c
o

,-./ .0123 45678 9:.: ,1;327< =>
ÞlSA Score
<10%
25-49.9%
50-74.9%
21.7%
Pov. Rate
10-24.9%
>75%
http://nces.ed.gov/surveys/pisa/
http://nasspblogs.org/principaldifference/2010/12/pisa_its_poverty_not_stupid_1.html
www.newyorkprincipals.org / @scfeeney
Poverty and US Schools — 2011
http://www.southerneducation.org/Programs/P-12-Program/Early-Ed/NewMajority.aspx
www.newyorkprincipals.org / @scfeeney
http://www.washingtonpost.com/local/education/poverty-increasing-in-public-schools/
2013/10/16/94defd76-36d9-11e3-80c6-7e6dd8d22d8f_graphic.html
www.newyorkprincipals.org / @scfeeney
Teachers who
receive top value-
added ratings are
more likely to be in
schools with fewer
poor students
www.newyorkprincipals.org / @scfeeney
Poverty and “Student Growth” in NJ
http://schoolfinance101.wordpress.com/2013/04/12/friday-am-graphs-just-how-biased-are-
njs-growth-percentile-measures-school-level/
www.newyorkprincipals.org / @scfeeney
“Student Growth” and Proficiency in NJ
http://schoolfinance101.wordpress.com/2013/04/12/friday-am-graphs-just-how-biased-are-
njs-growth-percentile-measures-school-level/
www.newyorkprincipals.org / @scfeeney
Contradictory Information
! At Wheatley, 92% of our 8th Graders take the
Algebra Regents exam. All but 1 passed (99.1%
passing rate)
! 73.5% of these students passed the Algebra Regents
at the “College and Career Readiness” score of 80
YET…
! Only 39% of our 8th graders “passed” the easier
8th grade mathematics assessment
IN FACT…
! 59% of the students categorized as not meeting
standards on 8th grade test subsequently earned a
score of at least 80 on the Algebra Regents
www.newyorkprincipals.org / @scfeeney
A Letter to Parents
about Testing

www.newyorkprincipals.org
www.newyorkprincipals.org / @scfeeney
What We Know
1. NYS Testing Has Increased Dramatically
2. The Tests were Too Long
3. Ambiguous Questions Appeared throughout the Exams
4. Children have Reacted Viscerally to the Tests
5. The Low Passing Rate was Predicted
6. The College Readiness Benchmark is Irresponsibly
Inflated
7. State Measures are Contradictory
8. Students Labeled as Failures are Forced Out of Classes
9. The Achievement Gap is Widening
10. The Tests are Putting Financial Strains on Schools
11. The Tests are Threatening Other State Initiatives
www.newyorkprincipals.org / @scfeeney
What We Do Not Know
1. How these tests will help our students
2. How these tests will help improve student skills or
understanding
3. The underlying cause of low test scores
4. What to expect next year
5. How much this is costing already-strained
taxpayers
www.newyorkprincipals.org / @scfeeney
So if not through tests,
what does
accountability look like?
www.newyorkprincipals.org / @scfeeney
Montgomery County, MD
Peer Assistance and Review (PAR) program
! Master teachers mentor new and struggling
teachers
! Performance standards based on National
Board for Professional Teaching Standards
! If mentoring does not work, PAR panel (8
teachers, 8 principals) can vote to fire a teacher
! Collaborative process built upon support and
trust
! Largest school system in Maryland; 16th largest
in nation (over 144,000 students)
www.newyorkprincipals.org / @scfeeney
Cincinnati Public Schools
Teacher Evaluation System (TES)
! Based on Danielson framework
! Mid-career teachers trained as Peer Evaluators, a
role they assume for three years before returning
to classroom
! 5-year rotation of a comprehensive evaluation
! Research indicates a strong correlation between
TES scores for teachers and difference in student
achievement growth
! Evaluations associated with a significant increase
in student scores in mathematics
www.newyorkprincipals.org / @scfeeney
Massachusetts RTTT Model
Educator Evaluation Model
! Applies to all educators, including teachers, principals and
superintendents
! Four performance standards (similar to Danielson), with
demonstration items that can be provided by teacher
! Four performance ratings: Unsatisfactory, Needs
Improvement, Proficient, Exemplary. Bands, NOT points are
assigned to teachers
! The length and characteristics of an individual’s Educator
Plan varies in length and characteristics: Improvement
Plan, Directed Plan (1-yr), Self-Directed plan (1-yr), Self-
directed plan (2-yrs)
! Trends of multiple measures of student learning outcomes
given to all educators as a lens through which to view the
overall evaluation. This Impact on Student Learning can
change the type of educator plan
www.newyorkprincipals.org / @scfeeney
Final Thoughts
www.newyorkprincipals.org / @scfeeney
What if they are wrong?
! How much damage has been done?
! How much money has been spent?
! How much are we changing schools?
! How are we changing student perceptions of school?
www.newyorkprincipals.org / @scfeeney
Leadership
! After we released our APPR letter, Commissioner King
dismissed our concerns, stating that “It’s not at all
surprising [that the introduction of a new evaluation system]
would produce anxiety.” (NYT, 11/27/11)
! Chancellor Tisch blamed parents for passing test stress on
to their children (Brian Lehrer Show, 8/15/13)
! King blamed NYSUT for the stress related to students taking
the tests (Times-Union, 6/5/13)
! In his “New York State Assessment Update” presentation,
King stated that the increase in testing was all “optional”
and the result of local district decisions
www.newyorkprincipals.org / @scfeeney
Not everything that counts can
be counted, and not everything
that can be counted counts.
(Einstein)
www.newyorkprincipals.org / @scfeeney
What gets measured gets done.
www.newyorkprincipals.org / @scfeeney
What gets nurtured grows.
www.newyorkprincipals.org / @scfeeney
Research Notes
Baker, Bruce T. School Finance 101 Blog (located at
http://schoolfinance101.wordpress.com)
Mitchell, Kenneth (2012) Federal Mandates on Local Education: Costs
and Consequences. Lower Hudson Council of School Superintendents
http://www.newpaltz.edu/crreo/brief_8_education.pdf
King, John (2013). “Teaching is the Core” letter dated 24 October
2013 http://usny.nysed.gov/docs/teaching-is-the-core.pdf
Montgomery County Education Association. Teachers Guide to the
Peer Assistance and Review Program and the Teacher Evaluation
System http://www.mceanea.org/pdf/PAR2012-13MCEAGuide.pdf
New York State Assessment Update (October 2013)
http://www.regents.nysed.gov/meetings/2013Meetings/
October2013/AssessmentUpdate.pdf
O’Donnell, P. and Exner, R. (June 18, 2013) “Grading the Teachers:
Teachers in Richer Schools Score Higher on Value-Added Measure”
http://stateimpact.npr.org/ohio/2013/06/18/grading-the-teachers-
teachers-in-richer-schools-score-higher-on-value-added-measure _

www.newyorkprincipals.org / @scfeeney
Research Notes
Rogers, Thomas (September 2013) Testimony to the Senate Education
Committee Hearing at Suffolk Community College on 9/17/2013
http://www.nysenate.gov/files/Dr.%20Thomas%20Rogers.pdf.
Project on the Next Generation of Teachers, Harvard Graduate School
of Education. A Users Guide to Peer Assistance and Review
http://www.gse.harvard.edu/~ngt/par/resources/
users_guide_to_par.pdf
Samtani, Hiten (23 May 2012) “More Parents Are Saying No to
Pearson’s Field Tests”
Shepard, L. & Smith (1990) “Synthesis of Research on Grade
Retention” Educational Leadership, 84-88.
Southern Education Foundation (October 2013). “A New Majority: Low
Income Students in the South and Nation.”
http://www.southerneducation.org/getattachment/0bc70ce1-
d375-4ff6-8340-f9b3452ee088/A-New-Majority-Low-Income-
Students-in-the-South-an.aspx