Supporting Effective Instruction: Technology and Professional Development

Carol McDonald Connor Frederick J. Morrison Christopher Schatschneider Barry Fishman

Coming to PD through the back door
 Individualizing

Student Instruction Study  RFT to test the impact of child characteristics-by-instruction interactions on student outcomes
 Aptitude

by treatment interactions

Designing the PD Protocol

Used the research base to develop a “state of the art” professional development protocol,

delivered to all treatment group teachers

We assumed that, with our coaching, the teachers would be able to implement ISI in the first experimental year with enough fidelity so that we could test our hypotheses

most PD research suggests that it takes 2 to 3 years for teachers to fully implement a new practice

What was the Intervention?
 Created

algorithms based on the HLM results in our Beyond the Reading Wars paper  Assessment to Instruction (A2i) software created to compute recommended amounts and types of instruction for each child
 Embedded

in planning software design to scaffold research-based reading practices

Multiple Dimensions of Instruction
Teacher Managed
Code Focused
Alphabet activities Letter Sight-Sound Phonological Awareness Onset-rime, blending and segmenting Word Segmentation Vocabulary Teacher Read Aloud Student Read Aloud, Choral Group Writing, Writing Instruction, Model Writing Listening Comprehension Discussion

Child Managed
Spelling phonics worksheets, handwriting activities decoding activities

Meaning Focused

Student Buddy Reading Sustained Silent Reading Reading Comprehension worksheets Student Individual Writing Pair writing

Algorithm Results TM-CF

TMCFa = ((End of Year Target - (.2* LW GE))/(.05 + (.05 * LW)))+13. TMCF_Recommended = (TMCFa - (.82 * Month )).

Algorithm Results for CM-MF

CMMFa = ((3.76 – End of year target + (1.4 * Vocabulary AE))/(.30)) - 14. CMMFsl = 10-(.24*CMMFa). CMMF_Recommended= CMMFa + .5*(CMMFsl * Month).

PD Protocol

Focused on using A2i to plan instruction and then on implementing the recommended amounts for each child in the classroom

Grouping
 

small group rather than whole class using A2i recommended groupings (homogeneous)

   

Classroom organization Effective child-managed activities Individualizing time, content, and delivery

no one way to individualize instruction

Research-based practice

Mentor or Coaching Model
    

“Research Partners” classroom based – 2 hours bi-weekly School level meetings Individual meetings Individualized
 

Same amount of time Content and focus varied

Did the PD work?
 Evidence

from child outcomes  Evidence from A2i software tracking  Evidence from changes in teacher knowledge  Evidence from classroom observations

Evidence from Child Outcomes
HLM - Treatment versus Control Student Reading Comprehension Outcomes
40 7 Sp g Pa g rin ssa e Com re e sion WScore p hn 48 6 46 6 44 6 42 6 40 6 Tre tm n a et Con trol Tre tm n a et Con trol

Mean scores controlling for fall vocabulary, passage comprehension, letter-word reading, curriculum, FARL, and Reading First status. 464 = GE 1.8, 468 = GE 2.0, n = 616 students

Evidence from Software Tracking
A2i Use and Reading Comprehension
42 7 40 7

Sp ring Com re n p he sion WScore

AE = 8.2 years
48 6 46 6 44 6 42 6 40 6 Control 9 m 0 in 1 0m 8 in 2 0m 7 in 3 0m 6 in

AE = 6.0 years

HLM fitted growth curves controlling for fall vocabulary, letter-word reading, curriculum, FARL, and Reading First status. 464 = GE 1.8, 468 = GE 2.0,

Evidence from changes in Teacher Knowledge

Teacher Knowledge Test Descriptives and Reliability

  

Assesses teachers’ understanding of English phonology, orthography, and morphology, and concepts of literacy acquisition and instruction 34 multiple choice and 11 short answer items Administered fall and spring Cronbach’s alpha = .87.

Teachers in the treatment group had significantly higher spring TKS scores compared to control teachers
 

Controlling for school SES status, other PD opportunities, teacher credentials and fall TKS scores standardize beta = .40 Fall TKS and A2i total use did not correlate Spring TKS and A2i correlation = .58

A2i and TKS correlations
 

Importance of Teacher Knowledge
Scores on the TKS ranged from 9 to 36 (M = 23.45, SD = 7.27).

First Graders – end of year grade equivalent score of 1.9 = 428

Evidence from Classroom Observations
Child-managed Pair 4.1. Literacy Codes: 4.1.2. Phoneme Awareness 4.1.2. Phoneme Awareness 4.1.3. Syllable Awareness 4.1.4. Morpheme Awareness 4.1.5. Onset/Rime Awareness 4.1.6. Word ID/Decoding 4.1.7. Word ID/Encoding 4.1.8. Fluency 4.1.9. Print Concepts 4.1.2. Phoneme Awareness 4.1.10. Oral Language 4.1.2.2. Blending 4.1.11. Print Vocabulary 4.1.2.3. Elision/Initial 4.1.12. Reading Comprehension 4.1.2.4. Elision/Final 4.1.13. Text Reading 4.1.2.5. Elision/Vowel 4.1.14. Writing 4.1.2.6. Elision/Medial 4.1.15. Library 4.1.16. Assessment 4.1.2.7. Substitution/Initial

4.1.2.8. Substitution/Final 4.1.2.9. Substitution/Vowel 4.1.2.10 Substitution/Medial 4.1.2.11 Segmenting/Counting

TCM Small-group Code-focused

Teacher-Managed Instruction
2 5 2 0 1 5 M /d y in a 1 0 5 0 Tre tm n a et Con trol

Small Group
TM -CF TCM -CF TM F -M TCM F -M

Whole Class

1 8 1 6 1 4 1 2 1 0 8 6 4 2 0 TCM -CF TCM F -M Tre tm n a et Con trol

Child Managed Instruction
2 0 1 8 1 6 1 4 1 2 M /D y 1 in a 0 8 6 4 2 0

Tre tm n a et Con trol

CM -SG-CF CM d -In -CF CM -SG-M CM -In F -SM d M F

Winter Observed – A2i recommended amounts
1 0 5 0 -5 -1 0 -1 5 -2 0 -2 5 TCM -CF TCM F -M CM -CF CM F -M Tre tm nt a e Control

Distance From Recommendation Absolute Values

2 5

Simple Differences
M /D y in a

2 0 1 5 1 0 5 0

*
Tre tm n a et Con trol

TCM -CF

TCM F -M

CM -CF

CM F -M

Effect of Distance from Recommendations
Winter TM-CF DFR
41 7 40 7 49 6 Spring PC Wscore 48 6 47 6 46 6 45 6 44 6 43 6 42 6 41 6 4 3 TCM D (m -CF FT inutes/da y) 0 2 .1

Winter CM-MF DFR
40 6 48 5 Spring LWWscore 46 5 44 5 42 5 40 5 48 4 46 4 6 2 CM F D (m -M FT inute s/da y) 0 1 .0

464 = 1.8, 470 = 2.1 or about a 3 month difference in GE

450 = 1.9, 458 = 2.5 .6 GE or about a 5 and a half month difference in GE

HLM - DFR predicting student outcomes

Cumulative fall, winter and spring DFR for TM-CF and CM-MF

DFR Treatment teachers < DFR Control Teachers

 

Total Amounts of instruction did not predict student outcome growth (residualized change) Cumulative TM-CF and CM-MF DFR negatively predicted both Passage Comprehension and LetterWord Identification residualized change
 

TM-CF DFR amount and change fall to spring CM-MF DFR amount

Discussion

Finding effective ways to change teacher practice quickly are critical
 

For Random Field Trials Improving student learning Practice-based PD versus General Knowledge Focus on A2i, which was designed to scaffold the kinds of instruction research suggests are more effective in improving students’ reading skills Did the technology really make a difference?

Why did our PD work?
 

We don’t know – and will need to conduct a RFT to find out