You are on page 1of 3

Practice:

Get quality data into your evaluator’s hands


Key Action: Manage logistics and communications for data collection

SAMPLE MATERIAL: Timeline for Data Collection and Analysis

Purpose: This sample timeline outlines ongoing tasks, milestones, and


deliverables for data collection and analysis during a district’s magnet
program evaluation. To keep the process on track, the table clearly lays
out all the key tasks for the evaluation team and allocates adequate
time to develop instruments, collect data, and deal with issues that may
impact data analysis. You may find this document useful as you create
or evaluate your own timeline.

Source: From 2007 Magnet Schools Assistance Program grant application,


developed by St. Paul Public Schools (SPPS), in collaboration with their
external evaluators, Mid-continent Research for Education and Learning
(McREL) and Geoffrey Borman, professor of education, University of
Wisconsin-Madison.

1
Practice: Get quality data into your evaluator’s hands
Key Action: Manage logistics and communications for data collection

Table 3: Data Collection and Analysis Timeline

Year 1 Year 2 Year 3


Tasks

Summer 2008

Summer 2009

Summer 2010
--- ongoing work

Spring 2008

Spring 2009

Spring 2010
Winter 2007

Winter 2008

Winter 2009
Ÿinterim milestone

Fall 2007

Fall 2008

Fall 2009
8 final deliverable

Overall Study Design


Finalize study design plans: research (Borman) --
8
(Nov 30)
Finalize study design plans: evaluation (McREL) --
8
(Nov 30)
Finalize randomization plan; construct and sign MOU -- 8
--
(Borman, McREL, SPPS) (Mar/Apr)
Randomly assign students to treatment and control
-- Ÿ
schools
Instruments
Adapt / develop data collection instruments for assessing -- -- 8
(Mar 28)
implementation (McREL / Borman)
Adapt / develop data collection instruments: Teacher -- -- 8
(Mar 28)
Outcomes (McREL / Borman)
Adapt / develop data collection instruments: Student -- -- 8
(Mar 28)
Isolation (McREL / Borman)
Adapt / develop data collection instruments: Student -- -- 8
(Mar 28)
Engagement (McREL / Borman)
Adapt / develop data collection instruments: Student -- -- 8
(Mar 28)
Preparedness for Future (McREL / Borman)
Data Collection
Download student data (MCA-II); screen and clean -- -- Ÿ -- -- -- Ÿ -- -- -- Ÿ
(Aug) (Aug) (Aug)
(SPPS)
Download student data (SAT-10); screen and clean -- -- Ÿ -- -- -- Ÿ -- -- -- Ÿ
(SPPS)

2
Practice: Get quality data into your evaluator’s hands
Key Action: Manage logistics and communications for data collection

Year 1 Year 2 Year 3


Tasks

Summer 2008

Summer 2009

Summer 2010
--- ongoing work

Spring 2008

Spring 2009

Spring 2010
Winter 2007

Winter 2008

Winter 2009
Ÿinterim milestone

Fall 2007

Fall 2008

Fall 2009
8 final deliverable

Collect IMPLEMENTATION data (McREL, SPPS) -- -- -- Ÿ -- -- -- Ÿ -- -- Ÿ

Collect TEACHER data (McREL, SPPS) Ÿ Ÿ Ÿ

Collect STUDENT data (incl. attendance data, honors Ÿ Ÿ


Ÿ
course enrollment data)
Analysis
Analyze descriptive data (McREL) -- -- -- -- -- -- -- -- Ÿ

Analyze student achievement data (Borman, McREL) -- -- -- -- -- -- -- -- Ÿ

Reporting
Write Summary Report (McREL, Borman) 8 (Feb) 8 (Feb)
Write Annual Performance report (McREL, Borman) 8 -- 8
(Jul) (Jul)
Integrate findings from descriptive and inferential -- -- -- -- Ÿ
analyses (McREL, Borman)
Write final report (McREL, Borman) -- -- -- -- 8

As shown in Table 3, deliverables include the following:


o Research Plan (Borman, November 30, 2007)
o Evaluation Plan (McREL, November 30, 2007)
o Randomization Plan (Borman, McREL & SPPS, February 29, 2008)
o Memoranda of Understanding (SPPS, February 29, 2008)
o Start-up Year Summary Report (per Rigorous Evaluation direction, February 29, 2008)
o Instrument development will be collaborative and ongoing; the following are targeted due dates:
ƒ Instrument: Implementation (McREL, March 28, 2008)
ƒ Instrument: Teacher Outcomes (McREL, March 28, 2008)
ƒ Instrument: Student Outcomes (isolation, engagement, preparedness for future) (McREL, March 28, 2008)
o Year 1 Annual Performance Report (McREL, Borman, July 2008)
o Year 2 Summary Report (per Rigorous Evaluation direction, February, 2009)
o Interim Report (McREL, Borman, September, 2009)
o Year 2 Annual Performance Report (McREL, Borman, July 2009)
o Final Report (McREL, Borman, June 2010)