Welcome to Scribd, the world's digital library. Read, publish, and share books and documents. See more
Download
Standard view
Full view
of .
Look up keyword
Like this
1Activity
0 of .
Results for:
No results containing your search query
P. 1
Anticipating Data Collection Issues for Rigorous Evaluation

Anticipating Data Collection Issues for Rigorous Evaluation

Ratings: (0)|Views: 30 |Likes:

More info:

Published by: wested-magnet-schools on Jan 16, 2009
Copyright:Attribution Non-commercial

Availability:

Read on Scribd mobile: iPhone, iPad and Android.
download as DOC, PDF, TXT or read online from Scribd
See more
See less

11/15/2013

pdf

text

original

Practice:
Get quality data into your evaluator\u2019s hands
Key Action:
Troubleshoot data issues as they emerge
TOOL: Anticipating Data Collection Issues for Rigorous Evaluation
Purpose:

As you plan and begin your data collection, it is
important to anticipate issues common to rigorous
evaluations of magnet programs. Use this tool to think
about and address specific issues that may negatively
impact your evaluation findings.

Instructions:

1. Review the list to learn about common data
collection issues that may negatively impact your
evaluation findings.

2. Based on the context of your particular district and
your understanding of the data collection issues,
identify the issues that are most likely to affect your
evaluation that need proactive problem solving.

3. Determine how your evaluation team will take
measures

to address these issues and/or build in
checkpoints to routinely monitor these issues
throughout the process of data collection.

1
Practice:
Get quality data into your evaluator\u2019s hands
Key Action:
Troubleshoot data issues as they emerge
Anticipating Data Collection Issues for Rigorous Evaluation
Rigorous evaluation issue

How this may
impact our
evaluation

Ways to address
the issue
Lack of treatment fidelity: To accurately assess

the impact of the magnet program, you must
determine the extent to which key elements (or
\u201ctreatment\u201d) of a magnet program were
implemented as originally intended. Before you
begin data collection, figure out how you will
measure \u201ctreatment fidelity,\u201d or the degree to which
planned program activities are conducted and
participants are reached. In other words, be clear
about how you will know if you are doing what you
said you would do.

Treatment \u201ccross-over\u201d in control or
comparison group: The control or comparison

group may engage in some or all of the elements of
the magnet program being evaluated. Indeed, few
schools or classrooms are intervention-free, and it
is difficult to anticipate all the ways best educational
practices may reach students. It is also difficult to
predict ahead of time which schools will adopt what
programs within the time frame of the evaluation
study. For this reason, it is important to be clear up
front with control and comparison schools about the
purpose of the evaluation and the need to limit
treatment crossover. You will also need to keep
regular close contact with all groups to document
what they are doing.

Selection bias: In a quasi-experimental design,

where students are not selected through random
assignment, statistical techniques may account for
some differences between the treatment and
comparison groups. Evaluators will try to make the
best matches between schools and individual
students to control for variables. However, when
students choose to participate in magnet programs,
it can be difficult to eliminate the effect of this
selection bias on student outcomes. You need to
minimize the effect of difficult-to-measure
differences between your treatment and
comparison group (e.g., level of motivation, parent
involvement).

Adapted from: U.S. Department of Education, Office of Safe and Drug-Free Schools. (2007). Mobilizing for evidence-based character
education (p. 36). Washington, DC: Author. The entire guide can be downloaded at www.ed.gov/programs/charactered/mobilizing.pdf
(last accessed December 10, 2008).
2
Type here

You're Reading a Free Preview

Download
/*********** DO NOT ALTER ANYTHING BELOW THIS LINE ! ************/ var s_code=s.t();if(s_code)document.write(s_code)//-->