Page 5: Understand Challenges When Comparing Data

It is crucial for principals to know what data are being reported and to understand the data they are
interpreting. Because state laws require states and districts to report the number of students tested
but do not require them to report enrollment numbers, it is often impossible to determine whether all
or some students with disabilities participated in the tests. It is common for states to:

 Not clearly differentiate scores of students with disabilities

 Not differentiate scores of students who take the test with accommodations versus students
who take an alternate assessment
 Aggregate—or include—the scores from tests taken with “nonapproved” accommodations
with standard test scores
 Not report scores of tests taken with nonstandard accommodations and not indicate that they
are not reporting them

Caution must be used when interpreting data for groups. When the highest performing students in
special education move to general education and the lowest performing students in general
education move to special education, the performance of special education students appears to not
improve over time. It is important to keep track of mobility in and out of special education and to look
at data in a number of ways.

One of the things that you have to be particularly concerned with in making decisions is the effect of
when there aren’t very many students in the group, a small change can make a big difference. And if
a principal is thinking about the performance of students with disabilities in their large scale
assessments, they may look at the performance of a group of students and notice, well, our
performance doesn’t seem to be as good this year as it was last year. Are we making progress as a
school? Or are we making progress as a program? When there are only a half a dozen or a dozen
students in a group, you might only have ten or fifteen students with disabilities in your large-scale
assessment. Any one of those individuals might make a great deal of improvement, might not
improve this year. One student could move in or move away, and now we have a very big change in
our data that are really being caused by a very small, we talk about it as a small n. So we’re always
concerned when we’re looking at the effects of our programs on, particularly groups of students with
disabilities, on the effects of those small groups.

The problem of interpreting data for small groups of students can be somewhat alleviated by
conducting additional assessments, including contextually relevant methods such as portfolios of
student work or progress tests throughout the year.