Welcome to Scribd, the world's digital library. Read, publish, and share books and documents. See more
Download
Standard view
Full view
of .
Look up keyword
Like this
1Activity
0 of .
Results for:
No results containing your search query
P. 1
POLICY EVALUATION VERSUS EXPLANATION OF OUTCOMES IN EDUCATION: THAT IS, IS IT THE TEACHERS? IS IT THE PARENTS? Richard Startz Department of Economics University of California, Santa Barbara Santa Barbara

POLICY EVALUATION VERSUS EXPLANATION OF OUTCOMES IN EDUCATION: THAT IS, IS IT THE TEACHERS? IS IT THE PARENTS? Richard Startz Department of Economics University of California, Santa Barbara Santa Barbara

Ratings: (0)|Views: 7|Likes:
Published by Peter C. Cook
Policy Brief

POLICY EVALUATION VERSUS EXPLANATION OF OUTCOMES IN EDUCATION: THAT IS, IS IT THE TEACHERS? IS IT THE PARENTS?

Richard Startz Department of Economics University of California, Santa Barbara Santa Barbara, CA 9306-920 startz@econ.ucsb.edu

Abstract Education reform advocates who base policy decisions on empirical research often face the argument that because background variables explain so much of student outcomes, it follows that policy interventions cannot be effective. This po
Policy Brief

POLICY EVALUATION VERSUS EXPLANATION OF OUTCOMES IN EDUCATION: THAT IS, IS IT THE TEACHERS? IS IT THE PARENTS?

Richard Startz Department of Economics University of California, Santa Barbara Santa Barbara, CA 9306-920 startz@econ.ucsb.edu

Abstract Education reform advocates who base policy decisions on empirical research often face the argument that because background variables explain so much of student outcomes, it follows that policy interventions cannot be effective. This po

More info:

Published by: Peter C. Cook on Oct 31, 2012
Copyright:Attribution Non-commercial

Availability:

Read on Scribd mobile: iPhone, iPad and Android.
download as PDF, TXT or read online from Scribd
See more
See less

09/25/2014

pdf

text

original

 
Policy Brief 
POLICY EVALUATION VERSUSEXPLANATION OF OUTCOMES INEDUCATION: THAT IS, IS IT THETEACHERS? IS IT THE PARENTS?
Abstract
Education reform advocates who base policy decisionson empirical research often face the argument that be-cause background variables explain so much of studentoutcomes, it follows that policy interventions cannot beeffective. This policy brief explains the logical fallacy intheargument,illustratingwithtwoexamples,onetakenfrom the teacher quality literature and one taken fromthe class size literature.
Richard Startz
Department of EconomicsUniversity of California,Santa BarbaraSanta Barbara, CA 93
06-92
0startz@econ.ucsb.edu
360
c
20
2 Association for Education Finance and Policy
 
Richard Startz
1. INTRODUCTION
Consider the following two statements:
.
Most of the variability in student outcomes is explained by out-of-schoolfactors (e.g., socioeconomic status).
2.
Policy reforms, for example, measures to raise the quality of teachers, areeffective means of improving student outcomes.Both of these statements are true. Unfortunately, the widespread (correct)belief in the importance of out-of-school factors is often used to (incorrectly)argue that this implies there is relatively little room left for effective policymeasures. This brief explains how the importance of out-of-school factors andtheeffectivenessofpolicyaremutuallyconsistent,andthereforethecommonlyadvanced dichotomy between “fix poverty” and “fix schools” approaches isfalse.Advocates of a variety of educational reforms often face opposition basedon the argument that success in school is overwhelmingly explained by back-groundmeasures:socioeconomicstatus,parentalcontributionstowardeduca-tion, community conditions, etc. The argument goes that if background doesa much better job of explaining student outcomes than do teacher quality,class size, school organization, or other inside-the-system items on a reformagenda, there is little point in pursuing a school-based reform agenda. Inpointoffact,researchquiteoftendoesshowthatbackgroundvariablesexplaina much higher fraction of student outcomes than is explained by policy orpublicschoolresources. However, decomposing explanationof outcomesintocomponent parts depends on both the impact of each input and how muchthat input varied in the observed data. In choosing among policies, we careabout the impact of each input (and the respective costs), but how much theinputs have varied historically is not relevant.Considerthisexample.Supposewelookedatstudentoutcomesinaschoolwith uniformly good teachers but large variation in family backgrounds. Allthe variation in student outcomes would be attributable to family background.But this does not speak to what would have happened if the school had evenbetter teachers.Now suppose we had a school in which both teacher quality and familybackground varied and that we were allowed to randomly match teachersand students so that each classroom had students with the same range of family backgrounds. All the variation of student outcomes within a classroomwould still be attributable to different family backgrounds. But the averagedifference in outcomes across classrooms would tell us how much differencea good teacher makes (subject to the usual issues of statistical variation and
361
 
POLICY VERSUS EXPLANATION OF OUTCOMES
the possibility that something other than teacher quality also varied acrossclassrooms). If a good teacher makes a big difference, and if we have aneffectivewayofraisingaverageteacherquality,itisworthinvestinginteacher-oriented policies.The need to untangle the confusion between policy effects and analysisof variance seems to be a permanent feature across a variety of social sciencetopics (see, e.g., Manski 20

). In education there is a long history of respectedpolicy advocates and social scientists focusing on the extent to which variationis explained by nonschool factors. Hanushek and Kain (
972, footnote 3) de-scribe this as the “no-school-effect conclusion.” An early example appears inthe Coleman report:It is known that socioeconomic factors bear a strong relation to aca-demic achievement. When these factors are statistically controlled,however, it appears that differences between schools account for onlya small fraction of differences in pupil achievement.
Hanushek and Kain (
972, p.
36) go on to explain:The closest the [Coleman]
Report 
comes to identifying policy instru-ments are its estimates of the
. . .
contribution of individual variablesto explained variance.
. . .
The proportion of explained variance doesnot identify policy instruments and gives little indication of the policyleverageprovidedbydifferentvariables.Parameterestimatesaremuchmore useful in this respect.
. . .
Explained variance
. . .
is simply not avery interesting concept
. . .
to the policy-maker.Morerecentcommentatorsalsoseguefromtheobservationthatmostofthevariability in student outcomes comes from outside school to the conclusionthat in-school factors, notably quality teachers, cannot accomplish the goals of educational reform (note the final sentence):[A] central thesis in today’s school reform discussion [is] the idea thatteachers are the most important factor determining student achieve-ment. But this proposition is false. Hanushek has released studiesshowing that teacher quality accounts for about 7.5–
0 percent of stu-dent test score gains. Several other high-quality analyses echo thisfinding, and while estimates vary a bit, there is a relative consensus:
. Coleman et al. (
966, pp. 2
22) and cited in Hanushek and Kain (
972, footnote 3). Note that“differences in schools” presumably references differences by race. Ehrenberg and Brewer (
995)show that teacher ability mattered even in the original Coleman data.
362

You're Reading a Free Preview

Download
scribd
/*********** DO NOT ALTER ANYTHING BELOW THIS LINE ! ************/ var s_code=s.t();if(s_code)document.write(s_code)//-->