Welcome to Scribd, the world's digital library. Read, publish, and share books and documents. See more
Download
Standard view
Full view
of .
Save to My Library
Look up keyword
Like this
0Activity
0 of .
Results for:
No results containing your search query
P. 1
Science

Science

Ratings: (0)|Views: 0 |Likes:
Published by phyco1954

More info:

Published by: phyco1954 on Mar 13, 2012
Copyright:Attribution Non-commercial

Availability:

Read on Scribd mobile: iPhone, iPad and Android.
download as PDF, TXT or read online from Scribd
See more
See less

05/13/2014

pdf

text

original

 
Economics of Education Review 25 (2006) 327–333
Measurement error, education production anddata envelopment analysis
John Ruggiero
Ã
Department of Economics and Finance, University of Dayton, Dayton, OH 45469-2251, USA
Received 30 September 2004; accepted 18 March 2005
Abstract
Data Envelopment Analysis has become a popular tool for evaluating the efficiency of decision making units. Thenonparametric approach has been widely applied to educational production. The approach is, however, deterministicand leads to biased estimates of performance in the presence of measurement error. Numerous simulation studiesconfirm the effect that measurement error has on cross-sectional deterministic models of efficiency. It is also known thatpanel data models have the ability to smooth out measurement error, leading to more reliable efficiency estimates. Inthis paper, we exploit known properties of educational production to show that aggregation can also have a smoothingeffect on production with measurement error, suggesting that efficiency analyses are more reliable than previouslybelieved.
r
2005 Published by Elsevier Ltd.
JEL Classification:
I21
Keywords:
Educational production; Data envelopment analysis
1. Introduction
Data Envelopment Analysis (DEA) is a nonpara-metric mathematical programming approach to themeasurement of efficiency that was introduced in theoperations research literature byCharnes, Cooper, andRhodes (1978)andBanker, Charnes, and Cooper(1984). Using linear programming, an observed decisionmaking unit (DMU) is evaluated relative to theproduction frontier, which consists of combinations of observed production possibilities using minimal assump-tions. The primary advantage of the approach is theability to handle multiple inputs and multiple outputs,particularly in the case when input prices are unavail-able. The linear programming approach has withstoodthe test of time and has become an acceptable approachfor efficiency evaluation.One important application of DEA is to the analysisof educational production. Many states have undergonelegal challenges because school districts are not provid-ing educational services efficiently and outcomes are notadequate. Reform has moved away from traditionalissues like equity to adequacy and efficiency. Theimportant policy implication is that school districtsneed to spend their money more wisely and increasetheir outcomes to acceptable levels. One populartechnique that has been used for measuring efficiencyin education is DEA. Research using DEA to measureperformance of educational production in the UnitedStates includeBessent, Bessent, Kennington, andReagan (1982),Fa ¨re, Grosskopf, and Weber (1989),
ARTICLE IN PRESS
www.elsevier.com/locate/econedurev0272-7757/$-see front matter
r
2005 Published by Elsevier Ltd.doi:10.1016/j.econedurev.2005.03.003
Ã
Tel.: +19372292258; fax: +19372292477.
E-mail address:
ruggiero@notes.udayton.edu.
 
Ray (1991),McCarty and Yaisawarng (1993),Ruggiero (1996, 2001), andDuncombe, Miner, and Ruggiero(1997). DEA studies analyzing performance in othercountries includeSilva Portela and Thanassoulis (2001),Farren (2002)andMun˜iz (2002). The ability of DEA to measure efficiency is dependenton the nature of observed production. Numerous studieshave shown that the performance of DEA deterioratesin the presence of measurement error and otherstatistical noise. Simulations inBanker, Gadh, and Gorr(1993),Ruggiero (1999), andBifulco and Bretschneider (2001), for example, clearly show that deterministicmodels including DEA are sensitive to measurementerror in cross-sectional analyses. Given an econometri-cian’s view of the world, the use of DEA could beproblematic and potentially harmful given the importantpolicy implications.Gong and Sickles (1992)andRuggiero (2004)use simulation to show that theproblem of measurement error is alleviated whenefficiency is estimated using panel data. These modelsassume that the production technology is the sameacross time and use averaging to smooth productionwith respect to measurement error.In a separate but related literature,Hanushek (1979)discusses other important issues with empirical educa-tional production analyses. One of the important issuesinvolves the choice of aggregation. Conceptually,education occurs at the student level, but most analysesrely on aggregated data (primarily due to availability)to analyze educational production. One potentialadvantage, which is exploited in this paper, is thesmoothing of production with respect to measurementerror. Hence, aggregation, like panel data, helpsalleviate the problem of measurement error. In parti-cular, we assume that educational production withmeasurement error occurs at a disaggregated level andconsider efficiency measurement using both disaggregateand aggregate data.Simulation analysis is used to show that the use of DEA on disaggregated data in the presence of measure-ment error leads to biased efficiency estimation,confirming the results of previous analyses. We alsoapply DEA to aggregated data and show that aggrega-tion can lead to unbiased efficiency estimates. Theseresults represent an important contribution to the DEAliterature: aggregation effectively controls for statisticalnoise. This suggests that DEA can be used effectively toidentify inefficiency of school districts (schools), wheredata are aggregated from the school (classroom) level.We note that such aggregation can be applied to otherareas such as analyzing bank performance whereproduction occurs at the branch level. For purposes of this paper, we consider district data as the aggregate of school data. The rest of the paper is organized asfollows. The next section provides a model of educa-tional production and efficiency using DEA. The thirdsection analyzes the performance of DEA using simula-tion analysis and the last section concludes.
2. Educational production
We describe the educational production process asfollows. Assume that each of 
s
schools uses a vector
¼ð
x
1
; :::;
x
Þ
of 
discretionary school inputs toproduce a vector
¼ð
 y
1
; :::;
y
Þ
of 
outcomes. Forschool
, inputs are given by
 j 
¼ð
x
1
 j 
; :::;
x
Mj 
Þ
andoutputs by
 j 
¼ð
 y
1
 j 
; :::;
y
Tj 
Þ
. For purposes of thispaper, we ignore the role of non-discretionary socio-economic variables to focus on measurement error andaggregation. This further allows comparisons to theapproach used by Bifulco and Bretshneider (2001).Ruggiero (1998)extended DEA to the case of multiplenon-discretionary inputs; allowing measurement errorwith non-discretionary inputs is a trivial extension of Ruggiero’s model using the approach of this paper.Assuming variable returns to scale, technical effi-ciency can be estimated at the school level using theBanker, Charnes, and Cooper (1984)input-orientedmodel:Min
y
o
l
1
;:::;
l
S
s
:
t.
X
S
¼
1
l
 y
ti 
X
 y
to
8
t
¼
1
; :::;
X
S
¼
1
l
x
mi 
p
y
o
x
mo
8
m
¼
1
; :::;
X
S
¼
1
l
¼
1
l
!
0
8
¼
1
; :::;
S
.
ð
1
Þ
In the absence of measurement error and with asufficiently large sample size, this model works well inmeasuring relative performance with multiple inputs andoutputs. The returns to scale assumption can betightened to allow only constant returns by excludingthe convexity constraint; leading to theCharnes,Cooper, and Rhodes (1978)model. The analysis of agiven school seeks a maximum equi-proportional con-traction of all inputs consistent with frontier productionwith all outputs at least as high as the levels of the schoolunder analysis. Alternatively, we could have focused onthe output oriented models.Model (1) leads to biased efficiency estimates in thepresence of measurement error. Numerous simulationanalyses confirm that even moderate levels of measure-ment error can lead to severe bias. For this reason, theusefulness of DEA to analyze educational productionmust be questioned. Consider measurement error in one
ARTICLE IN PRESS
J. Ruggiero / Economics of Education Review 25 (2006) 327–333
328
 
output
, with
~
 y
 j 
¼
 y
 j 
þ
 j 
, where
~
 y
 ji 
¼
 y
 ji 
þ
 ji 
for school
. We note that this extendsRuggiero (2004)byconsidering additive error in the output. The BCCmodel for measuring efficiency of a given school is thengiven byMin
y
o
l
1
;:::;
l
S
s
:
t.
X
S
¼
1
l
 y
ti 
X
 y
to
8
t
¼
1
; :::;
;
t
a
 j 
X
S
¼
1
l
~
 y
 ji 
X
~
 y
 jo
X
S
¼
1
l
x
mi 
p
y
o
x
mo
8
m
¼
1
; :::;
X
S
¼
1
l
¼
1
l
X
0
8
¼
1
; :::;
S
.
ð
2
Þ
As discussed inRuggiero (2004), measurement errorbiases efficiency estimation for two reasons. First, theright-hand side of the contaminated output constraintdistorts the measurement by comparing frontier outputto a level of output not consistent with that school. Thisresults from measurement error for the school underanalysis. The second effect involves the location of the‘‘frontier output’’; measurement error in other schoolswill lead to a biased location of the true frontier. For afurther discussion as it relates to panel data, seeRuggiero (2004).We consider the aggregation of the data to the schooldistrict level. We assume that each of the
S
schools canbe assigned to one and only one of the
D
schooldistricts. Defining
S
as the number of schools indistrict
for
¼
1
; :::;
D
, we construct an index thatidentifies the school according to the district to which itbelongs. Then, for each district
, we can define a vectorfor each input
m
and output
t
given by
md 
¼ð
x
md 
;
1
; :::;
x
md 
;
S
Þ
and
td 
¼ð
 y
td 
;
1
; :::;
 y
td 
;
S
Þ
;
respectively.We note that
D
¼
P
D
¼
1
S
prior to estimatingefficiency, we first average school level data to thedistrict level for each district
¯ x
md 
¼
1
S
X
S
¼
1
x
md 
;
8
m
,
¯ 
y
td 
¼
1
S
X
S
¼
1
 y
td 
;
8
t
a
 j 
;
and
¯ 
y
 jd 
¼
1
S
X
S
¼
1
~
 y
 jd 
;
:
With the additional assumption that the
ð
Þ¼
0, theerror term is averaged out with a sufficiently largenumber of schools in the district. In the simulationanalysis performed in the next section, we vary thenumber of schools within a district to be 5, 10 or 20 andshow that averaging data even with few schools caneffectively eliminate measurement error problems. Oneconcern raised by an anonymous reviewer was theassumption of aggregating from the school level to thedistrict level. We note that state policy typically funds atthe district level and compares district performance onstate tests. And, in cases where this is not true,aggregation could happen from the student or classroomlevel to the school level. For space consideration, weshow only the aggregation from the school to the districtlevel; the results from a lower level to the school levelfollow.
3. Simulation analysis
We assume that production can be characterized bythe conversion of discretionary inputs into output. Asmentioned above, non-discretionary inputs can beeffectively incorporated into the DEA model (seeRuggiero, 1998). In this simulation, we focus exclusivelyon the impact of measurement error. In particular, weassume that three inputs
x
1
,
x
 2
and
x
3
are used byschools to produce one output
y
according to thefollowing stochastic production function:
 y
¼
u
À
1
vx
0
:
41
x
0
:
42
x
0
:
23
,where
u
and
v
represent inefficiency and statistical noise,respectively. Input data were generated uniformly on theinterval (5,10) for each input. Measurement error wasgenerated log-normal with standard deviation
s
v
where
s
v
took on values 0.1, 0.2 and 0.3. Notably,
s
v
¼
0
:
2represented high measurement error in theBifulcoand Bretschneider (2001)and hence, we consider aneven higher measurement error variance. Theinefficiency component ln
u
was generated half-normalwith standard deviation 0.2. By varying the standarddeviation of the measurement error, we effectivelyallow varying ratios of measurement error toinefficiency variances. Finally, we consider variouscombinations of schools and school districts; we varythe number of schools within each district to take onvalues of 5, 10 and 20 and we allow 50, 100 and 200districts.Initially, each school within a given district had thesame level of inefficiency. In the last simulation, weallow school inefficiency to vary within a district. Threemeasures of performance are considered in this study:mean absolute deviation (MAD) between true andmeasured efficiency, the correlation coefficient and therank correlation coefficient. Because the assumed
ARTICLE IN PRESS
J. Ruggiero / Economics of Education Review 25 (2006) 327–333
329

You're Reading a Free Preview

Download
/*********** DO NOT ALTER ANYTHING BELOW THIS LINE ! ************/ var s_code=s.t();if(s_code)document.write(s_code)//-->