or Trut

The John Locke Foundation

is a501(c)(3) nonprot, nonpartisan researchinstitute dedicated to improving public policy debate in North Carolina. Viewpoints expressed by authors do not necessarilyrefect those o the sta or board o the Locke Foundation.

200 W. Morgan, #200Raleigh, NC 27601

phone:

919-828-3876

ax:

919-821-5117www.johnlocke.org

spot

light

id dropout prevention grants work?Because o research limitations, the politicians and advocates whoclaim that dropout prevention grants have worked cannot substanti-ate that claim. The problem is that they cannot establish a causal connectionbetween the grant program and changes in the dropout rate.

While a numbero grant recipient schools had lower dropout rates, there is no evidence that thegrants themselves were the primary cause or the decline. Similarly, there is noevidence that the grants generated any increases in school dropout rates.Indeed, correlation does not suggest causation. As one well-regarded re-search textbook points out, “Due to lack o randomization, manipulation, andother types o control characteristics o experimental studies, it is difcult toestablish cause-eect relationships with any great degree o confdence.”

A number o other actors – existing districtwide initiatives, school policy chang-es, higher quality teachers and administrators, etc. – may have accounted orlower dropout rates among grant recipient schools. The burden o proo allson the deenders o the dropout grant initiative, particularly the Democraticleadership o the North Carolina state legislature who claim that the millionso dollars spent on the programs have directly lowered dropout rates.Rather than make an earnest attempt to evaluate the programs, deenderso the initiative complain that independent evaluations o the grant programsare invalid because “schools didn’t receive the money until halway throughthe school year.”

First, a number o these programs were already in existence

No. 371 – March 24, 2009

ropout

revention

rants

Legislators need to rethink their approach to the dropout problem

key acts:• Onl 14 o h 100 hool h rivd rvirom dropou prvnion grn ripin hd ubnill lowr dropour nd highr grduion r rom h 2006-07 o h 2007-08 hoolr.• O h fv p o ripin wrdd grn, grn o non-prof orgni-zion pprd o hv h mo u.• urhr mpiril rrh i rquird o g  ompl piur o hrlionhip, i n, bwn dropou prvnion grn nd dropou r.

more >>

when they were awarded a grant, so there was no “up and running” period. More importantly, the act remains thatlegislators expected grant recipients to demonstrate the eectiveness o their programs by the end o the 2008 calen-dar year (when their grant unding expires or is renewed).

The legislation reestablishing the committee on dropoutprevention says,“SECTION 7.14.(e) Evaluation. – The Committee shall evaluate the impact o the dropout pre-vention grants awarded under S.L. 2007-323 and under this section. In evaluating the impacto the grants, the Committee shall consider … the extent to which the program or initiative hasimproved students’ attendance, test scores, persistence, and graduation rates…”

The mandatory progress report instructed grant recipients to provide a[l]ist o names o schools where students are enrolled. Include the our-year cohort graduationrate or each school and LEA. I program/initiative is serving middle school students, includethe our-year cohort graduation rate o their recipient high schools.”

Indeed, one o the only ways that grantees could have demonstrated the eectiveness o their programs wouldhave been to show that they raised graduation rates or the 2007-08 school year.Furthermore, in my previous study o the dropout prevention grants, I discussed the pitalls o unding a programor a year or two i one cannot adequately measure its eectiveness. In the conclusion o the report, I pointed out,The results outlined in Tables 1–3 should not suggest that the grant programs directly low-ered or raised graduation rates, but they do suggest a troublesome downward slide in districtrates that the dropout grants were designed to stop. Further research will be required to get acomplete picture o the relationship, i any, between dropout prevention grants and graduationrates.There are two obstacles to that research eort. First, many o the dropout prevention grantswill not register an immediate, quantifable eect on a district or school graduation rate be-cause the programs were designed to reach at-risk elementary and middle school students.More importantly, it will be difcult or grant recipients to establish direct, causal connectionsbetween the dropout prevention programs and district or school graduation rates. Programsshould not receive additional unding and/or replication based on anecdotal evidence. Instead,grant recipients should be able to quantiy their program’s ability to retain students and sig-nifcantly increase the district or school graduation rate.

One o the purposes o my previous study was to point out how unreasonable legislative expectations and claimswere. In that study, I acknowledged how oolish it was or state legislators to expect grantees to demonstrate immedi-ate results. Nevertheless, this and previous evaluations o the grants are simply compiling data required under statelaw and urther mandated by the Department o Public Instruction and The Committee on Dropout Prevention.

Dropou Prvnion Grn: Prliminr Rul

Out o the 100 schools examined, 45 improved their dropout rates at a higher rate than their respective schooldistricts. On the other hand, 55 schools ailed to improve dropout rates relative to their districts. O the 45 schoolsthat improved their dropout rates relative to changes in their school district, only 14 also had higher graduation rates.Put simply, only 14 o the 100 schools that received services rom dropout prevention grants had substantially lowerdropout rates and higher graduation rates rom the 2006-07 to the 2007-08 school year (see

appndix

). As a whole, the schools that received dropout prevention grant unds did not appear substantially to raise gradu-ation rates or lower dropout rates compared with district and state averages. From the 2006-07 to the 2007-08 schoolyear, the average graduation rate at grant recipient schools dropped rom 73.1 percent to 71.5 percent, while the

statewide graduation rate increased 0.8 percent during the same period. The average dropout rate or grant recipientschools declined rom 7.2 percent to 6.7 percent, consistent with the average district decline o 0.5 percent. At the sametime, there was a 0.3 percent decrease in statewide dropout rate.

Dropou Prvnion Grn b tp

Among the fve types o recipients awarded grants, grants to non-proft organizations appeared to have the mostsuccess. There are two caveats to this observation. First, there were ew dropout prevention grants awarded to schools,aith-based organizations, or colleges in the frst round o grants, so the sample is too small to determine i these kindso entities could have been successul in lowering the dropout rate. Second, as mentioned above, there is no evidencethat the grants or the type o grants actually caused the dropout rate to change.

tbl 1. Prormn o Non-Prof Grn Ripin (minimum hr hool)

Grant Recipient Schools Avg. Dropout Rate Change(Schools) Avg. Dropout Rate Change(Districts)SchoolsOutperorm Districts? Non-ProtGrant Recipient Rank

Futures or Kids19-1.1-0.5Yes1Cross Country or Youth3-1.0-0.5Yes2 YWCA o Asheville & WesternCarolina7-0.5-0.4Yes3Communities in Schools50.00.4Yes4Harriet Webster Task Force ForStudent Success80.2-0.4No5Operation Homework30.4-0.1No6

Among the non-proft grantees, schools where Futures or Kids, Cross Country or Youth, and the YWCA o Ashe-ville and Western Carolina had the highest improvement (decline) in their dropout rates (see Table 1). Communitiesin Schools did not lower schools’ dropout rates, but unlike the districts where the school implemented the program,it did not raise the dropout rates either. Dropout prevention projects operated by the Harriet Webster Task Force ForStudent Success and Operation Homework may have allowed dropout rates to increase.

tbl 2. Prormn o shool Diri Grn Ripin (minimum hr hool)

Grant Recipient Schools Avg. Dropout Rate Change(Schools) Avg. Dropout Rate Change(Districts)SchoolsOutperorm Districts? Non-ProtGrant Recipient Rank

Harnett County Schools 4-1.7-1.4Yes1Randolph County Schools 3-0.7-0.5Yes2Cleveland County Schools 40.00.5Yes3McDowell County Schools3-0.7-1.0No4Buncombe County Schools 3-0.6-0.9No5Chatham County Schools3-0.4-0.7No6Pitt County Schools 5-0.1-0.5No7Beauort County Schools 30.0-0.4No8Burke County Public Schools 30.2-1.4No9

Among schools and school districts, Harnett and Randolph counties had the greatest improvement in dropout