Regression Discontinuity Designs RegressionDiscontinuityDesigns

EITM2011
Chris Berry ChrisBerry
Todays Agenda Today sAgenda
I Methodological Overview I. MethodologicalOverview
II. DiscussionofExemplaryPapers
i i G id III. PractitionersGuide
IV. Stataexamples
METHODOLOGICAL OVERVIEW
RegressionDiscontinuity
METHODOLOGICALOVERVIEW
Important References ImportantReferences
RDmethodswereintroducedbyThistlewaite andCampbell
(1960) See Cook (2008) for an historical perspective (1960).SeeCook(2008)foranhistoricalperspective.
Recentapplicationsinpoliticsincludeanalysesofthe
incumbencyeffect(Lee,2008),electoralcompetitionon
policy (Lee Moretti and Butler 2004) the effect of gender policy(Lee,Moretti,andButler2004),theeffectofgender
onlegislatorbehavior(Rehavi nd),thevalueofaseatinthe
legisalture (EggersandHainmueller 2009),theeffectof
mayoral party ID on policy (Ferreria and Gyourko 2009). mayoralpartyIDonpolicy(Ferreria andGyourko 2009).
Recentimportanttheoreticalworkhasdealtwith
identificationissues(Hahn,Todd,andVanDer Klaauw,2001),
optimal estimation (Porter, 2003), tests for validity of the optimalestimation(Porter,2003),testsforvalidityofthe
design(McCrary,2008),bandwidthselection(Imbens and
Kalyanaraman 2011).
General surveys include Lee and Lemieux (2009) Van Der GeneralsurveysincludeLeeandLemieux(2009),VanDer
Klaauw(2008),andImbensandLemieux(2008).
Todaysdiscussionborrowsfromallofthem
RegressionDiscontinuityBasics g y
ThebasicideabehindtheRDdesignisthatassignment
tothetreatmentisdetermined,eithercompletelyor , p y
partly,bythevalueofanassignment(orforcing)
variable(thecovariateX)beingoneithersideofafixed
threshold.
Assignmenttotreatmentbycovariatevalue,assignallunits
withX
i
c totreatment
RD estimates the local average treatment effect (LATE) RDestimatesthelocalaveragetreatmenteffect(LATE)
ofthetreatmentatX = c
RDislikearandomizedexperimentatthecutpoint X = c
h d ll d d h h TheRDdesignisgenerallyregardedashavingthe
greatestinternalvalidityofallquasiexperimental
methods.Itsexternalvalidityismorelimited,sincethe
estimatedtreatmenteffectislocaltothediscontinuity.
RDScatterplot:PositiveTreatmentEffect
Outcome Outcome
(Y)
Assignment Variable (T)
Cutting Point
RDScatterplot:NoTreatmentEffect
Outcome Outcome
(Y)
Assignment Variable (T)
Cutting Point
RD Setup Formally RDSetupFormally
Startwiththeusualpotentialoutcomesframework
TheunitleveltreatmenteffectisY
i
(1) Y
i
(0),whereY
i
(1)isthe
outcomethatwouldoccuriftheunitwereexposedto
treatment; Y
i
(0) if not. The familiar problem is that we cannot treatment;Y
i
(0)ifnot.Thefamiliarproblemisthatwecannot
observethepairY
i
(0)andY
i
(1)simultaneously.
AssumeabinarytreatmentvariableT
i
andacontinuous
assignment (or forcing) variable X assignment(orforcing)variableX
i
Inasharpregressiondiscontinuitydesign:
T
i
= 1{X
i
c} T
i
1{X
i
c}
Wherecisacutoffsuchthatallunitsabovereceivethe
treatmentandallthosebelowdonot
Defining the Causal Estimate DefiningtheCausalEstimate
Underfairlyweakassumptions,theeffectofthe
treatment is identified by: treatmentisidentifiedby:
| | | |
| | | |
c c
c c
+ +
+ = + =
| +
X Y E X Y E
c X Y E c X Y E
i i i i
| ) 0 ( li | ) 1 ( li
| lim | lim
0 0
Whichis:
RD
=E[Y
i
(1) Y
i
(0)|X
i
=c].
| | | | c c
c c
+ = + = =
| +
c X Y E c X Y E
i i i i
| ) 0 ( lim | ) 1 ( lim
0 0
Thisistheaveragetreatmenteffectatthecutoffpoint,a
particularLATE.
This inference is possible because the functions E[Y(1)|X] ThisinferenceispossiblebecausethefunctionsE[Y(1)|X]
andE[Y(0)|X]are(assumedtobe)continuousinx.
Someextrapolationisrequiredbecausebydesignthere p q y g
arenounitswithX
i
=cforwhomweobserveY
i
(0).
RDComparedtoOtherObservationalMethods
b h d d d f ff Rememberthetwoassumptionsneededtoidentifytreatmenteffects
fromobservationaldatausingregression/matching(Kosukes slide#31):
overlap andunconfoundedness.
You get these two assumptions for free in a randomized experiment Yougetthesetwoassumptionsforfreeinarandomizedexperiment.
Ingeneral,unconfoundedness isnotconsideredaparticularlycredible
assumption,andtheothermethodswerestudyingthisweekcanbe
thoughtofaswaysformakingitmoreplausible.
RDisspecialinthefollowingways:
Unconfoundedness issatisfiedbydefinition.WhenXc,Tisalways1;when
X<c,Tisalways0.AfterconditioningonX,thereisnovariationleftinT,so
it cannot be correlated with any other factor RD is a special case of itcannotbecorrelatedwithanyotherfactor.RDisaspecialcaseof
selectiononobservables.
Bycontrast,overlap isclearlyviolatedinanRDsincewecannotever
observetreatmentandnontreatmentforthesamevalueofX.The
i i i i i d f h f il f l continuityassumptionisrequiredtocompensateforthefailureofoverlap
(moreonthisbelow).
NoteasubtlebutimportantdifferencefromIV:theassignmentvariableis
allowedtohaveadirectimpactontheoutcome,notjustonthetreatment, p j
butnotadiscontinuousimpact.
Theestimatedeffectoftheassignmentvariableontheoutcomemayreflectatrue
causaleffectoraspuriouscorrelationduetocorrelationwithunobservables.
RDComparedtoanExperiment
RDisoftendescribedasaclosecousinofarandomizedexperimentorasa
localrandomizedexperiment.
Coughey andSekhon argueagainstthisconceptualization,forreasonswewill
l t b t it i th d t di h th l i d seelater,butitisworthunderstandingwhytheanalogyismade
Consideranexperimentinwhicheachparticipantisassignedarandomly
generatednumber,v,fromauniformdistributionovertherange[0,1].
Unitswithv0.5assignedtotreatment;unitswithv<0.5assignedtocontrol g ; g
ThisrandomizedexperimentcanbethoughtofasanRDwheretheassignment
variableX=vandthecutoffisat0.5.
Becausetheassignmentvariableisrandom,thecurvesE[Y(0)|X]andE[Y(1)|X]are
flat And we know that they are flat flat.Andweknowthattheyareflat.
TheATEcanbecomputedasthedifferenceinthemeanvalueofYoneithersideof
thecutoff.
Becausethefunctionsareflateverywhere,theoptimalbandwidthistouseall
the data thedata
NotethattherearetwomainwaysinwhichanRDdiffersfromarandomized
experimentinactuality
ThefunctionalformofE[Y(0)|X]andE[Y(0)|X]neednotbeflat(orlinearor
t i ) d t b k monotonic)andmaynotevenbeknown.
Itmaybepossibleforunitstoaltertheirassignmenttotreatmentbymanipulating
theforcingvariableinawaythatisnotpossiblewhenitisassignedatrandomby
theinvestigator.
ExperimentasRD
Outcome Outcome
(Y)
Assignment Variable (T)
(Random)
Cutting Point
0.50
(Random)
EstimationBasics1
Wehavenowdefinedacausaleffectasthe
differenceoftwofunctionsatapoint.Howdowe
estimate that? There are 3 general approaches estimatethat?Thereare3generalapproaches.
Approach#1:Comparemeans
In the data we never observe E[Y(0)|X=c] that is there Inthedata,weneverobserveE[Y(0)|X c],thatisthere
arenounitsatthecutoffthatdontgetthetreatment,
butinprincipleitcanbeapproximatedarbitrarilywellby
E[Y(0)|X=c]. E[Y(0)|X c ].
Thereforeweestimate:
| | | | c c = + = c X Y E c X Y E | |
Thisisthedifferenceinmeansforthosejustaboveand
belowthecutoff.
Thisisanonparametricapproach.Agreatvirtueisthatit p pp g
doesnotdependoncorrectspecificationoffunctional
forms.
Estimation Basics 2 EstimationBasics2
NotethatIsaidinprinciplewecanestimatemeans
arbitrarilyclosetothecutoff.Inpractice,thisdependson y p , p
havinglotsofdatawithin ofthecutoff.Supposeyou
dont.
Approach #2: OLS (with Polynomials) Approach#2:OLS(withPolynomials)
TheoriginalRDdesign(Thistlewaite andCampbell1960)was
implementedbyOLS.
where isthecausaleffectofinterestand isanerrorterm.
q | t o + + + = X T Y
Thisregressiondistinguishesthenonlinearanddiscontinuous
jumpfromthesmoothlinearfunction.
OLSwithonelinearterminXisseldomusedanymore
b h f i l f i becausethefunctionalformassumptionsareverystrong.
Whatarethey?
Estimation Basics 3 EstimationBasics3
Supposetheunderlyingfunctionsarenonlinearandmaybeunknown.In
particular,supposeyouwanttoestimate
wheref(X)isasmoothnonlinearfunctionofX.
Perhaps the simplest way to approximate f(X) is via OLS with polynomials in X
q t o + + + = ) (X f T Y
Perhapsthesimplestwaytoapproximatef(X)isviaOLSwithpolynomialsinX.
Commonpracticeistofitdifferentpolynomialfunctionsoneachsideofthe
cutoffbyincludinginteractionsbetweenTandX.
Modelingf(X)withapthorderpolynomialinthiswayleadsto
Centering X at the cutoff prior to running the regression ensures that the
q | | | t
| | | o
+ + + + +
+ + + + + =
p
p
p
p
TX TX TX T
X X X Y
...
...
2
2 1
0
2
02 01
CenteringXatthecutoffpriortorunningtheregressionensuresthatthe
coefficientonTisthetreatmenteffect.
Commonpractice,forwhateverreason,seemstousea4
th
orderpolynomial,
thoughyoushouldbesurethatyourresultsarerobusttootherspecifications
( hi b l ) (moreonthisbelow).
EstimationBasics4
OLSwithpolynomialsisaparticularlysimplewayofallowinga
flexiblefunctionalforminX.Adrawbackisthatitprovidesglobal
estimatesoftheregressionfunctionthatusedatafarfromthe
cutoff.
Theremanyareotherways,buttheRDsetupposesacoupleof
problems for standard nonparametric smoothers problemsforstandardnonparametricsmoothers.
Weareinterestedintheestimateofafunctionataboundarypoint.
(Forwhythisisaproblem,seeHTVorImbens andLemieux.)
Standardnonparametrickernelregressiondoesnotworkwellhere p g
ThisleadstoApproach#3:LocalLinearRegression
Insteadoflocallyfittingaconstantfunction(e.g.,themean),fitlinear
regressionstoobservationswithinsomebandwidthofthecutoff
Arectangularkernelseemstoworkbest(seeImbens andLemieux),
butoptimalbandwidthselectionisanopenquestion
Aseriousdiscussionoflocallinearregressionisbeyondthescopeof
thislecture.See,forexample,FanandGijbels (1996) , p , j ( )
But,really,werejusttalkingaboutrunningregressionsondatanear
thecutoff.
RDPitfall:MistakingNonlinearity
forDiscontinuity
Consequences of using an incorrect functional Consequencesofusinganincorrectfunctional
formarepotentiallymoresevereforRDthanfor
othermethodswearestudyingthisweek
Misspecificationofthefunctionalformmay
generateabiasinthetreatmenteffect
Th t it ti f thi t i h Themostcommonsituationofthistypeiswhen
anunaccountedfornonlinearityinthe
conditionalmeanfunctionismistakenfora
discontinuity
Eachofthe3estimationmethodsdealswiththis
issue in a different way issueinadifferentway
Nonlinearity Mistaken for Discontinuity NonlinearityMistakenforDiscontinuity
Outcome Outcome
(Y)
False discontinuity
Assignment Variable (T)
Cutting Point
Polynomial Regression PolynomialRegression
Outcome Outcome
(Y)
Assignment Variable (T)
Cutting Point
LocalLinearRegression
Outcome Outcome
(Y)
Assignment Variable (T)
Cutting Point
CompareMeans
Outcome Outcome
(Y)
Assignment Variable (T)
Cutting Point
CompareMeans:SmallerBandwidth
Outcome Outcome
(Y)
Assignment Variable (T)
Cutting Point
Manipulation
Ifindividualshavecontrolovertheassignmentvariable,thenwe
shouldexpectthemtosortinto(outof)treatmentiftreatmentis
desirable(undesirable) ( )
Thinkofameanstestedincomesupportprogram,oranelection
Thosejustabovethethresholdwillbeamixtureofthosewhowould
passedandthosewhobarelyfailedwithoutmanipulation.
f i di id l h i l h i i bl Ifindividualshaveprecisecontrolovertheassignmentvariable,we
wouldexpectthedensityofXtobezerojustbelowthethreshold
butpositivejustabovethethreshold(assumingthetreatmentis
desirable). desirable).
McCrary(2008)providesaformaltestformaniupulaiton ofthe
assignmentvariableinanRD.TheideaisthatthemarginaldensityofX
shouldbecontinuouswithoutmanipulationandhencewelookfor
discontinuities in the density around the threshold discontinuitiesinthedensityaroundthethreshold.
Howprecisemustthemanipulationmustbeinordertothreatenthe
RDdesign?SeeLeeandLemieux(2010).
ThismeansthatwhenyourunanRDyoumustknowsomething y y g
aboutthemechanismgeneratingtheassignmentvariableandhow
susceptibleitcouldbetomanipulation.
ExampleofManipulation
A i t i hi h th i d $14 000 lif f Anincomesupportprograminwhichthoseearningunder$14,000qualifyfor
support
SimulateddatafromMcCrary2008
Assessing the Validity of an RD AssessingtheValidityofanRD
Itisimpossibletotestthecontinuityassumptiondirectly,but
we can test some implications of it wecantestsomeimplicationsofit
Namely,allobservedpredeterminedcharacteristicsshould
haveidenticaldistributionsoneithersideofthecutoff,inthe
limit as we approach smaller and smaller bandwidths That is limit,asweapproachsmallerandsmallerbandwidths.Thatis,
thereshouldbenodiscontinuitiesintheobservables.
Againthereisananalogytoanexperiment:wecannottest
whether unobserved characteristics are balanced but we can whetherunobservedcharacteristicsarebalanced,butwecan
testtheobservables.Rejectioncallstherandomizationinto
question.
A subtle point in the RD context is that a finding a AsubtlepointintheRDcontextisthatafindinga
discontinuityinobservablecovariatesindicatesaviolationof
thecontinuityassumption,notaviolationof
unconfoundedness which is satisfied by definition (More on unconfoundedness,whichissatisfiedbydefinition.(Moreon
thisbelow)
The Role for Covariates in RD TheRoleforCovariatesinRD
Inprinciple,covariatesarenotneededfor p p ,
identificationinRD,buttheycanhelpreduce
samplingvariabilityintheestimatorandimprove
precision precision
Thisisastandardargumentwhichalsosupports
inclusionofcovariatesinanalysesofrandomizedtrials
Addingcovariatesshouldnotaffectthepoint
estimateoftheeffect(verymuch).Ifitdoes,
there is a problem. thereisaproblem.
Thewiderthebandwidththemoreimportantit
maybetoincludecovariates.
Graphical Analysis GraphicalAnalysis
GraphicalinspectionisanintegralpartofanyRD
analysis.
3typesofgraphsshouldalwaysbeproduced,where
assignment variable is graphed against: assignmentvariableisgraphedagainst:
1:theoutcome
2:othercovariates
3 d it f 3:densityofcases
1shouldshowadiscontinuity;2and3shouldshowno
discontinuity
Ifyoucan'tseethemainresultwithsuchasimple
graph,it'sprobablynotthere
If you see a discontinuity in 2 or 3 be worried Ifyouseeadiscontinuityin2or3,beworried
BandwidthSelection
ForLocalLinearRegression
Bandwidthselectionrepresentsthefamiliartradeoffbetweenbiasandprecision
Whenthelocalregressionfunctionismoreorlesslinear,thereisntmuchofatradeoffsobandwidthcan
belarger. g
Therearetwogeneralmethodsforselectingbandwidth
Adhoc,orsubstantivelyderived(e.g.,electionsbetween4852%areclose)
Datadriven
Optimalbandwidthmethods(Imbens andKalyanaraman)
Crossvalidationmethods(LudwigandMiller;Imbens andLemieux) ( g ; )
ForPolynomialRegression
Choosingtheorderofthepolynomialisanalogtothechoiceofbandwidth
Twoapproaches
UsetheAkaike informationcriterion(AIC)formodelselection:AIC=Nln(
2
)+2p,where
2
(shouldhavea
h t) i th d f th i d i th b f d l t hat)isthemeansquarederroroftheregressionandpisthenumberofmodelparameters
Selectanaturalsetofbins(asyouwouldforanRDgraph)andaddbindummiestothemodelandtest
theirjointsignificance.Addhigherordertermstothepolynomialuntilthebindummiesarenolonger
jointlysignificant.
Thisalsoturnsouttobeatestforthepresenceofdiscontinuitiesintheregressionfunctionatpointsotherthenthe
cutoff,whichyoullwanttodoanyway
Inbothcases
Inpractice,youmaywanttofocusonresultsfortheoptimalbandwidth,butit'simportantto
testforlotsofdifferentbandwidths.Thinkoftheoptimalbandwidthonlyasastartingpoint.
Ifresultscriticallydependonaparticularbandwidth,theyarelesscredibleandchoiceof
bandwidth requires a substantive justification bandwidthrequiresasubstantivejustification.
Inprinciple,theoptimalbandwidthfortestingdiscontinuitiesincovariatesmaynotbethesame
astheoptimalbandwidthforthetreatment.Again,followthepracticeoftestingrobustnessto
variationsinbandwidth.
FuzzyRD
h b b l f h d l Supposetheprobabilityoftreatmentchangesdiscontinuously
atathreshold,butnotfrom0to1.Thisisasituationfor
applyingFRD.
h h f f h h NotethatthefuzzinessinFRDcomesfromthechangein
probabilityoftreatment,notfuzzinessaboutthethreshold
InsharpRDdesigns,thejumpintheoutcomeatthecutoffisthe
estimate of the ca sal impact of the treatment In a FRD design estimateofthecausalimpactofthetreatment.InaFRDdesign,
thejumpintheoutcomeisdividedbythejumpinthe
probabilityoftreatmentatthecutofftoproducethelocalWald
estimate (equivalent to a local IV estimate) of the causal impact estimate(equivalenttoalocalIVestimate)ofthecausalimpact.
(InSRD,thejumpisone,sothedivisionisinconsequential,but
thisdemonstratestherelationshipbetweenSRDandFRD).
The important point to remember is that fuzzy RD is numerically TheimportantpointtorememberisthatfuzzyRDisnumerically
equivalent(andconceptuallysimilar)toIV(seeMostlyHarmless
sec.6.2)
An additional graph is needed when doing FRD: probability of AnadditionalgraphisneededwhendoingFRD:probabilityof
treatmentbyassignmentvariableshouldshowadiscontinuous
probabilityatthethreshold
DISCUSSION OF PAPERS
RegressionDiscontinuity
DISCUSSIONOFPAPERS
DoVotersAffectorElectPolicies?
byLee,Moretti,andButler(LMB)
Motivation Motivation
Motivation:2fundamentallydifferentviewsoftheroleof
elections
Convergence:Competitionforvotesdrivescandidatestoseek
middlegroundpolicies,compromise(medianvotertheorem).
Voters affect policy choices of politicians Votersaffectpolicychoicesofpoliticians.
Divergence:Votersselectcandidates,whothenenacttheirown
preferredpolicies.Voterselectpolicies.
Which of the two we see in practice depends on whether can Whichofthetwoweseeinpracticedependsonwhethercan
makecrediblepromisestoimplementpoliciesthatarenotat
theirownblisspoint(crediblecommitmentsarefacilitatedby
repeatinteractions)
Thegoalofthepaperistoexaminewhichphenomenonis
moreempiricallyrelevantforUSpolitics,specificallyvoting
intheHouse
Theory (vastly simplified) Theory(vastlysimplified)
Consider2parties,DandR
Rsblisspointis0,Dsblisspointisc(>0) p p ( )
TheprobabilitythatDwinstheelectionisP
IfDwinselection,policyxisimplemented;ifRwins,yis
implemented implemented
P*representstheunderlyingpopularityofpartyD,orthe
probabilitythatDwouldwinifx=candy=0.Anincreasein
P* represents an exogenous increase in Ds popularity P representsanexogenousincreaseinD spopularity
Whendx*/dP*anddy*/dP*>0,wesaythatvotersaffect
candidatespolicychoices
* denoted equilibrium *denotedequilibrium
Whendx*/dP*anddy*/dP*=0,wesaythatvotersmerely
electpoliticianswithfixedpolicies.Thatis,anincreasein
P* does nothing to the equilibrium policies of the parties P doesnothingtotheequilibriumpoliciesoftheparties.
Estimating Framework EstimatingFramework
Therollcallvotingrecordoftherepresentativeinthe
district following election t is districtfollowingelectiontis
RC
t
=(1 D
t
)y
t
+D
t
x
t
Where D is the indicator for whether D won That is WhereD
t
istheindicatorforwhetherDwon.Thatis,
onlythewinningcandidatespolicyisobservable
Theexpressioncanbetransformedinto: p
This simply parameterizes the derivatives from y
prior slide as
0
.
Italsoallowsanindependenteffectofparty,
1
.
EstimatingFrameworkContd
WecannotobserveP*soequation(2)cannot
be directly estimated bedirectlyestimated
ButsupposewecouldrandomizeD
t
.ThenD
t
o ld be independent of P* and Then wouldbeindependentofP*
t
and
t
.Then:
E hi d li d i d b i d Everythingunderlinedinredcanbeestimated
fromthedata
WhyThisWorks
Theelectcomponentis
is estimated by the difference in voting
1
isestimatedbythedifference in voting
records between the parties at time t
ThefractionofdistrictswonbyDemocratsint+1 y
isanestimateof
Because we can estimate the total effect, ,
of a Dem victory in t on RCt+1 we can then of a Dem victory in t on RCt+1, we can then
net out the elect component to implicitly get
the affect component p
RandomassignmentofD
t
iscrucial.Without it,
equation (5) would reflect
1
and that Dem
districts have more liberal bliss points districts have more liberal bliss points
GraphicalEstimateofEquation4
20
Graphical Estimate of Eqn. 5 GraphicalEstimateofEqn.5
4 45
Graphical Estimate of Eqn. 6 GraphicalEstimateofEqn.6
0.50
Statistical Results StatisticalResults
Robustness, Etc. Robustness,Etc.
No other observable district attributes change Nootherobservabledistrictattributeschange
atthediscontinuity
Results hold for many other measures of Resultsholdformanyothermeasuresof
representativevotingrecords
R l b ll i f i f Resultsrobusttoallowingforvarioussortsof
districtheterogeneity
Results(smallaffectcomponent)stableover
time
RegressionDiscontinuityDesigns
and Popular Elections: Implications andPopularElections:Implications
ofProIncumbentBiasinCloseU.S.
HouseRaces
byCaughey andSekhon (CS)
Basic Argument BasicArgument
Closeelectionsarenotlikeotherelections
Strategicpoliticalactorshavestrongincentivestotargettheir
resourceswheretheywillhavethegreatestmarginalimpact
Thereisanincumbencyadvantageeveninveryclose
elections
Theincumbentwinsdisproportionatelyandhasgreaterfinancial
resources
Thisfinding,alongwithothercovariateimbalancesatthe
cutpoint,callsintoquestiontheLMBincumbency
advantageresultsand,moregenerally,theassumptionthat
t i l l ti d d l outcomesincloseelectionsareasgoodasrandomly
assigned
NotethatCScritiqueLee(2008),notLMB,buttheimplications
for the incumbency advantage results in both papers are the fortheincumbencyadvantageresultsinbothpapersarethe
same
TheLee(&McCrary)TestsforManipulation
A graph like (A) led Lee, and separately McCrary, to conclude that there is no manipulation.
However, (B) and (C) begin to suggest another story. Remember, the concern is with the
incumbent partys vote share, not the Democratic vote share.
DensityoftheAssignmentVariable
Key Takeaway: The candidate of the incumbent party is about three times more likely to win
election by half a percentage point or less than to lose by a similar margin. The density of this
variable appears to diverge rather than converge in the neighborhood of the cut-point.
Covariate Imbalance CovariateImbalance
BasedoncorrectingsomeofLeesdataandaddingsome
newvariables,CSfindimbalanceatthecutoffinthe
following:
Democraticmargininthepreviouselection
thepartiesrelativecampaignexpenditures
1stdimensionNOMINATEscoreofthecurrentincumbent
whethertheDemocratic(Republican)candidateisthecurrent
b incumbent
numberoftermstheDemocrat(Republican)hasservedinthe
U.S.HouseofRepresentatives
h th th D t (R bli ) h liti l whethertheDemocrat(Republican)hasmorepolitical
experiencethantheRepublican(Democrat)
CongressionalQuarterlysOctoberpredictionofwhich
candidate will win the race candidatewillwintherace
CovariateImbalanceGraph
SensitivitytoBandwidthSelection
PotentialMechanisms
Notlikelybeoutrightfraud,becausesignificanceoflagged
voteshareisincreasingovertimeandwebelievepotential
for fraud has been decreasing forfraudhasbeendecreasing
Controloverrecountsdoesnotappeartobethekey
becausetheyrarelyhappenandevenmorerarelychange
the outcome theoutcome.
Butwedontneedanexplanationbasedonvotecounting.
Differencesbetweenwinnersandlosersinincumbency,
money political experience and other pre election money,politicalexperience,andotherpreelection
resourcesareevidentfarbeforeanyvotesarecast,
counted,ormanipulated.
These differences can be seen in elections expected to be Thesedifferencescanbeseeninelectionsexpectedtobe
closeexanteandinthosethatwereinfactdecidedbya
narrowmargin.
These facts contradict the idea that resources Thesefactscontradicttheideathatresources,
expectations,andallelseshouldbebalancedintheclosest
elections.
Regression to the Mean Explanation? RegressiontotheMeanExplanation?
Disturbingly,themostseverelyimbalancedcovariatesarethoseclosely
related to the partisanship of the current occupant of the House seat relatedtothepartisanshipofthecurrentoccupantoftheHouseseat
thatis,thelaggedvalueoftreatmentandthesecondorderlagofthe
outcome.Onesuchcovariate,notshowninthebalancetable,isthe
generalpartisanswingamongallHouseracesinagivenyearrelativeto
the previous election In good Democratic years such as 1948 1958 1964 thepreviouselection.IngoodDemocraticyears,suchas1948,1958,1964,
1974,and2006,closeelectionsareoverwhelminglyconcentratedin
normallyRepublicanseats.Conversely,Democratheldseatstendtobe
closelycontestedinbadDemocraticyears,like1946,1966,1980,and
1994 Because the incumbent party is much more likely to prevail in close 1994.Becausetheincumbentpartyismuchmorelikelytoprevailinclose
races,closeDemocraticvictoriesaremuchmorelikelytooccurinbad
Democraticyears,andcloseRepublicanvictoriestooccurinbadyearsfor
Republicans.Totheextentthatbadelectionsforonepartytendtobe
followed by more normal ones this pattern suggests that the apparent followedbymorenormalones,thispatternsuggeststhattheapparent
incumbentpartyadvantagemaybecontaminatedbyregressiontothe
meaneffects.
Lessons from LMB & CS LessonsfromLMB&CS
Thisisacautionarytale
LMBarevery goodscholars.
Theydidalmosteverythingright
Theydonotdoalottojustifyfunctionalformorshowrobustnesstodifferentbandwidths
Remember,theLMBresultsforDemocraticvote(eqn.5)arenotimplicatedinthis
critique.Thisisthebestpartofthepaperanyway,IMHO.
Whatcanyoulearnfromthisexchange:
Trytofindproblemsinyourdesignbeforesomeoneelsedoesitforyou
Identifyandcollectaccuratedataontheobservablecovariatesmostlikelytoreveal
sorting at the cutpoint This may not be the covariates that happen to be sitting in sortingatthecut point.Thismaynotbethecovariatesthathappentobesittingin
yourdataset.
Laggedvaluesofthetreatmentvariablearealwaysagoodidea.Inelections,thepartythat
currentlycontrolstheoffice.
Automatedbandwidthselectionalgorithmsdonotguaranteegoodresults.Theyare g g g y
justastartingpoint.
ForRDpurposes,whatconstitutesacloseelectionappearstobecloserthanthe
4852%bandwidthwidelyuseduptonow.CSgetmostoftheirresultsusing49.5
50.5%.
GiventhecurrentfetishwithRDinpoliticalscience,understandthatitisnotafact
ofnaturethatcloseelectionsarerandom.Rememberthiswhenyousee(orsetout
towrite)thenextRDpaperoncloseelections.
GUIDE TO PRACTICE
RegressionDiscontinuity
GUIDETOPRACTICE
Summary of RD Assumptions SummaryofRDAssumptions
Thetreatmentisdeterminedatleastinpartby e t eat e t s dete ed at east pa t by
theassignmentvariable
Thereisadiscontinuityintheleveloftreatment y
atsomecutoffvalueoftheassignmentvariable
(selectiononobservablesatthecutpoint)
Unitscannotpreciselymanipulatetheassignment
variabletoinfluencewhethertheyreceivethe
treatment or not treatmentornot
Othervariablesthataffectthetreatmentdonot
change discontinuously at the cutoff changediscontinuouslyatthecutoff
Internal & External Validity
The strength of the RD design is its internal validity,
Internal&ExternalValidity
arguably the strongest of any quasi-experimental design
External validity may be limited
Sharp RD (SRD) provides estimates for the subpopulation Sharp RD (SRD) provides estimates for the subpopulation
with X=c, that is those right at the cutoff of the assignment
variable.
The discontinuity is a weighted average treatment effect The discontinuity is a weighted average treatment effect
where weights are proportional to the ex ante likelihood
that an individuals realization of X will be close to the
threshold.
Fuzzy RD (FRD) restricts the estimates further to compliers
at the cutoff (more on this below)
You need to justify extrapolation to other subpopulations You need to justify extrapolation to other subpopulations
(e.g., treatment homogeneity)
Threats to an RD Analysis ThreatstoanRDAnalysis
TherearethreegeneraltypesofthreatstoanRD g yp
design
1. Othervariableschangediscontinuouslyatthe
cutoff cutoff
Testforjumpsincovariates,includingpretreatment
valuesoftheoutcomeandthetreatment
2. Therearediscontinuitiesatothervaluesofthe
assignmentvariable
3 M i l ti f th i t i bl 3. Manipulationoftheassignmentvariable
Testforcontinuityinthedensityoftheassignment
variableatthecutoff
StepsforSharpRDAnalysis
1. Graphthedatabycomputingtheaveragevalueoftheoutcomevariable
overasetofbins.
The bin width has to be large enough to have a sufficient amount of precision Thebinwidthhastobelargeenoughtohaveasufficientamountofprecision
sothattheplotslookssmoothoneithersideofthecutoffvalue,butatthe
sametimesmallenoughtomakethejumparoundthecutoffvalueclear.
2. Estimatethetreatmenteffectbyrunninglinearregressionsonbothsidesof
the cutoff point. thecutoffpoint.
Witharectangularkernel,thesearejuststandardregressionestimatedwithina
binofwidthhonbothsidesofthecutoffpoint.Notethat:Standarderrorscan
becomputedusingstandardleastsquaremethods(robuststandarderrors).The
optimalbandwidthcanbechosenusingcrossvalidationorothermethods. p g
3. Therobustnessoftheresultsshouldbeassessedbyemployingvarious
specificationtests.
Lookingatpossiblejumpsinthevalueofothercovariatesatthecutoffpoint
Testing for possible discontinuities in the conditional density of the forcing Testingforpossiblediscontinuitiesintheconditionaldensityoftheforcing
variable
Lookingwhethertheaverageoutcomeisdiscontinuousatothervaluesofthe
forcingvariable
Using various values of the bandwidth with and without other covariates that Usingvariousvaluesofthebandwidth,withandwithoutothercovariatesthat
maybeavailable.
Steps for Fuzzy RD Analysis StepsforFuzzyRDAnalysis
1. Graphtheaverageoutcomesoverasetofbinsasinthe
caseofSRD,butalsographtheprobabilityoftreatment.
2. Estimatethetreatmenteffectusing2SLS,whichis
numericallyequivalenttocomputingtheratiointhe
estimateofthejump(atthecutoffpoint)intheoutcome
variableoverthejumpinthetreatmentvariable.
Standarderrorscanbecomputedusingtheusual(robust)
2SLS d d 2SLSstandarderrors
Theoptimalbandwidthcanagainbechosenusingoneofthe
methodsdiscussedabove.
3 Th b t f th lt b d i th 3. Therobustnessoftheresultscanbeassessedusingthe
variousspecificationtestsmentionedinthecaseofSRD
designs.
EvaluatinganRDPaper
(P ibl Y O ) (PossiblyYourOwn)
Doestheauthorshowconvincinglythat
Treatmentchangesdiscontinuouslyatthecutpoint eat e t c a ges d sco t uous y at t e cutpo t
Outcomeschangediscontinuouslyatthecutpoint
Othercovariatesdonotchangediscontinuouslyatthecutpoint
Pretreatmentoutcomesdonotchangeatthecutpoint
Th i i l ti f th i t i bl (b hi th Thereisnomanipulationoftheassignmentvariable(bunchingnearthe
cutpoint)
Arethebasicresultsevidentfromasimplegraph?
Aretheresultsrobusttodifferentfunctionalformassumptionsaboutthe p
assignmentvariable
Forexample,parametricandnonparametricfits,differentbandwidths,etc.
Couldotherpossiblyunobservedtreatmentschangediscontinuouslyat
the cutoff (bundling of institutions) thecutoff(bundlingofinstitutions)
Forexample,18
th
birthdaymarksadiscontinuouschangeineligibilitytovote,
butalsoeligibilityfordraft,sentencingasanadult,andlotsofotherthings,
whichmayormaynotberelevantdependingontheoutcomeinquestion
External validity Externalvalidity
Arecasesnearthecutpoint differentfromcasesfarfromthecutpoint inother
ways?Dothesedifferencesmakethemmoreorlessrelevantfroma
theoreticalorpolicyperspective?
EXAMPLES IN STATA
RegressionDiscontinuity
EXAMPLESINSTATA
Regression Discontinuity Designs RegressionDiscontinuityDesigns
EITM2011
Chris Berry ChrisBerry