05 Regression With RCT

Program Evaluation

Instructor: Dániel Horn

(slides: Gábor Kézdi)
Content
 Regression compares means
 Comparing means in an RCT
 Heterogeneous potential outcomes,
heterogeneous treatment effects, and the “error
term” of the regression
 Control variables in RCT regressions

ProgEval 05 Regression & RCT 2

Regressions Compare Means
•  
or

 Average Y for observations with X=0

 On average, Y is higher by units among observations that

have X higher by one unit.

ProgEval 05 Regression & RCT 3

Regressions and Binary RHS Variable
•  
or

 where

 Average Y for observations with X=0

 On average, Y is higher by units among observations that

have X=1 compared to observations that have X=0

ProgEval 05 Regression & RCT 4

Impact Evaluation and Regression
•  
or

 where
 if non-treated
 if treated

 Average Y among non-treated

 Average difference in Y between treated vs. non-treated

ProgEval 05 Regression & RCT 5
Impact Evaluation and Regression
•  
or

 where if non-treated, if treated

 If Wi is randomly assigned, β = ATE

 If Wi is not randomly assigned, β may not be ATE

 Endogeneity: Wi may be correlated with ui
i.e., participants and non-participants may be different in terms
of unobserved things that make them have different outcomes
apart from the effect of the program
ProgEval 05 Regression & RCT 6
More Precisely
Yi = Yi(1)Wi + Yi(0)(1 − Wi )

= Yi(0) + [Yi(1) − Yi(0)]Wi

= αi + βiWi where αi=Yi(0)i and βi= Yi(1)i − Yi(0)

= α + βWi + (αi−α) + (βi−β)Wi

= α + βWi + ui

 we want β=E[βi]=E[Yi(1)−Yi(0)]=ATE

 and also α=E[αi]=E[Yi(0)]

 Note that ui = (αi−α) + (βi−β)Wi

ProgEval 05 Regression & RCT 7
Random Assignment and Regression
 Consequences of random assignment
E[Yi|Wi=0] = E[Y(0)i]
so that α = E[Yi|Wi=0] = E[Y(0)i] = E[αi]

E[Yi|Wi=1] − E[Yi|Wi=0] = E[Y(1)i] − E[Y(0)i]

so that β = E[Yi|Wi=1] − E[Yi|Wi=0] = E[Y(1)i] − E[Y(0)i] =
= E[Y(1)i −Y(0)i] = E[βi] = ATE

ProgEval 05 Regression & RCT 8

Endogeneity/Exogeneity in Regression
 Exogeneity: Wi is uncorrelated with u i
 That is, treatment (Wi) is unrelated both with

 (αi−α): heterogeneity in untreated potential outcomes

 and (βi−β)Wi: heterogeneity in the treatment effect

 Endogeneity: Wi is correlated with ui

 That is, treatment (Wi) is related to either

 (αi−α): heterogeneity in untreated potential outcomes

 or (βi−β)Wi: heterogeneity in the treatment effect

 Regression estimates ATE under exogeneity

ProgEval 05 Regression & RCT 9

Regression: The Effect of NSW
•  

 Female (AFDC) subsample

 Yi: employment (0 or 1) at month 26

 Wi: being assigned to the treatment group (0 or 1)

 interpret α

 interpret β

 What is ui?

ProgEval 05 Regression & RCT 10

Control Variables in RCT Regressions
•  
 Yi: outcome variable

 Wi: being assigned to the treatment group (0 or 1)

 Random assignment

 Xi: predetermined control variables

 Measured before intervention or not affected by intervention

 What to we expect for ?

ProgEval 05 Regression & RCT 11

Control Variables in RCT Regressions
•  
 Does it make sense to include in RCT regression?
 If is zero?

 If is non-zero?

 X uncorrelated with W

 Don’t expect to be different whether X included or not

 But consider SE of

ProgEval 05 Regression & RCT 12

Control Variables in RCT Regressions
• SE
  of
 Simple case: one control variable, homoskedasticity

 SE may be smaller if X included

 (residual variance) smaller if X explains a lot of Y

ProgEval 05 Regression & RCT 13

Summary
 Compare average outcomes in treatment group and non-
treatment group in regression
 If assignment is random coeff on assignment gives good
estimate of ATE (also ATET)
 Random assignment makes assignment variable
exogenous in regression
 Assignment variable endogenous in general otherwise

 Including predetermined control variables in regression

 Does not affect estimate of ATE

 But may lead to more precise estimate of ATE (lower SE)

ProgEval 05 Regression & RCT 14