Difference in Difference

for Impact Evaluation

Moises Neil V. Seriño, PhD

Visayas Socio-Economic Research and Data Analytics Center
College of Management and Economics
Visayas State University
 World Bank’s Impact
Evaluation in Practice

1. Randomization
2. Regression discontinuity
3. Instrumental variables
4. Difference-in-difference
5. Matching (Propensity score matching)
Framework of Diff in Diff

Outcomes
Baseline Follow Up

Program group

impact

Counterfactual

time
Intervention
 Difference in Difference
(DID)
• DID relies on comparisons of the
differences between treatment and control
groups, before and after an intervention
– Also called “double difference” approach,
previous methods discussed are “single
difference” approaches

• DID accounts for selection on

unobservables as long as the
unobservables are time invariant
 Identifying Assumption
• Whatever happened to the control
group over time is what would have
happened to the treatment group in the
absence of the program.
Effect of program using
only pre- & post- data
from T group (ignoring
general time trend).

Baseline Follow up
 Identifying Assumption
• Whatever happened to the control
group over time is what would have
happened to the treatment group in the
absence of the program.

Effect of program using

only T & C comparison
from post-intervention
(ignoring pre-existing
differences between T &
C groups).

Baseline Follow up
 Identifying Assumption
• Whatever happened to the control
group over time is what would have
happened to the treatment group in the
absence of the program.

Effect of program
difference-in-difference
(taking into account pre-
existing differences
between T & C and
general time trend).

Baseline Follow up
 Typical Data Requirements

• Outcomes observed for two groups for two

time periods
• One of the groups is treated in the second period
but not the first period
• Second group is not exposed to the treatment
during either period.

• Requires pre-intervention baseline survey of

“potential” participants and non-participants
and then follow-up after intervention
 Typical Data Requirements

• Panel data usually required – same

individuals observed in the two time
periods

• With panel data approach, we will be

able to control for time invariant
unobservable characteristics
 Technically (Algebraic)

DID = E (Y − Y T1 = 1) − E (Y − Y T1 = 0 )
1
T
0
T
1
C
0
C

where t is time period (t = 0,1),

T C
Yt and Yt are outcomes for the two groups at t
T1 =1 is treatment at t = 1
T1 =0 is no treatment at t = 1
 Technically (Regression)
• With a two-period repeated cross-section data,
one can specify the following regression

Yit = α + β Ti1t + ρTi1 + γ t + ε it

• The β parameter is the DID estimate

• The separate T and t terms accounts for the
separate mean effects of treatment and time
• Can add covariates X in specification
 DID Approach

• In technology impact evaluation

literature, DID seemed to be looked at
favorably
– Having a baseline survey is important
– Plan in advance of technology roll-out
Example
Example
Example
 Questions?
Clarifications?
 Actual Example
• Workshop
• We will use EXCEL / State for
estimating DID
Thank you very much