Health Policy 77 (2006) 221–232

Evaluating program effects on institutional delivery in Peru

Michael J. McQuestion a,∗ , Anibal Velasquez b
a Johns Hopkins Bloomberg School of Public Health, Population and Family Health Sciences,
615 N Wolfe Street, E-4142, Baltimore, MD 21205, USA
b Consultant, PHRPlus, Antequera 777 Piso 8, San Isidro, Lima, Peru

Abstract

We evaluate the joint effects of two targeted Peruvian health programs on a mother’s choice of whether to deliver in a public
emergency obstetric care (EmOC) facility. The national maternal and child health insurance, or SMI Program, provided delivery
care coverage to Peru’s poorest households beginning in 1998. During 1996–2002, Proyecto 2000 sought to improve the quality
of EmOC and increase utilization of public EmOC facilities in the districts reporting the highest maternal and neonatal mortality
levels. Our data come from the Proyecto 2000 endline evaluation, which sampled 5335 mothers living in the catchment areas
of 29 treatment and 29 matched control EmOC facilities. Using propensity scoring and two quality of care indices, we find
significantly higher quality of care in Proyecto 2000 treatment facilities. Using variance components logistic models, we find a
mother enrolled in the SMI Program was more likely to have delivered her last child in a public EmOC, controlling for household
constraints. Residence in a Proyecto 2000 treatment area did not significantly affect the choice. A cross-level interaction term
was insignificant, indicating the two program effects were independent.
© 2005 Elsevier Ireland Ltd. All rights reserved.

Keywords: Quality of care; Evaluation; Developing countries; Safe motherhood

1. Introduction educated women over the period. To correct this, the

This study examines two very different efforts
to increase institutional delivery in Peru. During
1992–1997, Peru implemented large-scale health sec-
tor decentralization reforms. The reforms were criti-
cized for widening health disparities, particularly in
hospital utilization [1]. Peru’s DHS III (1996) and DHS
IV (2000) surveys documented a relative decline in
physician-assisted deliveries among rural and poorly
∗ Corresponding author. Tel.: +1 410 502 6037;
fax: +1 410 955 2303.
E-mail address: mmcquest@jhsph.edu (M.J. McQuestion).
Peruvian Ministry of Health initiated a series of tar-
geted maternal and child health interventions, two of
which we evaluate. The first intervention was Proyecto
2000, a USAID-funded effort begun in 1996 in the 12
of Peru’s 25 departmentos reporting the highest mater-
nal mortality levels. Proyecto 2000 aimed to increase
the proportion and quality of institutional deliveries,
thereby reducing maternal mortality and improving
birth outcomes. The project began with mass media,
health education and social mobilization efforts pro-
moting delivery in the nearest public emergency obstet-
ric care (EmOC) facility. Its emphasis, however, was
on improving the quality of services on offer. The sec-

0168-8510/$ – see front matter © 2005 Elsevier Ireland Ltd. All rights reserved.
doi:10.1016/j.healthpol.2005.07.007
222 M.J. McQuestion, A. Velasquez / Health Policy 77 (2006) 221–232
ond intervention was the Maternal and Child Health
Insurance (SMI) Program, launched in 1998. The SMI
Program covered most maternal and child health costs,
including institutional delivery in public EmOC facil-
ities [2]. It was a means-tested program in that only
households in the poorest wealth quintile were eligi-
ble to participate. By 2000, this program was reaching
about 50% of eligible households in two pilot regions
[3], and the following year it was extended nationwide.
The two programs thus incorporated different target-
ing strategies. Proyecto 2000 targeted high-risk dis-
tritos, specific EmOC facilities and their surrounding
communities while the SMI Program directly targeted
the country’s poorest households. Did either program
increase EmOC utilization? In this study we use quasi-
experimental data to probe this question. We model a
woman’s choice of where she delivered her last baby,
conditional on exposure to these two programs.
2. Background
2.1. Recent perinatal health trends
Demographic data show perinatal health in Peru
improved over this period. The country’s neonatal
death rate fell from 27 to 18 deaths per 1000 live
births during the 1990s [4]. Peru’s estimated mater-
nal mortality ratio also fell, from 265/100,000 live
births in 1990–1996 to 185 in 1994–2000 [5], yet
it remained third highest among 14 Latin American
countries reporting in 1999 [6]. Maternal and neona-
tal mortality are largely influenced by two factors: a
woman’s decision whether or not to utilize institutional
delivery care and the quality of that care. High-quality
EmOC can prevent an estimated one-third of mater-
nal deaths [7], and 40–62% of neonatal deaths [8].
Regarding maternal behaviors, the 1996 DHS III survey
showed that 55% of women who had given birth in the
previous 5 years did so at home. Another 38% used pub-
lic health care facilities and 5% used private delivery
facilities [9]. Over the succeeding 5 years, the propor-
tion of home deliveries fell to 47%, the public sector’s
share rose to 48% and the proportion using private
facilities stayed at about 5% [10]. There are no com-
parable EmOC quality of care estimates, however, a
recent qualitative study ranked Peru second of 13 Latin
American countries evaluated in terms of maternal and
neonatal program effort [11,12]. It is thus plausible that
the observed perinatal health improvements were due
to increased institutional deliveries that in turn resulted
from program improvements.
There were other important factors affecting mater-
nal and perinatal health in Peru over this period. The
country’s per capita GNP grew by a mean 2.4% per
annum during the 1990s [13], an improvement over
the chaotic 1980s. Total fertility rates declined from 4.8
in 1986 to 2.1 in 2000, lengthening birth intervals and
reducing the proportion of high-parity births [4]. These
changing background forces may have been more deci-
sive health behavioral determinants that the program
effects we attempt to elucidate.
2.2. Maternal health risk factors
In Peru, as elsewhere, it is the poorest, most remote
and most socially excluded women who least use
maternal health services [14], and are at highest risk
of maternal, perinatal and post-perinatal mortality
[15,16]. A 2000 survey in Peru’s Ayacucho Depart-
ment, for example, found that only about one-fourth of
women with complications were delivered in adequate
EmOC facilities [17]. In Peru’s DHS IV survey some
83% of women identified at least one barrier to access-
ing local maternal health services. Expense was the
leading problem, followed by lack of female caregivers
[10]. Other cultural factors act as barriers to EmOC uti-
lization, particularly among the 47% of Peruvians who
do not speak Spanish as their first language. Reports
of discrimination and mistreatment by health work-
ers are commonplace [18,19]. The DHS data suggest
that more high-risk women chose to utilize the public
EmOC facilities over this period.
2.3. National SMI Program
The Fujimori Administration instituted the SMI Pro-
gram in 1998. It was Peru’s first attempt to subsidize
preventive and maternal care for low-income pregnant
women, mothers and children ages 0–4 years. Many
saw it as an attempt to restore basic health rights that
had been infringed by decentralization. In 2001, the
program was supplanted by a national Integral Health
Insurance Plan, which offered a wider gamut of tar-
geted benefits to low-income Peruvians of all ages.
Until 1998, any woman could have accessed any public
 M.J. McQuestion, A. Velasquez / Health Policy 77 (2006) 221–232
EmOC facility where she had to pay fees for service on
a sliding scale. The targeted insurance programs elim-
inated these fees for the eligible poor. However, by
2001 many eligible households were still not enrolled
in the program. Peru’s public health system still lacked
the infrastructure and level of performance needed to
extend MCH services to all those eligible. Production
levels remained exceedingly low. The median num-
ber of consultations that year was less than three per
day in half of the Ministry of Health’s peripheral PHC
facilities. To date there has been no comprehensive
evaluation of these targeted insurance efforts [20].
2.4. Proyecto 2000
2.4.1. Phase I
We describe Proyecto 2000 in greater detail because
it generated the data we analyze. Proyecto 2000 was
implemented by a team of Ministry of Health and exter-
nal expert consultants. The team sought to make the
Ministry’s EmOC services culturally acceptable and to
ensure that the facilities delivered high-quality care. A
hallmark of Proyecto 2000 and other Safe Motherhood
projects is an emphasis on making services “woman-
friendly”. An EmOC facility is woman-friendly if: (a)
it is easily accessible and convenient to use; (b) high-
quality services are offered; (c) local cultural beliefs
and social norms are incorporated into treatment proto-
cols and (d) confidentiality is guaranteed, information
is shared and clients’ choices are respected [21]. The
Proyecto 2000 team worked at facility and commu-
nity levels to accomplish these aims. At baseline, team
members and Regional Ministry of Health educators
gathered and analyzed qualitative data on mothers’
perceptions and preferences regarding pregnancy and
childbirth. They used these data to mount a multime-
dia Safe Motherhood campaign in the treatment areas.
In addition, expert trainers trained 3692 community-
based traditional birth attendants (promotoras), and
EmOC staff formally engaged newly constituted com-
munity health committees (Comites Locales de Admin-
istracion en Salud) in their catchment areas. Facility
inputs included physical plant improvements, retrain-
ing of 409 facility-based providers, incorporation of
local birthing practices into clinical protocols and the
introduction of a continuous quality of care (“autoeval-
uacion”) model in some 89 public hospitals and health
centers. In brief, the autoevaluacion model incorpo-
223
rated the Donabedian continuous quality of care [22],
and the McCarthy and Maine maternal mortality deter-
minants frameworks [23]. The autoevaluacion instru-
ment included a battery of detailed indicators regarding
essential obstetric and neonatal care, physical facilities,
patient interaction and management. The expectation
was that greater autonomy and participation in the
self-appraisal process would stimulate improved staff
performance, and the resulting improved quality of care
would generate more institutional deliveries as client
satisfaction improved. All facilities were expected to
attain quality of care improvements sufficient to merit
formal certification by expert evaluators. These 89
facilities comprised the original treatment arm.
2.4.2. Midterm evaluation (2000)
As of 1998, 72 treatment facilities were still active
in the program, all of which had attained formal qual-
ity of care certification as high-quality perinatal care
centers [24]. By October 2000, the number of active
treatment facilities had fallen to 60. At that time a
midterm evaluation was carried out. An external evalu-
ation team examined a random sample of 37 treatment
facilities. They also identified a group of 37 similar
EmOC facilities not exposed to the project to serve as
a comparison group. The control facilities were drawn
from six Ministry of Health districts (DISAs) with ser-
vice population characteristics (literacy, contraceptive
prevalence, use of institutional delivery services, mal-
nutrition and poverty levels) similar to the Proyecto
2000 areas. The control facilities had received only rou-
tine Ministry of Health supervision over the period. The
mid-term evaluation was entirely facility-based. Expert
observers used standardized checklists and institutional
record reviews to assess the quality of EOC on offer.
They found evidence of improved quality of care and a
relative increase in the numbers of institutional deliver-
ies in the treatment group facilities as compared to the
control facilities (Table 1). Additionally, the observers
interviewed samples of prenatal clients. They found
users of treatment facilities were more knowledgeable
about pregnancy, more satisfied with their experiences
and more likely intended to deliver their babies in that
treatment area facility [25].
2.4.3. Phase II (2001–2002)
During Phase II, Proyecto 2000 inputs were con-
centrated on the 31 treatment facilities judged to have
224 M.J. McQuestion, A. Velasquez / Health Policy 77 (2006) 221–232

Table 1 offer. Our task is thus to disentangle two distinct treat-

Selected EOC facility indicators, Proyecto 2000 ment effects, one operating through the health sys-
Variable Control facilities Treatment facilities tem, the other directly on household health production.
Mean S.D. Mean S.D. We expect the two effects will be synergetic: insured
1997
women in high-quality EmOC catchment areas ought
Institutional 1463 1937 1486 845 to be the most likely to use that facility.
births With these points in mind, we model the probability
Prop births 0.07 0.08 0.10 0.09 a Peruvian mother chose to deliver her youngest child
<2.5 kg at the nearest public EmOC facility, conditional on the
Prop births 0.24 0.13 0.19 0.08
c-section
quality of care at that facility, her household constraints,
2000 SMI Program participation, and whether her commu-
Institutional 1052 1434 1542 681 nity and facility participated in Proyecto 2000.
births
Prop births 0.07 0.04 0.09 0.07
<2.5 kg 3. Data and methods
Prop births 0.24 0.10 0.23 0.11
c-section 3.1. Facility data
Ob-gyns 7.92 7.61 7.50 5.37
Births/ob-gyn 140 59 284 152
The Proyecto 2000 evaluators collected a second
2002 round of endline evaluation data in mid-2002 and it is
Autoevaluacion 26.80 8.30 51.20 14.40
these data we analyze in the present paper. The Phase
scorea
EmOC capacity 69.30 8.90 72.19 7.18 II treatment group included all 19 Phase I hospitals
score and a subset of 12 Phase I health centers. The eval-
a
uators selected a new control group, consisting of 15
First principal component of nine-factor index.
of the Phase I control establishments and 14 additional
establishments. As in Phase I, the 14 new control facili-
performed best in Phase I. Project supervisors regu-
ties were purposively selected from six newly matched
larly visited these facilities to ensure the autoevalua-
DISAs that were unexposed to the project. Expert teams
ciones were performed in each facility each quarter.
again evaluated essential obstetric care in the EmOC
Project data show the autoevaluaciones were in fact
facilities using the same extensive standard checklist
implemented. Of the 29 treatment establishments that
used in the midterm evaluation. They also evaluated the
participated to endline (2002), all carried out at least
quality of services using the autoevaluacion instrument
two autoevaluaciones, 25/29 carried out three, 13/29
itself. Thirdly, they collected selected service indica-
carried out four and 3/29 carried out five. The auto-
tors routinely reported by each facility to the Ministry
evaluacion scores reported by the facilities increased
of Health. We used these data to derive two EmOC
with each round (Fig. 1). These data indicate the institu-
quality of care measures, which we described below.
tional Proyecto 2000 interventions were implemented
and suggest the interventions could have been strong 3.2. Household data
enough to improve the quality of EmOC services on
To assess changes in local utilization patterns and
measure SMI Program participation, the Proyecto 2000
evaluators carried out a household survey in all treat-
ment and control facility service areas. The survey
instrument incorporated selected items from Peru’s
DHS III and DHS IV survey questionnaires [26],
particularly household characteristics, birth histories
and pregnancy-related behaviors. Sampling procedures
Fig. 1. Autoevaluacion scores by evaluation round, 2000–2001, were similar to those used in the DHS. Peru’s 1993 cen-
Proyecto 2000. sus of households provided the sampling frame. Within
 M.J. McQuestion, A. Velasquez / Health Policy 77 (2006) 221–232
each Proyecto 2000 catchment area, census tracts were
listed and selected at random. Within each tract house-
holds were selected systematically following cardinal
directions from the approximate center of each cluster.
Ten women who had given birth in the previous 5 years
were surveyed in each cluster. The measures we derive
from these household-level data described below. We
merged the facility and household data to make a hier-
archical dataset consisting of 5335 women nested in
420 clusters in 58 facility catchment areas.
3.3. Facility quality of care measures
Our two quality of care and basic EmOC capac-
ity measures, along with other facility indicators, are
shown in Table 1. The first measure (autevaluacion
score) is the first principal component (eigenvector)
from a factor score analysis of nine items from the
autoevaluacion checklist. The nine items were: blood
is routinely filtered, an incinerator is present, there
are generic versus proprietary drugs are in the phar-
macy, there is an up-to-date list of all drugs dispensed,
patients receive health educational messages, patient
satisfaction is measured, remedial activities to improve
patient satisfaction were implemented, there is a local
community advisory committee, staff meets at least
every 3 months, feedback on performance is given at
those meetings. We computed Cronbach’s alpha and the
Kaiser–Meyer–Olkin measure of sampling adequacy
[27] for these nine items. The resulting coefficients
were, respectively, 0.70 and 0.65 (results not shown).
We conclude the nine items are tapping a common
underlying construct but we note that 0.80 is the con-
ventional “gold standard” for both measures [27].
The second measure, EmOC capacity score, is the
percent score on a battery of 711 items the evaluators
used to assess the technical capacity of a facility to deal
with obstetric emergencies. The evaluators grouped the
indicators into nine categories: human resources, pre-
natal and obstetric equipment, radiology, pharmacy,
delivery room equipment, neonatal care unit, maternity
ward, operating room and blood bank.
As Table 1 shows, treatment facilities scored higher
on both the autoevaluacion quality of care index and
EOC capacity score. This apparent improvement could
be a true difference due to the Proyecto 2000 inputs
or it could be an artifact of the non-random match-
ing of treatment and control facilities, attrition or other
225
sources of bias. To explore this further we used four
of the routinely reported EmOC facility indicators to
compute a propensity score for the assignment pro-
cess. The aim of propensity scoring is to make assign-
ment “strongly ignorable” by blocking observations
on observables [28,29]. The outcome is the dummy
variable indicating assignment to treatment or control
group. The covariates we used are: number of obstetri-
cians and gynecologists on staff, number of maternal
deaths in 2000, number of caesarian sections performed
in 2000 and the proportion of all deliveries performed
outside of the facility. We generated a balanced score
with matched pairs of facilities falling into eight blocks
(results not shown). We then used the propensity score
to generate three alternative non-parametric treatment
effects estimates for each quality of care measure.
3.4. Household measures
We control for several household risk factors in our
models. Maternal education is a positive predictor of
maternal behaviors in Peru [14]. Other important fac-
tors include maternal age, number of births and socioe-
conomic status [30]. Maternal educational attainment
is coded using terciles, where 1 = no or primary educa-
tion, 2 = some secondary and 3 = completed secondary
and higher. Another dummy variable is coded one for
women who have had three or more live births, zero
otherwise. To control for household wealth we use the
Filmer–Pritchett method [31], wherein weights from
principal components are applied to a list of household
assets, scores are summed and ranked and each house-
hold is assigned to one of five wealth quintiles. We add
additional dummy variables to control for whether the
last child was born in 1998, 1999, 2000 or 2001. We
use a binary dummy variable to indicate whether or not
the household participates in the SMI Program.
As shown in Table 2, the characteristics of Proyecto
2000 sample households were broadly comparable
across treatment and control areas. Only ethnicity var-
ied: treatment area women were less likely to be Span-
ish speakers. Delivery patterns also appear similar
across the study arms. Four of every five women in both
treatment and control areas delivered their last babies in
some kind of institution. Though the matching appears
adequate, the Proyecto 2000 sample is not a nationally
representative sample. Table 2 shows the same indica-
tors computed from Peru’s DHS IV survey. The DHS
226 M.J. McQuestion, A. Velasquez / Health Policy 77 (2006) 221–232

Table 2
Sample characteristics and maternal health indicators, women giving birth in previous 5 years, Peru 1996–2002
Variable Proyecto 2000 t-Test DHS IV

Control facilities Treatment facilities Matched to P2000 Full sample

Mean S.D. Mean S.D. Mean S.D. Mean S.D.

Last birth institutional 0.83 0.37 0.82 0.38 0.77 0.21 0.53 0.50
Last birth prenatal care 0.91 0.29 0.91 0.29 0.87 0.11 0.78 0.42
Mothers characteristics
Age (years) 26.64 6.62 27.67 7.10 −5.38a 29.32 1.41 29.21 7.05
No. live births 2.57 1.70 2.66 1.88 2.92 0.60 3.51 2.47
Educational level
Primary 0.21 0.40 0.23 0.42 −2.37a 0.31 0.15 0.45 0.50
Secondary 0.44 0.50 0.44 0.50 0.38 0.15 0.32 0.47
Superior 0.35 0.48 0.32 0.47 0.26 0.13 0.14 0.34
Union status
Married 0.38 0.48 0.38 0.49 0.42 0.16 0.41 0.49
Consensual 0.48 0.50 0.47 0.50 0.43 0.16 0.47 0.50
Divorced/separated/widow 0.15 0.36 0.15 0.36 0.08 0.05 0.07 0.25
Rural origin 0.39 0.49 0.36 0.48 0.26 0.19 0.40 0.49
Non-Spanish speaker 0.02 0.15 0.08 0.27 −9.30a 0.10 0.17 0.22 0.42
Households
Electricity 0.93 0.25 0.90 0.30 −4.05a 0.80 0.20 0.52 0.50
Safe water 0.86 0.35 0.81 0.39 4.64a 0.97 0.12 0.81 0.39
Durable floor 0.53 0.50 0.56 0.50 −2.40a 0.44 0.11 0.53 0.50
Safe toilet 0.59 0.49 0.62 0.49 −2.09a 0.75 0.19 0.60 0.49
n 2514 2821 5826 13832
a Significant at p < 0.05 level.

IV sample is a nationally representative weighted sam- cator variables that control for the effects of both pro-
ple drawn from 589 of the 1789 distritos enumerated grams. We interpret their slopes as indirect treatment
in Peru’s 1993 household census. We used the distrito estimates. Given the heteroscedastic treatments and the
identifiers to match the DHS IV and Proyecto 2000 data many suspected unobserved variables that could have
(n = 68 matched distritos). The Proyecto 2000 sample affected mothers’ delivery choices, we fit two-level ran-
is somewhat better educated, more likely to be Spanish- dom effects models of the form:
speaking and living at a slightly higher socioeconomic
yij = πij + εij
level than the DHS subsample from the same distri-
tos. Compared to the national DHS sample, women in logit(πij ) = β0j + β1 Xij + β2 Tij + β3 Iij
the Proyecto 2000 distritos were more intensive mater- +β4 Pj + β5 Iij Pj
nal health service users, better educated, more likely to β0j = δ0j + δ01 z1j + κj
speak Spanish and less likely to have households with
electricity. Accordingly, all inferences we will make
are limited to the Proyecto 2000 sample data. εij ∼ N(0, 1), cov(Xij , Pj , Iij , Tij , εij ) = 0
κj ∼ N(0, σκ2 ), cov(z1j , κj ) = 0
3.5. Behavioral model cov(εij , κj ) = 0
As mentioned, we estimate a facility-level Proyecto In this model πij is the probability mother i in EmOC
2000 treatment effect using propensity scoring. Here facility service area j chose institutional delivery yij ,
we describe our behavioral model, which includes indi- and εij is an individual error term. β1 is a parame-
 M.J. McQuestion, A. Velasquez / Health Policy 77 (2006) 221–232 227

ter measuring individual effects due to household and
individual covariates Xij . β2 measures the effect of time,
specified as dummy variables for child i’s birth year Tij .
β3 controls for SMI Program participation, indicated
by Ii , which is coded one for participants, zero for non-
participants. β4 controls for being in a Proyecto 2000
treatment facility area, indicated by Pj , a dummy vari-
able coded one for Proyecto 2000 treatment distritos,
zero otherwise. We include β5 to capture any cross-
level interaction between the two treatments. This term
also adjusts for the possibility the insurance program
was not uniformly implemented across the Proyecto
2000 areas. β0j is a random facility-level intercept,
δ0j and δ01 are parameters, z1j is a dummy variable
for facility and kj is a facility-level random effect. If
the variance of kj , denoted as σk2 , is significant, then
we know there are unobserved variable effects which
might otherwise have biased the fixed effect parameters
in a conventional model.
To fit the behavioral model we must make several
assumptions. We assume that each mother is influenced
solely by her own EmOC facility. We further assume
that all mothers in the Proyecto 2000 treatment areas
were equally exposed to the treatments and that access
to the nearest Ministry of Health EmOC facility did not
differ between treatment and control areas. This was
not the case for the SMI Program, which was not uni-
formly implemented and was means-tested. Although
we lack any SMI Program data, we assume that all eli-
gible households with access did enroll. Because the
subsidy was conditioned on use of the nearest Ministry
EmOC facility we assume that any SMI Program effect
contributed fully to the likelihood of our outcome.
We estimate a series of nested multilevel models
using Stata’s gllamm program [32]. The program uses
a maximum likelihood algorithm with adaptive quadra-
ture to model latent variables as random effects. One
advantage of gllamm over other multilevel programs
is that it generates log-likelihood statistics useful for
comparing model fits. All standard errors are estimated
using the Huber–White sandwich estimator to adjust
for the clustered survey design effect [33].
4. Results
4.1. Institutional model
Our institutional treatment effects are shown in
Table 3. The estimates include population-average
treatment effects (ATE) produced by radius matching
and nearest neighbor matching algorithms. Following
Imbens (2003), we also estimate the within-sample
ATE. For comparison, we report the slope of a simple
OLS model with the treatment dummy the sole regres-
sor. The two indicators are measured on different met-
rics so their treatment effect estimates are not directly
comparable. What we expect are consistent estimates
for each indicator. Inferences are based on Wald tests.
With the outcome the EmOC capacity score, popula-
tion and sample ATE estimates were all significant.
With the autoevaluacion factor score as the outcome,

Table 3
Facility-level treatment effects estimates, conditioned on propensity scorea , Proyecto 2000
EmOC capacity (n = 52 facilities) Autoevaluation (n = 55 facilities)

Coefficient S.D. (Pairs) Coefficient S.D. (Pairs)

OLS slope 12.2** 2.8 0.55** 0.28
Radius matching ATTb 11.7** 2.6 (26t,18c) 0.5 0.43 (6t,16c)
Nearest neighbor
Random draw ATT 12.1** 2.7 (26t,11c) 0.55* 0.32 (26t,11c)
Equal weights ATT 12.1** 2.7 (26t,11c) 0.55 0.36 (26t,11c)
Sample ATEc 12.8** 2.9 (43t,43c) 0.65** 0.27 (n = 41t,41c)
a Propensity score variables: number of ob-gyns, no. maternal deaths 2000, no. caesarian sections 2000, proportion of all cases delivered in

facility in 2000.
b ATT, average treatment effect on the treated.
c ATE, average treatment effect.
* Significant at 0.10 < p < 0.05 level.
** Significant at p < 0.05 level.
228 M.J. McQuestion, A. Velasquez / Health Policy 77 (2006) 221–232

we find a significant sample ATE (0.65) but only one
population average treatment effect, and it is marginally
significant. The EmOC capacity score is apparently a
more sensitive quality of care measure. We conclude
the estimates are robust and that Proyecto 2000 inputs
did improve the quality of care in the EmOC treatment
facilities. Our main interest, however, is in measuring
any health behavioral impacts and assessing whether
they are linked to facility quality of care improvements,
to the provision of MCH insurance or a combination of
the two.
4.2. Behavioral model
Our behavioral model results are shown in Table 4.
We show exponentiated slopes (odds ratios) to ease
interpretation. In Model 1, the reference household
model, covariate effects are signed as expected. The
more educated and wealthier the woman, the more
likely she delivered in the EmOC facility. Those who
do not speak Spanish and have had three or more live
births are less likely to choose institutional delivery.
The dummy variables for birth years 1999, 2000 and
2001 capture unmeasured variables that are associated
with EmOC delivery. Those net effects are positive
compared with those in 1997 and 1998, the omitted
categories. There is a significant random effect, indi-
cating that women’s decisions to deliver in the public
EmOC facility are correlated in some catchment areas
more than in others due to omitted variables that jointly
affect their behaviors.
Model 2 results show that living in a Proyecto 2000
area has no significant effect on delivery choice. Model
3, in contrast, shows that the odds of institutional deliv-
ery for women covered by the SMI Program were twice
the odds for women not covered. Controlling for insur-
ance removes upward biases on the highest education
and wealth dummies. The difference in log-likelihoods
shows that Model 3 is also a significantly better-fitting
model than Models 1 or 2. Model 4 includes an inter-
action term between the highest wealth quintile and
the insurance dummies. These better-off households
were ineligible for coverage and the negative interac-
tion term captures this fact. Controlling this interac-
tion further decreases the direct effects of being in the
wealthiest quintile. The most dramatic effect, however,
is a seven-fold increase the odds of EmOC delivery for
the insured women. This pattern is consistent with the
fact only the poorest households were eligible for the
SMI Program. In Model 5 we add a cross-level interac-

Table 4
Two-level logistic regression delivery models, exponentiated effects, Proyecto 2000
Variablea Coefficient (S.E.)

Model 1 Model 2 Model 3 Model 4 Model 5

Non-Spanish speaker 0.38** (0.07) 0.38** (0.07) 0.36** (0.07) 0.36** (0.07) 0.36** (0.07)
Some secondary education 2.69** (0.27) 2.69** (0.27) 2.59** (0.26) 2.61** (0.27) 2.61** (0.27)
Complete secondary or more 5.63** (0.83) 5.63** (0.83) 4.81** (0.72) 4.75** (0.71) 4.76** (0.71)
Three or more live births 0.66** (0.06) 0.66** (0.06) 0.64** (0.06) 0.64** (0.06) 0.64** (0.06)
60–79th wealth quintile 2.23** (0.31) 2.24** (0.31) 2.10** (0.30) 2.13** (0.30) 2.13** (0.30)
80–100th wealth quintile 3.26** (0.61) 3.27** (0.62) 2.86** (0.54) 2.25** (0.46) 2.26** (0.46)
Born 1999 1.30** (0.15) 1.30** (0.15) 1.29** (0.15) 1.29** (0.15) 1.29** (0.15)
Born 2000 1.75** (0.22) 1.75** (0.22) 1.76** (0.22) 1.76** (0.22) 1.76** (0.22)
Born 2001 1.62** (0.20) 1.62** (0.20) 1.64** (0.20) 1.63** (0.20) 1.64** (0.20)
Insured 2.02** (0.25) 15.71** (14.52) 14.96** (13.98)
Insured 80–100th quintile* 0.34** (0.16) 0.34** (0.16)
P2000 treatment area 0.79 (0.25) 0.80 (0.26)
P2000 area insured* 1.08 (0.26)
Level-two random effect σk2 1.06** (0.27) 1.05** (0.26) 1.11** (0.28) 1.00** (0.28) 1.09** (0.24)
Log-likelihood −1797.4 −1797.1 −1780.1 −1777.0 −1776.8
n 5190 5190 5190 5190 5190
a Omitted categories: education secondary and beyond, Spanish speaker, one or two live births, lowest three wealth quintiles, born 1998, no

insurance.
* Significant at 0.10 < p < 0.05 level.
** Significant at p < 0.05 level.
 M.J. McQuestion, A. Velasquez / Health Policy 77 (2006) 221–232
Fig. 2. Real and simulated posterior means Models 1–5.
tion term to test whether the two programs interacted to
affect EmOC probabilities. The interaction is insignif-
icant.
Comparisons of the models with their conventional
single-equation logit analogs show that, in each case,
the random effects specification improved model fit
(results not shown). To check whether the normality
assumption was met, we standardized and plotted the
posterior means from each model. All distributions
were near normal but somewhat negatively skewed.
The skewness was due to three clinics (two treatment,
one control) whose means were more than two standard
deviations below the sample means. We used gllamm’s
post-estimation command gllasim to resample the pos-
terior means. Results are shown in Fig. 2. The simulated
distributions were normal and no longer skewed. Fol-
lowing arguments by Longford [34], the three clinics
are thus not true outliers; their apparent outlier status is
a feature of the realized sample. We leave them in the
dataset and conclude that the models are robust.
5. Discussion and conclusions
5.1. Project impacts
Our results show that Proyecto 2000 improved the
quality of care on offer but did not directly increase the
229
probability of delivery in Ministry of Health EmOC
facilities. Nor was there an interaction between the
system-level Proyecto 2000 inputs and the household-
level SMI Program. Though they targeted the same
sub-population, each program operated independently.
The only behavioral impact we document is that of the
SMI Program. It shows, simply, that reducing out-of-
pocket costs increases EmOC utilization. The poorest
Peruvian women clearly benefited from the targeted
insurance program, however, the household risk fac-
tor effects remained consistently negative across the
models, indicating that neither program significantly
reduced socioeconomic or ethnic disparities in EmOC
utilization.
Behavioral impacts due to Proyecto 2000 may have
been too weak to be detectable or may have occurred
after the endline survey. As shown in the DHS data,
the share of births delivered in Ministry EmOC
facilities rose nationwide during the period. Looking
at our sample, we also see increasingly positive
period effects, represented by the slopes on the birth
year dummies in our models. The forces propelling
those increases were likely more decisive than any
attributable to Proyecto 2000. A lagged Proyecto 2000
treatment effect would be plausible for two reasons.
First, only about 40% of the women surveyed gave
birth during the project’s most intensive second phase
(2000–2002). Although the dummy variables for
230 M.J. McQuestion, A. Velasquez / Health Policy 77 (2006) 221–232
birth year capture a rising probability of EmOC use,
the majority of women interviewed may have been
unaware of any local improvements when they made
their birthing decisions, or any improvements made
may not have been noticeable. Second, delivery behav-
iors may be socially mediated. If so, the observation
period may have been too short for social learning
or other endogenous social processes to reach some
theoretical threshold level of women. The data did
not permit us to test this hypothesis, however, social
forces are one possible source of the consistent cor-
relation of birthing decisions within catchment areas
captured by the random effects. Future studies would
do well to explore these social aspects of maternal
behaviors.
5.2. Limitations
There are a number of methodological shortcomings
in this study. The relatively rich quasi-experimental
Proyecto 2000 data allowed us to estimate a treat-
ment effect for that program. However, we lacked
any kind of design for evaluating the SMI Program.
Strong designs are needed in order to evaluate such tar-
geted programs. A recent example was Gertler’s 2000
[35] evaluation of Mexico’s Progresa Program. In that
study, Gertler used difference-in-differences estimators
and panel data from households in randomly sam-
pled treatment and control areas to show the program
increased school enrollment and health services utiliza-
tion and improved health outcomes. Had panel data
or even repeated cross-sectional data from the same
catchment areas been available we might have detected
household-level Proyecto 2000 treatment effects.
Our study also faced obvious sampling problems.
Attrition of the Proyecto 2000 facilities during Phase
I and the replacement of 14 of the original control
facilities with new ones at endline are likely sources
of sample selection bias. If the attriting EmOC facil-
ities were the stronger institutions then any treatment
effect would be underestimated. We lacked the data
necessary to assess this. The targeted nature of the two
programs presents another potential source of bias in
that the characteristics of facilities and households not
given treatment are likely to differ from those that did
receive treatments. We estimated Proyecto 2000 treat-
ment effects using a propensity score balanced on just
four observable covariates; many other, unmeasured
covariates could differ systematically across the two
groups. Regarding the SMI Program, the beneficiaries
we observed may differ from other potential beneficia-
ries in Proyecto 2000 catchment areas where the insur-
ance program had not yet been implemented. A more
general problem are background disturbances caused
by the constantly evolving mix of EmOC services many
Peruvian communities faced as public health services
decentralized and to some extent recentralized. In this
fluid policy environment, perceptions of EmOC qual-
ity, perhaps the legitimacy of public health services in
general, were in flux.
5.3. Policy implications
Peru’s SMI Program proved an effective means of
inducing high-risk women to use public EmOC facil-
ities. We document here its short-term impacts. They
show that cost is a significant barrier to many women.
However, such subsidized programs are generally fis-
cally unsustainable, particularly in poor countries. Fur-
ther, they may not be efficacious. The subsidies could
merely act as side payments for compliance and when
the subsidies end, the desired behavior, here use of
EmOC, may end too. The long-term sustainability
of targeted subsidy programs is an area where more
research is needed.
Proyecto 2000 sought to induce greater EmOC uti-
lization through more elaborate, technical strategies. It
theorized that improving institutional quality of care,
educating the public and working with communities
would be sufficient to induce behavioral change. We
lacked data on the latter but the data we did have
showed the first goal was achieved. Improved quality,
our results suggest, is not sufficient to change delivery
behaviors. Something else is needed. Recently, Gilson
[36] proposed a theory wherein trust, initially between
client and provider and later between community and
the state, is a necessary condition for communities to
become healthier. For this to happen people must per-
ceive the quality of care to be high and the public health
services to be legitimate. If out-of-pocket cost is a bar-
rier, then targeted subsidies may be warranted as an
interim measure. Studies elsewhere have shown even
the poorest people are willing to pay for health ser-
vices they value [37,38]. Though our interaction term
was insignificant, we encourage other researchers to
test this hypothesis.
 M.J. McQuestion, A. Velasquez / Health Policy 77 (2006) 221–232 231

Acknowledgements [13] United Nations Development Programme. Human Develop-

ment Reports: Human Development Indicators, 2003. Website:
http://www.undp.org/hdr2003/indicator/cty f PER.html
Technical assistance for Proyecto 2000 was pro-
accessed 4/2004.
vided by Pathfinder International with funding from [14] Elo IT. Utilization of maternal health-care services in Peru:
USAID-Peru, Contract 527-0366-C5049-00. Support the role of women’s education. Health Transition Review
for our study was provided by the Bill and Melinda 1992;2:49–69.
Gates Institute for Population and Reproductive [15] Vallenas G. Maternal mortality in Peru. Revista Peruana
de Poblacion 1993;3:33–56. Website: http://www.paho.org/
Health, Department of Population and Family Health
English/DD/AIS/cp 604.htm accessed 4/2004.
Sciences, Johns Hopkins Bloomberg School of Public [16] de Meer K, Bergman R, Kusner JS. Socio-cultural determi-
Health. nants of child mortality in southern Peru: including some
methodological considerations. Social Science and Medicine
1993;36:317–31.
[17] Averting Maternal Death and Disability Working Group on
References Indicators. Using UN process indicators to assess needs in emer-
gency obstetric services: Pakistan, Peru and Vietnam. Inter-
national Journal of Gynaecology and Obstetrics 2002;78:275–
[1] Arroyo-Laguna J. Greater autonomy for public hospitals in
82.
Lima and the consequences of that on the use of health ser-
[18] Meentzen A. Estrategias de desarrollo culturalmente adecuadas
vices, 1988–1997. Pan American Journal of Health 1999;6:301–
para mujeres indigenas. Unpublished document. Washington,
210.
DC: Unidad de Pueblos Indigenas y Desarrollo Comunitario.
[2] Ministry of Health. Lineamientos de polı́ticas del sector salud
Banco Interamericano de Desarrollo, 2000.
1995–2000: hacia un sector salud con eficiencia, equidad y cal-
[19] Pan American Health Organization. Situacion de salud de los
idad. Lima: Ministerio de Salud; 1996.
pueblos indigenas de Peru. Document OPS/HSP/HSO/98.08.
[3] Pan American Health Organization. Core select data indicators.
Washington, DC: Pan American Health Organization; 1998.
Peru, 2004.
[20] Francke BP, editor. Politicas de Salud 2001–2006. Diagnos-
[4] MEASURE DHS+. Country statistics: Peru. Demographic and
tico y Propuesta 5. Lima: Consorcio de investigacion eco-
Health Surveys, 2004. Website: http://www.measuredhs.com/
nomica y social, 2001. Available at: http://www.consorcio.org/
countries/country.cfm?ctry id=33 accessed 4/2004.
CIES/html/diag5.asp.
[5] Hill K, Abou-Zahr C, Wardlaw T. Estimates of maternal mor-
[21] Donnay F. Maternal survival in developing countries: what has
tality for 19/95. Bulletin of the World Health Organization
been done, what can be achieved in the next decade. Interna-
2001;79:182–93.
tional Journal of Gynaecology and Obstetrics 2000;70:89–97.
[6] Pan American Health Organization. Regional strategy for
[22] Donabedian A. The definition of quality and approaches to
maternal mortality and morbidity reduction, Document
its assessment. Ann Arbor, Michigan: Health Administration
CSP26/14. Washington, DC: Pan American Health Organiza-
Press; 1980.
tion; 2002.
[23] McCarthy J, Maine D. A framework for analysing the deter-
[7] Murray SF, Davies S, Kumwenda Phiri R, Ahmed Y. Tools
minants of maternal mortality. Studies in Family Planning
for monitoring the effectiveness of district maternity referral
1992;23:23–33.
systems. Health Policy and Planning 2001;16(4):353–61.
[24] Benavides B, Seclen JE, Novata J, Velasquez A. Impacto del
[8] World Health Organization. Mother-baby package. Implement-
mejoramiento continuo de la calidad en los servicios materno
ing safe motherhood in countries. Document FHE/MSM/94.11.
perinatales del Peru. La experiencia del Proyecto 2000. Unpub-
Maternal health and safe motherhood programme. Geneva:
lished document. Lima: Pathfinder Fund, 2000.
World Health Organization; 1994.
[25] Seclen J, Benavides B, Jacoby E, Velasquez A, Watanabe E.
[9] Instituto Nacional de Estadistica e Informatica. Encuesta
Existe una relacion entre los programas de mejora de la calidad
demografica y de salud familiar 1996. Lima: Instituto Nacional
y la satisfaccion de usuarias de atencion prenatal? Experiencia
de Estadistica e Informatica, 1996. Accessed at: http://www.
en hospitals de Peru. Revista Panamericana de Salud Publica
paho.org/English/SHA/coredata/tabulator/newTabulator.htm
2004;16:149–57.
23 October 2004.
[26] EASURE DHS+. Peru 2000 DHSIV Final Report. FR-120.
[10] Instituto Nacional de Estadistica e Informatica. Encuesta
Calverton, MD, 2000.
demografica y de salud familiar 2000. Lima: Instituto Nacional
[27] Cureton EE, D’Agostino RB. Factor analysis: an applied
de Estadistica e Informatica; 2000.
approach. Hillsdale, NJ: Erlbaum; 1983.
[11] Ross JA, Campbell OMR, Bulatao R. The maternal and neona-
[28] Rosenbaum PR, Rubin DB. The central role of the propensity
tal program effort index. Tropical Medicine and International
score in observational studies for causal effects. Biometrika
Health 2001;6:787–98.
1983;70:41–55.
[12] Futures Group. Herramienta para la incidencia politica en la
[29] Imben G. Nonparametric estimation of average treatment
salud materna: MNPI (indice de esfuerzo de programa materno
effects under exogeneity: a review. Technical Working Paper
y neonatal). Lima: Futures Group; 2002.
232 M.J. McQuestion, A. Velasquez / Health Policy 77 (2006) 221–232
293. Cambridge MA: National Bureau of Economic Research,
2003. http://www.nber.org/papers/T0294.
[30] AbouZahr CL, Wardlaw T. Antenatal care in developing
countries. Promises, achievements and missed opportunities.
Geneva: World Health Organization; 2003.
[31] Filmer D, Pritchett L. Estimating wealth effects without
expenditure data or tears: an application to educational
enrollments in states of India. Demography 2001;38:115–
32.
[32] Rabe-Hesketh S, Skrondal A, Pickles A. GLLAMM manual.
Paper 160. UC Berkeley Division of Biostatistics Working
Paper Series. The Berkeley Electronic Press, 2004. http://www.
bepress.com/ucbbiostat/paper160.
[33] Stata Corporation. Stata statistical software. Release 6.0. Col-
lege Station, TX: Stata Corporation; 1999.
[34] Longford NL. Simulation-based diagnostics in random-
coefficient models. Journal of the Royal Statistical Society A
2001;164(Part 2):259–73.
[35] Gertler P. Final report. The impact of Progresa on health.
Unpublished paper. International Food Policy Research Insti-
tute. Washington, DC: International Food Policy Research Insti-
tute, 2000.
[36] Gilson L. Trust and the development of health care as a social
institution. Social Science and Medicine 2003;56:1453–68.
[37] Acharya L, Cleland J. Maternal and child health services in rural
Nepal: does access or quality matter more? Health Policy and
Planning 2000;15:223–9.
[38] Ross G, Zeballos J, Infante A. La calidad y la reforma del sector
salud en America Latina y el Caribe. Revista Panamericana de
Salud Publica 2002;9:93–7.