The impact of updating health benefits plans on health technologies

usage and expenditures: the case of Colombia

Oscar Espinosa1,2 † , Jhonathan Rodrı́guez1 , Diego Ávila1 , Paul Rodrı́guez-Lesmes1,3 ,

Hernán Enrı́quez1,4 , Sergio Basto1 , Giancarlo Romano1 , and Lorena Mesa1
†
E-mail: oscar.espinosa@iets.org.co; Phone: +57 313 463 6126
1
Instituto de Evaluación Tecnológica en Salud (IETS)
2
Universidad Nacional de Colombia
3
Universidad del Rosario
4
Universidad Sergio Arboleda

July 1, 2021

PLEASE DO NOT DISTRIBUTE THIS DOCUMENT WITHOUT THE CONSENT OF THE

IETS AND MINHACIENDA

Abstract
The expansion of health benefit plans (HBP) is a central concern for maintaining universal
health coverage of countries while keeping health expenditures under control. Using adminis-
trative data on yearly usage (frequency and number of users) and expenditures per user, we
estimate the impact on usage and expenditures of health technologies after their inclusion into
the Colombian HBP. This is attained by comparing technologies (drugs and procedures) before
and after their inclusion, which is decided by the Government yearly, against other technologies
which are not included but are still used in the system due to a particularity of the Colombian
compulsory health insurance system. For such comparison, we use first a Callaway-Sant’Anna’s
Difference-in-Differences with Multiple Time Periods method, and second a synthetic control
strategy. On average, we find a substantial increase in usage of included technologies. Also,
there is an increase on expenditure levels per user of such technologies. However, there is vari-
ability on the response over time.
Keywords:
Health benefits plan, universal health coverage, health expenditures.
JEL classification: C55, I13, I18.
Acknowledgements:
Project financed jointly between the Ministerio de Hacienda y Crédito Público (Colombia)
and the Instituto de Evaluación Tecnológica en Salud, through Administrative Agreement 7016-
2020.
Conflict of interest statement:
The authors declare, under the methodology established by the Instituto de Evaluación Tec-
nológica en Salud (IETS), that there is no invalidating conflict of interest of financial, intellectual,
property, family, or any other type that could affect the development of this analysis and subse-
quent document.
1 Introduction
Achieving universal health coverage (UHC) is one of the Sustainable Development Goals (SDG)
targets, as part of the general objective of promoting the well-being of societies. UHC is the pursuit
of a set of good quality health goods and services that all individuals can access regardless of their
circumstances, the Health Benefit Plans (HBP) (Cotlear, Nagpal, Smith, Tandon, & Cortez, 2015;
Glassman, Giedion, Ú., & Smith, 2017). Coverage has three primary dimensions: who is covered
by insurance, what services are included in the plan, and what proportion of the costs people cover
(Etienne, Asamoa-Baah, & Evans, 2010; Schreyögg, Stargardt, Velasco-Garrido, & Busse, 2005).
Currently, several countries are achieving nearly universal coverage with high levels of financial
protection (Lozano et al., 2020; McKee, Balabanova, Basu, Ricciardi, & Stuckler, 2013). Yet,
Lozano et al. (2020) show that there are countries with similar levels of health expenditures but
with vast differences in the effective access to health services. This indicates the need for prioritising
services and establishing explicit HBP is one of the standard tools for it. Maintaining an explicit
plan is an arduous task in practice, with barriers such as legal provisions, political pressures from
interest groups, and above all, the technical and administrative capacity to update it (Glassman
et al., 2017). The challenge in putting this idea into practice lies in the financing, which involves
defining financial sources and establishing affordable budgets with public resources. Today, and
increasingly so, no society can afford everything, so it is vital to understand if the updating process
of an HBP helps to prioritise access to services while containing costs.
The general objective of this analysis is to evaluate the impact of the public policy of progressive
updating of the Colombian HBP, which will make it possible to determine the effect of the inclusion
of technology in terms of its use and its costs. As several middle-income countries, Colombia
provides health insurance coverage to nearly all its citizens (97% in 2016). However, it has still
several shortcomings in terms of quality and timeliness of health services (Herrero & Herrera, 2018;
OECD, 2018).
Yet, the institutional context of the country provides a unique setting for evaluation: the health
system combines a mandatory medical insurance that contains an explicit HBP with a clear process
of periodic updating, with an implicit rationing scheme with which some technologies not covered by
the HBP are financed. Therefore, it is possible to observe how the use and expenditures on specific
health technologies initially financed under an implicit system, change after they are included in
the explicit HBP.
Most of the literature related to HBP studies the introduction of HBP on populations with-
out prior access to health insurance, not the marginal expansion of benefits. This is the case
of poor and vulnerable populations in Bangladesh, Colombia, Georgia, India, Mexico, Philipines
(Aggarwal, 2010; El Omari & Karasneh, 2020; Erlangga, 2018; Galárraga, Sosa-Rubı́, Salinas-
Rodrı́guez, & Sesma-Vázquez, 2010; Giedion, Ú., Panopoulou, & Gomez-Fraga, 2009; Gotsadze,
Zoidze, Rukhadze, Shengelia, & Chkhaidze, 2015; Khan et al., 2020; Miller, Pinto, & Vera-Hernández,
2013), or more general populations in Burkina Faso, China, Ghana, Nigeria, Sri Lanka and Vietnam
(Fink, Robyn, Sié, & Sauerborn, 2013; Gaag et al., 2013; Gnawali et al., 2009; Green, Hollingsworth,
& Yang, 2021; Kumara & Samaratunge, 2019; Lambon-Quayefio & Owoo, 2017; Nguyen, 2011;
Thuong, 2020). This literature considers the impact of households’ out-of-pocket expenditures,
usage of health services, and in some cases health status. The expansion of benefits plans have
been only explored by the Colombian Government (Unión Temporal: Econometrı́a S.A. – SEI –
SIGIL Consulting Group S.A., 2011), which studied the expansion of the health plan of the poor in
Colombia (Regimen Subsidiado, RS) in 2012. This expansion matched the RS HBP to the standard
HBP of those Colombians who pay a prime via payroll taxes (Regimen Contributivo, RC)1 . For
a chosen set of 11 technologies, the authors found an increase on the usage of health services by
individuals in the RS.
1
HPB is understood as the package that is financed through the Capitation Payment Unit.

2
 We contribute to the literature by being the first study, to our knowledge, that explores the
implications in terms of health expenditure and usage of health technologies at the technology
level. We do so by reconstructing the counterfactual usage of technology using other technologies
that are still not included in the HBP via a synthetic control strategy, between 2012 and 2018. This
is possible due to the existence of administrative datasets that cover all purchases of technologies
(medications, devices, and services) needed for the attention of every citizen covered under the RC
system. These datasets exist to compute the capitation payment per individual provided to each
insurer.
In the following section, we describe the background of the Colombian HBP and how it compares
with other health systems. Next, the datasets are presented, as well as the details of the synthetic
control strategy. Finally, discussion and conclusions follow the results of the exercise.

2 Background: the Colombian HBP in context

2.1 The development and functioning of the Colombian HBP
Colombian HBP was set by Law 100 of 1993, which stated the basis of the UHC system of the
country. It defined the managed competition between health insurers of both the RC and the RS
(Escobar, Giedion, Ú., Giuffrida, & Glassman, 2009; Giedion, Ú. & Villar, 2009). People employed
in the formal sector pay a payroll tax that gives them access to the RC; people in the informal
sector or who are not able to pay, access the health system through the RS, which covers health
services through public subsidies.
Individuals choose an insurance company (which are specific to RC and RS) and can move
freely between them. The insurance companies have to organise and guarantee access to healthcare,
defined by the explicit HBP but not limited to it, to their affiliates. Allocation of resources to each
insurer is based on capitation with a risk adjustment based on age, gender, and residence area.
At the beginning of the system, guaranteed healthcare services were restricted to an explicit
HBP called Plan Obligatorio de Salud (POS), specific for RC (based on the de facto HBP of the
former national health insurer) and a less generous one for the RS (based on prior social programs).
Both POS defined the basis of the budget in health care, apart from public health, under the direct
control of local governments. The Law also stated that the POS must be updated, and eventually,
POS from the RC and RS should converge into a unique HBP. However, there was no indication
on how to update each of them and when the plans were going to be unified (Giedion, Ú. et al.,
2009, 2014). Such organisation was the result of the financial efforts of a country where only one in
six individuals was eligible for the RC, and it started with coverage of 30% of the entire population
(Escobar et al., 2009). By 2008, nearly 90% of the population were already covered, and one out
three individuals was part of the RC, with one of the lowest out-of-pocket expenditure levels of the
continent OECD (2018).
Over time, the lack of a transparent process of updating of the POS resulted in two particular
situations. First, insurers could pay for the supply of technologies outside the POS if a team of
doctors establish its need. Another option was to ask for the technology to be provided via judiciary
rulings (tutelas) that protect the right to health. In both cases, these resources are reimbursed by
the government outside their capitation contract. These reimbursements are processed as recobros,
essentially a fee-for-service mechanism. As a result, the initial budget planning failed, and the
system started to suffer problems related to financial sustainability, organisation and functionality
(Unión Temporal: Econometrı́a S.A. – SEI – SIGIL Consulting Group S.A., 2011). In addition to
putting pressure on spending, the payment mechanism for technologies not financed by capitation
generates financial imbalances in both insurers and providers.
In 2008 the Constitutional Court ruling T-760/2008 forced the State to implement both promises
of the Law 100 of 1993, and to find a solution to the crisis. The unification of the HBP for both

3
RC and RS started gradually, and the process was finished by 2012 (Unión Temporal: Econometrı́a
S.A. – SEI – SIGIL Consulting Group S.A., 2011). In terms of the updating HBP process, Law
1438 of 2011 established that the Ministry of Health and Social Protection (MinSalud) had to
develop the updating process. In response to this, the Ministry of Health has issued a series of
resolutions to update the HBP. We will describe these resolutions, which essentially include the
specific technologies to add into the HBP, presented in Figure 1.
In Colombia, the criteria used for the inclusion of new technologies in the HBP are related with: i)
cost-effectiveness; ii) effectiveness; iii) budget impact; iv) safety; v) maximising the improvement of
population health status; vi) burden of disease; vii) severity of disease; viii) equity; ix) affordability,
and x) access.
Finally, a large reform took place in 2015 with the enactment of Law 1751 of 2015, a constitu-
tional amendment, that introduced a major change in the health system’s set up. Instead of the
explicit list of health benefits of the POS, the law stated that any technology could potentially be
publicly funded unless it is explicitly defined that it shouldn’t. The procedure to ask for a recobro
was systematised using a digital tool managed by each prescribing physician. Yet HBP, now called
Plan de Beneficios en Salud - PBS, still establishes the capitation payment to each insurer.

Figure 1: Timeline of HBP development in Colombia

1993 Law 100: Introduction of POS
2008 Sentence T-760: Government forced to implement the updating process
POS 2011 Law 1438: Updating process defined
2012
2013 Resolution 5521
2014 Resolution 5926
Data 2015 Law 1751: Role of HBP changed
2015 Resolution 5592
PBS 2016 Resolutions 0001, 6408
2017 Resolutions 0374, 1687, 5269
2018 Resolutions 0046, 5857
2019 Resolution 3512, Circular 017
2020 Resolution 2481
2021 Resolution 0163
Notes: Own elaboration.

2.2 The Colombian HBP in context

In this section we contrast the Colombian HBP system with a group of reference countries from dif-
ferent continents: Canada, Chile, England, Ethiopia, India, Kenya, Malaysia, Mexico, Netherlands,
Norway, and Uruguay. All of these countries are committed to attaining UHC, but the actual level
of effective coverage varies greatly among them. Lozano et al. (2020) constructed a measure (0 to
100) of effective coverage based on 23 indicators, including elements such as health promotion and
treatment of diseases. Panel A of Table 1 shows that countries such as Ethiopia, India and Kenya
have values of 52 or below, while others such as Canada, the Netherlands, or Norway have levels
of 90 or above. Moreover, in all these countries but England, there is some explicit HBP in place.
Colombia, as explained above, has an explicit HBP but with a large outside option that allows for
exceptions. HBP plans can be decided at national or regional levels (Canada, Ethiopia, India),
and in some cases, only the administration occurs at the regional level (Mexico, Norway and New
Zealand). For some countries, HBP applies only for specific groups of the population, like in Kenya;
in others, it is the core of the entire health system.

4
 In terms of the number and nature of criteria on which HBP are defined and updated in the
referral countries, these were divided into three distinct categories, following Hayati, Bastani, Kabir,
Kavosi, and Sobhani (2018): intervention-related criteria, disease-related criteria, and community-
related criteria. Panel B of Table 1 presents the criteria and our classification for each of the
countries. Appendix A presents in detail how we developed this classification. The most widely
used criteria were cost-effectiveness, effectiveness, budget impact and safety in the intervention-
related category, equity, affordability and access in all countries included in the referencing. The
burden of disease is also widely used, especially in developing countries such as Chile, Mexico,
Uruguay, Ethiopia, Kenya and India. On the other hand, innovation and severity of disease are less
used, the former mainly in high-income countries.

Table 1: Criteria for updating HBP in a selected group of countries

ru nds
an
a

al
sia

rla

ay
ia
bi

y
a

Ze
o
an
op

wa
ad

om

gu
ya

ay

ic

he
le

gl

ew
hi

ex
an

en
di

al

or

et
hi

ol

En
Et

In

M
K

U
C

C
Panel A. Characteristics of the system
Advance of the UHC 90 74 74 47 47 88 52 67 61 94 83 90 69
Explicit HBP Yes Yes Mix Yes Yes No Yes Yes Yes Yes Yes Yes Yes
Central system No Yes Yes No No Yes Yes Yes Mix Mix Mix Yes Yes
Panel B. Determinant Criteria of HBP
Intervention-related criteria
Cost-effectiveness X X X X X X X X X X X X X
Effectiveness X X X X X X X X X X X X X
Budget impact X X X X X X X X X X X X X
Safety X X X X X X X X X X X X X
Sustainability X X X X X X X X X X X
Cost of interventions X X X X X X X X X X X X
Maximising the improvement of
X X X X X X X X X X
population health status
Innovation X X X X X X
Disease-related criteria
Burden of disease X X X X X X X
Severity of disease X X X X X X
Community-related criteria
Equity X X X X X X X X X X X X X
Affordability X X X X X X X X X X X X X
Access X X X X X X X X X X X X X
Notes: Own elaboration based on Hayati et al. (2018) criteria categories. Advance of the UHC is measured with the effective coverage of
health services index from (Lozano et al., 2020).

3 Data and methodology

3.1 Data
We consider data between 2012 and 20192 . As shown in Figure 1, our unit of analysis is the
technology, which could be a medicament or a procedure. Procedures are identified in Colombia
according to CUP code, and medications are classified according to the CUM code3 . Therefore,
our dataset considers for each technology, its frequency of usage, its number of unique users, its
2
We use information only for Entidades Prestadoras de Salud (EPS) from RC that approve the validation system
of the Ministry of Health. This EPS have approximately 85.54% of the insured persons of the RC.
3
We use the Anatomical/Therapeutic/Chemical classification (ATC codes) as a system for identification per active
principle or chemical substance (the 5th level) in the pharmaceutical products.

5
expenditures, the geographic area where the technology was accessed, and its status with respect
to the HBP.
There are 9,119 technologies which were not part of the HBP at least once between 2012 and
2019 in the dataset: 7,412 procedures and 1,706 drugs. As our objective is to estimate impacts on
the usage of technologies, we restrict the sample to technologies for which there are recorded usage
at least since 2012. It results in 5,700 procedures and 1,218 drugs (6,917 technologies: 75.9% of the
original sample).

3.1.1 HBP updates

For the analysis, we include a dummy variable that indicates if the technology is part of the HBP
or not. The suficiencia dataset indicates if the technology is part of the HBP as explained above,
but this is also contrasted with the specific resolutions (see Figure 1) as some technologies might be
included but not used in a particular year. Figure 2 shows the total number of technologies added
to the HBP per year.
In the administrative records, we observe 702 technologies included in the HBP and used at some
point between 2014 and 2019. Not all technologies included are eventually used in the health system.
After restricting the data to those used in 2012, we have left with 339 of them: 109 procedures and
230 drugs, a 48,3% of the original sample. This selection means that our estimates correspond to
the inclusion into the HBP of drugs and procedures that have been available for several years in
the market and does not consider completely new technologies.

Figure 2: Inclusion process of technologies in the Colombian HBP

300
Number of technologies added

200

100

0
2012 2013 2014 2015 2016 2017 2018 2019

Notes: Total number of technologies observed to be included in the HBP.

3.1.2 Dependent variables: technologies use and expenditures

To obtain the number of uses of specific technologies and the total expenditures on each of them,
we use two alternative data sources.

6
 The first source of information was the health spending records for each service rendered by
health insurers in the RC between 2012 and 2019, as part of the HBP, which manages funds from
mandatory contributions (Suficiencia, from here onwards). Suficiencia is a restricted dataset which
the Ministry of Health uses to calculate the amount of money that must be transferred every year
to each insurer via capitation.
The second source of information refers to the fee-for-service payment records of technologies
that insurers claim back from the Government (Recobros). As explained above, this is the case
of products not included in the HBP, which are allowed to be used case by case for individual
patients. This dataset only has the name of the technology and the name of the entity that gave
the technology, so the CUP and ATC codes and geographical area were added by the authors (see
appendix B for further details).
With this information, we compute our main dependent variables for specific technologies per
year, which are:

1. Frequency of usage: total units divided by the number of unique users;

2. Number of unique users: total patients that have a record of usage divided by the number of
insured persons in each year;

3. Expenditures per user: the total amount of money in COP of 2019, divided by the total
number of users.

Figure 3 presents the distributions of the variables described above. On average, there are 18,000
users per technology each year, 31,655 records of usage per year, and an expenditure per user on
average 1.210.879 COP per year (370 USD).4

3.2 Empirical Strategy

Consider only technologies not included in the HBP by 2012. There is a staggered adoption of
treatment: inclusion occurs on different periods, and once a health service is included, it will remain
treated forever. While technologies can be excluded from the HBP, an exclusion is a different sort
of treatment.

3.2.1 CS DiD estimator

We follow Callaway and Sant’Anna (2020) DiD estimator. This estimator applies when there is
variation in treatment timing. The standard strategy, the two-way fixed effects model, delivers
biased results (Athey and Imbes, 2021; Boruskay and Jaravel, 2017). 5 In such an econometric
4
Using an exchange rate of 3,281 COP per UDS, the average of 2019.
5
The TWFE model is
Yi,t = αi + αt + δDi,t + ui,t (1)
where Di,t is a dummy that takes the value of 1 if the technology i is part of the HBP on quarter t, and otherwise.
The regression includes fixed effects by technology (αi ) and quarter (αt ).
Its natural extension is the event study:
−2
X L
X
Yi,t = αi + αt + δklead Di,t
k
+ δklags Di,t
k
+ ui,t (2)
k=−K k=0

k
where Di,t is an indicator for technology i being k periods away from initial treatment at time t. For example, if
0 −2 2
technology A is included in 2015Q1 into the HBP, Dit = 1 only in 2015Q1, Dit = 1 only in 2014Q3, and Dit =1
−1 leads
only in 2015Q2. Here Dit is excluded from the regression to avoid perfect collinearity. Thus, δk would assess
violations of the pre-trends assumptions (if any of these coefficients is significant); and δklags would show the impacts
specific per periods after the inclusion into the HPS.
The models above provides unbiased estimates if all inclusions occur at the same time.

7
Figure 3: Distribution of the main outcomes of interest
Panel A. Unique users (1000s)
Kernel density estimate
.15

.1

Density

.05

0
-10 -5 0 5 10
Unique users (1000s)
kernel = epanechnikov, bandwidth = 0.3167

Panel B. Frequency (1000s)

Kernel density estimate
.15

.1
Density

.05

0
0 5 10 15 20
Unique users (1000s)
kernel = epanechnikov, bandwidth = 0.3286

Panel C. Expenditure per individual (Million COP)

Kernel density estimate
.25

.2

.15
Density

.1

.05

0
0 5 10 15 20
Expenditure per unique users (Million COP)
kernel = epanechnikov, bandwidth = 0.2039

Notes: Own calculation using Epanechnikov kernels.

8
model, under the staggered adoption, already treated units end up being used as part of the control
group, which does not allow to identify the average treatment on the treated (ATT). CS model
ensures that each treated technology’s comparison group consists only of units that have not been
treated yet.
The parameters of interest represent the ATT at period t (out of T periods) for technologies
included in quarter g ∈ G (the set of periods where there is an inclusion of a technology):

AT T (g, t) = E[Yi,t (g) − Yi,t (0)|Gi,g = 1] , f or t≥g (3)

where Yi,t is one of the four outcome variables for technology i out of J technologies in quarter
t. Yi,t (g) corresponds to its value if the technology started to be treated in period g (a particular
resolution), and Yi,t (0) its counterfactual value in case of no treatment. Gi,g takes the value of
1 if the technology i is included in the HBP in quarter g, and 0 otherwise (treatment start-time
dummies).
Without covariates and anticipation effects, the central assumption is the conditional parallel
trends based on Not-Yet-Treated groups (CS assumption 5).

E[Yi,t (0) − Yi,t−1 (0)|Gi,g = 1] = E[Yi,t (0) − Yi,t−1 (0)|Di,s=0 , Gi,g = 0] (4)

for each g and each (s, t) ∈ {2, ..., T } × {2, ..., T } such that t ≥ g and s > t. In the equation, Di,t
takes the value of 1 if the technology i is in the HBP at period t. Essentially, the time variation
on the outcome for technologies included into the HBP in period g if no-treatment occurs, can be
recovered with those technologies that have not been included by period t in the HBP (they’ll be
included in period s). Given this assumption, the semi-parametric estimator is defined as:6

ATˆ T (g, t) = E[Yi,t − Yi,g−1 |Gi,g = 1] − E[Yi,t − Yi,g−1 |Di,t = 0] (5)

The ATT(g,t) estimators are aggregated using a weighted average. We compute:

(i) A general result (group time-average) given by
T X
X T
δ= 1{g ≤ t}AT T (g, t)P (G = g) (6)
g=2 t=2

where P (G = g) is a weight based on the number of technologies included in the HBP at period g.
(ii) A event-study aggregation (dynamic effects). For each period τ after the inclusion into the
HBP:
XT
δτ = 1{g + τ ≤ T }AT T (g, g + τ )P (G = g|G + τ ≤ T ) (7)
g=2

3.2.2 Synthetic control

We follow Cavallo, Galiani, Noy, and Pantano (2013), who, in turn, extended Abadie and Gardeaz-
abal (2003) and Abadie, Diamond, and Hainmueller (2010) work on synthetic controls for multiple
treated units. This strategy constructs a control group for a each treated technology by weighting
control technologies in such a way that they resembles the behaviour of the treated technology over
time (prior to their introduction into the HBP).
6
An estimator conditional on covariates is implemented as well. Such version is similar but includes weights based
on the probability of a technology to be included into the HBP in a given period. See Callaway and Sant’Anna (2020)
for further details.

9
 If only one technology would be studied (unit i=1, one of the G treated units), an estimate of
the impact in quarter τ is obtained as follows:
J
1 X (1)
δ1,τ = Y1,τ − wj Yj,τ (8)
G
j=G+1

(1)
where weights wj are derived for each of the control technologies (j ∈ {G + 1, J}). Weights
are
p derived in this0 case from a constrained quadratic optimisation that minimises the distance
(1) (1) (1) J
((X1 − X0 W ) V (X1 − X0 W ) where W = {wj }j=G+1 and Xk∈{0,1} correspond to the vari-
ables to match. We match technologies on the values of Y prior to the inclusion into the HBP.
Matrix V , a symmetric and positive semidefinite matrix, is chosen so that the root mean squared
prediction error (RMSPE) of the pre-inclusion period is minimised.7 The procedure outlined above
for a single treated unit can be extrapolated to multiple treatment units, as in the case of (Cavallo
et al., 2013). As a result, a general estimate can be obtained:
G G J
1 X 1 X X (j)
δτ = Yj,τ − wk Yk,τ (9)
G G
j=1 j=1 k=G+1

For inference, the standard strategy is to use a permutation test. It is implemented by repeating
the analysis described above for each of the control units, as if they were treated (as placebos), in
order to compute an empirical distribution of the random differences that result from the procedure.
This distribution allow us to test if the estimated effects is statistically different from the placebo
results.

4 Results
4.1 Main findings
Table 2 presents the results of the CS-DiD. For the comparison group, we only consider technologies
that were not still included in the HBP by 2019; in that way, we are able to estimate effects for several
years after the inclusion into the HBP. After the inclusion into the HBP, the overall effects show an
increase of nearly 25,000 new unique users, 59,000 more entries per technology, and around 2 million
COP (619 USD) more on expenditures. All these results were significant at the 95% confidence level.
Yet, the dynamic specification shows differences in the pre-inclusion period, which are significant
at the 95% level, which usually introduces a sign of caution for potential parallel-trends violation.
However, Figure 4 shows that the differences observed post-intervention are considerably larger than
those pre-intervention. Therefore, while there might be doubts on the precise estimate, it is unlikely
that the observed results are drive-by differential trends prior to the inclusion into the HBP. The
graph also makes evident that current data allows for solid conclusions, at most three years after
the inclusion.
If we consider the group-time ATT (table 4 in the appendix), it is clear that the frequency and
number of users results are driven mainly by the 2016 inclusions. The expenditure results are less
dependent on a specific year of inclusions.
7
The RMSPE is the computed with the same distance described before, but X corresponds to the value of the
outcome variable at t = τ

10
 Table 2: Impact of inclusion into HBP Callaway-Sant’Anna DiD

Unique users Frequency Expenditure per user

time ATT Std. Error ATT Std. Error ATT Std. Error
-6 -1.0921 0.3533 * -1.7996 0.8176 0.0575 0.143
-5 -1.2358 0.2232 * -2.3642 0.4674 * -0.1852 0.3861
-4 -0.6325 0.1639 * -1.5108 0.3801 * -0.0772 0.116
-3 -1.3741 0.2411 * -2.4896 0.4534 * -0.2993 0.6713
-2 -0.9994 0.252 * -1.5583 0.5592 * -0.2833 0.4225
-1 0.0954 0.8753 0.8002 2.2112 0.3586 0.2807
0 23.4295 5.4195 * 55.2106 13.7498 * 2.077 0.3517 *
1 31.1943 7.201 * 73.6472 20.1473 * 1.6025 0.3106 *
2 31.1246 8.1417 * 69.1677 21.8165 * 2.4591 0.4281 *
3 37.4068 9.8883 * 91.5944 27.5977 * 1.7127 0.9028
4 -11.6927 15.1952 -21.8724 56.45 1.9506 0.8519
5 -14.876 18.567 -22.024 47.7737 3.0526 1.2327 *
Overall 24.9532 6.9535 * 59.6443 17.2733 * 2.0308 0.3446 *
Notes: Coefficients obtained after a Callaway-Sant’Anna DiD. Standard errors are pre-
sented in parentheses. Significant at 95% level: *.

11
 Figure 4: Dynamic effects Callaway-Sant’Anna DiD
Panel A. Unique users (1000s)
Average Effect by Length of Exposure

40
post
0 0
1

−40

−6 −5 −4 −3 −2 −1 0 1 2 3 4 5

Panel B. Frequency (1000s)

Average Effect by Length of Exposure

100
post
0 0
1
−100

−6 −5 −4 −3 −2 −1 0 1 2 3 4 5

Panel C. Expenditure per individual

Average Effect by Length of Exposure
6

4
post
2 0
1
0

−2
−6 −5 −4 −3 −2 −1 0 1 2 3 4 5

4.2 Synthetic control

The CS DiD presents striking differences in some of the outcomes, but there were doubts about
the comparability of some technologies before the inclusion. For instance, the 2016 cohort include

12
technologies for which there is an important jump in frequencies and unique users (see Figure 6 in
the appendix). For this reason, the synthetic control strategy will construct a comparison group
where pre-trends are similar.
Table 3 presents the results per cohort for the proposed outcome variables, which are presented
in levels. Figures 7 and 8 in the appendix show the average time trends of the included technologies
and their synthetic controls. First, it presents those included by the resolution of 2013 and which
entered effectively in 2014 into the HBP. There is evidence of an increase in the total number of
users and their frequency in such a case. For the case of expenditures, there is an observed increase
but only three years after the inclusion. As for the 2017 cohort, similar results are obtained but
smaller in magnitude.
Second, the increased usage also happens for the 2016 cohort, for which there is a substantial
increase in both unique users and frequency, as documented with the CS DiD results. Panel B of
Figure 7 shows that the large increase in usage comes from comparing technologies with very low
levels of usage prior to 2016.
Third, for the 2018 and 2019 cohorts, there is a decrease in the number of unique users and
frequencies, but such changes cannot be rejected to be equal to zero. Instead, there is evidence of
an increase on expenditures per user.
Lastly, the overall results indicate an increase for at least two years in the number of unique
users and frequency (see figure 5). For expenditures per user, the increase only holds for the first
year. This result, which incorporates all technology inclusions, indicates that the increases in those
variables dominate the observed reductions for specific cohorts. It is also in line with the CS DiD
result, but numbers are larger in magnitude for the usage but smaller for the expenditure per user.8
8
Only computed for the first two years after the inclusion as such estimates incorporate most of the inclusions
irrespective of the cohort.

13
Table 3: Impact of inclusion into HBP Synthetic Control: Group-Time Average Treatment Effects

Unique users Frequency Expenditure per user

Group Time ATT p-value ATT p-value ATT p-value
2014 2014 42.695 < 0.001 * 94.573 < 0.001 * 0.578 0.598
2014 2015 63.085 < 0.001 * 161.323 < 0.001 * 0.767 0.542
2014 2016 39.295 0.041 * 91.807 0.06 1.323 0.451
2014 2017 37.879 0.074 98.496 0.069 1.56 0.402
2014 2018 27.892 0.138 48.011 0.208 3.566 < 0.001 *
2014 2019 29.615 0.143 55.26 0.206 3.642 < 0.001 *
2016 2016 100.473 < 0.001 * 239.115 < 0.001 * 0.551 0.62
2016 2017 109.199 < 0.001 * 254.059 < 0.001 * -1.578 0.202
2016 2018 109.347 < 0.001 * 234.332 < 0.001 * 0.379 0.837
2016 2019 109.648 < 0.001 * 258.864 < 0.001 * 1.375 0.054
2017 2017 3.523 < 0.001 * 3.345 0.015 * -0.66 0.329
2017 2018 2.459 0.139 1.495 0.32 -0.539 0.343
2017 2019 4.009 0.047 * 2.787 0.193 0.042 0.723
2018 2018 -2.922 0.217 -8.328 0.195 4.329 < 0.001 *
2018 2019 -0.798 0.614 -1.026 0.814 1.776 < 0.001 *
2019 2019 -2.826 1 -16.461 1 0.929 < 0.001 *
Overall 0 71.157 < 0.001 * 160.935 < 0.001 * 1.232 0.001 *
Overall 1 82.572 < 0.001 * 189.197 < 0.001 * 0.238 0.885
Notes: Coefficients obtained after a synthetic control implemented with synth runner
package in Stata 16. P-values are derived from permutation tests after 1’000,000 placebo
averages. Significant at 95% level: *.

14
 Figure 5: Outcomes over time: inclusions and their synthetic control

100
200
80

Unique users
150

Frequency
60
100
40

20 50

0 0
-1 0 1 2 -1 0 1 2
Lead Lead

3
Expenditure per user

2.5

1.5

-1 0 1 2
Lead

Inclusions
Not in the HBP

5 Discussion and Conclusions

The main findings of the study show that the progressive update of the Health Benefit Plan in
Colombia during the period of study increased the frequency of use of technology and the number
of people who demand technologies included in this update; this aspect has a positive impact on the
access and effective coverage. This result reaffirms the premise that various authors have established
around the expected role of benefit plans as a mechanism that allows improving coverage through
explicit prioritisation (Giedion, Ú. et al., 2014; Glassman et al., 2017; Mejia-Mejia & Moreno-
Viscaya, 2014; Riascos & Camelo, 2014). This fact is related to the behaviour of patients and
prescribers and the representation of the HBP as a realisation of the right to health since by having
greater awareness of the technologies that both patients and prescribers can access, they are prone
to their use (Dı́az-Uribe, 2017; Glassman et al., 2017).
Faced with the exercise of the right to health based on the existence of an HBP that highlights
its content, some authors have considered that updating this HBP favours the empowerment of the
population to the extent that legal mechanisms exist to guarantee access, quality, opportunity and
financial protection and also represents the tangible part of the technical and political decisions
adopted by a health system (Sabignoso, 2018). In this regard, it is important to mention that, in
the case of Colombia, historically there has been a trend towards health judicialisation linked to the
requirement to comply with the provision of services and technologies included in the HBP, through
a mechanism based on the Constitution called Tutela (which allows to claim before the judges, at
any time and place, the immediate judicial protection of the citizen’s fundamental rights). This
is reflected in the reports on tutelas in the health sector prepared by the Ombudsman’s Office, in
which it is reported that in 2018, the percentage of tutelas in demand for health services contained
in the HPS increased to 81.33% of total tutelas, this being the highest figure since 2003 and the
highest in the contributory regime to date; similar figures are found for 2019 (Defensorı́a del Pueblo,
2018). This phenomenon is quite striking and is differential from other Latin American contexts, in

15
which judicialisation is to a greater extent to guarantee access to technologies not included in the
HBP, while in Colombia 80% is to guarantee technologies already included.
Previous studies in Colombia, such as the one carried out by Núñez et al. regarding the updating
of the plans, evidenced a significant reduction in the gap in access for people from the RS to
previously uncovered technologies (Núñez et al., 2015). Such a reduction could also have an impact
on equity, being this equity criterion one of the most valued and requested aspects on a global
scale as a goal of health systems (Braveman & Gruskin, 2003; Chang, 2002; Dawes, 2016; Etienne,
2013; Sen, 2002; Ye & Rodriguez, 2021). However, although the impact evaluation recognises and
shows a significant increase in the timely access to relevant health technologies in each process of
updating the HBP-UPC, there is still a long way to go in terms of the effective achievement of
equity, even more so considering that the evidence shows that Colombia is one of the countries with
the highest inequality indicators in Latin America (Medina, Hernandez-Ortiz, & Martinez-Perez,
2021), so that there is still a lot of work to overcome the social and equity gaps that have an impact
on the population health.
Another of the relevant findings of this study shows that, between 2014 and 2018, the increase in
the frequencies of use of the technologies included in the update did not mean a rise in expenditure
per person, while in the last year of the update analysed (2019), an increase in spending was
found concerning the inclusion of new technologies in the HBP. Thus, during the longest period
of observation, the expenditure per capita in the health system did not have significant changes
in relation to the update, which is in line with the postulates of various authors, who assure that
explicit prioritisation is a mechanism that can improve the efficiency of the sector. This is because
it allows supporting the decisions about what is financed and what is not through pre-established
criteria socially and politically accepted, as well as improving the efficiency in the allocation of
resources to cover those interventions that have the most impact on the collective health of the
population (Sabignoso, 2018, p.5).
For 2019, the WHO reported that Colombia had a per capita health expenditure of USD 459.2,
well below the average for OECD countries, which was USD 3,608 for the same year (Bank, 2021).
Although this value is low compared to spending in developed countries, national authors have
expressed concern about the sustained increase of health care spending in Colombia because this
represents an important fiscal effort for the country that generates a latent risk of sustainability
(Gutierrez & Gómez-Parra, 2018). Hence the importance of monitoring and evaluating the impact
of the HBP progressive update processes on the system’s spending structure, mainly because it
has been identified that 44% of the growth in health spending is explained by the pressure that
technological changes cause on the health system (Gutierrez & Gómez-Parra, 2018).
On the other hand, in methodological terms, this study allows us to consider the relevance
of the adequate planning of the impact evaluation processes of the HBP and its updating given
its technical complexity, due to several aspects that intersect with the availability of specific data
and instruments needed to carry out this task. Ate the same time, the impact estimation can be
not easy to perform because the health outcomes are the product of different interactions between
social determinants of health. For this reason, methodological decisions regarded with what kind
of variables can be used for the estimation are often complex and must be carefully founded.
Finally, it is important to highlight that the architecture of an HBP should consider the possi-
bilities of its impact; that is, its design should be structured with clear and transparent updating
objectives and criteria, in which the political, scientific and social dimensions are articulated as
a premise that bases its proper implementation. Based on this rationale, the HBP should be de-
signed by overcoming the idea that it is only an explicit list of health technologies. In addition,
the HBP updating process must include a comprehensive health technology assessment not only for
the inclusion also for the whole life cycle of technology. This implies developing post-introduction
evaluations in the HBP, strengthening monitoring schemes of the services included to determine the
technology financing continuity or not.

16
References
Abadie, A., Diamond, A., & Hainmueller, J. (2010). Synthetic control methods for comparative
case studies: Estimating the effect of california’s tobacco control program. Journal of the
American statistical Association, 105 (490), 493–505.
Abadie, A., & Gardeazabal, J. (2003). The economic costs of conflict: A case study of the basque
country. American economic review , 93 (1), 113–132.
Aggarwal, A. (2010). Impact evaluation of india’s “yeshasvini” community-based health insurance
programme. Health Economics, 19 , 5–35. doi: 10.1002/hec
Bank, W. (2021). World health organization - global health expenditure database. Retrieved from
https://apps.who.int/nha/database
Braveman, P., & Gruskin, S. (2003, 4). Defining equity in health. Journal of Epidemiology and
Community Health, 57 , 254. doi: 10.1136/jech.57.4.254
Callaway, B., & Sant’Anna, P. (2020). Difference-in-differences with multiple time periods. Journal
of Econometrics.
Cavallo, E., Galiani, S., Noy, I., & Pantano, J. (2013). Catastrophic natural disasters and economic
growth. Review of Economics and Statistics, 95 (5), 1549–1561.
Chang, W. (2002, 7). The meaning and goals of equity in health. Journal of Epidemiology and
Community Health, 56 , 488. doi: 10.1136/jech.56.7.488
Cotlear, D., Nagpal, S., Smith, O., Tandon, A., & Cortez, R. (2015). Going universal: how 24
developing countries are implementing universal health coverage from the bottom up. World
Bank Publications.
Dawes, D. (2016). Why health equity matters in an era of health care transformation. Retrieved
from https://www.aamc.org/news-insights/insights/why-health-equity-matters-era
-health-care-transformation
Defensorı́a del Pueblo. (2018). La tutela y los derechos a la salud y a la seguridad social. Bogotá
D.C.: Defensorı́a del Pueblo. Retrieved from https://www.defensoria.gov.co/public/
pdf/Tutela-los-derechos-de-la-salud-2018.pdf
Dı́az-Uribe, A. (2017). La participación ciudadana en el ajuste de los planes de beneficios en salud.
un análisis de los casos de colombia y chile (Tech. Rep.). Bogotá D.C.: Pontificia Universidad
Javeriana. Retrieved from http://hdl.handle.net/10554/36062
El Omari, S., & Karasneh, M. (2020). Social health insurance in the philippines: do the poor really
benefit? Journal of Economics and Finance, 45 , 171–187. doi: 10.1007/s12197-020-09525-5
Erlangga, D. (2018). The impact evaluation of public health insurance in indonesia on access to
care, financial protection, and health status. University of York.
Escobar, M., Giedion, Ú., Giuffrida, A., & Glassman, A. (2009). Colombia: After a decade of health
system reform. From Few to Many, 1 .
Etienne, C. (2013, 2). Equity in health systems. Revista Panamericana de Salud Pública, 33 , 79-82.
doi: 10.1590/S1020-49892013000200001
Etienne, C., Asamoa-Baah, A., & Evans, D. (2010). Health systems financing: the path to universal
coverage. World Health Organization.
Fink, G., Robyn, P., Sié, A., & Sauerborn, R. (2013). Does health insurance improve health?
evidence from a randomized community-based insurance rollout in rural burkina faso. Journal
of Health Economics, 32 (6), 1043–1056. doi: 10.1016/j.jhealeco.2013.08.003
Gaag, J., Gustafsson-Wright, E., Tanovic, Z., Kramer, B., Janssens, W., Duynhouwer, A., &
Durowade, K. (2013). A short-term impact evaluation of the health insurace fund program in
cetral kwara state. Nigeria: In Aiid.
Galárraga, O., Sosa-Rubı́, S., Salinas-Rodrı́guez, A., & Sesma-Vázquez, S. (2010). Health insur-
ance for the poor: Impact on catastrophic and out-of-pocket health expenditures in mexico.
European Journal of Health Economics, 11 (5), 437–447. doi: 10.1007/s10198-009-0180-3

17
Giedion, Ú., Panopoulou, G., & Gomez-Fraga, S. (2009). Diseño y ajuste de los planes explı́citos
de beneficios: el caso de colombia y méxico. In Financiamiento del desarrollo.
Giedion, Ú., Tristao, I., Escobar, L., Bitrán, R., Cañón, O., Molins, S., . . . Prado, L. B. (2014).
Planes de beneficios en salud de américa latina: Una comparación regional (Ú. Giedion, I. Tris-
tao, & R. Bitrán, Eds.). Washington D.C.: Banco Interamericano de Desarrollo. Retrieved
from https://publications.iadb.org/es/planes-de-beneficios-en-salud-de-america
-latina-una-comparacion-regional
Giedion, Ú., & Villar, M. (2009). Colombia’s universal health insurance system. Health Affairs,
28 (3), 853–863.
Glassman, A., Giedion, Ú., & Smith, P. (2017). What’s in, what’s out: Designing ben-
efits for universal health coverage. Washington D.C.: Center for Global Develop-
ment. Retrieved from https://www.cgdev.org/sites/default/files/whats-in-whats
-out-designing-benefits-final.pdf
Gnawali, D., Pokhrel, S., Sié, A., Sanon, M., De Allegri, M., Souares, A., & Sauerborn, R.
(2009). The effect of community-based health insurance on the utilization of modern
health care services: Evidence from burkina faso. Health Policy, 90 (2–3), 214–222. doi:
10.1016/j.healthpol.2008.09.015
Gotsadze, G., Zoidze, A., Rukhadze, N., Shengelia, N., & Chkhaidze, N. (2015). An impact eval-
uation of medical insurance for poor in georgia: preliminary results and policy implications.
Health Policy and Planning, 30 , 2–13.
Green, C., Hollingsworth, B., & Yang, M. (2021). The impact of social health insurance on rural
populations. The European Journal of Health Economics, 22 (3), 473–483.
Gutierrez, C., & Gómez-Parra, N. (2018). El sistema de salud colombiano en las próximas décadas:
cómo avanzar hacia la sostenibilidad y la calidad en la atención (Tech. Rep.). Retrieved
from https://www.fedesarrollo.org.co/sites/default/files/archivosciadernos/CDF
No 60 Marzo 2018.pdf
Hayati, R., Bastani, P., Kabir, M., Kavosi, Z., & Sobhani, G. (2018). Scoping literature review on
the basic health benefit package and its determinant criteria. Globalization and Health, 14 (1),
26. doi: 10.1186/s12992-018-0345-x
Herrero, M., & Herrera, F. (2018). ODS en colombia: Los retos para 2030. rograma de Las Naciones
Unidas Para El Desarrollo.
Khan, J., Ahmed, S., Sultana, M., Sarker, A., Chakrovorty, S., Rahman, M., & Niessen, L. (2020).
The effect of a community-based health insurance on the out-of-pocket payments for utilizing
medically trained providers in bangladesh. International Health, 12 (4), 287–298. doi: 10.1093/
inthealth/ihz083
Kumara, A., & Samaratunge, R. (2019). Health insurance ownership and its impact on health-
care utilization: Evidence from an emerging market economy with a free healthcare policy.
International Journal of Social Economics, 47 (2), 244–267. doi: 10.1108/IJSE-05-2019-0333
Lambon-Quayefio, M., & Owoo, N. (2017). Determinants and the impact of the national health
insurance on neonatal mortality in ghana. Health Economics Review , 7 (1). doi: 10.1186/
s13561-017-0169-z
Lozano, R., Fullman, N., Mumford, J., Knight, M., Barthelemy, C., Abbafati, C., . . . Abedi, A.
(2020). Measuring universal health coverage based on an index of effective coverage of health
services in 204 countries and territories, 1990–2019: a systematic analysis for the global burden
of disease study 2019. The Lancet, 396 (10258), 1250–1284.
McKee, M., Balabanova, D., Basu, S., Ricciardi, W., & Stuckler, D. (2013). Universal health
coverage: a quest for all countries but under threat in some. Value in Health, 16 (1), S39–
S45.
Medina, I., Hernandez-Ortiz, M., & Martinez-Perez, J. (2021). Economic growth, human de-
velopment and inequality. International Journal of Social Relevance & Concern, 9 . doi:

18
 10.26821/ijsrc.9.2.2021.9208
Mejia-Mejia, A., & Moreno-Viscaya, M. (2014). Evaluación económica para la toma de decisiones en
salud en colombia. Polı́ticas en salud: Monitor estrátegico, 70-75. Retrieved from https://www
.minsalud.gov.co/sites/rid/Lists/BibliotecaDigital/RIDE/IA/SSA/Articulo9.pdf
Miller, G., Pinto, D., & Vera-Hernández, M. (2013). Risk protection, service use, and health out-
comes under colombia’s health insurance program for the poor. American Economic Journal:
Applied Economics, 5 (4), 61–91.
Nguyen, C. (2011). The impact of voluntary health insurance on health care utilization and out-of-
pocket payments: new evidence for vietnam. Health Ec, 8 , 946–966. doi: 10.1002/hec.1768
Núñez, J., Castañeda, C., Wiesner, D., Romero, J., Parra, J., Hurtado, C., . . . Trujillo, A. (2015).
Evaluación de resultados e impacto de la unificación del plan obligatorio de salud – pos, evalu-
ación de resultados de su más reciente actualización y evaluación de los procesos implementa-
dos en el marco de la unificación y actualización del pos. Bogotá D.C.: Fedesarrollo. Retrieved
from http://hdl.handle.net/11445/3480
OECD. (2018). Reviews of health systems: Colombia 2016. Paris: OECD Publishing.
Riascos, A., & Camelo, S. (2014). Una estimación del costo y cambios en el bienestar de los
colombianos con el nuevo plan de beneficios en salud. Revista De Economı́a Del Rosario, 17 ,
299-314. doi: rev.econ.rosario.17.02.2014.04
Sabignoso, M. (2018). Breve 20: Planes de beneficios en salud y financiamiento basado en resul-
tados: Dos aliados para la cobertura universal y efectiva en salud. Washington D.C.: Banco
Interamericano de Desarrollo. doi: 10.18235/0002199
Schreyögg, J., Stargardt, T., Velasco-Garrido, M., & Busse, R. (2005). Defining the “health benefit
basket” in nine european countries. The European Journal of Health Economics, 6 (1), 2–10.
Sen, A. (2002). ¿por qué la equidad en salud? Revista Panamericana de Salud Pública, 11 . doi:
10.1590/s1020-49892002000500005
Thuong, N. (2020). Impact of health insurance on healthcare utilisation patterns in vietnam: a
survey-based analysis with propensity score matching method. BMJ Open, 10 (10), 1–12. doi:
10.1136/bmjopen-2020-040062
Unión Temporal: Econometrı́a S.A. – SEI – SIGIL Consulting Group S.A. (2011). Estudio de
diagnóstico de la situación actual del acceso, uso racional y calidad de medicamentos, insumos
y dispositivos médicos, que incluya la evaluación de la polı́tica farmacéutica nacional definida
en el año 2003. Bogotá D.C.. Retrieved from https://bit.ly/3yde7GN
Ye, W., & Rodriguez, J. (2021). Highly vulnerable communities and the affordable care act:
Health insurance coverage effects, 2010–2018. Social Science & Medicine, 270 , 113670. doi:
10.1016/j.socscimed.2021.113670

19
A Determining Criteria of HBP in a selected group of countries
In theory, almost all of these criteria are used in the definition and updating of HBP in virtually
all countries included in the referencing. However, this may not be the case and, at least in the
literature consulted, it was not possible to establish the distance between official statements of
intent and what actually happens in practice. On the other hand, some criteria often lack clear
definitions or are difficult to instrumentalise and implement, such as sustainability, innovation,
equity, affordability and access insofar as they link debatable concepts such as those associated
with ethical and normative considerations in the value-related category with other concepts on
which there is not complete agreement, such as health or sustainability.

B Construction of the dataset

Variable name Description Source database

Municipality code. Allows to identify the
CODMUNI Suficiencia
UPC analysis zones.
Date on which the technology was re-
FECHASERV Suficiencia
quired
ACTIVIDAD Identifies the technology in demand Suficiencia
VALORTOTAL Spending on demanded technology Suficiencia
User identifier. It allows to obtain the
IDEANONIMA unique people who demanded health tech- Suficiencia
nologies.
VALORRECOBRADO Spending on demanded technology Recobros
Month in which the technology was re-
MESSUMINISTRO Recobros
quired
ANOSUMINISTRO Year in which the technology was required Recobros
NIT of the service provider. It is used
NITPROVEEDOR to identify the UPC analysis zones in this Recobros
base.
NOMBREPROVEEDOR Name of service provider Recobros
CODMEDSERPREST Technology code Recobros
NOMMEDSERPREST Technology name Recobros

20
C Additional tables and figures

Table 4: Impact of inclusion into HBP Callaway-Sant’Anna DiD: Group-Time Average Treatment
Effects

Unique users Frequency Expenditure per user

Group Time ATT Std. Error ATT Std. Error ATT Std. Error
2014 2013 1.8371 4.3096 4.877 10.4777 0.0649 0.1543
2014 2014 -1.9357 0.8163 -2.653 1.0278 1.2495 0.4275 *
2014 2015 7.8172 12.7812 29.1418 47.1893 0.8323 0.309 *
2014 2016 -6.8722 8.6598 -4.4448 28.4392 1.6068 0.8332
2014 2017 -6.7966 7.1851 -1.8329 27.943 2.7761 2.591
2014 2018 -11.6927 15.5495 -21.8724 47.2277 1.9506 0.8752
2014 2019 -14.876 18.0563 -22.024 50.5928 3.0526 1.2419

2016 2013 -1.427 0.355 * -2.6555 0.6511 * -0.5747 1.2934

2016 2014 -0.9414 0.4152 -1.2106 0.7495 -0.3838 0.8243
2016 2015 0.3425 0.3454 -0.418 0.8224 -0.2512 0.0845 *
2016 2016 58.6682 12.8852 * 134.8986 35.4922 * 1.0678 0.4438
2016 2017 63.4461 15.6249 * 142.2233 42.5629 * 0.7023 0.4854
2016 2018 62.4534 14.8478 * 132.2031 36.926 * 1.8251 0.707
2016 2019 60.4424 14.7462 * 140.2818 46.0294 * 1.1585 0.4189 *

2017 2013 -1.4316 0.3551 * -2.6616 0.6446 * 0.0972 0.1426

2017 2014 -0.8975 0.4149 -1.0468 0.7286 0.3357 0.0929 *
2017 2015 0.3372 0.3459 -0.433 0.8308 -0.2847 0.0673 *
2017 2016 -4.0582 0.9566 * -6.906 2.1413 * 0.9841 0.6762
2017 2017 -0.7911 0.2116 * -1.8658 0.7845 3.1367 0.9576 *
2017 2018 -0.9344 0.4292 -0.5311 1.1899 5.4799 0.902 *
2017 2019 -4.4608 0.7596 * -7 2.0201 * 5.7874 1 *

2018 2013 -1.3032 0.3613 * -2.606 0.6604 * -0.8154 0.825

2018 2014 -0.9412 0.4272 -1.2484 0.7469 -0.0584 0.3181
2018 2015 0.4126 0.3407 -0.4172 0.8614 -0.1985 0.1537
2018 2016 -4.1264 0.9693 * -7.1407 2.1609 * 0.3706 0.3533
2018 2017 0.1359 0.4722 0.7349 1.3502 2.511 2.5313
2018 2018 -0.4608 0.6714 1.0938 1.7141 5.8428 2.191
2018 2019 -3.1043 0.8608 * -2.1576 2.3957 1.7643 1.2101

2019 2013 -1.0921 0.3889 * -1.7996 0.8483 0.0575 0.1463

2019 2014 -1.179 0.4296 * -2.1606 0.8459 0.3454 0.1338
2019 2015 0.4014 0.3479 -0.6173 0.8967 -0.2618 0.2081
2019 2016 -3.1824 1.1605 * -5.1361 2.2324 -0.176 0.1151
2019 2017 0.1659 0.5468 1.0136 1.4848 -0.5319 0.1606 *
2019 2018 0.6317 0.8701 5.6084 1.7315 * 0.2208 0.4846
2019 2019 1.1744 1.6863 7.9773 4.4166 2.1861 0.6408 *
Notes: Coefficients obtained after a Callaway-Sant’Anna DiD. Standard errors are presented
in parentheses. Significant at 95% level: *.

21
 Figure 6: Unique users and inclusion cohorts over time
70 150

60

100
50
Unique users

Unique users
40

50

30

20
0
2012 2014 2016 2018 2020 2012 2014 2016 2018 2020
ano ano

Inclusion in 2014 Not in the HBP Inclusion in 2016 Not in the HBP

30 30

20 20
Unique users

Unique users

10 10

0 0
2012 2014 2016 2018 2020 2012 2014 2016 2018 2020
ano ano

Inclusion in 2017 Not in the HBP Inclusion in 2018 Not in the HBP

22
 Figure 7: Synthetic control: group-time trends (I)
Panel A. Inclusions in 2014
80 200

Unique users 60 150

Frequency
40 100

20 50

0 0
-2 0 2 4 6 -2 0 2 4 6
Lead Lead
Expenditure per user

0
-2 0 2 4 6
Lead

Inclusions 2014
Not in the HBP

Panel B. Inclusions in 2016

150 300
Unique users

Frequency

100 200

50 100

0 0
-4 -2 0 2 4 -4 -2 0 2 4
Lead Lead

5
Expenditure per user

0
-4 -2 0 2 4
Lead

Inclusions 2016
Not in the HBP

23
 Figure 8: Synthetic control: group-time trends (II)
Panel A. Inclusions in 2017

6 8

Unique users 6

Frequency
4
4

2
2

0 0
-4 -2 0 2 4 -4 -2 0 2 4
Lead Lead

2
Expenditure per user

1.5

.5

0
-4 -2 0 2 4
Lead

Inclusions 2018
Not in the HBP

Panel B. Inclusions in 2018

6 15
5
Unique users

Frequency

10
4

3
5
2

1 0
-6 -4 -2 0 2 -6 -4 -2 0 2
Lead Lead

5
Expenditure per user

0
-6 -4 -2 0 2
Lead

Inclusions 2018
Not in the HBP

24