Professional Documents
Culture Documents
Cuadernos de Administración 0120-3592: Issn
Cuadernos de Administración 0120-3592: Issn
ISSN: 0120-3592
revistascientificasjaveriana@gmail.com
Pontificia Universidad Javeriana
Colombia
How to cite
Complete issue
Scientific Information System
More information about this article Network of Scientific Journals from Latin America, the Caribbean, Spain and Portugal
Journal's homepage in redalyc.org Non-profit academic project, developed under the open access initiative
Performance-based Potable
Water and Sewer Service
Regulation.
The Regulatory Model*
Rui Cunha Marques**
Fabio Hernando Garzón Contreras***
* The authors are indebted to two anonymous referees for their suggestions and comments. This paper is the result of a
research carried out by Technical University of Lisbon and Pontificia Universidad Javeriana, Cali, Colombia, during
the year 2006. The article was received on 19-01-2007 and was approved on 23-10-2007.
** Assistant Professor. Research Member of the Center of Urban and Regional Systems (CESUR), DECivil, Instituto
Superior Técnico, Technical University of Lisbon, Portugal. E-mail: rcmar@civil.ist.utl.pt
*** Magíster en Geotecnia e Hidraúlica, Instituto Moscovita de Ingeniería Hidráulica y Adecuación de Tierras, Rusia, Moscú,
1990; Ingeniero hidrotécnico, Instituto Moscovita de Ingeniería Hidráulica y Adecuación de Tierras, Rusia, Moscú, 1989.
Assistant Professor. Director of the Research Group in Advanced Management of Urban Water, Pontificia Universidad
Javeriana, Cali, Colombia. E-mail: fgarzon@puj.edu.co
Abstract Resumen
In recent years the Colombian potable water En los últimos años, el sector de agua potable
and sewer service sector has experienced y saneamiento básico colombiano ha experi-
strong changes promoted by the regulating mentado fuertes cambios promovidos por el
body (Comisión de Regulación de Agua Po- ente regulador (Comisión de Regulación de
table y Saneamiento Básico [Potable Water Agua Potable y Saneamiento Básico). En
and Sewer System Regulation Commis- este sentido, el ente regulador ha puesto en
sion]). In that sense, the regulating body has marcha una nueva estructura tarifaria que
initiated a new rate structure that currently rige el sector en la actualidad y que se basa
governs the sector, based on the comparative en la eficiencia comparativa del desempeño
efficiency of the performance of the sector de las empresas que participan en él. Den-
companies. Data envelop analysis (DEA) is tro de este modelo, el análisis envolvente de
one of the fundamental components in the datos (DEA, por sus siglas en inglés) es una
model. The authors highlight that, although de las piezas fundamentales. Se destaca que,
this methodology has been extensively used aunque esta metodología se ha empleado
for energy regulation, its use is still very extensivamente en la regulación energética,
limited in the waterworks sector. This article, en el sector del agua es aún muy limitado su
a product of joint research conducted by Uni- uso. Este artículo, producto de la investig-
versidad Técnica de Lisboa and Pontificia ación conjunta de la Universidad Técnica de
Universidad Javeriana de Cali, presents and Lisboa y la Pontificia Universidad Javeriana
discusses the comparative efficiency model de Cali, presenta y discute el modelo de efi-
adopted in Colombia and concludes that, ciencia comparativa adoptado en Colombia y
in spite of some problems whose solutions concluye que, a pesar de algunos problemas
are suggested herein, this is an important cuya solución aquí se plantea, este es un paso
step towards quality and excellence for the importante hacia la calidad y excelencia del
Colombian water sector. sector del agua colombiano.
are a few companies with regional coverage, disinfect water (Fernández, 2004). This, in
in other words, they serve more than one turn, is attributable to the weak technical ca-
municipality. Nearly all of them are public pacity of the smaller utilities, as well as cash
entities and between them they encompass flow problems that restrict the availability
about 70 municipalities. At April 2004, a of resources to purchase basic production
little more than 90 municipal administrations inputs. Regarding continuity of service, two
had placed their supply in the hands of the thirds of households report access to a 24-
private sector by means of concessions or hour water service.
similar types of contract (Ministerio de De-
sarrollo Económico de Colombia, 2002). Colombia’s major infrastructure sector re-
forms of the 1990s are rooted in the 1991
In the decade between 1990 and 2000 urban Constitution, and developed in the 1994
water coverage rose from 86,5% to 94,4%, Public Utilities Law (Law 142/94). Whereas
and rural coverage from 23,7% to 38% the new Constitution clearly places ultimate
without taking into account unconventional responsibility for service provision with the
supply systems, or 68% including them. Government, Law 142/94 marks an impor-
These figures show the good performance tant shift by recognizing the importance
in the sector. If we calculate the Millen- of private participation in the provision of
nium Development Goals, taking 1990 as services, establishing regulatory frameworks
the reference defined by the United Nations, and introducing bold liberalization mea-
we find that in December 2002 these targets sures across the utilities. The promotion of
had been surpassed by Colombia’s water competition (wherever possible) and sound
services in both urban and rural areas, the economical regulation (wherever necessary)
latter including non-conventional systems are seen as the two main instruments for
(Fernández, 2004). assuring the improvement and expansion
of infrastructure services. Furthermore, the
Despite the advances made in coverage and law promotes the adoption of cost recovery
the considerable financial resources chan- tariffs for the utilities and establishes limits
neled to the sector by the country, one of the on the extent of cross-subsidization between
major deficiencies is found in the quality of customers.
water supplied, with related effects of mor-
tality, especially among children. A recent In this way, during the 1990s, sustained ef-
survey found that 99% of the population in forts were made to raise water tariffs closer
the 23 largest cities were receiving water of to cost recovery levels, raising the average
potable quality. However, in the rest of the domestic tariff from US$0,33 per cubic
country, almost 60% of the samples fail to meter in 1990 to US$0,78 per cubic meter
pass the minimum quality standards. One in 2001 (Fernández, 2004). Given that 88%
of the main reasons for this is that almost a of households in Colombia are metered, this
third of the smaller municipalities, although large tariff increase led to a strong demand
equipped with water treatment plants, do not reaction, reducing average household con-
make the necessary purchases of chlorine to sumption from 34 to 19 cubic meters per
month over the same period, which implies regulation and for the use of benchmarking
a price elasticity of demand around –0,3 in this domain. The regulatory process in
(Barbero, 2004). Moreover, water tariffs which the performance of a regulated utility
also fall within the national cross-subsidy is based on the performance of other utilities
framework. Due to significant adjustments in within the same sector is known as YC. Its
the cross-subsidy framework over time, price main advantages are the fact that it offers
trends for specific customer groups may vary strong incentives towards efficiency and in-
significantly from the overall average. An novation, either in its operation and capital
important consequence of rising prices has costs (OPEX and CAPEX, respectively) or
been the postponement of major investments in its capital expenses, as well as a fair recov-
in the expansion of water supply capacity. ery of costs and remuneration of the capital
invested. This process also fosters the shar-
The reforms which took place in the water ing and transparency of information, which
sector in the 1990s created the Regulatory is generally asymmetrical and imperfect.
Commission for Water and Basic Sanitation Moreover, it also has several disadvantages
Services (Comisión de Regulación de Agua such as the difficulties intrinsic to its work-
Potable y Saneamiento Básico, hereafter ing principles (minimum number of utilities,
CRA) whose functions include the sector comparability requirement and verifiability),
economic regulation, particularly the defini- to the adopted methodology (information
tion of methodologies and tariff formulas. quality and quantity, heterogeneity, discre-
tionary…), to the hypothesis of collusive
2. Yardstick Competition using behavior among the regulated utilities and
DEA Benchmarking Technique to the kind of commitment the regulator is
able to assume when confronted with the
WSS have several features that require regulated utilities (Marques, 2006).
regulation. Among them there are the econo-
mies of scale and the economies of scope, In the scope of the water sector there are usu-
the provision of essential services to the ally two YC approaches (Marques, 2006).
society well being and a quasi-public good, The first relies on the benchmarking used to
the existence of asymmetric information, the set the operators’ prices and tariffs (hereafter
need of very high and long-lived assets, and called price YC). The kinds of benchmarking
occurrence of externalities (Marques, 2005). used are diverse, depending on the actors and
These market failures sometimes lead to on the features of the countries involved. The
abuses by the utilities, which then provide second approach of YC application concerns
an inefficient service, characterized by high sunshine regulation, which comprises the
prices and poor quality. Thus, the presence comparison and public discussion of the
of regulation is fundamental for the protec- operators’ performance. Sunshine regula-
tion of the various stakeholders (customers, tion is very popular in the water sector, not
Government and regulated companies). only because it is easily applicable but also
The magnitude of market failures and the because it is better accepted by the water
large number of players call for incentive utilities. Several countries, such as Portugal,
Australia and Argentina have applied this YC Denmark have used this methodology in
regulatory method with good outcomes. the past, even though other countries (e. g.
Norway) are thinking about using it, mainly
Colombia implemented an ambitious regu- due to the good results achieved in the
latory process based on price YC that uses energy sector. In the countries and sectors
DEA benchmarking technique as its main where DEA has been applied the reduction
regulatory tool. The Colombian regulator of OPEX has been noticeable, although its
(CRA) uses DEA in the establishment of implementation has always been controver-
targets of the tariff system. DEA studies sial (see, for example, about the regulation
have been gaining interest since the 1980s in of electricity utilities in Nordic countries,
very different production areas with diverse Agrell, Bogetoft, and Tind, 2005).
aims. The situation is similar in the utilities
sector. In the water sector there is also a DEA is a technique based on mathematical
rising interest in this issue. Over the last programming to evaluate the productive ef-
two decades, DEA has already been used ficiency of homogeneous units (here WSS).
in the WSS in many countries (USA, UK, DEA builds the non-parametric frontier
Australia, Japan, Denmark, Italy, Brazil and formed by the union of a group of linear
others), by different actors (water utilities, segments (piece-wise surface), which in-
clude the WSS observed. The relative mea-
regulators, financial agents and academics)
surement of efficiency is achieved through
with such different objectives as economic
the comparison of the efficiency of the WSS
regulation, investment appraisal, efficiency
analyzed with that of the other WSS which
earnings or research (Marques and Monteiro,
remain on that frontier (best practices). From
2004). However, the number of studies is
these WSS, those that use similar inputs and
still small when compared, for example,
outputs combinations are taken as bench-
with the electricity sector and it is expected
marks and, simultaneously, are the targets
that its application will grow. In Europe a of the WSS under analysis (peer group and
considerable number of countries use DEA best practices).
in the economic regulation of electricity in
different stages (Jamasb and Pollitt, 2001). The standard DEA model, called CCR
Nevertheless, it is also true that in Europe (Charnes, Cooper and Rhodes), was derived
the water sector is not very regulated (by a from Farrell’s work (Farrell, 1957) and was
sector-specific regulator) and, for this reason, developed in 1978 assuming constant returns
the use of DEA with regulatory ends, espe- to scale (Charnes, Cooper, and Rhodes,
cially as applied in Colombia, is still scarce. 1978). In 1984, this model was extended to
Only the Office of Water Services (Ofwat) account for variable returns to scale (Banker,
in England and Wales, the Australian Capital Charnes, and Cooper, 1984), originating
Territory (ACT) Government in Australia the model known as BCC (Banker, Charnes
and the Danish Competition Authority in and Cooper). The BCC model deems vari-
able returns to scale by adding a convexity
On April 1, 2006, the Office of Water Services (Of-
wat) was replaced by the Water Services Regulation constraint ensuring that an inefficient WSS
Authority (Ofwat). is only compared against WSS of similar
Min: hk -ε ∑ si + ∑ s j
i =1
cal and non-discretionary variables, conges-
j=1
Figure 1
Technical efficiency and slacks
WSS to each inefficient WSS with a similar financial health and in the customers budget,
combination of inputs and outputs; (b) the given that the shifting of an input, sometimes
ability to deal easily with multiple inputs only in units used (e. g. capital in quantities
and outputs; (c) the adoption of best prac- or in monetary units) has important con-
tices as elements of comparison, rather than sequences in efficiency values. The same
average values; (d) the non-assumption of a happens with the sampling change (number
functional form for the frontier or for the of WSS) or with a choice of production
term of inefficiency; (e) the decomposition
technology (e. g. constant or variable returns
of efficiency into several components, and (f)
its conservative nature. However, the DEA to scale), which can be very troublesome to
technique has various problems, such as: (a) compute. Some of these problems have been
the sensitivity to the outliers, in particular the minimized with the help of the bootstrap
WSS onto the frontier; (b) the difficulty of methodology (Simar and Wilson, 1998 and
measuring the associated error and testing 2000). With regard to the identification of
the results and the specified models statisti- outliers and to the inclusion of the operation-
cally, and (c) the complexity of integrating al environment in the analysis, there was also
the environmental variables.
relevant progress recently (see, respectively,
From the operational point of view, the lack Simar, 2003; Simar and Wilson, 2007) de-
of statistical results makes its practical use rived from the re-sampling (bootstrapping)
difficult. For instance, it is very tough for a procedure and from the order-m concept.
regulator to take a decision based on DEA
scores with direct effects on a company’s
3. The Regulatory Model adopted The average administrative costs for the WS
in Colombia and for the SS are the following:
Articles translated by the authors.
Where PDEA is the % of the operator’s rela- are: service continuity of 80%, a level of
tive efficiency obtained by the application water measurement of 70%, an efficient
of DEA. collection ratio of 60% and a relationship
of WS customers to SS users of 50%. In
The value of 0,046 claims to include the order to take the outliers into account the
costs and the explanatory variables which following relationship was also observed for
are not accounted for in the DEA model. The a confidence interval of 95%:
value of 1,03 corresponds to the maximum AC>ACaverage ±3×σAC () (7)
value applied to the cost compared consider-
ing an efficiency of 100% and a maximum where σAC is the standard deviation of aver-
return of 3% over the capital expenses. The age administrative costs.
cost obtained will be the ceiling (cap) for
the prices, while a minimum of 50% of that With regard to the portion of variable cost
value will also be considered (floor). that is determined by the sum of the three
parts mentioned above, it is important to
The DEA model adopted is computed for stress the way in which OPEX are calcu-
two different groups of water companies, lated: they are obtained separately for the
one for small companies with 2.500 to WS and the SS by means of the formulas:
25.000 costumers and another for the large e
companies with more than 25.000 costum-
OPEX ws = OPEX ws i
+ OPEX ws (8)
ers. The intention is to include the effect of
(9)
economies of scale. The model adopted is OPEX ss = OPEX ssi + OPEX sse
of the CCR type (which achieves constant
returns to scale) and it is oriented towards Where:
the minimization of inputs. It uses several OPEX ws corresponds to the average WS
non-controllable variables and adopts a pro- operation cost.
cedure to simplify calculations (inversion of i
variables). The inputs are the administrative OPEX ws regards the average WS operation
costs and the outputs are the number of WS cost specific to each company.
customers (non-controllable), the number e
OPEX ws is the average WS efficient opera-
of SS users (non-controllable), the number of tion cost.
customers who have water meters installed
(controllable), the number of customers OPEX ss corresponds to the average SS
in levels 1 and 2 of socio-economic clas- operation cost.
sification attended to (non-controllable), OPEX ssi concerns the average SS operation
the number of billing complaints decided cost specific to each company.
in favor of the consumer (controllable), the
density of customers per km of mains (non- OPEX sse refers to the average SS efficient
controllable) and the number of attended operation cost.
commercial and industrial customers (non-
controllable). Sample quality is controlled The first term of the right-hand side of
by minimum parameters for the WSS which formulas 8 and 9 comprises the OPEX
part corresponding to energy and reagents (benchmarking) comes from the equations
costs that depend on each WSS and cannot given below for the WS and the SS:
be closely controlled by them. The second
term refers to the remaining OPEX, more TOPEX DEA × S ws
controllable by each company, and therefore
e
OPEX ws = (11)
PVws × ( 1 − L )
determined on the basis of the performance
of other WSS. The first term regarding the OPEX sse =
TOPEX DEA × ( 1 − S ws )
specific operation cost of WS is determined RS ss × ( 1 − L ) (12)
by the following formula (for the SS there
are other equations which are not given for TOPEXDEA being the WSS total operation
reasons of space): cost obtained by the DEA model, the RSss
being the revenue water volume associated
(EC ws + RC ws ) with the SS and the remaining parameters
OPEXiws = +
PVws × (1 − L) being as already mentioned. The efficient
T operation cost (TOPEXDEA) will be obtained
RWws
RWws +
× 0,57 × (NRW − L) (10) from:
1 − NRW
TOPEXDEA=OPEX×E (13)
opted is again of the CCR kind, and it draws technique mentioned above, discussed in
on the idea of input minimization, although this article.
it employs a trick by using non-controllable
variables. The inputs of the model adopted For example, the specified models use
are the OPEX and the outputs (all non- constant returns to scale technology (CCR
controllable). The elements are the volume model). For the monopolistic and lumpy
of water produced, the revenue sewerage sectors the adoption of the CCR model is
volume, the volume of pumped water, the not the best one, at least in the short-term.
number of treatment plants actually used The WSS usually only have flexibility to
by the system (computed by a formula adjust their size in the long term. Under
which takes into account scale effects) the these circumstances, the BCC model would
equivalent length of mains (which includes be preferred. WSS can be highly penal-
the complexity and size of the network) ized when using this model, and could be
and raw water quality. The procedures used wrongly seen to be inefficient. Besides, if
are similar to those described in the ad- the inefficiency decreases substantially the
ministrative costs model regarding sample CCR model will raise the number of outli-
standardization. ers according to the criteria adopted and,
consequently, it will reduce the size of the
The data used in the CRA-DEA model are sample. The trick related to the inversion of
based on standardized accounting, on man- the model is correct from a theoretical point
agement information reported by the WSS of view, but increases the complexity and the
and on the elements required by the CRA. meaning of the model’s results.
For this first regulatory period the data sup-
plied considered the years 2002 and 2003. From a different perspective, the number of
variables is very large. So, rather than con-
4. Discussion of the Regulatory sidering the non-controllable variables in the
Model Adopted model, it would be better to employ a second
stage and apply the Tobit regression, which
The Colombian regulatory process has is very easy to implement. Another and,
been conducted transparently by the CRA. more robust alternative (the Tobit regression
The tariff system for the WSS defined in produces biased results), consists of apply-
Resolution 287 includes the fixed costs ing a second stage considering the bootstrap
corresponding to the average administra- technique, even though this is more complex
tive costs and the variable costs obtained (see Simar and Wilson, 2007). It also appears
by the sum of the OPEX plus the invest- that there is some kind of undesirable overlap
ment costs and the environmental charges between some of the variables.
by unit of consumption. The regulatory
model uses the DEA method to determine Concerning the OPEX model, the variable
the efficient administrative costs and the related to energy (as well as reagent costs)
efficient OPEX. These costs have aroused should be modeled separately and not in-
some criticism in the Colombian regulatory cluded in the DEA model. As the energy
process related to the drawbacks of the DEA efficiency questions are very relevant, rep-
resenting a significant part of the total cost, around 50% of system input volume. The
the energy variable is not easily controllable output to be considered should be the rev-
and therefore should be taken out of the enue water volume rather than the produced
model. Even then, the energy cost should be water volume. Water losses can also be taken
regulated by incentives, for instance by con- as an explanatory factor. Other hypotheses
trolling the performance indicator of stan- for the water losses inclusion can be to
dardized energy consumption measured by consider this factor as an undesirable vari-
the ratio between annual energy consump- able (see, e.g. Seiford and Zhu, 2002). At
tion in pumping and the sum of the volume all events, the authors think that the basic
per pump head in hundreds of meters. The model should consider as output the revenue
reference value of this performance indica- water volume and the water losses should be
tor, which represents the average amount of analyzed separately, regardless of the use
energy consumed per m3 at a pump head of of DEA techniques. Its relation to the tariff
100 m, is between 0,35 and 0,45 kWh/m3 system should only aim to provide rewards
at 100 meters. or penalties to the best or worst performances
(carrot and stick). In that case, for example,
The variable length of mains is also relevant, water losses can have a direct relationship
and if the aim is just to analyze the OPEX with the rate of return of the operators. The
(as in Colombia) it should be considered as reduction in water losses is linked to the
output. Nevertheless, it is important to stress investments made, which influence OPEX
that the fact of having a very large network as well as CAPEX. The mixture and transfer-
does not necessarily mean a high OPEX. It ence between CAPEX and OPEX is another
is even more important to know whether the pertinent issue. OPEX cannot be analyzed
system has urban or rural characteristics, independently from CAPEX. In the latter,
since the former usually has higher OPEX. the investment levels should be carefully
For that, one might compute the density of supervised to avoid transference of costs
customers per km of mains, or the density between CAPEX and OPEX (and in this case
of service connections. There are also other between CAPEX and administrative costs)
important explanatory factors that should and also “gold plate” practices.
be included in the DEA analysis in some
way, such as number of customers, revenue The presence of outliers is a very important
from industrial and commercial water vol- issue as well. Although this matter is carefully
umes, network aging, number of failures, taken into account by the CRA, the authors
water losses, customer density, and peak would try to split the sample according to
factors. Extreme situations (e. g. floods and its size into several clusters. The DEA tech-
droughts) or the particular conditions of each nique would then be applied, followed by an
WSS (e. g. availability or quality of water analysis of the efficient peers and a sensitivity
resources) are not considered in the regula- analysis. If doubts arise or there were ques-
tory process either. tions from the operators about the quality of
the model, the statistical inference (bootstrap
Water losses are a very significant factor to methodology) would be applied. It should be
be taken into account. The average loss is noticed that in addition to the recent method
referring to identify outliers (Simar, 2003) to the theme, there was a brief description
there are other techniques available. Some of the water sector in Colombia. Then, it
of them very prompt, such as the peer count discussed the YC regulatory model and the
index, the superefficiency, the peer index DEA benchmarking technique. Next it de-
and the Wilson method (see, e. g., Torgersen, scribed the major features of the Colombian
Førsund, and Kittelsen, 1996). regulatory model and finally, it provided
some comments and recommendations about
Finally, note that, at the beginning of the the model applied.
regulatory process, operators do not have the
same development level, and that the splitting Colombia has been making a considerable
of the population classes (2.500 and 25.000 effort to meet targets of high performance in
and >25.000 customers) should be analyzed the water sector. As a rule, it has been able to
in terms of the outcomes obtained. reach its objectives surpassing its neighbor
countries and those with similar character-
Concerning the administrative costs, and istics with regard the water sector patterns
under the principle that two separate models accomplished, both in the WSS and the SS.
were used, a model with one input and two
outputs (one corresponding to the fixed por- The Colombian regulator adopted an am-
tion of administrative costs and another to the bitious regulatory model based on an YC
variable part of OPEX) would be applied, that
approach. The YC model consists of the
is, the administrative costs and the number of
regulation of an operator, based on the
water and wastewater customers. The other
performance evaluation of the other opera-
variables would be considered as non-explan-
tors from the same sector. The advantages
atory in a second stage. For the OPEX base
of using YC are undisputable as proved
model, the OPEX would be used as input and
by empirical cases in progress worldwide.
the water and wastewater revenue volumes,
the number of customers and the mains length The regulatory process has also applied the
would be the outputs. The various explanatory principles of transparency and neutrality and
factors mentioned would also be employed, there are benefits from a sound institutional
including the water catchment quality (or the and regulatory framework, appropriate to
degree of treatment). the realities of the sector and Colombia in
general.
It should be noted that in DEA application,
as in performance measurement in general, The Colombian YC model uses DEA bench-
it is better to perform complex analyses marking as its main regulatory tool. The
of simple models than simple analyses of benefits of employing this technique are also
complex models. unquestionable. The role of benchmarking
in the water sector is fundamental due to its
Conclusions particular features and so the DEA technique
is one of the most effective tools for regula-
This article analyzed the regulation of the tory benchmarking although it also presents
WSS in Colombia. After an introduction some problems.
Charnes, A., Cooper W. and Rhodes E. (1978). Mea- Seiford, L. and Zhu, J. (2002). Modeling undesirable
suring the efficiency of decision making units. factors in efficiency evaluation. European Jour-
European Journal of Operational Research, 2 nal of Operational Research, 142 (1), 16-20.
(6), 429-444.
Shleifer, A. (1985). A theory of yardstick com-
Cooper, W., Seiford, L. and Tone, K. (2000). Data petition. Rand Journal of Economics, 6 (3),
envelopment analysis: A comprehensive text 319-327.
with models, applications, references and DEA-
solver software. Boston: Kluwer Academic Simar, L. (2003). Detecting outliers in frontier mod-
Publishers. els: A simple approach. Journal of Productivity
Analysis, 20 (3), 391-424.
Farrell, M. (1957). The measurement of produc-
tive efficiency. Journal of the Royal Statistical — and Wilson, P. (1998). Sensitivity analysis of
Society, 120A (3), 353-290. efficiency scores: how to bootstrap in non-