MEASURING OVERALL HEALTH SYSTEM PERFORMANCE FOR 191 COUNTRIES

Ajay Tandon, Christopher JL Murray Jeremy A Lauer David B Evans GPE Discussion Paper Series: No. 30 EIP/GPE/EQC World Health Organization

1. Introduction Performance of health systems has been a major concern of policy makers for many years. Many countries have recently introduced reforms in the health sector with the explicit aim of improving performance (1,2). There exists an extensive literature on health sector reform, and recent debates have emerged on how best to measure performance so that the impact of reforms can be assessed (3). Measurement of performance requires an explicit framework defining the goals of a health system against which outcomes can be judged and performance quantified (4). In a previous paper, Evans et al. (5) describe how the performance of countries in terms of meeting one important goal – that of maximising population health – can be measured. In this companion paper, we assess the performance of countries in terms of achieving a broader set of health system outcomes. In addition to considering health, we include attainment in terms of 4 other indicators linked to the intrinsic goals of a health system. The analytical framework used for characterising the goals of a health system is derived from Murray and Frenk (6). They differentiate intrinsic goals of the health system from instrumental goals. In their framework, an intrinsic goal is one: (a) whose attainment can be raised while holding other intrinsic goals constant (i.e., there is at least partial independence among the different intrinsic goals), and (b) raising the attainment of which is in itself desirable, irrespective of any other considerations. Instrumental goals, on the other hand, are goals that are pursued to attain the instrinsic goals. Murray and Frenk identify three intrinsic goals of a health system (Figure 1).

Health system goals
Level Health Responsiveness Fairness in financing Quality
✔ ✔

Distribution
✔ ✔ ✔

Efficiency Efficiency

Equity

Figure 1: Health System Goals

The first is improvement in the health of the population (both in terms of levels attained and distribution). The second is enhanced responsiveness of the health system to the legitimate expectations of the population. Responsiveness in this context explicitly refers

2

. represents the minimum level of attainment that would exist even in the absence of any health system inputs (e.g. This indicates what a system is achieving relative to its potential at given input levels. 1 Health-improving responsiveness dimensions of the system would be included in the attainment of the goal of improving population health. both the level of responsiveness and its distribution are important. The concept is illustrated in Figure 2. 3 . among other factors. the entire population will not be dead in the absence of a functioning health system). in which the health system as a whole is viewed as a macro-level production unit. and reflects respect of persons and client orientation in the delivery of health services. relative to the maximum it could be expected to achieve given its level of resources and non-health system determinants.2 Methodologically. See de Silva et al. labelled “minimum”.to the non-health improving dimensions of the interactions of the populace with the health system.1 As with health outcomes. (5). or the maximum possible level of attainment that can be achieved for given levels of inputs. overall performance measures how well a country achieves all five goals of the health system simultaneously. 100 Maximum possible 80 Overall goal attainment c 60 b 40 Minimum 20 a 0 2 4 6 Inputs to overall goal 8 Figure 2: Health System Performance (Overall Efficiency) The upper line represents the frontier. Murray and Frenk define overall system performance as b/(b+c). (18) for additional details. The third intrinsic goal is fairness in financing and financial risk protection. The framework of frontier production functions (a concept typically used in the measurement of the technical efficiency of firms and farms) is applied. The vertical axis measures overall goal attainment and inputs are measured on the horizontal axis. (19) for details. 2 See Murray et al. Assume a country is observed to achieve (a+b) units of the overall goal attainment. overall health system performance in relation to this broader set of goals is assessed in a similar fashion as described in Evans et al. Adjustment is also made for the fact that overall goal attainment may not be zero in the absence of a modern health system. The aim is to ensure that poor households should not pay a higher share of their discretionary expenditure on health than richer households. Specifically. and all households should be protected against catastrophic financial losses related to ill health. The lower line.

the first step was to combine the individual attainments on all five goals of the health system into a single number. The composite index is a weighted average of the five component goals specified above. health. 4 . education.. (7). Estimation Methods a) Composite Index In order to assess overall efficiency.4 The idea of using a weighted average as an index of several goals is not new. responsiveness-level.1] interval. country attainment on all five indicators (i. 2. an index based on the average of three indicators: longevity. which we call the composite index. 3 Technical efficiency is typically defined as (a+b)/(a+b+c) in Figure 2. Figure 3 reports the rank correlation between attainment on each of the individual components and attainment on the overall composite index for the 191 countries which are members of the World Health Organization (WHO) in 1997. Factors such as health and educational levels of the populace are not viewed as instrumental goals aimed at achieving higher productivity and thereby higher income levels. but are viewed as intrinsic goals of development. and health. and 25% for fairness in financing.5% for the distribution of responsiveness. 12. The HDI is commonly used to assess the state of development of a country. responsiveness-distribution. A recent example is the Human Development Index (HDI). educational attainment (including literacy and enrolment). These weights are based on a survey carried out by WHO to elicit stated preferences of individuals in their relative valuations of the goals of the health system. 12. and fair-financing) were rescaled restricting them to the [0. (20) for details of the survey. 25% for health inequality. 5 For an extension of the HDI that incorporates inequalities in income.5% for the level of responsiveness. 4 See Gakidou et al. The primary difference between performance and technical efficiency is that the former accounts for the non-zero outcome even in the absence of inputs.3 Accordingly. Then the following weights were used to construct the overall composite measure: 25% for health (DALE). First. we use the term “overall efficiency” to refer to overall health system performance in the remainder of this paper. health inequality.5 A similar idea underlies the construction of the composite index as a measure of the overall attainment of the intrinsic goals of the health system. and income per capita.The idea is very similar to that of technical efficiency in the frontier production function literature. see Hicks (21).e.

e. For the purposes of this analysis. See Murray et al.. without considering uncertainty in the valuations of the different components. countries which rank high on levels of health also seem to do well on level of responsiveness. (8) for additional details regarding the weighting scheme and a sensitivity analysis of the impact of changes in these weights on the overall attainment of the health system as measured by the composite index. we chose to use a fixed-effects panel data model in the 5 . as are rankings on health level with health distribution. one way to identify it is to estimate it from the data. Rankings on responsiveness level are also highly correlated with responsiveness distribution. There is a large literature on this topic.e. is the estimation of the maximum attainable composite index (the frontier) given resource inputs and other non health-system determinants of goal attainment. Individual Goal Attainment versus Composite Attainment As can be seen from the bottom row of Figure 3. b) Methodology The econometric methodology for measuring efficiency on the composite index (i. For reasons elaborated in Evans et al. (5) for more details]. overall efficiency) is identical to that for measuring efficiency on health [See Evans et al. Rankings on fair financing do not seem to be correlated with ranks on any of the other components. The problem. especially in the areas of agricultural and industrial economics. the ranks on DALE and health inequality are the most highly correlated with the overall composite. e. This implies that countries which have major inequalities in health or responsiveness are equally as likely to score well on fair financing as countries which have less inequality in these variables.Responsiveness Level 191 Responsiveness Distribution 1 Fair Finance 191 Health Inequality 1 DALE 191 COMPOSITE 1 1 191 1 191 1 191 Figure 3: Rank Correlation. Since this frontier is not directly observable. with countries which are ranked high on these two components also ranking high on the composite index. the weights used in the construction of the composite index have been used consistently... This is likely a product of the relatively high rank correlation between some of the goals.g. from an econometric standpoint. (5). And countries which achieve relatively high levels of health are no less likely to have unfair financial systems as countries that achieve relatively low health outcomes. i.

and the ui’s represent country-specific inefficiencies. Xit ) . (vi) E (Yit | ui = 0. Mathematically. In order to ensure that all the estimated ui’s are positive. the country with the maximum αi is assumed to be the reference and is deemed fully efficient. Technical efficiency is defined as: TEi = E (Yit | ui. Xit ) − Mit .ui) is now country-specific. E (Yit | ui = 0. The econometric methodology of the fixed-effects model is elaborated below. vit is the error term representing random noise with mean zero. ˆ ˆ α = max(αi ) . Ei = E (Yit | ui. The term ui A 0 measures country-specific technical inefficiency. Estimation of the minimum level of goal attainment in the absence of a health system is described later.estimation of the frontier. (ii) where the new intercept αi = (α . and estimates can be found by using a standard fixed-effects model. (i) The dependent variable Yit is the composite index of country i in time t. α represents the frontier intercept. 6 . Consider a functional relationship where the simultaneous attainment of the goals of the health systems are a function of resource inputs and other non-health system determinants. COMPOSITEi max − COMPOSITEi min (vii) Where the COMPOSITE index in the above equation refers to the expected value for country i estimated from the model. and ˆ ˆ ˆ ui = α − αi . Xit ) (v) Overall efficiency (Ei) was based on this definition of technical efficiency with the difference that the minimum output (Mit) that would be achieved in the absence of a health system was subtracted. (iii) (iv) This normalisation ensures non-negative ui’s. Xit ) − Mit In less technical terms: Ei = COMPOSITEi − COMPOSITEi min . It is constrained to be always non-negative. In equation form. The above model can be rewritten as: Yit = αi + X ′itβ + vit . and X´ is a vector of independent variables. this can be written as: Yit = α + X ′itβ + vit − ui .

data on three general types of variable are necessary. X2). Accordingly. Second.10). Details of its construction have been described earlier. For the two-input case (X1. it is necessary to include the effect of controllable non-health-system determinants of health. and third. It is important to note that. Inputs considered included total health expenditure per capita (public and private) in 1997 international dollars (using purchasing power parities. As a proxy for non-health systems inputs. or PPPs. There. to convert from local currency units). d) Data To measure efficiency using the production function approach. 7 . One of the most versatile is the translog (or the transcendental logarithmic) model. While every country had an observation for 1997. the translog model can be written as follows (all variables in logs): Yit = αi + β 1 X 1it + β 2 X 2it + β 3( X 1it ) 2 + β 4( X 2it ) 2 + β 5( X 1it )( X 2it ) + vit . it is necessary to identify an appropriate outcome indicator that represents the output of the health system. overall efficiency combines both technical and allocative efficiency. with some missing data for some countries and years.. In effect. the remaining 141 countries were complete in all panel years). First. The composite index was considered to be the output of the health system.e. the translog function is a second-order Taylor-series approximation to an unknown functional form (9. Modern production studies generally use a flexible form. about 50 countries had observations only for that year (i. Our panel covers the years from 1993 to 1997 for all 191 member countries of WHO. But overall efficiency in our specification is not just a function of technical efficiency. We estimated the full translog model as well as nested versions of the model including the Cobb-Douglas log-linear formulation. Both the Cobb-Douglas and the Constant Elasticity of Substitution (CES) production functions can be derived as restricted formulations of the translog function (11). we considered educational attainment (as measured by average years of schooling in the population older than 15 years).c) Model Specification Different functional formulations of the fixed-effect model were estimated. efficiency relates only to technical efficiency – whether the observed combination of inputs produces the maximum possible output. it is necessary to measure the health-system inputs that contribute to producing that output. by using health expenditure as the health system input to the production of health outcomes.13). the interpretation of overall efficiency differs significantly to the interpretation of efficiency from many existing production function studies. and the Cobb-Douglas log-linear with each of the square terms and the interaction term separately. It will also vary according to the choices each country makes about the mix of interventions purchased with the available health expenditures. The data sources and methods of calculation of health expenditure are described elsewhere (12.

This linear relation was similarly used to predict. and if there is no responsiveness to distribute. everyone in the population is equally well (or poorly) off with respect to a non-existent system of health financing.6 In the case of the composite index. although health inequalities surely would exist in the absence of a health system. i. zero progress can be claimed. In that paper. with respect to the health system goal of reducing inequalities. Furthermore. responsiveness-level receives a zero score. for example. the choice of the normalisation has no intrinsic importance. minimum DALE was always A 15. a sub-sample of the cross-section of 25 countries for which data was compiled at around the turn of the century was investigated. Since a non-existent health system is clearly completely unresponsive..1] after normalisation. and the minimum frontier production function for health as a function of literacy was obtained [see Evans et al. however. respectively. measured in terms of disability adjusted life expectancy or DALEs. the equation for the bottom frontier is as follows (where 25 is the weight on health level in the overall attainment measure): é (DALE i − DALE min ) ù COMPOSITEi min = 37. In other words.5×1 = 37. since each component of the composite index is normalised on the [0. it is assumed that the score for the other two components (“health inequalities” and “responsiveness-level”) would be zero in the absence of a health system (25×0+12.5×0 = 0). For similar reasons. at current levels of literacy. So long as the observed DALE values in the sample are restricted to [0. (5). some amount of health or other health-system goal attainment is to be expected despite no resource inputs to the health sector. (12) and Evans et al. the health levels that would be achieved in the absence of a health system. (5) for more details].5). a similar argument can be made for responsiveness-distribution.e. the component accounting for health level (DALE) is similarly normalised for calculation for the minimum attainable bound. 25×1+12. that in Evans et al. However. “fair financing” and “responsiveness-distribution”) have little or no meaning in the absence of a health system. 7 In order to obtain an expression for DALEmin.5. two components of the overall attainment measure (i. and so long as the same bounds are used in calculating the composite score and in calculating the minimum. f) Uncertainty 6 Unlike a traditional production setting. 8 . 7 Note.5 + 25 × ê ë DALE max − DALE min (8) The value of DALEmin and DALEmax were set at 20 and 80.1] interval. that would be expected even in the absence of a modern health system. (5).. Thus.e) Minimum Frontier What level of composite index could be expected in the absence of a health system? This is analogous to the question posed in Evans et al. For this reason. these two components of the composite index are given full scores in determination of the minimum (in calculating the weighted average this would entail that attainment of these goals be given a score of 37.e. the goal was to estimate the minimum level of health.

Logged Variables. Panel Estimates (1993– 1997).4.5% of the coefficient estimates from the 1000 regressions described earlier.0057769 . Mean and Uncertainty Interval) for the Frontier Health Production Function.0065223 0.0.0065666 0. where the 80% uncertainty intervals for the rank order were derived from the distribution of the overall efficiency index.0225598 4.1731853 The table also shows the 95% uncertainty intervals around the estimated coefficients.136329 0. Max(u) is the maximum deviation from this average and.8 Rank order was based on the mean value of the overall efficiency index for each country.04963 Mean 0. Coefficient Estimates (Median.0654469 0. The simulated distributions of the individual coefficients are graphed in Figure 5. The 80% uncertainty intervals on the overall efficiency index reflect the estimated distribution of the efficiency index derived from these 1000 different regressions. Table 1. where each of the 1000 estimates reflected a single draw from the distribution of the composite index.110499 0. In brief. None of the uncertainty intervals include zero. 191 Member Countries of WHO. 3. 95% uncertainty intervals were used because they offer greater discriminatory power against the null hypothesis that the coefficients are equal to zero. 8 See Evans et al.11182 0. Coefficient Estimate Health expenditure Average years of schooling Median 0. The constant term represents the average fixed effect in the sample.1736141 Uncertainty Interval (95%) 0.Uncertainty in the COMPOSITE index reflects the underlying uncertainty in the estimation of each of the five goals of the health systems for all 191 countries. gives the intercept for the frontier.0. we were able to construct a distribution consisting of 1000 draws on the composite index for each country.0363105 . when added to the constant. Monte Carlo simulation techniques were used. the overall efficiency measure.0187357 . Results Results from the preferred functional form estimated using the fixed effect model are reported in Table 1. They were derived by omitting the lowest and the highest 2.0223382 Constant Max (u) 4.0.0076745 0.0281929 4. Here. (5) for details. In order to derive the confidence intervals around our statistic of interest. These uncertainty intervals are not the statistical confidence intervals of the individual regressions. 9 .1871777 Square average years of schooling 0.0496496 0.0.1631771 .076119 . the efficiency index was estimated using the fixed-effect model for all countries 1000 different times. Given that we had a 1000 random draws on the values of each of the five goals of the health system for each country.

08 Density 100 0 0 . Table 2.. As can be seen from the test statistic.15 0 . Where this correlation is present.035 30 80 60 20 Density Density 40 10 20 0 4 4.2 Figure 5: Distribution of the Coefficient Estimates for Log of Health Expenditure. Constant. 191 WHO Member Countries. a significant difference in the coefficient estimates is indicative of correlation between the individual effects and the regressors.0051401 -0.008 Log of health expenditure .16 .03 Square log of average years of schooling .0119787 0.009 .006 .02 . and Maximum Value of the Country-Specific Fixed Effect We also tested statistically whether we should use a fixed effects or random effects model. Log of Average Years of Schooling.015 .06 Log of average years of schooling . Assuming that the model is correctly specified. performance) for each country are reported in Annex Table 1.0054362 -0.0494743 0.05 Constant 4. Panel Estimates (1993-1997).1000 60 200 150 40 Density 500 Density 20 50 0 .0546144 0.1 4. Square of Log of Average Years of Schooling.02 .000 The resulting estimates of overall efficiency (i.005 . Coefficients Composite index Fixed-Effects Random-Effects Difference Health expenditure Average years of schooling Square of average years of schooling m2(3) p value 0.15).04 .0065425 0.007 . the null of no correlation is rejected and a fixed-effects model is clearly preferable. the estimates using a random-effects model will be biased (14.0227706 0.0152011 59.e. Table 2 reports the coefficient estimates where the expected value of the composite index is used as the dependent variable.0379717 -0. The Hausman test is a test of equality between the coefficients estimated via the fixed-effects and random-effects models. All Variables in Logs.025 . along with the uncertainty interval around the efficiency 10 . Hausman Specification Test: Coefficient Estimates using Expected Value of the Composite Index as Dependent Variable.02 0.18 Maximum fixed effect .

log of average years of schooling and the square of average years of schooling as regressors – is both parsimonious and maps most closely to the reference standard (Figure 7). the full version of the translog is. a priori. Furthermore. The efficiency index ranges from a maximum of 0. Spain Iceland Sweden Finland Cuba Thailand Albania Egypt Nicaragua Ukraine Libyan Arab Jamahiriya Panama Iraq Ecuador Samoa Vanuatu Ghana Burundi Kyrgyzstan Madagascar Democratic People's Republic of Korea South Africa Lesotho Sierra Leone 0 20 40 60 80 100 120 140 160 180 200 Figure 6: Uncertainty Intervals. Ranking of Overall Efficiency. 191 countries According to the interpretation of the translog model as a second-order approximation to an unknown functional form.index. when parsimony is added as a criterion for model choice. However. Italy. We compare rank order correlation in Figure 7 since efficiency will invariably increase with the addition of terms on the right hand side of the regression (unless the added term is completely collinear with another. the appropriate criterion for judging predictive stability across models is rank order correlation. This overlap in the rankings is illustrated graphically in Figure 6. However. for instance. it will always explain additional sample variance). that the rank orders of the top three countries (France. the model we report – with log of health expenditure. In any given regression. and France was not the best-performing country in all 1000 runs. It would not be possible to say. there is substantial overlap of the confidence intervals for several countries. the country with the maximum fixed effect will have a score of 1. and San Marino) with respect to the overall efficiency were significantly different from each other. the reported scores are averaged over 1000 runs. the reference standard.994 for France and a minimum of 0 for Sierra Leone. Thus. 11 .

we explored whether the inclusion of possible other non-health system determinants of health would make a difference to the ranking – in addition to our proxy for non-health system determinants. Since other possible direct explanators were difficult to identify and measure for all countries in our sample. high performers perform well in all specifications. and average years of schooling squared was estimated. and the efficiency analysis and ranks were recomputed. This residual can be interpreted as the part of income which might act through mechanisms other than health expenditures and education – or possible other pathways (called POSOTHER). and accordingly the efficiency index is higher with POSOTHER. For this reason. average years of schooling. 12 . and because it is not possible to explain which determinants of efficiency picked up by POSOTHER are controllable inputs or not. the residual of the dependent variable that is left unexplained is smaller. POSOTHER was added to the fixed effects regression as a proxy for these possible other pathways related to income. average years of schooling. The residual from the regression of income on health expenditure per capita. To further test robustness. The rank correlations are extremely high no matter what combination of variables is included. the correlation between the rankings under the two sets of estimates is very high (0. Conversely. However.9974) showing that inclusion of POSOTHER does not have a significant impact on the relative rankings of the countries based on their efficiency in producing health. When any new explanator is added to an equation. we defined a new variable obtained by regressing income per capita on the regressors already in our efficiency equation. suggesting that poor performers perform poorly in all specifications and rank is not an artefact of the choice of model.Health expenditure Years of schooling 191 Health expenditure Years of schooling Square of health expenditure 1 Health expenditure Years of schooling Square of years of schooling 191 Health expenditure Years of schooling Square of health expenditure Square of years of schooling 1 Health expenditure Years of schooling Interaction 191 Health expenditure Years of schooling Square of health expenditu Square of years of schoolin Interaction 1 1 191 1 191 1 191 Figure 7: Rank Correlation Matrix for Different Model Specifications The above results (Figure 7) show clearly that the rank of the different countries is very robust to the functional form of the translog regression. we chose to use the more parsimonious form of the equation reported above.

(5). in addition to the more traditional goal of population health. 13 .5 0 2 4 6 Log of health expenditure per capita 8 Figure 8: Health Efficiency versus Health Expenditure per Capita 9 Efficiency on health is from Evans et al.4. fair financing. By way of comparison. Figures 8 and 9 report the estimates of efficiency on health as well as overall efficiency (with uncertainty intervals) against health expenditure per capita (in log). Unlike previous work in this area. Discussion This paper has introduced a new way of measuring the efficiency of health systems. and health inequality. we have specifically defined the broad set of goals of the health system such as responsiveness (both level and distribution).9 1 Efficiency on health .

5 0 2 4 6 Log of health expenditure per capita 8 Figure 9: Overall Efficiency versus Health Expenditure per Capita Several things are notable: first. fair financing. One interpretation of this could be that there are diminishing returns to increasing the inputs of resources devoted to producing health (say due to biological limits on life expectancy). but there the decline is less obvious. This association of overall efficiency with resource inputs is also evident from the country rankings: industrialised countries are dominant among the better performers. which reflects the fact that there are uncertainty intervals around each of the components of the composite index.1 Overall efficiency . and high AIDS prevalence is likely to have a deleterious effect on overall efficiency. This is also the case for overall efficiency. efficiency on health appears to increase with health expenditure per capita and then perhaps to decline slightly. Most of those countries that are ranked low tend to be those in Sub-Saharan Africa where a combination of factors related to economic problems. civil unrest. Figure 10 plots the geographical distribution of the overall efficiency score. there is greater uncertainty related with the estimates of overall efficiency compared to efficiency on health. Secondly. but that the composite index would not be subject to strong diminishing returns because greater expenditure can be used to further the goals of responsiveness. 14 . and reductions in health inequality.

0.361 .200 . All rights reserved or of its authorities.0.849 .0.994 0.0.760 0.848 0.0. 1997 Estimates Higher overall goal attainment can be achieved by increasing health expenditure.495 . city or area  WHO 2000.674 0. Dotted lines on maps represent approximate border lines for which there may not yet be full agreement.199 Lowest performance No Data The boundaries and names shown and the designations used on this map do not imply the expression of any opinion whatsoever on the part of the World Health Organization concerning the legal status of any country.591 0. 191 WHO Member States. territory.928 . which plots predicted levels of overall goal attainment as a function of health expenditure and educational attainment for our preferred frontier equation.360 0 .Highest performance 0.0.0.927 0.761 . This implies moving along the expansion path in the spirit of the World Health Report 1999 (16). or concerning the delimitation of its frontiers or boundaries. This is illustrated in Figure 11.675 .494 0.0.0. Figure 10: Global Distribution of Overall Efficiency.592 . 15 .

This is unlikely to be the case. or is perfectly efficient. It is also important to note that the analysis does not imply that countries with high efficiency scores cannot improve their performance. something that is discussed further in the World Health Report 2000 (17). This work draws attention to the fact that some countries are doing better than others in terms of achieving their potential. to increase efficiency as well. Future work will aim to identify determinants of this relative performance: whether exogenous factors such as institutional quality and population density have an impact on efficiency. This raises the question of how to increase efficiency. There is an implicit overestimation of efficiency in the model since the estimates assume that the best-performing country in the sample has an efficiency of 1. Panel Estimates (1993–1997) However. the analysis also suggests that overall goal attainment can be increased without increasing health expenditure: there is considerable room in all countries. at all levels of health expenditure. given their inputs.95 composite index 85 75 5 average 100 health expenditure 1000 2 years of schooling 10 Figure 11: Logarithmic-Scale Plot of the Expected Value of the Overall Health System Achievement Frontier Production Function. 16 . but at present we have no way of knowing the extent of the overestimation. 191 Member Countries of WHO.

This is the focus of ongoing work at WHO. and over time within the same country.Since it is not restricted by possible biological limits on healthy life-span. It is an indicator that is feasible to measure regularly enabling comparison between countries. overall efficiency is a more representative measure of the true efficiency of health systems than one based on health status alone. say to conduct a comparison of health systems at the district or state level. 17 . the goal is to stimulate action that will eventually improve the overall performance of health systems in countries and contribute to improving the welfare of people. An important benefit from the debate that is likely to accompany this exercise will be development of improved data sources and estimation methods. Such intra-national and inter-temporal analyses of efficiency would be particularly important for countries introducing health system reforms. In taking this first step towards measuring efficiency. and we hope that this study encourages all countries to routinely measure the inputs and outputs of their health systems. The framework for measuring overall efficiency may also be applied at a sub-national level.

943 0.837 0.808 0.908 0.964 0.809 0. Bolivarian Index 0.917 0.872 0.735 0.894 0.994 0.793 0.965 0. Overall efficiency in all WHO member states Overall efficiency Rank 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 Uncertainty Interval 1 1 1 2 3 2 4 4 7 8 8 10 10 13 12 14 14 16 14 17 18 14 20 16 22 22 23 26 18 27 27 28 22 32 31 33 35 34 36 36 38 37 39 41 41 41 36 43 42 45 38 41 47 50 5 5 6 7 7 11 8 14 12 11 12 15 16 19 20 21 21 21 22 24 24 29 26 30 27 32 33 32 39 32 33 34 43 36 41 40 44 46 44 48 45 48 53 51 52 53 59 54 55 59 67 67 62 64 Member State France Italy San Marino Andorra Malta Singapore Spain Oman Austria Japan Norway Portugal Monaco Greece Iceland Luxembourg Netherlands United Kingdom Ireland Switzerland Belgium Colombia Sweden Cyprus Germany Saudi Arabia United Arab Emirates Israel Morocco Canada Finland Australia Chile Denmark Dominica Costa Rica United States of America Slovenia Cuba Brunei Darussalam New Zealand Bahrain Croatia Qatar Kuwait Barbados Thailand Czech Republic Malaysia Poland Dominican Republic Tunisia Jamaica Venezuela.957 0.781 0.945 0.910 0.972 0.802 0.937 0.830 0.966 0.882 0.947 0.978 0.825 0.993 0.862 0.845 0.945 0.915 0.928 0.866 0.914 0.785 0.812 0.883 0.978 0.868 0.810 0.861 0.926 0.754 0.893 0.852 0.890 0.759 0.914 0.840 0.838 0.947 0.772 0.804 0.921 0.879 0.911 0.859 0.831 0.000 1.965 0.928 0.803 18 .925 0.959 0.827 0.948 0.825 0.921 0.939 0.762 0.955 0.861 0.852 0.819 0.834 0.845 0.942 0.876 0.916 0.909 0.789 0.782 0.902 0.805 0.849 0.817 0.925 0.830 0.930 0.895 0.871 0.824 0.961 0.932 0.991 0.959 0.870 0.982 0.985 0.932 0.881 0.870 0.813 0.834 0.807 0.982 0.834 0.973 0.775 Uncertainty Interval 0.973 0.972 0.931 0.838 0.854 0.929 0.812 0.874 0.924 0.988 0.881 0.886 0.903 0.000 0.894 0.914 0.849 0.985 0.790 0.957 0.881 0.916 0.793 0.938 0.933 0.946 0.948 0.859 0.913 0.918 0.832 0.824 0.779 0.942 0.ANNEX Table 1.816 0.897 0.884 0.782 0.998 0.000 1.958 0.891 0.939 0.829 0.903 0.808 0.815 0.997 0.906 0.848 0.745 1.741 0.816 0.

600 0.733 0.800 0.741 0.781 0.774 0.756 0.695 0.615 0.773 0.660 0.723 0.645 0.583 0.628 0.697 0.683 0.714 0.632 0.700 0.666 0.684 0.768 0.707 0.665 0.774 0.55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 41 51 47 55 50 53 52 54 49 50 55 59 53 59 58 60 58 64 65 63 66 68 68 57 70 68 70 75 75 75 74 76 79 69 83 84 85 85 83 87 87 90 78 94 92 90 92 94 91 86 94 94 95 94 93 91 88 105 95 103 94 - 75 63 77 67 78 73 74 73 81 80 80 74 81 75 81 81 83 84 82 83 81 84 85 99 88 93 92 88 88 91 96 93 96 111 107 106 104 107 110 108 107 106 123 107 110 113 114 113 117 126 118 120 121 120 122 123 125 118 127 121 130 Republic of Albania Seychelles Paraguay Republic of Korea Senegal Philippines Mexico Slovakia Egypt Kazakhstan Uruguay Hungary Trinidad and Tobago Saint Lucia Belize Turkey Nicaragua Belarus Lithuania Saint Vincent and the Grenadines Argentina Sri Lanka Estonia Guatemala Ukraine Solomon Islands Algeria Palau Jordan Mauritius Grenada Antigua and Barbuda Libyan Arab Jamahiriya Bangladesh The former Yugoslav Republic of Macedonia Bosnia and Herzegovina Lebanon Indonesia Iran.806 0.745 0.638 0.695 0.660 0.739 0.639 0.716 0. Islamic Republic of Bahamas Panama Fiji Benin Nauru Romania Saint Kitts and Nevis Republic of Moldova Bulgaria Iraq Armenia Latvia Yugoslavia Cook Islands Syrian Arab Republic Azerbaijan Suriname Ecuador India Cape Verde Georgia El Salvador 0.647 0.617 0.619 0.656 0.699 0.734 0.797 0.708 0.707 0.722 0.608 0.664 0.752 0.747 0.623 0.750 0.765 0.713 0.629 0.732 0.686 0.643 0.743 0.561 0.665 0.620 0.687 0.717 0.626 0.630 0.674 0.653 0.755 0.672 0.764 0.784 0.678 0.720 0.719 0.718 0.655 0.589 0.657 0.630 0.664 0.723 0.734 0.630 0.667 19 .582 0.664 0.647 0.664 0.674 0.589 0.638 0.696 0.690 0.693 0.750 0.694 0.669 0.566 0.625 0.628 0.761 0.742 0.711 0.713 0.659 0.730 0.675 0.684 0.611 0.719 0.714 0.675 0.782 0.682 0.639 0.702 0.747 0.671 0.695 0.798 0.657 0.686 0.736 0.544 - 0.617 0.701 0.698 0.710 0.669 0.720 0.583 0.688 0.721 0.630 0.692 0.802 0.740 0.789 0.688 0.586 0.642 0.599 0.772 0.664 0.740 0.637 0.755 0.789 0.834 0.665 0.759 0.565 0.632 0.652 0.722 0.691 0.624 0.664 0.573 0.770 0.679 0.664 0.705 0.689 0.754 0.618 0.719 0.689 0.691 0.664 0.630 0.661 0.776 0.709 0.740 0.597 0.642 0.698 0.617 0.571 0.627 0.754 0.752 0.722 0.

479 0.504 0.268 0.551 0.429 0.619 0.497 0.599 0.470 0.526 0.399 0.410 0.611 0.522 0.516 0.401 0.340 0.180 158 164 158 163 161 164 161 164 164 180 179 180 180 182 181 184 182 183 Tonga Uzbekistan Comoros Samoa Yemen Niue Pakistan Micronesia.579 0.471 0.529 0.431 0.596 0.547 0.517 0.573 0.234 0.390 0.368 0. Federated States of Bhutan Brazil Bolivia Vanuatu Guyana Peru Russian Federation Honduras Burkina Faso Sao Tome and Principe Sudan Ghana Tuvalu Côte d'Ivoire Haiti Gabon Kenya Marshall Islands Kiribati Burundi China Mongolia Gambia Maldives Papua New Guinea Uganda Nepal Kyrgyzstan Togo Turkmenistan Tajikistan Zimbabwe United Republic of Tanzania Djibouti Eritrea Madagascar Viet Nam Guinea Mauritania Mali Cameroon Lao People's Democratic Republic Congo Democratic People's Republic of Korea Namibia Botswana Niger Equatorial Guinea Rwanda Afghanistan Cambodia South Africa Guinea-Bissau 0.314 0.463 0.374 0.262 0.329 0.361 0.494 0.251 0.266 0.614 0.549 0.626 0.334 0.319 0.501 0.526 0.572 0.384 0.611 0.357 0.518 0.376 0.575 0.668 0.482 0.544 0.672 0.483 0.449 0.429 0.595 0.337 0.526 0.420 0.543 0.397 0.567 0.443 0.533 0.278 .457 0.400 0.575 0.447 0.302 .400 0.414 0.490 0.393 0.689 0.497 0.544 0.612 0.618 0.0.582 0.416 0.571 0.455 0.356 0.439 0.517 0.610 0.593 0.509 0.592 0.467 0.615 0.277 0.559 0.339 0.355 0.522 0.524 0.116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 106 92 86 114 92 114 109 114 111 111 112 118 119 122 126 115 114 124 119 118 130 124 120 129 130 133 135 125 125 134 135 138 137 136 138 143 142 143 147 143 145 149 152 149 155 155 154 156 150 157 - 121 134 139 126 140 129 132 131 136 136 135 138 140 138 136 145 147 144 151 150 145 149 152 149 148 147 150 157 162 154 154 154 159 158 159 157 158 161 163 167 166 168 170 170 166 170 172 176 181 178 160 .325 0.577 0.504 0.534 0.535 0.527 0.458 0.594 0.495 0.375 0.505 0.328 0.414 0.288 0.554 0.532 0.385 0.477 0.413 0.464 0.572 0.411 0.428 0.607 0.490 0.632 0.384 0.517 0.472 0.516 0.511 0.549 0.268 0.455 0.337 0.583 0.456 0.446 0.553 0.239 0.482 0.512 0.587 0.594 0.352 0.327 0.298 - 0.354 0.485 0.584 0.461 0.338 0.531 0.469 0.520 0.589 0.389 0.322 0.427 0.381 0.410 0.425 0.0.176 157 .404 0.481 0.422 0.284 0.353 0.427 0.463 0.541 0.398 0.543 0.563 0.452 0.366 0.459 0.430 0.246 0.392 0.564 0.375 20 .373 0.

306 0.138 0.191 Swaziland Chad Somalia Ethiopia Angola Zambia Lesotho Mozambique Malawi Liberia Nigeria Democratic Republic of the Congo Central African Republic Myanmar Sierra Leone 0.251 0.190 175 .094 0.339 0.000 .303 0.339 0.275 0.117 0.171 0.343 0.363 0.198 0.177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 166 167 167 173 172 170 174 170 171 180 183 185 - 184 183 186 185 186 186 186 187 188 189 189 189 179 .231 0.311 0.305 0.205 0.286 0.000 .276 0.100 - 0.332 0.0.191 190 .0.215 0.319 0.326 0.176 0.251 0.234 0.369 0.200 0.079 21 .199 0.000 0.156 0.260 0.186 0.0.269 0.000 .266 0.282 0.204 0.174 0.232 0.369 0.

47(1):69-83. Health care reform: informing difficult choices. Jorgenson DW. A WHO framework for health system performance assessment.28. Collins C.J. World Health Organization. 1995. Switzerland. 2000 (Global Programme on Evidence for Health Policy Discussion Paper No. Maynard A. Int J Health Plann.. Frenk.A.J.J.. Bloor K. New York..L. Evans. 1999 (Global Programme on Evidence for Health Policy Discussion Paper No. American economic review 1975.6. Goldstein H. Murray. Spiegelhalter DJ.D. literacy. Switzerland. Smith P.10(4):247-64.D. Green A.65(3):367-83.. The Use of Performance Indicators in the Public Sector. Elements of Econometrics. 11. United Nations Development Programme.J. Hunter D. 2000 (Global Programme on Evidence for Health Policy Discussion Paper No. Macmillan Publishing. Lau LJ. 1986 12.A. 1999 8.7. Christensen LR. educational attainment. Journal of the Royal Statistical Society Series A 1996. Geneva. Oxford University Press.L. New York. et al.) 7.29. et al. Tandon. Murray.C.C. Tandon. absolute poverty. 5.) 22 . Christensen LR. Estimates of income per capita. World Health Organization.L. Evans. Health sector reform and the interpretation of policy context.Reference List 1.E.C. 4. Bendib. Geneva. Human Development Report 1999. Geneva..) 9. World Health Organization. World Health Organization. Geneva. Switzerland. Health Policy 1999. Transcendental logarithmic production frontiers. 2000 (Global Programme on Evidence for Health Policy Discussion Paper No. Lau LJ. 3. Overall health system achievement for 191 countries.L. and Frenk. League Tables and Their Limitations: Statistical Issues in Comparisons of Institutional Performance.J. Jorgenson DW.153(1):53-72. Murray. 2. and income Gini coefficients for The World Health Report 2000.Manage. Transcendental logarithmic utility functions.) 6. 10. Journal of the Royal Statistical Society Series A 1990. Kmenta.. Review of economics and statistics 1973.55(1):2845.E. Tandon.59(3):385-443. Switzerland. The comparative efficiency of national health systems in producing health: an analysis of 191 countries. et al.J.A.

. A Guide to Econometrics.E.. Prentice Hall. Econometric Analysis. De Silva. 1997 15.L. Measuring preferences on health system performance assessment. WHO. Hicks DA. 4th edition.C. World Health Organization. World Health Organization. 2000 (Global Programme on Evidence for Health Policy Discussion Paper No.) 20.21. Switzerland. Murray. Cambridge. Upper Saddle River.H. 2000 (Global Programme on Evidence for Health Policy Discussion Paper No. World Health Organization. Geneva.F..P. Greene.) 14. WHO. World Health Organization.) 19. The World Health Report 1999: Making a Difference. 23 .A. World Development 1997. Measuring responsiveness: results of a key informants survey in 35 countries. Switzerland.) 21.P.P.E. Frenk. The Inequality-Adjusted Human Development Index: A constructive proposal. and Murray. 1998 16. Knaul.27.L.J.24.N..J.20. Defining and measuring fairness of financial contribution. 3rd edition. Switzerland. Geneva. World Health Organization. Switzerland. NJ. Switzerland. Gakidou. 2000 18.25(8):1283-98. Geneva. Geneva. The MIT Press. and Hernández. Switzerland. 2000 (Global Programme on Evidence for Health Policy Discussion Paper No. World Health Organization.J. Musgrove. Kennedy. 1999 17.P. World Health Report 2000. Geneva.J. Geneva. Pouillier.13. National Health accounts for 191 countries in 1997. 2000 (Global Programme on Evidence for Health Policy Discussion Paper No. and Valentine. et al.W.C.