Proceedings of the 37th International Conference on Computers and Industrial Engineering,

October 20-23, 2007, Alexandria, Egypt, edited by M. H. Elwany, A. B. Eltawil

DATA ENVELOPMENT ANALYSIS OF NATIONAL HEALTHCARE SYSTEMS

AND THEIR RELATIVE EFFICIENCIES
James Benneyan
Mechanical and Industrial Engineering
Northeastern University
United States
benneyan@coe.neu.edu

Mehmet Erkan Ceyhan

Aysun Sunnetci
Mechanical and Industrial Engineering
Northeastern University
United States
{ceyhan.m, sunnetic.a}@neu.edu

ABSTRACT health (Roemer, 1991), and the evaluation of the effi-

We summarize results of several data envelopment
analysis (DEA) studies to identify countries with the
most efficient healthcare systems in terms of translat-
ing resources consumed into outputs produced.
These analyses identify 27-65 countries on empirical
performance frontiers based on six key dimensions of
healthcare systems – clinical outcomes, health ad-
justed life years, access, equity, safety, and resources
– using modified DEA methods for rationally con-
straining weights given to each measure and han-
dling proportional data such as outcome rates. The
DEA models also produce targets for each measure
that would move an inefficient healthcare system to
the current best performance frontier. Results are
reported for 180 countries using data obtained pri-
marily from the World Health Organization (WHO),
both altogether and separately within each of four
WHO economic development categories. Overall,
very few countries were found to be efficient, sug-
gesting significant opportunities for improvements
through combining best practices.
KEYWORDS
Performance frontiers, DEA, econometrics, world
health organization
1. INTRODUCTION
Although several efforts recently have attempted to
evaluate and rank national healthcare systems, this
remains a complex endeavor due to multiple desir-
able and undesirable system characteristics, available
resources, and socioeconomic differences. Health-
care systems are integrated combinations of several
activities intended to promote, restore, and maintain
ciency and weighting by which multiple types of re-
sources are consumed to produce multiple types of
outputs to accomplish these objectives frequently is
largely subjective.
The World Health Organization (2000) summarized
the healthcare performance of 193 countries world-
wide using several dozen measures that describe the
overall level of health, distribution of health in the
population, responsiveness of each healthcare sys-
tem, resources expended, and distribution of services.
The WHO also used these results to rank each
healthcare system, although the study received a fair
amount of criticism due to the data collection and
estimation methods (Alan, 2001), methodology
(Jamison & Sandbu, 2001), choice of healthcare
components (Starfield, 2000) and fairness of USA’s
ranking (University of Maine, 2001).
Other studies include several publications by the
Commonwealth Fund (Davis et al, 2006; Davis et al
2004) that compared the performance of 6 countries
(Australia, Canada, Germany, New Zealand, the
United Kingdom, and the United States) using a sub-
jective scorecard ranking methodology based on 37
indicators in 5 key areas: health outcomes, quality,
access, efficiency, and equity. Kumar et al (2004)
also compared 18 healthcare systems using a Pareto
optimization approach to identify any countries that
are equal or better in all criteria, comparing Austra-
lia, Austria, Belgium, Canada, Finland, France, Ger-
many, Iceland, Ireland, Italy, Japan, Luxemburg,
Netherlands, Spain, Sweden, Switeria, UK, and USA
and based on data from Organisation for Economic
Cooperation and Development and focused on five
categories: healthcare expenditures, physician ser-

251
 Proceedings of the 37th International Conference on Computers and Industrial Engineering,
October 20-23, 2007, Alexandria, Egypt, edited by M. H. Elwany, A. B. Eltawil
vices, pharmaceutical services, life expectancy, and
infant mortality.
In contrast, we use data envelopment analysis (DEA)
to evaluate and rank the relative performance of all
countries, and to identify combinations of achievable
targets that would cause poorly performing countries
to become world-class. DEA has been successfully
used to study other healthcare issues, such as hospital
performance (Zhu, 2003), public policy efficiency
(Coppola et al, 2003), and cardiac surgeon perform-
ance (Chilingerian, 1995), and offers several advan-
tages for the current analysis.
2. METHODOLOGY
2.1. Envelopment analysis
Originally introduced by Charnes et al (1978), DEA
is a production frontier estimation method that solves
a series of transposed fractional programs to deter-
mine the relative efficiency of multiple systems
(here, countries) at consuming multiple inputs (e.g.,
doctors, resources, staff, etc) in order to produce
multiple outputs (e.g., clinical outcomes, access, sat-
isfaction, etc). The relative efficiency of a given
country e (“decision making unit”, or DMU, in DEA
terminology) is found by solving the formulation
v1 I e ,1 + v 2 I e , 2 + ... + v N I e , N
Minimize z e =
u1Oe ,1 + u 2 Oe , 2 + ... + u M Oe , M
v1 I 1,1 + v 2 I 1, 2 + ... + v M I 1, M
s.t. 0 ≤
≤1
u 1 O1,1 + u 2 O1, 2 + ... + u N O1, N
M
v1 I k ,1 + v 2 I k , 2 + ... + v M I k , M
0≤ ≤1
u1 O k ,1 + u 2 O k , 2 + ... + u N O k , N
u j > 0, j = 1,..., M
v i > 0, i = 1,..., N ,
where k is the number of countries, M is the number
of outputs, N is the number of inputs, and vi and uj
are the weights placed on input i and output j, respec-
tively. In practice, this model is converted to its lin-
ear programming equivalent (Charnes and Cooper,
1962), and 2nd and 3rd stage dual programs are solved
for each DMU separately to find their efficiency
score, target values, optimal weights, and reference
sets (where some weighted combination of these
DMU’s would move the country on the frontier).
In addition to the above constant returns to scale
(CRS) formulation, we solved variations that assume
variable returns-to-scale (VRS) relationships between
the inputs and outputs (Banker, 1984)seek to mini-
mize the weighted ratio by producing more outputs
with the given level of inputs (output-oriented), and
consume fewer inputs to produce the current level of
outputs (input oriented).
2.2. Data elements and sources
Table 1 summarizes the data elements used in this
analysis and their designation as inputs or outputs. In
some cases, surrogate measures were identified for a
general dimension (e.g, immunization rates as a pan-
system marker for prevention), with a total of 5 in-
puts and 6 outputs. All data were gathered from the
WHO website (http://www.who.int/en/), with the
exception of the adverse event (‘misadventure’) rates
(a safety measure) collected from WrongDiagno-
sis.com.
Data element or surrogate measure
Healthy life expectancy (O)
Outcomes
Care and
Adult mortality rate (O)
Infant mortality (O)
Morbidity surrogate measure (TB rate) (O)
Resources
Per capita total expenditure (I)
Cost &
(1)
Doctors and nurses per 1000 capita (I)
Hospital beds per 1000 (I)
Equity
Weighted combination of urban-to-rural under 5
(2)
yr mortality rate, upper-to-lower wealth quartile,
and none-to-high education mother ratios (O)
Preven-
tion
Surrogate measure (immunization rate) (I)
(3)
Safety
Incidence rate of medical misadventure (O)
(4)
graphics
(5)
Demo-
Median age (I)
Table 1. Data elements or surrogate measures used
Table 2 illustrates a subset of these data (only for 10
countries) in the interest of space; the full report is
available at www.coe.neu.edu/research/qpl.
Although a small amount of data were able to be re-
placed via multiple regression, thirteen of the 193
countries still needed to be eliminated because most
of their data were missing (Cayman Islands, Cook
Islands, Lao People’s Democratic Public, Monaco,
252
 Proceedings of the 37th International Conference on Computers and Industrial Engineering,
October 20-23, 2007, Alexandria, Egypt, edited by M. H. Elwany, A. B. Eltawil

Inputs Outputs
Per capita Trained Immu- Healthy life Adult Infant TB Adverse
Hospital Median
Country health ex- medical nization expectancy mortality mortal- preva- event Equity
beds age
penditure people rate at birth rate ity rate lence rate
Canada 36 $2,669 12.09 0.07 38.9 72.0546 0.007 0.995 1 n/a n/a
China 23.11 $61 2.11 0.1767 32.7 64.1163 0.0118 0.974 0.998 n/a n/a
India 6.9 $27 1.4 0.4 24.9 53.4510 0.014 0.938 0.997 999398 0.65
Jamaica 18 $164 2.5 0.22 23 65.0673 0.0116 0.983 0.999 n/a n/a
Japan 129.34 $2,662 9.77 0.01 42.9 75.0632 0.0055 0.997 0.999 n/a n/a
Pakistan 6.8 $13 1.2 0.3433 19.8 53.3 0.0137 0.92 0.997 667219 0.37185
Russian
99 $167 12.31 0.03 38.4 58.8524 0.0202 0.987 0.998 n/a n/a
Federation
Turkey 26 $257 3.04 0.19 28.1 62.0305 0.0144 0.972 0.999 288746 0.41694
USA 33 $5,711 11.93 0.0633 36.5 69.2953 0.0083 0.994 1 n/a n/a
Venezuela 9 $146 2.73 0.1733 26 64.1974 0.0093 0.984 0.999 n/a n/a

Table 2. Illustration of data inputs for subset of countries (n/a = not available)

Montenegro, Nauru, Niue, Saint Vincent & Grenadi-
nes. Saint Marino, Sao Tome and Principe, Solomon
Islands, Timor-Leste, and Tuvalu), leaving 180 coun-
tries with all measures except those for equity and
safety. The equity and safety measures were avail-
able for only 120 and 102 of the remaining 180 coun-
tries, respectively, but with only 39 countries in
common.
As summarized in Section 3, additional analyses
therefore were conducted on (1) this smaller subset of
the 39 counties that included equity and safety meas-
ures and (2) the larger set of 180 countries but now
separated into each of the four WHO economic de-
velopment categories (based in gross national income
per capita) and analyzed individually by category.
Since classic DEA models assume all inputs and out-
puts are unbounded above, the proportional data ele-
ments (mortality, morbidity, and prevention rates)
were handled both via standard models and by fol-
lowing an odds ratio approach (Benneyan and Sun-
netci, 2006), where output j and input i for DMU k
are converted to
Ok , j
OkOR, j = , 0 < O kOR, j < ∞,
1 − Ok , j
and
I k ,i
I kOR,i = , 0 < I kOR,i < ∞,
1 − I k ,i
respectively, and results are back-converted to obtain
targets after the analysis is complete.
Additionally, in some cases solving traditional DEA
models produces “irrational” results for the optimal
weights in the sense that less important measures can
receive significantly more emphasis than more im-
portant outcomes, such as satisfaction dominating
life expectancy. Section 3.4 therefore presents re-
sults where additional constraints are added to the
above formulation to force relative ordering of
weights, e.g., v1 > v2 > v3 and v2 > v4.
3. RESULTS
3.1. Full data set (without equity and
safety measures)
Table 3 summarizes the results of the unrestricted
constant-retruns-to-scale output-oriented model
(CRS-O) for the larger data set (i.e., without safety
and equity and with no weight restrictions), again
just for a sample of countries given space limitations.
The first and second rows for each country contain
the target values and weights, respectively. One
hundred and fifteen of the 180 countries were not on
the efficiency frontier. Note that a country can be
efficient or inefficient regardless of whether it has
limited or abundant inputs. For example, Jamaica
and Japan are both efficient whereas the United
(6)
States and Turkey are inefficient.
Table 4 summarizes the reference sets for these same
countries, with the tabulated percentages normalized
to sum to 100% (essentially representing the percent
(7)
contribution of each DMU in the reference set to a
given country’s hypothetical composite in order to be
efficient). Note that efficient countries do not have
any countries (other than themselves) in their refer-
ence sets. Although not presented here in detail,
Figure 1 summarizes the percent of efficient DMUs

253
 Proceedings of the 37th International Conference on Computers and Industrial Engineering,
October 20-23, 2007, Alexandria, Egypt, edited by M. H. Elwany, A. B. Eltawil

Inputs Outputs
Per cap. Trained Immun Healthy life Adult Infant TB
Median
Country Score Beds spend- medical ization expectancy mortality mortal- preva-
age
ing people rate at birth rate ity rate lence
36 $2,669 11.9 0.07 38.9 77.78 0.0058 0.00412 0.00003
Canada 0.818
0.011 0 0 0.34 1.26 0.12 0.0021 0.0008 0.00001
23.11 $61 2.1 0.1767 32.7 71.17 0.0077 0.01895 0.0006
China 0.624
0.007 0.002 0.1134 0.8149 1.46 0.559 0 0.00001 0
5.982 $27 1.3 0.3976 24.9 60.94 0 0.03357 0.00073
India 0.667
0 0.001 0.0551 0 4.35 0.283 0.0096 0 0
18 $164 2.4998 0.22 23 65.067 0.0116 0.017 0.00009
Jamaica 1
0 0 0.0713 0 2.75 0 0.0076 0.00026 0.00003
129.37 $2,662 9.7704 0.01 42.9 75.063 0.0055 0.003 0.00039
Japan 1
0 0.0002 0.0012 15.068 0.28 0 0.0015 0.00218 0
2.368 $13 0.97 0.272 19.8 56.240 0.0122 0.04858 0.0009
Pakistan 0.902
0 0.003 0 0 4.39 0.238 0.0085 0 0
Russian 50.002 $167 10.3 0.03 38.4 78.046 0.0081 0.00527 0.00015
Federa- 0.422
tion 0 0.001 0 39.535 1.77 0.368 0 0.00624 0
26 $11 3.0424 0.19 28.1 70.667 0.0077 0.01862 0.0003
Turkey 0.645
0 0 0.085 0.3903 2.92 0.607 0 0 0
33 $2,163 11.3 0.0633 36.5 73.178 0.0070 0.00509 0.00003
USA 0.847
0.012 0 0 0.5504 1.32 0.00001 0.0042 0.00071 0.00001
Vene- 9 $146 2.7 0.1733 26 64.788 0.0091 0.0156 0.00051
0.938
zuela 0.034 0.001 0 1.1486 1.18 0.374 0.0018 0.00188 0.00002

Table 3. Results for unrestricted CRS output-oriented model (full data set): efficiency scores, target values, and weights

Country Reference Set

Canada Jordan (30.8%), Sweden (24.8%), Mexico (18.3%), Oman (10.8%), Iceland (7.8%), Guatemala (7.6%)

China Syrian Arab Rep. (14.7%), Bhutan (11.0%), Eritrea (9.4%), Comoros (5.0%), Vietnam (3.6%)

India Comoros (86.4%), Cape Verde (9.8%), Uganda (2.9%), Guatemala (0.9%)

Jamaica Jamaica (100%)

Japan Japan (100%)

Pakistan Comoros (96.7%), Zambia (2.5%), Guatemala (0.7%)

Russian
Syrian Arab Rep. (59.9%), Oman (21.8%), Seychelles (20.5%), Singapore (2.7%)
Federation

Turkey Nicaragua (48%), Belize (43.6%), Jamaica (5.0%), Oman (3.5%)

USA Jordan (65.5%), Sweden (22.7%), Iceland (6.2%), Guatemala (4.5%), Mexico (1.2%)

El Salvador (40.5%), Comoros (33.3%), Morocco (9.1%), Syrian Arab Rep. (8.3%), Singapore (3.5%), Mexico (4.2%),
Venezuela
Jordan (1.1%)

Table 4. Reference sets for unrestricted CRS output-oriented model (full data set), listed in decreasing order

254
 Proceedings of the 37th International Conference on Computers and Industrial Engineering,
October 20-23, 2007, Alexandria, Egypt, edited by M. H. Elwany, A. B. Eltawil

when solving the DEA model under different returns-to- Efficient only in grouped Efficient in both analyses
scale and input-versus-output orientation assumptions, analysis (global efficiency)
both with and without the odds ratio transformation of rate Afghanistan Kenya Antigua & Mexico
data. Andorra Kuwait Le- Barbuda Morocco
Bahrain sotho Bangladesh Mozambique
Percent efficient healthcare systems Liberia Belarus
Brunei Darus- Nepal
100 salam Libyan Arab Belize Nicaragua
Burkina Faso Namibia Benin Malaysia
80 Philippines
Cambodia Bhutan Niger
Percent efficient

60.6 Qatar
55.6 56.7 Canada Burundi Oman
60
45.6 Central Afri- Republic Cape Verde Panama
36.1 36.1 can of Korea
40 Chile Paraguay
Republic Saudi Arabia
Comoros Rwanda
20 Chad Slovakia
Costa Rica Seychelles
Colombia Togo
Cyprus Singapore
0 Cuba Uzbekistan
CRS-I-R VRS-I-R CRS-O-R VRS-O-R VRS-I VRS-O Czech Slovenia
Democratic Venezuela Republic
Type of model Somalia
Republic of Yemen Dominica Spain
the Congo
Figure 1. Percent efficient countries for each DEA model Zimbabwe Ecuador
Djibouti Sri Lanka
(I: input orientation; O: output orientation; R: odds ratio trans- El Salvador Swaziland
Equatorial
formation) Eritrea
Guinea Sweden
Grenada Ethiopia Switzerland
3.2. Relative efficiencies within each Hungary Finland Syrian
WHO economic group Indonesia Gambia Arab Republic
Iran Guatemala Tajikistan
Tables 5 and 6 (on following page) summarize corre- Jamahiriya Honduras Tonga
sponding results for these same countries when the Madagascar Iceland Uganda
analysis was run separately within each of the 4 Malawi Israel United Re-
WHO economic development groups, with the num- Jamaica public
Maldives
of Tanzania
bers in parentheses indicating the group each country Mali Japan
Vietnam
belongs to. Note that although Venezuela and Can- Mauritius Jordan
Zambia
ada were inefficient in the combined analysis they Kyrgyzstan

now are efficient within their groups, whereas Ja-
maica and Japan were efficient in both the combined
and grouped analyses. This makes intuitive sense, as
a DMU that is efficient in a particular data set must
also be efficient in any subsets, whereas the converse
must not be true, resulting in fewer (96) of the 180
countries now not being on efficiency frontiers.
Table 7 summarizes all countries that were efficient
in only the grouped analysis (left-hand columns) and
those that were efficient in both sets of analyses
(right-hand columns), herein termed “globally-
efficient”. Conversely, those countries that were in-
efficient in both the studies might be thought of as
“absolute inefficient”, since partitioning the data set
did not allow them to join the frontier. For example,
the United States healthcare system exhibits absolute
inefficiency, presumably because it consumes higher
levels of resources but does not efficiently translate
these to proportionally higher levels of outputs (even
under variable-returns-to-scale assumptions).
Table 7. Summary of globally and within-group efficient
healthcare systems
3.3. Reduced data set, with equity and
safety measures
As mentioned above, including equity and safety
measures in the analysis - although desirable - sig-
nificantly reduces the number of countries with com-
plete data (from 180 to 39). Tables 8 and 9 summa-
rize DEA results for a sample of 10 countries for
which these measures were available, analyzed over-
all (i.e., not stratified into the WHO economic
groups) since there were not enough remaining coun-
tries in some groups.
As shown, DEA results of the smaller data set are
significantly different than the results of the larger
data set (even if just these 39 are used without the
two new measures). In general, with more measures
but fewer DMUs it is more likely that a particular
country will appear efficient.

255
 Proceedings of the 37th International Conference on Computers and Industrial Engineering,
October 20-23, 2007, Alexandria, Egypt, edited by M. H. Elwany, A. B. Eltawil

Inputs Outputs
Per cap. Trained Immun Healthy life Adult Infant TB
Median
Country Score Beds spend- medical ization expectancy mortality mortal- preva-
age
ing people rate at birth rate ity rate lence

Canada 36 $2669 12.1 0.07 38.9 72.1 0.00701 0.005 0.00002

1
(4) 0.00878 0 0 0.5 0.4 0 0.00387 0 0.00004

China 22.3 $61 2.1 0.18 31.5 68.9 0.00942 0.01408 0.00082
0.81
(2) 0 0 0.3 1.3789 0 0.6 0 0 0

India 2.9 $16 1.3 0.36 24.9 63.5 0.0092 0.03643 0.00066
0.66
(1) 0.01876 0 0 0 3.8 0.7 0 0 0

Jamaica 18 $164 2.5 0.22 23 65.1 0.0116 0.017 0.00009

1
(2) 0.00067 0 0 0.4917 2.9 0.2 0.00355 0.00301 0.00001

Japan 129.37 $2662 9.8 0.01 42.9 75.1 0.00552 0.003 0.00039
1
(4) 0 0 0 9.85 0 0 0.00333 0.00111 0.00001

Pakistan 3.97 $13 1.1 0.26 19.8 56.2 0.01222 0.04902 0.0009
0.88
(1) 0 0 0 0.0355 3.8 0.7 0 2.78407 0
Russian 71.19 $167 12.3 0.03 38.4 74.8 0.0084 0.00631 0.000001
Federa- 0.48
tion (3) 0 0 0.1 13.65 0.8 0.6 0 0.00226 0.00009

Turkey 19.33 $257 3 0.087 28.1 69.3 0.00793 0.01211 0.00032

0.72
(3) 0 0 0.1 0 1.7 0.6 0 0 0.00003

USA 33 $4,028 11.9 0.035 36.50 70.3 0.00768 0.00323 0.00004

0.99
(4) 0.01107 0 0.05 0 0.14 0 0 0 0.00004
Vene- 9 $146 2.7 0.17 26 64.2 0.00934 0.016 0.00052
zuela 1
(3) 0 0 0.1 0.32 0 0 0.00612 0.00404 0.00005

Table 5. Results for unrestricted CRS output-oriented model (grouped data): efficiency scores, target values, and weights

Country Reference Set

Canada
Canada (100%)
(4)

China (2) Morocco (35.5%), Indonesia (28.5%), Sri Lanka (20.7%)

India (1) Comoros (100%)

Jamaica
Jamaica (100%)
(2)

Japan (4) Japan (100%)

Pakistan
Comoros (96.7%), Zambia (2.1%), Tajikistan (1.2%)
(1)
Russian
Federation Seychelles (58%), Malaysia (22.6%), Oman (15.7%), Libyan Arab Jam. (2.4%), Panama (1.3)
(3)

Turkey (3) Belize (49.1%), Malaysia (26%), Costa Rica (17.1%), Venezuela (7.9%)

USA (4) Sweden (78.5%), Iceland (15.2%), Cyprus (6.3%)

Venezuela
Venezuela (100%)
(3)

Table 6. Reference sets for unrestricted CRS output-oriented model (grouped data), listed in decreasing order

256
 Proceedings of the 37th International Conference on Computers and Industrial Engineering,
October 20-23, 2007, Alexandria, Egypt, edited by M. H. Elwany, A. B. Eltawil

Inputs Outputs
Per cap. Trained Immun Healthy life Adult Infant TB Adverse
Median
Country Score Beds spend- medical ization expectancy mortality mortal- preva- event Equity
age
ing people rate at birth rate ity rate lence rate

Bangla- 3.4 $14 0.39 0.19 22.2 54.3 0.01402 0.05 0.004 592382 0.39
1
desh 0.03881 0.00230 0.02 3.36 0.008 0.83457 0.00000 0 0.00001 0 0.009
Central 5.8 $12 0.39 0.34 18.4 44.1 0.01434 0.096 0.004 13973 0.55
African 0.89
Republic 0.00000 0.00444 0.23 0 0.05 0.00000 0.01602 0 0 112.7 0.01
12.7 $16 0.9 .19 19.9 51.0 0.01357 0.04 0.003 82803 0.63
Ghana 0.95
0.00000 0.01386 0 0.46 0.04 0.34881 0.00942 0 0 8.97 0.002
6.9 $27 1.04 0.4 24.9 59.7 0.01081 0.04 0.002 2741 0.55
India 0.78
0.01772 0.00657 0 0.0004 0.4 0.87087 0.00000 0 0 0 0
17.0 $177 5.3 0.04 23 60.9 0.01333 0.023 0.00005 23517 0.8
Jordan 1
0.00000 0.00564 0 0.04 0 0.00000 0.00000 0.00023 0.00005 65 0.01

Mada- 4.2 $8 0.6 0.4 17.5 48.6 0.01525 0.076 0.00351 1866 0.54
1
gascar 0.00000 0.00038 0.0006 2.5 0 0.00000 0.00000 0.002 0 1819.9 0.0007
6.8 $13 1.2 0.34 19.8 53.3 0.01373 0.08 0.003 667219 0.37
Pakistan 1
0.01025 0.00546 0 0.38 0.04 0.29408 0.00917 0 0 0 0.01

South 28.7 $508 4.85 0.11 40.6 73.7 0.00504 0.016 0.005 81757 0.74
0.72
Africa 0.00000 0.00000 0.03 0.007 0.03 0.00000 0.00699 0 0.007 0 0.0008
13.5 $179 3.04 0.19 28.1 68.0 0.00821 0.02 0.008 23881 0.6
Turkey 0.77
0.00000 0.00000 0.0005 0.06 0.05 0.60442 0.00000 0 0 0 0

Zim- 5.0 $40 0.9 0.17 19.9 33.6 0.01257 0.08 0.01 15389 0.78
1
babwe 0.00693 0.00021 0 0.31 0.04 0.00000 0.01145 0.007 0.01 169.01 0.004

Table 8. Results for unrestricted CRS output-oriented model (reduced data set): efficiency scores, target values, and weights

Country Reference Set

Bangla- Bangladesh (100%)

desh
Central
African Ethiopia (48.5%), Chad (30.3%), Uganda (10.4%), Madagascar (10.3%), Botswana (0.5%)
Republic

Ghana Uganda (40.4%), Ethiopia (30.8%), %), Sri Lanka (13%), Pakistan (9.3Vietnam (5.7%), Madagascar (0.8%)

India Madagascar (51%), Nicaragua (22%), Bangladesh (18%), Pakistan (9%)

Jordan Jordan (100%)

Madagas-
Madagascar (100%)
car

Pakistan Pakistan (100%)

South
Mexico (76%), Zambia (19%), Guatemala (4.8%), Botswana (0.2%)
Africa

Turkey Nicaragua (39.8%), Paraguay (32.9%), Mexico (22.4%), Jordan (4.7%)

Zimbabwe Zimbabwe

Table 9. Reference sets for unrestricted CRS output-oriented model (reduced data set), listed in decreasing order

257
 Proceedings of the 37th International Conference on Computers and Industrial Engineering,
October 20-23, 2007, Alexandria, Egypt, edited by M. H. Elwany, A. B. Eltawil

3.4 Rational weight restrictions the weight restrictions force it to be the lowest valued
data element.
Note in Table 8 that some efficient countries assign
zero weights to important measures, essentially ig-
3.4. Commonwealth fund study
noring them in computing an overall score. For ex-
ample, Jordan and Madagascar are efficient by plac- In a related analysis, Table 13 summarizes the results
ing no weight on the number of beds per capita (a of the Commonwealth Fund’s evaluation of 6 coun-
measure representing resources consumed), life ex- tries, with each scored on a 1 (best) to 6 (worst) scale
pectancy, and adult mortality rates. in several categories. (Davis et al, 2007). Each cate-
gory is coded with an “I” or “O” to indicate its use in
Similar “irrational weighting” results also occur for
the DEA model as an input or output, respectively.
Japan, Jamaica, Venezuela, and many other countries
in the above analyses. Table 10 summarizes the Tables 14 and 15 summarize the efficiency scores,
number and percent of times each data element re- targets, and reference sets that result from the con-
ceived zero weight (for the unrestricted DEA results stant returns-to-scale output oriented model, using an
for the data set in Section 3.3), ranging from 10 to 62 odds ratio transformation for the scalar data and the
percent.. following output weight restrictions:
vHealthy lives > vEquity, (12)
Frequency (per-
Data element
cent) of 0 weight vQuality > vEquity, and (13)
Hospital Beds 14 (36%) vEquity > vAccess . (14)
Trained medical people 13 (33%)
As previously, the first and second rows for each
Per capita expenditure on health 13 (33%)
country contain the target values and weights, respec-
Median age 4 (10%) tively. Here, all outputs received weights larger than
Immunization rate among 1-year-olds 8 (21%) zero whereas the efficiency input measure received
Healthy life expectancy at birth 18 (46%) zero weights for all countries. Note that the United
TB prevalence 23 (59%)
States, Canada, and New Zealand (interestingly) have
very low DEA scores, whereas the UK and Australia
Adult mortality rate 17 (44%)
both are on the best practice frontier (score = 1) and
Infant mortality rate 24 (62%) Germany is in the middle (0.43). As shown by the
Incidence rate for medical misadventure 18 (46%) target values, New Zealand and Germany are ineffi-
Equity 6 (15%) cient because their healthcare systems should be able
to produce greater amounts of outputs (quality care,
Table 10. Number (percent) of times each data element is ig- access, equity, and health lives) for the amount of
nored (of 39 total)
resources they currently consume.
As suggested in Section 2, one solution to this prob-
lem is to impose some type of weight restrictions to Austra- Ger-
New
ensure that, for example, mortality receives more Measure Canada Zea- UK US
lia many
land
weight than satisfaction. Tables 11 and 12 (on fol-
Quality
lowing page) therefore summarize the results using 3.5 5 2 3.5 1 6
care (O)
the following weight constraints:
Access (O) 3 5 1 2 4 6
vAdult mortality rate > vTB prevalence , (8)
vInfant mortality rate > vTB prevalence , (9) Equity (O) 2 5 4 3 1 6

vHealthy life expectancy at birth > vTB prevalence , and (10) Healthy
1 3 2 4.5 4.5 6
lives (O)
vTB prevalence > vIncidence rate for medical misadventure . (11)
Efficiency
Note that the efficiency scores and targets change 4 5 3 2 1 6
(I)
significantly, although some (but fewer) measures $ per
$2,876 $3,165 $3,005 $2,083 $2,546 $6,102
still receive zero weights. For instance, median age Capita (I)
now does not receive any zero weights and healthy
Table 13. Commonwealth Fund data (1 = best, 6 = worst, O =
life expectancy at birth received zero weight only output, I = input)
once. Conversely, the medical misadventure rate
received an increased number of zero weights, since

258
 Proceedings of the 37th International Conference on Computers and Industrial Engineering,
October 20-23, 2007, Alexandria, Egypt, edited by M. H. Elwany, A. B. Eltawil

Inputs Outputs
Per cap. Trained Immun Healthy life Adult Infant TB Adverse
Median
Country Score Beds spend- medical ization expectancy mortality mortal- preva- event Equity
age
ing people rate at birth rate ity rate lence rate
Bangla- 3.4 $ 14 0.4 0.19 22.2 54.3 0.014 0.056 0.0044 592382 0.4
1
desh 0.03 0.0017 0.3 0 0.03 0.013 0.009 0.02 0.0001 0 0.01
Central 6.017 $ 12 0.4 0.29 18.4 62.69 0.01 0.03 0.0007 683753 0.5
African 0.89
Republic 0 0.0042 0.2 0 0.05 0.0001 0.016 0.0001 0.0001 0.0001 0.01
12.67 $ 16 0.9 0.19 19.9 51 0.01 0.04 0.003 82799 0.6
Ghana 0.95
0 0.014 0 0.5 0.04 0.35 0.01 0 0 0 0.001
6.9 $ 27 1.04 0.4 24.9 59.71 0.011 0.04 0.0016 2741 0.55
India 0.77
0.0177 0.007 0 0.0004 0.04 0.87 0 0 0 0 0
17 $ 177 5.27 0.04 23 60.94 0.01 0.023 0.0001 23517 0.002
Jordan 1
0 0.0003 0.04 0 0.03 0.0001 0.0065 0.0046 0 0.00002 0.9
Mada- 4.2 $8 0.6 0.4 17.5 48.60 0.0153 0.08 0.0035 1866 0.54
1
gascar 0.009 0.00198 0.1 0 0.05 0.067 0.0142 0 0 0.00004 0.004
6.8 $ 13 1.2 0.34 19.8 53.3 0.0137 0.08 0.003 667218 0.4
Pakistan 1
0.0049 0.0065 0 0.3 0.04 0.3411 0.0082 0 0 0 0.01
South 28.738 $ 508.5 4.9 0.11 40.6 52.37 0.005 0.04 0.005 3011 0.7
0.72
Africa 0 0 0.03 0.007 0.03 0 0.007 0 0.007 0 0.001
13.45 178.6 3.04 0.19 28.1 67.96 0.008 0.02 0.0003 23888 0.57
Turkey 0.77
0 0 0.0005 0.06 0.04 0.6 0 0 0 0 0
Zim- 2.85 29.71 0.88 0.006 3.8 20.76 0.01 0.12 0.0003 140100 0.4
1
babwe 0.0073 0.00053 0.09 0 0.04 0.000037 0.01145 0 0 0 0.002

Table 11. Results for CRS output-oriented model with weight restrictions (reduced data set): efficiency scores, target values, and
weights

Country Reference Set

Bangla- Bangladesh (100%)

desh
Central
African Ethiopia (72.67%), Chad (26.7%), Botswana (0.5%), Jordan (0.06%)
Republic

Ghana Uganda (42.72%), Ethiopia (32.69%), Sri Lanka (13.91%), Pakistan (10.68%)

India Madagascar (50.86%), Nicaragua (22%), Bangladesh (17.96%), Pakistan (9%)

Jordan Jordan (100%)

Madagas-
Madagascar (100%)
car

Pakistan Pakistan (100%)

South
Mexico (76%), Zambia (19%), Guatemala (4.8%), Botswana (0.2%)
Africa

Turkey Nicaragua (39.8%), Paraguay (32.9%), Mexico (22.4%), Jordan (4.7%)

Zimbabwe Zimbabwe (100%)

Table 12. Reference sets for CRS output-oriented model with weight restrictions (reduced data set), listed in decreasing order

259
 Proceedings of the 37th International Conference on Computers and Industrial Engineering,
October 20-23, 2007, Alexandria, Egypt, edited by M. H. Elwany, A. B. Eltawil

erage score. Figure 2 illustrates the weak correlation

Score
$ per Quality Heal.
Eff. Access Equity between the rank orders from the WHO study and
capita Care Lives
those produced by our DEA analysis (for the CRS
Australia

$2,876 6.0 3.5 3.0 2.0 1.0 output-oriented unrestricted overall model). Interest-
1.0000

ingly, the agreement is fairly weak, with a correlation

0.0004 0 0.001 0.001 0.001 0.002
of r2 = 0.0196 (p = 0.0613). Thirteen of the WHO’s
best performing countries are “absolutely inefficient”
Canada

$3,165 4.1 1.0 1.0 1.0 1.0

0.0033

(i.e., both overall and in their respective economic

0.1 0 0.24 0.24 0.24 0.54
groups), with the exceptions of only Japan and Swit-
zerland. Further, some countries with the fewest
Germany

$3,005 6.0 1.4 1.0 3.0 1.5

0.4344

healthcare resources are ranked low by the WHO but

0.001 0 0.002 0.002 0.002 0.004 are efficient under the DEA approach, whereas some
countries with the highest outputs are ranked high by
New Zea-

$2,083 5.1 1.0 1.0 1.0 1.0

0.0091
land

the WHO but are inefficient with respect to DEA.

0.05 0 0.13 0.13 0.13 0.303
Comparison of WHO vs. DEA rankings
$2,546 6.0 1.0 4.0 1.0 4.5
1.0000

200
UK

0.0004 0 0.001 0.001 0.001 0.002

150
$6,102 1.0 1.0 1.0 1.0 1.0
.00001

DEA Ranking
US

37.6 0 95.4 95.42 95.42 214.71 100

Table 14. Results for restricted CRS output-oriented model:

efficiency scores, target values, and weights 50
r 2 = .0196

Country Reference Set 0

Australia Australia (100%) 0 50 100 150 200

Canada Australia (80.3%), UK (19.7%) WHO Ranking

Germany UK (98.5%), Australia (1.5%)
New Zealand UK (88.3%), Australia (12.7%)
UK UK (100%)
USA UK (59.9%), Australia (40.1%)
Table 15. Reference sets for restricted CRS output-oriented
model
The United States and Canada also were inefficient
for all other forms of DEA models, and both with
and without weight restrictions. The UK and Austra-
lia were efficient in all analysis, and Germany was
efficient only in models which do not have any
weight restriction constraints. The percentages listed
for the reference sets in Table 15 in a sense indicate
the extent by which each inefficient country would
need to emulate the UK and Australia in order to be-
come efficient.
4. COMPARISON OF RESULTS
The WHO and Commonwealth Fund studies both
include overall rankings of healthcare systems, based
on somewhat subjective equal (Commonwealth
Fund) or unequal (WHO) weightings of their indi-
vidual measures to compute an overall weighted av-
Figure 2. Weak correlation of DEA and WHO overall rankings
(r2 = 0.0196, p = 0.0613)
5. CONCLUSIONS
Data envelopment analysis appears to be a useful
approach that can complement other comparative
studies of national healthcare systems, often produc-
ing different conclusions. These results underscore
the value of this approach and of more explicitly
considering the levels of resources consumed relative
to outputs produced. This type of analyses might be
particularly useful in policy making, for benchmark-
ing, and when considering national healthcare system
reform or re-design. Further work ideally should
include more complete data and additional perform-
ance measures, such as patient satisfaction, diagnos-
tic error, patient-centeredness, and others.
REFERENCES
Alan W (2001), “Science or Marketing at WHO? A Com-
mentary on World Health 2000”, Health Economics,
10(2): 93-100.
Banker R, Charnes A, Cooper W (1984), “Some Models
for Estimating Technical and Scale Inefficiencies in

260
 Proceedings of the 37th International Conference on Computers and Industrial Engineering,
October 20-23, 2007, Alexandria, Egypt, edited by M. H. Elwany, A. B. Eltawil

Data Envelopment Analysis”, Management Science,

30(4), 1078-1092.
Benneyan J, Sünnetçi A (2006). “Handling Proportional
Data in Data Envelopment Analysis”, in preparation.
Charnes A, Cooper W (1962), “Programming with Linear
Fractional Functionals”, Naval Research Logistics
Quarterly, 9, 181-186.
Chilingerian J (1995), “Evaluating Physician Efficiency in
Hospitals: A Multivariate Analysis of Best Practices”,
European Journal of Operational Research, 80, 548-
574.
Coppola M, Ozcan Y, Bogacki R (2003), “Evaluation of
Performance of dental Providers on Posterior Restora-
tions: Does Experience Matter? A Data Envelopment
Analysis (DEA) Approach”, Journal of Medical Sys-
tems, 27(5):447-458.
Davis K, Schoen C, Schoenbaum S, Doty M, Holmgren A,
Kriss J, Shea K (2007), “Mirror, Mirror on the Wall:
An International Update on the Comparative Perform-
ance of American Health Care”, The Commonwealth
Fund, Boston.
Davis K, Schoen C, Schoenbaum S, Audet A, Doty M,
Tenney K (2004), “Mirror, Mirror on the Wall: Look-
ing at the Quality of American Health Care through the
Patient’s Lens”, The Commonwealth Fund, Boston.
Jamison D, Sandbu M (2001), “Global Health: WHO
Ranking of Health System Performance”, Science,
293(5535): 1595-1596.
Starfield B (2000), “Is US health really the best in the
world?”, Journal of American Medical Association,
284: 483-500.
University of Maine, Bureau of Labor Education, (2001).
“US Healthcare System: The Best in The World or Just
the Most Expensive?”, accessed June 12, 2007,
http://dll.umaine.edu/ble/U.S.%20HCweb.pdf

261