Efficiency of Health Care Systems what it is and how to measure and compare it by Gernot Pehnelt

Gernot.pehnelt@ecipe.org --DRAFT-Please do not cite without authors permission. Comments and feedback will be much appreciated

Introduction

Health often referred to as the highest good (summum bonum) and the provision of health care services have been a major issue in national and international politics and, not least, for the very individual, indeed. The question on how effective, efficient and fair health care systems around the globe perform has attracted much attention and an intensive and controversial debate not only in academic and political circles. In recent years, quite a few studies have been conducted in order to evaluate and compare the efficiency of health care systems. Given the different theoretical and methodological approaches, it is rather unsurprising that the results of these studies are far from conclusive. In this study, we will discuss the question what efficiency of health care systems actually means, how it can be measured and finally how the performance of health care systems can be compared on a reasonable and fair basis. In order to do so, we will first develop a theoretical framework of the very nature of the production process of health care systems. Based on these theoretical considerations, various approaches to evaluate the performance of health care systems will be assessed and a procedure to measure the efficiency of different health care systems will be developed and discussed.

The Health Care System A Productive Entity

The Output Dimension According to the World Health Report 2000 (WHO 2000) health systems are defined as comprising all the organizations, institutions and resources that are devoted to producing health actions. A health action is defined as any effort, whether in personal health care, public 1

health services or through intersectoral initiatives, whose primary purpose is to promote, restore or maintain health. In other words, health care systems can be described as productive entities, consisting of various components or sub-divisions, whose primary goal and economically speaking output is the improvement of the health care status of the (potential) patients.1 The value of healthy time gained by health actions can manifest itself in various forms. Healthy time can be used for productive work generating income for the very individual and creating an output and additional wealth for the society. Healthy time could also be used for leisure activities yielding to additional perceived personal utility. Last but not least, living in a better health state has a value to the individual in its own right, e.g. the absence of pain, higher mobility and better quality of life (Drummond et al. 2005). Sure, there are various other goals that contribute to the performance of a health care system including accessibility, fairness, a sustainable and just financing, customer satisfaction, transparency, respect for the patient etc. However, these instrumental goals are rather means to the very end of health care production which is protecting and improving the health status of the entire population over peoples whole life cycle. Once the achievement of this final goal can be measured, there might be no reason to integrate every single outcome or subordinate target explicitly into the efficiency analysis. Unfortunately, the improvement of the health status of the patients due to health actions cannot be measured directly. This is, first, because there is no exact measure or even a good proxy of the (virtual) health status of the population in the absence of any health care system.2 Second, the actual health status of any individual within a society with all the relevant physical and mental dimensions3 can hardly be quantified at a certain point in time, let alone in a dynamic perspective. Third, as we will discuss later, there are many factors outside the health care system that might influence the health status of individuals. To sum up, many questions about health system performance have no clear or simple answers because outcomes are hard to measure and it is hard to disentangle the health systems contribution from other factors (WHO 2000). Different outcome measures have been used to approximate the performance of health care systems. Generic outcome measures often used in utility analysis are quality-adjusted lifeyears (QALYs)4, healthy-years equivalent (HYE)5, the disability-adjusted life-years (DALY)6,
1

The term potential patients basically includes the entire population of a country since every individual has a certain demand for health services, let it be in cases of urgent emergencies, a demand for general diagnostic procedures, basic services or some sort of optional demand. Even very simple and indigenous societies have some sort of a health care system and use resources to perform health actions. By the way, following the definition of health given by the World Health Organization (WHO) whereby, health is a state of complete physical, mental and social well-being and not merely the absence of disease or infirmity, almost every human being might suffer from certain health deficits. QALYs basically take into account the quality and the quantity of the life years added by a medical intervention. Each year in perfect health is assigned the value of 1 down to a value of 0 for death. If the extra years were not being lived in full health, for example if the patient suffers constant pain or be confined to a wheelchair, then the extra life-years would be weighted with a value between 0 and 1 to account for this. HYE is calculated o the basis of the number of years of perfect health that has the same utility as the lifetime path of health states under consideration (Gold et al. 1996). The DALY is a health gap measure that extends the concept of potential years of life lost due to premature death to include equivalent years of healthy life lost by virtue of being in states of poor health or disability. It combines in one measure the time lived with disability and the time lost due to premature mortality. One

age specific mortality rates, infant mortality or simply the life expectancy at birth. On the one hand, these indicators are appropriate proxies for the final outcome of a health care system and other factors that influence the health status of human beings. On the other hand, the output of health care systems is multidimensional, by all means. First, because of the massive technological progress and accumulation of knowledge about the factors that influence the health status, certain diseases and their transmission, etc. the scope of modern health care systems is much more diversified than a couple of decades before. Health education, for instance, becomes more and more important when it comes to combating some of the most widespread diseases in Western society, such as obesity and diabetes. Second, people do no longer occupy the health care system for just the relief of pain and treatment of physical limitations and emotional disorders. They are also looking for diagnostic procedures on a regular basis without a direct indication, for advice on diet, child-rearing and other behaviour. Third, maintaining and improving a functioning health care system with sufficient capacities, decent facilities and well educated experts meets a certain optional demand that every individual has. That is why offering state-of-the-art health care services or providing the opportunity to use health care resources is an output itself. 7 Fourth, product and process innovation and technical progress in diagnostics, surgery, medication etc. also contribute to the performance of health care systems. The higher the rate of technological progress and accumulation of knowledge, the more effective will be the identification of disease patterns and, finally, the treatment of patients. Furthermore, under certain (unfortunate) circumstances, such as a natural disaster or a high prevalence of certain diseases (diabetes, HIV/Aids etc.) because of the populations genetic exposure or life-style patterns the health care system of a particular country might have to carry out a lot of health actions in order to maintain a decent level of health amongst the population whereas the health care system of another country simply does not face these rather structural problems that cannot be influenced in the short run. Saying this, it has to be recognized that to perform health actions is an (intermediate) output itself. Overall, there might be a case for integrating some of these additional or intermediate outputs when it comes to the assessment of the performance of health care systems.

The Input Dimension In order to assess the productivity of health care system, one has to define the actual boundaries of a health system regarding the very inputs that are used to produce the multidimensional output. There have been quite a few attempts to define boundaries that separate the health system from elements outside of it (e.g. Terris 1978, Hsiao 1992, Chernichovsky 1995). There is little doubt about that certain facilities and services such as hospitals and primary care are elementary parts of the input domain of health care systems. There is more controversy around components that have health-enhancing benefits but are not
DALY can be thought of as one lost year of healthy life and the burden of disease as a measurement of the gap between current health status and an ideal situation where everyone lives into old age free of disease and disability (WHO 2008). One important objective of modern health systems is to respond to peoples expectations regarding not only the actual quantity and quality of care but also the availability of decent treatment just in case.

primarily intended to influence the overall level of health within a society. 8 Examples for such factors are the educational system or urban infrastructure. Furthermore, there are some regulations and actions that clearly aim at the safety or health of individuals but are still not referred to as inputs within the health care system. For example, the prohibition of smoking in public buildings does most probably influence the smoking habits of individuals and most definitely contribute to the health of non-smokers who work in public buildings. However, the costs that are related to the enforcement of non-smoking rules are usually not integrated as an input of the production process of health care systems. The same holds for environmental and labour standards, certain safety rules and other regulations. One could argue that health education at school is primarily intended to promote health and should therefore be recognized in productivity analyses. Well, looking straight forward at first glance, this issue is again far from trivial. First, the question is whether deeper understanding of how diseases are spread, how important individual hygienic practices and a decent nutrition are and how certain habits may influence the risk of getting ill is an input or an output of health care system. On the one hand, the resources used to teach the students these things are means to an intended end; that is improved health. Therefore, these resources can be referred to as inputs in the production process of the health system. On the other hand, enhancing peoples knowledge about health related issues and giving them the opportunity to choose their individual life-style and habits rationally on the basis of a well-founded cognition of the complex sphere of health is most definitely beneficial and therefore an output itself. Second, from a practical point of view, one must confess that it would be almost impossible to separate the resources that have been used primarily in order to directly or indirectly promote health, let it be in public schools, businesses or sports clubs. How many hours a year does the 2nd grade teacher at school spend to teach things that are primarily intended to promote childrens health or their knowledge about health related issues? How many resources does a medium-sized enterprise invest to maintain or improve the health of its employees and what will be the result? Even if some schools or businesses account for these things, comprehensive information about these health-related inputs/outputs do not exist on a broader scale and will most probably never exist. Each health system should be judged according to the resources actually at its disposal, not according to other resources which in principle could have been devoted to health but were used for something else (WHO 2000), even if these activities outside the health care system do have an impact on the health status of the population. We call factors that are outside the defined boundaries of the health care system non-health care determinants. To make one thing clear, these non-health care determinants may play a major role in assessing the overall performance of health care systems and are of special importance in explaining differences between countries. That is why they surely have to be recognized in any analysis, though maybe not explicitly as direct inputs in the very production process of health care systems. However, given the considerations discussed above, we prefer a rather narrow definition of the boundaries of health care systems. It is the resources that are employed in the health care
8

Clearly, all boundary definitions are arbitrary but to undertake an assessment of health system performance an operational boundary must be proposed. (Murray and Frenk 1999)

sector and, indeed, health care expenditures that matter most when it comes to an assessment of the efficiency of health care systems. Some of the resources used in the production process in the narrower sense can be measured relatively easily. It is the labour employed in the health sector, the capital that has been invested in health care facilities, the utilities and other health care expenses that can be measured in monetary terms. However, there are still some important inputs that are rather intangible, such as the specific know-how and the level of technology. These inputs can be estimated by certain proxies as the number of high-tech equipment such as MRT or CT or the ratio of examined medical personnel employed in the health care sector.

The health care production process at a glance The production process of health care systems can be described as follows. Resources that lie within the boundaries, referred to as health care resources, are used to provide health care services in order to improve the health status of the population.9 The primary output of this process is the difference between the initial health status of the population (without any health services) and the actual health status that has been achieved due to the health actions performed by the health system (the delta in Figure 1).
non-health care determinants:
average income education sanitation, nutrition environment, polution etc. life-style other socioeconomic factors

initial health status population (potential patients)

health care resources

health care system

regulation and other policies

population actual health status

Figure 1: The health care system as a productive entity

The initial (and actual) health status of the population as well as the health care system itself are highly influenced by factors that lie more or less outside the boundaries of the health care systems, such as the average income, the educational level, sanitation and nutrition, the
9

The patients are the most important physical input of the production process, so to speak.

= output of the health care system

environment, genetic exposure, the life-style and other socioeconomic factors.10 These nonhealth care determinants might be even more important for the health status of a countrys population. In fact, some evidence suggests that health systems make little or even no difference regarding the overall level of health. Comparisons across countries show that while per capita income, education, income inequality or cultural characteristics are strongly related to the actual health status there seems to be little independent connection between the level of health and health care inputs such as doctors or hospital beds (Cochrane et al. 1978), total health expenditure (Musgrove 1996), expenditure only on conditions amenable to medical care (Mackenbach 1991), or public spending on health (Filmer and Pritchett 1999). It is not surprising to find that these relations are weak in rich countries, since many causes of death and disability are already controlled and there are many different ways to spend health system resources, with quite varying effects on health status. But health system expenditure often seems to make little difference even in poor countries with high infant and child mortality, which should be a priority to reduce (WHO 2000).11 Figure 2 translates the complex health care system into terms of production. The initial (secondary) inputs such as medical personnel and other labour, equipment, facilities and other capital, know-how, utilities and other resources are used and combined in the production process. The very nature of this production process in terms of a production function remains pretty much unclear and can be referred to as a black box. The intermediate output of this production process are the health actions that are performed to reach the final goal. These secondary outputs such as diagnoses, surgeries and other procedures, outpatient treatments and care days, medication, information and documentation are used as primary inputs in order to cure the patients, ease their pain, weaken the symptoms and maintain or improve overall health which is the final and primary output of the health care system.

10

11

Maybe, some of these factors can be influenced by certain health policies and regulations, at least in the long run. However, many of these non-health care determinants lie completely outside even a broad definition of what health systems are. There is little evidence for a significant impact of higher cost on beneficiary outcome. Some regions in the US have markedly higher cost than others, with no higher patient satisfaction or health improvement (Dartmouth 1998).

capital labour know-how utilities other resources

health care resources / secondary inputs

diagnose

production process (Black Box") (combination of secondary inputs)

process 1 process 2

procedures and

surgeries
patient & care days medication documentation

process n

primary input / secondary output

cure easing the pain weakening the

symptoms
improve overall health

primary output

Figure 2: Inputs and outputs in the health care production process

As discussed before, the secondary inputs can be measured relatively easily by counting the number employees or the full-time equivalents (FTE) in the different sub-sectors of the health care system and collecting data about health care expenses. The same is true for the secondary outputs that are basically well documented in health care statistics. The difficult part is to quantify the primary output. The generic measures or a combination of some of these measures mentioned above seem to be the most appropriate way to estimate the final outcome of the production process.

The assessment of health system performance

The concept of relative performance Given the tremendous importance of non-health care determinants for the initial and actual health status of the population, international comparisons of the performance of health care systems are not a trivial task. Just comparing the outcome or output-input-ratios might be not appropriate to get a reasonable and fair assessment of the overall performance and efficiency of different health systems around the world. In the following we explain the difficulties that come around and develop an approach that is able to overcome some of these problems. Murray and Frenk (1999) introduced a concept that compares the actual performance of a health care system with its potential, taking into account that the actual outcome that is the health status of the population would not be zero if there was no health care system at all. Even without any health care expenditure or a functioning health care system, health levels 7

would not be zero because the entire population would not be dead or seriously ill. The initial health status, so to speak, of a countrys population is supposed to be highly correlated with various factors such as the average income, the educational level, environmental issues, infrastructure, sanitation, nutrition and the like (non-health care determinants). The outcome of a health care system could be defined as the difference between the observed health status of the entire population and the initial (virtual) health status of the population given a certain level of these factors. Therefore, performance of the health system involves relating goal attainment to what could be achieved under certain circumstances that cannot be addressed by the health care system directly. In other words, performance is a relative concept. A rich country has higher levels of health than a poor one, but which country has a higher level of performance relative to health system resources? Murray and Frenk (1999) argue that performance should be assessed relative to the worst and best that can be achieved for a given set of circumstances. Figure 3 illustrates an example.12 The upper lines (hmaxi) show the maximum attainable level of health (y-axis) for the given level of health expenditures (x-axis), given the non-health care determinants of health status such as the average income, the educational level or environmental pollution for every country i. The stars (hmini) represent the minimum level of health given the specific non-health system circumstances in this particular country i. The circles (Hi) show the actual observed health status in each country i. What can be seen is that country A certainly has a lower level of health than country B initially (hminA < hminB) and actually (HA < HB). However, both countries show exactly the same health care system performance in absolute terms, no matter if this performance is measured by the distance between minimum level and the actual level of health (p* = improvement due to the health care system) or the distance from the observed health status to the maximum attainable frontier (p = potential improvement / level of inefficiency). In relative terms, the performance of country As health care system is even better than the performance of B. Notice that even if country A invested as much health care resources as country B, the initial health status of country As population would still quite below the health status of country Bs population (hminA < hminB). That is due to unfavourable circumstances such as certain socioeconomic, life-style or environmental factors. That is why the maximum attainable health status the individual efficient frontier at a given amount of health expenditures is also lower in country A than in country B (FhmaxA < FhmaxB). 13

12

13

The examples and illustration given by Murray and Frenk (1999) and elsewhere (e.g. Evans et al. 2001) are somehow misleading. That is why we use a different approach to illustrate the very problem. The shape of the efficient frontier implicitly assumes diminishing marginal returns to investments in health care services. We will discuss this issue later in this chapter. The slope of the minimum level of health (Fhmin) can be expected to have a curved slope as well. However, since this detail is not relevant here, we skip this discussion.

h max

Fh max h max
B

Fh max Fh max
A

p health status h max p H

A A A

p H
B

H h min

C C

Fh min p*
B

p*

Country C
A

h min

Fh min

Country B p* Fh min
A A

h min

Country A health system resources Figure 3: Health system performance

There might be another country (C) that has the same actual health status as country B (HC = HB). At first glance, country C would probably ranked first among the three countries if it comes to a simple ratio analysis of the performance of the health care system since country C uses fewer resources than country B and reaches the same overall health status. However, the performance of country Cs health care system seems to be not much better or even weaker than Bs performance, since the improvement due to the health care system is lower in country C than in country B (p*C < p*B) and the lack of efficiency compared to the individual efficient frontier (Fhmax) is higher in country C (pC > pB). Because of the favourable nonhealth care determinants of health status in country C, this country could reach a much higher overall health status than it does (see the efficient frontier FhmaxC). C faces the highest individual efficient frontier among the three countries. To assess the relative performance of health care systems, it would be desirable to develop an individual benchmark, an upper limit or frontier, corresponding to the best practice that could be achieved by the health system under consideration. This frontier would represent the level of attainment which a health system might achieve under the given non-health care determinants, but which no country surpasses (Donabedian et al. 1982). With such a frontier approach it would be possible to see how much of the potential has been realized. In other words, comparing actual attainment with potential shows how far from the relevant frontier of maximal performance is each countrys health system (WHO 2000). In economic terms, the distance between the actual outcome and the potential would be referred to as slack or inefficiency. Now we draw our attention to the slope of the best-practice frontiers in Figure 3 which leads us to a very important fact that have not been recognized by the vast majority of empirical studies. It is very reasonable to assume diminishing returns in the production of health 9

concerning the final outcome, regardless the measure applied. Coming from a very poor level of overall health, i.e. a low life-expectancy, it might take just a few relatively inexpensive measures to improve the average health and life-expectancy of the population. The improvement of hygiene in hospitals, for instance, has dramatically reduced hospital mortality in the 20th century. The cost of the related measures, such a sterilizing surgical instruments or simply encouraging medical personnel to wash ones hands and provide the necessary facilities to do so, are comparably low. The same holds for basic vaccination. Some diseases have almost been wiped out in many industrialised countries in the last century due to vaccination. By the way, some of the most effective measures that are able to enhance the overall health status and improve the life-expectancy lie outside the health care system. The tremendous increase in life-expectancy and the quality of life since the mid-1800s where due to better nutrition, improvements in urban sanitation and personal hygiene that came along with a better understanding of how diseases spread and, not least, the rise of capitalism, an enhanced division of labour and the subsequent economic growth and increasing social welfare. Technically speaking, economic growth pushed the frontiers upwards and increased the overall level of health by improving the initial health status. Furthermore, technological progress and a more efficient use of resources made it possible to invest more in health, which would be represented by a countrys move to the right in Figure 3. Once a country has already reached a high level of health, it takes much more effort to further reduce morbidity and increase life-expectancy. Compared to the rather basic health actions mentioned above, the treatment of certain diseases like cancer are very expensive. Although modern chemo and radiation therapy together with subsidiary medication are quite successful, they are not able to increase the average life expectancy within the society by years or even decades like some of the measures discussed above could. The same is true for almost every intensive care measure. Intensive care, a surgery or decent medication can save ones life, ease the pain or even cure a disease and therefore add some additional life-time. However, in the end, even if one disease can be successfully defeated, it just means replacing one particular cause of death by another one. The higher the average level of health (e.g. the lifeexpectancy) is, the harder and more expensive it gets to further improve the health and life expectancy of the population. Why is this important for efficiency analyses of health care systems? Well, economically speaking, the health system faces diminishing marginal returns beyond a certain level, the production function becomes very flat. That is why country B in Figure 3 has to invest much more into its health care system (in absolute and relative terms) to get a marginal improvement of the overall health status than country A. In other words, the marginal productivity of country As health care system at this certain point of economic development is by nature higher than the marginal productivity of country Bs health system. Any investment in health actions in country A (movement to the right in Figure 3) would add quite a lot to the average level of health in this country, whereas country B must invest quite a lot to increase the overall health of its population just a bit. What can be drawn is that, because of various reasons, it is a trivial task to actually compare the efficiency of country A and country B regarding the production of health. 10

However, there have been attempts to compare the efficiency of different health care systems and some of them are quite promising. Since some studies do not take the relative nature of efficiency into account, we, in the following, rather focus on studies that do incorporate the relative concept of performance.

Data Envelopment Analysis an appropriate tool to measure relative efficiency Based on the work of Farell (1957)14 Charnes et al. (1978) developed the Data Envelopment Analysis (DEA) to measure the efficiency of productive entities, so called decision making units (DMUs). This method, which has been advanced during the last decades 15 , is very appropriate in assessing the (relative) efficiency of DMUs that produce multidimensional outputs and use certain inputs that cannot be measured in monetary terms. Model (1) calculates the relative efficiency of DMU 0 in comparison to every other DMU in the observed sample of n DMUs. Every DMU j is a productive entity that uses the i = 1,..., m inputs (xij > 0) to produce r = 1,..., s outputs (yrj > 0). The model chooses the individual weights of every single input and output (qi, pr) in a way that maximizes the efficiency score ho.
s

p r y ro max h o
p, q s r 1 m

qi xo
i 1

p r y rj NB:
r 1 m

1 q i x ij

j 1, ..., n

(1)

i 1

pr, qi

1,..., s

1,..., m.

The optimization model (2) the so called envelopment-form calculates the DEA-Score o* that shows to which level the DMU 0 has to reduce the inputs x to become efficient. Efficient units get a DEA-Score of 1.
min o
n

NB:
j 1

j y rj
n j 1

y ro j x ij 0

r i

1,..., s

(2)
1,..., m

x io j 0

j 1,..., n.

14 15

For an overview on the origins of Data Envelopment Analysis see Forsund and Sarafoglou (2002). See for instance Lthgren and Tambour (1996), Andersen and Petersen (1993), Cooper et al. (1996) and Sengupa (1989). Seiford (1996) provides an outline of the early developments and a bibliography of the DEA.

11

Figure 4 shows an example with seven DMUs that produce the same output (quantity and quality) with two inputs.
Input 2 A F

C
2

G D E

Input 1

Figure 4: DEA Example with 7 DMUs, two inputs and one output

The efficient DMUs A, B, C, D and E lie on the frontier.16 The frontier envelops the two inefficient DMUs F and G. In order to be ranked efficient, F and G had to reduce both inputs to reach point C and G, respectively. As can be seen, the DEA is a relative concept of efficiency. The efficiency of every DMU is assessed relatively to the performance of real DMUs or convex combinations of existing DMUs. The benchmark, so to speak, for DMU F is DMU C. The benchmark for DMU C is constituted by a combination of the efficient peers C and D. DEA explicitly incorporates the relative concept of efficiency by measuring the relative distance of every DMU to the best-practice frontier. Furthermore, the peers or benchmarks of every inefficient DMU are generally similar to that DMU in terms of the relative importance of the inputs and outputs in the production process. With DEA one can also take into account variable returns to scale. Some technologies possess increasing returns to scale, whereas other production processes might come along with constant or decreasing returns to scale. Under the assumption of constant returns to scale (CRS), the efficient frontier FCRS in Figure 5 is determined by the position vector from the origin through the points C and A. The efficient DMUs A and C lie on this frontier and are identified as efficient (ACRS*= CCRS*= 1). The other DMUs (B, D and E) are ranked inefficient by the CRS-model (BCRS* DCRS* ECRS*< 1). The magnitude of the inefficiency is quantified by , , the relative distance of the inefficient DMU to the frontier. In case of DMU B this is the ration BB/BB. In order to become efficient, DMU B had to reduce the input by the distance BB.
16

The frontier is determined by linear sections (convex combinations) between two efficient DMUs at a time. Note that DMUs A and E lie on the frontier although they could reduce one input by one unit. The DMUs A and E exhibit some slack and are called weakly efficient.

12

Output

FCRS D A C E FVRS

0
Figure 5: DEA CRS- and VRS-model

Input

Under the assumption of variable returns to scale (VRS) the best-practice frontier FVRS is determined by the DMUs A, B, C and D and the respective convex combinations of the neighbouring DMUs. The VRS-model identifies 4 of the 5 DMUs as efficient, taking into account that up to a certain size, the production process possesses increasing returns to scale and beyond a certain threshold decreasing returns to scale. In other words, the DMUs B and D produce technically efficient but are too small (DMU B) or too large (DMU D) to be fully efficient. DMUs B and D are scale inefficient. The magnitude of the scale inefficiency of a DMU 0 can also be measured using the DEA-scores calculated by CRS- and VRS models:
o CRS* o VRS* o

(3)

One of the biggest advantages of DEA is the possibility to integrate various multidimensional inputs AND outputs into the efficiency analysis. This is of special importance in the health care sectors where the output is multidimensional and some inputs and especially the output can hardly be measured in monetary terms. Furthermore, there is no need to specify a certain type of production function. Since very little is known about the functional form of health care production, the endogenous so to speak empirical way to identify the best-practice frontier is another major advantage of this non-parametric technique. DEA can also be used to measure productivity changes over time. Figure 6 illustrates how the dynamic productivity change can be measured in a disaggregated way. DMU D, for instance, is identified in both points of time (t and t+1). The technical efficiency of the inefficient DMU D at time t is 0J/0Dt and 0M/0Dt+1at time t+1, respectively (DCRS*< 1). In order to assess the dynamic productivity change of DMU D, its technical efficiency relatively to the frontier in time t and time t+1 will be measured. The geometric mean of these two rates of change shows 13

the change of the total factor productivity of DMU D (TFPD) over the whole period. This is the Malmquist-Index of this particular DMU (MID).
Input 2 FCRS time t +1 FCRS time t A
t+1

A
t+1

D K B

t+1

D B
t

J C =C
t t+1

M L

0 Figure 6: Disaggregated Malmquist-Index

Input 1

The Malmquist-Index can be separated into two measures of efficiency change, an efficiency change (catch up) component and a technology change (frontier shift) component:
MI D 0M 0D t 1 0J 0D t 0K 0M 0J 0L
1 2

(4)

catch up (E)

frontier shift (F)

The first term on the right hand side (catch up) of equation (4) measures the change of the distance of DMU D to the frontier in time t+1 (0M/0Dt+1) relative to the distance to the frontier in time t (0J/0Dt). If the score for this catch up component is higher than 1, the DMU has enhanced its relative technical efficiency, it has cough up.17 The term on the right hand side of equation (4) measures the shift of the frontier over time. Values of this component bigger than 1 indicate dynamic productivity gains within the sample or, so to speak, the underlying production technology. Using the disaggregated Malmquist-Index one cannot only identify the productivity change of every single DMU that are due to individual efforts but also efficiency changes that are due to technological progress or other factors that influence the productivity of the whole sample. Again, this tool is very useful to assess the efficiency of productive entities in health care because it incorporates a relative efficiency concept and can handle multidimensional inputs and outputs.

17

If the score for the catch up component iss maller than 1, one actually had to call it fall behind.

14

Efficiency analysis in health care some applications and results Because of its specific advantages, during the last 20 years, Data Envelopment Analysis and other non- or semi-parametric approaches have been used increasingly to measure the efficiency of health care services. The vast majority of these studies focus on certain facilities such as hospitals (e.g. Grosskopf and Valdmanis 1987, Borden 1988, Ozcan and Luke 1993, Burgess and Wilson 1995, Grosskopf et al. 2001, Biorn et al. 2003 and Prior 2006), nursing homes (e.g. Nyman and Bricker 1989, Kooreman, P. 1994, Rosko et al. 1995 and Chattopadhyay and Ray 1996) or primary health care centers (e.g. Huang and McLaughlin 1989, Szczepura et al. 1993 and Athanasios et al. 2002).18 Interestingly, there are not many applications of non-parametric techniques to measure and compare the efficiency of health care systems as a whole.19

Fre et al. (1997), analyzing a sample of 19 OECD countries, found a very widespread and rapid productivity growth between 1974 and 1989 among the 10 countries with complete data with Denmark and the US showing the highest cumulative productivity growth in this period (> 30%). The main driver of this productivity growth was technical change. Alfonso and St. Aubyn (2006) estimate semi-parametric models of health production using a DEA-based two-stage approach for OECD countries for the years 2000-2003. Input variables include medical technology indicators and health employment. Output is measured by indicators such as life expectancy and infant mortality. The authors use a principal component analysis (PCA) of the outputs life expectancy, infant survival rate and potential number of years of life not lost. This reduces the dimensionality of multivariate output to one single measure which has its advantages when the sample size is relatively small. However, by running PCA one gets an artificial output measure that might not represent the output of every health care system well enough. In the basic model, among 21 OECD countries, Canada, Finland, Japan, Korea, Spain, Sweden and the US show the highest efficiency. The efficiency of the Czech Republic, the Slovak Republic and especially Hungary falls way behind the sample median. Inefficiency in the health sector has been found strongly related factors that are, at least in the short to medium run, beyond the control of the health system: the GDP per capita, the level of education, smoking habits, and obesity. In a next step, the authors take into account differences in national income, obesity and tobacco consumption and correct the efficiency score for these factors. Comparing the ranks and efficiency scores of the uncorrected model with the scores and ranks resulting from the correction, significant changes occur. Some countries poorly ranked previously are now closer to the production possibility frontier e.g. Denmark, the Czech Republic, Hungary, the Slovak Republic, and the UK. On the other hand, countries like Canada, Japan, Sweden and the US receive remarkably lower efficiency scores after the correction.
18

19

For a review of non-parametric studies in health care see Hollingsworth et al. (1999) and Hollingsworth (2003). Gupta et al. (1997) use an approach that is similar to DEA (free disposal hull FDH) to analyze the efficiency of government spending on health and education in Africa with life expectancy and infant mortality as the main outputs.

15

Spinks and Hollingsworth (2005) go even further in integrating socioeconomic factors into the models. They use measures for education (school expectancy years), employment (unemployment rates), and income (GDP per capita) as inputs, additional to total health expenditure. Life expectancy at birth is the only output. The authors calculated the technical efficiency of OECD countries based on the data provided by the OECD and the WHO respectively. In the WHO setting, they use a different output measure (disability adjusted life expectancy DALE). Turkey, Mexico, Korea, Greece, Spain and Japan reach full efficiency in both models (OECD and WHO dataset). Italy and France lie on the frontier only in the WHO setting. Iceland and Switzerland are identified as efficient only in the OECD setting. The most remarkable efficiency deficits have been found for Denmark, the US, Belgium and Germany.20 Spinks and Hollingsworth (2005) also calculate changes in efficiency between 1995 and 2000 (OECD dataset) and 1993 and 1997 (WHO database) using the disaggregated MalmquistIndex. The mean value of the catch up component is 0.961 for the OECD dataset indicating that overall, member countries have moved slightly away from the frontier, representing a decrease in technical efficiency. Similarly, the mean technological change value (frontier shift) of 0.995 would suggest that that the efficiency of the whole technology has declined slightly. Overall, the mean efficiency of the whole sample have decreased between 1995 and 2000 (mean Malmquist-Index = 0.956). Interestingly, results using the WHO dataset move conversely to results for the OECD dataset. Overall, the technical efficiency has improved (mean catch up component = 1.041), whereas the efficient frontier has retracted (mean frontier shift = 0.974). The total factor productivity seems to have increased slightly between 1993 and 1997. (MI = 1.014). Looking at the individual scores of the disaggregated Malmquist-Index calculated by Spinks and Hollingsworth (2005) on the basis of the two different datasets draw a rather dubious light on the results since for the majority of countries the scores move to the opposite direction using the two different datasets. Since the two periods do not differ considerably, the odd result maybe due to the different output measure (OECD: life expectancy, WHO: DALE). Both approaches discussed above (Alfonso and St. Aubyn 2006; Spinks and Hollingsworth 2005) incorporate broad socioeconomic or life-style measures more or less explicitly into the efficiency analysis. As argued before, some measures of socioeconomic status, such as the average income, the level of overall economic development, education and environmental factors have clearly shown to be associated with health status (Macintyre 1993, Kaplan et al. 1996, Laporte 2002). In other words, different countries may face different individual efficiency frontiers. That is why the idea to correct the efficiency scores for some important factors that cannot be influenced by the health care system, at least not in the short run, follows the concept introduced above (see again Figure 3). However, the dramatic changes of the ranks and especially some individual efficiency scores due to the correction draw a rather dubious light on the methodological approach used by Alfonso and St. Aubyn (2005). Furthermore, integrating factors that are not under control of the productive entity to be
20

Notice that some of these findings are quite contrary to the results documented by Alfonso and St. Aubyn (2006).

16

assessed directly into the efficiency analysis (Spinks and Hollingsworth 2005), is in conflict with the very nature of efficiency analysis.

Towards a concept to measure the efficiency of health care systems discussion

Efficiency in health care is relative. That is why the actual performance of a health system has to be assessed relative to the performance of a group of peers that can be reasonably compared to the system under evaluation. Although factors that lie outside the boundaries of the health system might be even more important in determining the health status of the population than the actions performed by the health system, it does not make sense to incorporate factors that cannot be controlled by the decision makers within the health system into the efficiency analysis, at least not explicitly as inputs. However, these factors surely have to be taken into account when it comes to an assessment of the (relative) performance of the health system. Therefore, we suggest a clear cut two-step procedure. First, the relative efficiency of health care systems has to be measured by including all inputs that are used to actually produce the output. Since the output of health systems is multidimensional, it might be not enough to focus solely on life expectancy or other rather broad measures of the health status of the population as the output measure. It could be necessary to integrate some of the secondary outputs (e.g. some health actions that have been actually performed) into the analysis in order to get a reasonable and fair evaluation of the efficiency of health systems. This is not least due to the diminishing marginal returns to investment in health care that most probably causes a lower marginal, though not necessarily average, productivity in countries that already invest a lot into the health system. Once the efficiency of the health systems has been calculated on the basis of factors that are under control of the health system, the causes of differences in the relative efficiency can be investigated, using regression analysis with the efficiency scores as the dependent variable and certain socioeconomic and other factors as explanatory variables. The results can be used to further elaborate and interpret the efficiency assessment. Using an appropriate research design, one may even calculate specific elasticities of certain factors that affect the efficiency of health care systems. These elasticities could be used to correct the efficiency scores derived in the first step in order to take into account the different best practice frontiers the different countries might face. Because of its relative and non-parametric concept and the possibility to integrate multiple inputs and outputs of different dimension into the analysis, the Data Envelopment Analysis (DEA) is a very appropriate and powerful tool to assess the efficiency of health care systems.

Chapter on the design or reference to the second paper

17

18

References

Afonso, A. and St. Aubyn, M. (2006): Relative Efficiency of Health Provision: a DEA Approach with Non-discretionary Inputs. Working Papers 2006/33, Department of Economics at the School of Economics and Management (ISEG), Technical University of Lisbon. Athanasios, I.Z.; Tsakos, G.; Economou, C. and Kyriopoulos, J. (2002): Using DEA to Evaluate Efficiency and Formulate Policy Within a Greek National Primary Health Care Network. Journal of Medical Systems, Vol. 26(4), pp. 285-292. Biron, E.; Hagen, T.P.; Iversen, T. and Magnussen, J. (2003): The Effect of Activity-Based Financing on Hospital Efficiency: A Panel Data Analysis of DEA Efficiency Scores 1992-2000. Health Care Management Science, Vol. 6(4), pp. 271-283. Borden, J.P. (1988): An Assessment of the Impact of Diagnosis-Relatd-Groups (DRG)-Based Reimbursement on the Technical Efficiency of New Jersey Hospitals Using Data Envelopment Analysis. Journal of Accounting and Public Policy, Vol. 7(2), pp. 77-96. Burgess, J.F. and Wilson, P.W. (1995): Decomposing Hospital Productivity Changes, 1985 1988: A Nonparametric Malmquist Approach. Journal of Productivity Analysis, Vol. 6(4), pp. 343-363. Chattopadhyay, S. and Ray, C.S. (1996): Technical, scale and size efficiency in nursing home care: a non-parametric analysis of Connecticut homes. Health Economics, Vol. 5(4), pp. 363-373. Charnes, A; Cooper, W.W. and Rhodes, E. (1978): Measuring the efficiency of decision making units. European Journal of Operational Research, Vol. 2(6), pp. 429-444. Cochrane, A.L.; Leger, A.S.St. and Moore, F. (1978): Health service "input" and mortality "output" in developed countries. Journal of Epidemiology and Community Health, 32(3), pp.200-205. Cooper, W.W.; Thomson, R.G. and Thrall, R.M. (1996): Introduction: Extensions and new developments in DEA. Annals of Operations Research, Vol. 66(1), pp. 3-45. Dartmouth (1998): The Dartmouth Atlas of Health Care 1998. The Dartmouth Institute for Health Policy and Clinical Practice, Center for Health Policy Research, Dartmouth Medical School. Dartmouth (2008): Tracking the Care of Patients with Severe Chronic Illness. The Dartmouth Atlas of Health Care 2008. The Dartmouth Institute for Health Policy and Clinical Practice, Center for Health Policy Research, Dartmouth Medical School. Donabedian, A.; Wheeler, J.R.C.; Wyszewianski, L. (1982): Quality, cost, and health: an integrative model. Medical Care, Vol. 20, pp. 975-992.

19

Drummond, M.F.; Sculpher, M.J.; Torrance, G.W.; O'Brien, B.J. and Stoddart, G.L. (2005): Methods for the Economic Evaluation of Health Care Programmes. 3rd Edition, Oxford University Press. Evans, D.B.; Tandon, A.; Murray, C.J.L. and Lauer, J.A. (2001): The comparative efficiency of national health systems in producing health: an analysis of 191 countries. Global Programme on Evidence for Health Policy Discussion Paper No. 29, World Health Organization, Geneva. Fre, R.; Grosskopf, S.; Lindgren, B. and Poullier, J.-P. (1997): Productivity Growth in Health-Care Delivery. Medical Care, Vol. 35(4), pp. 354-366. Farrel, M.J. (1957): The measurement of productive efficiency. Journal of Royal Statistical Society, Series A, Vol. 120(3), pp. 253-281. Filmer, D. and Pritchett, L. (1999): The impact of public spending on health: does money matter? Social Science and Medicine, Vol. 49(10), pp. 1309-1323. Forsund, F.R. and Sarafoglou, N. (2002): On the Origins of Data Envelopment analysis. Journal of Productivity Analysis, Vol. 17(1-2), pp. 23-40. Gold, M.R.; Siegel, J.E.; Russel, L.B.; Weinstein, M. (eds) (1996): Cost-effectiveness in health and medicine. New York: Oxford University Press, 1996. Grosskopf, S.; Margaritis, D. and Valdmanis, V. (2001): Comparing Teaching and Nonteaching Hospitals: A Frontier Approach (Teaching vs. Non-teaching Hospitals). Health Care Management Science, Vol. 4(2), pp. 83-90. Grosskopf, S. and Valdmanis, V. (1987): Measuring Hospital Performance. A Non-parametric Approach. Journal of Health Economics, Vol. 6(2), pp. 89-107. Gupta, S.; Honjo, K. and Verhoeven, M. (1997): The efficiency of government expenditures: Experiences from Africa. IMF Working Paper No. WP/97/15, International Monetary Fund, Washington D.C. Huang, Y.G. and McLaughlin, C.P. (1989): Relative efficiency in rural primary health care: an application of data envelopment analysis. Health Services Research, Vol. 24(2), pp. 143-158. Hollingsworth, B. (2003): Non-Parametric and Parametric Applications Measuring Efficiency in Health Care. Health Care Management Science, Vol. 6(4), pp. 203-218. Hollingsworth, B.; Dawson, P.J. and Maniadakis, N. (1999): Efficiency measurement of health care: a review of non-parametric methods and applications. Health Care Management Science, Vol. 2(3), pp.161-172. Kaplan, G.A.; Pamuk, E.R.; Lynch, J.W.; Cohen, R.D. and Balfour, J.L. (1996): Inequality in income and mortality in the United States: analysis of mortality and potential pathways. British Medical Journal, Vol. 312(7037), pp. 999-1003. Kooreman, P. (1994): Nursing home care in The Netherlands: a nonparametric efficiency analysis. Journal of Health Economics, Vol. 13(3), pp. 301-316. 20

Laporte, A. (2002): A note on the use of a single inequality index in testing the effect of income distribution on mortality. British Medical Journal, Vol. 55(9), pp. 1561-70. Lthgren, M. and Tambour, M. (1996): Alternative Approaches to Estimate Returns to Scale in DEA-Models. Working Paper No. 90, Stockholm School of Economics, Stockholm. Macintyre, S.; Maciver, S. and Sooman, A. (1993): Area, class and health: should we be focusing on places or people? Journal of Social Policy, Vol. 22, pp. 213-34. Mackenbach, J.P. (1991): Health care expenditure and mortality from amenable conditions in the European Community. Health Policy, Vol. 19, pp. 245-255. Murray, C.J.L. and Frenk, J. (1999): A WHO framework for health system performance assessment. Global Programme on Evidence for Health Policy Discussion Paper No. 6, World Health Organization, Geneva. Musgrove, P. (1996): Public and private roles in health: theory and financing patterns. Washington D.C., The World Bank, 1996 (World Bank Discussion Paper No. 339). Nyman, J.A. and Bricker, D.L. (1989): Profit incentives and technical efficiency in the production of nursing home care. The Review of Economics and Statistics, Vol. 71(4), pp. 586-594. Ozcan, Y.A. and Luke, R.D. (1993): A National Study of the Efficiency of Hospitals in Urban Markets. Health Services Research, Vol. 27(6), pp. 719-740. Prior, D. (2006): Efficiency and total quality management in health care organizations: A dynamic frontier approach. Annals of Operations Research, Vol. 145(1), pp. 281-299. Rosko, M.D.; Chilingerian, J.S.; Zinn, J.S. and Aaronson, W.E. (1995): The effects of ownership, operating environment, and strategic choices on nursing-home efficiency. Medical Care, Vol. 33(10), pp. 1001-1021. Seiford, L.M. (1996): Data Envelopment Analysis: The evolution of the state of the art (19781995). Journal of Productivity Analysis, Vol. 7(2-3), pp. 99-137. Sengupta, J.K. (1989): Nonlinear measures of technical efficiency. Computers & Operations Research, Vol. 16(1), pp. 55-65. Spinks, J. and Hollingsworth, B. (2005): Health production and the socioeconomic determinants of health in OECD countries: the use of efficiency models. Centre for Health Economics, Working Paper 151. Szczepura, A.; Davis, A.; Fletcher, C.J. and Boussofiane, A. (1993) Efficiency and effectiveness in general practice, Journal of Management in Medicine, Vol. 7(5), pp. 36-47. WHO (2000): The World Health Report 2000. Health Systems: Improving Perforance. World Health Organization, Geneva, 2000. WHO (2008): Health statistics and health information systems. Disability adjusted life years (DALY). [http://www.who.int/healthinfo/boddaly/en/] 21