Professional Documents
Culture Documents
WHO MSD MSB 00.2i PDF
WHO MSD MSB 00.2i PDF
2i
Workbook 8
Economic
Evaluations
WHO
World Health Organization
UNDCP
United Nations International Drug Control Programme
EMCDDA
European Monitoring Center on Drugs and Drug Addiction
This document is not a formal publication of the World Health Organization (WHO) and all rights are reserved by
the Organization. The document may, however, be freely reviewed, abstracted, reproduced and translated, in
part or in whole but not for sale nor for use in conjunction with commercial purposes. The views expressed in
documents by named authors are solely the responsibility of those authors.
Acknowledgements
Table of contents
Sample questionnaires 30
Discounting tables 31
References 31
Overview of
Workbook Series
This workbook is part of a series formation that comes from these
intended to educate programme evaluation activities.
planners, managers, staff and other
decision-makers about the evalua- This workbook (Workbook 8) is
tion of services and systems for the about economic evaluation. Eco-
treatment of psychoactive substance nomic evaluations involve the iden-
use disorders. The objective of this tification, measurement and valu-
series is to enhance their capacity ation, and then comparison of the
for carrying out evaluation activities. costs (inputs) and benefits (out-
The broader goal of the workbooks comes) of two or more alternative
is to enhance treatment efficiency treatments or activities.
and cost-effectiveness using the in-
Introductory Workbook
Framework Workbook
Foundation Workbooks
Workbook 1: Planning Evaluations
Workbook 2: Implementing Evaluations
Specialised Workbooks
Workbook 3: Needs Assessment Evaluations
Workbook 4: Process Evaluations
Workbook 5: Cost Evaluations
Workbook 6: Client Satisfaction Evaluations
Workbook 7: Outcome Evaluations
Workbook 8: Economic Evaluations
What is an
economic
evaluation?
Economic evaluations involve the identifica- Economic evaluations differ according to
tion, measurement, and valuation, and then their scope and intent. They can have a
comparison of the costs (inputs) and ben- very narrow focus, whereby evaluators are
efits (outcomes) of two or more alternative only concerned about the resource conse-
PSU treatments or activities. quences for their agency. In these evalua-
tions, any new intervention which shifts
In economic evaluations, the costs and con- costs to another agency may be preferred.
sequences of alternative interventions or sce- Alternatively, economic evaluations can
narios are compared to examine the best use examine wider social costs. In these evalu-
of the scarce resources. The specific ques- ations, a new intervention that shifts costs
tion being addressed may be: but does not reduce total costs may not
have good value. Similarly, for health
a comparison of the costs and benefits of agencies, the outcomes of prime impor-
a new intervention to some current thera- tance are likely to be the health of the indi-
peutic approach vidual user. From a societal perspective,
a comparison of the costs and benefits however, the costs of crime and other so-
between treatment and prevention activi- cial effects may be of greatest concern.
ties
Why do
an economic
evaluation?
Economic Using any resource for the treatment of PSU Full economic evaluations are rarely com-
evaluation is disorders means the opportunity to use that pleted. One reason is that economic evalua-
one of the resource for something else is lost. There- tions are resource intensive and typically re-
tools available fore, cost-effectiveness (or value for money quire a high level of research expertise. It is
to help choose spent on treatment services) is of central con- important, prior to undertaking this type of
wisely from a cern in most health care and government sys- study, to determine whether a full economic
range of tems. Economic evaluation is one of the tools evaluation is warranted or required. For some
alternatives available to help choose wisely from a range research questions, answers can be ad-
and implement of alternatives and implement efficient re- dressed through a cost evaluation (Work-
efficient sources. book 5), which is generally less intensive to
resources. complete.
In general, economists prefer the widest
possible societal perspective: general ques- Full economic evaluations should only be
tions about the use of scarce resources and undertaken after an initial analysis to gauge
societal well-being. However, in certain the usefulness of the study. Prospective eco-
cases, policy makers may wish to know nomic analyses are best undertaken along-
the answers to narrower questions, for side other evaluations, particularly outcome
example, restricting the perspective to studies (Workbook 7). In themselves, eco-
health outcomes and health care expendi- nomic components of research need not be
ture, or restricting it to a specific area, for excessively expensive. There is, however,
example, the effects on crime and the crimi- great merit in examining the economic de-
nal justice system. sign from the beginning of a research plan-
ning process as results may affect the overall
design of the study as well as the detail of
data collection.
How to do a
cost evaluation:
general steps
Alternatively, you may want to know whether More general questions, for example, about
a new therapy should be adopted. The pro- the overall level of funding of substance use
viders of substance use service may be in- treatments in a local area, or whether the
terested in the detailed analysis of the costs balance between prevention and treatment
and consequences for their own organisation. funding is bringing the highest health gains,
Funders of services may be interested in the require a broader approach and perspective.
wider implications for health service delivery Answering such broad questions requires
under their jurisdiction. National or state au- data at a more aggregate level and assumes
thorities may be more interested in the soci- that many of the detailed evaluations have
etal perspective. It is important that you as- already been undertaken.
sess the potential consequences of taking any
narrower perspective than that of society.
is usually some attempt to evaluate the new ent outcomes from compliance to mainte-
therapy compared to some form of minimal nance therapies. Glazer and Ereshefsky
intervention. (1996) present a model with antipsychotic
therapy that could be adapted. In their model,
The case example located at the end of this the first step is to identify all the treatments in
workbook presents an evaluation of a rela- current clinical practice, then to identify the
tively new case approach versus usual care possible outcomes of each of the treatment
for people with severe mental illness and PS alternatives (e.g., client will comply or not
dependence. Usual care was defined as comply, client will remain abstinent from PSU
participation in a community-based PSU sup- or begin using PSU again, etc.). The next step
port group (alcoholics/narcotics anonymous). is to establish the estimated probability of
each of the outcome combinations (or path-
Some questions require more complex ways) for each treatment alternative. Ide-
evaluation designs. For example, you may ally, these probabilities should be derived
want to evaluate a system of treatments or from previous outcome evaluations (Work-
some form of stepped care. These types of book 7). Costs for each treatment alterna-
questions lend themselves to a decision-tree tive are compiled separately. Finally, costs
approach. Zarkin and colleagues (1994) cre- for each treatment alternative are compared
ated one structure to consider the impact of by multiplying the costs associated with each
evaluating outcomes and costs over more outcome pathway by the cumulative prob-
than one treatment episode. This approach ability that a client will reach this particular
assumes that a PSU client will have repeated outcome. After repeating this process for
encounters with treatment services through- each of the outcome pathways associated
out his/her lifetime, and helps to identify al- with each treatment alternative, costs can be
ternative PSU policy interventions that might added together to yield the total cost of that
affect outcomes. Considering any one PSU treatment strategy.
clients lifetime history, he/she will be on a
particular branch of a decision tree at any Another approach would be to use a deci-
given time (e.g., A) stopped PSU after initial sion analysis to evaluate a stepped care
treatment vs. B) continued to use PS after programme where failures are entered into
initial treatment and was admitted for further different or progressively more intensive
treatment vs. C) continued to use PS after therapies. The advantage of the decision- tree
initial treatment but refused additional treat- approach is that thought must be given to all
ment. By estimating the proportion of the total the alternative courses. The disadvantage is
client population that may follow each of that data are required on the probabilities of
these paths, you can examine the effect of outcomes at different stages and for evalua-
potential policy changes in terms of numbers tion to be feasible at each of these stages.
of clients affected, costs, and expected out-
comes. For example, you can assess the pro- Given the current scarcity of economic evalu-
portion of your PSU population that is clas- ations, it is important to generate a body of
sified into category C (above), and estimate well-conducted research. However, studies
whether an intervention directed at motivat- such as these need to be resourced at an
ing this category of patients to return to treat- appropriate level. The danger is that with so
ment is as cost effective as directing similar few studies conducted, the results of a study
resources at initial intervention efforts. that is designed to answer a very specific
question, with very selected alternatives be-
Alternative structures could be composed ing considered, may be inappropriately
looking, for example, at the potential differ- generalised.
The advantage If two interventions have the same individual An example of a cost minimisation study is
of the cost health effects, then the one which can be judged an examination of several standard metha-
minimisation as the most value for money will be the inter- done facilities in which client group and
approach is vention which minimises the net costs. It is, how- expected consequences are assumed to be
that the ever, a strong assumption to assume that indi- the same. The figures in the table on the
measurement vidual health effects are the same between two next page illustrate some potential results.
problem is or more alternative treatments (or treatment They are taken from a study by Bradley
reduced to just scenarios if more complex questions are being and colleagues (1994), and it should be
examining posed). It would be an even stronger assump- stressed that this study was only concerned
resource tion to include all other benefits of treatment as with costs and no claims were made about
consequences. equal. The advantage of the cost minimisation the outcomes of the different sites. Of the
However, the approach is that the measurement problem is three sites reported in the table below, Site
assumptions reduced to just examining resource conse- A was hospital based whereas Sites B and
are difficult to quences. However, the assumptions are diffi- C were free standing facilities. In this case,
justify prior to cult to justify prior to any experimental study. Site A is the cost minimising option, be-
any This method, therefore, has only limited appli- cause of the lower average staff costs.
experimental cation within the PSU field.
study.
Illustrative cost minimisation analysis for 3 standard methadone
treatment programmes (1990)
Site A Site B Site C
Contracted services
(laboratory tests, pharmacists, accountants etc) 39,339 237,205 131,645
Cost-effectiveness Analysis
The majority of the published economic struct some cost-effectiveness ratio indi-
evaluations have been cost-effectiveness cating the net costs required for each unit
analyses. In this type of economic evalua- of outcome. For some health care inter-
tion, the effect of treatment is measured in ventions, the natural health unit outcome
a single natural health unit. Costs and other measure may be best reflected by deaths
consequences also are measured in mon- avoided or gains in life years. Most PSU
etary terms in the same way as for cost studies have used some measure of PSU
minimisation analysis. The requirement for rather than a health measure, for example,
an economic study to have a single, prin- the net costs per abstinent day, or per per-
cipal outcome measure is needed to con- centage reduction in PSU.
In this type One down side of this technique is that using The screening costs of applying an alco-
of economic a single measure is that the total effects of hol use questionnaire, 2 minutes, are cal-
evaluation, PSU treatment may not be reflected in any culated to be between 0.8 and 2.40
the effect of one health or PSU variable. Further, many per person, total costs 160 to 480.
treatment is of the effects of treatment on the individual The questionnaire is estimated to suggest
measured in a may have wider impact on the quality of life that 46 people would need a brief inter-
single natural than just that of health. Hence, a narrow uni- vention (36 true positive and 10 false
health unit. dimensional outcome measure used as a negative)
Costs and comparison may fail to measure the full It is estimated that 15 minutes is needed
other impact of the different therapies and lead to to deliver each intervention, with addi-
consequences a misleading conclusion on the relative tional costs of leaflets, etc., giving a cost
also are worth of the therapies under consideration. of each brief intervention of between 8
measured in and 20
monetary terms The choice of outcome measure not only af-
For the 46 people receiving the interven-
in the same fects the validity of the study, but also the
tion, this yields a service cost of between
way as for cost use of study results. PSU quantity measures
368 and 920
minimisation may be preferred by therapists as being the
analysis. only measures relevant to their client group. Total programme costs including the
However, funders of treatment could not use screening are therefore between 528 and
such studies to examine the comparative 1400
worth of expanding PSU treatments vs. ex- The cost per at risk drinker is between
panding vaccination programmes because of 14.6 and 38.9
the lack of a common generic health status Expressing this in terms of effectiveness
measure. evidence yields an average extra costs of
2.4 to 6.5 per unit of reduced alco-
To illustrate the use of cost-effectiveness hol consumption.
studies, consider the following example of the
effectiveness of brief interventions compared Note this example is conducted from the pri-
to a control intervention for those drinking mary care perspective and only a limited
alcohol above a low risk level. In this ex- range of cost and consequences are exam-
ample, both costs and effects are measured ined.
as the excess over the control intervention.
Cost-utility Analysis
Systematic reviews of the effectiveness
evidence suggest that alcohol consump- All resources have an opportunity cost: op-
tion is reduced on average by 20 per cent portunities to do something different with re-
following brief interventions. Assume that sources are lost when resources are com-
in a hypothetical problem- PSU popula- mitted in a certain direction. Within health
tion, this would translate into a reduction care, there is a need to make decisions on
of 6.02 alcohol units per person per the balance of resources, for example, be-
week. (One unit is equal to 8 grams of tween terminal care and prevention interven-
alcohol). tions. Such comparisons, however, require
Based on these results, administrators some common outcome measures that can
want to implement a screening incorporate quantity and quality of life
programme and delivery opportunistic changes. Such measures can be seen as mea-
brief interventions in a primary care set- sures of utility (or value of health) to indi-
ting for 100 men and 100 women. Be- viduals. Economic evaluations using such
fore doing so, they want to understand outcome measures are hence called cost-
the cost-effectiveness. utility studies.
.. this method Drummond et al (1997) suggest this method Note that not all QALYs are calculated in
should be should be used when quality of life is the the same way. Scoring for different ques-
used when important outcome. PSU cost-utility studies tionnaires may be based on different ideas
quality of life might involve the evaluation of social care about what constitutes quality of life. For
is the programmes designed to help individuals example, is regular fainting worse than
important who have been in long-term residential chronic pain? Different quality of life mea-
outcome. programmes. Or, it might be used to com- sures will weight or value these items dif-
pare interventions that have effects both on ferently. Different QALY measures also
the length and quality of life. Finally, there have different health dimensions. The
are those programmes that have a range of EuroQol EQ-5D, for example, has 5 di-
different outcomes arising from interventions mensions: mobility, self-care, usual activ-
and some common measure is required to ity, pain/discomfort and anxiety/depression
make comparisons between them. (Dolan et al., 1995). Another measure, the
WHOQOL-BREF, has four dimensions:
There are a number of different aspects to physical health, psychological, social rela-
constructing and using health utility measures tionships, and environment. Copies of both
in economic evaluations. It is necessary to measures and scoring instructions are lo-
identify, measure, and value the health gains cated in Workbook 1, Appendix 2.
from any extension of life and improved qual-
ity of life. Some treatments may improve both All other costs and resource consequences
aspects, but others may influence only the are measured and valued in a similar way
length of life or the quality of life. Whereas as in all the other types of economic evalu-
cost-effectiveness studies measure the out- ations.
come a particular point in time, for example
one year after treatment ends, cost-utility As an example of this type of study, con-
measures must estimate how long the treat- sider the three treatment programmes de-
ment effects will last. scribed in the previous cost-minimisation
table. If a cost utility study of these three
Most cost utility studies measure quality programmes were completed, it would have
adjusted life years (QALYs) among their been possible to chart both differences in any
participants. A QALY is based on the idea overdose or other mortality while in the
that categorising people merely as alive or programme and the improvement in general
dead (i.e., quantity of life) does not cap- health.
ture adequately multiple states of health, or
quality of life, that exist in individuals lives Using this example, assume that Site C (with
following PSU treatment. QALYs assign the greater proportionate staff input) resulted in
score of 1.000 to a (hypothetical) person each client having large health improvements
who is in a state of perfect health. Then, de- using a standard quality of life measures.
ductions from 1.000 are taken for different Combining the health and reductions in mor-
symptom reports while answering quality of life tality yielded the following average total health
questions. For example, use of a cane may re- gains in the three programmes over a year
duce a persons QALY by.060 (1.000 - programme. No future health gains were
0.060), while wheezing or shortness or breath thought to arise from these programmes, a
may reduce QALY by.257 (1.000 - 0.257). minimum estimate position.
Combining these findings with cost data hence reduced annual cost) yielded the
(assuring the original cost figures were ad- following average cost per QALY figures
justed for the premature mortality (and shown in the table below.
All other costs and consequences are as- of risk or asking individuals to put monetary
sumed to be the same between the sites. In values on different health states using a will-
this example, Site C has a far lower cost per ingness-to-pay approach (see Johannesson
QALY than the other two sites. Given the et al. (1996) for a review of these methods).
different mortality rates, there would be dif-
ferential health gains in future years even if all An illustration of the results gained by will-
existing clients reverted back to former lev- ingness to pay methods can be drawn losses
els of PSU. associated with fatal road accidents in the
United Kingdom. The government agency
Cost-benefit Analysis reviewed the available estimates from all dif-
ferent methods, and choose a value in 1987
In cost-benefit analysis, all individual benefits of 500,000 per life. This compared to a fig-
are measured in monetary terms. This means ure of 283,000 calculated using a foregone
that all costs and consequences are measured earnings method. In 1996 terms, this con-
in the same units. The method is useful when verts to a value of 23,000 for each lost
there are a wide range of diverse outcomes (gained) life year.
associated with the treatments being evalu-
ated. Because the results can be expressed Willingness-to-pay methods may be seen as
in terms of whether the monetary value of an alternative measurement system to that
benefits outweighs the costs, such studies are used for utility measures. The differences
often seen to provide more powerful argu- between them may be in the weighting sys-
ments for implementing programmes (or not) tem used for different groups of the popula-
than other forms of economic evaluation. tion. Utility measures usually have an equity
However, the relevance of any study to de- element built in, with one quality adjusted life
cision-making depends on the alternative year being deemed of equal value to all indi-
options being evaluated and the scope of the viduals. This is not always the case with mon-
evaluation. etary measures as some may be biased to
giving greater weight to those with more in-
In cost-benefit Measuring health gains in monetary terms is come. This method may be appropriate in
analysis, all sometimes viewed as problematic. For ex- health care systems were individuals are re-
individual ample, market values of the value of life, sponsible for paying for health care.
benefits are based on foregone earnings have been
measured in thought to undervalue some groups in soci- French and colleagues (1996) have proposed
monetary ety, particularly older and poorer people. recently a method for estimating the mon-
terms. This method of valuation is now rarely used. etary value of PSU treatments. The method-
Other methods include using market values ology they proposed is a mixture between
using health utility measures and monetary mated from available epidemiological data
valuations. Estimates of quality adjusted life and it is not clear that avoided disease can
years are calculated for different age/gen- be accurately estimated for the range of dif-
der and race cohorts and a dollar value of ferent PSU. Also, by concentrating on
a QALY applied. The average value was avoided disease, the measure may fail to cap-
taken from assuming a value $5 million for ture the full individual benefit of treatment.
the statistical life of an average white male This may be particularly important for treat-
at age 38. In essence, the methodology ments applied to less dependent users.
proposed could be used within either a
cost-utility or cost-benefit analysis. Differ- The fullest study examining both costs and
ent disease outcomes were related to a benefits in monetary terms was conducted
general index of values of different health by Gerstein and colleagues (1994). This
states. For example, compared to a per- was a before and after study without a
fect health value of 1 and death as 0, a control group and therefore can be
moderately severe case of Hepatitis B was criticised on methodological grounds. The
thought to generate a value of 0.96 over 2 study, however, illustrates the size of po-
months duration. This is an interesting low tential gains to a number of agencies. Con-
cost methodology for estimating individual sequences included criminal justice costs,
health gains. It may have useful applica- an estimate of victim losses from crime,
tions for PS injection users who are vul- health care cost and productivity conse-
nerable to a number of different diseases quences. The summary figures are given in
with measurable consequences. It may be below, showing all modalities apart from
possible to provide reasonable estimates methadone discharge resulted in greater
of the avoided cases by treatment. How- benefits than costs for an average episode
ever, these data would need to be esti- of treatment.
productivity costs
Capital
- land
- buildings
- equipment
Running costs
- paid staff
- volunteers
- administrative and managerial costs
- consumables including drugs prescribed and their dispensing costs, toxicology
costs etc.
Leisure time and other costs associated with input to treatment Costs
- pain, distress etc. associated with changing habits, or with process of treatment
4. Productivity costs
It may be The areas that may be included in your analysis it may not always be possible to measure and
relatively easy will depend on your situation. For example, value all these effects, some analysis can be
to measure an evaluation from the health service per- undertaken to check whether they are likely
and value the spective may only concentrate on the cost to to differ among the alternatives being evalu-
provision of providers and any other consequent cost for ated.
the costs of other health agencies. If a wider perspective
treatment. is taken, more variables may be analyses. In A controversial area is whether costs arising
Some other these situations, it is important to avoid double from lost productivity while in treatment
potential counting. For example, individuals may lose should be included in different economic
consequences income while undertaking treatment. Includ- evaluations. For long residential treatments,
pose more ing an allowance for the full cost of lost pro- the estimates of these costs can be consid-
problems. ductivity from the time spent in treatment erable. A sizeable group of substance users
would mean that the lost labour resource in treatment may have been unemployed for
would be erroneously counted twice. It should some time and therefore the valuation of this
also be noted that the list refers only to re- item will depend crucially on whether the
source costs those actions which mean estimates are based on some unadjusted
there is a loss of scarce raw materials, land, value of time, or adjusted for labour market
labour or capital. Many PS users may be in demand conditions i.e. adjusted for the risk
receipt of welfare payments from the state of unemployment.
these are transfers from one group (the
taxpayer) to another group. Changes in A number of questions also arise in applying
transfer payments are not included in eco- values to material resources. For example,
nomic evaluations. However, such changes some resources may be more expensive in
may be of prime interest to state or na- rural areas because of transportation costs
tional governments and may need some whereas others, including buildings, may be
separate analysis. more expensive in urban areas because of
scarcity. It is helpful to present results in re-
It may be relatively easy to measure and value source use terms as well as applying mon-
the provision of the costs of treatment. Some etary values so that individual readers can
other potential consequences pose more relate the results to their own situation.
problems. For example, it is often difficult to
trace the full impact of different treatments More detail on how individual treatments
on other agencies. Alternatively, certain treat- may be costed and further discussion of
ments may be associated with more distress some costing issues are contained in Work-
both to the individual and their family. While book 5.
Reduced use of resources from other social care and welfare services
5. Productivity benefits
Most studies are likely to include a number riods especially of frequently used health ser-
of different individual outcome measures. For vices. This is one of the important challenges
example, many controlled trials may include to researchers.
some measure of substance use, some spe-
cific substance-related outcome measure and As well as the benefits to the health care sys-
a more generic health status instrument. For tem, PSU treatments also are likely to re-
economic evaluations, there may be a need duce the use of other social care and welfare
to value the health benefits and a need to agencies. It should be noted again that these
choose both the method of valuation and the resources relate to services received rather
groups from which values are sought, for than changes to welfare benefits, which are
example, from PS users or the whole popu- transfer payments not resource costs. The
lation. difficulty for evaluators is tracing such changes
in use and finding some means of valuing the
Most existing general health measures are
PSU... focused on health-related quantity and qual-
diverse range of effects that may arise. It
affects a ity of life changes. This makes sense because
would of course be far too costly and time
number of other maximising health gain is a very important
consuming to individually trace all potential
dimensions of objective. PSU, however, affects a number
effects. Some may be excluded from the
life quality of of other dimensions of life quality of both
analysis but the effects of such exclusions has
both users and users and their families. Some PSU-related
to be considered carefully. It is clear, for ex-
their families. outcome measures have attempted to include
ample, that there is a large impact on the costs
to the criminal justice system from treating
some of these dimensions. There is a need
some dependent drug users. It would be im-
to evaluate whether such non-health ben-
portant to consider whether the alternative
efits to PS users can be measured and val-
treatments being evaluated had differential
ued perhaps through willingness to pay or
effects on crime before it was decided to
preference-based measures on total health
exclude them, even if the main focus of the
and non-health-related quality of life. At the
study was the impact on health care services.
current time, there are no such ready-to-use
Some important benefits from successful
economic measures.
treatments may be more and difficult to mea-
The current body of cost-offset and cost sure. For example, an adequate system of
outcome studies suggest that one of the con- services for PS users may well have a
sequences of individuals receiving treatment favourable impact on a community and the
is the reduction in demand for other health environment over and above some of the
services. These potential gains could be size- reductions in direct problems. For example,
able. The ease by which health use can be reductions in crime rates may also produce
measured varies with health care systems. reductions in the fear of crime among com-
Not all Where there is some charging mechanism, munity members. Not all consequences of
consequences even if individuals do not pay directly be- treatments may be beneficial. It is important
of treatments cause of social insurance, there may be to consider possible negative consequences
may be records of all health care use, including the of the alternative interventions being evalu-
beneficial. resource cost of the different treatment epi- ated.
It is important sodes. However, in many countries it may
to consider be necessary to ask individuals about their Finally, there are the benefits that result from
possible use of health care over a period before, dur- gains in productivity. As with productivity
negative ing and after treatment and then use average costs, there is some debate about the inclu-
consequences values of the costs of such use. Cost-offset sion of such effects and if included how they
of the studies using insurance record data can track should be measured. Treatment is likely to
alternative individuals within the plan over considerable improve employment prospects and increase
interventions periods. This would be more difficult to the productivity of those in work. However,
being achieve with self-report data. It is not clear the actual changes will depend on the state
evaluated. how accurate recall would be over long pe- of the local labour market.
as semi-fixed and obviously the extra costs Illustration of incremental costs was given in
of treating an extra client fall until the thera- the study by Bradley et al (1994): see the
pist has a full load. table below. In this example, an extra client
is assumed only to require more
Other costs and benefits also may change consumables, such as methadone. However,
according to the level of activity. It is there- an extra 25 clients would require more staff
fore important to undertake additional and other programme costs, although not
measurement and modelling of costs and ben- extra building space. In this example, Site C
efits to fully understand the processes of differ- has the lowest marginal costs for 1 and 25
ent therapies as they expand or contract. clients.
For some research questions, the whole study approach would be an appropriate way of
could be framed in an incremental way. Many evaluating some stepped programme of care.
policy decisions concern how much each In this case, the additional benefits and costs
programme should be funded rather than a of providing extra units of care against the
choice of funding only one. The incremental alternative of no additional care.
The alternative is to compare costs and out- Programme 2 has more benefits and more
comes directly: costs than programme 1. Again decision
is unclear.
(Costs1 - Costs2 )/ (Outcomes1 - Outcomes2)
With cost-benefit analysis, the benefits and
to give a net figure for the difference be- costs are measured in the same unit, money
tween programme 1 and 2 (if the design of values, and the results may be presented as
the study is comparing two alternatives in the net benefits of the alternatives (benefits -
this way). This is usually calculated using costs) or in terms of cost-benefit ratios (costs/
incremental costs and benefits and can benefits). Ratios are not particularly useful
clearly be adapted for some of the more and can be manipulated because some ben-
complex economic evaluation designs (see efits can be redefined as averted costs and
Drummond et al., 1997). affect the ratio. Ratios do not give any ideas
of the size of the scale of benefits or costs.
With a simple study comparing two alterna- This may be important when comparing
tive programmes, there are four possible re- programmes.
sults:
The results will be more complex if a more
Programme 1 has more benefits and lower complex design is used. Similarly, there may
costs than programme 2 (at all levels of be subsidiary analysis to consider if the de-
implementation). In this case programme sign of the study allows. For example, it may
1 clearly dominates programme 2. be possible to consider whether costs or
benefits vary with severity or other charac-
Programme 2 has more benefits and lower teristics. However, unlike in other areas, there
costs than programme 1 and 2 will domi- are no standard measures for case-mix for
nate 1. substance use services.
Programme 1 has more benefits but also
more costs than programme 2. In this case
the decision is not so simple and may de-
pend on whether incremental figures show
an advantage of one over another.
Put the information from this workbook to 3 Using the information contained in this
use for your own setting. Complete these workbook, choose the study design that
exercises below. is most appropriate for your research
questions and resources.
Remember to use the information from
Workbooks 1 and 2 to help you complete a Cost-minimisation
full evaluation plan. Review that information
now, if you have not already done so. Cost-effectiveness
Cost-utility
1 Decide whether a full economic evalua- Cost-benefit
tion is needed or warranted, given your
research questions and your research re- Partial economic evaluation
sources. Is a partial economic evaluation
(reviewed in this workbook) or a cost
evaluation (Workbook 5) more appro- 4 List programme cost sources that you
priate? want to evaluate. If evaluating services
across agencies, decide the common
measurement(s) you will use. Meet with
2 Decide the scope of your study and the planners from the other agency(ies) to
treatment alternatives that you want to achieve consensus on the evaluation
evaluate. Will you conduct an economic methods.
evaluation within an agency, across sev-
eral agencies, or evaluate wider social We have started the list as an aide for
costs? you. Cross out the sources that do not
apply to your situation, and add others
Within an agency that are not already listed.
Conclusion
and a practical
recommendation
In this workbook, we have outlined the ba- Return to the expected user(s) of the evalu-
sic principles and practices of economic ation with specific recommendations based
evaluations within PSU services and sys- on your results. List your recommenda-
tems. In undertaking economic evaluations, tions, link them logically to your results, and
it is essential that you pay close attention suggest a period for implementation of
to the principles and practices of planning changes. The example below illustrates this
and implementation as outlined in Work- technique.
books 1 and 2. Trade-offs have to be made
as to the rigour with which you collect and Based on the finding that programme A,
analyse information to answer your evalu- compared to programme B, results in 20%
ation questions, and the resources you cost savings yet equivalent quality of life
have available. You must strive to achieve outcomes, we recommend that
the best possible information with the time programme A is adopted on a larger-scale
and resources available to you. You must basis.
carefully document the limitations of your
findings and conclusions. With these prin- Remember, economic evaluations are a
ciples in mind, you will be able to under- critical step to better understanding the day
take practical and useful cost evaluations to day functioning of your PSU services.
within your treatment service or system. It is important to use the information that
economic evaluations provide to redirect
After completing your evaluation, you want treatment services. Through careful exami-
to ensure that your results are put to prac- nation of your results, you can develop
tical use. One way is to report your results helpful recommendations for your
in written form (described in Workbook programme. In this way, you can take im-
2, Step 4). It is equally important, how- portant steps to create a healthy culture
ever, to explore what the results mean for for evaluation within your organisation.
your programme. Do changes need to hap-
pen? If so, what is the best way to accom-
plish this?
References
Bradley, C.J., French, M.T. and Rachal, Gerstein, D.R., Johnson, R.A., Harwood,
J.V. Financing and cost of standard and H.J., Fountain, D., Suter, N. and Malloy, K.
enhanced methadone treatment. Journal Evaluation recovery services: the Califor-
of Substance Abuse Treatment, 1994, nia drug and alcohol treatment assessment
11: 433-442. (CALDATA). California Department of Al-
cohol and Drug Programs: Sacramento,
Dolan, P., Gudex, C., Kind, P. and Will- 1994.
iams, A. A social tariff for the EuroQol:
results from a UK general population Glazer, W.M. and Ereshefsky, L. A
survey. Centre for Health Economics Dis- pharmaeconomic model of outpatient antip-
cussion Paper 138, University of sychotic therapy in revolving door schizo-
York:York, 1995. phrenic patients. Journal of Clinical Psy-
chiatry, 1996, 57(8): 337-345.
Drummond, M.F., OBrien, B., Stoddart,
G.L. & Torrance, G.W. Methods for the Johannesson, M., Jnsson, B., & Karlsson,
Economic Evaluation of Health Care G. Outcome measurement in economic
Programmes. 2nd edition, Oxford Medical evaluation. Health Economics, 1996, 5(4):
Publication: Oxford, 1997. 279-296.
French, M.T., Mauskopf, J.A., Teague, J.L.. Zarkin, G.A., French, M.T., Anderson, D.W.
& Roland, E.J. Estimating the dollar value & Bradley, C.J. A conceptual framework
health outcomes from drug-abuse interven- for the economic evaluation of substance
tions. Medical Care, 1996, 34(9): 890-910. abuse interventions. Evaluation and Pro-
gram Planning, 1994,17(4): 409-414.
Comments about
case example
The following case example presents a modi- that were incurred by participants. Societal
fied cost-effectiveness economic evaluation. costs included psychiatric, medical, legal, and
The evaluation compared a case management family resources. Results indicated that both
approach to a usual care approach for treatments resulted in cost savings. However,
people with severe mental illness and PSU the case management approach also resulted
dependence. In this evaluation, case man- in significantly improved psychiatric symp-
agement included intense individualised as- toms and role functioning relative to usual
sistance and monitoring by a team of profes- care. Evaluators did not provide a cost-ef-
sionals. Usual care was defined as fectiveness ratio, thus departing from stan-
participation in a community-based PSU sup- dard cost-effectiveness evaluation techniques
port group (Alcoholics/Narcotics Anony- as described in this workbook. As a result,
mous). Costs of providing each programme their conclusions had to be based on a gen-
were calculated on a per client basis, and eral overview of cost and outcome data rather
compared against changes in societal costs than a single measure of effectiveness.
Case example of a
economic evaluation
Cost-effectiveness evaluation of
substance misuse interventions
By
The author alone is Teh-wei Hu, Ph.D.
responsible for the
views expressed in
this case example.
Who was asking the ers of public health and other services. The
case management program involved inten-
question(s) and why sive individualised assistance and monitoring
did they want the by a team of clinicians and paraprofession-
als. The clients involved in the case manage-
information? ment program were compared to those in-
volved with Alcoholics Anonymous or
Towards the late 1980s, a California county Narcotics Anonymous (AA/NA), a form of
mental health administrator was very much supportive counselling to help people work
concerned about the rising costs of mental through the 12-step recovery process. Staff
health care and the prevalence of substance met with clients both individually and in groups
misuse. The administrator was searching for and provided an additional three to four hours
a services program which might be able to per week of services in addition to their men-
contain the rising costs of care, and, perhaps tal health services. Staff actively engaged in
at the same time, improve the mental health teaching patients the 12-step recovery ap-
status and functioning of clients. In response, proach, and linked them to existing AA/NA
the county allocated funds for carrying out meetings in the community. Thus, the 12-step
alternative models for treatment. At the same AA/NA approach served as the control
time, evaluation funding from the federal gov- group or as a usual care group.
ernment was received to evaluate the costs
and outcomes of this experiment. Qualified patients were randomly assigned
into the two intervention programs. In the fi-
This case study focuses on the treatment of nal analysis, 39 patients were in the AA/NA
people with severe mental illness and alco- program and 45 patients were in the case
hol and drug abuse in a California county management program. It was anticipated that
(Jerrell & Hu, 1996; Jerrell, Hu, & Ridgeley, the individualised assistance and monitoring
1994; Jerrell & Ridgeley, 1995). A case within the case management program would
management program was developed for this produce superior outcomes for this client
client population since they are very high us- population and be more cost-effective.
clinical public guardian records. Informa- reflect only a shifting of existing resources.
tion regarding average cost per unit of However, they are an important indicator
these services were obtained from the rel- reflecting the government/ taxpayer share
evant departments. of the cost of illness. The amount of trans-
fer payments also serves as a useful out-
Costs incurred by the family in providing come indicator demonstrating treatment
care to these clients were included in the providers success in connecting clients to
cost estimation procedures. This included, entitlement programs that are likely to en-
for example, the actual family expenditures hance overall income level and, therefore,
for treatment, transportation, legal services, quality of life. These data are collected
as well as the time family members spent from client interviews as well as from pub-
with the client in treatment and transpor- lic or private guardian records.
tation. These data were obtained from fam-
ily/ care giver interviews. Market value of After all the public and private service el-
the transportation costs and wage rate of ements related to the evaluation were iden-
services were then used to estimate the tified, we determined a standard unit of
costs to the family. The issue of whether measurement for each type of services and
maintenance costs, such as food, lodging, obtained the unit cost of each service. This
and clothing should be included in this type unit cost was then multiplied by the num-
of analysis is debatable. One approach is ber of units of services and summed to
to treat all maintenance costs as treatment obtain various types of subtotal and total
costs (Rice, Kelman, & Miller 1992). An- costs.
other approach is to treat only a portion
of the total as treatment costs (McGuire et
al., 1987). Maintenance costs incurred
while living with family members or alone How were the
in a house are not usually considered treat-
ment costs because these maintenance ex- data analysed?
penditures are part of daily living expenses,
even of persons who are ill. On the other Cost and outcome data were compiled for
hand, in the context of this study, the costs the 6 months prior to each clients entry
of employing a paid caretaker to assist a into the study, and then for each 6-month
client with basic daily living activities should period that they remained in the study. Cost
be considered as treatment costs. The lat- and outcome variable were found using
ter reasoning was adopted in these analy- statistical computing software. Each cost
ses. In this study, we performed a sepa- and outcome variable was summarised
rate accounting of all these maintenance using the mean, variance (standard devia-
costs to reflect the magnitude of those costs tion), minimum, and maximum value, the
of daily living. These data were collected use of which provides a basic understand-
in client and family/ care giver interviews. ing of these variables and also helps to
check for any possible outliers or unrea-
Finally, to understand the relevant financial sonable values. A number of cross-tabu-
burdens among various sectors of society, lations were constructed with socio-demo-
transfer payments were also recorded in our graphic variables (age, gender, ethnicity,
data set. Transfer payments are from one etc.) to provide a description of the study
party to another (i.e., parents to children, populations between study groups. To
government welfare payments, etc.) that are evaluate the possible differences be-
not accompanied by an exchange of services tween programs, multiple regression
or goods of comparable value. These pay- analysis was used for cost and outcome
ments are not treatment costs because they data analysis.
What did they find out? The findings from these effectiveness mea-
surements indicate that the case manage-
ment program provides some improvement
Psychiatric symptoms include depression, in psychiatric symptoms, life satisfaction
manic episodes, and schizophrenia while measures, and independent living, but no
using drugs and alcohol. Compared to the significant improvement in drug or alcohol
AA/NA program, patients under case man- symptoms.
agement had an overall lower mean value
of schizophrenia symptoms and depres- Tables 1 and 2 provide a summary of men-
sion. There was no difference in drug and tal health costs and average societal costs
alcohol use. These results were obtained for the two interventions. The detailed cat-
after controlling for sociodemographic dif- egories of these costs changes over a 12-
ferences and baseline illness condition us- month period are illustrated. It can be seen
ing multiple regression analysis. Overall life that both programs significantly reduced
satisfaction measures showed that case mental health costs from the baseline pe-
management program patients improved in riod: AA/NA program reduced costs by
terms of their living situation, global satis- 50%, while the case management program
faction with life situation, and mental health reduced costs by 41.2%. Similarly, total
condition, as compared to AA/NA pro- average societal costs were reduced
gram patients. The overall role functioning 46.8% for the AA/NA program, the case
measures show that case management pro- management program reduced costs by
gram patients improved in independent liv- 39.7%.
ing, but were rated lower in extended so-
cial involvement in the community, as When comparing effectiveness measure-
compared to AA/ NA program patients. ments, it seems that case management is
Again, all these findings are based on mul- more cost effective than the AA/NA pro-
tiple regression models, controlling for gram. The AA/NA reduced costs by 10%,
other sociodemographic factors. but had less improvement in patient out-
comes. On the other hand, the case man-
Changes of psychosocial outcome mea- agement program achieved both cost re-
sures scores were measured and com- duction (4%), and improved some of the
pared between the baseline and 12-month psychosocial conditions of participating
period through regression analysis. It was patients.
found that adjustment of family interaction
(SAS) was improved by 0.75 for case Given the nature of multiple outcome mea-
management clients, as compared to AA/ sures in numerous scales, it is very difficult
NA clients. Similarly, the score of Global to provide a meaningful cost-effectiveness
Satisfaction of Life (SLS) for higher for ratio. However, it is clear from this analy-
case management clients by 1.74. Further- sis that the case management program has
more, mental health symptoms (Schizo- not only reduced (or saved) the costs of
phrenia, -1.88; Depression, -2.19; Mania, treatment, but also improved the outcomes
-0.96) were all reduced among case man- of participating clients. In other words, it
agement clients as compared to AA/NA achieved both cost savings and improved
clients. effectiveness.
Table 1: Changes in average mental health service costs per client for two dual
diagnosis treatment programs
Table 2: Comparison of average societal cost per client for 12-month time periods for
two dual diagnosis treatments
Medical
Legal
Family
Derogatis, L.R., & Spenser, P.M. Brief McGuire, T., Dickey, G., Shiveley, E., &
Symptom Inventory: Administration and Strumwasser, I. Differences in resource use
Scoring Procedures Manual. Washington, and cost among facilities treating alcohol, drug
D.C.: Clinical Psychometric Evaluation, abuse, and mental health disorders: Implica-
1982. tions for design of a prospective payment
system. American Journal of Psychiatry,
Hu, T., & Jerrell, J. Cost-effectiveness of 1987, 144:616-20.
alternative approaches in treating severely
mentally ill in California. Schizophrenic Bul- Rice, D., Kelman, S., & Miller L. The eco-
letin,1991, 17: 461-468. nomic burden of mental illness. Hospital and
Community Psychiatry, 1992, 43:1227-
Jerrell, J., Hu, T., & Ridgeley, M. Cost-ef- 32.
fectiveness of substance disorder interven-
tion for people with severe mental illness. The Schooler, N., Weissman, M., & Hogarty, G.
Journal of Mental Health Administration, Social Adjustment Scalefor Schizophrenics.
1994, Summer:83-297. In: W. Hargreaves, C. Atkisson, & J.
Sorenson (Eds.).Resource Material for
Jerrell, J., & Ridgeley, M. (1995). Compara- Community Mental,1979.
tive effectiveness of three approaches to serv-
ing people with severe mental illness and sub- Health Program Evaluators, USHEW
stance abuse disorders. Journal of Nervous Publication No (ADM) 799328 Washing-
and Mental Disorders, 183: 586-96. ton DC: U.S. Government Printing Office,
pp. 203-303.