International Journal of Drug Policy 20 (2009) 475479

Contents lists available at ScienceDirect

International Journal of Drug Policy

journal homepage: www.elsevier.com/locate/drugpo

Commentary

Methods for comparing drug policiesThe utility of composite Drug Harm Indexes
Alison Ritter
Drug Policy Modelling Program, National Drug and Alcohol Research Centre, University of New South Wales, Sydney 2052, Australia

a r t i c l e

i n f o

a b s t r a c t
One of the challenges for drug policy research is being able to compare policy options and outcomes. The development of indexes, such as the UK Drug Harm Index or the UNODC Illicit Drug Index is a way to systematically enable such comparisons. An Index is a single common metric that represents the diverse outcomes or consequences of drug use. An Index may be used for performance monitoring within one country/region over time; to establish societal benet of drug policies as expressed in social costs saved; to compare countries or regions; or for comparative policy analysis. Clarity of purpose is important in how an Index is used. The consequences or outcomes that can be combined into a single Index include health consequences, crime consequences, public amenity, pain and suffering, labour market outcomes, and drug manufacture and trafcking activity. The choice of outcomes for inclusion is driven by the purpose but also often by practical considerations, such as data availability. The weighting of the consequences is an important consideration in translating the outcomes into a common metric. A monetary unit has a number of advantages: it is a unit that can be measured across diverse impacts; it gives implicit weighting of harms; and it is intuitive for policy makers and community. On the other hand, it represents an economic perspective. No one Index will be regarded as suitable and appropriate by every stakeholder and ongoing research effort on Indexes is an important foundational research activity to advance illicit drug policy. 2009 Elsevier B.V. All rights reserved.

Article history: Received 23 June 2008 Received in revised form 28 December 2008 Accepted 26 February 2009

Keywords: Harm Index Measurement Drug policy

Introduction Measuring the success or otherwise of policies is fundamental to continuous social, economic and community improvement. For complex social problems, such as illicit drug use, it is a difcult task. One central reason for difculty is that there is not necessarily any agreement on what the outcomes should be. The second challenge is that drug policy transects the multiple domains of law enforcement, treatment, harm reduction and prevention. Each of these policy interventions impacts on different aspects of the problem, and is measured differently. Thus, any endeavour to measure the success of policy needs to include consideration of multiple domains and impacts. Many elds now have developed a single metric which attempts to encapsulate multiple indicators. The ecological footprint is one example. The ecological footprint attempts to measure the human demand on nature by calculating what would be needed to sustain a population and comparing this to the actual productive land and marine area. As a simple, elegant method it has received both pop-

ular attention (one can calculate ones own ecological footprint) as well as criticism for the oversimplication that such an Index entails. Another example closer to home is the DALY (Disability Adjusted Life Year)a composite measure of the burden of disease that includes both morbidity and mortality. The DALY can be used to compare diseases, compare populations, evaluate priority policy areas and monitor changes over time (Murray, Salomon, Mathers, & Lopez, 2002). The essence of a composite index, such as the ecological footprint or the DALY is that it combines different components to produce a single measure of impact. Drug policy analysis would be substantially enhanced if we could develop an Index that can provide a summative measure of the impact of drug use and associated harms. In this paper, the existing small number of illicit drug Indexes is reviewed, followed by an examination of the purposes of Indexes, the chosen outcome variables for inclusion, and the common metrics that have been employed. The paper concludes with a discussion of how Indexes could be used to advance drug policy. Existing Drug Policy Indexes

Tel.: +61 29385 0236; fax: +61 29385 0222. E-mail address: Alison.ritter@unsw.edu.au. 0955-3959/$ see front matter 2009 Elsevier B.V. All rights reserved. doi:10.1016/j.drugpo.2009.02.012

There are three notable existing indexes: the United Nations Ofce on Drugs and Crime (UNODC) Illicit Drug Index, the UK Drug

476

A. Ritter / International Journal of Drug Policy 20 (2009) 475479

Harm Index, and the Australian Federal Police Drug Harm Index. The UNODC Illicit Drug Index (IDI) has as its primary purpose to compare countries/regions. As stated by UNODC, their aim is to establish a single, standard and comparable measure of a countrys overall drug problem (United Nations Ofce on Drugs and Crime, 2005, p. 166). The UNODC Illicit Drug Index effectively measures the extent of the problem. Production, trafcking and abuse were the three components chosen as the drug indicators for the IDI. These factors were selected due to data availability and also because these factors were assumed to be closely related to the drug problem (United Nations Ofce on Drugs and Crime, 2005). To calculate the production index two methods were implemented. Firstly, plant-based drug production estimates were determined through land surveys. To derive synthetic drug production estimates, data on number of consumers and typical amounts consumed per user, seizures of end products, and seizures of precursors were used. The production estimates were then converted into typical consumption units to reect the differing typical dose between drug type. These consumption units were then weighted by a harm/risk factor (discussed below) to take into account the fact that certain drugs have higher abuse risk and associated harms. The trafcking sub-index resulted from two indicators: reported drug seizures and drug route indicators. Reported drug seizures were compiled from law enforcement data. Similar to the production index, the amounts were converted into typical doses and weighted by the harm factor. The drug route indicators were compiled from data collected as part of the UNODCs annual report questionnaire. Member states were asked questions related to the place of origin, transit and destination of drugs seized on their territory. The abuse sub-index was calculated as the number of users multiplied by average annual dose and then weighted by the harm/risk factor. The harm/risk factor was generated to accommodate the difference in the number and the depth of the harms and risk between drug types. The elements selected to establish the risks and harms for each drug type were: treatment demand, injecting drug use, toxicity, and deaths. The UNODC Index is weighted heavily towards production and trafcking, and as such, regions that produce illicit drugs receive much higher scores than consumer countries. The Index has only been published once, in the 2005 World Drug Report (UNODC, 2005), although work continues on developing the metric (personal communication). The UK Drug Harm Index has a different purpose: to monitor the progress of the UK Drug Strategy over time (MacDonald, Tinsley, Collingwood, Jamieson, & Pudney, 2005). It comprises three parts: drug-related domestic and commercial crime; community problems (community perceptions of drug dealing, drug dealing offences); and health harms (BBV, mortality, overdoses, mental health and behavioural problems, neonatal problems) that collectively cover 19 harms (MacDonald et al., 2005). The Home Ofce acknowledges that the list of harms is limited, but has been pragmatic in choosing harms that are quantiable with existing reliable data. The harms are weighted through analysis of the social costs associated with each harm. The Index was set at an arbitrary value of 100 in the year 1998. Subsequent years have seen a decrease relative to the original 1998 value of 100 (Goodwin, 2007). In the most recent UK Drug Strategy publication, they note a 28.4% reduction in the Drug Harm Index between 2002 and 2005 (HM Government, 2008). There have been a number of critiques of the UK Drug Harm Index (see for example Newcombe, 2006). In unrelated work from the UK, Nutt and colleagues have developed a scale that assesses the harms of different drugs (Nutt, King, Saulsbury, & Blakemore, 2007). The purpose of the scale is to assess the potential harms of individual drugs, enabling them to be ranked against each other. Using Delphi techniques and consul-

tation with experts, judgements were made on three main factors of harms: physical harm, dependence, and social harms. Within physical harm, consideration was given to acute, chronic and intravenous harms. Within dependence, consideration was given to intensity of pleasure, psychological dependence and physical dependence. And within the social harms factor, intoxication, other social harms and health care costs were considered. The method produced a mean harm score for each of 20 different drugs (Nutt et al., 2007). Importantly, while this work by Nutt and colleagues refers to drug harms, it is not a Drug Harm Index in the same way as the others described herein, and will not be further referred to. The Australian Federal Police (AFP) have been developing a Drug Harm Index over a number of years, the latest version of which was published in 2006 (McFadden, 2006). The purpose of the AFP Index is to assess the value to the Australian community of drugs seized by the AFP. The Index represents the dollar value of harm that would have ensued had the seized drugs reached the community (McFadden, 2006, p. 68). An economic cost per kilogram of drug consumed is estimated, through combining prevalence and consumption against social cost estimates. The areas covered by the AFP Harm Index include tangible costs: labour costs, health care, road accidents, crime, resources used in abusive consumption, and intangible costs: loss of life and pain and suffering. Because the source material for the social costs did not separately identify drug classes, various estimates and multipliers were used to divide the social costs between the different drug classes of interest for the AFP. Estimates of consumption were then divided by the social costs to produce a harm per kilogram. In an extension of the AFP work, Moore (2007) estimated the per annum social costs associated with cannabis, cocaine, amphetamine and heroin use. Importantly, he distinguished between non-dependent and dependent users, deriving two estimates for each drug type. The social costs covered health, crime and road accidents (Moore, 2007). The estimate for the social cost associated with one non-dependent cannabis user per annum was $190 (AUD) and for a dependent cannabis user $11,000 per annum. The substantial difference in social cost between a non-dependent and a dependent user was replicated for heroin: $2000 (AUD) per annum for a non-dependent heroin user and $105,000 (AUD) for a dependent user. Social costs per annum for cocaine and amphetamine were also derived. Sensitivity analyses (95% condence intervals) revealed that the plausible range for the estimates for dependent users of cannabis was between $6998 and $17,437 social cost per annum; for cocaine between $12,107 and $24,548 social cost per annum; for opiates between $55,330 and $115,222; and for amphetamines between $18,258 and $48,757 (see Moore, 2007 for details). While the number of published Indexes has been small, there appears to be substantial interest in the development of Indexes. For example, New Zealand has recently developed an Illegal Drug Harm Index (Slack et al., 2008); and the Victoria Police in Australia are developing a Drug Harm Index to assist operational police to direct resources most effectively. The purpose of an Index The purposes of an Index can be categorised as: performance monitoring within one country/region over time (for example the UK DHI); establishing social costs saved (for example the AFP DHI); comparisons between countries or regions (for example the UNODC IDI); and policy analysis (for example Moores work). The purpose of an Index is highly relevant to the outcomes to be compiled; and the unit of measurement. Clarity of purpose assists in understanding the chosen methods. If the purpose of an Index is performance monitoring, then the outcomes to be included will be

A. Ritter / International Journal of Drug Policy 20 (2009) 475479

477

different from one where the purpose is to demonstrate returns on investment. The intended usage or purpose of an Index also relates to how it is constructed. For some purposes, the distinction between types of drug users will be vital. For example, given the substantial heterogeneity in drug use and associated harms, consideration should be given to whether drug users are divided into recreational and dependent users, or treated as a homogenous group. Moores (2007) research demonstrated that the Index associated with non-dependent use is vastly different from that of dependent use (as measured by social cost). If an Index is used to compare policy responses targeted towards dependent users versus those targeted towards non-dependent/occasional users, such differentiation within the Index will be important. A second consideration is whether an Index distinguishes drug typecannabis from opiates and so on. If the purpose is to monitor overall drug problems across a region, then such differentiation may not be important. However in combining all drugs into one Index, consideration needs to be given to a method for weighting the harms associated with different drugs. The UNODC Index resolved this by applying a harm/risk factor for each drug type and creating a hypothetical reference drug against which other drugs were then benchmarked (UNODC, 2005). If the purpose is to compare policy responses then distinctions between drug types are likely to be important. Moores (2007) approach was to develop an Index for each drug class. Consequences or outcomes to be included in an Index Outcomes from effective drug policy may be dened as a reduction in the total number of users; reductions in the amount of use per user; reductions in harm or consequences of use; and/or reductions in the availability of drugs. There is no consensus about which outcomes are the most important. The development of an Index requires explicit consideration and selection of the outcomes to be included. Reductions in drug use (such as the numbers of people using drugs, or the amounts consumed) are obvious measures of policy impact. However some interventions do not directly measure these outcomes. Notably the outcomes from law enforcement interventions are not measured in use reduction terms but in arrests, seizures and crop destruction. Likewise for prevention policies, the outcome measure is not reduction in prevalence or consumption but deferred use reduction in the future. In addition, a focus on use reduction (prevalence and consumption) does not accommodate reductions in the harms associated with drug use, such as reduced criminal activity, or lives saved. Drug policy has powerful effects on the harms associated with drug use and we would want to include those harms in a measure of policy outcome. An Index focussed solely on use (prevalence, quantity consumed) would not enable inclusion of harms. In addition, the relationship between use (prevalence or quantity) is not linear with harm, and varies by drug type and using context. Thus use is not a proxy for harm. Instead of focussing on the use of drugs per se, another option is to focus on the consequences or harms arising from that use. In terms of community concern and nancial burden, it is the harmful consequences associated with drug use that should be ameliorated. The consequences from drug use can be categorised into health-related, crime-related, community-related, labour-market and productivity, and pain and suffering. There are other potential advantages to using consequences as a measure of policy outcome: for example it enables distinction between the different bearers of the harm (MacCoun & Reuter, 2001). In their classic text Drug War Heresies, MacCoun and Reuter provide a taxonomy of drug-related harms. Using categories of harm (health, social and economic functioning, safety and public order, criminal justice), they distinguish

between the bearers of the harm and the primary source of the harm. The bearers of the harm include: users, dealers, intimates, employers, neighbourhood and society. The sources of harm can be from use itself, from illegal status and/or from enforcement. Measuring use or measuring the consequences of use are not mutually exclusive. Consequences include, perforce, the number of individuals using the drug and a proxy of quantity of use. An inclusive approach would therefore cover prevalence and consumption as well as the consequences associated with use. In the process of considering the outcomes that are to be combined, one must also consider the perspective taken. If the perspective of government is considered, then only those outcomes of direct concern to government would be included. That is, those outcomes which governments have control over, and can manipulate/alter. Alternately, a societal perspective could be taken resulting in the inclusion of many outcomes that reside largely outside the control of direct government action such as private consequences, for example insurance premiums. There is also the perspective of the drug userthe outcomes of drug policy for a drug user are likely to include substantially different considerations from those discussed thus far. For example, an Index that does not include the benets associated with drug use to offset the negative consequences would be seen as inadequate if one were taking a user perspective. As will be apparent from the above, the choice of outcome variables or consequences to be included in an Index is not value neutral. The UNODC Index concentrates on drug manufacturing and trafcking as these variables are considered representative of a countrys drug problem by the UNODC (United Nations Ofce on Drugs and Crime, 2005, p. 166). Others would argue that manufacturing and trafcking are not the primary indicators of a drug problem. Another value-driven consideration is whether outcome variables should be weighted positively or negatively. Treatment demand is one such variable. The UNODC Index uses treatment demand as a measure of the extent of problematic use; but it could equally reect positive policy responses. There are a limited number of outcomes, or consequences, measured in the existing Indexes. These include: health consequences; crime consequences; labour market impacts (productivity and welfare); community outcomes such as social cohesion, public amenity, fear of crime; manufacture and trafcking activity; and pain and suffering. Unfortunately, there are limited available data covering each of these outcomes. Signicant foundational research would be required to include all the outcomes of interest. Where good data exist, such as in hospital inpatient services or in road accidents, the data are used extensively. Where limited data exist, such as in the area of public amenity, the domain is not included. This point is made overt in the UK Index (MacDonald et al., 2005). Likewise, Moore (2007) acknowledges the potential signicance of labour market impacts but does not include them due to the absence of available data. On the other hand, road trafc accidents are included because good data exist, but these may represent only very marginal impacts relative to labour market impacts. To date, it appears that a pragmatic response has been taken in Index development that is to proceed with known data, acknowledging the gaps. However, there may be some who would argue that it would be preferable to wait for more fulsome data before proceeding with an overall Index. Different metrics Having considered the purpose, the outcomes to be included and the perspective taken, it remains to determine the metric for the Index. The outcomes need to be summed/pooled using a common metric. The two metrics used to date are a single number and a monetary unit. In other elds the metric has been global hectares

478

A. Ritter / International Journal of Drug Policy 20 (2009) 475479

(ecological footprint) and years of life (DALY). We should not rule out alternate metrics in the drugs eld. A numeric value for an Index is derived by summing the outcomes. The UNODC Illicit Drug Index and the UK Drug Harm Index both use a number as the common metric. A simple count mechanism to derive the number is problematic in that it does not incorporate any weighting of outcomes. That is, it counts one burglary similarly to one death. Therefore the outcomes need to be weighted by some means: a harmfulness weighting, the monetary value they represent, or by some community valuing process. Once weighted, the number metric is likely to require some transformation, depending upon the purpose of the Index. The UK DHI, with the purpose of performance monitoring, set the number at 100 for the reference year, and then subsequent years are compared as an increase or decrease relative to 100. The alternate option is for the common metric to be dollarsthe monetary value of the outcomes. The monetary value for each outcome represents the nancial burden to society of drug use. There are a number of advantages to using monetary value as the common metric. It enables the different outcomes to be combined readily when they are all converted into the same metric. The social cost data are available; at least for health and crime outcomes. The use of monetary value as the common metric deals with the problem of assigning weights to the outcomes. Effectively by using the social cost of each consequence, they are automatically weighted by the economic value they represent. However, some people will argue that social cost is not the most important way of valuing and comparing consequences (burglary is worse than hospitalisation; what value a life?). Finally, the advantage of using social costs as the common metric is that it is one that is readily understood and makes intuitive sense for policy analyses: it represents the potential cost savings that could accrue under the policy scenario being examined. It has an implicit meaning, as compared to a number. Using Indexes in policy analysis Those indexes developed to monitor change over time can be used in a reasonably straightforward manner. One concern is the interpretation of changes in an Indexa rise in some harms and a fall in others may see no change in the Index itself. Likewise, drug substitution (decreases in use of one drug and increases in use of another) may not be captured by a single Index. In both these cases, the Index provides a net analysis of drug policy, but does not enable a nuanced interpretation of changes in the Index. This reinforces the importance of stakeholder understanding of the purposes and limitations associated with an Index. Indeed, it is the power and simplicity of using a single Index that is also its Achilles heel. An Index can be used to compare different policy options. The Drug Policy Modelling Program (Ritter et al., 2007) uses models or simulations as a primary method to evaluate policy options. Within these models, an Index can be applied to compare outcomes from the modelled scenarios. Two examples, using Moores (2007) gures, are provided. An Index can be used to estimate the effect of a new cannabis treatment policy that provides compulsory treatment to all cannabis users detected by police. The effect referred to here is any reduction in social cost associated with the intervention. Given non-dependent cannabis users bear a social cost of approximately $190 AUD per annum and dependent users a social cost per annum of approximately $11,000, we can estimate the effect of such a policy under a model that assumes both various ratios of dependent to non-dependent users detected; and various ratios of purported treatment success. The social cost savings under these various scenarios can be compared to the cost associated with providing the new intervention.

A second example compares a new treatment for methamphetamine dependence with the effect of law enforcement directed at seizing methamphetamine supplies. Moores (2007) working estimates provide a social cost per annum estimate for methamphetamine dependent people of approximately $45,000. If we assume that half of the dependent people receive the intervention, and the success rate is 40%, we can estimate potential costs savings. This can then be compared to the law enforcement intervention, where Moores (2007) estimate suggests that the social cost per gram of methamphetamine is $6500 per annum. Thus if police seize 100 g from a clandestine laboratory, the social cost savings would be $65,000 per annum. We can then compare $65,000 with the estimated treatment savings. In both examples it should be noted that these are not costeffectiveness analyses per se, rather they provide the opportunity to test scenarios under various conditions (such as success rate, population target) and draw broad conclusions about policy impact, for example in relation to estimating break even points, or marginal gains with increased success rates. It appears that an Index can be usefully applied to examine policy outcomes. However, there are a number of caveats. Careful assessment of what an Index includes and excludes by way of outcomes would be required for each particular analysis undertaken. In addition, clear and detailed exposition in the Index is vital for its successful application in policy research. There is a likelihood that an Index is used for purposes other than intended. In a eld such as illicit drug policy that carries strong emotional valence and is often value-laden, it is quite conceivable that an Index designed for one purpose is used for an alternate purpose that may draw fundamentally different conclusions. While this is probably unavoidable, it behoves researchers to ensure clear exposition and detailed documentation of limitations to usage. In my opinion, the benets of a set of transparent, public domain common metrics that can be used for policy analysis outweigh the risks of misuse. Conclusions Drug policy analysis whether concerned with monitoring progress over time, comparisons between regions or policy analysis that compares interventions all benet from a common metrican Index that quanties, weighs and combines all the consequences of drug use into a single common metric. The purpose of an Index will determine the methodological approach. Agreement on the outcomes to be included use and consequences is required. Where there is agreement on the variables to be included, pragmatic constraints on available data may compromise the Index. The metric needs to be decided upon. Numeric summation of the various outcomes, especially where they cross law enforcement, treatment, prevention and harm reduction arenas, is problematic in the absence of some form of weighting. One option is the application of the social costs for each outcome. The advantage of social cost as a common metric lies in its implicit meaning to both decision makers and the community alike. However, it does represent an economic perspective of the value of drug consequences. The drive to develop an Index that can sum across domains is apparent. Various groups have developed or are working on Indexes. There is clearly interest and commitment to such an approach, although the diverse purposes should be noted. There will never be a perfect Indexand the exercise itself will be criticised by some who see that data do not exist, and what data do exist must be manipulated and distorted to achieve a solitary Index. Those who think about details and work at a microlevel will not be satised with the assumptions and methods that are required to develop an Index. Despite these criticisms, we believe it is worthwhile. A transparent, well-documented Index that can be

A. Ritter / International Journal of Drug Policy 20 (2009) 475479

479

used to broadly compare countries or regions; monitor progress; direct resources; or compare policy options is a valuable step forward. Acknowledgements This work forms part of the Drug Policy Modelling Program (DPMP) and was funded by the Colonial Foundation Trust. The author is funded by an NHMRC Career Development Award. References
Goodwin, A. (2007). Measuring the Harm from Illegal Drugs: The Drug Harm Index 2005. Home Ofce. On line report, Accessed 3/5/08. http://www.homeofce. gov.uk/rds/pdfs07/rdsolr2207.pdf. HM Government. (2008). Drugs: Protecting Families and Communities: The 2008 drug strategy. Retrieved 29/2/08, from http://drugs.homeofce.gov.uk/publicationsearch/drug-strategy/drug-strategy-2008-2018?view=Binary. MacCoun, R., & Reuter, P. (2001). Drug War Heresies. UK: Cambridge University Press.

MacDonald, Z., Tinsley, L., Collingwood, J., Jamieson, P., & Pudney, S. (2005). Measuring the Harm from Illegal Drugs using the Drug Harm Index. Retrieved 6/2/07, from http://www.homeofce.gov.uk/rds/pdfs05/rdsolr2405.pdf. McFadden, M. (2006). The Australian Federal Police Drug Harm Index: A new methodology for quantifying success in combating drug use. Australian Journal of Public Administration, 65(4), 6881. Moore, T. (2007). Monograph No. 14: Working estimates of the social costs per gram and per user for cannabis, cocaine, opiates and amphetamines. Sydney: National Drug and Alcohol Research Centre. Murray, C. J. L., Salomon, J. A., Mathers, C. D., & Lopez, A. D. (2002). Summary measures of population health: Concepts, ethics, measurement and applications. Geneva: World Health Organisation. Newcombe, R. (2006). A review of the UK Drug Strategy PSA targets and Drug Harm Index. Manchester: Lifeline. Nutt, D., King, L. A., Saulsbury, W., & Blakemore, C. (2007). Development of a rational scale to assess the harm of drugs of potential misuse. The Lancet, 369(March 24), 10471053. Ritter, A., Bammer, G., Hamilton, M., Mazerolle, L., & The DPMP Team. (2007). Effective drug policy: A new approach demonstrated in the Drug Policy Modelling Program. Drug and Alcohol Review, 26(3), 265271. Slack, A., ODea, D., Sheerin, I., Norman, D., Wu, J., & Nana, G. (2008). New Zealand Drug Harm Index: Report to the New Zealand Police. Wellington: BERL House. United Nations Ofce on Drugs and Crime (UNODC). (2005). World Drug Report 2005: Volumes I and II. UNODC.