Natasha

Tusikov PhD Candidate, Australian National University, Email: natasha.tusikov@anu.edu.au

Measuring Organised Crime-Related Harms: Exploring Five Policing Methods

A final version of this article appears in Crime Law and Social Change, online publication November 2011.

Abstract Many law enforcement agencies around the world have adapted risk assessment methodology to analyse organised crime. These assessments, which are intended to provide law enforcement management with rigourous analysis to enable rational and objective decision-making processes, are an integral part of intelligence-led policing. Despite the prevalence of these assessments, as the assessments and their methodologies are often tightly restricted within the law enforcement community, it is often unclear how law enforcement defines, analyses and makes decisions about organised crime. While the use of risk assessment methodology to analyse organised crime in policing is generally under-evaluated, critics point to serious methodological weaknesses. Another aspect that is less explored in the scholarly literature is how law enforcement conceptualises and measures the impact or harm from organised crime and uses this analysis to inform priority-setting processes. This article explores how law enforcement assess organised crime-related harm by examining five policing methodsone each from Australia and the Netherlands and three from the United Kingdom. The article finds that the methods have significant shortcomings: the main concepts are generally ill-defined and the operationalisation of these concepts is problematic. More importantly, the problems evident in the harm methods raise several critical questions, specifically whether measuring organised crime-related harms is empirically feasible and, if so, can be undertaken in a manner that meaningfully informs law enforcements decision-making and limits undue political interference. Key words: organised crime, harm, risk, priority setting Introduction In the last decade, many law enforcement agencies around the world have adopted risk assessment methodologies to assess organised crime, including in Australia, the Netherlands and the United Kingdom (see [44]; [51]). These assessments, adapted from business and scientific risk literature (see [5]) conceptualise organised crime as a probabilistic risk that can be assessed, mapped and managed in structured analyses that determine the relative risk from specific criminal networks. The widespread but largely under-evaluated adoption of risk assessment methodology (exceptions include [47]; [44]; [16]; [51]) is a part of policing management strategies that emphasise proactive, preventive, and crime-reduction approaches, such as intelligence-led policing. Intelligence-led policing is a widely adopted but critically under- theorised and relatively unproven management strategy (e.g., [15]). The intelligence-led approach is predicated on sound intelligence practices and processes and is intended to inform a rational and objective decision-making process (see [31]). Law enforcements conceptualisation of organised crime as a risk to be governed is also shaped by the risk society discourse that sees police as key providers and communicators of risk knowledge intended to reduce societys collective insecurity ([10]). The study of organised crime-related consequences or harms has been traditionally over- shadowed by a focus on threat ([47]). As Maguire ([27]: 328) argues, how the relative seriousness of crimes is determined and by whom are critical questions in relation to law enforcement priority-setting processes, particularly as only a limited number of initiatives can

1
Electronic copy available at: http://ssrn.com/abstract=1978182

Natasha Tusikov PhD Candidate, Australian National University, Email: natasha.tusikov@anu.edu.au

be undertaken at any one time. Critics, however, argue that law enforcement often lacks an empirical basis for its findings, resulting in weak priority-setting processes and the arbitrary designation of crime priorities (see [44]). As risk assessments on organised crime and their related methodologies are generally tightly restricted to the law enforcement community, it is often unclear how law enforcement defines, assesses and makes decisions about organised crime, leading to criticisms of methodological opacity and unnecessary secrecy (see [45]). While an assessments findings must remain classified, the underlying methodology should be as rigourous as possible, meaning that law enforcement should, where possible, make public its methodology for a critical review. This entails what Klerks ([22]: 97) calls a delicate balance between democratic transparency and the need to protect information. This article assesses five methods used to measure organised crime-related harmsone from Australia and the Netherlands, and two from the United Kingdom. It examines the reliability and validity of each methods operationalisation of harm, building upon recent evaluations of assessments on organised crime (particularly [16]; [51]). This article also builds upon research that explores the practical application of risk assessment methodology to organised crime (particularly [2]; [46]; [33]; [47]; [1]; [8]) and seeks to develop further risk assessment methodology. This article is organised into three sections. The first section situates risk assessment methodologies as practiced by law enforcement within the broader risk and policing literature. Next, the article examines the concept of harm in risk assessments and explores the theoretical and practical challenges involved in the conceptualisation and measurement of consequences from organised crime. The final section outlines and then assesses five law enforcement methods used to assess the harm caused by organised crime. Risk Assessment Methodology Risk has become a defining word of the 21st century ([14]). In the risk society characterised by a pervasive and consuming sense of insecurity and fear, there is a constant need for information about risks. Ericson and Haggerty ([10]: 6) note that risk communication and management systems proliferate as fear builds upon itself and the production of security becomes central to public and private institutions. Police hold a central but not exclusive role in this society as key providers and communicators of risk knowledge ([10]: 5). Crimes are conceptualised as calculable and probabilistic events with unequal social distribution with some individuals facing higher risks for particular events than others. Police are responsible for assessing and mapping the actors and spaces that pose hazards to the public in an attempt to make the world more secure by ever more perfect knowledge of risk ([10]: 295). The emphasis on technology, rationality and probabilistic calculations of risks is described as the scientification of policing ([11]). The risk discourse underlies policing management strategies that emphasise proactive, preventive and crime-reduction approaches, such as problem-oriented policing, community policing and intelligence-led policing. These strategies share, as Maguire ([27]: 316, italics in original) notes, a strategic, future-oriented and targeted approach to crime control, focusing upon the identification, analysis and 'management' of persisting and developing 'problems' or 'risks' (which may be particular people, activities or areas). The management of persistent,

2
Electronic copy available at: http://ssrn.com/abstract=1978182

Natasha Tusikov PhD Candidate, Australian National University, Email: natasha.tusikov@anu.edu.au

serious, and organised crime-related risks is at the core of intelligence-led policing which was created, in part, as a response to dissatisfaction with traditional policing practices and concern over growing levels of criminality (including a perceived increase in organised crime) ([31]). This policing strategy also emphasises that policing resources are finite and must be allocated toward high-risk activities and actors. Organised crime is therefore understood as a risk that can be differentiated from other crime types. Actors and illicit activities can be assessed, mapped, ranked and targeted according to the levels of threat and risk posed. The role of police as knowledge workers ([10]) becomes in intelligence-led policing a relatively specialised function that is performed, in theory, by trained intelligence experts in specialised roles. At the core of the intelligence function is the (largely civilian) analytical component. The emphasis on rationality within intelligence-led policing requires a transition from the analytical function that was relatively intuitive and unstructured to a more formalised approach based on specific technological tools and methodologies that purports to provide a more objective assessment of crime ([20]; [11]). Despite decades of calling for the professionalisation of intelligence analysis, practitioners and academics alike debate whether it is (or is becoming) a discipline (e.g., [12]). Critics assert that the research methods used in intelligence analysis are not rigourous but rather fairly plastic and designed for working with dirty data in manufacturing an intelligence product ([20]: 50). Police knowledge about organised crime risks is communicated, in part, through risk assessments, a concept borrowed from business and scientific literature (see [5]). Definitions of threat and risk vary according to the assessors needs; however, the fundamental components of a risk assessment model are a realistic recognition of hazard, a scale of risk and a defined protocol ([5]: 182). In the context of organised crime, a risk assessment would involve an assessment of the nature and magnitude of the threat (i.e., actors posing risk), the probability and frequency of a threat occurring (i.e., achieving specific criminal aims), and the determination of the nature and scale of the impact (i.e., harm), intentional or inadvertent, should the threat be realised (see [46]; [5]). Law enforcements risk assessments are intended to direct rational and evidence-based decision-making by allocating resources toward high-risk issues (see [46]; [49]; [1]). The transition of police from front-line responders to knowledge workers who are expected to deliver a more rational, objective and cost-effective evaluation and targeting of particular crime risks is problematic ([20]; see also [15]). Despite lofty claims by law enforcement of fundamental shifts in organisational and operational strategies from reactive to proactive approaches to crime control, researchers question the actual level of change that has occurred ([18]; [27]; [32]). Skepticism of law enforcements claims of analytical objectivity and rationality ([35]; [20]) is evident in critiques of law enforcement and governmental assessments of organised crime. These critiques point to serious problems in both confidential and public assessments of organised crime, particularly weak methodology (see, for example, for the Netherlands [47]; [22]; [51]; for Germany [48]; for Europol [45]; for Canada [51]; and for the United Kingdom and Belgium [16]; [47]). Scholars question the rigour and validity of these assessments, cautioning that findings may reflect idiosyncratic law enforcement interests rather than an objective measure of organised crime ([9]), or, more bluntly, warning that the assessments are more ritual incantation than a knowledge-based process ([45]: 261).

Natasha Tusikov PhD Candidate, Australian National University, Email: natasha.tusikov@anu.edu.au

Despite the interest in both the academic and law enforcement communities in applying risk assessment methodology to organised crime, Dorn ([7]: 292) rightly argues that efforts to operationalise seriousness and harm are in their infancy. There has also been relatively little comparative analysis of how law enforcement agencies conceptualise and measure harm, or evaluations of the techniques used to measure harm (notable exceptions are [50]; [9]; [17]; [54]). This gap is partly explained by weak or undeveloped methodologies (see [47]; [22]; [51]; [16]) or unnecessarily secretive processes (see [45]). As well, academics and practitioners have traditionally accorded greater attention to the definition and measurement of threat over harm ([47]). However, where the concept of harm is incorporated into risk methodology, as Zoutendijk ([51]) notes, few assessments explicitly define what is meant by the concept or indicate how it is measured. Moreover, these assessments generally lack a structured approach to harm ([47]; [16]; [51]). This paper first describes harm and then discusses key theoretical and practical challenges related to measuring organised crime-related harms before exploring four case studies. Defining Harm Organised crime is one of the most contested terms in academic criminology ([36]: 490), a phantom ([43]) characterised by a diversity of activities, participants, criminal capabilities and different organising structures. Measuring organised crime first requires, as Von Lampe ([49]: 85) argues, conceptual clarity and certainty, a considerable challenge for a phenomenon that continues to provoke debate regarding its constituent elements (see [1]; Von Lampes website).1 Organised crime is alternately described with a focus on organisation (actor- and network-level analysis), or market-level dynamics (both licit and illicit), or a combination of both, and each conceptualisation results in a different focal point for analysis. When the analytical focus is on criminal organisations, damage from organised criminality is understood as negative effects that can be directly attributed to specific criminal actors and networks. Hamilton-Smith and Mackenzie ([16]) describe a framework in which risk is determined based on scores of criminal networks attributes, calculations based on networks criminal activities, and a combination of both. Harm can be measured at the criminal network- level using, for example, criminal-attribute scales (see Klerks 2000 in [46]), criminal offence- scales or statute-penalty scales. When the analytical focus is on what can be termed sector- or market-related harms, including consequences to the legal market (see [1]), harm is generally understood in broad (potentially overlapping) categories, which cover a range of damages that are not attributable to specific criminal networks, with effects ranging from physical, economic, psychological, political, social, community, intellectual to environmental (e.g., [26]; [2]; [46]). At the sector level, these effects are most commonly qualitatively assessed, though some research estimates crime costs using these categories (e.g., [30]). Costs related to organised crime are generally divided into tangible effects that have clearly identifiable victims and concrete damages (e.g., medical bills) or intangible effects, such as fear of crime or the loss of quality of life (e.g., [6]). Network-level (i.e., micro) and sector-level (i.e., macro) approaches can also be combined to determine the
1

See Klaus Von Lampes Organised Crime Research website at http://www.organisedcrime.de/OCDEF1.htm.

Natasha Tusikov PhD Candidate, Australian National University, Email: natasha.tusikov@anu.edu.au

relative level of harm from specific criminal networks and, more broadly, assess organised crime-related harms that cannot be attributed to specific criminal networks. Challenges in the Analysis of Organised Crime-Related Harms The analysis of organised crime-related harms suffers from the same vulnerabilities that trouble all organised crime research: a dearth of standardised data on organised crime and victimisation, a lack of common (or complementary) methodological approaches and records- management systems, restrictive privacy laws, and an often-secretive policing culture (see [35]). Research challenges related to organised crime-related harms are not further discussed in this article as they are effectively covered elsewhere (see particularly [28]; [21]). This section instead explores the disjuncture between law enforcements conceptualisation of organised crime-related harms and critical questions raised in the academic harm literature. The concept of harm varies according to how one defines and categorises the behaviour that results in the damage. When harm is conceptualised within a legalist framework, the focus is upon violations of criminal laws, whereas socio-legal definitions of crime include civil or regulatory violations, particularly in reference to corporate crime (see [50]). A shift from a legalist focus to a socio-legal framework would enable law enforcement to assess organised crime-related harms more comprehensively and also likely provide more attention to corporate harms. Some researchers recommend a departure from the legal framework altogether to enable analysis of activities that are legal but damaging, such as the use of drift-nets in commercial fisheries (see [50]). However, such a move would be outside law enforcements mandate and the ostensible purpose of its risk assessments. Law enforcement risk assessments, not surprisingly, also tend to exclude damage inadvertent and otherwiseresulting from the states interventions against organised crime. Such collateral damage is well known, particularly in relation to anti-drug measures, for example, research indicates that law enforcement interventions in illicit drug markets could unintentionally increase violence ([40]: 6). The relationship between counter-measures against organised crime and collateral damage should be assessed, especially if these measures generate greater harms than the crime targeted ([8]: 35) or if law enforcement incorrectly attributes harm caused by its counter-measures as solely caused by organised crime. This presupposes, as Von Lampe ([48]) argues, that law enforcement can critically and effectively evaluate its practices. It also presumes that law enforcement has both the will and capacity to revise or halt harmful counter- measures. Law enforcement risk assessments generally reflect a normative approach to organised crime- related harms which recognises little distinction between harm from conflictual crimes (mala in se) versus consensual crimes (mala prohibita). Consequences from conflictual crimes, such as fraud or theft, generally have clearly identifiable victims and more concrete damages. In contrast, harms from consensual crimes (e.g., illicit drugs) are difficult to determine, as the goods are voluntarily demanded and supplied but can result in negative socio-economic effects (see [4]). Some negative effects occur irrespective of the activitys legality (e.g., overdoses or lost productivity due to alcohol), while others (e.g., illicit market-related violence) would decline significantly were the activity legalised. When an activity is legally permitted in some settings and not others, analysis must distinguish whether the negative effects are related to organised crime or are intrinsic to the activity ([26]: 46). Analysis must consider which consequences related to consensual crimes should be measured, if any, and how these should be calculated.

Natasha Tusikov PhD Candidate, Australian National University, Email: natasha.tusikov@anu.edu.au

Law enforcement risk assessments rarely include a consideration of positive effects from organised criminality as there appears an implicit assumption that all organised crime-related activities are non-consensual. Many scholars, in contrast, note that organised crime can provide the rule of law where government is absent and inject wealth into economies, both developing (e.g. [37]) and developed (e.g., [42]; [41]). On balance, however, there appears to be an inverse correlation between organised crime prevalence and a states economic wealth, as Van Dijk ([41]) determined in his composite organised crime index. Harm analysis should consider, as Maltz ([26]: 43) argues, both the economic losses to victims and economic gains to criminal organisations, with a recognition that the losses and gains are not necessarily complementary (e.g., property may be destroyed). Determining the net cost of organised crime may provide a more accurate accounting of harms, although this costing is often politically untenable and problematic to calculate with any accuracy. An important step would involve a shift by law enforcement from an assumption of harm in all organised crime-related activities to a determination of damage. Discussion of the above issues invariably raises more questions than answers and demonstrates a need for a critical discussion of the socio-historical construction of harm and assumptions underlying current understandings of the concept. If the analysis of organised crime-related harms is to become more critical, comprehensive and rigourous, a serious examination of such issues is necessary. Law enforcement should make clear how it defines and measures organised crime-related harms. Five Approaches to Assessing Harm This section examines harm methods constructed by five policing agencies: the Metropolitan Police Service (MPS) in the United Kingdom, National Policing Improvement Agency (NPIA) in the United Kingdom, the Association of Chief Police Officers (ACPO) in the United Kingdom, Australian Crime Commission (ACC), and the Netherlands Police Agency (in Dutch, KLPD).2 There has been little public discussion of these methods as they are part of classified assessments, except the 2008 Dutch assessment that was publicly released. Hamilton-Smith and Mackenzie ([16]) provide a valuable critique of two methods: the 2006-version of the MPS methodology (this article examines the 2008 amended version) and NPIAs methodology that forms the basis of national assessments undertaken on behalf of ACPO. These five agencies have similar goals in their assessment of organised crime. In strategic terms, the assessments are intended to guide law enforcement decision-making and government policy-making in regards to current and emerging risks from organised crime. In operational terms, the assessments are used to advise priority-setting processes in relation to organised crime-related issues or criminal networks. With the exception of the MPS, these methods are designed to measure harms at the national level.3 Although the methods vary, each agency takes a primarily qualitative approach to harm measurement with a strong but not exclusive reliance on law enforcement data. Intelligence personnel, most often analysts, are primarily
2 3

Korps landelijke politiediensten. The focus and mandate of risk assessment methodologies in relation to organised crime will likely shift in the United Kingdom following significant changes outline by the Home Office which include replacing the Serious Organised Crime Agency (SOCA) with a National Crime Agency (NCA) and phasing out the National Policing Improvement Agency and folding some of its functions within the NCA and perhaps incorporating other functions within the Home Office ([19]).

Natasha Tusikov PhD Candidate, Australian National University, Email: natasha.tusikov@anu.edu.au

responsible for constructing, undertaking, and evaluating their agencys assessments, with limited external involvement. The five agencies operationalise the measurement of organised crime-related harms by focusing on criminal network-specific attributes and activities (e.g., offence scales), or, more broadly, assessing sector-related harms, such as the collective damage from organised crime-related vehicle crime. The assessments that are produced on behalf of ACPO combine both approaches. Despite having similar aims, the five agencies have different approaches to risk assessment methodology. As discussed earlier in this article, risk assessments are comprised of various components depending upon the assessments aim and audience. The following section assesses the reliability and validity of each methods operationalisation of organised crime-related harm (i.e., how that concept will be measured) and, in doing so, builds upon recent evaluations of law enforcement and governmental risk assessments on organised crime (see [16]; [51]). Reliability refers to the consistency of measurement (i.e., results can be reproduced) while validity refers to the extent to which a concept fits the methods used to measure it. United Kingdom: Metropolitan Police Service (MPS) In the United Kingdom, the MPS uses the Criminal Network Harm Assessment Matrix to identify criminal networks (including gangs) causing the greatest harm.4 In September 2008, the MPS amended the 2006 version of its matrix5 and scores each criminal network in three separate categories: 1) harm; 2) criminal capacity and capability; and, 3) other risks (organisational risk and community harm). Harm is assessed using the maximum-statute penalties of 29 offences considered integral to criminal networks in three categories: social (12 offences); economic (15 offences); and, political (two terrorism offences). For each offence, intelligence analysts assess a criminal networks involvement as high, medium or low (based on given definitions). High scores receive the maximum-statute sentence, medium scores receive half, low scores a quarter, and a life sentence is scored as 25 years. For example, a criminal network highly involved in firearms importation (designated an economic harm) would score seven (maximum sentence), whereas medium-level involvement would score four (rounded up from 3.5), and a low would score two. The measurement of criminal capacity and capability (a measure of threat) is determined through six attributes (rated high, medium, or low with related numeric values) that is based upon an abbreviated version of the Sleipnir capability measurement technique (see [38]). Each network is also scored (high, medium or low) for the level of operational risk its activities pose to law enforcement or government, and the level of community impact caused. Criminal networks are prioritised for operational and intelligence resources primarily based on their levels of harm; however, networks with low harm scores but high scores in the other two categories will be highlighted in the priority-setting process. United Kingdom: National Policing Improvement Agency (NPIA) NPIA, a policing organisation funded by the Home Office, developed a methodology to assess a broad range of crime issues, including organised crime, for the National Strategic Assessment. This assessment is produced on behalf of the Association of Chief Police Officers (ACPO) in a
4

The MPS Criminal Network Harm Assessment Matrix Manual (2008) is a protected document. For a detailed discussion of the 2006 version of the MPS Harm Assessment Matrix, see [16].

Natasha Tusikov PhD Candidate, Australian National University, Email: natasha.tusikov@anu.edu.au

collaborative effort by law enforcement agencies across the United Kingdom. It is comprised of situation reports assessing factors including volume, victim groups, and trend direction and establishes priorities within a broad range of crime issues, not solely organised crime.6 Each crime issue is assessed in four harm categories: physical; economic; psychological; social/community; and, geographical distribution, and scored as low, medium or high with corresponding numeric values. Extra weighting factors can be applied to issues that are a police force or district priority, a government priority, a signal crime, or that generate public concern. United Kingdom: Association of Chief Police Officers (ACPO) ACPO, an independent strategic body which coordinates the direction and development of policing services, manages a project called organised crime group mapping which determines the threat and harm levels of organised crime networks in the United Kingdom. This process was developed by NPIA and is described in the NPIAs guidance manuals and it informs the National Strategic Assessment that is produced on behalf of ACPO. Harm is assessed in six categories: injury; community; reputation/political (i.e., public attitudes of police impartiality and effectiveness); cross-border/geography; economic; and criminal capacity/capability (a measure of threat).7 Each category is scored as nil, low, medium, or high with corresponding numeric scores that are aggregated into a total impact score. A multiplier, based on the probability of criminal activities reoccurring (i.e., unlikely, possible, probable, or highly probable) gives a total numerical score for each criminal network. Australia: Australian Crime Commission (ACC) As part of its annual national assessment on organised and serious crime, the ACC undertakes a comparative analysis intended to determine changes in the nature and level of harm amongst various crime issues.8 Subject-matter experts from law enforcement, government, industry and academia use a structured questionnaire created by the ACC to ascribe a numeric score to each crime issue based on defined criteria in three categories: political, social and economic. The political component analyses the nature and level of government interest (federal and state/territory), interest from law enforcement and regulatory agencies, and legislative and regulatory efforts. It includes an assessment of media coverage, political pressure, and the perceived and actual level of community concern, along with related action taken by those communities. The economic component examines the illicit profits generated and the cost to Australian society. The assessment includes an analysis of enabling activities, such as identity theft and money laundering. The ACC then analyses the results, compiles an average score and ascribes a harm level for each issue. The Netherlands: Netherlands Police Agency (KLPD) Every four years, the Department of International Police Information (IPOL) of the KLPD undertakes a national assessment of organised crime with involvement from the Police Academy of the Netherlands, regional police forces and KLPDs National Crime Squad ([24]). The KLPDs assessments research questions and issues are determined by a working party comprised of the National Public Prosecutors Office, National Office of the Public Prosecution
6

The Organised Crime Group Mapping Manual (September 2010) is a protected document produced by the Organised Crime Partnership Board and published by the NPIA on behalf of ACPO. 7 Ibid. 8 The ACCs Harms Assessment Process (2008) is a protected document.

Natasha Tusikov PhD Candidate, Australian National University, Email: natasha.tusikov@anu.edu.au

Service, the Ministry of the Interior, the Ministry of Justice, and the IPOL. The Police Academy produces a study of crime-relevant factors (crime-generating or crime-inhibitive factors) that contextualises the assessment. It focuses on meso-level trends (e.g., socio-economic factors) that will affect law enforcement investigative practices ([23]: 10). In the assessment, organised crime is understood as criminal phenomena involving the structural cooperation between people, with a degree of consistency in the collaboration, for financial or material gain ([24]: 15). Consequences are divided into seven direct harms: health (physical, mental and public); economic; environmental; infrastructure; disruption of socio- economic relationships; influence over judicial procedures; and, influence of political or administrative decision-making. There are three indirect harms: precautionary behaviour (e.g., installing security devices); harm to certain interests or relations (such as loss of reputation); and, enabling activities, such identity fraud. There are no objective criteria for determining each criminal activitys harm. The assessment team collaboratively determines the harm level in a process described as interactive subjectivity ([22]: 95; see [51]). The article now moves to discuss in greater detail the vulnerabilities in these harm methods. Critique of the Methods Measuring the harm caused by organised crime first assumes that the phenomenon can be clearly conceptualised ([48]) and, secondly, that it can be measured in a valid and reliable manner, a challenge that is additionally complicated when analysis is undertaken across multiple jurisdictions and agencies. As Klerks ([22]: 97) notes in his observation of the Dutch assessment process, it is difficult to capture the synthesis of multiple reports on a broad range of crime- related issues in a single rigourous methodology. To address the inherent difficulties involved in assessing organised crime, in addition to the methodological challenges that Klerks notes, the case-study agencies have instituted several good practices. The ACC and KLPD both solicit analysis from external subject-matter experts that undoubtedly provides a richer and more diverse assessment of organised crime-related harms than a single police agency could produce. This practice is suited to analysis of broad crime issues (e.g., real estate fraud) as information is available from many sources. Broad consultation with external experts is a relatively resource- and time-intensive process; moreover, it is dependent on data quality and availability and analysis unimpeded by police or political interference. The ACC, for example, retains control over the selection of crime issues and subject-matter experts, compilation of findings, and setting of priorities. Observing the Dutch assessment process, Klerks ([22]: 95) acknowledges that the involvement of qualified academics further enhances the process but argues that this consultative process cannot replace the open forum of academia. In contrast, criminal network-focused analysis (e.g., MPS) is undertaken by police as it relies almost exclusively on police information sources, though external experts could be given access to police databases to determine how data collection, collation and analytical processes could be strengthened. The NPIA created multiple guidance manuals to ensure a standard approach in terms of techniques, formats and terminology in relation to the assessment process throughout the United Kingdom. Intelligence analysts interviewed about ACPOs risk assessment process were positive about the basic utility, simplicity and practicality of ACPOs risk assessment matrix and stated that prioritisation processes align well with intuitive judgments ([16]: 13). Despite the

Natasha Tusikov PhD Candidate, Australian National University, Email: natasha.tusikov@anu.edu.au

volume of guidance manuals produced and the analysts positive reviews, it remains unclear whether the assessments findings have a greater validity and reliability than other law enforcement assessments. The perceived alignment between prioritised risks and analysts intuitive judgments may indicate the validity and reliability of the process. Conversely, it may also indicate cultural and institutional biases from law enforcement toward certain types of organising structures and activities. A key difficulty with the methods is definitional rigour. This is an obvious challenge as organised crime is alternately understood as a collection of individuals or activities, or a combination of both ([51]). Where organised crime and harm are defined, the wording is relatively vague and open to different interpretations. The methods also tend to qualify their definitions of organised crime or harm with the adjectives significant or serious without specifying how this characteristic is defined or measured. Organised crime is differentiated from serious crime in several methods though the relevance or utility of such a distinction is not clear. Given these definitional challenges, there are problems of validity with the operationalisation of the concept of harm as it is unclear what is being measured and if that is representative of the concept. Moreover, the methods also have problems with reliability as different interpretations of the concepts are possible and thus the conceptualisation and measurement of harms could vary amongst analysts. All the methods conceptualise harm in broad categories (e.g., economic or social harms) that are vaguely defined and not mutually exclusive (i.e., possible overlap amongst categories). The methods also lack specified indicators to measure the presence or absence of harm. However, even if these indicators were clearly defined, Von Lampe ([48]) argues that when law enforcement and policy makers control the selection and analysis of harm-related indicators there are potential problems of institutional and cultural bias that result in poor validity, particularly if analysis is also based primarily on law enforcement sources. These problems elucidate the case-study agencies difficulties in operationalising harm and illustrate the challenges in measuring such a concept with any degree of validity and reliability. This articles findings are similar to recent evaluations of law enforcement and governmental assessments of organised crime (see [16]; [51]). Criminal network-based harm methods (e.g., MPS and ACPO) are intended to assist senior law enforcement management set enforcement priorities by ranking criminal networks according to their relative levels of harm and threat. The methods used by MPS and ACPO are practical solutions to a complex problem. However, these methods assume that organised criminality can be differentiated clearly from non-organised crime and that organised illicit activities can be accurately and reliably attributed to specific criminal networks. Social networks, however, are not static; networks fuzzy boundaries make it difficult to determine which actors should be included within the network (see [29]). Attributing illicit activities to particular networks is somewhat easier when actors self-identify as part of a criminal organisation or are especially overt in their criminality. However, even when a network is somewhat delineated, it is problematic to assume that all offences are committed as part of or to benefit a criminal organisation. Criminal-offence scales presume that organised criminality can be reduced to a list of integral offences (see MPS); however, these scales tend to demonstrate law enforcements traditional preoccupation with drug- or violence-related offences and lesser attention to other types of

10

Natasha Tusikov PhD Candidate, Australian National University, Email: natasha.tusikov@anu.edu.au

crime, such as fraud. Such scales therefore represent a relatively narrow conceptualisation of organised crime, an understanding that is likely attenuated by the privileging of law enforcement information sources that contain offence-specific data over other data sources that could offer relevant information. Using specific offences to measure damage is additionally complicated because, as Fox and Freiberg ([13]: 166) observe: the same offences can vary widely in the manner of their execution, in the harm they cause and in the interests upon which they infringe. These challenges are compounded by potential measurement problems. For example, the MPS appears to allow criminal networks that traffic drugs or weapons to also be scored for possession offences, essentially double-scoring trafficking offences. Using sentences to determine crime seriousness, as the MPS does, is well established in criminology. The use of maximum sentences would appear to facilitate consistent scoring throughout the MPS, minimise the discretion of personnel in scoring, and lend legitimacy and objectivity to the process ([16]: 13). The integrity of the scoring process, however, depends on the validity and reliability of the intelligence assessed, the quality of analysis, and the consistency of scoring a criminal networks level of involvement, which is a relatively subjective process (see [16]). Moreover, the use of sentences to establish seriousness assumes that the seriousness of an offence is matched by a penalty of equivalent gravity, consistency between jurisdictions in sentencing, and similarities between the ordering of offences according to maximum penalties and the ranking of offences according to penalties imposed, which may not be the case ([13]: 171). As Dorn and Van der Bunt ([8]: 35) rightly argue, harm should be conceptualised separately from the response to crime, as using sentence lengths to indicate seriousness is circular in that current priorities are set using historical legislative and judicial judgments. Three of the case studies formally incorporate state or media pressure as criteria (ACC, MPS and NPIA). These criteria are an explicit acknowledgement that these factors influence but (ideally) do not dominate decision-making. However, these criteria are questionable in terms of validity because of the clear potential for politicisation and the troublesome problem of circularity in which past concerns and expenditures dictate future efforts. Moreover, it is unclear what indicators are used to measure state or media pressure or how law enforcement would measure these criteria with any degree of validity and reliability. Using public concern as a factor in measuring crime seriousness has clear precedent in criminology (e.g., [39]) and, as Cohen ([4]: 269) points out, surveying the public has resulted in relatively consistent rankings over time and across populations. Public opinion research regarding crime, however, has clear limitation as there are misperceptions about crime frequency and injury severity (see [4]) and much of this research does not concentrate on organised crime (for an exception, see [3]). Moreover, as decades of criminological research demonstrates, the public perception of organised crime is often sensationalised, imbued with racism and anti-immigrant rhetoric and prone to the creation of folk devils (e.g., [25]), making accurate and reliable measurement difficult. The public may view organised crime as serious depending upon how the phenomenon is conceptualised and the moral panic and folk devils de jourhowever, as Levi ([25]: 53) argues in relation to white-collar crime, viewing something as being serious is not the same as wanting it prioritised by police or wanting it more heavily sentenced than other crimes.

11

Natasha Tusikov PhD Candidate, Australian National University, Email: natasha.tusikov@anu.edu.au

Several of the case studies make reference to the reputational harm to law enforcement, government or particular interests, such as the financial sector. Like state and media pressure, loss of reputation is a problematic criterion in terms of validity. It is unclear if reputational damage (to state or commercial interests) can and should be categorised as harm caused by criminals and how such a loss could be reliably valued ([8]: 32; see also [16]). Reputational loss is in the eye of the beholder: what some institutions may decry as damage to reputation, others may regard as a well-deserved rebuke for ineffective policies or poor performance or dismiss as political in-fighting. As Dorn and Van der Bunt ([8]: 32) observe, there is no clear or objective arbitration between opposing views of reputational damage and, in any case, they argue that it is often reaction from policy-makers, media and the public that causes such damage, not criminals per se. Conclusions Despite its critique of the harm methods, this article fully recognises that they are intended to operate within assessment processes that have an ambitiously broad scope (both in terms of criminality and geography) and a varied clientele with diverse requirements. The (mostly) civilian intelligence analysts responsible for the methods are likely aware of the methods methodological shortcomings. However, the construction, implementation and evaluation of these methods require dedicated time and managerial support and such resources are often in short supply within the reactive environment of law enforcement. This article also appreciates the difficulty of applying risk assessment methodology to the rather nebulous and inherently controversial concept of organised crime. There are several key problems with the methods: the main concepts are generally ill-defined, the operationalisation of these concepts is problematic, and the separation of organised and serious crime is not clear nor is the significance or utility of such a distinction. These findings are similar to those in recent studies (see especially [16]; [51]). There are several problematic harm criteria, in particularpolitical/law enforcement interest, media/public attention, and reputational harmthat can potentially politicise and distort harm measurement and priority- setting processes. The case-study agencies each have varying processes to strengthen analysis and facilitate consistent measurement; however, the measurement and subsequent ranking are largely subjective. The problems evident in the harm methods raise several critical questions, specifically whether measuring and ranking organised crime-related harms (either from specific criminal networks or, more broadly, from crime issues) are empirically feasible and, if so, can be undertaken in a manner that meaningfully informs law enforcements decision-making and limits politicisation. A related question is the comprehensiveness of harm measurement, particularly if methods tend to favour data-rich or former high-priority issues over lesser-known, data-poor issues. The circularity of these information processes results in self-perpetuating priority-setting processes in which certain issues or organisations are repeatedly targeted because more is known about them ([36]) or because the state allocated considerable resources in the past but did not achieve its goals ([8]: 15). Some issues designated as priorities are so broad and generic (e.g., financial crime) that they are undisputable as priorities ([17]: 497) but are relatively meaningless in terms of directing decision-making and should therefore be treated with some skepticism ([9]). While determining how methods can be strengthened for practical application within law enforcement is part of a much larger discussion, this article concludes with a few suggestions.

12

Natasha Tusikov PhD Candidate, Australian National University, Email: natasha.tusikov@anu.edu.au

Key concepts, particularly, organised crime, harm, threat, serious crime and significant crime, should be clearly defined in all documents, even if such concepts have been previously defined internally or in earlier documents. Such definitions should accord as closely as possible to social science standards of reliability and validity (see [51]). Given the challenges involved in defining and measuring political/law enforcement interest, media/public attention, and reputational harm, and the general weakness of these criteria, policing agencies should move away from measuring these areas. Instead, it would be more effective to focus on developing strong indicators to measure the nature and magnitude of specific types of harm that use information from multiple state and non-state sources. In terms of strengthening the implementation and use of the analysis of harm, the NPIAs guidance manuals offer a practical and useful way to establish minimum standards and ensure consistent usage by multiple policing agencies across a broad geographical area. Policing agencies should also look to other components of risk assessment methodology that would provide a more comprehensive risk analysis of organised crime. Law enforcements risk assessments, for example, would be strengthened by the inclusion of measures to determine probability of specific threats and harms occurring and the vulnerability of certain populations or institutions to organised crime (see [1]; [46]). The current interest from both the scholarly and law enforcement communities in applying risk assessment methodology to organised crime is promising. There is a great deal of theoretical and practical research to be undertaken in terms of conceptualising and operationalising organised crime-related harms and developing rigourous methodologies that have real practical utility to law enforcement decision-making and priority-setting processes. Such research is essential if we are to develop further criminal intelligence analysis (see [12]; [20]) and strengthen law enforcements use of risk assessment methodology. Acknowledgements The MPS, ACC and NPIA graciously granted the author permission to discuss these methods provided that restricted information was not released. While ACPO leads the Organised Crime Group Mapping project, the underlying methodology was created by NPIA which granted permission to discuss this project. The information on the Dutch National Threat Assessment was publicly available.

13

Natasha Tusikov PhD Candidate, Australian National University, Email: natasha.tusikov@anu.edu.au

References 1. Albanese, J., (2008). Risk assessment in organised crime: Developing a market and product-based model to determine threat levels. Journal of Contemporary Criminal Justice, 24, 263-273. 2. Black, C., De Ruyver, B., and Vander Beken, T., (2001). Reporting on organised crime: A shift from description to explanation in the Belgian Annual Report on Organised Crime, Antwerpen: Maklu Publishers. 3. Bullock, K., Chowdhury, R., and Hollings P., (2009). Public concerns about organised crime. Research Report 16, Research, Development and Statistics Directorate, Home Office United Kingdom, www.rds.homeoffice.gov.uk/rds/pdfs09/horr16c.pdf. Accessed 11 May 2010. 4. Cohen, M., (2000). Measuring the costs and benefits of crime and justice. Measurement and analysis of crime and justice, Criminal Justice 2000, Vol. 4, Washington, D.C.: Office of Justice Programs, U.S. Department of Justice. www.ncjrs.gov/criminal_justice2000/vol_4/04f.pdf. Accessed 12 January 2009. 5. Di Nicola, A., and McCallister, A., (2006). Existing experiences of risk assessment. European Journal of Criminal Justice Policy Research, 12, 179-187. 6. Dolan, P., and Peasgood, T., (2007). Estimating the economic and social costs of the fear of crime. British Journal of Criminology, 47, 121-132. 7. Dorn, N., (2009). The end of organised crime in the European Union. Crime, Law, and Social Change, 51, 283-295. 8. Dorn, N., and Van de Bunt, H., (2009). Bad thoughts: Towards an organised crime harm assessment and prioritisation system (OCHAPS), Supported by the Ministry of Justice, The Hague, The Netherlands, www.wodc.nl/images/1703_volledige%20tekst_tcm44- 257202.pdf. Accessed 14 May 2010. 9. Edwards, A., and Gill, P., (2003). After transnational organised crime? The politics of public safety. In A. Edwards and P. Gill (Eds.), Transnational organised crime: Perspectives on global security (pp. 264-281). New York: Routledge. 10. Ericson, R. V., and Haggerty, K. D., (1997). Policing the risk society. Toronto, ON: University of Toronto Press. 11. Ericson, R., and Shearing, C., (1986). The scientification of police work. In G. Bohme and N. Stehr (Eds.), The knowledge society: The growing impact of scientific knowledge on social relations (pp. XXXX). Dordrecht: D. Reidel. 12. Fisher, R., and Johnston, R., (2008). Is intelligence analysis a discipline? In R. Z. George and J. B. Bruce (Eds.), Analyzing intelligence: Origins, obstacles, and innovations (pp. 55- 68). Washington, D.C.: Georgetown University Press.

14

Natasha Tusikov PhD Candidate, Australian National University, Email: natasha.tusikov@anu.edu.au

13. Fox, R.G., and Freiberg, A., (1990). Ranking offence seriousness in reviewing statutory maximum penalties. Australian and New Zealand Journal of Criminology, 23, 165-191. 14. Garland, D., (2003). The rise of risk. In R.V. Ericson and A. Doyle (Eds.), Risk and morality. Toronto, ON: University of Toronto Press. 15. Gill, P., (2000). Rounding up the usual suspects? Developments in contemporary law enforcement intelligence. Aldershot, Hampshire: Ashgate Publishing. 16. Hamilton-Smith, N., and Mackenzie, S., (2010). The geometry of shadows: a critical review of organised crime risk assessments. Policing and Society. 20(3), 257-279. 17. Harfield, C., (2008). The organisation of organised crime policing and its international context. Criminology and Criminal Justice, 8, 483-507. 18. Heaton, R., (2000). The prospects for intelligence-led policing: Some historical and quantitative Considerations. Policing and Society, 9(4), 337-56. 19. Home Office (2010), Policing in the 21st century: Reconnecting police with the people. http://www.homeoffice.gov.uk/publications/consultations/policing-21st-century/ 20. Innes, M., Fielding, N., and Cope, N., (2005). The appliance of science? The theory and practice of criminal intelligence analysis. British Journal of Criminology, 45(1), 39-57. 21. Johnston, P., Schneider, S., Neily, J., Parkes, W., and Lachaine, D., (2009). Developing and applying an organised crime harm index: A scoping and feasibility study. Public Safety Canada: Lansdowne Technologies Inc. 22. Klerks, P., (2007). Methodological aspects of the Dutch National Threat Assessment. Trends in Organised Crime, 10, 91-101. 23. Klerks, P., and Kop, N., (2008). Societal trends and crime-relevant factors: An overview for the Dutch National Threat Assessment on Organised Crime 2008-2010. The Netherlands: Police Academy of the Netherlands, Lectorate for Crime Control and Investigative Science. www.policefuturists.org/pdf/SocietalTrendsandCrime- relevantFactors.pdf. Accessed 12 January 2009. 24. Korps landelijke politiediensten (2008). National Threat Assessment 2008 Organised Crime. [National Police Agency of the Netherlands]. http://www.politie.nl/KLPD/Images/200830e%20%20%20%20national%20%20threat%2 0assessment%202008_tcm35-504488.pdf. Accessed 12 January 2009. 25. Levi, M., (2009). Suite revenge? The shaping of folk devils and moral panics about white- collar crimes. British Journal of Criminology, 49, 48-67. 26. Maltz, M., (1990). Measuring the effectiveness of organised crime control efforts. University of Illinois at Chicago, IL: Office of International Criminal Justice Press.

15

Natasha Tusikov PhD Candidate, Australian National University, Email: natasha.tusikov@anu.edu.au

27. McGuire, M., (2000). Policing by risks and targets: Some consequences of intelligence- led crime control. Policing and Society, 9(1), 315-36. 28. Melchers, R., (2007). Academic contributions to the measurement of organised crime: A literature search. Public Safety and Emergency Preparedness Canada: Research, Evaluation and Coordination Division. 29. Morselli, C., (2009). Inside criminal networks. New York, NY: Springer Science+Business Media. 30. Porteous, S., (1998). Organised crime impact study. Ottawa, ON: Public Works and Government Services of Canada. 31. Ratcliffe, J. H., (2008). Intelligence-led policing. Cullompton: Willan Publishing. 32. Ratcliffe, J. H., (2005). The effectiveness of police intelligence management: A New Zealand case study. Police Practice and Research, 6(5), 435-451. 33. Savona, E. U., (2006). A methodology for measuring the harm caused by organised crime. Forum on Crime and Society, 5(1), 69-88. 34. Sheptycki, J., (2003). The governance of organised crime in Canada. Canadian Journal of Sociology, 28(4), 489-516. 35. Sheptycki, J., (2004). Organisational pathologies in police intelligence systems; some contributions to the lexicon of intelligence-led policing. European Journal of Criminology, 1(3), 307-332. 36. Sheptycki, J., (2007). Police ethnography in the house of serious and organised crime. In A. Henry and D. Smith (Eds.), Transformations of policing (pp. 5177). Aldershot, Hampshire: Ashgate Publishing. 37. Shelley, L., (1999). Identifying, counting and categorising transnational criminal organisations. Transnational Organised Crime, 5(1), 1-18. 38. Strang, S. J., (2005). Project Sleipnir: An analytical technique for operational priority setting. Ottawa, ON: Royal Canadian Mounted Police, https://analysis.mitre.org/proceedings/Final_Papers_Files/135_Camera_Ready_Paper.p df. Accessed 17 April 2008. 39. Stylianou, S., (2003). Measuring crime seriousness perceptions: What have we learned and what else do we want to know. Journal of Criminal Justice, 31, 37-56. 40. Urban Health Research Initiative (2010). Effect of drug law enforcement on drug-related violence: Evidence from a scientific review. British Columbia Centre for Excellence in HIV/AIDS, B.C.: Urban Health Research Initiative, http://uhri.cfenet.ubc.ca. Accessed 18 March 2010.

16

Natasha Tusikov PhD Candidate, Australian National University, Email: natasha.tusikov@anu.edu.au

41. Van Dijk, J., (2007). Mafia markers: assessing organised crime. Trends in Organised Crime, 10, 39-56. 42. Van Duyne, P.C., (2003). Money laundering policy: Fears and facts. In P.C. Duyne, K. von Lampe, J.L. Newell (Eds.), Criminal finances and organising crime in Europe (pp. 72-109), Nijmegen: Wolf Legal Publishers. 43. Van Duyne P.C., (1996). Phantom and threat of organised crime. Crime, Law, and Social Change, 24, 341-377. 44. Van Duyne P.C., and van Dijck, M., (2007). Assessing organised crime: The sad state of an impossible art. In F. Bovenkerk and M. Levi (Eds.), The organised crime community: essays in honor of Alan A. Block (pp. 101-124), New York, NY: Springer. 45. Van Duyne P.C., and Vander Beken, T., (2009). The incantations of the EU organised crime policy making. Crime Law, and Social Change, 51, 261-281. 46. Vander Beken, T., (2004). Risky business: A risk-based methodology to measure organised crime. Crime, Law, and Social Change. 41(5), 471-516. 47. Vander Beken, T., Savona, E. U., Korsell, L. E., Defruytier, M., Di Nicola, A., Herber, A., Bucquoye, A., Dormaels, A., Curtol, F., Fumarulo, S., Gibson, S., and Zoffi, P., (2006). Measuring organised crime in Europe: A feasibility study of a risk-based methodology across the European Union. Antwerp: Maklu Publishers. 48. Von Lampe, K., (2004). Making the second step before the first: Assessing organised crimeThe case of Germany. Crime, Law, and Social Change, 42, 227-259. 49. Von Lampe, K., (2004). Measuring organised crime: A critique of current approaches. In P. Van Duyne, M. Jager, K. Von Lampe, and J. Newell (Eds.), Threats and phantoms of organised crime, corruption and terrorism; rhetoric and critical perspectives (pp. 85- 116). Nijmegen: Wolf Legal Publishers. 50. White, R., (2008). Crimes against nature: Environmental criminology and ecological justice, Cullompton: Willan Publishing. 51. Zoutendijk, A. J., (2010). Organised crime threat assessments: a critical review. Crime, Law, and Social Change, 54, 63-86.

17