Risk Assessment and Risk Management in Policing

Robert E. Worden
School of Criminal Justice
The University at Albany, SUNY
Albany, NY, USA
and
The John F. Finn Institute for Public Safety, Inc.
Albany, NY, USA

Christopher J. Harris
Department of Criminal Justice and Criminology
University of Massachusetts – Lowell
Lowell, MA, USA

Sarah J. McLean
The John F. Finn Institute for Public Safety, Inc.
Albany, NY, USA

Forthcoming in Policing: An International Journal of Police Strategies & Management

© Emerald Group Publishing Limited

Corresponding author: Robert E. Worden

Corresponding author’s e-mail: rworden@albany.edu
 Biographical Details

Robert E. Worden is associate professor of criminal justice and public policy at the University at

Albany, State University of New York, and the Director of the John F. Finn Institute. He holds a

Ph.D. in political science from the University of North Carolina at Chapel Hill. His scholarship

has appeared in Justice Quarterly, Criminology, Law & Society Review, and other academic

journals, and his research has been funded by the National Institute of Justice, the Bureau of

Justice Assistance, the New York State Division of Criminal Justice Services, and other

sponsors.

Christopher J. Harris is an assistant professor in the Department of Criminal Justice &

Criminology at UMass Lowell and received his Ph.D. in criminal justice from the University at

Albany, State University of New York. His research interests are primarily in police

accountability and public perceptions of police, as well as evaluation research of police

initiatives. He is author of Pathways of Police Misconduct (2010) from Carolina Academic

Press.

Sarah J. McLean is the Associate Director and the Director of Research and Technical

Assistance at the John F. Finn Institute. She holds a Ph.D. in criminal justice from the University

at Albany, State University of New York, with a specialization in policy and process. At the

Institute she participates in research that examines the effectiveness of strategic and

programmatic crime reduction initiatives. She previously worked as a research associate at the

Hindelang Criminal Justice Research Center of the University at Albany, and prior to that as a

research associate at the National Center for Mental Health and Juvenile Justice (NCMHJJ).

ii
 Risk Assessment and Risk Management in Policing
Abstract

Purpose
The purpose of this paper is to critique contemporary tools for assessing and managing the risk
of police misconduct and suggest directions for their improvement.

Design/methodology/approach
We draw on extant literature, synthesizing several lines of inquiry to summarize what we know
about patterns of police misconduct, and what we know about assessing and managing police
misconduct. Then we draw from the literature on offender risk assessment in criminal justice to
draw lessons for assessing and managing the risk of police misconduct.

Findings
We found that there is good reason to believe that the tools used to assess the risk of misconduct
make suboptimal predictions about officer performance because they rely on limited information
of dubious value, but also that the predictive models on which the tools are based could be
improved by better emulating procedures for assessing offenders’ risk of recidivism.

Research limitations/implications
Future research should examine cross-sectional and longitudinal patterns of misconduct and
associations between risk-related outputs and enforcement activity, develop better measures of
criterion variables, and evaluate the predictive accuracy of risk assessment tools.

Practical implications
Police managers should make better use of the information available to them, improve the
quantity and quality of information if feasible, and cooperate in the necessary research.

Originality/value
This paper offers a new synthesis of extant research to demonstrate the limitations of
contemporary provisions for assessing the risk of police misconduct, and potential avenues for
useful research and improved practice.

Keywords
Police misconduct; early intervention system; citizen complaints; police use of force; police
management; police supervision; police performance

Article Classification
Research Paper

iii
 Introduction

Police misconduct has a corrosive effect on police organizations, the communities they

serve, and the relationship between the two.1 When police deviate from law or policy, it affects

the legitimacy of the police, the tenor of police-community relations, and the cooperation of the

public with law enforcement. Social science has examined several forms of police misconduct:

the use of excessive force; illegal searches and seizures; bias in police stops; discourtesy and

other unprofessional conduct. But across more than four decades of research on these topics, the

accumulated empirical evidence is not voluminous, and our understanding of the well-springs of

police misconduct and the organizational mechanisms by which misconduct can be better

regulated is not very deep (see, in general, National Research Council, 2004, p. 290).

For their part, police executives have increasingly assumed responsibility not only for

sanctioning acts of misconduct but also for managing the risk of misconduct by their officers.

Risk management does not rely on any one organizational mechanism, but early warning systems

– also known as early intervention (EI) systems – have emerged as a key tool for managing the

risk of misconduct as they have proliferated across the U.S. EI systems have even become an

accreditation requirement and a core element of Justice Department “pattern or practice”

litigation and consent decrees (Harmon, 2009; Walker, 2003a), treated as a “best practice” of

police administration. 2

The logic that underlies EI systems is rooted in the empirical observation that small

numbers of officers account for disproportionate fractions of citizen complaints, uses of physical

force, and other symptoms of police misconduct. By monitoring indicators of police action that

might signify misconduct, or “risk-related” outputs (Bobb, 2009), police administrators identify

officers who display symptoms of recurring problematic conduct, and intervene soon after such

symptoms appear with counseling or retraining. Early intervention would presumably serve to
prevent a substantial volume of police misconduct. Thus EI systems are structured to detect

short-term spikes in indicators that bear presumptive relationships to police misconduct: one or

multiple indicators tracked over periods ranging from 3 months to a year. Commercially

available software flags officers who reach or exceed specified thresholds on indicators.

EI systems are in an early stage of development, even in 2012, and the purposes that EI

systems are intended to serve have evolved. The more elaborate versions of EI systems – now

sometimes known as performance measurement information systems, such as Pittsburgh’s PARS

– are intended to identify not only “problem officers” – officers with a chronic propensity to

abuse their authority – but also others with performance problems as well (Walker, 2003b).

Furthermore, as Walker (2003b, pp. 10-11) has observed, such systems might serve to engage the

entire organizational system in improving performance, and not serve only to fix attention on bad

cops. However, as we explain below, we believe that the current structure of EI systems is not

commensurate with the aspirations that their advocates have for them.

We draw on extant literature, synthesizing several lines of inquiry to summarize (1) what

we know about patterns of police misconduct, and (2) what we know about assessing and

managing police misconduct. Then we draw from the literature on offender risk assessment in

criminal justice more generally to draw lessons for assessing and managing the risk of police

misconduct and directions for future research.

What We Know about Patterns of Police Misconduct

Previous research suggests that the incidence of misconduct by police tends to exhibit a

skewed distribution across officers, concentrated disproportionately among small subsets of

individuals, at least at any given point in time. It also shows, however, that the behaviors

captured in performance indicators are correlated with officer activity, and these behaviors are

2
ambiguous as symptoms of misconduct. Further, some research shows that these behaviors are

associated with some characteristics of police officers, such as their age and pre-employment

histories. Moreover, some recent research suggests that, at the individual level, these behaviors

display substantial variation over time, as officers’ careers unfold; some officers appear to desist

from misconduct early in their careers, while a small number persist. It may in addition be

possible to isolate different syndromes of problem behavior.

Concentrations of Potentially Problematic Behavior

The cross-sectional distributions of complaints, use of force, etc., across officers are very

skewed, with a small number of officers accounting for a disproportionate number of these

incidents. For example, following the Rodney King incident, the Independent Commission

formed to investigate the Los Angeles Police Department (LAPD) found a set of 44 problem

officers who together over a five-year period accounted for less than one percent of the officer

population and fifteen percent of citizen complaints and incidents involving the use of force

(Independent Commission on the Los Angeles Police Department, 1991). Analyses in other

agencies, such as the Houston Police Department, the Boston Police Department, the Los

Angeles Sheriff’s Department, and the Kansas City Police Department, all demonstrate that such

a skewed distribution is not unique to LAPD, as small numbers of officers in those agencies

accounted for disproportionately large fractions of citizen complaints as well (Walker, 2003b).

Likewise, use of force reports are completed disproportionately by a small number of officers.

Lersch, et al. (2006), examining data on officers in a large southeastern department, found that

while the average number of use of force reports was 41 in the year 2000, just over seven percent

of the patrol officers in this agency each filed over 100 such reports (and in one case, as many as

245). In addition, Worden et al. (2003; also Worden et al. 2012b), in their study of one agency’s

3
EIS, found that many indicators of problem behaviors such as citizen complaints, use of force

reports, police vehicle crashes, and the use of sick leave also are disproportionately concentrated

when considering these indicators for one- and two-year time periods.

Correlations with Officer Activity

We also know, however, that the indicators that are included in (or recommended for) EI

systems correlate with indicators of officer activity. Officers who make more arrests also tend to

use force more frequently, and they are also more frequently the subjects of citizen complaints.

Lersch (2002), using 1997 data from a southeastern police department, found that measures of

officer productivity, particularly the mean number of felony and misdemeanor arrests, were

moderately correlated with citizen complaints. Also, high complaint officers (those with two or

more complaints) were significantly more likely to make a greater number of felony arrests than

low complaint officers (those with fewer than two complaints). Similarly, Brandl, et al. (2001)

examined data on citizen complaints about excessive force recorded by a large mid-western

police department in one year. Officers who were in the high complaint group (those with three

or more excessive force complaints) made significantly more arrests than officers in the low

complaint group (those with two or fewer such complaints). In addition, Hassell and Archbold

(2010), using four years of complaint data from a mid-western police department, found that the

average number of citations is positively related to the frequency of citizen complaints. Lersch,

et al. (2006) also found that officers who were selected for an agency’s EIS based on their use of

force reports were significantly more likely to have personnel complaints filed against them, but

also averaged significantly more arrests during their one year data collection period.

Taken together, these findings would suggest that common indicators of problem

behaviors employed by EI systems are correlated with indicators of officers’ enforcement

4
activity and hence their exposure to situations that put them at risk of the use of force or citizen

complaints. Thus one might contend, as Lersch (2002) has, that EI systems unfairly select and

punish productive officers who in fact may not be problematic. We return to this important issue

below.

Ambiguity of the Indicators

These aggregate correlations are subjects of concern at least partly due to both

conventional wisdom and some empirical evidence about individual incidents. We know that

neither every citizen complaint nor every use of force report signifies misconduct. Complaints

can be filed based on misunderstanding, calculations of legal advantage, or malice, and of course

many uses of force by police are quite legitimate and often even reflect restraint. Indeed, when

the use of force is disaggregated and examined in the context of individual police-citizen

interactions, we find that police often use no greater and sometimes lesser forms of force – if any

at all – than the resistance of citizens would appear to justify (e.g., Alpert and Dunham, 1997;

Terrill, 2005).3

It is sometimes observed that where we find smoke in the form of frequent uses of force

or numerous citizen complaints, then surely the fire of misconduct burns, even if it is not readily

visible. But a different metaphor might apply: like an internal combustion engine, routine police

work, when it is performed properly or even skillfully, can be expected to produce exhaust in the

form of use-of-force reports and citizen complaints. We need to separate the smoke of

misconduct from the exhaust of routine policing, and to date this analytic feat has not been

accomplished. Walker (2003b) cites the virtues of taking account of police activity, and also

using peers as benchmarks, but it appears to be seldom done and such adjustments are of

unknown validity.

5
Risk and Protective Factors

We know that misconduct is related to a number of pre-service and academy indicators,

which we might think of as “risk factors.” For example, Cohen and Chaiken (1973) found that

officers’ records of dismissals in prior jobs or military discipline were associated with later

police rule violations. Also, the Mollen Commission (Commission to Investigate Allegations of

Police Corruption and the Anti-Corruption Procedures of the Police Department, 1994) found a

high prevalence of prior arrests among suspended and dismissed officers in the New York City

Police Department (NYPD). Cuttler and Muchinsky (2006) found that personality traits and

work history predict “dysfunctional job performance.” Greene, et al. (2004) found that among

Philadelphia police, those who at the time of their application had previously been arrested, or

ticketed for a traffic violation, or had their driver license suspended were upon their assignment

to the street more likely to be the subjects of complaints for physical abuse and for verbal abuse.

So too were those who had applied for a gun permit prior to becoming police officers. Similarly,

in their analysis of career-ending misconduct, Fyfe and Kane (2005) found that pre-employment

histories of arrest, along with failures in other jobs (relating to either disciplinary problems or

shortcomings of reliability), and (younger) age at appointment, were all associated with

involuntary separation from the NYPD. Academy performance was also predictive of later

termination, with rates of involuntary separation for all types of police misconduct being

inversely associated with Police Academy academic averages.

Fyfe and Kane further found that education appeared to be a protective factor; those with

some college, associate’s degrees, and bachelor’s degrees were all less likely to be involuntarily

separated. They also found that length of service was associated with a lower risk of involuntary

separation. Such a pattern is affirmed by Harris’ (2009b) research, which found that for most

6
officers, the risk of personnel complaints is highest early in their careers and declines steadily

over the course of their careers; it appears that police misconduct and length of service are

related in much the same way that criminal offending and age are related, forming an experience-

misconduct curve similar to the well-known age-crime curve. Thus many officers who are the

subjects of complaints discontinue (without EIS intervention) whatever behavior gives rise to

those complaints.4

Multiple Trajectories of Misconduct

We also have some evidence that the cross-sectional patterns overstate the degree to

which complaints and other events are concentrated among presumptively problem officers.

Worden, et al. (2003, 2012b) analyzed several indicators for all sworn members of a large police

department in the northeastern U.S.: nearly fifteen years of data on personnel complaints, police

vehicle accidents, secondary arrests, arrests more generally, civil litigation, and sick leave usage,

as well as six years of use of force data (i.e., all of the data that were in machine-readable form).

In a nutshell, they found that: (1) each year, complaints and other indicators displayed the same

kind of skewed distribution that others have reported in other agencies, in that a small number of

officers accounted for a large fraction of the events, but (2) it was not, for the most part, the same

officers who appeared in the tail of the distribution from year to year, such that (3) officers who

were documentable "problems" at any one point in time were problematic only for a brief time,

and only a small subset of those were chronically problematic.

Using a subset of the same data to examine a large cohort of officers followed for a

lengthy portion of their careers, Harris (2010) found evidence of divergent citizen complaint

patterns, or trajectories, over time. Specifically, he found three groups of officers with distinct

longitudinal patterns – low-, mid-, and high-rate trajectories – underlying the aggregate

7
relationship between experience as an officer and citizen complaints. Each of these groups

significantly differed in terms of when they began their problem behaviors, when they ended,

and how frequently problem behaviors occurred between these two career points. Also, while the

mid- and high-rate officers are overrepresented in citizen complaints based on their percentage of

the police force, each displays an early peak and a steady decline over time.5 Thus, both groups

may be problematic, but the degree of problematic behavior varies over time both within and

across each group.

Multiple Problems

The public, the courts, scholars, and police executives have tended to think of problem

officers as those who use improper force. We sometimes assume that officers who exhibit

conduct problems, e.g., in the form of rudeness, will escalate their misconduct to the abuse of

force. But officers could exhibit different problems, including underproductivity, and as EI

systems have evolved, they have incorporated additional indicators and make claims that they

address different problems. We do not know much about the nature of these problems, and we

know even less about the short- and longer-term causes of these problems. Research on police

use of force has little to say about the use of improper force (Harris, 2009a), and even less about

the influences on misconduct more generally other than situational factors. Characteristics of

officers such as race, sex, and years of service appear as independent variables in some research,

but the more proximate causes of their behavior (e.g., authoritarian personality traits or substance

abuse) are subjects on which little scientific evidence has accumulated. Some of the more

nuanced treatments of these patterns appear in ethnographic typologies of police officers (e.g.,

Brown, 1981; Muir, 1977; see generally Worden, 1995b), but for the most part, these questions

have not attracted the attention of social science.6

8
 What We Know about Assessing and Managing Misconduct Risk

Most of what we know about managing the risk of police misconduct concerns the recent

establishment and operation of EI systems, on which hopes for positive change have been

pinned. That is not to say that social science is devoid of findings on the effects of administrative

interventions on risk-related police behavior. We know that administrative rule-making can have

a substantial effect on the use of lethal force (Fyfe, 1979, 1988a). One study suggests that

training can have a salutary effect on the use of force (Fyfe, 1988b). And it’s possible – though

hardly an established scientific fact – that police organization cultures could be managed to

produce socially beneficial effects on police compliance with law and policy (Worden and

Catlin, 2002; Davis and Mateu-Gelabert, 1999). But what we know with the greatest confidence,

based on research in corrections and not in policing, is that risk assessment and risk management

is best performed according to some established principles, principles with which current EI

systems are not in conformance.

Early Intervention Systems

As we discussed above, numerous investigations of police departments have shown that a

small number of police officers are repeatedly the subjects of citizen complaints or the authors of

use-of-force reports. The first documentation of these officers was a 1981 report by the U.S.

Commission on Civil Rights, which found that in the Houston Police Department, 12 percent of

the officers who had received citizen complaints accounted for 41 percent of all such complaints.

In view of this finding, the Commission (1981) recommended that “A system should be devised

in each department to assist officials in the early identification of violence-prone officers” (p.

159). Other investigative bodies have made similar recommendations, with the Independent

Commission on the Los Angeles Police Department (1991) noting that the small group of 44

9
problem officers was readily identifiable from police department records. Police agencies have

not, until recently, excelled in compiling and tracking information in terms of which potential

problems could be identified. For many years, it was noted by some police scholars that

supervisors often knew (or believed they knew) who their problem officers were, but seldom

took administrative action against them (Goldstein, 1977).

An EIS appears to be at least a partial cure for these ills, as it necessitates the systematic

collection and analysis of various performance indicators in a search for misconduct risks. An

EIS is a behavioral management device for law enforcement agencies, designed to identify

officers whose behavior appears problematic and subject those officers to some form of non-

punitive intervention. A national evaluation of EI systems conducted by the Police Executive

Research Forum concluded that these systems “are a significant and growing aspect of American

law enforcement,” as about one-quarter of local police agencies serving populations of 50,000 or

more had some version of an EIS at the time of their survey, in 1998 (Walker, et al., 2000).

Designed and used properly, EI systems would appear to represent a promising mechanism to

enhance police accountability and service.

Operationally, EI systems have four component parts: performance indicators,

identification and selection criteria, intervention(s), and post-intervention monitoring (Walker

and Milligan, 2005). The performance indicators captured by a department’s EIS database

include records of the officer activities that are considered risk-related, such as citizen

complaints, use of force reports, and administrative investigations. These indicators and

associated thresholds enable administrators to identify officers whose behavior appears

problematic, warranting some form of response. The intervention component of EI systems

normally consists of counseling or retraining of officers. Finally, the department (supervisors)

10
may continue to monitor officers’ performance for a specified period of time following an

intervention to ensure that problem behaviors have been corrected.

The development of EI systems remains in the early stages, however, and the evidence on

their effectiveness is thin and weak. To date, and to our knowledge, only six agencies’ EI

systems have been the subjects of outcome evaluations (Bobb, 2009; Macintyre, et al., 2008;

Walker, et al., 2000; Worden, et al., 2012a). With the exception of Worden, et al. (2012a), these

studies report positive findings about the effects of EI systems’ interventions, despite differences

in the systems’ selection mechanisms and criteria, as well as in the nature and content of the

interventions. Yet the studies that report positive findings all rest on fairly weak designs. Most

are one-group, pre/post designs, with a limited range of measured outcomes. They provide for

either no control group or a control group that differs in key respects, and with no use of

statistical controls to adjust for those differences. Thus they are vulnerable to several threats to

internal validity, including maturation, history and regression to the mean. Overall, we would

assess the strength of this body of research as no greater than a three on the five-point Maryland

Scientific Methods Scale (Sherman and Gottfredson, 1997), with methodology that is rigorous in

some respects and weak in others (see Worden, et al., 2012a).

The one study that does not find positive effects (Worden, et al. 2012a) was based on a

stronger design, providing for a control group matched on sex, race, academy class, and pre-

intervention complaints. Based on a panel analysis, the authors find that the EIS intervention had

no effects on rates of personnel complaints, uses of force, or secondary arrests. Moreover, the

authors find that officers who were subject to EIS intervention made fewer arrests (particularly

proactive arrests) after the intervention (and compared to the matched controls), indicating that

the intervention had the unintended effect of deterring officers selected from being proactive.

11
 In addition to a weak knowledge base on their effectiveness, neither research nor

professional consensus offers guidance about EIS structure. There is currently no agreed-upon

number of performance indicators to include in EI systems (though some are recommended; see

Walker, 2003b), nor have standards been developed as “best practices” to identify officers with

performance problems. Some systems rely on selection criteria that include only citizen

complaints, while others use multiple indicators, including complaints, use of force reports,

involvement in civil litigation, and violations of administrative rules to identify officers whose

performance warrants intervention (Walker, et al., 2000). But whether they use one or multiple

indicators, most systems are simple “time-and-numbers” systems that provide for numerical

thresholds of specified events over specified time frames – e.g., three complaints over a twelve-

month period – as the criterion by which officers are selected for intervention (Walker, et al.,

2000, p. 4.4). No consensus exists on the proper thresholds, and the validity of the predictions on

which these systems are implicitly based – that is, that an individual officer with, say, three

complaints within twelve months will subsequently display a pattern of behavior that constitutes

a problem, including further complaints, warranting preventive intervention – has not been

established by empirical research.7

One would be appropriately skeptical of such predictions. First, phenomena that occur

with such a low frequency are notoriously difficult to predict with a high level of accuracy (see,

e.g., Gottfredson 1987). Second, as noted above, the events on which the predictions are based

are ambiguous as indicators of police misconduct. Police work involves the use of force on

occasion, even when the work is skillfully performed, and citizen complaints could be expected

to sometimes arise even when police officers do their jobs properly. Third, widely used “time-

and-numbers” systems typically do not take into account other factors, such as the nature of

12
officers’ assignments, which may affect the probability that they will have occasion to use force,

or the likelihood that their actions will be the subjects of complaints. While it is probably true

that the best prediction of behavior during any given time period is based on behavior during the

immediately preceding time period, the predictive mechanisms on which most EI systems are

based have not been validated by empirical research and likely leave some room – much room,

we think – for improvement.

EI systems that rely on such crude selection criteria may be needlessly costly. While EI

systems are designed to be non-punitive and operate outside the police disciplinary system more

generally, and although errors in the form of selecting for intervention officers who do not in fact

require it (the false positives) are inevitable, large numbers of false positives can have important

implications for police organizations. First, resources are wasted insofar as interventions are

applied to personnel who do not need them. Second, officers subject to an EIS intervention may

suffer an informal stigma in their selection, and so the officers who are false positives carry the

burden of such a stigma unnecessarily. Third, and perhaps most importantly, a moderate to high

rate of false positives is likely to reduce the legitimacy of an EIS in the eyes of line officers and

the immediate supervisors who are responsible for them. If officers see peers or subordinates

who, in their estimation, are solid performers and who are nevertheless selected for EIS

intervention, then they are likely to see the system as broken, and that would adversely affect the

potential of such a system for organizational reform.8

What We Should Learn about Risk Assessment

The criminal justice field knows something about how offenders can be assessed for their

risk of recidivism, and about how interventions can be applied with cognizance of the forces that

affect that risk. Efforts to assess and manage police officers’ risk of misconduct might benefit
13
from a closer examination of the lessons learned in the correctional field. So we next review the

development and use of risk assessment instruments in corrections, and then we derive from that

review some clues about promising directions for the further development of EI systems and for

research on police misconduct.

Correctional Risk Assessment

A key principle of modern risk assessment is that such assessments should be actuarial in

nature – that is, they should be “structured, quantitative, and empirically linked to a relevant

criterion” (Bonta, 2002, p. 356). Best practice in correctional risk assessment has progressed

from a reliance on clinical or professional judgments of risk to actuarial assessments based on a

set of risk factors that is evidence-based; each factor bears a demonstrated association with

recidivism, and these factors are combined to form an index or scale that predicts recidivism

more accurately than any one factor does (see Bonta and Andrews, 2007, pp. 3-4). The

combination can be a simple summation of binary factors, or the factors might be assigned

weights that reflect their relative predictive values. The numerical scores on the scale, which

signify offenders’ likelihoods of reoffending, can be transformed into ordinal categories for

application, such as low-, medium-, and high-risk categories. The instruments are validated by

applying them to “validation samples” of offenders, on whom data on the risk factors and

recidivism are retrospectively collected and analyzed to evaluate predictive accuracy. (In some

cases, large samples are drawn and then randomly separated into “estimation” and “validation”

groups [Schwalbe, 2009].) In general, actuarial assessments of risk outperform clinical

judgments in that the actuarial instruments better predict criminal behaviors and better

differentiate between low- and high-risk offenders (Grove and Meehl, 1996), although clinicians

sometimes resist the application of actuarial instruments.

14
 Consider, for example, the Salient Factor Score (SFS). The SFS is based mainly on

information that can be found in an offender’s file, such as the number of prior convictions, the

number of prior incarcerations, age at first commitment, the nature of the commitment offense,

and previous violations of probation or parole. An early version of the SFS provided for scores

ranging from 0 (for the worst risks) to 11 (for the best risks). The U.S. Parole Commission used

versions of the SFS for more than twenty years in making parole selection and revocation

decisions (Hoffman, 1994; Hoffman and Beck, 2004). Research on the SFS, using several

different samples of parolees, has found correlations with recidivism from .27 up to .41

(Hoffman, et al., 1978; Hoffman, 1983; Hoffman and Beck, 1984). To illustrate: in a three-year

follow-up of 1971 releasees, Hoffman, et al. (1978) found that 83 percent of the “very good”

risks (with scores of 9 to 11) succeeded in the sense that they were not arrested for a new

offense, compared with 62 percent of the “good” risks (scores of 6 to 8), 43 percent of the “fair”

risks (scores of 4-5), and 29 percent of the “poor” risks (scores of 3 or less).

Actuarial risk assessment instruments such as the SFS represent a second generation of

assessment tools and an advance over the first-generation clinical judgments (see Bonta and

Andrews, 2007). Although they are reasonably successful in predicting recidivism, these

instruments rely predominantly on risk factors that are static and immutable (e.g., criminal

history), and thus of no utility in guiding rehabilitative interventions. Some advocates argued

that risk assessment instruments should be useful not only for predicting offenders' recidivism

but also for informing rehabilitation planning (Heilbrun, 1997), giving rise to another generation

of risk assessment instruments.

A new principle of risk assessment emerged, which stressed that assessment of risk must

be tied to risk reduction (Andrews, et al., 1990). This principle of risk emphasized that the most

15
intensive treatment should be directed toward those offenders with the highest probability of

recidivism, since research demonstrated that intensive interventions for low-risk offenders can

actually increase recidivism (Lowenkamp and Latessa, 2004). Moreover, this new principle

stressed targeting criminogenic needs, which are crime-producing factors strongly associated

with risk, but which can change over time. Such needs are termed dynamic risk factors in order

to differentiate them from immutable, static risk factors. Dynamic risk factors (e.g., antisocial

attitudes, substance abuse) have been incorporated into the newer third-generation risk

assessment instruments, both to allow offender risk scores to change and also to assist

correctional staff charged with managing offender risk to identify targets for intervention.

Research has shown that changes in dynamic risk factors are associated with changes in

recidivism, and successfully addressing these dynamic risk factors can contribute to an offender's

reduction in risk (Bonta, 2002).

We would note that while third-generation instruments more readily allow those charged

with managing risk to target dynamic factors associated with continued offending, they are much

more time and resource intensive to administer than second-generation instruments. For

example, one of the most commonly used third-generation instruments in correctional settings is

the Level of Service Inventory-Revised (LSI-R). This instrument requires an in-person interview

which can take between 45 minutes and 2 hours to administer, and not every correctional

institution has the resources to administer such a lengthy assessment, which is why several

second-generation instruments are still in use today. Some agencies have taken to using second-

generation risk assessments as screening tools to sort offenders into categories of risk, then

focusing lengthier third-generation assessments on those offenders deemed to be highest risk as a

16
way to direct scarce resources to those who would presumably benefit from them the most

(Latessa and Lovins, 2010).

We would further note that the third-generation risk assessment instruments are based on

a large volume of research. Indeed, Andrews, et al. (2006, p. 8) characterize the “theoretical,

empirical and applied progress within the psychology of criminal conduct” as “nothing less than

revolutionary,” and this body of research was “crucial to effective correctional treatment.” By

comparison, the theoretical, empirical, and applied progress in explaining police misconduct has

been much less impressive and far short of revolutionary.

Lessons for Policing

From the experience with risk assessment in corrections we would draw three broad

lessons for policing: (1) intervention is based on predictions of future performance (even if the

predictions are based partly or entirely on past performance); (2) extant research provides a

foundation for the construction of risk assessment instruments that are likely to prove superior to

the assessments that contemporary EIS selection criteria effect; (3) further improvements in risk

assessment will turn on further progress in understanding police misconduct.

Intervention is Based on Predictions. First, it is important that we acknowledge the

predictive exercise that inheres in the operation of EI systems, so that we can take steps to assess

the validity of the predictions and to improve their accuracy. As we indicated above, EI systems

make implied predictions by identifying officers who would presumably engage in (further)

misconduct in the absence of an intervention – officers at high(er) risk. Where they are based on

explicit, statistical criteria in the form of specified indicators and associated thresholds, the EIS

assessments are in a limited sense actuarial rather than merely clinical, though EIS selection

through the “informal performance review” that is part of the EI systems in many agencies (one-

17
third of those surveyed by Walker, et al., 2000, p. 4.4) is unstructured and clinical in nature.

However, even the quantified EIS selection procedures are based on intuitive professional

judgments about thresholds and not on scientific evidence about the factors associated with

police misconduct, and so they tend to be simplistic. Indeed, given the ambiguity of the

indicators, the conventional time-and-numbers selection might be inferior to unstructured clinical

judgments.

The predictive accuracy of current EIS selection criteria is not known, in general, as

hardly any research has been undertaken for the purpose of validating assessments of misconduct

risk. In the only analysis of its kind to our knowledge, Worden, et al. (2003, 2012c) tested the

predictive accuracy of conventional time-and-numbers selection criteria using longitudinal data

on multiple indicators from one large agency. Simulating the application of various EIS

selection criteria and assessing their predictive performance against composite, two-year indices

of (later) problem behavior, they found that the customary EIS criteria (e.g., three complaints in

twelve months) generated what appeared to be a large proportion of false positives, as well as

false negatives. These patterns held (among the more than 7,000 officers included in the

analysis) both with and without the 124 officers who had been through the EIS intervention.

Adjusting thresholds up or down served to strike different balances between false positives and

false negatives, of course, but each of the conventional time-and-numbers systems generated

large numbers of errors.

Along similar lines, and using a subset of the same data, Harris (2012) formed a cohort of

officers who entered the department between 1987 and 1990 to follow from the start of their

careers until the data collection period ended in 2001. Using the concepts from the criminal

career paradigm of the residual number of offenses (RCO) and residual career length (RCL),

18
Harris estimated the accuracy of applying the typical EIS selection criterion of three personnel

complaints in a year to predict whether or not these officers would be in high RCO or RCL

groups compared to those who did not meet that criterion. His results showed that the predictive

accuracy was very low.

In their study of a large southeastern police department’s experience with an EIS, Lersch,

et al. (2006) found that for the calendar year 2000, officers selected by the EIS were younger,

less experienced, and made more arrests, filed more use of force reports, and used higher levels

of force when compared to non-selected officers. These results suggest that officers who are

flagged may represent a large number of false positives. That is, because the EIS did not take

assignment or productivity measures into account, it flagged for intervention officers who are

generally younger and more proactive, but who are not necessarily officers in need of counseling

or retraining. Also of note is that while the EIS identified and selected officers on a quarterly

basis, of the 33 officers identified, 27 (82 percent) were identified in only one time period. While

this study included data for only a single year, its findings are consistent with the expectation

that officers who are identified as problematic in one time period will not be problematic in

others, even in the absence of some intervention. In addition, Bazley, et al. (2009), using the

same data, found that the frequency and intensity of use of force reports were predictive of which

officers were selected by the department’s EIS. However, when examining the force used by

officers relative to the resistance of suspects, the authors found that officers who used lower

levels of force to handle higher levels of suspect resistance were more likely to be identified by

the EIS. Among those officers who tended to use more force than called for relative to suspect

resistance, none was identified by the EIS. Again, these results call into question the ability of EI

19
systems to properly flag officers who are in need of intervention, and points to difficulties when

these systems simply rely on a “time-and-numbers” approach.

Build on Extant Research. Extant research findings point to risk and protective factors

that could be incorporated into risk assessment instruments that are likely to perform better than

current EI systems in identifying the officers at the highest risk of misconduct, who are the most

suitable candidates for an intervention of some kind. Contemporary EI systems are rather crude

risk assessment mechanisms compared with the instruments – even the second-generation

instruments – used in correctional systems to assess offenders’ risk of recidivism. There are

limits to how closely officer risk assessment can emulate offender risk assessment, to be sure.

But insofar as police occupational deviance parallels other forms of deviance, and insofar as

correctional risk assessment serves purposes of both prediction and intervention (and not

punishment9), the structure and operation of EI systems and managing misconduct risks probably

can be improved by better emulating correctional risk assessment.

For example, by focusing on only the last three to twelve months’ of police performance,

EI systems discard historical information on police behavior that is probably of some – perhaps

much – predictive value; histories of complaints, use of force, etc., could be readily incorporated

into officer risk assessment. Corroborating this supposition, Harris (2012) found that the

prediction of membership in the high RCO or RCL group could be improved by taking into

account officers’ entire complaint histories, instead of relying on only the most recent year of

complaint data. For another example, no information on pre-service, background factors or on

academy performance is incorporated into the risk assessment that an EIS performs, even though

the limited empirical evidence now available (and reviewed above) points toward several risk

factors that appear to have predictive value.10 Histories of arrest, and/or of disciplinary action or

20
termination in jobs that preceded police employment, are empirically established risk factors and

they might add to the predictive accuracy of risk assessment instruments. Length of service is an

empirically demonstrated protective factor, and it too would almost surely improve the predictive

accuracy of EIS selection. Length of service at the first sustained complaint might even add to

predictive success. Thus it may well be possible to improve the accuracy of the predictions on

which EI systems rest even in advance of revolutionary progress in research on police

misconduct. A risk assessment instrument that includes histories of risk-related outputs such as

citizen complaints and administrative investigations, length of service, and pre-service arrest or

employment discipline or termination might form a viable second-generation instrument that is

feasible and economical, not unlike the SFS in correctional settings.

In this connection we would add that the management of individual officers’ risk for

misconduct need not (and should not) turn only on an EIS. As in a community correctional

setting, where offenders with different risk scores are subjected to supervision (e.g., by probation

officers) of different intensity, so too might officers of varying risk be subjected to supervisory

oversight of varying kind and intensity. For an officer whose assessed risk for misconduct is

high, supervisors could be mandated to closely scrutinize the officer’s reports of arrests and use

of force, observe a minimum number of the officer’s interactions with citizens, and in other

instances to follow up afterwards with citizens with whom the officer had contact to hear the

citizens’ reports of their subjective experiences. One agency in which we have done research

requires supervisors to complete Service Quality Control Reports on all officers, with additional

SQCRs for officers assessed – albeit only clinically – as higher-risk.

The Need for Further Research. If risk assessment and risk management in policing is to

improve along the lines that we outline above, then much of that improvement will be based on

21
research that has not yet been done. We describe the contours of that research here. Research on

criminal behavior, and especially research on the psychology of criminal conduct that Andrews,

et al. (2006, p. 9) characterize as “a key component of the fast-growing area of a justice and/or

forensic psychology,” has been decades in the making. Police research has a long way to go.

That will require not only that researchers – whether they are based in universities, research

institutes, or law enforcement agencies –address these research questions, but also that police

agencies cooperate in granting access to data and in allowing the dissemination of findings.

The indicators and associated thresholds that are parts of current EI systems remain

largely untested, such that the extent and the limits of their risk assessment value is not known;

none of the EIS selection protocols has been validated. One of the foremost challenges in

conducting such analysis is the construction of the criterion variable – the outcome that risk

assessment predicts. Offender risk assessments have been tested against the criterion of

recidivism – e.g., rearrest, or reconviction. Measures of recidivism based on the outcomes of

enforcement and/or adjudication share some flaws with readily available measures of police

misconduct: they capture only the behaviors that come to the attention of authorities and are

recorded by them; the measures that are based on a higher evidentiary standard, and therefore

exclude (most) occasions on which the subject is incorrectly accused, provide a still more

deflated count by excluding some occasions on which the accusations are not incorrect.11 But for

all of the reasons that we discussed above, in a consideration of the ambiguity of the indicators,

measures of misconduct suffer from flaws that are much greater than those that afflict measures

of recidivism. We would not purport to be able to offer a definitive solution to this problem here

and now. It will probably involve some degree of triangulation, and it will surely entail some

22
mechanism to control for officers’ varying levels of enforcement activity (see, e.g., Worden, et

al., 2012b).

We would surmise that such research would be facilitated by descriptions of the cross-

sectional and longitudinal patterns of performance – across officers and over time – and the

associations among those performance indicators, and further examination of the relationship

between indicators of risk-related behavior and indicators of police activity, which would be of

particular value. We need to know to what degree counts of risk-related outputs are a function of

activity, and the conditions under which that function varies. We also need to ascertain whether

officers whose counts of risk-related outputs are high are simply very active or practice an

operational style that contributes to both their productivity and their frequency of complaints

(Terrill and McCluskey, 2002). Such research would be vital in determining the validity of

various indicators of police misconduct, and would help to separate the smoke of misconduct

from the exhaust of routine policing.

With acceptable measures of criterion variables, inquiry could assess the relative

accuracy of contemporary EIS selection mechanisms and the alternatives, predictive models that

take better advantage of the information available to police and that take better account of the

longitudinal patterns and risk and protective factors. Extant research leaves the predictive

accuracy of EIS selection very much in doubt. Moreover, research has yet to determine the

extent to which the prediction of future misconduct could be enhanced by taking advantage of

longer histories of misconduct indicators, or by the inclusion of officer characteristics such as

length of service, age at appointment, educational attainment, or pre-employment histories of

arrest. Such research is required to determine what indicators or characteristics have sufficient

predictive value, how the indicators should be combined, as well as what thresholds (or range of

23
thresholds, or index score) are optimal for classifying an officer as high-risk for possible

intervention. Some research of this kind would parallel correctional research on risk factors

using “estimation” samples to estimate the strength of the relationships between individual risk

factors and subsequent misconduct – the criterion variable – and other research would test

predictions of misconduct using other “validation” samples.

Much more research is required on the nature of officers’ performance problems, the

psycho-social factors that give rise to those problems, and the modes of intervention that have

better chances of success in addressing those factors. This is the long-term research agenda that

might allow for the development of third-generation instruments in policing that serve the dual

purposes of assessing risk and assessing needs. It is important to note that most high-risk

offenders are not at high risk for recidivism because they have one risk or need factor, but rather

because they have multiple factors (Latessa and Lovins, 2010). In short, different offenders have

a different collection of needs, which call for specifically targeted interventions. We have reason

to believe that even among officers at high risk of misconduct, different officers have different

problems – that is, their misconduct may take different forms – and/or they would have different

needs that call for different interventions. For example, Scrivner (1994) discovered five groups

of officers among those referred to police psychologists due to their use of excessive force,

including: officers with personality disorders; officers whose job-related experiences – e.g.,

traumatic incidents such as police shootings – put the officers at risk for abusing force; young

and inexperienced officers who were also “highly impressionable and impulsive”; officers who

developed inappropriate patrol styles; and officers with personal problems. Whether risk

assessment and needs assessment should be performed with the same or different instruments

24
(see Gottfredson and Moriarty, 2006), risk management should take structured account of

different needs and apply interventions accordingly.12

The basic research on which risk assessment in policing is most likely to capitalize is

research that develops individual-level explanations for misconduct, focusing on officers’

characteristics, backgrounds, experiences, and attitudes. Previous research has not neglected

these kinds of variables (see National Research Council, 2004, chapter 4), but from the

unexplained variation in behavioral outcomes one might reasonably infer that there is much that

remains to be learned; moreover, research has often dwelled on behavioral constructs that do not

differentiate proper from improper behavior (e.g., on the use of force without distinguishing the

misuse of force from the appropriate use of force). Hints about the explanatory promise of

individual variables can be found in several studies. Worden (1995a) found that officers with

more negative attitudes toward citizens are more likely to use force improperly. More recently,

analyzing attitudes as a bundle rather than one by one, Terrill, et al. (2003) found that officers

whose occupational attitudes conform more closely to the tenets of the traditional police culture

are more prone to the use of their coercive authority. Similarly, Paoline and Terrill (2005) found

that such officers are more likely to conduct searches during traffic stops. Further efforts to

uncover the individual factors that affect forms of police misconduct – whether it is the abuse of

physical force or improper searches or even simple discourtesy – are investments that may yield

dividends of improved prediction and also better-guided intervention.

Conclusions

Managing the risk of police misconduct does not, need not, and should not turn only on

the operation of an early intervention system. Officers’ conduct can be and surely often is

regulated informally by peers and supervisors. As Stephens (2011) observes, it is not unusual for

25
officer’s peers to be aware of a pattern of misconduct long before the officer’s behavior results in

discipline or is detected by an EIS; the earliest intervention can come from peers and first-line

supervisors. The failures of such regulation, however, have been scandalous and dramatic at

times, and although the cohesiveness of “the” police culture has almost certainly been overstated

in the past, it remains true that police work by its nature generates strong centripetal forces

among officers – and supervisors – that can impede functional regulation of performance on the

street. Organizational culture and context are important parts of managing misconduct, but even

so, formalized systems for assessing and managing the risks of misconduct have a place in

contemporary police administration, and some version of an EIS would be a major component of

such a system.

The successful operation of an EIS, however, rests to a significant degree on its capacity

to identify officers at genuinely high risk of continued misconduct in the future, and to

discriminate them from officers who are not (but who may be active and productive). Errors in

prediction are inevitable, but they should be minimized, and informed decisions should be made

at any level of predictive accuracy about the desired balance of false positives and false

negatives. False positives – officers erroneously identified as problems requiring intervention –

result in wasted intervention resources, inappropriately labeled officers, and perhaps eroded

legitimacy of the EIS and even the risk management function more generally. Furthermore, it

may be in policing, as in corrections, that intervening with low-risk subjects is not merely

wasteful but counterproductive. A high level of predictive accuracy is very important in

operating an EIS or any such risk management mechanism.

We explained the reasons that should make one skeptical of the predictive accuracy of

commonly used EIS selection criteria. We believe that contemporary provisions for assessing

26
the risks of police misconduct leave substantial room for improvement – improvements whose

benefits would ripple through the risk management process. Some steps, such as extending the

time periods over which performance data are accumulated to capture more (or all) of officers’

histories, and the incorporation of information on risk and protective factors in officers’

personnel files, could be taken fairly quickly. Other steps will await the generation of empirical

evidence on patterns of misconduct, including the nature of the relationships between

performance indicators and enforcement activity, on the predictive accuracy of alternative risk

assessment tools, and on the well-springs of various forms of misconduct. These are important

issues on which progress can be made through collaborative partnerships between researchers

and police managers.

27
 References

Alpert, G.P., and Dunham, R.G. (1997), The Force Factor: Measuring Police Use of Force
Relative to Suspect Resistance, Police Executive Research Forum, Washington.

Andrews, D. A., Bonta, J., and Hoge, R. (1990), “Classification for effective rehabilitation:
Rediscovering psychology,” Criminal Justice and Behavior, Vol. 17, pp. 19–52.

Andrews, D.A., Bonta, J., and Wormith, J.S. (2006), “The recent past and near future of risk
and/or need assessment,” Crime & Delinquency, Vol. 52, pp. 7-27.

Bazley, T.D., Mieczkowski, T., and Lersch, K.M. (2009), “Early intervention program criteria:
Evaluating officer use of force,” Justice Quarterly, Vol. 26, pp. 107-124.

Bobb, M.J., and staff (2009), Los Angeles County Sheriff’s Department: 27th Semiannual Report,
Police Assessment Resource Center, Los Angeles.

Bonta, J. (2002), “Offender risk assessment: Guidelines for selection and use,” Criminal Justice
and Behavior, Vol. 29, pp. 355-379.

Bonta, J., and Andrews, D.A. (2007), Risk-Need-Responsivity Model for Offender Assessment
and Rehabilitation, Public Safety Canada, available at
http://www.publicsafety.gc.ca/res/cor/rep/risk_need_200706-eng.aspx (accessed
September 4, 2012).

Brandl, S.G., Stroshine, M.S., and Frank, J. (2001), “Who are the complaint-prone officers? An
examination of the relationship between police officers’ attributes, arrest activity,
assignment, and citizens’ complaints about excessive force,” Journal of Criminal Justice,
Vol. 29, pp. 521-529.

Brown, M.K. (1981), Working the Street: Police Discretion and the Dilemmas of Reform,
Russell Sage, New York.

City of Pittsburgh Citizen Police Review Board (1998), Rules and Operating Procedures, Author,
available at http://www.city.pittsburgh.pa.us/cprb/assets/cprbrules.pdf (accessed
November 30, 2012).

Cohen, B., and Chaiken, J.M. (1972), Police Background Characteristics and Performance,
RAND Corporation, New York.

Commission to Investigate Allegations of Police Corruption and the Anti-Corruption Procedures

of the Police Department (1994), Commission Report, City of New York, New York.

Cuttler, M.J., and Muchinsky, P.M. (2006), “Prediction of law enforcement training performance
and dysfunctional job performance with general mental ability, personality, and life
history variables,” Criminal Justice and Behavior, Vol. 33, pp. 3-25.

28
Davis, R.C., and Mateu-Gelabert, P. (1999), Respectful and Effective Policing: Two Examples in
the South Bronx, Vera Institute of Justice, New York.

Fyfe, J.J. (1979), “Administrative interventions on police shooting discretion,” Journal of

Criminal Justice, Vol. 7, pp. 309-323.

Fyfe, J.J. (1988a), “Police use of deadly force: Research and reform,” Justice Quarterly, Vol. 5,
pp. 165-205.

Fyfe, J.J. (1988b), The Metro-Dade Police/Citizen Violence Reduction Project: Final Report,
Police Foundation, Washington.

Fyfe, J. J., and Kane, R. (2005), Bad Cops: A Study of Career-Ending Misconduct among New
York City Police Officers, Report to the National Institute of Justice, John Jay College of
Criminal Justice, New York.

Garner, J.H., Maxwell, C.D., and Heraux, C. (2002), “Characteristics associated with the
prevalence and severity of force used by the police,” Justice Quarterly, Vol. 19, pp. 705-
746.

Goldstein, H. (1977), Policing a Free Society, Ballinger, Cambridge, Mass.

Gottfredson, S.D. (1987) “Prediction: An overview of selected methodological issues,” in

Gottfredson, D.M. and Tonry, M. (eds.), Prediction and Classification in Criminal
Justice Decision Making, Vol. 9 of Crime and Justice: A Review of Research, University
of Chicago Press, Chicago.

Gottfredson, S.D., and Moriarty, L.J. (2006), “Statistical risk assessment: Old problems and new
applications,” Crime & Delinquency, Vol. 52, pp. 178-200.

Grant, J.D., and Grant, J. (1996), “Officer selection and the prevention of abuse of force,” in
Geller, W.A., and Toch, H. (eds.), Police Violence: Understanding and Controlling
Police Abuse of Force, Yale University Press, New Haven, CT.

Greene, J.R., Piquero, A.R., Hickman, M.J., and Lawton. B.A. (2004), Police Integrity and
Accountability in Philadelphia: Predicting and Assessing Police Misconduct, Report to
the National Institute of Justice, Temple University, Philadelphia.

Grove,W. M., and Meehl, P. E. (1996), “Comparative efficiency of informal (subjective,

impressionistic) and formal (mechanical, algorithmic) prediction procedures: The
clinical-statistical controversy,” Psychology, Public Policy, and Law, Vol. 2, pp. 293-
323.

Hannah-Moffat, K. (2012), “Actuarial sentencing: An ‘unsettled’ proposition,” Justice

Quarterly, in press.

29
Harmon, R. (2009), “Promoting civil rights through proactive policing reform,” Stanford Law
Review, Vol. 62, pp. 1-68.

Harris, C. (2009a), “Police use of improper force: A systematic review of the evidence,” Victims
& Offenders, Vol. 4, 25-41.

Harris, C. (2009b), “Exploring the relationship between experience and problem behaviors: A
longitudinal analysis of officers from a large cohort,” Police Quarterly, Vol. 12, pp. 192-
213.

Harris, C. (2010), “Problem officers? An analysis of problem behavior patterns from a large
cohort,” Journal of Criminal Justice, Vol. 38, pp. 216-225.

Harris, C. (2011) “Problem behaviors in the later portion of officers’ careers,” Policing: An
International Journal of Police Strategies and Management, Vol. 34, pp. 135-152.

Harris, C. (2012), “The residual career patterns of police problem behaviors,” Journal of
Criminal Justice, forthcoming.

Hassell, K., and Archbold. C. (2010), “Widening the scope on complaints of police misconduct,”
Policing: An International Journal of Police Strategies & Management, Vol. 33, pp. 473-
489.

Heilbrun, K. (1997), “Prediction versus management models relevant to risk assessment: The
importance of legal decision-making context,” Law and Human Behavior, Vol. 21, pp.
347-359.

Hickman, M. (2008), “On the context of police cynicism and problem behavior,” Applied
Psychology in Criminal Justice, Vol. 4, pp. 1-44.

Hoffman, P.B. (1983), “Screening for risk: A revised Salient Factor Score (SFS 81),” Journal of
Criminal Justice, Vol. 11, pp. 539-547.

Hoffman, P.B. (1994), “Twenty years of operational use of a risk prediction instrument: The
United States Parole Commission’s Salient Factor Score,” Journal of Criminal Justice,
Vol. 22, pp. 477-494.

Hoffman, P.B., and Beck, J. L. (1984) “Burnout – age at release from prison and recidivism,”
Journal of Criminal Justice, Vol. 12, pp. 617-623.

Hoffman, P.B., and Beck, J. L. (2004) The United States Parole Commission’s Expedited
Revocation Procedure, U.S. Parole Commission Research Unit Report Forty-five, United
States Parole Commission, Washington.

Hoffman, P.B., Stone-Meierhoefer, B., and Beck, J.L. (1978), “Salient Factor Score and release
behavior: Three validation samples,” Law and Human Behavior, Vol. 2, pp. 47-63.
30
Independent Commission on the Los Angeles Police Department (1991), Report of the
Independent Commission on the Los Angeles Police Department, Author, Los Angeles.

Latessa, E.J., and Lovins, B. (2010), “The role of offender risk assessment: A policy maker
guide,” Victims and Offenders, Vol. 5, pp. 203-219.

Lersch, K.M. (2002), “Are citizen complaints just another measure of officer productivity? An
analysis of citizen complaints and officer activity measures,” Police Practice and
Research, Vol. 3, pp. 135-147.

Lersch, K.M., Bazley, T., and Mieczkowski. T. (2006), “Early intervention programs: An
effective police accountability tool, or punishment of the productive?” Policing: An
International Journal of Police Strategies & Management, Vol. 29, pp. 58-76.

Lowenkamp, C. T., and Latessa, E. J. (2004), “Understanding the risk principle: How and why
correctional interventions can harm low-risk offenders,” In Topics in Community
Corrections (pp. 3–8). Washington, DC: Department of Justice.

Macintyre, S., Prenzler, T., and Chapman. J. (2008), “Early intervention to reduce complaints:
An Australian Victoria Police initiative,” International Journal of Police Science and
Management, Vol. 10, pp. 238-250.

Muir, W.K., Jr. (1977), Police: Streetcorner Politicians, University of Chicago Press, Chicago.

National Research Council (2004), Fairness and Effectiveness in Policing: The Evidence,
Committee to Review Research on Police Policy and Practices, Wesley Skogan and
Kathleen Frydl (eds.), Committee on Law and Justice, Division of Behavioral and
Social and Behavioral Sciences and Education, National Academies Press, Washington,
DC.

Paoline, E.A. III, and Terrill, W. (2005), “The impact of police culture on traffic stop searches:
An analysis of attitudes and behavior,” Policing: An International Journal of Police
Strategies and Management, Vol. 28, pp. 455-472.

Schwalbe, C. S. (2008), “Risk assessment stability: A revalidation study of the Arizona

risk/needs assessment instrument,” Research on Social Work Practice, Vol. 19, pp. 205–
213.

Scrivner, E.M. (1994), Controlling Police Use of Excessive Force: The Role of the Police
Psychologist, Research in Brief, National Institute of Justice, Washington.

Sherman, L. W., and Gottfredson. D. (1997), “Appendix: Methodology for this report,” in
Sherman, L.W., Gottfredson, D., MacKenzie, D., Eck, J., Reuter, P., and Bushway, S.,
Preventing Crime: What Works, What Doesn’t, What’s Promising, University of
Maryland.

31
Stephens, D.W. (2011), Police Discipline: A Case for Change, National Institute of Justice,
Washington.
Terrill, W. (2005), “Police use of force: A transactional approach,” Justice Quarterly, Vol. 22,
pp. 107-138.

Terrill, W., and Mastrofski, S.D. (2002), “Situational and officer-based determinants of police
coercion,” Justice Quarterly, Vol. 19, pp. 215-248.

Terrill, W., and McCluskey. J. (2002), “Citizen complaints and problem officers: Examining
officer behavior,” Journal of Criminal Justice, Vol. 30, pp. 143-155.

Terrill, W., Paoline, E.A., III, and Manning, P.K. (2003), “Police culture and coercion,”
Criminology, Vol. 41, pp. 1003-1034.

U.S. Commission on Civil Rights (1981), Who is Guarding the Guardians? Author,
Washington.

U.S. Department of Justice (2001), Principles for Promoting Police Integrity, Author,
Washington.

Walker, S. (2003a), “The new paradigm of police accountability: The U.S. Justice Department
‘pattern or practice’ suits in context,” St. Louis University Public Law Review, Vol. 22,
pp. 3-52.

Walker, S. (2003b), Early Intervention Systems for Law Enforcement Agencies: A Planning and
Management Guide, Office of Community Oriented Policing Services, Washington.

Walker, S., Alpert, G.P., and Kenney. D.J. (2000), Responding to the Problem Police Officer: A
National Study of Early Warning Systems, Report to the National Institute of Justice,
National Institute of Justice, Washington.

Walker, S., and Milligan. S.O. (2005), Supervision and Intervention within Early Intervention
Systems: A Guide for Law Enforcement Chief Executives, Police Executive Research
Forum, Washington.

Worden, R.E. (1995a), “The ‘causes’ of police brutality: Theory and evidence on police use of
force,” in Geller, W.A., and Toch, H. (eds.), And Justice for All: Understanding and
Controlling Police Abuse of Force, Police Executive Research Forum, Washington.

Worden, R.E. (1995b), “Police officers’ belief systems: A framework for analysis,” American
Journal of Police, Vol. 14, pp. 49-81.

Worden, R.E., and Catlin, S.E. (2002), “The use and abuse of force by police,” in Lersch, K.M.
(ed.), Policing and Misconduct, Prentice Hall, Englewood Cliffs, N.J.

32
Worden, R.E., Pratte, M.A., Catlin, S.E., Harris, C., and Schlief, S.L. (2003), “Problem Officers,
Problem Behavior, and Early Warning Systems,” presented at the Annual Meeting of the
Academy of Criminal Justice Sciences.

Worden, R.E., Kim, M., Harris, C.J., Pratte, M.A., Dorn, S.E., and Hyland, S.S. (2012a),
“Intervention with problem officers: An outcome evaluation of an EIS intervention,”
Criminal Justice and Behavior, published on-line, doi 10.1177/0093854812458095.

Worden, R.E., Harris, C.J., Pratte, M.A., Dorn, S.E., and Hyland, S.S. (2012b), “Problem
officers and problem behavior: Cross-sectional and longitudinal patterns of police
misconduct,” working paper, University at Albany, Albany.

Worden, R.E., Harris, C.J., Pratte, M.A., Dorn, S.E., and Hyland, S.S. (2012c), “Early
intervention for police: Problems of risk assessment and prediction,” working paper,
University at Albany, Albany.

33
 Notes

1
By “misconduct” we mean “any alleged improper or illegal act, omission or decision” by a police officer that
directly affects the person or property of an individual by reason of a violation of any general, standing or special
order or guideline of the police department, a violation of any federal law, state law or municipal ordinance, or “any
act otherwise evidencing improper or unbecoming conduct” (City of Pittsburgh Citizen Police Review Board, 1998,
p. 2).
2
The Department of Justice’s 2001 report, Principles for Promoting Police Integrity, recommends EI systems as a
means for enhancing police accountability, and the Commission on the Accreditation of Law Enforcement Agencies
(CALEA) adopted a standard (1999: Standard 35.1.15) requiring all large agencies to have an EIS.
3
Furthermore, even when it is clear that force is used improperly, it may not be clear what led to its use. Fyfe
(1988b) observes that some improper uses of force are wanton and malicious acts by officers who know better,
while others are misjudgments that stem from lack of adequate training and preparation; at other times, their
judgments might be affected by the emotional intensity of the situation, the level of illumination, or other factors.
These are obviously different kinds of problems calling for different interventions.
4
Research on police use of force – much of which does not concentrate on improper uses of force – report some
mixed findings on the effects of officers’ characteristics on their use of force, with officers age or length of service,
sex, and education emerging as predictors in some studies (Garner, et al., 2002; Terrill and Mastrofski, 2002;
Worden, 1995a).
5
Another study by Harris (2011), which examines a cohort followed from the middle to later portion of their career,
finds that officers in the high-rate trajectory displayed a 26 percent increase in their rates of citizen complaints near
their retirement (between years 17 and 23). Such results suggest that a universal decline for all officers past their
initial peak may not hold across the career course of the most problematic officers.
6
For exceptions, see Cuttler and Muchinsky (2006), Hickman (2008), Terrill, et al. (2003), and Paoline and Terrill
(2005).
7
Walker (2003b, p. 4) argues that EI systems “do not attempt to predict officer performance based on background
characteristics or other factors, rather they indicate that current performance levels, while not warranting
disciplinary action, still warrant improvement.” But the prediction on which EIS intervention is based – that recent
unsatisfactory performance will continue without intervention – is unmistakable.
8
If officers, and the unions that represent them, are skeptical of EI systems, the process whereby officers are
selected for intervention might be one very good reason. There are other reasons, to be sure: insofar as the
intervention is unwelcome, and insofar as it rests on no demonstrable violation of policy, it could be reasonably
perceived as an adverse action that is unjustified. Thus it is all the more important that selection rest on firm ground.
9
The use of risk assessment for the purpose of punishment (e.g., in criminal sentencing) raises a number of thorny
issues; see Hannah-Moffat (2012).
10
Walker (2003b) makes the point that pre-service factors do not suffice to predict performance, and certainly he is
not the first to observe the limited effectiveness of pre-employment psychological screening (see Grant and Grant,
1996). But we need not rely exclusively on background factors to assess and manage the risk of misconduct; we can
take advantage of their predictive value, in conjunction with other data on actual performance.
11
Examining involuntary separation from the agency, Fyfe and Kane’s (2005) outcome is still more restrictive.
12
Mandated counseling by field supervisors may be one intervention with the potential to accomplish this, but
ideally it would be structured.

34