See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/317790929

Determining When to Conduct a Violence Risk Assessment: Development and

Initial Validation of the Fordham Risk Screening Tool (FRST)

Article in Law and Human Behavior · June 2017

DOI: 10.1037/lhb0000247

CITATIONS READS

16 1,308

8 authors, including:

Barry Rosenfeld Charity Wijetunga

Fordham University Fordham University
267 PUBLICATIONS 15,078 CITATIONS 7 PUBLICATIONS 52 CITATIONS

SEE PROFILE SEE PROFILE

Jacqueline Howe Alicia Nijdam-Jones

Fordham University University of Manitoba
7 PUBLICATIONS 159 CITATIONS 30 PUBLICATIONS 474 CITATIONS

SEE PROFILE SEE PROFILE

All content following this page was uploaded by Alicia Nijdam-Jones on 23 June 2017.

The user has requested enhancement of the downloaded file.

Law and Human Behavior
Determining When to Conduct a Violence Risk
Assessment: Development and Initial Validation of the
Fordham Risk Screening Tool (FRST)
Barry Rosenfeld, Melodie Foellmi, Ali Khadivi, Charity Wijetunga, Jacqueline Howe, Alicia Nijdam-
Jones, Shana Grover, and Merrill Rotter
Online First Publication, June 22, 2017. http://dx.doi.org/10.1037/lhb0000247

CITATION
Rosenfeld, B., Foellmi, M., Khadivi, A., Wijetunga, C., Howe, J., Nijdam-Jones, A., Grover, S., &
Rotter, M. (2017, June 22). Determining When to Conduct a Violence Risk Assessment:
Development and Initial Validation of the Fordham Risk Screening Tool (FRST). Law and Human
Behavior. Advance online publication. http://dx.doi.org/10.1037/lhb0000247
 Law and Human Behavior © 2017 American Psychological Association
2017, Vol. 0, No. 999, 000 0147-7307/17/$12.00 http://dx.doi.org/10.1037/lhb0000247

Determining When to Conduct a Violence Risk Assessment: Development

and Initial Validation of the Fordham Risk Screening Tool (FRST)
Barry Rosenfeld and Melodie Foellmi Ali Khadivi
Fordham University Albert Einstein College of Medicine

Charity Wijetunga, Jacqueline Howe, Shana Grover

and Alicia Nijdam-Jones New School for Social Research, New York,
Fordham University New York

Merrill Rotter
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

Albert Einstein College of Medicine

This document is copyrighted by the American Psychological Association or one of its allied publishers.

Techniques to assess violence risk are increasingly common, but no systematic approach exists to help
clinicians decide which psychiatric patients are most in need of a violence risk assessment. The Fordham
Risk Screening Tool (FRST) was designed to fill this void, providing a structured, systematic approach
to screening psychiatric patients and determining the need for further, more thorough violence risk
assessment. The FRST was administered to a sample of 210 consecutive admissions to the civil
psychiatric units of an urban medical center, 159 of whom were subsequently evaluated using the
Historical Clinical Risk Management-20, version 3, to determine violence risk. The FRST showed a high
degree of sensitivity (93%) in identifying patients subsequently deemed to be at high risk for violence
(based on the Case Prioritization risk rating). The FRST also identified all of the patients (100%) rated
high in potential for severe violence (based on the Serious Physical Harm Historical Clinical Risk
Management-20, version 3, summary risk rating). Sensitivity was more modest when individuals rated
as moderate risk were included as the criterion (rather than only those identified as high risk). Specificity
was also moderate, screening out approximately half of all participants as not needing further risk
assessment. A systematic approach to risk screening is clearly needed to prioritize psychiatric admissions
for thorough risk assessment, and the FRST appears to be a potentially valuable step in that process.

Keywords: violence, risk assessment, screening, triage, psychiatric patients

The use of structured instruments that can help guide clinical
decisions about violence risk is well established (Singh, Grann, &
Fazel, 2011; Singh et al., 2014). In some settings, a thorough violence
Barry Rosenfeld and Melodie Foellmi, Department of Psychology, Fordham
University; Ali Khadivi, Department of Psychiatry, Albert Einstein College of
Medicine; Charity Wijetunga, Jacqueline Howe, and Alicia Nijdam-Jones, Depart-
ment of Psychology, Fordham University; Shana Grover, Department of Psychol-
ogy, New School for Social Research, New York, New York; Merrill Rotter,
Department of Psychiatry, Albert Einstein College of Medicine.
The authors express their appreciation to Michael Greenstein and Mat-
thew Grover, who provided input during the preliminary development of
the FRST. In addition, Katarina Furjanić assisted in the initial data collec-
tion phase of this project.
The Fordham Risk Screening Tool (FRST) was designed to provide a
structured, systematic approach to screening psychiatric patients to determine
the need for a more comprehensive violence risk assessment. In a sample of
newly admitted psychiatric patients, the FRST showed high sensitivity in
identifying patients subsequently deemed to be at high risk for violence while
still screening out approximately half of all patients as not needing further
evaluation. This instrument has the potential to help prioritize psychiatric
admissions to optimize institutional resources while still identifying those
patients who may pose a high risk for violent behavior.
Correspondence concerning this article should be addressed to Barry
Rosenfeld, Department of Psychology, Fordham University, Bronx, NY
10458. E-mail: rosenfeld@fordham.edu
1
risk assessment is a required element of the clinical or forensic
evaluation (e.g., determining whether a patient acquitted not guilty by
reason of insanity if suitable for release into the community). In many
settings; however, the decision as to whether a thorough evaluation of
violence risk is necessary is made on a case-by-case basis. Unfortu-
nately, little guidance exists as to how clinicians or administrators
should make the decision to utilize a risk assessment instrument.
Although tools to assess violence risk are readily available, a thorough
violence risk assessment requires considerable time and resources,
both of which are in short supply in most clinical settings (Magen,
Richards, & Ley, 2013; Thomas, Ellis, Konrad, Holzer, & Morrissey,
2009). Viljoen, McLachlan, and Vincent (2010) estimated that a
thorough violence risk assessment requires approximately 15 hours
for a trained evaluator, far exceeding the resources available in most
clinical settings (although a less thorough assessment can certainly be
done more quickly; Smith & White, 2007). Hence, institutions are
likely forced to triage admissions to determine where to focus their
resources. To date, the process of determining when or whether to
conduct a violence risk assessment has been largely unstructured,
untested, and inconsistent, with considerable potential for error
(Foellmi, Rosenfeld, Rotter, & Khadivi, 2014; Heilbrun, Yasuhara, &
Shah, 2010). Specialized risk-screening tools have been developed for
specific types of violence (e.g., intimate partner violence; Rabin,
Jennings, Campbell, & Bair-Merritt, 2009), but there is no empirically
supported systematic approach to helping clinicians decide which
 2 ROSENFELD ET AL.
psychiatric patients are most in need of more thorough violence risk
assessment.
Several published instruments have been described as violence-
screening tools; however, these instruments are more accurately char-
acterized as brief measures that gauge the likelihood of future vio-
lence. Instruments such as the V-RISK-10 (Bjørkly, Hartvig, Heggen,
Brauer, & Moger, 2009), the Clinically Feasible Iterative Classifica-
tion Tree (Monahan et al., 2000; an abbreviated variation on the
Classification of Violence Risk, Monahan et al., 2006), and the
Violence Screening Checklist (McNiel & Binder, 1994) do not
provide a true screening function, which is typically conceptual-
ized as casting a broad net to identify a subgroup of individuals
who require further examination (Pauker & Kassirer, 1980).
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
Rather, these instruments are intended to efficiently differentiate
This document is copyrighted by the American Psychological Association or one of its allied publishers.
higher and lower risk individuals, a goal more consistent with
triage—a rapid approach designed to determine how to prioritize
assessment or intervention resources (Iserson & Moskop, 2007;
Raffle & Gray, 2009). In the context of violence risk, a triage
approach would determine which patients need a violence risk
assessment most urgently. A violence risk-screening instrument,
on the other hand, would identify those patients who need further
evaluation—that is, a comprehensive violence risk assessment.
Hence, screening tools are most effective when the have a very
high degree of sensitivity (i.e., identifying all or most of the
individuals with a designated condition) and a meaningful level of
specificity (i.e., to eliminate a sufficient number of cases as not
needing further attention) or the reverse (near perfect specificity
and an adequate level of positive predictive accuracy). Other
indices of classification accuracy, such as the tool’s positive pre-
dictive value, are less useful in the context of screening because
they are unduly influenced by the base rate of the condition under
investigation (Rosenfeld, Sands, & van Gorp, 2000).
Singh et al. (2012) developed one such instrument, intended for
use with psychiatric patients diagnosed with schizophrenia. Their
algorithm, based on demographic and family history variables, gen-
erated a very high degree of specificity (99%) in differentiating
patients who were violent over a 5-year follow-up period. Although
this approach could (in theory) help identify a subset of patients that
pose little or no risk of future violence, their algorithm screened out a
relatively small proportion of patients (approximately 34%). More-
over, given the focus on actual violence as their outcome measure
(based on a conviction for a violent offense) and their retrospective
approach (using outcome data to identify predictor variables), this
instrument provides limited guidance in identifying individuals in
need of a more thorough violence risk assessment (a function best
described as screening).
In response to the need for a systematic approach to violence risk
screening and following the recommendations of the New York
State/New York City Mental Health-Criminal Justice Panel’s report
(New York State/New York City Mental Health-Criminal Justice
Panel, 2008; our research team created the Fordham Risk Screening
Tool [FRST]). Drawing on face-valid content, the FRST is a flow
chart designed to help clinicians decide which patients need a thor-
ough violence risk assessment. The information considered by the
FRST should, in principle, be readily available either before or during
a standard intake evaluation. However, because clinicians may fail to
ask potentially relevant questions or consider potentially important
information (e.g., Coontz, Lidz, & Mulvey, 1994; Skeem, Manchak,
Lidz, & Mulvey, 2013), we developed the FRST algorithm (including
specific questions and prompts) to guide this process.
Given that the goal of screening is to identify cases in which a
condition might be present (in this case, high risk of future vio-
lence), the criterion that the FRST is designed to predict is a high
risk rating on a well-validated risk assessment instrument. This
criterion is ideal for a screening instrument, given that a wide
range of intervening variables can impact whether an individual
actually engages in violence (e.g., intervention, incapacitation).
Indeed, one result of a thorough risk assessment is the identifica-
tion and implementation of effective risk-management strategies,
hopefully resulting in a lower risk of violence in the future. It
follows that individuals deemed to represent a high risk of future
violence should receive aggressive interventions intended to re-
duce the likelihood of actual violence, whereas individuals deemed
to be low risk may require little or no further intervention (An-
drews, Bonta, & Hoge, 1990; Litwack, 2001). Although risk-
management strategies may attenuate the predictive accuracy of
violence risk assessment tools, it does not impact the assessment of
a screening tool, just as the availability of a treatment for tuber-
culosis would not impact the accuracy of a screening technique
designed to identify cases of tuberculosis. Thus, the focus of
research using the FRST has been to identify that subset of
psychiatric patients that warrant a high risk rating when further
evaluated while simultaneously screening out a meaningful pro-
portion of individuals as not needing further assessment of their
violence risk.
The FRST received preliminary empirical support in an unpub-
lished pilot study of 64 psychiatric inpatients at a large state psychi-
atric hospital in the Bronx, New York (Rosenfeld, Foellmi, Howe, &
Rotter, 2013). In that study, the FRST significantly predicted high risk
ratings based on the Historical Clinical Risk Management-20, version
3 (HCR-20V3; Douglas, Hart, Webster, & Belfrage, 2013), a widely
used violence risk assessment instrument. Importantly, this pilot study
generated very few false negative results, patients who were not
identified by the FRST as needing a comprehensive evaluation but
were deemed to present a high risk of violence based on HCR-20V3
evaluations. Following this pilot study, we made minor modifications
to the instrument (e.g., adding three variables related to the patient’s
current clinical presentation) and initiated a larger trial to evaluate the
utility of this instrument in an acute care psychiatric setting. Specif-
ically, we sought to examine the accuracy of the FRST in identifying
patients thought to present a high risk of violence on the basis of a
thorough violence risk assessment (based on the HCR-20V3 Case
Prioritization summary risk rating). Simultaneously, we sought to
minimize both false-positive classifications (i.e., FRST recommenda-
tions that a risk assessment is needed yet ultimately generating a low
risk rating) and false-negative classifications (high-risk patients who
were not identified by the FRST as needing a risk assessment). Given
the importance of serious violence risk, we also examined the predic-
tive accuracy of the FRST with respect to the HCR-20V3 Severe
Physical Harm risk rating.
Method
Participants
Study participants were a nonoverlapping, consecutive sample
of psychiatric patients (N ⫽ 210) admitted to a large, private,
 VIOLENCE RISK SCREENING
nonprofit hospital in New York City. Most participants were
brought to the hospital by the police (n ⫽ 34, 16.0%) or emergency
medical personnel (n ⫽ 97, 45.5%), and the vast majority of
patients were involuntarily hospitalized (n ⫽ 158, 75.2%). Of
those 210 patients, HCR-20V3 ratings were available for 159
individuals (75.7%); the most frequent cause of missing HCR-
20V3 data was rapid discharge (n ⫽ 36 of 51, 70.1%). There were
no significant differences between individuals for whom the HCR-
20V3 outcome data were available and those screened by the FRST
but not evaluated with the HCR-20V3. Thus, descriptive data
regarding the sample for which both FRST and HCR-20V3 data
were available are summarized here.
The sample included 120 males (57.1%) and 90 females
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
(42.9%), ranging in age from 18 to 68 years old (M ⫽ 37.5, SD ⫽
This document is copyrighted by the American Psychological Association or one of its allied publishers.
14.2). Half of the participants were African American (n ⫽ 108,
53.5%), whereas 62 (30.7%) identified as Hispanic and 17 (8.4%)
as Caucasian, non-Hispanic; 15 individuals (7.4%) were classified
as an other racial/ethnic group, and these data were missing for
eight individuals (3.8%). Education level ranged from 3 to 18
years (M ⫽ 11.7, SD ⫽ 2.6), and approximately half the partici-
pants (n ⫽ 86 55.1%) had a high school education or higher (data
on educational achievement were missing for 57 participants). The
majority of study participants were single/never married (n ⫽ 147,
69.3%), but 21 (9.9%) were married or cohabiting, and 12 (5.7%)
were separated, divorced, or widowed; data regarding marital
status were missing for 32 individuals (15.1%). The most common
types of psychiatric disorders (based on the treatment team’s
discharge diagnoses, contained in the patient’s medical record)
were psychotic disorders, including schizophrenia, schizoaffective
and other psychotic disorders (n ⫽ 129, 60.6%), mood disorders
(n ⫽ 73, 34.3%), substance abuse disorders (n ⫽ 54; 25.4%), and
personality disorder (primarily borderline or antisocial; n ⫽ 12,
5.6%; percentages exceed 100% due to comorbidity).
Instruments
Fordham Risk Screening Tool (FRST). As described above,
the FRST (see Appendix) is an algorithm intended to determine the
need for a comprehensive violence risk assessment in psychiatric
inpatients. Based on face-valid variables that should raise concerns
about the possibility of violence, the FRST classifies an individual
as needing a violence risk assessment when the patient displays
recent and severe violent behavior, threats or ideation. Recent is
operationalized as the preceding 6 months, and severe reflects
behavior, threats, or ideation that has or could plausibly result in
physical harm that requires medical attention. In addition to the
historical FRST risk factors that form the basis of the core algo-
rithm, the tool also elicits clinician ratings of three current risk
factors that should be considered: agitation/hostility, paranoia or
threat/control override symptoms, and refusal of medication.
The FRST received preliminary empirical support from a pilot
study (N ⫽ 64) conducted at a large public psychiatric hospital in
the Bronx, New York (Rosenfeld et al., 2013), which demonstrated
strong interrater reliability and predictive validity. In that study,
the FRST accurately identified 79% (15 of 19) of those patients
who were subsequently rated (by an independent evaluator), based
on the HCR-20V3, as high risk for violence, and 100% of those
identified as having a high risk of severe violence (n ⫽ 8). In the
present study, interrater reliability for FRST ratings was assessed
3
by having two raters code every third assessment (n ⫽ 97). The
intraclass correlation coefficient (two-way mixed model, absolute
agreement) of .81, 95% confidence interval (CI) [.73, .87], indi-
cated excellent interrater reliability (based on interpretive guide-
lines provided by Chichetti, 1994).
Historical Clinical Risk Management-20, version 3
(HCR-20V3). The HCR-20V3 (Douglas, Hart, Webster, & Bel-
frage, 2013) is an empirically supported violence risk assessment
instrument that compiles 20 historical, clinical, and risk manage-
ment factors into a series of structured professional judgments
regarding violence risk. The HCR-20V3 prompts the clinician to
consider which violence risk factors are present and most relevant
to the individual’s violence risk. Specifically, the clinician rates
each item’s presence (rated as present, partially present, or absent)
and relevance (rated as low, moderate, high, or omit); item ratings
can be omitted when insufficient evidence is available to reliably
rate the item. Although higher presence and relevance ratings
typically indicate a greater likelihood of violence, no summary
scores are generated by this instrument (Douglas, Hart, Webster, &
Belfrage, 2013). Instead, the clinician uses risk and relevance
ratings to generate a risk formulation and engage in scenario
planning that are subsequently used to make three summary vio-
lence risk ratings (each of which is rated as low, medium, or high):
overall Case Prioritization, risk of Serious Physical Harm, and risk
of Imminent Violence. Only the Case Prioritization and Serious
Physical Harm risk ratings were utilized in this study.
The HCR-20V3 is the latest iteration of the HCR-20 (Webster et
al., 1997), which has been evaluated in more than 200 empirical
studies (Campbell, French, & Gendreau, 2009; Guy, Douglas, &
Hendry, 2010). When used to assess for future violence risk,
HCR-20 item totals (the sum of presence ratings for the individual
items) and summary risk ratings significantly predict violent behav-
ior, with correlation coefficients ranging from .22 to .28 (Campbell,
French, & Gendreau, 2009). Area under the curve (AUC) estimates
typically range from .60 to .79 (Douglas & Reeves, 2010), with a
median AUC value of .67 across 45 studies (Guy et al., 2010). In the
present study, interrater reliability was established by having indepen-
dent evaluators assess violence risk using the HCR-20V3 in approxi-
mately one fourth of all cases. This procedure yielded strong interrater
reliability, with an intraclass correlation coefficient (two-way, mixed
model, absolute agreement; n ⫽ 54) of .72, 95% CI [.57, .83], for the
Case Prioritization summary risk rating and .63, 95% CI [.44, .77], for
the Serious Physical Harm rating.
Procedures
Following admission to one of the study institution’s three
psychiatric units, patients were interviewed using the FRST by a
study research assistant (all of whom were doctoral students in
clinical psychology). Approximately 1 week after admission, each
patient was also interviewed by a second graduate student using a
structured interview developed for this study to rate the HCR-20V3
and generate risk estimates. Because staffing limitations precluded
interviewing every patient admitted to the institution (particularly
given the labor-intensive interview process used for the HCR-
20V3, typically requiring approximately 4 hours per assessment),
and the three different hospital units had somewhat different
characteristics, we focused on one unit at a time for 2-month
periods, rotating between the three units twice each. This proce-
 4 ROSENFELD ET AL.
dure resulted in approximately 1 year of data collection (although
data collection was suspended briefly during the summer months
when research assistant staffing was less available).
Inclusion criteria for the study were the following: older than 18
years of age, fluent in English, and the absence of extreme cog-
nitive limitations that would preclude meaningful informed con-
sent or reliable data collection. Because the study was imple-
mented as part of a performance improvement initiative, in an
effort to improve violence risk assessments, the hospital and uni-
versity institutional review boards concluded that informed con-
sent was not necessary. Rather, patient assent was deemed appro-
priate, given the need for participation in the clinical interview
used to generate HCR-20V3 ratings. The clinicians who conducted
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
FRST and HCR-20 V3 ratings were supervised by the hospital’s
This document is copyrighted by the American Psychological Association or one of its allied publishers.
senior psychologist and provided written violence risk assessment
reports for the hospital treatment teams. Permission was granted by
the Institutional Review Boards of Fordham University and Bronx-
Lebanon Hospital to analyze the data retrospectively.
Results
The FRST identified 78 of 210 patients (35.9%) as needing a
violence risk assessment based on the primary decision making algo-
rithm. Thirty of these 78 patients (38.5%) were identified solely by the
presence of recent and severe violent behavior, whereas the remainder
were identified through a combination of violent behavior, threats,
and/or thoughts. In addition to cases identified by the core FRST
algorithm, an additional 20 patients (9.2%) were classified by the
evaluating clinician as needing a risk assessment for other reasons,
resulting in a total of 98 of 210 individuals (45.2%) considered to
need a violence risk assessment. Although clinicians did not specify
the precise reason for this decision, an analysis of FRST elements
revealed that the three variables based on the patient’s current clinical
were significantly associated with clinician perceptions that a violence
risk assessment was needed. For example, among the 132 participants
who were not identified by the FRST algorithm (i.e., recent and severe
violent behavior, threats, or ideation), 40 displayed evidence of par-
anoid ideation and 16 of these individuals (40.0%) were considered
(by the evaluating clinician) to need a violence risk assessment, ␹2(1,
N ⫽ 130) ⫽ 26.89, p ⬍ .001, ␾ ⫽ .46. Similarly, of the 23 partici-
pants who displayed agitation during the FRST interview, 10 (43.5%)
were deemed to need a violence risk assessment for other reasons,
␹2(1, N ⫽ 129) ⫽ 18.42, p ⬍ .001, ␾ ⫽ .38. Evidence of medication
noncompliance was more modestly associated with clinician determi-
nations that a violence risk assessment was needed, ␹2(1, N ⫽ 129) ⫽
4.69, p ⫽ .03, ␾ ⫽ .19, because 9 of the 33 individuals (27.3%)
identified as noncompliant with psychotropic medication were
deemed to require a violence risk assessment.
HCR-20V3 ratings were available for 159 of the 210 study
participants (75.7%). Of these, 14 individuals (8.8%) were classi-
fied as high risk based on the HCR-20V3 Case Prioritization rating,
whereas 52 (32.7%) were rated as moderate risk and 93 (58.5%) as
low risk (see Table 1). Thirteen of the 14 high-risk individuals
(92.8%) were identified as needing a violence risk assessment by
the FRST, as were 39 of 52 moderate-risk individuals (75.0%).
Thus, sensitivity was 92.8%, 95% CI [.64, .99], in identifying high
risk individuals, and 78.8%, 95% CI [.67, .88], in differentiating
high- or moderate-risk patients from those rated as low risk. On the
other hand, of the 93 participants (58.5%) classified as low risk on
the HCR-20V3, only 28 were identified as needing a violence risk
assessment by the FRST (i.e., a false-positive rate of 30.1%).
Hence, specificity was 53.8%, 95% CI [.45, .62], in identifying
low- and moderate-risk individuals and 69.9%, 95% CI [.59, .79],
when identifying only low-risk individuals.
Far fewer individuals were rated as high risk for Serious Phys-
ical Harm (n ⫽ 5; 3.1%). Of these, all five were identified by the
FRST as needing a violence risk assessment, as were 31 of 40
individuals (77.5%) rated as moderate risk identified by the FRST
(see Table 2). Thus, sensitivity of the FRST was 100%, 95% CI
[.46, 1.00], in identifying individuals rated as high risk for Serious
Physical Harm, and 80%, 95% CI [.66, .99], in identifying those at
high or moderate risk for Serious Physical Harm. Specificity was
51.3%, 95% CI [.43, .59], in identifying low and moderate risk for
Serious Physical Harm and 61.4%, 95% CI [.52, .70], when
identifying only those individuals deemed to be low risk for
Serious Physical Harm.
Finally, because the presence of paranoia or threat/control over-
ride symptoms was strongly associated with clinician determina-
tions that a violence risk assessment was needed for other reasons,
we examined the utility of this variable in bolstering the core
FRST algorithm. Specifically, we analyzed classification accuracy
when individuals were considered to require a violence risk as-
sessment based on the presence of either recent or severe violent
behavior, threats or ideation combined with the presence of current
paranoia, or threat/control override symptoms. This classification
approach yielded almost identical classification accuracy, identi-
fying 13 of 14 individuals rated as high risk for Case Prioritization
and five of five individuals at high risk for Serious Physical Harm.
When high- or moderate-risk cases were considered as the crite-
rion, this expanded algorithm resulted in sensitivity of 76% for
Case Prioritization and 78% for Serious Physical Harm.
Discussion
Few issues generate as much concern in mental health settings as
the potential for violence. Many expect that mental health profession-
als will be able to identify those individuals who represent a serious
risk of violence and thereby prevent or minimize the occurrence of
violence. Although important advances have occurred in the field of
violence risk assessment, the time and resources needed to adequately
assess whether an individual poses a significant risk of violence are
substantial. Given this dilemma, there is a clear need for an effective
method of screening psychiatric patients to determine where to apply
Table 1
FRST Classifications Based on Case Prioritization Violence
Risk Ratings
FRST decision
HCR-20 case Risk assessment Risk assessment Total
prioritization needed not needed sample
High risk 13 (92.9%) 1 (7.1%) 14 (8.8%)
Moderate risk 39 (75.0%) 13 (25.0%) 52 (32.2%)
Low risk 28 (30.1%) 65 (69.2%) 93 (58.5%)
80 (50.3%) 79 (49.7%) 159
Note. For Fordham Risk Screening Tool (FRST), specificity ⫽ 53.8%
when based on moderate- and low-risk ratings, 69.9% when based only on
low ratings.
 VIOLENCE RISK SCREENING
Table 2
FRST Classifications Based on Serious Physical Harm
Risk Ratings
FRST decision
HCR-20 serious Risk assessment Risk assessment Total
physical harm needed not needed sample
High risk 5 (100%) 0 (0%) 5 (3.2%)
Moderate risk 31 (77.5%) 9 (22.5%) 40 (25.2%)
Low risk 44 (38.6%) 70 (61.4%) 114 (71.7%)
81 (50.6%) 77 (49.4%) 159
Note. For Fordham Risk Screening Tool (FRST), specificity ⫽ 51.3%
when based on moderate- and low-risk ratings, 61.4% when based only on
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
low ratings.
This document is copyrighted by the American Psychological Association or one of its allied publishers.
those scarce resources. The FRST is intended to provide this function
by utilizing a structured, reliable, and objective approach to identify-
ing those psychiatric patients that are most likely to require a further,
more comprehensive assessment.
The results of this study provide preliminary support for the
FRST, demonstrating a very high degree of sensitivity in identi-
fying high-risk individuals. Moreover, the FRST identified more
than 80% of those individuals rated as moderate or high risk.
Simultaneously, the screening process guided by the FRST elim-
inated approximately half of all patients from needing further
evaluation regarding risk of violence. Indeed, even when the
criterion was expanded to include individuals identified as posing
a moderate risk of violence on the summary risk ratings, the FRST
retained sensitivity rates of approximately 80%. Thus, this study
provides strong support for the FRST in differentiating those
psychiatric patients who require a more comprehensive risk as-
sessment from those who do not.
Obviously, determining an adequate level of sensitivity for a
screening instrument is far more complex than simply generating
seemingly strong classification accuracy. Even a single false-negative
result (i.e., the failure to identify an individual who poses a high risk
of violence) could result in severe consequences, both for the indi-
viduals who are the target of the violence as well as the clinicians and
administrative staff who might be held responsible for failing to
prevent the harm. Ideally, a risk-screening instrument would have
perfect sensitivity, but that goal is likely impossible unless virtually all
individuals are screened in. Indeed, the results of this study are
perhaps as close to perfect accuracy as could be hoped for, albeit not
eliminating as many false-positive cases as might be desired. Never-
theless, continued research is necessary to identify additional indica-
tors that might help identify those high-risk individuals who are
missed by the FRST.
Similarly, the question of whether an instrument that screens out
only 50% of the population as not needing further risk assessment
is clinically useful. Given the time and resources needed to con-
duct a violence risk assessment, it may be impractical to evaluate
even half of all admissions to a psychiatric facility. We consider
this to be an administrative decision rather than a research deci-
sion. Just as determinations regarding what level of sensitivity is
considered adequate, administrators and clinicians will need to
determine whether a tool such as the FRST is appropriate for their
setting. These decisions involve weighing the risks and benefits of
not only sensitivity and specificity rates but also the cost of
5
maintaining the status quo. Our anecdotal observations, based on
feedback from clinicians and administrators across the United
States (e.g., in response to conference presentations regarding the
FRST) is that even a 50% reduction in the workload for clinicians
that conduct violence risk assessments is a substantial improve-
ment over the status quo (which often means conducting few, if
any, structured violence risk assessments). Of course, the FRST
need not (and should not) necessarily be considered a final judg-
ment; decisions not to conduct further assessment can always be
revisited if additional information or behavioral changes height-
ened concerns about possible violence.
A related question emerging from these results pertains to the
small decrement in classification accuracy that was observed
when clinician input (in the form of other reasons for a violence
risk assessment) was replaced with a more objective variable
(presence of paranoia or threat/control override symptoms).
Given the goal of increasing the objectivity and transparency in
decision making, reliance on objective sources of data is clearly
desirable, although retaining clinician input is also crucial.
Rejecting a clinician’s opinion that a violence risk assessment is
needed simply because the underlying basis for that opinion is
unclear appears unreasonable. However, the formal integration
of paranoia or threat/control override symptoms into the FRST
algorithm may diminish the frequency with which clinicians
feel the need to supplement the FRST algorithm, resulting in a
simplified (and likely faster) decision-making process. Further
research examining the utility of the FRST should continue to
examine whether this, or other modifications to the FRST,
enhance its predictive accuracy.
Finally, one might question whether violence risk, as mea-
sured by the HCR-20V3, is indeed the appropriate outcome
variable. Although numerous instruments are available to help
clinicians estimate the risk of future violence (Singh et al.,
2014), we chose to focus on an instrument that emphasized
dynamic risk factors and facilitated risk management, rather
than a) unstructured clinical judgment, b) an actuarial risk
assessment instrument, or c) actual violent behavior. In a clin-
ical setting, a primary goal of violence risk assessment is risk
management, and hence, determining whether, when, and how
to intervene is of critical importance. These goals are most
effectively achieved, in our opinion, through the use of an
instrument such as the HCR-20V3. Indeed, this study arose from
clinical practice as an attempt to respond to the challenges
imposed by limited resources, which precluded a systematic
assessment of violence risk for all patients admitted to the
hospital. Hence, our choice of the HCR-20V3 Case Prioritiza-
tion risk rating as the criterion against which to measure the
effectiveness of the FRST was deliberate because our goal was
to determine which patients might need aggressive risk man-
agement, not to warn staff or administrators as to which patients
require greater restrictions (as might result from the use of an
actuarial risk assessment instrument or actual violent behavior).
Despite the impressive predictive accuracy rates observed in this
study, there are a number of limitations that should be considered.
First, the 95% confidence intervals around our sensitivity rates were
relatively wide, suggesting that these results should be considered
preliminary. Without question, further research, with larger sample
sizes, is needed to examine the stability and reproducibility of these
findings across a range of settings. Relatedly, the general psychiatric
 6 ROSENFELD ET AL.
setting generated relatively few high-risk cases and even fewer that
were seen as high risk for violence that would result in serious
physical harm. As the rate of high violence risk cases increases, the
potential for higher error rates (false-negative results) may also in-
crease. Indeed, our unpublished pilot study included substantially
more high risk cases (23 of 63 patients evaluated or 37% vs. 14 of 159
or 8.8% in the present study), and 5 of 63 individuals (7.9%) were
classified as high risk for Serious Physical Harm (compared with 5 of
159 or 3.1% in the present study). Of course, in settings in which risk
of violence is greater (e.g., forensic or correctional facilities), the need
for a screening tool may decrease because the frequency of high-
violence-risk cases is likely high enough to justify a thorough assess-
ment on all patients.
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
Another important limitation in the present study is the
This document is copyrighted by the American Psychological Association or one of its allied publishers.
reliance on highly trained and supervised graduate students
rather than experienced staff members employed by the study
institution. This approach may overestimate the accuracy of
both FRST and HCR-20V3 ratings because treating clinicians
may not have the time or training (or motivation) to reliably rate
these instruments. Our strong interrater reliability supports the
rigor with which the FRST and HCR-20V3 were applied in this
study, but the lack of outcome data (i.e., actual violent behav-
ior) limits our ability to determine how accurate the HCR-20V3
risk estimates actually were. Finally, the modest sample size
limits a systematic analysis of misclassification errors, partic-
ularly given the small number of such errors. Future research
using a larger, more heterogeneous sample with more high-risk
individuals is needed to bolster the findings from this study.
These limitations notwithstanding, this study provides strong
preliminary support for the FRST as a violence risk screening
tool. Obviously we would caution that this study represents a
first step in the process of establishing an empirically supported
screening tool. Further research is needed to determine the
extent to which these results are generalizable to other settings
and populations. For example, because the FRST relies heavily
on the patient’s willingness to acknowledge violent thoughts,
threats, and actions, this approach may be more appropriate for
the mental health settings in which it was developed. It is
unclear whether the FRST would retain comparable accuracy in
a criminal justice setting, in which the association between
psychiatric symptoms and violence risk may be weaker. With-
out additional research to examine the strengths and limitations
of this approach to violence risk screening, clinicians will
continue to be left with little guidance as to how they should
prioritize their caseload in terms of assessing violence risk.
References
Andrews, D. A., Bonta, J., & Hoge, R. D. (1990). Classification for
effective rehabilitation: Rediscovering psychology. Criminal Justice and
Behavior, 17, 19 –52. http://dx.doi.org/10.1177/0093854890017001004
Bjørkly, S., Hartvig, P., Heggen, F. A., Brauer, H., & Moger, T. A. (2009).
Development of a brief screen for violence risk (V-RISK-10) in acute
and general psychiatry: An introduction with emphasis on findings from
a naturalistic test of interrater reliability. European Psychiatry, 24,
388 –394. http://dx.doi.org/10.1016/j.eurpsy.2009.07.004
Campbell, M. A., French, S., & Gendreau, P. (2009). The prediction of
violence in adult offenders: A meta-analytic comparison of instruments
and methods of assessment. Criminal Justice and Behavior, 36, 567–
590. http://dx.doi.org/10.1177/0093854809333610
Chichetti, D. V. (1994). Guidelines, criteria, and rules of thumb for
evaluating nominal and standardized assessment instruments in psychol-
ogy. Psychological Assessment, 6, 284 –290. http://dx.doi.org/10.1037/
1040-3590.6.4.284
Coontz, P. D., Lidz, C. W., & Mulvey, E. P. (1994). Gender and the
assessment of dangerousness in the psychiatric emergency room. Inter-
national Journal of Law and Psychiatry, 17, 369 –376. http://dx.doi.org/
10.1016/0160-2527(94)90013-2
Douglas, K. S., Hart, S. D., Webster, C. D., & Belfrage, H. (2013). HCR-20
(Version 3): Assessing Risk for Violence. Burnaby, BC, Canada: Mental
Health, Law, and Policy Institute, Simon Fraser University.
Douglas, K. S., & Reeves, K. A. (2010). Historical-Clinical-Risk
Management-20 (HCR-20) Violence Risk Assessment Scheme: Ratio-
nale, application, and empirical overview. In R. K. Otto & K. S. Douglas
(Eds.), Handbook of violence risk assessment (pp. 147–185). New York,
NY: Routledge/Taylor & Francis Group.
Foellmi, M., Rosenfeld, B., Rotter, M., & Khadivi, A. (2014, June).
Violence risk screening: The influence of clinician and patient variables.
Paper presented at the International Association of Forensic Mental
Health Service Conference, Toronto, Ontario, Canada.
Guy, L. S., Douglas, K. S., & Hendry, M. C. (2010). The role of psycho-
pathic personality disorder in violence risk assessments using the HCR-
20. Journal of Personality Disorders, 24, 551–580. http://dx.doi.org/10
.1521/pedi.2010.24.5.551
Heilbrun, K., Yasuhara, K., & Shah, S. (2010). Violence risk assessment
tools. In R. Otto & K. S. Douglas (Eds.), The handbook of violence risk
assessment (pp. 1–17). New York, NY: Routledge.
Iserson, K. V., & Moskop, J. C. (2007). Triage in medicine, part I:
Concept, history, and types. Annals of Emergency Medicine, 49, 275–
281. http://dx.doi.org/10.1016/j.annemergmed.2006.05.019
Litwack, T. R. (2001). Actuarial versus clinical assessments of dangerous-
ness. Psychology, Public Policy, and Law, 7, 409 – 443. http://dx.doi.org/
10.1037/1076-8971.7.2.409
Magen, J., Richards, M., & Ley, A. F. (2013). A proposal for the “next-
generation psychiatry residency”: Responding to challenges of the fu-
ture. Academic Psychiatry, 37, 375–379. http://dx.doi.org/10.1176/appi
.ap.13030024
McNiel, D. E., & Binder, R. L. (1994). Screening for risk of inpatient
violence: Validation of an actuarial tool. Law and Human Behavior, 18,
579 –586. http://dx.doi.org/10.1007/BF01499176
Monahan, J., Steadman, H. J., Appelbaum, P. S., Grisso, T., Mulvey, E. P., Roth,
L. H., . . . Silver, E. (2006). The classification of violence risk. Behavioral
Sciences & the Law, 24, 721–730. http://dx.doi.org/10.1002/bsl.725
Monahan, J., Steadman, H. J., Appelbaum, P. S., Robbins, P. C., Mulvey,
E. P., Silver, E., . . . Grisso, T. (2000). Developing a clinically useful
actuarial tool for assessing violence risk. British Journal of Psychiatry,
176, 312–319. http://dx.doi.org/10.1192/bjp.176.4.312
New York State/New York City Mental Health-Criminal Justice Panel.
(2008). Report and recommendations. Retrieved from https://urldefense.
proofpoint.com/v2/url?u⫽http-3A__www.criminaljustice.ny.gov
_pio_mh-2Dcjreport.pdf&d⫽DgIGaQ&c⫽aqMfXOEvEJQh2iQM
Cb7Wy8l0sPnURkcqADc2guUW8IM&r⫽ouz4Zk--_COQd_h4gl1
tySLjzB-d9OxYW9QpnEt-TqY&m⫽GOI5bcPkRLhRHTb-aCMIi
1SXuwwt7F07fML1vqqjkFU&s⫽BDI491h3Uz6d-zwnHR_rU49X
3gKi-6IVCUMDQg6hSSQ&e⫽
Pauker, S. G., & Kassirer, J. P. (1980). The threshold approach to clinical
decision making. New England Journal of Medicine, 302, 1109 –1117.
http://dx.doi.org/10.1056/NEJM198005153022003
Rabin, R. F., Jennings, J. M., Campbell, J. C., & Bair-Merritt, M. H.
(2009). Intimate partner violence screening tools: A systematic review.
American Journal of Preventive Medicine, 36, 439 – 445. e4. http://dx
.doi.org/10.1016/j.amepre.2009.01.024
 VIOLENCE RISK SCREENING
Raffle, A. E., & Gray, J. A. M. (2009). Screening: Evidence and practice.
International Journal of Epidemiology, 38, 1748 –1749. http://dx.doi
.org/10.1093/ije/dyn233
Rosenfeld, B., Foellmi, M., Howe, J., & Rotter, M. (2013, June). Devel-
opment and validation of the Bronx Risk Screening Tool (BRST). Paper
presented at the 12th annual meeting of the International Association of
Forensic Mental Health Services, Maastricht, Netherlands.
Rosenfeld, B., Sands, S. A., & Van Gorp, W. G. (2000). Have we forgotten
the base rate problem? Methodological issues in the detection of distor-
tion. Archives of Clinical Neuropsychology, 15, 349 –359. http://dx.doi
.org/10.1093/arclin/15.4.349
Singh, J. P., Desmarais, S. L., Hurducas, C., Arbach-Lucioni, K., Conde-
marin, C., Dean, K., . . . Ho, R. M. Y. (2014). International perspectives
on the practical application of violence risk assessment: A global survey
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
of 44 countries. International Journal of Forensic Mental Health, 13,
This document is copyrighted by the American Psychological Association or one of its allied publishers.
193–206. http://dx.doi.org/10.1080/14999013.2014.922141
Singh, J. P., Grann, M., & Fazel, S. (2011). A comparative study of
violence risk assessment tools: A systematic review and metaregres-
sion analysis of 68 studies involving 25,980 participants. Clinical
Psychology Review, 31, 499 –513. http://dx.doi.org/10.1016/j.cpr
.2010.11.009
(Appendix follows)
7
Singh, J. P., Grann, M., Lichtenstein, P., Långström, N., & Fazel, S.
(2012). A novel approach to determining violence risk in schizophrenia:
Developing a stepped strategy in 13,806 discharged patients. PLoS ONE,
7, e31727. http://dx.doi.org/10.1371/journal.pone.0031727
Skeem, J. L., Manchak, S. M., Lidz, C. W., & Mulvey, E. P. (2013). The
utility of patients’ self-perceptions of violence risk: Consider asking the
person who may know best. Psychiatric Services, 64, 410 – 415. http://
dx.doi.org/10.1176/appi.ps.001312012
Smith, H., & White, T. (2007). Feasibility of a structured risk assessment
tool in general adult psychiatry admissions. Psychiatrist, 31, 418 – 420.
Thomas, K. C., Ellis, A. R., Konrad, T. R., Holzer, C. E., & Morrissey, J. P.
(2009). County-level estimates of mental health professional shortage in
the United States. Psychiatric Services, 60, 1323–1328. http://dx.doi
.org/10.1176/ps.2009.60.10.1323
Viljoen, J. L., McLachlan, K., & Vincent, G. M. (2010). Assessing vio-
lence risk and psychopathy in juvenile and adult offenders: A survey of
clinical practices. Assessment, 17, 377–395. http://dx.doi.org/10.1177/
1073191109359587
Webster, C. D., Douglas, K. S., Eaves, D., & Hart, S. D. (1997). HCR-20:
Assessing the risk for violence (version 2). Vancouver, British Columbia,
Canada: Mental Health, Law, and Policy Institute, Simon Fraser Uni-
versity.
 8 ROSENFELD ET AL.

Appendix
Fordham Risk Screening Tool

1) Violent Behavior ⴱⴱIf both recent and severe are selected, a. Any information suggestive of violent thoughts?
please proceed to Section 5.
b. “Ever had thoughts about hurting someone? When was
a. Is there any information suggestive of violent behavior? the last time?”

b. “Have you ever physically hurt someone? When was the c. “What happened? Why didn’t you act on those
last time? What happened?” thoughts?”

c. “Ever tried to hurt someone? Ever come close? Anybody d. “How often did you think about hurting someone in the
injured?” last 6 months?”
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
This document is copyrighted by the American Psychological Association or one of its allied publishers.

d. “How many times have you hurt someone in the last 6 Recent (past 6 months): Yes No
months?” Severe: Yes No
Recent (past 6 months): Yes No 4) Supplemental questions ⴱⴱIf yes was selected for any ques-
Severe (requiring medical attention or acts that could have tion in Section 4, return to Section 1 and asks questions
resulted in serious harm): Yes No again focusing on new information.
2) Violent Threats ⴱⴱIf both recent and severe are selected in a. Have you ever?
any combination between Section 1 and Section 2, please
proceed to Section 5. i. Gotten into fights? Yes No
a. Is there any information suggestive of violent threats? ii. Had an order of protection taken out against you? Yes
No
b. “Have you ever threatened someone? When was the last
time? What happened?” iii. Been arrested on a violent charge? Yes No
c. “How many times did you threaten to harm someone in 5) Additional risk factors
the last 6 months?” Current evidence of:
d. “Ever made threats with a knife or a gun? When? What a. Agitation or hostility: Yes No
happened?”
b. Paranoid ideation or delusions related to control: Yes No
Recent (past 6 months): Yes No
Severe (threats of injury that could potentially require medical c. Treatment resistances (e.g., refusal to take medications):
attention if carried out): Yes No
Yes No

3) Violent thoughts ⴱⴱIf both recent and severe are selected in Received November 28, 2016
any combination between Section 1, Section 2, and Section 3, Revision received April 1, 2017
please proceed to Section 5. Accepted April 2, 2017 䡲

View publication stats