You are on page 1of 19

Measurement and Evaluation in Counseling and

Development

ISSN: 0748-1756 (Print) 1947-6302 (Online) Journal homepage: http://www.tandfonline.com/loi/uecd20

Selecting Suicide Ideation Assessment


Instruments: A Meta-Analytic Review

Bradley T. Erford, Jessica Jackson, Gerta Bardhoshi, Kelly Duncan & Zumra
Atalay

To cite this article: Bradley T. Erford, Jessica Jackson, Gerta Bardhoshi, Kelly Duncan
& Zumra Atalay (2017): Selecting Suicide Ideation Assessment Instruments: A Meta-
Analytic Review, Measurement and Evaluation in Counseling and Development, DOI:
10.1080/07481756.2017.1358062

To link to this article: https://doi.org/10.1080/07481756.2017.1358062

Published online: 20 Dec 2017.

Submit your article to this journal

Article views: 2

View related articles

View Crossmark data

Full Terms & Conditions of access and use can be found at


http://www.tandfonline.com/action/journalInformation?journalCode=uecd20

Download by: [University of Florida] Date: 22 December 2017, At: 01:52


MEASUREMENT AND EVALUATION IN COUNSELING AND DEVELOPMENT
, VOL. , NO. , –
https://doi.org/./..

Selecting Suicide Ideation Assessment Instruments: A Meta-Analytic


Review
Bradley T. Erforda , Jessica Jacksonb , Gerta Bardhoshic , Kelly Duncand , and Zumra Atalaye
a
Peabody College at Vanderbilt University, Human and Organizational Development, Nashville, TN, USA; b Loyola
University Maryland, Baltimore, MD, USA; c University of Iowa, Iowa City, IA, USA; d Northern University, Aberdeen, SD,
USA; e MEF University, Istanbul, Turkey

ABSTRACT KEYWORDS
Downloaded by [University of Florida] at 01:52 22 December 2017

Psychometric meta-analyses and reviews were provided for four commonly meta-analysis; suicide
used suicidal ideation instruments: the Beck Scale for Suicide Ideation, the assessment; suicide ideation
Suicide Ideation Questionnaire, the Suicide Probability Scale, and Columbia–
Suicide Severity Rating Scale. Practical and technical issues and best use rec-
ommendations for screening and outcome research are offered.

Suicide ideation is common, with a lifetime prevalence of over 10% in the U.S. population; thus, suicide
ideation is an important determinant of suicide risk and one that requires appropriate intervention (Nock
et al., 2013). Suicide was the 10th leading cause of death in individuals of all ages in the United States in
2012 and resulted in an estimated $34.6 billion in lost work and medical costs per year. Gender differences
exist among those completing suicides with males accounting for 79% of all suicides, making males four
times more likely than females to complete a suicide; at the same time, females are far more likely to
attempt suicide (McIntosh & Drapeau, 2014).
Differences in suicide completion by age also exist. Suicide is the second leading cause of death among
adults aged 25 to 34 years of age and the third leading cause among those aged 15 to 24 years. Among
males, suicide rates are highest for those aged 75 years and over. Racial and ethnic disparities also affect
suicide, with higher suicide rates recorded for American Indians and Alaska Natives than other ethnici-
ties, nearly 2.5 times higher than the national average suicide rate among the 15-to 34-year-old age group
(Centers for Disease Control and Prevention, 2012).
Individuals with an acute risk of suicidal behavior often show signs that can alert professionals and
nonprofessionals alike. Although long-term suicide risk factors encompass biological, emotional, and
environmental domains (Erford, 2013), presentation of suicidality might first be indicated by imminent
warning signs. The threat of harm to self, seeking access to lethal means such as weapons or pills, or
communicating through writings, social media, or songs about death and dying are important warning
signs. Drastic changes in mood, acts of recklessness or aggression, withdrawal from relationships, and
increased substance use might also indicate imminent risk (Chesin & Stanley, 2013). Suicidal risk also
increases the probability of cooccurrence with a variety of mental disorders such as depression, bipolar,
anxiety, eating, trauma-related, and adjustment disorders (Seligman & Reichenberg, 2012). Unfortu-
nately, even when individuals who have attempted suicide in the past year receive mental health care,
approximately half report unmet treatment needs, further increasing risk for reattempts and completion
(Han, Compton, Gfroerer, & McKeon, 2014).
Suicidal ideation is commonly encountered by mental health professionals working with adolescents
and adults, and assessment and treatment of these individuals can be very stressful (Granello & Granello,

CONTACT Bradley T. Erford bradley.t.erford@vanderbilt.edu Peabody College at Vanderbilt University, Human and Organiza-
tional Development,  Appleton Place, Nashville, TN -, USA.
©  Association for Assessment and Research in Counseling (AARC).
2 B. T. ERFORD ET AL.

2007). Although clinical interviews are almost always used for assessment of suicidal ideation, there is
no literature specifying how frequently standardized suicide assessment instruments are used in clinical
practice, or under what conditions the instruments are used. This leads to the question of whether inte-
gration of standardized instruments in conjunction with clinical interviews could lead to better practices
and triangulation of results from multiple sources.
Although there are a variety of instruments useful for assessing suicidal risk, little is known about the
psychometric integrity of these standardized instruments other than what is published in the test user
manuals and scattered throughout the literature in individual articles. The purpose of this article was to
provide a comprehensive synthesis of psychometric evidence from the extant literature to evaluate, com-
pare, and contrast four of the most commonly used and cited standardized suicide ideation instruments:
the Beck Scale for Suicide Ideation (BSS; Beck & Steer, 1991), the Suicide Ideation Questionnaire (SIQ;
Reynolds, 1988), the Suicide Probability Scale (SPS; Cull & Gill, 1992); and the Columbia–Suicide Sever-
ity Rating Scale (C–SSR; Columbia University, 2014). A discussion of practical issues, technical strengths
and weaknesses, and best use recommendations for screening and counseling outcome research will
inform clinicians and researchers on the selection process of suicide ideation assessment instruments.
Downloaded by [University of Florida] at 01:52 22 December 2017

Furthermore, frequency of use of these instruments in suicide research could inform counselors of the
relative usefulness of these instruments in clinical practice.

Methodological Considerations
The psychometric meta-analytic results that follow were derived from journal articles, dissertations,
and other electronically available sources that met the following criteria: (a) used the English version
of the main instrument (other language translations and brief or modified versions are designated
as such and presented separately, as appropriate); and (b) provided some type of reliability, validity,
or nonconclinical sample mean data. Foreign language translated versions were not aggregated with
English language versions because they generally did not conform to best practices translation proce-
dures (AERA/APA/NCME, 2014, Standard 9.7). The studies included in this meta-analysis were inde-
pendent of the standardization samples or studies reported in the manual or issued by the instrument
authors when providing initial psychometric evidence for instrument use.
Redundant search procedures were undertaken to identify candidate studies. Electronic searches were
followed by hand searches of synthesis and selected article reference lists. PsychINFO, ERIC, Academic
Search Premier, Cochrane Central Register of Controlled Trials, and MEDLINE articles for the year of
and subsequent to publication of each instrument were included, inclusive of 2015. Key words included
the instrument name and acronym (e.g., Beck Scale for Suicide Ideation and BSS) for full-text search.
Additional candidate studies were then located by searching the reference lists of the selected articles and
synthesis studies. Full-text versions of each candidate article were inspected, selection criteria applied,
and all articles meeting the selection criteria were included in the analysis.
The primary variables of interest in each instrument psychometric meta-analysis were internal con-
sistency (α), test–retest reliability (rtt ), convergent correlations (r) with other suicide ideation or related
measures (e.g., depression, hopelessness), structural validity (i.e., exploratory factor analysis [EFA], con-
firmatory factor analysis [CFA]), diagnostic validity (e.g., percent of correct classifications, sensitivity,
specificity, positive predictive power, negative predictive power, area under the curve [AUC] estimates),
and descriptive statistics (i.e., means and standard deviations) from nonclinical samples for use in norms
comparisons. Sample size weightings for α and rtt coefficients were applied directly. All convergent
coefficients (r) were independent and corrected for sampling bias by first transforming r into z values
(½log[ (1+r)
(1−r)
]; Hedges & Olkin, 1985), then weighed by sample size, summed, averaged, and the grand z
back-transformed to r. The 95% CI (indicated in brackets [] throughout this article) was calculated to
test the null hypothesis of r = 0; the null was rejected if the entire 95% CI was > 0.

Review and Comprehensive Psychometric Meta-Analysis of the Beck Scale for Suicide
Ideation
The BSS is a self-report instrument used to identify and measure suicide ideation among adolescents and
adults (Beck & Steer, 1991). The BSS was developed as a self-report version of the clinician-report Scale
MEASUREMENT AND EVALUATION IN COUNSELING AND DEVELOPMENT 3

for Suicide Intention (SSI; Beck, Kovacs, & Weissman, 1979). The BSS contains 21 clusters of statements
with each containing three statements marked 0, 1, or 2. Statements designated with 0 indicate low suicide
ideation and statements designated with 2 indicate high suicide ideation. The 21 statements are broken
down into two parts. Part 1 measures the clients’ desire to live or die, their reasons for living or dying, and
their ideation of active or passive suicide attempts. If the client chooses the 0 statements on Clusters 4
and 5, they proceed to Cluster 20 and 21, designed to measure their past suicide behavior. If they choose
the 1 or 2 statements on Clusters 4 and 5, they move on to Cluster 6 and complete the test in order, where
factors such as duration and frequency of ideation, deterrents and reasons for attempts, and deception
and concealment are measured.
Administration requires only 5 to 10 minutes and can be completed independently by the client, or
the examiner could read the scale aloud and mark the responses for the client. This level C instrument
(Erford, 2013) should be scored by professionals with clinical training and experience, as the professional
might need to provide an appropriate therapeutic or crisis response based on the examinee’s score (Beck
& Steer, 1991). The BSS is scored by adding the raw score points from the selected statements (0, 1, or 2).
Total scores can range from 0 to 38 and there is no suggested criterion-related cutoff score. Indeed, any
Downloaded by [University of Florida] at 01:52 22 December 2017

positive response to any BSS item could reveal the presence of suicide intention and should be carefully
investigated by the clinician.
The BSS was developed for use with both inpatient and outpatient adult psychiatric clients and was
standardized on a group of 50 inpatients and 55 outpatients from a suburban general hospital (Beck,
Steer, & Ranieri, 1988) with 46% of participants describing prior suicide attempts. Cronbach’s coefficient
α for the subgroups of inpatient and outpatient suicide ideators was .90 and .87, respectively. One-week
test–retest stability was also studied on 60 inpatients, but was found to be low (r = .54), probably due
to the clinical improvement expected after a week-long hospitalization treatment regimen. Scores on
the BSS correlated significantly with scores on the SSI (Beck, Kovacs, & Weissman, 1979; r = .90), Beck
Depression Inventory (BDI; Beck, Steer, & Brown, 1996; r = .64), and Beck Hopelessness Scale (BHS;
Beck & Steer, 1993; r = .53).
The standardization sample was not very diverse and additional studies on the effectiveness of the BSS
with minorities are needed. The BSS has been translated into many languages, including Spanish, Urdu,
Farsi, Portuguese, Brazilian, and Norwegian. Additional studies have explored test validity and reliabil-
ity for use with inpatient populations diagnosed with schizophrenia, schizoaffective, or bipolar disorder
(Pinninti, Steer, Rissmiller, Nelson, & Beck, 2002), inmates (Way, Kaufman, Knoll, & Chlebowski, 2013),
people with sex disorders (Schützmann, Brinkmann, Schacht, & Richter-Appelt, 2009), and with a non-
clinical population of university students using the Norwegian language version (Chioqueta & Stiles,
2006), among others.
Of candidate articles available from 1993 through 2015, 420 were identified through computerized
and hand searches. Full-text review of all candidate articles eliminated 413 articles for violating one or
more of the inclusion criteria. Thus, only seven (j = 7) articles with a combined sample size of 1,059
participants were analyzed. Two studies reported αs (Bisconer & Gross, 2007; Steer, Rissmiller, Ranieri,
& Beck, 1993), combining for a grand α of .91 (.81–1.00; n = 397), consistent with the .90 and .87
reported by Beck and Steer (1991) for inpatient and outpatient samples, respectively. No test–retest stud-
ies were located in the literature. Concurrent validity comparisons (see Table 1) showed strong correla-
tions between the BSS and the C–SSR (Posner et al., 2011; n = 472, r = .52), SPS (Cull & Gill, 1992;
n = 67, r = .65), and Adult Suicide Ideation Questionnaire (ASIQ; Reynolds, 1988; n = 67, r = .64).
Interestingly, anxiety (Beck Anxiety Inventory [BAI]; Beck, 1993; n = 122, r = .55) was just as highly
correlated as the suicide-related constructs of depression (BDI; Beck, Steer, & Brown, 1996; n = 535,
r = .58) and hopelessness (BHS; Beck & Steer, 1993; n = 406, r = .55).
One EFA was located (Steer, Rissmiller, Ranieri, & Beck, 1993) with an appropriately powered sample
size of 330 psychiatric outpatients. A principal factor analysis with promax rotation generated a three-
factor solution on which all but six items loaded significantly. Factor I, desire for death, was composed of
Items 1, 2, 3, 5, and 8; Factor II, preparation for suicide, was composed of Items 12 to 18; and Factor III,
active suicidal desire, was composed of Items 4, 6, 7, and 9. These three factors were strongly intercorre-
lated (r = .50–.62). No CFAs were located, so the Steer et al. (1993) study provides the only evidence of
internal structural validity on the BSS.
4 B. T. ERFORD ET AL.

Table . Concurrent Validity Studies Using the Beck Scale for Suicide Ideation (BSS), Suicide Ideation Questionnaire (SIQ), Suicide Prob-
ability Scale (SPS), and Columbia–Suicide Severity Rating Scale (C–SSR).

Convergent Comparison j n r [% CI] Studies

BSS with
C–SSR   . [., .] Posner et al. ()
SPS   . [., .] Bisconer & Gross ()
Adult Suicide Ideations Questionnaire (ASIQ)   . [., .] Bisconer & Gross ()
Suicide Assessment Scale Self-Report (SUAS-S)   . [., .] Koldsland et al. ()
Beck Depression Inventory (BDI)   . [., .] Bisconer & Gross (); Bowen et al. ();
Cochrane-Brink et al. (); Hirsch ()
Mood Disorder Questionnaire (MDQ)   . [., .] Bowen et al. ()
Modified SAD PERSONS Scale (MSPS)   . [., .] Cochrane-Brink et al. ()
Beck Hopelessness Scale (BHS)   . [., .] Bisconer & Gross (); Cochrane-Brink et al.
(); Hirsch ()
Beck Anxiety Inventory (BAI)   . [., .] Bisconer & Gross (); Cochrane-Brink et al.
()
Affective Lability Scale (ALS)   . [., .] Bowen et al. ()
Perceived Stress Scale (PSS)   . [., .] Bowen et al. ()
Downloaded by [University of Florida] at 01:52 22 December 2017

Eysenck Neuroticism Scale (N–ERQ)   . [., .] Bowen et al. ()

SIQ with
Suicide Opinion Questionnaire (SOQ)   . [., .] Lester ()
Suicide Resiliency Inventory (SRI–)   –. [–., –.] Rutter et al. ()
Reasons for Living Inventory (RFL)   –. [–., –.] Kumar et al. ()
Suicide Intent Scale (SIS)   . [., .] Spirito et al. ()
BDI  , . [., .] Chang (); Reynolds (); Weber et al. ()
Reynolds Adolescent Depression Scale (RADS)   . [., .] Spirito et al. ()
Minnesota Multiphasic Personality Inventory   . [., .] Osman et al. ()
Depression subscale
BHS  , . [., .] Gencoz et al. (); Lynch et al. (); Reynolds
(); Weber et al. ()
Hopelessness Scale for Children (HSC)   . [., .] Spirito et al. ()
UCLA Loneliness Scale   . [., .] Weber et al. ()

SIQ–Junior High version (SIQ–JR) with


Reynolds Suicidal Behaviors Interview (SBI)   . [., .] Bergen (); King et al. (); Prinstein et al.
(); Reynolds (); Reynolds & Mazza
()
SPS with
RFL  , –. [–., –.] Eryilmaz (); Gutierrez et al. (); Kopper
et al. (); Osman et al. (); Osman &
Kopper ()
Center for Epidemiological Studies–Depression   . [., .] Morrison & O’Connor (); O’Connor & Noyce
()
BHS   . [., .] Bisconor & Gross (); D’Zurilla & Chang ();
Kopper et al. (); Lee (); Morrison &
O’Connor ()
BSS   . [., .] Bisconor & Gross (); Lee ()
BDI   . [., .] Bisconor & Gross (); Borthick ();
Demirbas et al. (); D’Zurilla & Chang ()
Positive Affect Negative Affect Schedule   . [., .] Gutierrez et al. ()
(PANAS+)
Positive Affect Negative Affect Schedule   . [., .] Gutierrez et al. ()
(PANAS-)
MMPI Depression subscale   . [., .] Gutierrez et al. ()
Response Style Questionnaire–Short Form   . [., .] Morrison & O’Connor ()
BAI   . [., .] Bisconor & Gross ()
SIQ   . [., .] Bisconor & Gross ()
Firestone Assessment of Self-Destructive  , . [., .] Firestone & Firestone ()
Thoughts

C–SSR with
BSS   . [., .] Posner et al. ()
Beck Lethality Scale   . [., .] Posner et al. ()
Montgomery-Asberg Depression Rating Scale   . [., .] Posner et al. ()
SIQ–JR   . [., .] Posner et al. ()

Note. j = number of articles in which the instrument was used as an outcome measure.
MEASUREMENT AND EVALUATION IN COUNSELING AND DEVELOPMENT 5

One study was located yielding evidence of diagnostic validity (also known as decision reliability;
see Erford, 2013) on the BSS (Cochrane-Brink, Lofchy, & Sakinofsky, 2000). At a recommended cutoff
score of ࣙ 24 raw score points, sensitivity = 1.00, specificity = .90, positive predictive power = 1.00,
and negative predictive power = .72, with an overall accuracy rate of close to 95%. These were quite
impressive and robust results. A second study was located that provided evidence for a 5-item short
form version of the BSS (Koldsland, Mehlum, Mellesdal, Walby, & Diep, 2007), concluding that a cutoff
score of ࣙ 1 was optimal, yielding sensitivity = .80, specificity = .78, positive predictive power = .77,
and negative predictive power = .40, with an overall accuracy rate of close to 79%. These results were less
promising, so clinicians and researchers will likely obtain more accurate hit rates using the full version
of the BSS. Finally, no study statistics were reported for nonclinical samples on the English version of the
BSS that could be used to estimate sample norms, although Ekramzadeh et al. (2012) reported a mean of
1.69 (SD = 2.25, n = 650) for a Farsi translation of the BSS, and Alexandrino-Silva et al. (2009) reported
a mean of 0.38 (SD = 1.50, n = 987) for a Portuguese translation of the BSS on a sample of Brazilian
medical students.
Downloaded by [University of Florida] at 01:52 22 December 2017

Review and Comprehensive Psychometric Meta-Analysis of the Suicide Ideation


Questionnaire and Junior High Version
The SIQ and SIQ–JR (Junior High version) measure the multifaceted concept of suicide ideation in ado-
lescents (Reynolds, 1988), and attempt to address the lack of appropriate screening for suicide ideation
in adolescence. The SIQ contains 30 items and the SIQ–JR contains 15 items. The client responds to each
item on a 7-point scale (i.e., 0–6) according to the temporal frequency of the agreement with the item in
the past month. Responses range from I never had this thought to almost every day and include I had this
thought before but not in the past month. Both versions can be administered individually, or in small or
large groups. The SIQ scores are calculated by summing the raw scores for each item. Scores from 0 to
180 are possible on the SIQ and from 0 to 90 on the SIQ–JR, with higher scores indicating more severe
suicide ideation. Respondents with a raw score of 41 or above on the SIQ, or 31 or above on the SIQ–JR,
should be further evaluated for suicide risk and potentially significant psychopathology. Scores of critical
items identified by Reynolds through theoretical means (as opposed to empirical means) can be listed as
well, as they might indicate the potential for serious self-harm behavior (Reynolds, 1988). Those items
are 2, 3, 4, 7, 8, 9, 13, and 18 on the SIQ and 2, 3, 4, 7, 8, and 9 on the SIQ–JR. Scores on two or more
of these specific items indicating that these thoughts occur a “couple of times a week” or “almost every
day” should be taken seriously regardless of overall score.
Reynolds (1988) reported an internal consistency reliability coefficient of .97 for the SIQ and .94 for
the SIQ–JR. Although suicidal ideation is not considered a stable mental state, 4-week test–retest relia-
bility of the SIQ (n = 801) was .72. The scores of the SIQ and SIQ–JR were quite consistent with scores
on the Reynolds Adolescent Depression Scale (RADS; Reynolds, 2002; r ranging from .55–.63), the BDI
(Beck, Steer, & Brown, 1996; r = .53) and Children’s Depression Inventory (CDI; Kovacs, 1992; r rang-
ing from .65–.70), and the BHS (Beck & Steer, 1993; r = .47). The SIQ and SIQ–JR have been used
on diverse populations, including adolescents with behavioral disorders, clinical adolescent populations
(Pinto, Whisman, & McCoy, 1997), and female college students 18 and 19 years old (Klosterman-Fields,
1985). The SIQ has also been expanded for use with adults (ASIQ; Reynolds, 1991) and published in
other languages including Chinese (Zhang, Yip, & Fu, 2014), French (Potard, Kubiszewski, Gimenes, &
Courtois, 2014), Korean, Arabic, and Afrikaan (South Africa).
Of candidate articles, 125 were identified through computerized and hand searches and full-text
review eliminated all but 35 articles for a combined sample size of 10,942 participants. Internal consis-
tency estimates of the SIQ were provided in 14 articles (Abdel-Khalek & Lester, 2007; Chang, 2002; Cross,
Cassady, & Miller, 2006; Gencoz, Vatan, Walker, & Lester, 2007; Gutierrez, Osman, Barrios, & Kopper,
2001; Horesh & Apter, 2006; Lynch, Cheavens, Morse, & Rosenthal, 2004; Miller & Esposito-Smythers,
2013; Osman et al., 1999; Pieneer, Rothman, & Van De Vijver, 2007; Pinto, Whisman, & McCoy, 1997;
Reynolds, 1991; Steyn, Vawda, Wyatt, Williams, & Madu, 2013; Walker, Lester, & Joe, 2006) for a com-
bined sample of 5,858 participants yielding an average α = .95 [.92, .98], a very high coefficient. A single
6 B. T. ERFORD ET AL.

test–retest reliability study was reported, and using a 1-week interval derived rtt = .86 [.77, .95] (n = 474;
Reynolds, 1991). An additional three articles exploring the internal consistency of a Korean translation
of the SIQ were also located and reported a combined α = .97 [.90, 1.00] (n = 751; J. Lee, Choi, Kim,
Park, & Shin, 2009; S-Y. Lee, 2011; Shin, 1992).
Concurrent validity coefficients (see Table 1) were robust with other measures of suicidal ideation,
such as the Suicide Opinion Questionnaire (SOQ; Domino, Moore, Westlake, & Gibson, 1982; j = 1,
n = 81, r = .70 [.48, .92]; Lester, 2004). Resilience to suicide was compared with the Suicide Resiliency
Inventory (SRI–25; Osman et al., 2004; j = 1, n = 239, r = –.67 [–.54, –.80]; Rutter, Freedenthal, &
Osman, 2008) and the Reasons for Living Inventory (RFL; Linehan, Goodstein, Nielsen, & Chiles, 1983;
j = 1, n = 320, r = –.66 [–.55, –.77]; Kumar, Atanu, Basu, & Das, 2008). Depression inventories, such as
the BDI (Beck, Steer, & Brown, 1996; j = 3, n = 1,030, r = .58 [.52, .64]; Chang, 2002; Reynolds, 1991;
Weber, Metha, & Nelson, 1997), RADS (Reynolds, 1987; j = 1, n = 151, r = .47 [.31, .63]; Spirito, Ster-
ling, Donaldson, & Arrigan, 1996), and Minnesota Multiphasic Personality Inventory (MMPI; Butcher,
Dahlstrom, Graham, Tellegen, & Kaemmer, 1989) Depression subscale (j = 1, n = 118, r = .64 [.46,
.82]; Osman et al., 1999) were moderately correlated. Hopelessness inventories such as the BHS (Beck &
Downloaded by [University of Florida] at 01:52 22 December 2017

Steer, 1993; j = 4, n = 1,485, r = .50 [.45, .55]; Gencoz et al., 2007; Lynch, Cheavens, Morse, & Rosenthal
2004; Reynolds, 1991; Weber et al., 1997) and Hopelessness Scale for Children (HSC; Kazdin, Rodgers,
& Colbus, 1986; j = 1, n = 151, r = .49 [.33, .65]; Spirito et al., 1996) also yielded moderate correlations
with the BSS.
Only one EFA for the SIQ was located. Abdel-Khalek and Lester (2007) reported a five-factor solu-
tion using principal components analysis (PCA) with a varimax rotation on a sample of 273 English-
speaking undergraduates in Kuwait. In the same article they found a highly similar factor solution on an
Arabic translation of the SIQ on scores from a sample of 460 Kuwaiti undergraduates. No CFAs of the
SIQ were located. Only one decision reliability study was published using the SIQ. Pinto et al. (1997),
on a sample of 226 adolescents, presented the cutoff score of 30 for optimal percent of correct classifi-
cation (72%) and sensitivity (.78, specificity = .65, positive predictive power = .73, negative predictive
power = .72), along with a cutoff score of 35 for optimal specificity (.65, percent of correct classification
= .70, sensitivity = .72, positive predictive power = .73, negative predictive power = .66). Osman et al.
(1999) used the 25-item ASIQ in a diagnostic validity study with 205 adult psychiatric patients, report-
ing the cutoff score of 14 for optimal sensitivity (.96, specificity = .79, positive predictive power = .73,
negative predictive power = .97), along with a cutoff score of 20 for optimal specificity (.92, sensitivity
= .80, positive predictive power = .85, negative predictive power = .89).
Descriptive statistics were reported for six nonclinical samples (Chang, 2002; Gencoz et al., 2007;
Kumar et al., 2008; Pieneer et al., 2007; Reynolds, 1991; Weber, Metha, & Nelsen 1997) resulting in a total
sample grand mean of 11.1 (SD = 15.8, n = 3,563). Two studies disaggregated the results by sex (Cross,
Cassady, & Miller, 2006; Reynolds, 1991) resulting in a male grand mean of 11.5 (SD = 13.7) and female
grand mean of 13.6 (SD = 13.8) in a relatively small sample size of only 389 participants. Clinicians
should use caution when using cutoff scores, such as the published raw score cutoff of 31 (Reynolds,
1988) because it might overidentify women or underidentify men engaging in suicidal ideation. Using
this data, a total sample grand mean and standard deviation of 11.1 (15.8) means that a cut score of 31
will identify about 10% of a population (z = 1.26) with suicidal ideation. When applied to the small
sample gender differential studies, a cutoff score of 31 is likely to identify 10% of females (z = 1.26), but
only 7.5% of males (z = 1.42). A cutoff score of 41 is likely to identify 2% of females (z = 2.00) and males
(z = 2.15).
A number of studies focused specifically on the SIQ–JR. Six studies (Ang & Ooi, 2004; Bergen, 1999;
Diamond, Creed, Gillham, Gallop, & Hamilton, 2012; Huth-Bocks, Kerr, Ivey, Kramer, & Kins, 2007;
Keane, Dick, Bechtold, & Manson, 1996; Reynolds & Mazza, 1999) combined to yield an overall average
α = .93 [.87, .99] (n = 977). Test–retest reliability of the SIQ–JR was studied in two articles combining for
a 1.5-month grand rtt = .74 [.62, .86] (n = 268; Kaminer, Burleson, Goldston, & Burke, 2006; Reynolds &
Mazza, 1999). Not surprisingly, as the length of the retest interval increased, rtt , decreased. For example,
Reynolds and Mazza (1999) reported an approximate 1-month rtt of .89 (n = 91), whereas Kaminer et al.
(2006) reported a 2-month rtt of .66 (n = 177). A robust SIQ–JR concurrent validity comparison was
MEASUREMENT AND EVALUATION IN COUNSELING AND DEVELOPMENT 7

reported with the Reynolds Suicidal Behaviors Interview (SBI; Reynolds, 1990) as five studies (Bergen,
1999; King, Hill, Naylor, Evans, & Shain, 1993; Prinstein, Boergers, Spirito, Little, & Grapentine, 2000;
Reynolds, 1990; Reynolds & Mazza, 1999) combined (n = 690) to yield a weighted average r = .66 [.59,
.73]. No EFA or CFA studies were located for the SIQ–JR.
Two diagnostic validity studies were identified using the SIQ–JR and both adopted a similar cut-
off score, even though two very diverse samples were used. Keane et al. (1996) assessed 163 Native
American adolescents and used a cutoff raw score of 30 to correctly classify 85% of the sample (sen-
sitivity = .80, specificity = .86, positive predictive power = .27, negative predictive power = .98).
Using a clinical sample of 289 adolescents hospitalized for psychiatric issues Huth-Bocks et al. (2007)
applied a cutoff score of 31 to produce an optimal sensitivity index of only .77 with a concomi-
tant low specificity of .41. Raising the cutoff score to 34 led to an optimal accuracy rate of only
about 65%, with more balanced sensitivity (.69) and specificity (.61) rates. Still, an overall accuracy
rate of 65% is very low for use in a clinical population, particularly assessing suicide risk. Finally,
descriptive statistics were reported for three nonclinical SIQ–JR samples (Kaminer et al., 2006; Keane
et al., 1996; Reynolds & Mazza, 1999) resulting in a total sample grand mean of 10.0 (SD = 15.0,
Downloaded by [University of Florida] at 01:52 22 December 2017

n = 431).

Review and Comprehensive Psychometric Meta-Analysis of the Suicide Probability Scale


The SPS (Cull & Gill, 1992) is a 36-item self-report measure designed to assess suicide risk in adoles-
cents and adults. The SPS is quick and easy to administer to individuals and groups, taking about 5 to
10 minutes. Clients should be directed to indicate how often each statement applies to them on a 4-point
scale, without being informed that the test assesses suicide. In fact, the measure does not mention sui-
cide in its title. This is considered a level C test (Erford, 2013), requiring a professional with a doctoral
degree or a clinical license possessing knowledge regarding suicide, clinical assessment, and the ethical
knowledge to work with clients who might present at a high risk for suicide. The items are scored across
four subscales: Hopelessness, Suicide Ideation, Negative Self-Evaluation, and Hostility. A total weighted
score, suicide probability score, and a normalized T score are also generated, to facilitate a more in-depth
interpretation of the scores. Thus, the SPS is the only instrument among the four reviewed in this arti-
cle to yield norm-referenced interpretive scores. The SPS is easily scored by hand, but scoring is also
available via western psychological services (WPS) Test Report Fax service, which immediately provides
users with scores and summaries in a single-page report.
The SPS was normed on a sample of more than 1,000 individuals across the general population, psy-
chiatric patients, and lethal suicide attempters. According to Cull and Gill (1992), the internal consis-
tency (α) of the SPS total score was .93, and for the four subscale scores were: Hopelessness, .85; Suicide
Ideation, .88; Negative Self-Evaluation, .58; and Hostility, .78. Test–retest reliability for the total score
was also examined after 10 days (rtt = .94, n = 478).
Computerized and hand searches identified 188 candidate articles for full-text review, with subse-
quent elimination of 150 articles that violated one or more inclusion criteria, leaving 38 articles to ana-
lyze with a total sample size of 9,254. Six of the eliminated articles provided usable results for foreign
language versions of the SPS and are reported in context later. The SPS has been translated into Turkish,
Korean, Swedish, and French.
The SPS internal consistency was strong across 10 studies at α = .91 [.87, .95] (n = 2,154; Bisconor &
Gross, 2007; Eltz et al., 2007; Gutierrez et al., 2001; Gutierrez et al., 2002; Morrison & O’Connor, 2008;
O’Connor & Noyce, 2008; Rudd, Joiner, & Rumzek, 2004; Stewart, 2009; Tatman, Greene, & Karr, 1993;
Ustad, 1998). Test–retest stability over a 1- to 3-month interval in two studies (Morrison & O’Connor,
2008; O’Connor & Noyce, 2008) was rtt = .71 [.60, .82] (n = 313). As can be seen in Table 1, the SPS
displayed consistent convergent correlations with other instruments measuring suicidal ideation (SRI
[Osman et al., 2004], r = –.54 [–.49, –.59], n = 1,543; BSI [Beck & Steer, 1991], r = .51 [.32, .70],
n = 112) and depression (Center for Epidemiological Studies–Depression scale [CES–D; Radloff, 1977],
r = .64 [.53, .75], n = 313; BDI [Beck, Steer, & Brown, 1996], r = .67 [.60, .74], n = 858; MMPI [Butcher
et al., 1989] Depression subscale, r = .68 [.59, .77], n = 475).
8 B. T. ERFORD ET AL.

Two studies reported EFA results, but only one of these provided any valuable information. Tatman
et al. (1993) performed a PCA with varimax rotation on 217 nonclinical adolescents, concluding the
existence of three factors that were highly correlated (r = .79–.90): suicidal despair (18 items), angry
frustration (13 items), and low self-efficacy (12 items). Eltz et al. (2007) administered the SPS to 226
adolescent psychiatric inpatients, but then proceeded to conduct EFAs on each of the original four SPS
subscales. Two CFAs were located. Eltz et al. (2007) indicated that the four-factor model was confirmed
but did not report fit statistics as evidence. Bagge and Osman (1998) also reported that the four-factor
model fit the data best, but the fit statistics were very low (comparative fit index = .79; nonnormed fit
index = .77), although the root mean square error of approximation (.06) indicted an adequate fit.
Two studies of diagnostic validity have been published using the full SPS; the published recommended
raw score cut1off is 78 (Cull & Gill, 1992). In a large sample study, Larzelere, Smith, Batenhorst, and Kelly
(1996) assessed 855 adolescents at Boys Town. The SPS results provided extremely low sensitivity (.28)
and positive predictive power (.09), but a very respectable specificity of .90, meaning that the SPS did a
good job of identifying who would not attempt suicide, but a very poor job of determining who would
attempt suicide. In a second study, Huth-Bocks et al. (2007) found an optimal balance of results using
Downloaded by [University of Florida] at 01:52 22 December 2017

the cutoff score of 78, but only identified about 65% of cases correctly (sensitivity = .65, specificity =
.64). A third study was located in which Anderson, Townsend, Everly, and Lating (1995) assessed only
52 patients, then selected the most highly correlated items from the original 36 SPS items to compose a
brief scale composed of 10 items (12, 15, 21, 24, 25, 29, 32, 33, 34, & 36) that resulted in 83% correct clas-
sifications. Although promising, it is important to point out that small sample results must be replicated
on larger, diverse samples before accepted for clinical decision-making purposes.
Means and standard deviations for nonclinical samples were reported in four studies (Bagge & Osman,
1998; Gutierrez et al., 2002; Kopper, Osman, & Barrios, 2001; Tatman et al., 1993), resulting in a grand
mean for the total sample (n = 1,300) of 55.7 (SD = 11.0). This is a large sample and an important
observation here is that a cutoff score of 78 would be almost exactly 2 SDs above the nonclinical partici-
pant mean, effectively identifying only the top 2% of the distribution. Interestingly, when disaggregated
by sex, the male total sample mean (M = 55.0) and standard deviation (SD = 9.0) across three studies
(n = 420; Bagge & Osman, 1998; Kaplan et al., 2007; Kopper et al., 2001) was slightly higher than the
female (n = 421) mean (53.0; SD = 9.7). However, when the scores are projected out 2 SDs above the
mean, the cutoff score at the 98th percentile is virtually identical: a raw score of 73, which is a 0.5 SD
again lower than the published cutoff raw score of 78.
A number of articles have been published that studied only the Suicide Ideation subscale of the SPS,
which is frequently used as a brief version and outcome variable. Eight articles (Dear, 2000; Heisel,
Flett, & Hewitt, 2003; O’Connor, Fraser, Whyte, MacHale, & Masterson, 2008; O’Connor, O’Connor, &
Marshall, 2007; O’Connor, Smyth, Ferguson, & Caoimhe Williams, 2013; O’Connor, Whyte, et al., 2007;
Rasmussen, O’Connor, & Brodie, 2008; Rasmussen et al., 2010) reported a combined and weighted α =
.87 [.81, .93] (n = 1,165). O’Connor et al. (2008) and O’Connor, Whyte, et al. (2007) reported a com-
bined 2-month test–retest coefficient (n = 504) of rtt = .43 [.34, .52], which was quite low. Concurrent
validity studies using the Suicide Ideation subscale yielded a coefficient with the CES–D scale (Radloff,
1977) of .58 [.42, .74] (n = 143; Heisel et al., 2003), a combined and weighted correlation across seven
studies using the BHS (Beck & Steer, 1993) of r = .60 [.54, .66] (n = 1,169; Heisel et al., 2003; O’Connor,
Whyte, et al., 2008; O’Connor et al., 2013; O’Connor et al., 2007; Rasmussen et al., 2008; Rutter, 2006;
Rutter & Soucar, 2002), and a correlation across five studies using the Hospital Anxiety and Depression
Scale–Depression subscale (HADS–D; Zigmond & Snaith, 1983) of r = .47 [.40, .54] (n = 764; O’Connor
et al., 2008; O’Connor et al., 2013; O’Connor et al., 2007; Rasmussen et al., 2010; Rasmussen et al., 2008).
Nonclinical Suicidal Ideation subscale sample means for both sexes combined (n = 463) yielded a
mean of 9.0 and standard deviation of 5.6 across two studies (O’Connor, O’Connor, et al., 2007; Rutter,
2006). No diagnostic validity studies using the SPS Suicidal Ideation subscale alone were located.
Finally, several additional studies were located that used translated versions of the SPS and are
reported here for convenience, even though they were not included in the English version identifica-
tion and results process. Lee (2011) reported α for the Korean translation of the SPS to be .90 [.80, 1.00]
(n = 406), and a concurrent validity coefficient with the Reasons for Living Inventory (RFL; Guttierez
MEASUREMENT AND EVALUATION IN COUNSELING AND DEVELOPMENT 9

et al., 2002) of r = –.54 [–.44, –.64]. Eskin (1993b) reported α for the Swedish translation of the SPS to be
.87 [.60, 1.00] (n = 52) and a 1-month test–retest reliability coefficient of .89 [.62, 1.00] (n = 474). Finally,
four studies of the Turkish translation of the SPS were located (Dilli, Dallar, & Cakir, 2010; Eskin, 1993a;
Eskin, Ertekan, Dereboy, & Demirkiran, 2007; Zeyrek, Gencoz, Bergman, & Lester, 2009), resulting in a
combined weighted α of .88 [.82, .94] (n = 1,240). Eskin (1993a) reported a 1-month rtt of .95 [.64, 1.00]
(n = 41). Concurrent correlations with the Turkish SPS were –.42 [.29, .55] (n = 214) with the RFL (Dilli
et al., 2010), .62 [.47, .77] (n = 180) with the BHS (Zeyrek et al., 2009), and .70 [.63, .77] (n = 805) with
the CDI (Eskin et al., 2007).

Review and Comprehensive Psychometric Meta-Analysis of the Columbia–Suicide


Severity Rating Scale
The C–SSR was designed to help predict suicidal behavior in adolescents and adults (Columbia Univer-
sity, 2014). It was created in response to a need for a more comprehensive assessment of critical behaviors
that put clients at risk for suicidal behavior. The C–SSR is highly praised as a quick, easy, and compre-
Downloaded by [University of Florida] at 01:52 22 December 2017

hensive tool to identify clients across various populations for referral to a mental health professional due
to high suicide risk. There are different versions of the scale, designed to fit different clients (e.g., general
population, clinical population, pediatric population), and different situations (e.g., first visit, follow-up
visit). Administrators read the questions aloud to the client and mark down the response according to
whether or not the client has felt this way in his or her lifetime or in the past month. According to the
responses, the administrator proceeds through the questions in order, or skips to the next section. The
first part of the C–SSR determines if the client has suicidal ideation. If the client does, the second section
of the C–SSR determines the severity of the suicidal ideation. The third section determines if the client
has engaged in suicidal behavior in his or her lifetime or in the past 3 months, and whether or not these
behaviors were interrupted by the client or by others. The last section of the C–SSR collects a brief his-
tory of suicide attempts, if applicable. This section asks for the date and a professional assessment of the
client’s most recent suicide attempt, most lethal suicide attempt, and initial or first suicide attempt. These
attempts are classified by the administrator as having actual or potential lethality. Once completed, the
C–SSR is scored, and the administrator can access resources and consult on the next appropriate course
of action with the client. If the client responds positively to any item on the suicide ideation section (1–
5), then he or she is identified as having suicide ideation. In the intensity of ideation section, clients rate
their “lifetime most-severe ideation” and “recent most-severe ideation” on a scale from 1 to 5, where 1 is
the least severe and 5 is the most severe. The scores from this section are then added to give an intensity
score, ranging from 0 to 25. If the client responds positively to any item in the suicidal behavior section,
then he or she is identified as having suicidal behavior.
The C–SSR was standardized across three populations: adolescent suicide attempters (n = 124),
depressed adolescents in a medication efficacy trial (n = 312), and adults presenting with psychi-
atric issues to an emergency department (n = 237). Using the data collected in a study of adoles-
cent suicide attempters, the internal consistency of the intensity subscale of the C–SSR was α = .95,
whereas the samples of adults with psychiatric issues and adolescents in the medication efficacy trial
both produced much lower coefficient alphas (α = .73; Posner et al., 2011). Using the data collected
from the other two populations, the internal consistency of the intensity subscales for both visits
was moderate, with α = .73. Convergent and divergent validity were examined by comparing the
scores between the worst-point score on the SSI (Beck, Kovacs, & Weissman, 1979) with the Sever-
ity subscale of the C–SSR (r = .52, n = 472), and the Intensity subscale of the C–SSR (r = .56, n
= 487). There was a strong correlation between the C–SSR Severity subscale and the Montgomery-
Asberg depression rating scale (MADRS) (Montgomery & Asberg, 1979) suicidal ideation item
(r = .63) and the BDI (Beck, Steer, & Brown, 1996) suicide item (r = .80). The C–SSR has been studied
with different populations, including adolescents in the juvenile justice system (Kerr, Gibson, Leve, &
DeGarmo, 2014), and suicidal patients receiving depression interventions who were evaluated for effi-
cacy (Hesdorffer et al., 2013; Wang et al., 2013), and has been translated into many different languages
for use in 114 different countries (Columbia University, 2014).
10 B. T. ERFORD ET AL.

Computerized and hand searches identified 148 candidate articles submitted to full-text review, result-
ing in only 3 usable articles with a combined sample size of 917. Combined internal consistency across
three samples from the Posner et al. (2011) article was only α = .84 [.78, .90] (n = 917), which is adequate
for screening-level purposes (Erford, 2013), but far lower than the other three instruments reviewed ear-
lier. Brent et al. (2009) reported test–retest reliability separately for the suicidal ideation items (rtt = .90
[.79, 1.00], n = 334) and suicidal behavior items (rtt = 1.00 [.89, 1.00], n = 334), coefficients that are
very respectable, and higher than α. Posner et al. (2011) reported significant concurrent validity correla-
tions between the C–SSR and the SIQ–JR (Reynolds, 1988; j = 1, n = 234, r = .36 [.23, .39]), BSS (Beck
& Steer, 1991; j = 2, n = 683, r = .58 [.51, .65]), and Beck Lethality Scale (Beck, Shuyler, & Herman,
1974; j = 1, n = 234, r = .79 [.66, .92]). No EFA or CFA evidence, or descriptive statistics on nonclinical
samples, was provided in the literature. Finally, Posner et al. (2011) reported diagnostic validity evidence
in two studies using several cutoff schemes and resulting in perfect prediction of sensitivity (1.00) and
near-perfect specificity predictions (.96–1.00), the best of any of the four instruments included in this
analysis. Although decision reliability is the most essential element of a screening or diagnostic scale
of suicidal ideation and behavior, the lower αs and concurrent correlations, and lack of internal struc-
Downloaded by [University of Florida] at 01:52 22 December 2017

tural (EFA, CFA) evidence are concerning, and much more psychometric information is needed to better
understand and use the C–SSR.

Implications for Counseling Practitioners and Researchers


The ability to confidently use assessments that identify suicidal ideation when working with clients is
paramount for counselors. Some estimates indicate that approximately one in four mental health pro-
fessionals will experience a client suicide (Granello & Granello, 2007). Accurate identification is an
essential step in the prevention and effective management of suicide ideation. Although no instru-
ment can be 100% conclusive, it can be invaluable for practitioners to have viable instruments they
can credibly incorporate in clinical assessment. This is the bottom line: Standardized assessments
should be used in conjunction with a clinical interview, pneumonics (e.g., IS PATH WARM, SLAP, NO
HOPE), and assessment of risk factors to provide a comprehensive assessment of client lethality (Erford,
2018).
Meta-analysis has gained popularity in recent years as a valuable method to synthesize research into
a single, more comprehensible product. This meta-analysis reviewed studies of four commonly used
suicide ideation assessments to inform the selection of suicide instruments based on criteria deemed
important by the clinician. Factors such as administration time, level of administrator training, popula-
tion match, and psychometric strength can be useful in determining selection.
For example, the BSS (Beck & Steer, 1991) can be completed in 5 to 10 minutes, but its administration
requires a trained clinician. Two of the total seven studies included in the meta-analysis using the BSS
reported high internal consistency, but we could not identify any studies reporting test–retest data. There
was strong evidence of concurrent validity. Internal structural and diagnostic validity were reported, but
by only a single study. Despite lack of reported statistics for nonclinical samples, the full version of the
BSS should provide clinicians with accuracy in identifying suicide ideation in adolescents and adults,
although this finding warrants some caution due to the low number of studies identified.
The SPS (Cull & Gill, 1992) is similarly quick to administer and score, although a highly trained
clinician is required for its administration to adolescents and adults. Our analysis of 38 articles identified
strong internal consistency across 10 studies, and test–retest stability in two studies. Strong evidence
of concurrent validity was also reported, as well as promising results pertaining to diagnostic validity.
However, studies reporting data fit for the four-factor model were inconclusive. The brief version of the
SPS, the Suicide Ideation subscale, was also evaluated across eight articles, with comparable psychometric
strength, although we were unable to identify any studies reporting data on diagnostic validity.
The SIQ (Reynolds, 1988) and SIQ–JR are intended for specific use with adolescents, and we were
able to identify 35 articles for this meta-analysis. Very high internal consistency across 14 studies was
reported for the SIQ, although test–retest reliability was reported in only a single study. Although the
SIQ displayed robust concurrent validity, similar to the BSS, we identified only one internal structural
MEASUREMENT AND EVALUATION IN COUNSELING AND DEVELOPMENT 11

Table . Characteristics and Meta-Analysis Psychometric Summary of the Beck Scale for Suicide Ideation (BSS), Suicide Ideation Ques-
tionnaire (SIQ), Suicide Probability Scale (SPS), and Columbia–Suicide Severity Rating Scale (C–SSR).

Reliability
Scale Age Items Time Response j n α rtt (T)

BSS Adult  – Self-report  , . N/A


SIQ (& SIQ–JR) +   Self-report  , . . ( week)
SPS +   Self-report  , . . (– months)
C–SSR +   Interview   . .

Validity
Scale Convergent Internal Diagnostic Interpretation

BSS Moderate Minimal Strong Criterion (+)


SIQ (& SIQ–JR) Strong Minimal Moderate Criterion (/)
SPS Moderate-strong Minimal Poor Normed ()
C–SSR Moderate-strong Minimal Excellent Criterion (+)
Downloaded by [University of Florida] at 01:52 22 December 2017

validity study. Although the SIQ–JR resulted in mostly similar psychometric data, the reported accuracy
rate for this instrument was very low for use with a clinical population.
The C–SSR (Columbia University, 2014) is intended to be a more comprehensive suicide assessment
for both adolescents and adults, but we were able to identify only three usable articles to include in this
meta-analysis. It is a structured clinical interview in comparison to the other three self-report instru-
ments. Internal consistency for the C–SSR was adequate, but lower than the other three instruments
reviewed in this study. Test–retest reliability and concurrent validity were also reported, but we could
not identify any evidence for internal structural validity. It should be noted, however, that the C–SSR
had near-perfect sensitivity and specificity, meeting or surpassing the other three instruments reviewed.
Although counselor education programs using council for accreditation of counseling and related
educational programs (CACREP) accreditation standards focus on clinical interviewing and suicide
assessment procedures, counselors in training and clinical practice could benefit from instruction and
training in the use of standardized assessments of suicide ideation and behaviors. Only the BSS is cur-
rently commonly taught in counselor education programs (Neukrug et al., 2013) and commonly used
in clinical practice (Peterson et al., 2014), although it was used in outcome studies less frequently than
all other instruments except for the relative newcomer, the C–SSR (Columbia University, 2014). The C–
SSR is probably the most commonly used suicide ideation screening instrument in suicide and depres-
sion research because of lethality safeguards required by the National Institute of Mental Health when
conducting federally funded research with this population. Unfortunately, researchers use the C–SSR
without reporting on its psychometric properties with the samples studied.
Each of the reviewed standardized instruments is easy to administer, score, and interpret, and each
has strengths and weaknesses to consider along with use. Table 2 presents an aggregated summary of
these strengths and weaknesses. A weakness common to all of the instruments reviewed is a lack of
validity or response style scales clinicians could use to help determine whether a client is attempting to
create a positive impression (i.e., make the clinician believe the client does not display suicidal ideation
and tendencies when indeed the client does), or a negative impression (i.e., make the clinician believe the
client does display suicidal ideation and tendencies when indeed the client does not) that could invalidate
results.
So what are the best choices for use by professional counselors? The C–SSR is an excellent instrument
to incorporate into a clinical interview to provide a more structured approach and its outstanding diag-
nostic validity results will help bolster the accuracy of decisions. When considering use of self-report
instruments to supplement the clinical interview and triangulate with conclusions of the pneumonics
and risk factors, the BSS is the best choice for use with adults and the SIQ (and SIQ–JR) is the best choice
for use with adolescents. Even though the SPS was the most frequently cited suicide ideation instru-
ment in the extant literature, the poor showing in diagnostic validity studies counterindicates confidence
12 B. T. ERFORD ET AL.

in usefulness. Thus, counselor educators are encouraged to incorporate the C–SSR, BSS, and SIQ into
course instruction and training of professional counselors. Until professional counselors are exposed
to and instructed in use of these and other standardized instruments, decision accuracy will suffer and
the standard of care expected of the American Counseling Association (ACA) Code of Ethics (ACA,
2014) might come up short. Triangulation of structured clinical interview, self-report, pneumonics, and
risk factor assessments related to suicide ideation and behaviors should lead to robust, consistent, and
accurate clinical decisions.
As this meta-analysis has indicated, there is a need for more studies with these instruments using
larger and more diverse samples. Although these four instruments might provide clinicians with tools to
initiate conversations about suicide and provide a structure to the overall evaluation process, their clinical
utility may be limited. Because suicide rates are indicated to be higher with marginalized populations,
it is important that the utility and generalizability of these suicide ideation instruments are studied with
these populations. Randomized assignment of clients and pretest–posttest design methodology could be
useful. Cumulative evidence of the efficacy of these instruments is warranted, and clinicians should not
substitute clinical judgment and evaluation for suicide screening based on these measures alone.
Downloaded by [University of Florida] at 01:52 22 December 2017

There is a call for evidence-based practice in the counseling field to inform assessment and treat-
ment practices. Meta-analysis has been identified as one of the most legitimate methods of summarizing
findings of empirically based research (Rosenthal et al., 2006). As such, researchers using assessment
instruments in clinical studies should consistently report reliability and validity data to facilitate psycho-
metric comparisons among different measures. As counselors become better consumers of research and
conduct quality studies, evidence for the selection of instrumentation for both practice and research will
improve.

Study Limitations
A systematic set of criteria were followed in the selection of articles for this meta-analysis. Extensive
electronic and hand searches of published literature were undertaken with the qualifiers that each study
reviewed must be in English and have some type of reliability or validity data. The results were then
pooled and analyzed quantitatively to further evaluate, compare, and contrast the four most commonly
used suicide ideation instruments.
Rosenthal, Hoyt, Ferrin, Miller, and Cohen (2006) pointed out that although using published literature
is a common methodological procedure, it does lend itself to three potential biases that are a limitation of
the findings. These include publication, search, and selection biases. Because sometimes only those stud-
ies with positive results are published, there is the possibility that publication bias has occurred. Because
studies with no significant results or even negative results might not have been chosen for publication
in a peer-reviewed journal, which was a criterion for inclusion in this meta-analysis, it was impossible
to include the results of those studies that in turn might have altered the analysis results. Additionally,
only articles that had been published in English were included, which means there could have been
other-language studies with potential impact not included. Not all journals have the same rigor for con-
sideration, so quality among the articles included might also vary.
There is the potential that search bias might have occurred in the selection process. Even though
strict criteria and procedures for inclusion were followed, relevant studies might have been omitted. The
key search words used included the instrument name and acronym—however, there is a possibility that
pertinent articles were excluded.
Selection bias is another potential limitation. The initial identification phase in the meta-analysis
process can produce long lists of potential studies for inclusion. On further analysis, many of these studies
might not be relevant. Discrepancies might have occurred also if studies of lower quality met the selection
criteria.
The most important limitation for this study was lack of psychometric data for some of the suicide
instruments included in this meta-analysis. Although the BSS generated the most candidate articles (420)
from 1993 to 2013, only seven articles met the selection criteria for English language and reported validity
and reliability data, significantly limiting the available studies with published psychometric data that we
MEASUREMENT AND EVALUATION IN COUNSELING AND DEVELOPMENT 13

could analyze. Similarly, searches for the SPS resulted in 188 candidate articles, but only 38 were fit for
inclusion. The C–SSR followed suit, with 148 candidate articles, leading to only three selected articles.
Finally, searches for the SIQ produced the lowest number of candidate articles, at 125, but yielded 35
included articles. Even those studies that fit the inclusion criteria did not consistently assess reliability
and validity for their chosen instruments, further limiting confidence for the recommendation of one
instrument over the other. Researchers are urged to report study reliability and validity data to help
bolster our collective understanding of how these instruments can be most effectively used in research
and clinical practice.

Notes on Contributors
Bradley T. Erford, Ph.D., is Professor in the Human Development Counseling Program of the Department of Human and
Organizational Development at Peabody College at Vanderbilt University.
Jessica Jackson, M.Ed., is a graduate of the school counseling program of the Education Specialties Department in the
School of Education at Loyola University Maryland.
Downloaded by [University of Florida] at 01:52 22 December 2017

Gerta Bardhoshi, Ph.D., is an assistant professor in the Department of Rehabilitation and Counseling in the School of
Education at the University of Iowa.
Kelly Duncan, Ph.D., is Dean and professor at Northern University, Aberdeen, SD.
Zumra Atalay is an associate professor and counseling department chair at MEF University, Istanbul, Turkey.

References
References marked with as asterisk indicate studies included in the meta-analysis.

Abdel-Khalek, A., & Lester, D. (2007). The psychometric properties and correlates of the Reynolds’ Sui-
cide Ideation Questionnaire with Kuwaiti and American students. Archives of Suicide Research, 11, 309–319.
doi:10.1080/13811110600896236

Alexandrino-Silva, C., Pereira, M. L., Bustamante, C., Ferraz, A. C., Baldassin, S., Andrade, A. G., & Alves, T. C. (2009).
Suicidal ideation among students enrolled in healthcare training programs: A cross-sectional study. Revista Brasileira
de Psiquiatria, 31, 338–344. doi:10.1590/S1516-44462009005000006
American Counseling Association. (2014). ACA code of ethics. Retrieved from http://www.counseling.org/
docs/ethics/2014-aca-code-of-ethics.pdf?sfvrsn=4
American Educational Research Association, American Psychological Association, and National Council on Measurement
in Education. (2014). Standards for educational and psychological testing. Arlington, VA: American Psychological Asso-
ciation.

Anderson, K. P., Townsend, S. T., Everly, G. S., Jr., & Lating, J. (1995). Suicide risk assessment in crisis situations: Clinical
utility of an abbreviated self-report measure. Crisis Intervention & Time-Limited Treatment, 1, 225–230.

Ang, R. P., & Ooi, Y. P. (2004). Impact of gender and parents’ marital status on adolescents’ suicidal ideation. The Interna-
tional Journal of Social Psychiatry, 50, 351–360.

Bagge, C., & Osman, A. (1998). The Suicide Probability Scale: Norms and factor structure. Psychological Reports, 83, 637–
638. doi:10.2466/pr0.1998.83.2.637
Beck, A. T. (1993). Beck Anxiety Inventory. San Antonio, TX: Psychological Corporation.
Beck, A. T., Kovacs, M., & Weissman, A. (1979). Assessment of suicidal intention: The scale for suicide intention. Journal
for Consulting and Clinical Psychology, 47, 343–352. doi:10.1037/0022-006X.47.2.343
Beck, A.T., Shuyler, D., & Herman, I. (1974). Development of suicidal intent scales. In A.T. Beck, H. L. P. Resnik, & D. J.
Lettieri (Eds.), The prediction of suicide (pp. 45–56). Bowie, MD: Charles Press.
Beck, A. T., & Steer, R. A. (1991). Manual for Beck Scale for Suicide Ideation. San Antonio, TX: Psychological Corporation
Beck, A. T., & Steer, R. (1993). Manual for the Beck Hopelessness Scale. San Antonio, TX: Psychological Corporation.
Beck, A. T., Steer, R. A., & Brown, G. K. (1996). Manual for the Beck Depression Inventory—Second edition. San Antonio,
TX: Psychological Corporation.
Beck, A. T., Steer, R. A., & Ranieri, W. F. (1988). Scale for suicide ideation: Psychometric properties of a
self-report version. Journal of Clinical Psychology, 44, 499–505. doi:10.1002/1097-4679(198807)44:4<499::AID-
JCLP2270440404>3.0.CO;2-6

Bergen, J. E. (1999). A model of suicide risk for sixth graders: Initial empirical evidence. Dissertation Abstracts Interna-
tional: Section B: The Sciences and Engineering, 60(4-B), 1883.

Bisconer, S. W., & Gross, D. M. (2007). Assessment of suicide risk in a psychiatric hospital. Professional Psychology: Research
and Practice, 38, 143–149. doi:10.1037/0735-7028.38.2.143
14 B. T. ERFORD ET AL.


Borthick, M. J. (1998). Gender role conflict and suicidal ideation in a late adolescent and young adult population: Age
18–24 sources. Dissertation Abstracts International: Section B: The Sciences and Engineering, 58(8-B), 4437.

Bowen, R., Baetz, M., Leuschen, C., & Kalynchuk, L. E. (2011). Predictors of suicidal thoughts: Mood instability versus
neuroticism. Personality & Individual Differences, 51, 1034–1038. doi:10.1016/j.paid.2011.08.015

Brent, D. A., Emslie, G. J., Clarke, G. N., Asarnow, J., Spirito, A., Ritz, L., & Vitiello, B. (2009). Predictors of sponta-
neous and systematically assessed suicidal adverse events in the treatment of SSRI-resistant depression in adolescents
(TORDIA) study. American Journal of Psychiatry, 166, 418–426. doi:10.1176/appi.ajp.2008.08070976
Butcher, J. N., Dahlstrom, W. G., Graham, J. R., Tellegen, A, & Kaemmer, B. (1989). The Minnesota Multiphasic Personality
Inventory–2 (MMPI–2): Manual for administration and scoring. Minneapolis, MN: University of Minnesota Press.
Centers for Disease Control and Prevention. (2012). Youth Risk Behavior Surveillance—United States, 2011 Surveillance
summaries. MMWR, 61(SS04), 1–162.

Chang, E. C. (2002). Examining the link between perfectionism and psychological maladjustment: Social problem solving
as a buffer. Cognitive Therapy & Research, 26, 581–595. doi:10.1023/A:1020329625158
Chesin, M., & Stanley, B. (2013). Risk assessment and psychosocial interventions for suicidal patients. Bipolar Disorders,
15, 584–593. doi:10.1111/bdi.12092
Chioqueta, A. P., & Stiles, T. C. (2006). Psychometric properties of the Beck Scale for Suicide Ideation: A Norwegian study
with university students. Nordic Journal of Psychiatry, 60, 400–404. doi:10.1080/08039480600937645

Cochrane-Brink, K. A., Lofchy, J. S., & Sakinofsky, I. (2000). Clinical rating scales in suicide risk assessment. General
Downloaded by [University of Florida] at 01:52 22 December 2017

Hospital Psychiatry, 22, 445–451. doi:10.1016/S0163-8343(00)00106-7


Columbia University. (2014). Understanding the C–SSRS. Retrieved from http://www.cssrs.columbia.edu/

Cross, T. L., Cassady, J. C., & Miller, K. A. (2006). Suicide ideation and personality characteristics among gifted adolescents.
Gifted Child Quarterly, 50, 295–306. doi:10.1177/001698620605000403
Cull, J. G., & Gill, W. S. (1992). Manual for the Suicide Probability Scale. Los Angeles, CA: Western.

Daigle, M. S., Labelle, R., & Côté, G. (2006). Further evidence of the validity of the Suicide Risk Assessment Scale for
prisoners. International Journal of Law and Psychiatry, 29, 343–354. doi:10.1016/j.ijlp.2006.01.004

Dear, G. E. (2000). Functional and dysfunctional impulsivity, depression, and suicidal ideation in a prison population.
Journal of Psychology, 134, 77–80. doi:10.1080/00223980009600850

Demirbas, H., Celik, S., Ilhan, I. O., & Doğan, Y. B. (2003). An examination of suicide probability in alcoholic inpatients.
Alcohol and Alcoholism, 38, 67–70.

Diamond, G., Creed, T., Gillham, J., Gallop, R., & Hamilton, J. L. (2012). Sexual trauma history does not moderate treat-
ment outcome in attachment-based family therapy (ABFT) for adolescents with suicide ideation. Journal of Family
Psychology, 26, 595–605. doi:10.1037/a0028414
Dıllı, D., Dallar, Y., & Çakir, İ. (2010). Psychological characteristics of adolescent suicide attempters presenting to a pediatric
emergency service. Turkish Journal of Medical Sciences, 40, 377–390. doi:10.3906/sag-0901-18
Domino, G., Moore, D., Westlake, L., & Gibson, L. (1982). Attitudes toward suicide: A factor analytic approach. Journal of
Clinical Psychology, 38, 257–262.

D’Zurilla, T. J., & Chang, E. C. (1998). Social problem-solving deficits and hopelessness, depression, and suicidal
risk in college. Journal of Clinical Psychology, 54, 1091–1107. doi:10.1002/(SICI)1097-4679(199812)54:8<1091::AID-
JCLP9>3.0.CO;2-J

Ekramzadeh, S., Javadpour, A., Draper, B., Mani, A., Withall, A., & Sahraian, A. (2012). Prevalence and correlates of suici-
dal thought and self-destructive behavior among an elderly hospital population in Iran. International Psychogeriatrics,
24, 1402–1408. doi:10.1017/S1041610212000245

Eltz, M., Evans, A. S., Celio, M., Dyl, J., Hunt, J., Armstrong, L., & Spirito, A. (2007). Suicide Probability Scale and its utility
with adolescent psychiatric patients. Child Psychiatry and Human Development, 38(1), 17–29. doi:10.1007/s10578-006-
0040-7
Erford, B. T. (2013). Assessment for counselors (2nd ed.). Belmont, CA: Cengage Wadsworth.
Erford, B. T. (Ed.). (2018). Orientation to the counseling profession: Advocacy, ethics and essential professional foundations
(3rd ed.). Columbus, OH: Pearson Education.

Eryilmaz, A. (2012). A model for subjective well-being in adolescence: Need satisfaction and reasons for living. Social
Indicators Research, 107, 561–574. doi:10.1007/s11205-011-9863-0
Eskin, M. (1993a). Reliability of the Turkish version of the Perceived Social Support from Friends and Family scales, Scale
for Interpersonal Behavior, and Suicide Probability Scale. Journal of Clinical Psychology, 49, 515–522. doi:10.1002/1097-
4679(199307)49:4<515::AID-JCLP2270490408>3.0.CO;2-K
Eskin, M. (1993b). Swedish translations of the Suicide Probability Scale, Perceived Social Support from Friends and Family
Scales, and the Scale for Interpersonal Behavior: A reliability analysis. Scandinavian Journal of Psychology, 34, 276–281.
doi:10.1111/j.1467-9450.1993.tb01122.x

Eskin, M., Ertekin, K., & Demir, H. (2008). Efficacy of a problem-solving therapy for depression and suicide potential in
adolescents and young adults. Cognitive Therapy & Research, 32, 227–245. doi:10.1007/s10608-007-9172-8
Eskin, M., Ertekin, K., Dereboy, C., & Demirkiran, F. (2007). Risk factors for and protective factors against adolescent
suicidal behavior in Turkey. Crisis, 28, 131–139. doi:10.1027/0227-5910.28.3.131

Firestone, R. W., & Firestone, L. (1998). Voices in suicide: The relationship between self-destructive thought processes,
maladaptive behavior, and self-destructive manifestations. Death Studies, 22, 411–443. doi:10.1080/074811898201443
MEASUREMENT AND EVALUATION IN COUNSELING AND DEVELOPMENT 15


Gençöz, F., Vatan, S., Walker, R. L., & Lester, D. (2007). Hopelessness and suicidality in Turkish and American respondents.
Omega: Journal of Death and Dying, 55, 311–319. doi:10.2190/OM.55.4.e
Granello, D. H., & Granello, P. F. (2007). Suicide: An essential guide for helping professionals and educators. Boston, MA:
Allyn & Bacon.

Gutierrez, P. M., Osman, A., Barrios, F. X., & Kopper, B. A. (2001). Development and initial validation of the Self-Harm
Behavior Questionnaire. Journal of Personality Assessment, 77, 475–490. doi:10.1207/S15327752JPA7703_08

Gutierrez, P. M., Osman, A., Barrios, F. X., Kopper, B. A., Baker, M. T., & Haraburda, C. M. (2002). Development of the
Reasons for Living Inventory for young adults. Journal of Clinical Psychology, 58, 339–357. doi:10.1002/jclp.1147
Han, B., Compton, W. M., Gfroerer, J., & McKeon, R. (2014). Mental health treatment patterns among adults with suicide
attempts in the United States. American Journal of Public Health, 104, 2359–2368. doi:10.2105/AJPH.2014.302163
Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta-analysis. Orlando, FL: Academic Press.

Heisel, M. J., Flett, G. L., & Hewitt, P. L. (2003). Social hopelessness and college student suicide ideation. Archives of Suicide
Research, 7, 221–235. doi:10.1080/13811110301557
Hesdorffer, D., French, J., Posner, K., DiVentura, B., Pollard, J., Sperling, M., & Kanner, A. (2013). Suicidal ideation and
behavior screening in intractable focal epilepsy eligible for drug trials. Epilepsia, 54, 879–887. doi:10.1111/epi.12128

Hewitt, P. L., Newton, J., Flett, G. L., & Callander, L. (1997). Perfectionism and suicide ideation in adolescent psychiatric
patients. Journal of Abnormal Child Psychology, 25, 95–101. doi:10.1023/A:1025723327188

Hirsch, J. K. (2004). An examination of optimism, pessimism and hopelessness as predictors of suicidal ideation in college
Downloaded by [University of Florida] at 01:52 22 December 2017

students: A test of a moderator model. Dissertation Abstracts International: Section B: The Sciences and Engineering,
64(10-B), 5218.

Horesh, N., & Apter, A. (2006). Self-disclosure, depression, anxiety, and suicidal behavior in adolescent psychiatric inpa-
tients. Crisis: The Journal of Crisis Intervention and Suicide Prevention, 27, 66–71. doi:10.1027/0227-5910.27.2.66

Huth-Bocks, A. C., Kerr, D. C., Ivey, A. Z., Kramer, A. C., & King, C. A. (2007). Assessment of psychiatrically hospitalized
suicidal adolescents: Self-report instruments as predictors of suicidal thoughts and behavior. Journal of the American
Academy of Child and Adolescent Psychiatry, 46, 387–395. doi:10.1097/chi.0b013e31802b9535

Kaminer, Y., Burleson, J. A., Goldston, D. B., & Burke, R. H. (2006). Suicidal ideation among adolescents with alcohol use
disorders during treatment and aftercare. American Journal on Addictions, 15, 43–49. doi:10.1080/10550490601006154

Kaplan, K. J., Ficker, L., Wallrabenstein, I., Dodge, N., Laird, P.-G., Thiel, K., & Folk, M. (2007). Why does Zeno the stoic
hold his breath? “Zenoism” as a new variable for studying suicide. Omega: Journal of Death and Dying, 56, 369–400.
doi:10.2190/OM.56.4.e
Kazdin, A. E., Rodgers, A., & Colbus, D. (1986). The Hopelessness Scale for Children: Psychometric characteristics and
concurrent validity. Journal of Consulting and Clinical Psychology, 54, 241–245.

Keane, E. M., Dick, R. W., Bechtold, D. W., & Manson, S. M. (1996). Predictive and concurrent validity of the Suici-
dal Ideation Questionnaire among American Indian adolescents. Journal of Abnormal Child Psychology, 24, 735–747.
doi:10.1007/BF01664737
Kerr, D. R., Gibson, B., Leve, L. D., & DeGarmo, D. S. (2014). Young adult follow-up of adolescent girls in juve-
nile justice using the Columbia Suicide Severity Rating Scale. Suicide & Life-Threatening Behavior, 44, 113–129.
doi:10.1111/sltb.12072

King, C. A., Hill, E. M., Naylor, M., Evans, T., & Shain, B. (1993). Alcohol consumption in relation to other predictors
of suicidality among adolescent girls. Journal of the American Academy of Child and Adolescent Psychiatry, 32, 82–88.
doi:10.1097/00004583-199301000-00012
Klosterman-Fields, S. J. (1985). Bulimia, binge eating, and suicidal ideation among college women (Unpublished doctoral
dissertation). University of Wisconsin, Madison, WI.

Koldsland, B. O., Mehlum, L., Mellesdal, L. S., Walby, F. A., & Diep, L. M. (2007). The suicide assessment scale: Psycho-
metric properties of a Norwegian language version. BMC Research Notes, 5, 417. doi:10.1186/1756-0500-5-417

Kopper, B. A., Osman, A., & Barrios, F. X. (2001). Assessment of suicidal ideation in young men and women: The incre-
mental validity of the MMPI–2 content scales. Death Studies, 25, 593–607. doi:10.1080/074811801753184318
Kovacs, M. (1992). Manual for the Children’s Depression Inventory. North Tonawanda, NY: Multi-Health Systems.

Kumar, D., Atanu, K., Basu, S., & Das, S. (2008). The roles of personality, stressful life events, meaning in life, reasons
for living on suicidal ideation: A study in college students. Journal of Projective Psychology & Mental Health, 15(1),
52–57.

Larzelere, R. E., Smith, G. L., Batenhorst, L. M., & Kelly, D. B. (1996). Predictive validity of the Suicide Probability Scale
among adolescents in group home treatment. Journal of the American Academy of Child and Adolescent Psychiatry, 35,
166–172. doi:10.1097/00004583-199602000-00009

Lee, J., Choi, H., Kim, M. J., Park, C. G., & Shin, D.-S. (2009). Anger as a predictor of suicidal ideation in middle-school
students in Korea: Gender difference in threshold point. Adolescence, 44, 433–446.
Lee, S. Y. (2011). Reasons for living and their moderating effects on Korean adolescents’ suicidal ideation. Death Studies,
35, 711–728. doi:10.1080/07481187.2011.553316

Lee, W. V. (1992). Concurrent validity of the Suicide Probability Scale. Dissertation Abstracts International, 53(5-A), 1402.

Lester, D. (2004). Derivation of a proxy measure of suicidal ideation from the Suicide Opinion Questionnaire. Perceptual
and Motor Skills, 99, 1046. doi:10.2466/pms.99.3.1046-104
16 B. T. ERFORD ET AL.

Linehan, M. M., Goodstein, J. L., Nielsen, S. L., & Chiles, J. A. (1983). Reasons for staying alive when you are thinking of
killing yourself: The Reasons for Living Inventory. Journal of Consulting and Clinical Psychology, 51, 276–286.

Lynch, T. R., Cheavens, J. S., Morse, J. Q., & Rosenthal, M. Z. (2004). A model predicting suicidal ideation and hopelessness
in depressed older adults: The impact of emotion inhibition and affect intensity. Aging & Mental Health, 8, 486–497.
doi:10.1080/13607860412331303775
McIntosh, J. L., & Drapeau, C. W. (for the American Association of Suicidology). (2014). U.S.A. suicide 2012: Official final
data. Washington, DC: American Association of Suicidology.

Miller, A. B., & Esposito-Smythers, C. (2013). How do cognitive distortions and substance-related problems affect
the relationship between child maltreatment and adolescent suicidal ideation? Psychology of Violence, 3, 340–353.
doi:10.1037/a0031355
Montgomery, S. A., & Asberg, M. (1979). A new depression scale designed to be sensitive to change. British Journal of
Psychiatry, 134, 382–389.

Morrison, R., & O’Connor, R. C. (2008). The role of rumination, attentional biases and stress in psychological distress.
British Journal of Psychology, 99, 191–209. doi:10.1348/000712607X216080

Morse, J. Q., & Lynch, T. R. (2004). A preliminary investigation of self-reported personality disorders in late
life: Prevalence, predictors of depressive severity, and clinical correlates. Aging & Mental Health, 8, 307–315.
doi:10.1080/13607860410001709674

Mundt, J. C., Greist, J. H., Gelenberg, A. J., Katzelnick, D. J., Jefferson, J. W., & Modell, J. G. (2010). Feasibility and validation
Downloaded by [University of Florida] at 01:52 22 December 2017

of a computer-automated Columbia–Suicide Severity Rating Scale using interactive voice response technology. Journal
of Psychiatric Research, 44, 1224–1228. doi:10.1016/j.jpsychires.2010.04.025
Neukrug, E., Peterson, C. H., Bonner, M., & Lomas, G. I. (2013). A national survey of assessment instruments taught by
counselor educators. Counselor Education & Supervision, 52, 207–221. doi:10.1002/j.1556-6978.2013.00038.x
Nock, M. K., Green, J. G., Hwang, I., McLaughlin, K. A., Sampson, N. A., Zaslavsky, A. M., & Kessler, R. C. (2013). Preva-
lence, correlates, and treatment of lifetime suicidal behavior among adolescents: Results from the National Comorbid-
ity Survey Replication adolescent supplement. JAMA Psychiatry, 70, 300–310.

O’Connor, D. B., O’Connor, R. C., & Marshal, R. (2007). Perfectionism and psychological distress: Evidence of the medi-
ating effects of rumination. European Journal of Personality, 21, 429–452. doi:10.1002/per.616

O’Connor, R. C., Fraser, L., Whyte, M.-C., MacHale, S., & Masterton, G. (2008). A comparison of specific positive future
expectancies and global hopelessness as predictors of suicidal ideation in a prospective study of repeat self-harmers.
Journal of Affective Disorders, 110, 207–214. doi:10.1016/j.jad.2008.01.008

O’Connor, R. C., & Noyce, R. (2008). Personality and cognitive processes: Self-criticism and different types of rumination
as predictors of suicidal ideation. Behaviour Research and Therapy, 46, 392–401. doi:10.1016/j.brat.2008.01.007

O’Connor, R. C., Smyth, R., Ferguson, E. R., & Caoimhe Williams, J. M. G. (2013). Psychological processes and repeat
suicidal behavior: A four-source prospective study. Journal of Consulting and Clinical Psychology, 81, 1137–1143.
doi:10.1037/a0033751

O’Connor, R. C., Whyte, M.-C., Fraser, L., Masterton, G., Miles, J., & MacHale, S. (2007). Predicting short-term outcome
in well-being following suicidal behaviour: The conjoint effects of social perfectionism and positive future thinking.
Behaviour Research and Therapy, 45, 1543–1555. doi:10.1016/j.brat.2006.11.006

Osman, A., Gilpin, A. R., Panak, W. F., Kopper, B. A., Barrios, F. X., Gutierrez, P. M., & Chiros, C. E. (2000). The Multi-
attitude Suicide Tendency Scale: Further validation with adolescent psychiatric inpatients. Suicide & Life-Threatening
Behavior, 30, 377–385.
Osman, A., Gutierrez, P. M., Muehlenkamp, J. J., Dix-Richardson, F., Barrios, F. X., & Kopper, B. A. (2004). Suicide Resilience
Inventory–25: Development and preliminary psychometric properties. Psychological Reports, 94, 1349–1360.

Osman, A., & Kopper, B. A. (1998). The Reasons for Living Inventory for Adolescents (RFL–A): Develop-
ment and psychometric properties. Journal of Clinical Psychology, 54, 1063–1078. doi:10.1002/(SICI)1097-
4679(199812)54:8<1063::AID-JCLP6>3.0.CO;2-Z

Osman, A., Kopper, B. A., Linehan, M. M., Barrios, F. X., Gutierrez, P. M., & Bagge, C. L. (1999). Validation of the Suicide
Ideation Questionnaire and the Reasons for Living Inventory in an adult psychiatric inpatient sample. Psychological
Assessment, 11, 115–123. doi:10.1037/1040-3590.11.2.115

Otzenberger, J. H. (2003). Personality characteristics relating to suicide risk of incarcerated adolescent males as measured
by the Millon Adolescent Clinical Inventory and the Suicide Probability Scale. Dissertation Abstracts International:
Section B: The Sciences and Engineering, 64(6-B), 2934.
Peterson, C. H., Lomas, G. I., Neukrug, E. S., & Bonner, M. W. (2014). Assessment use by counselors in the United
States: Implications for policy and practice. Journal of Counseling & Development, 92, 90–98. doi: 10.1002/j.1556-
6676.2014.00134.x

Pienaar, J., & Rothmann, S. (2005). Suicide ideation in the South African police service. South African Journal of Psychol-
ogy, 35, 58–72.

Pienaar, J., Rothmann, S., & Van De Vijver, F. J. R. (2007). Occupational stress, personality traits, coping strate-
gies, and suicide ideation in the South African police service. Criminal Justice and Behavior, 34, 246–258.
doi:10.1177/0093854806288708
MEASUREMENT AND EVALUATION IN COUNSELING AND DEVELOPMENT 17

Pinninti, N., Steer, R., Rissmiller, D., Nelson, S., & Beck, A. (2002). Use of the Beck Scale for Suicide Ideation with psychiatric
inpatients diagnosed with schizophrenia, schizoaffective, or bipolar disorders. Behaviour Research and Therapy, 40,
1071–1079. doi:10.1016/S0005-7967(02)00002-5
Pinto, A., Whisman, M. A., & McCoy, K. M. (1997). Suicidal ideation in adolescents: Psychometric properties of the Suicidal
Ideation Questionnaire in a clinical sample. Psychological Assessment, 9(1), 63–66. doi:10.1037/1040-3590.9.1.63

Posner, K., Brown, G. K., Stanley, B., Brent, D. A., Yershova, K. V., Oquendo, M. A., … Mann, J. J. (2011). The Columbia–
Suicide Severity Rating Scale: Initial validity and internal consistency findings from three multisite studies with ado-
lescents and adults. American Journal of Psychiatry, 168, 1266–1277. doi:10.1176/appi.ajp.2011.10111704
Potard, C., Kubiszewski, V., Gimenes, G., & Courtois, R. (2014). Validation of the French version of the Suicidal Ideation
Questionnaire among adolescents. Psychiatry Research, 215, 471–476. doi:10.1016/j.psychres.2013.11.025

Prinstein, M. J., Boergers, J., Spirito, A., Little, T. D., & Grapentine, W. L. (2000). Peer functioning, family dysfunction, and
psychological symptoms in a risk factor model for adolescent inpatients’ suicidal ideation severity. Journal of Clinical
Child Psychology, 29, 392–405. doi:10.1207/S15374424JCCP2903_10
Radloff, L. S. (1977). The CES–D scale: A self-report depression scale for research in the general population. Applied Psy-
chological Measurements, 1, 385–401.

Rasmussen, S. A., Fraser, L., Gotz, M., MacHale, S., Mackie, R., Masterton, G., … O’Connor, R. C. (2010). Elaborating the
cry of pain model of suicidality: Testing a psychological model in a sample of first-time and repeat self-harm patients.
British Journal of Clinical Psychology, 49, 15–30. doi:10.1348/014466509X415735
Downloaded by [University of Florida] at 01:52 22 December 2017


Rasmussen, S. A., O’Connor, R. C., & Brodie, D. (2008). The role of perfectionism and autobiographical memory in a
sample of parasuicide patients: An exploratory study. Crisis: The Journal of Crisis Intervention and Suicide Prevention,
29, 64–72. doi:10.1027/0227-5910.29.2.64
Reynolds, W. M. (1987). Reynolds Adolescent Depression Scale: Professional manual. Odessa, FL: Psychological Assessment
Resources.
Reynolds, W. M. (1988). Reynolds Suicidal Ideation Questionnaire: Professional manual. Odessa, FL: Psychological Assess-
ment Resources.

Reynolds, W. M. (1990). Development of a semi-structured clinical interview for suicidal behaviors in adolescents. Psy-
chological Assessment, 2, 382–390. doi:10.1037/1040-3590.2.4.382

Reynolds, W. M. (1991). Psychometric characteristics of the Adult Suicide Ideation Questionnaire with college students.
Journal of Personality Assessment, 56, 289–307. doi:10.1207/s15327752jpa5602_9
Reynolds, W. M. (2002). Manual for the Reynolds Adolescent Depression Scale—Second edition (RADS–2). Lutz, FL: Psycho-
logical Assessment Resources.

Reynolds, W. M., & Mazza, J. J. (1999). Assessment of suicidal ideation in inner-city children and young adolescents:
Reliability and validity of the Suicidal Ideation Questionnaire–Jr. School Psychology Review, 28, 17–30.
Rosenthal, D. A., Hoyt, W. T., Ferrin, J. M., Miller, S., & Cohen, N. D. (2006). Advanced methods in meta-analytic
research: Applications and implications for rehabilitation counseling research. Rehabilitation Counseling Bulletin, 49,
234–246.

Rudd, M. D., Joiner, T. E., Jr., & Rumzek, H. (2004). Childhood diagnoses and later risk for multiple suicide attempts.
Suicide and Life-Threatening Behavior, 34, 113–125. doi:10.1521/suli.34.2.113.32784

Rutter, P. A. (2006). Young adult suicide and sexual orientation: What should counselors know? Journal of LGBT Issues in
Counseling, 1, 33–48. doi:10.1300/J462v01n03̱04

Rutter, P. A., Freedenthal, S., & Osman, A. (2008). Assessing protection from suicidal risk: Psychometric properties of the
Suicide Resilience Inventory. Death Studies, 32, 142–153. doi:10.1080/07481180701801295

Rutter, P. A., & Soucar, E. (2002). Youth suicide and sexual orientation. Adolescence, 37, 289–299.
Schützmann, K., Brinkmann, L., Schacht, M., & Richter-Appelt, H. (2009). Psychological distress, self-harming behav-
ior, and suicidal tendencies in adults with disorders of sex development. Archives of Sexual Behavior, 38(1), 16–33.
doi:10.1007/s10508-007-9241-9

Selby, E. A., Yen, S., & Spirito, A. (2013). Time varying prediction of thoughts of death and suicidal ideation in ado-
lescents: Weekly ratings over 6-month follow-up. Journal of Clinical Child and Adolescent Psychology, 42, 481–495.
doi:10.1080/15374416.2012.736356
Seligman, L., & Reichenberg, L. W. (2012). Theories of counseling and psychotherapy: Systems, strategies and skills (4th ed.).
Upper Saddle River, NJ: Prentice-Hall.

Shin, M. S. (1992). An empirical study of the mechanism of suicide: Validation of the Scale for Escape from Self (Unpublished
doctoral dissertation). Yonsei University, Seoul, Korea.

Spirito, A., Sterling, C. M., Donaldson, D. L., & Arrigan, M. E. (1996). Factor analysis of the Suicide Intent Scale with
adolescent suicide attempters. Journal of Personality Assessment, 67, 90–101. doi:10.1207/s15327752jpa6701_7

Steer, R. A., Rissmiller, D. J., Ranieri, W. F., & Beck, A. T. (1993). Dimensions of suicidal ideation in psychiatric inpatients.
Behaviour Research and Therapy, 31, 229–236. doi:10.1016/0005-7967(93)90090-H

Stewart, V. (2009). Perceived social support, suicidal ideation, stigma, and help seeking behaviors among college students.
Dissertation Abstracts International: Section B: The Sciences and Engineering, 70(4-B), 2589.

Steyn, R., Vawda, N., Wyatt, G. E., Williams, J. K., & Madu, S. N. (2013). Posttraumatic stress disorder diagnos-
tic criteria and suicidal ideation in a South African Police sample. African Journal of Psychiatry, 16, 19–22.
doi:10.4314/ajpsy.v16i1.3
18 B. T. ERFORD ET AL.


Tatman, S. M., Greene, A. L., & Karr, L. C. (1993). Use of the Suicide Probability Scale (SPS) with adolescents. Suicide &
Life-Threatening Behavior, 23, 188–203. doi:10.1111/j.1943-278X.1993.tb00179.x

Ustad, K. L. (1998). Assessment of malingering in a jail referral population: Screening and comprehensive evaluation.
Dissertation Abstracts International: Section B: The Sciences and Engineering, 58(7-B), 3937.

Walker, R. L., Lester, D., & Joe, S. (2006). Lay theories of suicide: An examination of culturally relevant suicide beliefs
and attributions among African Americans and European Americans. Journal of Black Psychology, 32, 320–334.
doi:10.1177/0095798406290467
Wang, H., Xue, Y., Chen, Y., Zhang, R., Wang, H., Zhang, Y., … Tan, Q. (2013). Efficacy of repetitive transcranial magnetic
stimulation in the prevention of relapse of depression: Study protocol for a randomized controlled trial. Trials, 14, 338.
doi:10.1186/1745-6215-14-338
Way, B. B., Kaufman, A. R., Knoll, J. L., & Chlebowski, S. M. (2013). Suicidal ideation among inmate-patients in state
prison: Prevalence, reluctance to report, and treatment preferences. Behavioral Sciences & the Law, 31, 230–238.
doi:10.1002/bsl.2055

Weber, B., Metha, A., & Nelsen, E. (1997). Relationships among multiple suicide ideation risk factors in college students.
Journal of College Student Psychotherapy, 11, 49–64. doi:10.1300/J035v11n03_0

Wu, Y., & Newfield, S. A. (2007). Comparing data collected by computerized and written surveys for adolescence health
research. Journal of School Health, 77, 23–28. doi:10.1111/j.1746-1561.2007.00158.x

Yen, S., Gagnon, K., & Spirito, A. (2013). Borderline personality disorder in suicidal adolescents. Personality and Mental
Downloaded by [University of Florida] at 01:52 22 December 2017

Health, 7, 89–101. doi:10.1002/pmh.1216



Yen, S., Weinstock, L. M., Andover, M. S., Sheets, E. S., Selby, E. A., & Spirito, A. (2013). Prospective pre-
dictors of adolescent suicidality: 6-month post-hospitalization follow-up. Psychological Medicine, 43, 983–993.
doi:10.1017/S0033291712001912
Zeyrek, E. Y., Gencoz, F., Bergman, Y., & Lester, D. (2009). Suicidality, problem-solving skills, attachment style, and hope-
lessness in Turkish students. Death Studies, 33, 815–827. doi:10.1080/07481180903142407
Zhang, Y., Yip, P., & Fu, K. (2014). Validation of the Chinese version of the Reynolds’ Suicidal Ideation Questionnaire:
Psychometric properties and its short version. Health and Quality of Life Outcomes, 12, 33. doi:10.1186/1477-7525-12-
33
Zigmond, A. S., & Snaith, R. P. (1983). The Hospital Anxiety and Depression Scale. Acta Psychiatrica Scandinavica, 67,
361–370. doi:10.1111/j.1600-0447.1983.tb09716.x