Research

JAMA Psychiatry | Original Investigation

Severity and Variability of Depression Symptoms

Predicting Suicide Attempt in High-Risk Individuals
Nadine M. Melhem, PhD; Giovanna Porta, MS; Maria A. Oquendo, MD, PhD; Jamie Zelazny, PhD;
John G. Keilp, PhD; Satish Iyengar, PhD; Ainsley Burke, PhD; Boris Birmaher, MD; Barbara Stanley, PhD;
J. John Mann, MD; David A. Brent, MD

Supplemental content
IMPORTANCE Predicting suicidal behavior continues to be among the most challenging tasks
in psychiatry.

OBJECTIVES To examine the trajectories of clinical predictors of suicide attempt (specifically,

depression symptoms, hopelessness, impulsivity, aggression, impulsive aggression,
and irritability) for their ability to predict suicide attempt and to compute a risk score
for suicide attempts.

DESIGN, SETTING, AND PARTICIPANTS This is a longitudinal study of the offspring of parents
(or probands) with mood disorders who were recruited from inpatient units at Western
Psychiatric Institute and Clinic (Pittsburgh) and New York State Psychiatric Institute.
Participants were recruited from July 15, 1997, to September 6, 2005, and were followed up
through January 21, 2014. Probands and offspring (n = 663) were interviewed at baseline and
at yearly follow-ups for 12 years. Lifetime and current psychiatric disorders were assessed,
and self-reported questionnaires were administered. Model evaluation used 10-fold
cross-validation, which split the entire data set into 10 equal parts, fit the model to 90% of
the data (training set), and assessed it on the remaining 10% (test set) and repeated that
process 10 times. Preliminary analyses were performed from July 20, 2015, to October 5,
2016. Additional analyses were conducted from July 26, 2017, to July 24, 2018.

MAIN OUTCOMES AND MEASURES The broad definition of suicide attempt included actual,
interrupted, and aborted attempts as well as suicidal ideation that prompted emergency
referrals during the study. The narrow definition referred to actual attempt only.

RESULTS The sample of offspring (n = 663) was almost equally distributed by sex (316 female
[47.7%]) and had a mean (SD) age of 23.8 (8.5) years at the time of censored observations.
Among the 663 offspring, 71 (10.7%) had suicide attempts over the course of the study.
The trajectory of depression symptoms with the highest mean scores and variability over
time was the only trajectory to predict suicide attempt (odds ratio [OR], 4.72; 95% CI,
1.47-15.21; P = .01). In addition, we identified the following predictors: younger age (OR, 0.82;
Author Affiliations: Department of
95% CI, 0.74-0.90; P < .001), lifetime history of unipolar disorder (OR, 4.71; 95% CI,
Psychiatry, University of Pittsburgh
1.63-13.58; P = .004), lifetime history of bipolar disorder (OR, 3.4; 95% CI, 0.96-12.04; School of Medicine, Pittsburgh,
P = .06), history of childhood abuse (OR, 2.98; 95% CI, 1.40-6.38; P = .01), and proband Pennsylvania (Melhem, Zelazny,
actual attempt (OR, 2.24; 95% CI, 1.06-4.75; P = .04). Endorsing a score of 3 or higher on the Birmaher, Brent); University of
Pittsburgh Medical Center,
risk score tool resulted in high sensitivity (87.3%) and moderate specificity (63%; area under
Pittsburgh, Pennsylvania (Porta);
the curve = 0.80). Department of Psychiatry, Perelman
School of Medicine, University of
CONCLUSIONS AND RELEVANCE The specific predictors of suicide attempt identified are those Pennsylvania, Philadelphia
(Oquendo); Department of
that clinicians already assess during routine psychiatric evaluations; monitoring and treating Psychiatry, Columbia University,
depression symptoms to reduce their severity and fluctuation may attenuate the risk New York, New York (Keilp, Burke,
for suicidal behavior. Stanley, Mann); Department of
Statistics, University of Pittsburgh,
Pittsburgh, Pennsylvania (Iyengar).
Corresponding Author: Nadine M.
Melhem, PhD, Department of
Psychiatry, University of Pittsburgh
School of Medicine, 3811 O’Hara St,
JAMA Psychiatry. 2019;76(6):603-612. doi:10.1001/jamapsychiatry.2018.4513 Pittsburgh, PA 15213 (melhemnm@
Published online February 27, 2019. upmc.edu).

(Reprinted) 603
© 2019 American Medical Association. All rights reserved.

Downloaded From: https://jamanetwork.com/ on 06/04/2022

 Research Original Investigation
S
uicide is the second leading cause of death in the United
States among people aged 15 to 34 years.1 History of at-
tempts, mood disorders, and substance use disorders are
the most important clinical predictors of suicidal behavior.2-9 Ad-
ditional predictors include family history of suicidal behavior8,10;
impulsive aggression8,10,11; lethality of method12; insomnia12,13;
hopelessness, anxiety, and agitation7,14-18; and childhood adver-
sity and negative life experiences.19-25 In a meta-analysis of 50
years of research, the prediction of suicidal ideation, attempt, and
suicide deaths with these predictors was only slightly better than
chance.26 Machine learning approaches have been applied to es-
timating the risk for suicidal behavior using electronic health
records27-30 with improved performance.28,30,31 Predictors in-
cluded lifetime psychiatric diagnoses and substance abuse.28-31
Psychiatric diagnoses are well-established indicators of sui-
cidal behavior. However, diagnoses and stable or trait-like pre-
dictors are of limited value because the risk of suicidal behav-
ior varies during the course of psychiatric illness. Thus, it is
important to identify symptoms that vary over time. We ex-
amined the trajectories of impulsivity, aggression, impulsive
aggression, depression symptoms, irritability, and hopeless-
ness in a longitudinal study of offspring of parents with mood
disorders. We examined whether changes in these measures
over time predict attempt and time to onset of attempt above
and beyond psychiatric diagnoses and other predictors. In ad-
dition, we computed a risk score based on our models and as-
sessed its performance.
Methods
The study protocol was approved by the institutional review
boards at Western Psychiatric Institute and Clinic (Pittsburgh,
Pennsylvania) and New York State Psychiatric Institute (New
York). Written informed consent was obtained from adults, and
written parental consent and assent were obtained for those
younger than 18 years.
Sample
The sample consisted of 663 offspring of 318 parents with mood
disorders from the larger sample of 711 offspring of 337 par-
ents (eFigure 1 in the Supplement). Parents are referred here-
after as probands. Probands were recruited from inpatient units
at Western Psychiatric Institute and Clinic and New York State
Psychiatric Institute. All probands had a history of mood dis-
order, and 180 (56.6%) of them had a lifetime history of an ac-
tual suicide attempt. We excluded 48 offspring who were lost
to follow-up after the baseline assessment. We compared them
with the remaining 663 offspring and found them substan-
tially different in some demographic and clinical characteris-
tics, with the excluded offspring having higher rates of mood
and psychotic disorders (eTable 1 in the Supplement). Partici-
pants were recruited from July 15, 1997, to September 6, 2005,
and were followed through January 21, 2014.
We used a broad and a narrow definition of suicide at-
tempt in offspring. The broad definition included an actual at-
tempt or a suicide-related behavior (eg, interrupted, aborted,
and ambiguous).32-34 We also included suicidal ideation that
604 JAMA Psychiatry June 2019 Volume 76, Number 6 (Reprinted)
© 2019 American Medical Association. All rights reserved.
Downloaded From: https://jamanetwork.com/ on 06/04/2022
Severity and Variability of Depression Symptoms for Predicting Suicide Attempt
Key Points
Question What are the most important clinical predictors of
suicide attempt?
Findings In this longitudinal study of 663 offspring of parents
with mood disorders, the trajectory of depression symptoms
showing the highest mean scores and variability over time
predicted suicide attempt above and beyond psychiatric
diagnoses. Additional predictors were younger age (ⱕ30 years),
mood disorders, childhood abuse, and personal and parental
history of suicide attempt.
Meaning Predictors were identified that clinicians already assess
during routine psychiatric evaluation; clinicians should especially
monitor and treat depression symptoms to reduce the risk for
suicidal behavior.
prompted emergency referral during the study, consistent with
treatment studies.8,35 Our safety procedures included emer-
gency referrals for those with suicidal ideation with intent
and a plan at the time of assessment and thus may have pre-
vented suicidal behavior. Here, we refer to the broad defini-
tion as suicide attempt and the narrow definition, including ac-
tual attempts only, as actual attempt.
Assessment
Probands and offspring were interviewed at baseline and
yearly follow-ups for 12 years. We assessed lifetime and
current psychiatric disorders at baseline as well as since the
last assessment and at current follow-ups using the Schedule
for Affective Disorders and Schizophrenia for School-Age
Children—Present and Lifetime Version36 and the Structured
Clinical Interview for DSM-IV.37 Similarly, suicidal behavior
was assessed using the Columbia Suicide History Form and
the Medical Lethality Rating Scale.38 Self-reported question-
naires were administered to assess the current severity of
symptoms: the Beck Depression Inventory39 for those older
than 18 years and the Children’s Depression Inventory 40
for those younger than 18 years; the Beck Hopelessness
Scale 41 and the Hopelessness Scale for Children 42 ; the
Barratt Impulsiveness Scale43 in adults and the subscales of
emotionality, activity, sociability, and impulsivity in those
younger than 18 years 44 ; and the Buss-Durkee Hostility
Inventory 45 and its downward extension, the Children’s
Hostility Inventory,46 to measure impulsive aggression. The
Brown Goodwin Aggression Scale47 was used to assess life-
time history of aggressive behaviors at baseline, and
follow-up assessments covered the time frame since the last
assessment. We measured irritability using the irritability
items from the depression scales. History of childhood abuse
was assessed with the Childhood Trauma Questionnaire48
and an adapted instrument from the Abuse Dimensions
Inventory.49 Standardized scores were computed when dif-
ferent measures were used for those younger or older than
18 years.
Statistical Analysis
Preliminary analyses were performed from July 20, 2015, to
October 5, 2016. Additional analyses were conducted from July
jamapsychiatry.com
 Severity and Variability of Depression Symptoms for Predicting Suicide Attempt
26, 2017, to July 24, 2018. To identify the trajectories of symp-
toms for depression, hopelessness, irritability, impulsivity, ag-
gression, and impulsive aggression, we used the lcmm pack-
age in R (R Foundation for Statistical Computing) to fit mixed
models using latent classes. Observations were censored at the
time point prior to the first suicide attempt for offspring with
an event during the course of the study and at the time point
of last observation for those without attempt. We used the hlme
function in R to fit latent class linear mixed models, which take
into account within-subject variability. We used the unstruc-
tured variance-covariance matrix and allowed the variance-
covariance of the random effects to be class specific, which does
not impose any constraints on the model so as to better esti-
mate variability patterns. We used the Bayesian Information
Criteria to identify the best-fitting model. We examined the
concordance between the trajectories using percent agree-
ment and Cohen κ coefficient.
We compared the demographic and clinical characteris-
tics of those with and without attempt using χ2 tests; Fisher
exact test; and unpaired, 2-tailed t tests. Similarly, we com-
pared the resulting trajectories on these characteristics. We ap-
plied a Bonferroni correction for multiple comparisons for the
6 measures (2-sided α < .01 or α < .05/6 indicates statistical sig-
nificance): depression symptoms, hopelessness, irritability, im-
pulsivity, aggression, and impulsive aggression. We then com-
pared offspring with and without attempt on the resulting
trajectories for each of the 6 measures, and we examined
with logistic regression whether these trajectories predicted
attempt.
Regression models were conducted with the broad and nar-
row definitions of attempt. For all analyses, we used 2 sets of
regression models. Model 1 controlled for age, sex, race/
ethnicity, site, and income. We examined the trajectories of
income to account for changes over time (eFigure 2 in the
Supplement). We also controlled for proband lifetime history
of actual attempt because the study design included recruit-
ing probands, half of whom had such a history, and for off-
spring lifetime history of attempt at baseline. Model 2 con-
trolled for all variables in model 1 and for offspring lifetime
history of the most common psychiatric disorders in our sample
up to the time of attempt, history of childhood abuse, pro-
band history of bipolar disorders, psychiatric treatment at the
time point prior to attempt, and cluster B personality disor-
ders in offspring 18 years of age or older (n = 507). We then in-
cluded all symptom trajectories into 1 regression model.
Similarly, we used Cox proportional hazards regression
models to examine time to onset of attempt, with psychiatric
disorders and psychiatric treatment in model 2 as time-
varying covariates. Because of multiple siblings per family, we
estimated the variance-covariance matrix with correlated ob-
servations and computed clustered robust SEs. Regression
models were also used to predict incident attempt (broad defi-
nition). We were not able to use these models for incident ac-
tual attempt given their small number (n = 19). For model
evaluation, we used 10-fold cross-validation in which we split
the entire data set into 10 equal parts, fit the model to 90% of
the data (training set), and assessed it on the remaining 10%
(test set) and repeated that process 10 times.
jamapsychiatry.com
© 2019 American Medical Association. All rights reserved.
Downloaded From: https://jamanetwork.com/ on 06/04/2022
Original Investigation Research
We applied the multiple imputation by chained equa-
tions technique on covariates with missing data only in Stata,
ice command (StataCorp LLC).50,51 There were no missing-
ness or missing data on 1 variable for 75% of the sample, 12%
had missing data on 2 variables, and 13% had missing data on
3 or more variables. Similar results were obtained using the
original and imputed data sets; thus, we report the results from
the imputed data set. The final models on the nonimputed data
set are reported in eTable 16 and eTable 18 in the Supple-
ment. We also used machine learning regression approaches—
LASSO (least absolute selection and shrinkage operator)—by
using the glmnet package in R52 to determine a subset of pre-
dictors with the strongest effects. We computed a risk score
based on prediction models and examined area under the
curve, sensitivity, and specificity; positive predictive value; and
negative predictive value.
Results
The sample of offspring (n = 663) was almost equally distrib-
uted by sex (316 female [47.7%]) and had a mean (SD) age of
23.8 (8.5) years at the time of censored observations (Table 1).
Of the 663 offspring, 455 of 647 (70.3%) were white individu-
als, and the median (range) follow-up was 8.1 (1-15.4) years.
Compared with offspring who completed follow-up for at least
8 years, those with fewer than 8 years of follow-up or who were
lost to follow-up were younger (mean [SD] age, 22.1 [9.9] years
vs 26.2 [5.8] years; t = 6.70; df = 631.02; P < .001) and less likely
to have an anxiety disorder (21.3% vs 31.2%; χ 21= 8.36; P = .004).
Among the 663 offspring, 71 (10.7%) had suicide attempts
over the course of the study (eFigure 1 in the Supplement), 51
of which were first-time attempts for an incidence rate of 8.4%.
Among attempters, the mean (SD) number of suicide at-
tempts was 1.2 (0.6) and of actual attempts was 1.3 (0.7). The
mean (SD) lethality (for actual attempts only) was 1.7 (2), which
corresponded to physical damage for which medical atten-
tion was needed. The median (range) time from the last as-
sessment point to suicide attempt was 45 (1-126) weeks (30
[4-126] weeks for actual attempt).
Trajectory Analyses
We identified a 2-class model for hopelessness (mean [SD] score,
0.99 [0.78] vs –0.35 [0.34]; t185.9 = –21.5; P < .001; Cohen d = 2.74),
impulsivity (1.17 [0.60] vs –0.39 [0.57]; t641 = –29; P < .001; Co-
hen d = 2.70), aggression (1.13 [0.71] vs –0.26 [0.32]; t120.1 = –20.3;
P < .001; Cohen d = 3.4), and irritability (0.92 [0.60] vs –0.29
[0.36]; t195.8 = –24.1; P < .001; Cohen d = 2.83], in which class 2
on these measures consistently showed higher mean scores and
variability compared with class 1 (Figure 1; eTable 2 in the Supple-
ment). We also identified a 2-class model for impulsive aggres-
sion, in which class 2 showed higher mean scores compared with
class 1 (0.60 [0.62] vs –0.79 [0.41]; t607 = –33.8; P < .001; Cohen
d = 2.60), although the variability patterns between the 2 classes
were similar. For depression symptoms, we identified a 3-class
model, in which class 3 showed the highest mean scores and vari-
ability (2.08 [0.75] vs 0.59 [0.42] vs –0.44 [0.28] (class 1); F2,644
= 1216.7; P = <.001; Cohen d class 3 vs 2 = 2.97; Cohen d class 3
(Reprinted) JAMA Psychiatry June 2019 Volume 76, Number 6 605
 Research Original Investigation Severity and Variability of Depression Symptoms for Predicting Suicide Attempt

Table 1. Demographic and Clinical Characteristics and Trajectories of Symptoms for Offspring

No. (%)
Variable Total Sample Without Suicide Attempt With Suicide Attempt Test df P Value Cohen d
Totala 663 592 71 NA NA NA NA
Demographics and clinical characteristics
Age, mean (SD), y 23.8 (8.5) 24.5 (8.7) 18.5 (5.8) t: 7.63 111.43 <.001 0.70
Female sex 316 (47.7) 280 (47.3) 36 (50.7) χ 21: 0.30 1 .59 0.07
White race/ethnicity 455/647 (70.3) 411/576 (71.4) 44 (62.0) χ 21: 2.67 1 .10 −0.21
Pittsburgh site 443 (66.8) 391 (66.0) 52 (73.2) χ 21: 1.48 1 .22 0.15
Class 2 income 435/660 (65.9) 398/590 (67.5) 37/70 (52.9) χ 21: 5.94 1 .02 −0.31
Income, mean (SD)b 5.3 (3.0) 5.5 (3.0) 4.3 (2.8) t: 2.99 613 .003 0.39
Lifetime history of psychiatric disorders
Mood disordersc 360 (54.3) 301 (50.8) 59 (83.1) χ 21: 26.58 1 <.001 0.66
No mood disorder 303 (47.5) 291 (49.2) 12 (16.9)
Unipolar disorder 301 (45.4) 253 (42.7) 48 (67.6) χ 21: 26.95 2 <.001 0.63
Bipolar disorder 59 (8.9) 48 (8.1) 11 (15.5)
No. of depression episodes, mean (SD) 1.1 (2.2) 1.1 (2.2) 1.6 (1.9) t: −2.05 661 .04 0.26
Duration of depression episodes, 19.4 (27.2) 17.5 (26.0) 28.2 (31.2) t: –2.28 65.42 .03 0.40
mean (SD), mo
Anxiety 284 (42.8) 248 (41.9) 36 (50.7) χ 21: 2.01 1 .16 0.18
Disorders
Psychotic 7 (1.1) 5 (0.8) 2 (2.8) FET NA .17 0.19
Alcohol and substance use 220 (33.2) 196 (33.1) 24 (33.8) χ 21: 0.01 1 .91 0.01
Posttraumatic stress 87 (13.1) 74 (12.5) 13 (18.3) χ 21: 1.88 1 .17 0.17
Attention-deficit/hyperactivity 115/630 (18.3) 98/569 (17.2) 17/61 (27.9) χ 21: 4.18 1 .04 0.28
Cluster B personalityd 55/502 (11.0) 45/463 (9.7) 10/39 (25.6) FET NA .01 0.52
Proband
Bipolar 217/651 (33.3) 185/581 (31.8) 32/70 (45.7) χ 21: 5.41 1 .02 0.30
Attempt 391 (59.0) 339 (57.3) 52 (73.2) χ 21: 6.69 1 .01 0.33
History
Suicide attempt 58 (8.8) 38 (6.4) 20 (28.2) χ 21: 37.57 1 <.001 0.79
Childhood abuse 181/556 (32.6) 142/487 (29.2) 39/69 (56.5) χ 21: 20.61 1 <.001 0.60
Psychiatric treatment at time point prior
to censoring
Inpatient 18/633 (2.8) 11/563 (2.0) 7/70 (10.0) FET NA .002 0.49
Outpatient 53/144 (36.8) 33/111 (29.7) 20/33 (60.6) χ 21: 10.43 1 .001 0.66
Psychotherapy 532/607 (87.6) 482/550 (87.6) 50/57 (87.7) χ 21: 0.0003 1 .99 0.003
Antidepressants/mood stabilizers 97 (14.6) 80 (13.5) 17 (23.9) χ 21: 5.52 1 .02 0.30
Other psychotropic medications 69 (10.4) 62 (10.5) 7 (9.9) χ 21: 0.03 1 .87 −0.02
Nonpsychotropic medications 235 (35.4) 213 (36.0) 22 (31.0) χ 21: 0.69 1 .41 −0.10
Trajectories of symptoms
Depression symptoms class
2 174/647 (26.9) 151/577 (26.2) 23/70 (32.9) FET NA <.001 0.72
3 43/647 (6.6) 27/577 (4.7) 16/70 (22.9)
Class 2
Hopelessness 154 (25.4) 140 (24.1) 25 (35.7) χ 21: 4.42 1 .04 0.27
Impulsivity 151 (23.5) 125 (21.7) 26 (38.8) χ 21: 9.77 1 .002 0.41
Aggression 112/659 (17.0) 92/590 (15.6) 20/69 (29.0) χ 21: 7.85 1 .01 0.36
Impulsive aggression 349/637 (54.8) 297/569 (52.2) 52/68 (76.5) χ 21: 14.45 1 <.001 0.49
Irritability 159/649 (24.5) 131/578 (22.7) 28 (39.4) χ 21: 9.62 1 .002 0.39
Abbreviations: FET, Fisher exact test; NA, not applicable. 4 = $20 000 to $29 999; 5 = $30 000 to $39 999; 6 = $40 000 to $49 999;
a
The denominator used to calculate the percentages may be different for 7 = $50 000 to $59 999; 8 = $60 000 to $69 999; and 9 = ⱖ$70 000.
c
different items because of missing data. Denominators that vary from column Combining unipolar and bipolar disorders.
heading are included with number. d
Only offspring aged ⱖ18 years included (n = 507).
b
1 = <US $10 000; 2 = $10 000 to $14 999; 3 = $15 000 to $19 999;

606 JAMA Psychiatry June 2019 Volume 76, Number 6 (Reprinted) jamapsychiatry.com

© 2019 American Medical Association. All rights reserved.

Downloaded From: https://jamanetwork.com/ on 06/04/2022

 Severity and Variability of Depression Symptoms for Predicting Suicide Attempt Original Investigation Research

Figure 1. Longitudinal Trajectories for 6 Measures of Suicide Attempt

A Depression symptoms trajectory B Hopelessness trajectory

6 1.5

Class 1
Class 2
Class 3
1.0
4

Hopelessness Mean Score

Depression Mean Score

0.5

0
–0.5

–2 –1.0
0 1 2 3 4 5 6 7 8 9 10 11 12 0 1 2 3 4 5 6 7 8 9 10 11 12
Follow-up Time, y Follow-up Time, y

C Impulsivity trajectory D Aggression trajectory

2 2.0

1.5
Impulsivity Mean Score

Aggression Mean Score

1
1.0

0
0

–0.5

–1 –1.0
0 1 2 3 4 5 6 7 8 0 1 2 3 4 5 6 7 8 9 10 11 12
Follow-up Time, y Follow-up Time, y

E Impulsive aggression trajectory F Irritability trajectory

1.0 2.5
A, Depression symptoms class 1
(n = 430), class 2 (n = 174), and class
2.0 3 (n = 43). B, Hopelessness class 1
Impulsive Aggression Mean Score

0.5 (n = 485) and class 2 (n = 165).

C, Impulsivity class 1 (n = 492) and
Irritability Mean Score

class 2 (n = 151). D, Aggression class 1

0 (n = 547) and class 2 (n = 112).
1.0 E, Impulsive aggression class 1
(n = 288) and class 2 (n = 349).
F, Irritability class 1 (n = 490) and
–0.5
class 2 (n = 159). Only 34 offspring
0
had a follow-up duration beyond 12
years. Data from these follow-ups
–1.0 were truncated and removed from
the analyses. During the course of
this longitudinal study, the
–1.5 –1.0 impulsivity and impulsive aggression
0 1 2 3 4 5 6 7 8 0 1 2 3 4 5 6 7 8 9 10 11 12 questionnaires were eliminated at
Follow-up Time, y Follow-up Time, y year 8 to reduce the assessment
battery.

jamapsychiatry.com (Reprinted) JAMA Psychiatry June 2019 Volume 76, Number 6 607

© 2019 American Medical Association. All rights reserved.

Downloaded From: https://jamanetwork.com/ on 06/04/2022

 Research Original Investigation
vs 1 = 7.31; Cohen d class 2 vs 1 = 3.17). eTable 3 in the Supplement
shows the concordance between trajectories across the differ-
ent measures. Depression and hopelessness showed the high-
est agreement (κ = 0.42; 74.6% agreement). eTables 4 to 9 in the
Supplement show the demographic and clinical characteristics
of the trajectories.
Characteristics of Offspring With Suicide Attempt
Table 1 compares the offspring with and without suicide at-
tempt. In addition to some clinical characteristics, we also
found those with attempt to be more likely than those with-
out attempt to belong to class 3 depression symptoms with
higher mean and variability (16 [22.9%] vs 27 [4.7%]; P < .001;
Cohen d = 0.72) (Table 1). They were also more likely to be-
long to class 2 for impulsivity (26 [38.8%] vs 125 [21.7%];
χ 21 = 9.77; P = .002; Cohen d = 0.41), aggression (20 [29.0%]
vs 92 [15.6%]; χ 21 = 7.85; P = .01; Cohen d = 0.36), impulsive
aggression (52 [76.5%] vs 297 [52.2%]; χ 21 = 14.45; P < .001;
Cohen d = 0.49), and irritability (28 [39.4%] vs 131 [22.7%];
χ 21 = 9.62; P = .002; Cohen d = 0.39).
Individual Trajectories as Predictors of Suicide Attempt
Class 3 depression symptoms with higher mean and variabil-
ity was the only statistically significant trajectory that pre-
dicted increased risk for suicide attempt in offspring (model 1
odds ratio [OR], 6.53 [95% CI, 2.53-16.87; t = 3.88; P < .001];
model 2 OR, 3.39 [95% CI, 1.32-8.66; t = 2.55; P = .01) (eTable 10,
models 1-2 in the Supplement). Similar results were obtained
when looking at the narrow definition of actual attempt
(eTable 11, model 1 in the Supplement), incident attempt
(eTable 12, model 1 in the Supplement), and time to onset of
events (eTables 13-15, model 1 in the Supplement). However,
none of the trajectories predicted time to onset of any of the
events when controlling for psychiatric disorders and treat-
ments as time-varying covariates (eTables 13-15, model 2 in the
Supplement).
Combined Trajectories as Predictors of Suicidal Behavior
When we included the trajectories for all measures and con-
trolled for demographics and proband history of actual at-
tempt (Table 2, model 1; eTable 16, model 1 in the Supple-
ment), we found that class 3 depression symptoms was
associated with an almost 8-fold increased risk for suicide at-
tempt (OR, 7.69; 95% CI, 2.37-24.90; t = 3.40; P = .001) and was
the only statistically significant trajectory to predict attempt.
Class 3 depression symptoms remained the only statistically
significant trajectory to predict suicide attempt when control-
ling for additional clinical characteristics (OR, 4.72; 95% CI, 1.47-
15.21; t = 2.60; P = .01) (Table 2, model 2; eTable 16, model 2 in
the Supplement). When including cluster B personality disorders
in models, including offspring aged 18 years or older, we obtained
similar results (eTable 17 in the Supplement). Similar results were
obtained when using the narrow definition (Table 3; eTable 18
in the Supplement), incident attempt (eTable 19 in the Supple-
ment), time to onset of suicide attempt (Figure 2; eTable 20 in
the Supplement), and actual and incident attempts (eTables 21-
22 in the Supplement). However, some of the predictors were no
longer statistically significant, and some models were not stable
608 JAMA Psychiatry June 2019 Volume 76, Number 6 (Reprinted)
© 2019 American Medical Association. All rights reserved.
Downloaded From: https://jamanetwork.com/ on 06/04/2022
Severity and Variability of Depression Symptoms for Predicting Suicide Attempt
given the reduced sample size for the narrow and incident at-
tempt definitions. Using LASSO (eTable 23 in the Supplement),
we found the following predictors in addition to class 3 depres-
sion symptoms: younger age, a lifetime history of unipolar and
bipolar disorders, history of childhood abuse, and proband ac-
tual attempt.
We computed a risk score using the number of statistically
significant predictors endorsed from our models: younger age,
in which risk was defined as 30 years or younger given that 98%
of attempts occurred among offspring aged 30 years or younger;
class 3 depression symptoms; having a mood disorder; history
of suicide attempt; history of childhood abuse; and proband ac-
tual attempt. A risk score of 3 or higher resulted in the highest
sensitivity (87.3%) and moderate specificity (63%; area under the
curve = 0.80) for suicide attempt (eTable 24 in the Supplement).
The positive predictive value was low but increased as the preva-
lence of attempt increased. Similar results were obtained for ac-
tual and incident attempts (eTable 24 in the Supplement).
Discussion
The trajectory of depression symptoms with the most severe
depression symptoms and variability over time was the only
trajectory to predict suicide attempt in young adults. This find-
ing persisted even after controlling for psychiatric disorders
and history of attempt. In addition, younger age, mood disor-
der, childhood abuse, history of suicide attempt, and pro-
band actual attempt were statistically significant predictors.
The combination of these predictors showed high sensitivity
and moderate specificity.
These results highlight that the severity and variability of
depression symptoms may be the only indicator of attempt
above and beyond clinical characteristics. Fewer than half of
offspring in class 3 depression symptoms had a diagnosis of
bipolar disorder, 26 offspring (60.5%) had a diagnosis of ma-
jor depressive disorder, and only 14 (6.9%) had no diagnosis
of a mood disorder. Among those with a major depressive dis-
order diagnosis, 53.8% were younger than 25 years at the time
of the last censored observation and thus were still within the
age of risk for onset of bipolar disorder, had not yet fully ex-
pressed the diagnosis, or were misdiagnosed as having major
depressive disorder. The mean age at onset of bipolar disor-
der spectrum was 14 years, with a prodrome preceding the on-
set ranging between 2 and 10 years.53-55 These results are con-
sistent with findings in studies showing the substantially higher
risk of attempt in individuals with bipolar disorder.56,57
A large epidemiologic community sample of adults and a
meta-analysis of 27 studies with heterogeneous samples have
shown depression to be a predictor associated with suicidal ide-
ation but not attempt among those with suicidal ideation.7,58
However, our study is based on a high-risk sample of young adult
offspring of parents with mood disorders. When taking into ac-
count mood disorders and other psychiatric disorders as time-
varying covariates, we found that class 3 depression symptoms
continued to be associated with a 2 to 7 times increased hazard
of attempt. Our results underscore the severity and variability
in depression symptoms as a marker of risk for attempt.
jamapsychiatry.com
 Severity and Variability of Depression Symptoms for Predicting Suicide Attempt Original Investigation Research

Table 2. Prediction of Suicide Attempt (Broad Definition) Combining Symptom Trajectoriesa

Variable OR (95% CI) t Test P Value

Model 1
Depression symptoms classb
2 1.51 (0.67-3.39) 1.00 .32
3 7.69 (2.37-24.90) 3.40 .001
c
Class 2
Hopelessness 0.61 (0.25-1.51) −1.07 .29
Impulsivity 0.89 (0.42-1.91) −0.29 .77
Aggression 1.75 (0.80-3.84) 1.39 .16
Impulsive aggression 1.38 (0.65-2.94) 0.84 .40
Irritability 1.05 (0.51-2.17) 0.14 .89
Age 0.87 (0.82-0.92) −5.00 <.001
Sex, female vs male 1.04 (0.54-2.00) 0.12 .90
Race/ethnicity, white vs non-white 1.26 (0.67-2.39) 0.72 .47
Site, Pittsburgh vs New York 0.97 (0.47-2.01) −0.09 .93
Income, class 2c 0.53 (0.27-1.06) −1.80 .07
History of suicide attempt 3.26 (1.50-7.11) 2.98 .003
Proband suicide attempt 1.88 (0.99-3.56) 1.94 .05
Model 2
Depression symptoms classb
2 0.78 (0.33-1.86) −0.56 .57
3 4.72 (1.47-15.21) 2.60 .01
c
Class 2
Hopelessness 0.77 (0.31-1.95) −0.54 .59
Impulsivity 0.85 (0.35-2.04) −0.36 .71
Aggression 1.23 (0.55-2.78) 0.50 .62
Impulsive aggression 1.24 (0.52-2.94) 0.48 .63
Irritability 0.81 (0.37-1.78) −0.51 .61
Age 0.82 (0.74-0.90) −4.11 <.001
Sex, female vs male 0.74 (0.37-1.48) −0.85 .39
Race/ethnicity, white vs non-white 1.39 (0.68-2.84) 0.89 .37
Site, Pittsburgh vs New York 1.19 (0.52-2.72) 0.41 .68
Income, class 2c 0.48 (0.24-0.95) −2.11 .04
Disorder
Unipolar 4.71 (1.63-13.58) 2.87 .004
Bipolar 3.40 (0.96-12.04) 1.90 .06
Anxiety 0.93 (0.38-2.25) −0.17 .87
Alcohol and substance use 1.73 (0.76-3.92) 1.31 .19
Posttraumatic stress disorder 1.20 (0.38-3.73) 0.31 .76 Abbreviation: OR, odds ratio.
Attention-deficit/hyperactivity disorder 0.56 (0.21-1.49) −1.17 .24 a
The broad definition of suicide
History attempt included an actual attempt
or a suicide-related behavior
Suicide attempt 2.26 (0.98-5.23) 1.91 .06
(interrupted, aborted, or ambiguous
Childhood abuse 2.98 (1.40-6.38) 2.82 .01 or suicidal ideation that prompted
Proband emergency referral during the
study).
Actual attempt 2.24 (1.06-4.75) 2.11 .04 b
Dummy-coded variable with class 1
Bipolar disorder 1.20 (0.63-2.30) 0.55 .59 as the reference category.
Inpatient 1.83 (0.47-7.20) 0.87 .38 c
Class 2 compared with class 1.
Outpatient 1.60 (0.40-6.40) 0.68 .50 Model 1 pseudo R2 = 0.225; %
correctly classified: 91.4% (full
Psychotherapy 0.66 (0.19-2.28) −0.66 .51 model) and 90.4% (test model).
Antidepressant/mood stabilizer 1.91 (0.65-5.66) 1.18 .24 Model 2 pseudo R2 = 0.326; %
correctly classified: 92.4% (full
Other psychotropic medication 0.32 (0.08-1.22) −1.67 .09
model) and 90% (test model).

jamapsychiatry.com (Reprinted) JAMA Psychiatry June 2019 Volume 76, Number 6 609

© 2019 American Medical Association. All rights reserved.

Downloaded From: https://jamanetwork.com/ on 06/04/2022

 Research Original Investigation Severity and Variability of Depression Symptoms for Predicting Suicide Attempt

Table 3. Prediction of Actual Attempt (Narrow Definition) Combining Symptom Trajectoriesa

Variable OR (95% CI) t Test P Value

Model 1
Depression symptoms classb
2 0.85 (0.26-2.72) −0.28 .78
3 8.22 (1.71-39.57) 2.63 .01
c
Class 2
Hopelessness 0.48 (0.11-2.02) −1.00 .32
Impulsivity 0.86 (0.26-2.78) −0.26 .80
Aggression 1.19 (0.35-4.05) 0.28 .78
Impulsive aggression 2.38 (0.82-6.91) 1.60 .11
Irritability 0.62 (0.17-2.28) −0.71 .48
Age 0.87 (0.79-0.95) −3.01 .003
Sex, female vs male 1.44 (0.54-3.86) 0.73 .47
Race/ethnicity, white vs non-white 1.35 (0.55-3.36) 0.65 .51
Site, Pittsburgh vs New York 0.52 (0.18-1.50) −1.21 .23
Income, class 2c 0.75 (0.26-2.21) −0.52 .60
History of suicide attempt 5.73 (2.07-15.90) 3.36 .001
Proband actual attempt 2.89 (1.05-7.90) 2.06 .04
Model 2
Depression symptoms classb
2 0.45 (0.13-1.56) −1.26 .21
3 6.74 (1.33-34.05) 2.31 .02
c
Class 2
Hopelessness 0.54 (0.14-2.09) −0.89 .37
Impulsivity 0.72 (0.21-2.52) −0.51 .61
Aggression 0.76 (0.25-2.31) −0.48 .63
Impulsive aggression 2.68 (0.72-10.04) 1.47 .14
Irritability 0.37 (0.11-1.32) −1.53 .13
Age 0.80 (0.66-0.97) −2.24 .03
Sex, female vs male 1.48 (0.51-4.35) 0.72 .47
Race/ethnicity, white vs non-white 2.23 (0.85-5.84) 1.64 .10
Site, Pittsburgh vs New York 0.54 (0.15-1.92) −0.95 .34
Income, class 2c 0.74 (0.28-2.00) −0.58 .56
Disorder
Unipolar 3.40 (0.67-17.42) 1.47 .14
Bipolar 1.01 (0.14-7.36) 0.01 .99
Anxiety 0.43 (0.11-1.78) −1.16 .25
Alcohol and substance use 2.88 (0.79-10.49) 1.60 .11
Posttraumatic stress disorder 3.15 (0.56-17.79) 1.30 .19
Attention-deficit/hyperactivity disorder 1.35 (0.29-6.25) 0.39 .70
History
Suicide attempt 3.01 (0.86-10.49) 1.73 .08
Childhood abuse 6.99 (1.71-28.51) 2.72 .01 Abbreviation: OR, odds ratio.
a
Proband The narrow definition included
actual attempts only.
Actual attempt 4.70 (1.23-17.92) 2.26 .02 b
Dummy-coded variable with class 1
Bipolar disorder 0.93 (0.31-2.78) −0.13 .90 as the reference category.
Inpatient 3.16 (0.41-24.65) 1.10 .27 c
Class 2 compared with class 1.
Outpatient 0.85 (0.10-7.33) −0.15 .88 Model 1 pseudo R2 = 0.274; %
correctly classified: 96.2% (full
Psychotherapy 0.64 (0.12-3.56) −0.51 .61 model) and 95.4% (test model).
Antidepressant/mood stabilizer 1.92 (0.30-12.38) 0.69 .49 Model 2 pseudo R2 = 0.392; %
correctly classified: 96.8% (full
Other psychotropic medication 0.49 (0.09-2.77) −0.81 .42
model) and 94.5% (test model).

610 JAMA Psychiatry June 2019 Volume 76, Number 6 (Reprinted) jamapsychiatry.com

© 2019 American Medical Association. All rights reserved.

Downloaded From: https://jamanetwork.com/ on 06/04/2022

 Severity and Variability of Depression Symptoms for Predicting Suicide Attempt Original Investigation Research

Impulsivity and aggression have been reported to predict

Figure 2. Time to Onset of Suicide Attempt by Trajectories
attempt and conceptualized as traits rather than state- for Depression Symptoms
dependent constructs.59 However, we found that the trajec-
tories of impulsivity, aggression, and impulsive aggression did 1.00
not predict attempt when controlling for psychiatric disor-
ders, and class 2 on each trajectory of impulsivity and aggres-
0.75
sion showed variability over time and thus are not stable traits. Class 3

These results are also consistent with findings in a previous
study showing that impulsivity is relatively state dependent
and decreases in response to treatment of severe depression.60
We identified 6 key factors (younger age, class 3 depres-
sion symptoms, mood disorder, history of suicide attempt, his-
tory of childhood abuse, and proband actual attempt) that show
high sensitivity (87.3%) and moderate specificity (63%) in pre-
dicting suicide attempt. These results are consistent with
screening tests that are widely used in clinical practice such
as mammography, tomosynthesis, and blood-based breast can-
cer screening and Papanicolaou test for cervical cancer (sen-
sitivity = 0.72-0.985; specificity = 0.4-0.67).61
The risk score we generated showed improved perfor-
mance compared with other prediction models for suicide at-
tempt and suicide risk, most of which had low sensitivity and
high specificity.27-31 Weighing the cost of a false-positive against
a false-negative result for an outcome like suicidal behavior
in a high-risk population and the cost of screening using vari-
ables that are already collected as part of the medical history
assessment in clinical settings, our risk score is a valuable ad-
dition to tools for estimating the likelihood of a suicide at-
tempt in high-risk individuals. We acknowledge the need to
test its performance in an independent sample, and future stud-
ies need to incorporate clinical, behavioral, and biological vari-
ables to achieve higher performance.62
Strengths and Limitations
This study has several strengths. It is one of the few longitu-
dinal studies to examine the predictors of suicide attempt in
a well-characterized cohort with more than 12 years of follow-
up. This cohort comprised a large sample of offspring at high
risk for suicidal behavior. In addition, the study is one of the
few studies of its kind to focus on a relatively short window
of time from assessment to attempt (median time, 30-45
weeks). Simulation studies for latent class analysis have shown
that a sample size of 600 or more is useful when the effect size
(Cohen d) between classes is fewer than 1.63,64 The sample size
of 663 and the differences between classes were quite large
(Cohen d = 2.6-7.31; Figure 1).
This study also has limitations. Participating in the study
and applying our safety protocols may have reduced suicidal
events among the offspring in the sample. However, we in-
cluded suicide-related behaviors and conducted emergency
1 –Survival
0.50
Class 2
0.25
Class 1
0
0 5 10 15 16
Months Since Baseline
referrals during the study (30 of 663 [4.5%] participants had
referrals). With more than 150 participants per class, we were
able to identify the measures on which the classes differed with
large effect sizes. One case with a small class sample size was
class 3 depression symptoms (n = 43). However, suicide at-
tempt was a rare event, and as such it was not surprising that
the class that predicted attempt included a small subset of the
sample. We were not able to examine the trajectories of sui-
cidal ideation because our measure was focused on intent and
plan among those with suicidal ideation. As such, we did not
capture the full spectrum of ideation. The study did not in-
clude a measure of mood lability to assess rapid mood vari-
ability over shorter periods. Offspring lost to follow-up were
younger and showed lower rates of anxiety disorders com-
pared with those retained for longer periods. However, we con-
trolled for these variables in our models. The measure of irri-
tability was limited to 1 item on depression scales. Finally, we
used the same data set to estimate risk and to derive the risk
score. Although machine learning methods are often used for
large data sets, they are known to perform well for smaller
sample sizes.65,66 Future studies are needed to test our de-
rived risk score in independent samples.
Conclusions
We recommend that clinicians, in their assessment of depres-
sion, (1) pay particular attention to the severity of both cur-
rent and past depression and the variability in these symp-
toms, and (2) monitor and treat depression symptoms over time
to reduce symptom severity and fluctuation, and thus the like-
lihood for suicide attempt, in high-risk young adults.

ARTICLE INFORMATION
Accepted for Publication: November 25, 2018.
Published Online: February 27, 2019.
doi:10.1001/jamapsychiatry.2018.4513
Author Contributions: Drs Melhem and Brent had
full access to all of the data in the study and take
responsibility for the integrity of the data and the
accuracy of the data analysis.
Concept and design: Melhem, Keilp, Burke,
Birmaher, Stanley, Mann, Brent.
Acquisition, analysis, or interpretation of data:
Melhem, Porta, Oquendo, Zelazny, Keilp, Iyengar,
Burke, Birmaher, Stanley, Mann, Brent.
Drafting of the manuscript: Melhem.
Critical revision of the manuscript for important
intellectual content: All authors.
Statistical analysis: Melhem, Porta, Iyengar, Brent.
Obtained funding: Melhem, Keilp, Mann, Brent,
Oquendo.
Administrative, technical, or material support:
Melhem, Oquendo, Keilp, Mann, Brent.
Supervision: Melhem, Oquendo, Keilp, Mann, Brent.

jamapsychiatry.com (Reprinted) JAMA Psychiatry June 2019 Volume 76, Number 6 611

© 2019 American Medical Association. All rights reserved.

Downloaded From: https://jamanetwork.com/ on 06/04/2022

 Research Original Investigation
Conflict of Interest Disclosures: .Dr Melhem
reported receiving research support from the
National Institute of Mental Health (NIMH), Brain
and Behavior Research Foundation, and the
American Foundation for Suicide Prevention (AFSP)
. Dr Oquendo reported receiving royalties from
Research Foundation for Mental Hygiene for the
commercial use of the Columbia-Suicide Severity
Rating Scale, and owning stock in Bristol-Myers
Squibb. Drs Burke, Stanley, and Mann reported
receiving royalties from Research Foundation for
Mental Hygiene for the commercial use of the
Columbia-Suicide Severity Rating Scale. Dr Stanley
also reported receiving research support from
NIMH and AFSP. Dr Keilp reported receiving
support from AFSP and owning stocks in Pfizer and
Zoetis. Dr Birmaher reported receiving research
support from the National Institute of Mental
Health (NIMH) and royalties for publications from
Random House Inc, Lippincott Williams & Wilkins,
and UpToDate. Dr Iyengar reported receiving
research support from NIMH. Dr Brent receives
research support from NIMH, AFSP, and the Once
Upon a Time Foundation, honoraria from the
Klingenstein Third Generation Foundation royalties
for scientific board membership and grant review,
from Guilford Press, royalties from the electronic
self-rated version of the C-SSRS from ERT, Inc.,
royalties from performing duties as an UpToDate
Psychiatry Section Editor, payment for serving as an
Associate Editor for Psychological Medicine, and
consulting fees from Healthwise. No other
disclosures were reported.
Funding/Support: This study was supported by
R01 grants MH056612 (Dr Brent) and MH056390
(Dr Mann) and K01 grant MH077930 (Dr Melhem)
from the NIMH.
Role of the Funder/Sponsor: The funding sources
had no role in the design and conduct of the study;
collection, management, analysis, and
interpretation of the data; preparation, review,
or approval of the manuscript; and decision to
submit the manuscript for publication.
Additional Contributions: We would like to
acknowledge the families for their participation
in this study.
REFERENCES
1. Murphy SL, Xu J, Kochanek KD. Deaths: final data
for 2010. Natl Vital Stat Rep. 2013;61(4):1-117.
2. Coryell W, Young EA. Clinical predictors of
suicide in primary major depressive disorder.
J Clin Psychiatry. 2005;66(4):412-417. doi:10.4088/
JCP.v66n0401
3. Angst J, Angst F, Gerber-Werder R, Gamma A.
Suicide in 406 mood-disorder patients with and
without long-term medication: a 40 to 44 years’
follow-up. Arch Suicide Res. 2005;9(3):279-300.
doi:10.1080/13811110590929488
4. Yoshimasu K, Kiyohara C, Miyashita K; Stress
Research Group of the Japanese Society for
Hygiene. Suicidal risk factors and completed
suicide: meta-analyses based on psychological
autopsy studies. Environ Health Prev Med. 2008;13
(5):243-256. doi:10.1007/s12199-008-0037-x
5. Bernal M, Haro JM, Bernert S, et al;
ESEMED/MHEDEA Investigators. Risk factors for
suicidality in Europe: results from the ESEMED
study. J Affect Disord. 2007;101(1-3):27-34. doi:10.
1016/j.jad.2006.09.018
612 JAMA Psychiatry June 2019 Volume 76, Number 6 (Reprinted)
© 2019 American Medical Association. All rights reserved.
Downloaded From: https://jamanetwork.com/ on 06/04/2022
Severity and Variability of Depression Symptoms for Predicting Suicide Attempt
6. Nock MK, Borges G, Bromet EJ, et al.
Cross-national prevalence and risk factors for
suicidal ideation, plans and attempts. Br J Psychiatry.
2008;192(2):98-105. doi:10.1192/bjp.bp.107.040113
7. Nock MK, Hwang I, Sampson NA, Kessler RC.
Mental disorders, comorbidity and suicidal
behavior: results from the National Comorbidity
Survey Replication. Mol Psychiatry. 2010;15(8):
868-876. doi:10.1038/mp.2009.29
8. Melhem NM, Brent DA, Ziegler M, et al. Familial
pathways to early-onset suicidal behavior: familial
and individual antecedents of suicidal behavior. Am
J Psychiatry. 2007;164(9):1364-1370. doi:10.1176/
appi.ajp.2007.06091522
9. Schaffer A, Isometsä ET, Tondo L, et al.
International Society for Bipolar Disorders Task
Force on Suicide: meta-analyses and
meta-regression of correlates of suicide attempts
and suicide deaths in bipolar disorder. Bipolar Disord.
2015;17(1):1-16.doi:10.1111/bdi.12271
10. Brent DA, Melhem NM, Oquendo M, et al.
Familial pathways to early-onset suicide attempt:
a 5.6-year prospective study. JAMA Psychiatry.
2015;72(2):160-168. doi:10.1001/jamapsychiatry.
2014.2141
11. Nock MK, Ursano RJ, Heeringa SG, et al; Army
STARRS Collaborators. Mental disorders,
comorbidity, and pre-enlistment suicidal behavior
among new soldiers in the US Army: results from
the Army Study to Assess Risk and Resilience in
Servicemembers (Army STARRS). Suicide Life
Threat Behav. 2015;45(5):588-599. doi:10.1111/sltb.
12153
12. Goldstein TR, Bridge JA, Brent DA. Sleep
disturbance preceding completed suicide in
adolescents. J Consult Clin Psychol. 2008;76(1):
84-91. doi:10.1037/0022-006X.76.1.84
13. Bernert RA, Kim JS, Iwata NG, Perlis ML. Sleep
disturbances as an evidence-based suicide risk
factor. Curr Psychiatry Rep. 2015;17(3):554. doi:10.
1007/s11920-015-0554-4
14. Fawcett J, Busch KA, Jacobs D, Kravitz HM,
Fogg L. Suicide: a four-pathway clinical-biochemical
model. Ann N Y Acad Sci. 1997;836:288-301.
doi:10.1111/j.1749-6632.1997.tb52366.x
15. Mann JJ. Neurobiology of suicidal behaviour.
Nat Rev Neurosci. 2003;4(10):819-828. doi:10.
1038/nrn1220
16. Sareen J, Cox BJ, Afifi TO, et al. Anxiety
disorders and risk for suicidal ideation and suicide
attempts: a population-based longitudinal study of
adults. Arch Gen Psychiatry. 2005;62(11):1249-1257.
doi:10.1001/archpsyc.62.11.1249
17. ten Have M, de Graaf R, van Dorsselaer S, et al.
Incidence and course of suicidal ideation and
suicide attempts in the general population. Can J
Psychiatry. 2009;54(12):824-833. doi:10.1177/
070674370905401205
18. Beck AT, Steer RA, Kovacs M, Garrison B.
Hopelessness and eventual suicide: a 10-year
prospective study of patients hospitalized with
suicidal ideation. Am J Psychiatry. 1985;142(5):
559-563. doi:10.1176/ajp.142.5.559
19. Afifi TO, Boman J, Fleisher W, Sareen J.
The relationship between child abuse, parental
divorce, and lifetime mental disorders and
suicidality in a nationally representative adult
sample. Child Abuse Negl. 2009;33(3):139-147.
doi:10.1016/j.chiabu.2008.12.009
20. Haw C, Hawton K. Life problems and deliberate
self-harm: associations with gender, age, suicidal
intent and psychiatric and personality disorder.
J Affect Disord. 2008;109(1-2):139-148. doi:10.1016/
j.jad.2007.12.224
21. Troister T, Links PS, Cutcliffe J. Review of
predictors of suicide within 1 year of discharge from
a psychiatric hospital. Curr Psychiatry Rep. 2008;
10(1):60-65. doi:10.1007/s11920-008-0011-8
22. Stein DJ, Chiu WT, Hwang I, et al. Cross-national
analysis of the associations between traumatic
events and suicidal behavior: findings from the
WHO World Mental Health Surveys. PLoS One.
2010;5(5):e10574. doi:10.1371/journal.pone.0010574
23. Bruffaerts R, Demyttenaere K, Borges G, et al.
Childhood adversities as risk factors for onset and
persistence of suicidal behaviour. Br J Psychiatry.
2010;197(1):20-27. doi:10.1192/bjp.bp.109.074716
24. Zatti C, Rosa V, Barros A, et al. Childhood
trauma and suicide attempt: a meta-analysis of
longitudinal studies from the last decade.
Psychiatry Res. 2017;256:353-358. doi:10.1016/j.
psychres.2017.06.082
25. Aaltonen K, Näätänen P, Heikkinen M, et al.
Differences and similarities of risk factors for
suicidal ideation and attempts among patients with
depressive or bipolar disorders. J Affect Disord.
2016;193:318-330. doi:10.1016/j.jad.2015.12.033
26. Franklin JC, Ribeiro JD, Fox KR, et al. Risk
factors for suicidal thoughts and behaviors:
a meta-analysis of 50 years of research. Psychol Bull.
2017;143(2):187-232. doi:10.1037/bul0000084
27. Kessler RC, Warner CH, Ivany C, et al; Army
STARRS Collaborators. Predicting suicides after
psychiatric hospitalization in US Army soldiers: the
Army Study to Assess Risk and Resilience in
Servicemembers (Army STARRS). JAMA Psychiatry.
2015;72(1):49-57. doi:10.1001/jamapsychiatry.2014.
1754
28. Walsh CG, Ribeiro JD, Franklin JC. Predicting
risk of suicide attempts over time through machine
learning. Clin Psychol Sci. 2017;5(3):457-469. doi:10.
1177/2167702617691560
29. Barak-Corren Y, Castro VM, Javitt S, et al.
Predicting suicidal behavior from longitudinal
electronic health records. Am J Psychiatry. 2017;174
(2):154-162. doi:10.1176/appi.ajp.2016.16010077
30. Kessler RC, Stein MB, Petukhova MV, et al;
Army STARRS Collaborators. Predicting suicides
after outpatient mental health visits in the Army
Study to Assess Risk and Resilience in
Servicemembers (Army STARRS). Mol Psychiatry.
2017;22(4):544-551. doi:10.1038/mp.2016.110
31. Tran T, Luo W, Phung D, et al. Risk stratification
using data from electronic medical records better
predicts suicide risks than clinician assessments.
BMC Psychiatry. 2014;14:76-84. doi:10.1186/1471-
244X-14-76
32. Meyer RE, Salzman C, Youngstrom EA, et al.
Suicidality and risk of suicide–definition, drug safety
concerns, and a necessary target for drug
development: a consensus statement.
J Clin Psychiatry. 2010;71(8):e1-e21. doi:10.4088/
JCP.10cs06070blu
33. U.S. Department of Health and Human
Services, Food and Drug Administration, Center for
Drug Evaluation and Research (CDER). Guidance for
Industry. Suicidal ideation and behavior:
prospective assessment of occurrence in clinical
jamapsychiatry.com
 Severity and Variability of Depression Symptoms for Predicting Suicide Attempt
trials. https://www.fda.gov/drugs/
guidancecomplianceregulatoryinformation/
guidances/ucm315156.htm. Published August 2012.
Accessed January 21, 2019.
34. Crosby A, Ortega L, Melanson C. Self-Directed
Violence Surveillance: Uniform Definitions and
Recommended Data Elements. Atlanta, GA: Centers
for Disease Control and Prevention, National Center
for Injury Prevention and Control, Division of Violence
Prevention; 2011. https://www.cdc.gov/
violenceprevention/pdf/Self-Directed-Violence-a.
pdf. Accessed January 21, 2019.
35. Brent DA, Emslie GJ, Clarke GN, et al. Predictors
of spontaneous and systematically assessed
suicidal adverse events in the treatment of
SSRI-resistant depression in adolescents (TORDIA)
study. Am J Psychiatry. 2009;166(4):418-426.
doi:10.1176/appi.ajp.2008.08070976
36. Kaufman J, Birmaher B, Brent D, et al. Schedule
for Affective Disorders and Schizophrenia for
School-Age Children-Present and Lifetime Version
(K-SADS-PL): initial reliability and validity data.
J Am Acad Child Adolesc Psychiatry. 1997;36(7):
980-988. doi:10.1097/00004583-199707000-
00021
37. Spitzer RL, Williams JB, Gibbon M, First MB.
The Structured Clinical Interview for DSM-III-R
(SCID). I: history, rationale, and description. Arch
Gen Psychiatry. 1992;49(8):624-629. doi:10.1001/
archpsyc.1992.01820080032005
38. Mann JJ, McBride PA, Brown RP, et al.
Relationship between central and peripheral
serotonin indexes in depressed and suicidal
psychiatric inpatients. Arch Gen Psychiatry. 1992;49
(6):442-446. doi:10.1001/archpsyc.1992.
01820060022003
39. Beck AT, Ward CH, Mendelson M, Mock J,
Erbaugh J. An inventory for measuring depression.
Arch Gen Psychiatry. 1961;4:561-571. doi:10.1001/
archpsyc.1961.01710120031004
40. Kovacs M. The Children’s Depression,
Inventory (CDI). Psychopharmacol Bull. 1985;21(4):
995-998.
41. Beck AT, Weissman A, Lester D, Trexler L.
The measurement of pessimism: the Hopelessness
Scale. J Consult Clin Psychol. 1974;42(6):861-865.
doi:10.1037/h0037562
42. Kazdin AE, Rodgers A, Colbus D.
The Hopelessness Scale for Children: psychometric
characteristics and concurrent validity. J Consult
jamapsychiatry.com
© 2019 American Medical Association. All rights reserved.
Downloaded From: https://jamanetwork.com/ on 06/04/2022
Clin Psychol. 1986;54(2):241-245. doi:10.1037/
0022-006X.54.2.241
43. Barratt ES. Factor analysis of some
psychometric measures of impulsiveness and
anxiety. Psychol Rep. 1965;16:547-554. doi:10.
2466/pr0.1965.16.2.547
44. Plomin R. Childhood temperament. In: Lahey
BB, Kazdin AE, eds. Advances in Clinical Child
Psychology. New York: Plenum; 1983:45-92. doi:10.
1007/978-1-4613-9814-1_2
45. Buss AH, Durkee A. An inventory for assessing
different kinds of hostility. J Consult Psychol. 1957;
21(4):343-349. doi:10.1037/h0046900
46. Kazdin AE, Rodgers A, Colbus D, Siegel T.
Children’s Hostility Inventory: measurement of
aggression and hostility in psychiatric inpatient
children. J Clin Child Psychol. 1987;16:320-328.
doi:10.1207/s15374424jccp1604_5
47. Brown GL, Goodwin FK. Human aggression and
suicide. Suicide Life Threat Behav. 1986;16(2):
223-243. doi:10.1111/j.1943-278X.1986.tb00353.x
48. Bernstein DP, Fink L, Handelsman L, et al.
Initial reliability and validity of a new retrospective
measure of child abuse and neglect. Am J Psychiatry.
1994;151(8):1132-1136. doi:10.1176/ajp.151.8.1132
49. Chaffin M, Wherry JN, Newlin C, Crutchfield A,
Dykman R. The Abuse Dimensions Inventory: initial
data on a research measure of abuse severity.
J Interpers Violence. 1997;12(4):569-589. doi:10.
1177/088626097012004006
50. Stata Statistical Software [computer program].
Release 11. College Station, TX: StataCorp LLC;
2009.
51. Collins LM, Schafer JL, Kam CM. A comparison
of inclusive and restrictive strategies in modern
missing data procedures. Psychol Methods. 2001;6
(4):330-351. doi:10.1037/1082-989X.6.4.330
52. Friedman J, Hastie T, Tibshirani R.
Regularization paths for generalized linear models
via coordinate descent. J Stat Softw. 2010;33(1):1-22.
doi:10.18637/jss.v033.i01
53. Hafeman DM, Merranko J, Goldstein TR, et al.
Assessment of a person-level risk calculator to
predict new-onset bipolar spectrum disorder in
youth at familial risk. JAMA Psychiatry. 2017;74(8):
841-847. doi:10.1001/jamapsychiatry.2017.1763
54. Van Meter AR, Burke C, Youngstrom EA,
Faedda GL, Correll CU. The bipolar prodrome:
meta-analysis of symptom prevalence prior to initial
or recurrent mood episodes. J Am Acad Child
(Reprinted) JAMA Psychiatry June 2019 Volume 76, Number 6
Original Investigation Research
Adolesc Psychiatry. 2016;55(7):543-555. doi:10.
1016/j.jaac.2016.04.017
55. Conus P, Ward J, Hallam KT, et al. The proximal
prodrome to first episode mania–a new target for
early intervention. Bipolar Disord. 2008;10(5):
555-565. doi:10.1111/j.1399-5618.2008.00610.x
56. Pompili M, Gonda X, Serafini G, et al.
Epidemiology of suicide in bipolar disorders:
a systematic review of the literature. Bipolar Disord.
2013;15(5):457-490. doi:10.1111/bdi.12087
57. Grande I, Berk M, Birmaher B, Vieta E. Bipolar
disorder. Lancet. 2016;387(10027):1561-1572.
doi:10.1016/S0140-6736(15)00241-X
58. May A, Klonsky ED. What distinguishes suicide
attempters from suicide ideators? a meta-analysis
of potential factors. Clin Psychol Sci Pract. 2016;23
(1):5-20. doi:10.1111/cpsp.12136
59. Mann JJ, Arango VA, Avenevoli S, et al.
Candidate endophenotypes for genetic studies of
suicidal behavior. Biol Psychiatry. 2009;65(7):
556-563. doi:10.1016/j.biopsych.2008.11.021
60. Corruble E, Benyamina A, Bayle F, Falissard B,
Hardy P. Understanding impulsivity in severe
depression? a psychometrical contribution. Prog
Neuropsychopharmacol Biol Psychiatry. 2003;27
(5):829-833. doi:10.1016/S0278-5846(03)00115-5
61. Maxim LD, Niebo R, Utell MJ. Screening tests:
a review with examples. Inhal Toxicol. 2014;26(13):
811-828. doi:10.3109/08958378.2014.955932
62. Melhem NM, Munroe S, Marsland A, et al.
Blunted HPA axis activity prior to suicide attempt
and increased inflammation in attempters.
Psychoneuroendocrinology. 2017;77:284-294. doi:10.
1016/j.psyneuen.2017.01.001
63. Dziak JJ, Lanza ST, Tan X. Effect size, statistical
power and sample size requirements for the
bootstrap likelihood ratio test in latent class
analysis. Struct Equ Modeling. 2014;21(4):534-552.
doi:10.1080/10705511.2014.919819
64. Gudicha DW, Tekle FB, Vermunt JK. Power and
sample size computation for Wald tests in latent
class models. J Classif. 2016;33(1):30-51. doi:10.
1007/s00357-016-9199-1
65. Hastie T, Tibshirani R, Friedman J. The Elements
of Statistical Learning: Data Mining, Inference, and
Prediction. 2nd ed. New York: Springer; 2009.
doi:10.1007/978-0-387-84858-7
66. Izenman AJ. Modern Multivariate Statistical
Techniques: Regression, Classification, and Manifold
Learning. New York: Springer; 2013.
613