General Hospital Psychiatry 75 (2022) 68–74

Contents lists available at ScienceDirect

General Hospital Psychiatry

journal homepage: www.elsevier.com/locate/genhospsych

A virtual, pilot randomized trial of a brief intervention to prevent suicide in

an integrated healthcare setting
Natalie B. Riblet a, b, *, Lauren Kenneally a, Susan Stevens a, c, Bradley V. Watts a, c, d, Jiang Gui a, e,
Jenna Forehand a, Sarah Cornelius a, Glenna S. Rousseau a, c, Jonathan C. Schwartz a, c,
Brian Shiner a, b, f
a
Veterans Affairs Medical Center, 215 North Main Street, White River Junction, VT 05009, United States of America
b
Department of Psychiatry and Dartmouth Institute, Geisel School of Medicine, Dartmouth College, 1 Rope Ferry Road, Hanover, NH 03755, United States of America
c
Department of Psychiatry, Geisel School of Medicine, Dartmouth College, 1 Rope Ferry Road, Hanover, NH 03755, United States of America
d
Veterans Rural Health Resource Center, White River Junction VA Medical Center, 215 North Main Street, White River Junction, VT 05009, United States of America
e
Department of Biomedical Data Science, Dartmouth Institute, Community and Family Medicine, Geisel School of Medicine, Dartmouth College, 1 Rope Ferry Road,
Hanover, NH 03755, United States of America
f
National Center for PTSD, Mental Health Service, Veterans Affairs Medical Center, 215 North Main Street, White River Junction, VT 05009, United States of America

A R T I C L E I N F O A B S T R A C T

Keywords: Objective: Patients who die by suicide are often seen in primary care settings in the weeks leading to their death.
Brief educational intervention There has been little study of brief interventions to prevent suicide in these settings.
Suicide prevention Method: We conducted a virtual, pilot, randomized controlled trial of a brief suicide prevention strategy called
Integrated care
Veterans Affairs Brief Intervention and Contact Program (VA BIC) in patients who presented to a primary care
Veteran
mental health walk-in clinic for a new mental health intake appointment and were at risk for suicide. Our pri­
Remote research
mary aim was to assess feasibility. We measured our ability to recruit 20 patients. We measured the proportion of
enrolled patients who completed all study assessments. We assessed adherence among patients assigned to VA
BIC.
Results: Twenty patients were enrolled and 95% (N = 19) completed all study assessments. Among the 10 patients
assigned to VA BIC, 90% (N = 9) of patients completed all required intervention visits, and 100% (N = 10)
completed ≥70% of the required interventions visits.
Conclusion: It is feasible to conduct a virtual trial of VA BIC in an integrated care setting. Future research should
clarify the role of VA BIC as a suicide prevention strategy in integrated care settings using an adequately powered
design.
Clinical Trial Registration: NCT04054947.

1. Introduction
Suicide is the 10th leading cause of death for people of all ages in the
United States (US) [1]. It is well-established that patients are at espe­
cially high risk for suicide following an acute psychiatric hospitalization
[2]. However, there is also evidence to support that many patients who
die by suicide are seen in non-mental health settings such as primary
care in the weeks preceding their death [3,4]. Ahmedani et al. (2014)
found in an analysis of 4988 suicide deaths in patients enrolled in a
Health Maintenance Organization (HMO) plan that almost 50% (2844)
of decedents had a health care contact in the four weeks prior to their
death, and many of these were primary care (21%) or medical specialty
care (25%) contacts [3]. There is a critical need to identify effective
strategies to prevent death by suicide in patients who have limited direct
contact with the mental health care system.
There have been two primary approaches to mitigate suicide risk in
primary care settings. First, healthcare systems have focused on
educating primary care providers about suicide prevention, and
empowering these providers to address suicide risk [5,6]. Second,
healthcare systems have implemented collaborative care or integrated
care models in the hopes that that these models will improve the man­
agement of mental health symptoms and thereby, reduce suicide risk

* Corresponding author at: Veterans Affairs, Building 9, 215 North Main Street, White River Junction, VT 05009, United States of America.
E-mail address: Natalie.Riblet@dartmouth.edu (N.B. Riblet).

https://doi.org/10.1016/j.genhosppsych.2022.02.002
Received 22 December 2021; Received in revised form 14 February 2022; Accepted 15 February 2022
Available online 18 February 2022
0163-8343/Published by Elsevier Inc.
N.B. Riblet et al.
[5,7–11]. In a study of 20 primary care practices enrolled in the Pre­
vention of Suicide in Primary Care Elderly: Collaborative Trial (PROS­
PECT), Bruce et al. (2004) found that depression treatment guidelines in
conjunction with case management resulted in a significantly greater
reduction in suicidal ideation compared with usual care [8]. The in­
vestigators, however, were unable to assess the impact of the interven­
tion on suicide deaths because of the low event rate during the 24-month
follow-up [7]. In general, there is conflicting evidence about whether
educational interventions or collaborative/integrated care models are
effective at preventing suicide [6,9,12–14]. There are substantial bar­
riers to bringing these approaches to scale because of time constraints
[15,16] and limited or no access to mental health resources [17]. These
concerns point to the need to identify feasible interventions that can be
implemented in primary care settings to prevent suicide [5].
Surprisingly, unlike inpatient and emergency room settings [12,14],
brief interventions to prevent suicide in primary care or integrated care
settings have not been widely studied [18–20]. Dueweke (2019) con­
ducted a pre-post pilot study of a crisis planning intervention in 22 pa­
tients who presented with risk for suicide in an integrated care setting
[20]. They found that at four-month follow-up the intervention was
associated with lower suicidal intent, improved hope, and greater
coping efficacy [20]. The World Health Organization Brief Intervention
and Contact (WHO BIC) program is a similarly promising suicide pre­
vention strategy that has not been studied in primary care or integrated
care settings [21]. WHO BIC targets patients being discharged from an
emergency room or psychiatric hospital after a suicide attempt (SA) and
prevents suicide in this group [14,21]. The intervention begins at
discharge with an educational session about suicide facts and suicide
risk, motivation for treatment, and emphasis on self-efficacy and patient
responsibility. It is followed by regularly scheduled telephone or in-
person contacts over 18 months [21]. An adapted version of WHO BIC
(called the Veterans Affairs Brief Intervention and Contact Program, VA
BIC) has been developed for US veterans, and has shown promise in
reducing suicide risk after psychiatric hospitalization [22,23]. VA BIC
could be useful in primary care or integrated care settings because it
targets factors implicated in suicide risk during care transitions,
including poor treatment engagement and social connectedness
[22–26].
In order to gather preliminary information on the role of VA BIC as a
suicide prevention strategy in primary care or integrated care settings,
we conducted a pilot randomized controlled trial (RCT) of the VA BIC in
patients who presented to an integrated care setting for a new mental
health evaluation. Our primary objective was to assess feasibility in the
integrated care setting. Our secondary objective was to collect pre­
liminary estimates of effect sizes.
2. Method
2.1. Sample
We conducted a pilot, single-site, assessor-blinded, RCT comparing
VA BIC plus standard care to standard care alone at the White River
Junction Veterans Affairs Medical Center between August 2020 and May
2021. Patients were recruited from a collocated, collaborative care
clinic, referred to as the Primary Care-Mental Health Integration Clinic
(PC-MHI), located in the primary care clinic. The clinic is staffed by
prescribing clinicians including psychiatrists and advanced practice
nurses as well as psychotherapists including psychologists and social
workers. PC-MHI providers work closely with primary care providers to
manage patients with a full spectrum of mental health conditions. In
most cases, patients are referred to the PC-MHI clinic for a same day,
one-time mental health consult. Patients then return to primary care for
ongoing care.
Prior to the SARS-COV-2 pandemic, PC-MHI providers typically
evaluated new referrals in-person. However, in March 2020, the facility
instituted a new policy whereby PC-MHI providers were expected to
69
General Hospital Psychiatry 75 (2022) 68–74
evaluate new referrals using virtual care modalities including video and
telephone visits in response to the SARS-COV-2 pandemic physical
distancing requirements. The policy remained in effect throughout the
course of our study.
The study was approved by our local Institutional Review Board and
was registered at ClinicalTrials.gov (Trial no. NCT04054947).
2.2. Study design
Similar to other studies of VA BIC [22,23], we intended to use a
combination of in-person and virtual methods to carry out this study.
However, consistent with clinical care, in-person research at our facility
was put on hold due to the SARS-COV-2 pandemic. Therefore, we con­
verted all study procedures including informed consent to a virtual
platform with approval from our local IRB. Patients were compensated
for their participation.
Patients were eligible for study inclusion if they were seen in the PC-
MHI clinic for a new mental health intake appointment, were at least 18
years old, English speaking, eligible to receive VA services, and were
currently at risk for suicidal behavior. We based our determination of
risk on the results of a formal suicide assessment scale (i.e., clinical
version of Columbia Suicide Severity Rating-Scale, C-SSRS) and clinician
follow up questions of C-SSRS results documented in the intake note. We
also allowed entry for patients when the clinician documented infor­
mation in their intake note to support that the patient may be at risk for
suicidal behavior. We relied on clinician interpretation of suicide risk
based on scale responses and other questioning rather than a set score
because of no clear cut off score criteria in the literature [27,28]. We
excluded patients who were unable to provide informed consent or
imprisoned.
The study staff screened new intake appointments in the PC-MHI
daily to identify eligible patients. Patients who met inclusion criteria
were sent a letter describing the study, and then received a follow-up
phone call to confirm study eligibility and to assess interest in partici­
pation. If interested, the study staff obtained informed consent
telephonically.
Using fixed block randomization, patients were randomly assigned to
VA BIC plus standard care or standard care alone. Study personnel who
were not otherwise involved in recruitment or assessment indepen­
dently prepared allocation cards using sealed, opaque, sequentially
numbered envelopes. It was impossible to blind either the interven­
tionist or the patient to treatment allocation. However, the assessor
remained blinded to study assignment throughout the trial, and patients
were instructed not to reveal their assignment to the assessor.
After obtaining consent, a trained assessor met with the patient to
conduct the baseline assessment. The assessor then conducted two
follow-up assessments at one- and three- months after the baseline
assessment. Because the research involves a high-risk population [29],
patients received further evaluation by a study psychiatrist if their re­
sponses met pre-defined criteria on a standardized safety protocol.
2.3. Sample size
The primary aim of our pilot study was to determine the feasibility of
conducting research on VA BIC in the integrated care setting. While an
adaptation of WHO BIC was previously studied in VA populations at risk
for suicide, those populations were exclusively inpatient samples
[22,23]. Moreover, there was no prior literature in the VA or elsewhere
on adapting WHO BIC to an entirely virtual format. There is a lack of
consensus about the optimal approach to determine a sample size for
such a pilot study. Several strategies, however, have been proposed
[30]. We followed guidance from the National Center for Complemen­
tary Integrative Health at the National Institute of Health (NIH), which
recommends against the use of a power analysis to determine a sample
size for a pilot study, and suggests a researcher should select a sample
size that is both reasonable and feasible to achieve the stated objectives
N.B. Riblet et al.
[31]. We set our target recruitment goal at 20 patients (i.e., 10 subjects
per group) because this sample size would provide us with reasonable
insights on the feasibility of studying VA BIC in an integrated care
setting, and meet the budgetary constraints set by our funder.
2.4. VA BIC intervention
Unlike the original WHO BIC, VA BIC is a manualized intervention
that includes conversation guides to facilitate the session [22,23]. VA
BIC was shortened to 3 months. These adaptations were made to help
bring the program to scale in a clinical setting.
VA BIC aims to educate patients about suicide prevention, bolster
self-efficacy, and treatment engagement. A primary focus of the inter­
vention is to encourage and facilitate treatment engagement and social
connection. It incorporates aspects of motivational interviewing. It can
be delivered by a trained mental health provider such as a psychologist,
social worker or mental health nurse [22,23]. The role of the interven­
tionist can be best defined as that of a coach whose goal is to help pa­
tients stay connected to their treatment. The sessions do not include
mental health interventions such medication management or psycho­
therapy, rather it uses a standardized approach to encourage connection
between the patient and the primary treatment team.
Once randomized to VA BIC, the patient was contacted by the
interventionist by phone to briefly introduce them to VA BIC, and to
schedule the brief education visit. A packet of printed educational ma­
terials was mailed out to the patient in anticipation of the visit. The
patients then received a one-hour, one-on-one educational intervention
on suicide prevention either over phone or video. As part of this session,
the interventionist also introduced patients to safety planning. Patients
then received six additional contacts with the interventionist over the
course of three months. These visits occurred over phone or video.
During these sessions, the interventionist checked on the patient’s well-
being, encouraged self-monitoring of symptoms, affirmed progress,
reviewed the safety plan, assessed adherence with treatment, and
continued to build the patient’s sense of self-efficacy and motivation for
treatment engagement.
By design, the BIC program requires relatively little training [21].
Prior to the start of the study, the interventionist (Dr. Susan Stevens), a
clinical psychologist by training, was formally trained in the delivery of
VA BIC through didactics and video demonstrations. She completed
multiple practice cases to ensure competency. During the trial, she met
with an expert in VA BIC (Dr. Natalie Riblet) on a regular basis to review
cases, and to receive ongoing supervision and mentoring.
2.5. Standard care
Regardless of assignment, all patients received the standard mental
health care that is offered to all patients who are seen for a new mental
health evaluation in the PC-MHI clinic. This includes meeting with a
mental health provider and completing a clinical assessment. The pro­
vider works in collaboration with the patient to determine the appro­
priate treatment regimen. Patients are usually referred to primary care
for ongoing treatment.
2.6. Study measures
We collected socio-demographic data from participants using a
standardized intake form. We assessed baseline psychiatric diagnosis
using the MINI International Neuropsychiatric Interview (MINI)
[32,33]. We assessed feasibility in several ways. First, we measured our
ability to recruit 20 patients into the study. Second, we measured the
proportion of enrolled patients who completed the baseline and two
follow-up study assessments. Third, we assessed adherence among pa­
tients who were assigned to VA BIC. We measured both the proportion of
patients who completed all required intervention visits (i.e., brief edu­
cation plus six contact visits), and those who completed 70% or more of
70
General Hospital Psychiatry 75 (2022) 68–74
the required intervention visits (i.e., brief education plus 4 or more
contact visits). Finally, we reported any unanticipated challenges that
we observed in conducting research in patients at risk for suicide.
We also collected preliminary outcome data. We assessed suicidal
ideation using the Beck Scale for Suicidal Ideation (BSS) [34], and SAs
using the research version of the C-SSRS [35]. We assessed hopelessness
using the Beck Hopelessness Scale (BHS) [36]. We assessed social
connectedness in two ways. First, we used the validated Interpersonal
Needs Questionnaire-15 (INQ-15) scale that measures thwarted
belongingness (TB) and perceived burdensomeness (PB) [37]. Second,
we used the validated Multidimensional Scale of Perceived Social Sup­
port (MSPSS) [38]. The MSPSS assesses for overall support, and support
from a significant other, family, and friends [38].
We evaluated treatment engagement using a multidimensional
framework. First, we assessed self-management using the validated
Partners in Health (PIH) scale [39]. Second, we measured the patient’s
ability to cope with suicidal behavior using the validated Suicide-
Related Coping Measure (SRCM) scale [40]. The SRCM measures over­
all coping ability as well as external and internal coping ability. Third,
we measured intensity of treatment, defined as the total number of
mental health and primary care visits between baseline assessment and
the last study assessment. We abstracted utilization data from the VA
electronic medical record and asked patients about use of non-VA care
using a standardized form.
2.7. Analysis
We summarized baseline characteristics and measures of feasibility
using descriptive statistics. Using the intention-to-treat principle, we
developed a linear mixed-effects model to assess for differences in scores
between study arms and across time, including one- and three-months
post baseline assessment. We included treatment, time, and treatment-
time interaction in the model. This approach allowed us to account for
any missing follow-up data, and adjust for any baseline differences in
measures between study arms [41]. We compared health care utilization
data across study arms using two sample t-tests. We conducted all ana­
lyses in R.
We calculated preliminary effect sizes at each follow-up assessment.
To account for the between-group design, we used a t-statistic to
calculate a Cohen’s ds as well as a Hedges’ gs [42]. We defined an effect
size of 0.2 or less as small, 0.5 as medium, and 0.8 or greater as large.
3. Results
As shown in Supplementary Fig. 1, we were able to recruit 20 pa­
tients into our study. This was 65% of available, eligible patients (N =
31). Among the enrolled patients, 95% (N = 19) completed all study
assessments. Furthermore, among the 10 patients who were assigned to
VA BIC, 90% (N = 9) of patients completed all required intervention
visits, and 100% (N = 10) completed 70% or more of the required in­
terventions visits. One patient assigned to VA BIC was lost to follow-up
after the fifth VA BIC contact visit, and therefore, missed the last VA BIC
visit.
At study entry, patients on average reported mild symptoms of sui­
cidal ideation and hopelessness as well as mild problems with social
connectedness and engagement (see Table 1). Patients assigned to VA
BIC generally shared similar characteristics with patients assigned to
standard care. However, there were a few important differences.
Compared to patients assigned to VA BIC, patients assigned to standard
care were more often female (50% versus 20%), younger (mean age
48.1 years (Standard Deviation, SD: 17.6) versus 54.9 years (SD: 17.1)),
and had a lifetime history of a SA (50% versus 20%). Conversely, pa­
tients assigned to VA BIC reported higher mean suicide-related coping
scores than those assigned to standard care (mean 59.5(SD 21.9) vs 50.3
(SD 12.7). Of note, the majority of patients reported a history of SA via
overdose (e.g., aspirin). No SAs occurred within the three months prior
N.B. Riblet et al. General Hospital Psychiatry 75 (2022) 68–74

Table 1 Table 2
Key characteristics of enrolled population*. A description of extra study contacts and related co-interventions that occurred
VA BIC Cohort (N Standard care (N =
among patients randomized to receive the VA BIC plus standard care or standard
= 10) 10) care alone.

N (%) N (%) VA BIC Cohort Standard care

Gender, female 2 (20.0) 5 (50.0) (N = 10) (N = 10)
Mean age (y, SD) 54.9 (17.1) 48.1 (17.6) N (%) N (%)
Total patients 10 (100.0) 10 (100.0)
Demographic Variables
Race and Ethnicity Patients with extra study contacts due to acute safety concernsa
Black, Non-Hispanic 0 (0.0) 1 (10.0) Baseline assessmentb 1 (10.0) 0 (0.0)
White, Non-Hispanic 1 (100.0) 7 (70.0) First follow-up assessment 0 (0.0) 2 (20.0)
White, Hispanic 0 (0.0) 1 (10.0) Second follow-up assessment 0 (0.0) 1 (10.0)
Multiracial 0 (0.0) 1 (10.0)
Marital Status
Single 2 (20.0) 1 (10.0) Co-interventions resulting from extra study contactsc
Divorced/Widowed/Separated 2 (20.0) 2 (20.0) Total number of extra study contacts 1 (100.0) 3 (100.0)
Married 5 (50.0) 5 (50.0) Outpatient provider(s) notified of safety alert 1 (100.0) 3 (100.0)
Other 1 (10.0) 2 (20.0) Outpatient provider(s) scheduled a same-day
0 (0.0) 3 (100.0)
or next-day visit with the patient
Outpatient provider referred the patient back
Baseline Clinical Characteristics 0 (0.0) 1 (33.3)
to PC-MHI for further evaluation.
Number of Mental Health Conditionsa Patient was referred for ongoing psychiatric
0–1 Mental Health Conditions 5 (50.0) 4 (40.0) 0 (0.0) 1 (33.3)
care in the mental health clinic
2–3 Mental Health Conditions 3 (30.0) 3 (30.0) Patient was psychiatrically hospitalized 0 (0.0) 1 (33.3)
4+ Mental Health Conditions 2 (20.0) 3 (30.0)
Type of Mental Health Conditions Note: N = Number; % = percent; PC-MHI = Primary Care - Mental Health
Depressive Disorders 6 (60.0) 8 (80.0) Integration Clinic (PC-MHI), VA BIC = Veterans Affairs Brief Intervention and
Bipolar and Related Disorders 1 (10.0) 0 (0.0) Contact Program.
Anxiety Disorders 3 (30.0) 6 (60.0) a
The study made use of standardized safety protocol. If the patient’s re­
Trauma and Stressor Related sponses to the study assessments met pre-defined criteria, the patient had extra
0 (0.0) 1 (10.0)
Disorders
contact with study staff outside of the a priori study visits.
Obsessive Compulsive and Related b
Disorders
0 (0.0) 2 (20.0) Extra study contacts occurred prior to randomization.
c
Substance Related and Addictive Patients were assessed by a study psychiatrist and additional steps were
1 (10.0) 1 (10.0) taken as necessary by the staff psychiatrist and outpatient provider to ensure the
Disorders
History of Suicidal Behavior safety and well-being of the patient.
Lifetime history of one or more SA,
2 (20.0) 5 (50.0)
% (n)
trial of VA BIC in patients who present to an integrated care clinic for an
Note: N = Number; % = percent; SA = Suicide Attempt; SD = Standard Devia­ initial evaluation of a mental health concern. Our experience suggest
tion; VA BIC = Veterans Affairs Brief Intervention and Contact Program; y = that VA BIC could either be implemented or studied in a formal research
year. study in this setting using an entirely virtual format. Patient who are
a
As measured on the MINI International Neuropsychiatric Interview.
assigned to VA BIC appear to adhere to the intervention regimen con­
sisting of one telehealth educational visit followed by six telehealth
to study entry. In a few cases, the SA occurred ≥20 years ago. contact visits. VA BIC may benefit patients in integrated care settings by
At follow-up, two out of the 10 patients assigned to standard care had reducing suicidal ideation and decreasing hopelessness. These results,
extra contacts with the study personnel outside of the a priori study however, must be considered preliminary because of the limited samples
contacts (see Table 2). These contacts occurred because the patient’s size of this pilot study. Future research is needed to study the impact of
responses to the study assessment triggered the study safety protocol. VA BIC on suicide-related and other mental health outcomes in inte­
Patients received a range of co-interventions. Conversely, no patients grated care settings.
assigned to VA BIC required extra contact with the study personnel at There have been several clinical trials of WHO BIC in low- and
follow-up. middle-income countries [14]. In addition, there have been several
Data trends suggested that symptoms of suicidal ideation and small studies of an adaptation of WHO BIC (i.e., VA BIC) in US veterans
hopelessness improved at one- and three-month assessment among the discharged from an inpatient mental health unit [22,23]. Together,
VA BIC group (see Table 3). The largest reduction occurred at three- these studies have suggested that the BIC program may be useful in
months where scores fell by four points. At this level of improvement, reducing suicide risk after psychiatric discharge. There has been no
patients denied suicidal ideation and hopelessness. The corresponding study, however, of the BIC program in an integrated care setting. Unlike
effect sizes for VA BIC were in the medium to large range for suicidal prior studies [14,22,23], our study enrolled patients with comparably
ideation (gs = 0.71) and hopelessness (gs = 0.60) (see Table 4). There milder symptom burden, and substantially less risk of suicidal behavior.
was a medium effect on treatment engagement as measured by the Nevertheless, our preliminary findings suggested that in this population
number of non-VA BIC health care visits (mental health visits gs = 0.43; VA BIC may be associated with improvements in suicidal ideation and
primary care visits gs = 0.55). No SA occurred during the study. hopelessness. These results align with the limited body of research on
Although other outcomes improved, the effects were generally less brief suicide prevention strategies among patients at risk for suicide in
pronounced, and did not uniformly favor VA BIC. For example, patients primary care settings [19,20]. In a pre-post analysis of a crisis response
assigned to standard care reported greater improvements in coping and planning intervention in patients assessed for moderate suicide risk in an
thwarted belongingness than the VA BIC arm at three-month integrated care setting, Dueweke (2019) also found that patients expe­
assessment. rienced some improvement in suicidal intent, hopelessness, and coping
efficacy over a four-month period [20].
4. Discussion Overall, our findings suggested that it is feasible to study the VA BIC
program in an integrated care setting. It is noteworthy, however, that
Our pilot work demonstrates that it is feasible to conduct a virtual 20% of patients in the control arm receiving standard care arm had extra

71
N.B. Riblet et al. General Hospital Psychiatry 75 (2022) 68–74

Table 3 Health Scale; SD = Standard Deviation; SRCM = Suicide-Related Coping Mea­

Outcomes at one and three-month follow-up among patients randomized to sure; TB = Thwarted Belongingness; VA BIC = Veterans Affairs Brief Interven­
receive the VA BIC intervention plus standard care or standard care alone. tion and Contact Program.
VA BIC (N = Control (N = Between group
10) 10) differences
Table 4
Outcome measures Mean (SD) Mean (SD) Estimate Calculated effect sizes for VA BIC intervention plus standard care versus stan­
Suicidal Ideation (BSS) dard care alone.
Baseline 4.6 (11.7) 3.3 (6.7) 1.3
Outcome measure t- Cohen’s Hedges’ Favor VA BIC
Follow-up (1 M) 1.2 (17.8) 2.0 (10.3) − 0.8
statistic ds gs arm
Follow-up (3 M) 0.7 (17.9) 3.5 (10.3) − 2.8
Suicidal Ideation (BSS)
Assessment (1 M) − 0.86 − 0.38 − 0.0.37 Yesa
Hopelessness (BHS) Assessment (3 M) − 1.65 − 0.74 − 0.71 Yesc
Baseline 6.6 (9.8) 6.5 (5.6) 0.1
Follow-up (1 M) 3.1 (13.8) 3.7 (8.0) − 0.6
Follow-up (3 M) 2.3 (13.9) 4.7 (8.0) − 2.4 Hopelessness (BHS)
Assessment (1 M) − 0.39 − 0.18 − 0.17 Yesa
Assessment (3 M) − 1.39 − 0.62 − 0.60 Yesb
Perceived Burdensomeness (INQ-15, PB)
Baseline 13.0 (17.1) 12.2 (9.9) 0.8
Follow-up (1 M) 9.8 (26.8) 10.8 (15.4) − 1.0 Perceived Burdensomeness (INQ-PB)
Follow-up (3 M) 11.9 (26.9) 9.1 (15.4) 2.8 Assessment (1 M) − 0.48 − 0.21 − 0.21 Yesa
Assessment (3 M) 0.52 0.23 0.22 No

Thwarted Belongingness (INQ, TB)

Baseline 28.6 (27.9) 35.2 (16.1) − 6.6 Thwarted Belongingness (INQ-TB)
Follow-up (1 M) 24.2 (36.5) 26.9 (21.1) − 2.7 Assessment (1 M) 0.91 0.41 0.39 No
Follow-up (3 M) 22.4 (36.7) 24.4 (21.1) − 2.0 Assessment (3 M) 1.06 0.48 0.46 No

Self-Engagement (PIH) Self-Management (PIH)

Baseline 75.0 (29.9) 69.7 (17.3) 5.3 Assessment (1 M) − 0.88 − 0.39 − 0.38 No
Follow-up (1 M) 82.1 (39.7) 81.0 (22.9) 1.1 Assessment (3 M) − 0.57 − 0.26 − 0.25 No
Follow-up (3 M) 80.4 (40.0) 77.9 (22.9) 2.5

Overall Support (MSPSS)

Overall Social Support (MSPSS) Assessment (1 M) 0.13 0.06 0.06 Yesa
Baseline 4.6 (3.5) 4.4 (2.0) 0.2 Assessment (3 M) 0.83 0.37 0.36 Yesa
Follow-up (1 M) 5.4 (4.4) 5.2 (2.6) 0.2 Support from Significant
Follow-up (3 M) 5.9 (4.5) 5.3 (2.6) 0.6 Other
Support from Significant Assessment (1 M) 0.54 0.24 0.23 Yesa
Other Assessment (3 M) 0.64 0.29 0.28 Yesa
Baseline 5.1 (3.9) 5.1 (2.2) 0.0 Support from Family
Follow-up (1 M) 6.2 (5.4) 5.8 (3.1) 0.4 Assessment (1 M) − 1.05 − 0.47 − 0.45 No
Follow-up (3 M) 6.5 (5.5) 6.0 (3.1) 0.5 Assessment (3 M) 0.65 0.29 0.28 Yesa
Support from Family Support from Friends
Baseline 4.7 (4.8) 3.5 (2.8) 1.2 Assessment (1 M) 0.49 0.22 0.21 Yesa
Follow-up (1 M) 5.1 (5.6) 4.4 (3.2) 0.7 Assessment (3 M) 0.66 0.29 0.28 Yesa
Follow-up (3 M) 5.8 (5.6) 4.3 (3.2) 1.6
Support from Friends
Overall Coping with Suicidal Behavior (SRCM)
Baseline 4.0 (4.4) 4.7 (2.5) − 0.7
Assessment (1 M) − 1.38 − 0.62 − 0.59 No
Follow-up (1 M) 5.0 (5.9) 5.4 (3.4) − 0.4
Assessment (3 M) − 1.04 − 0.47 − 0.45 No
Follow-up (3 M) 5.3 (5.9) 5.5 (3.4) − 0.2
External Coping with Suicidal
Behavior
Overall Suicide-Related Coping (SRCM) Assessment (1 M) − 1.46 − 0.65 − 0.63 No
Baseline 59.5 (21.9) 50.3 (12.7) 9.2 Assessment (3 M) − 1.24 − 0.55 − 0.53 No
Follow-up (1 M) 60.6 (29.6) 56.4 (17.1) 4.2 Internal Coping with Suicidal Behavior
Follow-up (3 M) 63.6 (29.7) 58.2 (17.1) 5.4 Assessment (1 M) − 1.04 − 0.46 − 0.44 No
External Suicide-Related Assessment (3 M) − 0.31 − 0.14 − 0.13 No
Coping
Baseline 24.6 (9.9) 21.0 (5.7) 3.6
Healthcare utilization over three- month assessment periodd
Follow-up (1 M) 24.8 (14.7) 24.1 (8.5) 0.7
Total MH visits per patient − 0.99 0.44 0.42 Yesa
Follow-up (3 M) 26.4 (14.8) 25.3 (8.5) 1.1
Total PCP visits per patient − 1.23 0.55 0.52 Yesb
Internal Suicide-Related
Coping Note: BHS = Beck Hopelessness Scale; BSS = Beck Scale for Suicidal Ideation;
Baseline 24.4 (10.7) 20.7 (6.2) 3.7 INQ = Interpersonal Needs Questionnaire; M = Month; MH = Mental Health;
Follow-up (1 M) 25.2 (13.6) 23.1 (7.9) 2.1
MSPSS = Multidimensional Scale of Perceived Social Support; PB = Perceived
Follow-up (3 M) 25.8 (13.7) 22.6 (7.9) 3.2
Burdensomeness; PCP = Primary care provider; PIH = Partners in Health Scale;
SRCM = Suicide-Related Coping Measure; TB = Thwarted Belongingness; VA
Healthcare utilization over three- month assessment period BIC = Veterans Affairs Brief Intervention and Contact Program.
Total MH visits per a
Small effect.
7.0 (5.6) 4.6 (5.3) 2.4
patient b
Medium effect.
Total PCP visits per c
3.3 (4.1) 1.6 (1.6) 1.7 Nearly large effect.
patient d
t-statistic calculated based on results from independent two-sample t-test.
Note: BHS = Beck Hopelessness Scale; BSS = Beck Scale for Suicidal Ideation;
INQ = Interpersonal Needs Questionnaire; M = Month; MH = Mental Health;
MSPSS = Multidimensional Scale of Perceived Social Support; N = Number; PB
= Perceived Burdensomeness; PCP = Primary Care Provider; PIH = Partners in

72
N.B. Riblet et al.
contact with the study team outside of a priori scheduled study visits.
Based on research safety standards such as alerting clinicians to
increasing scores on standardized assessments of suicidality, these pa­
tients triggered the study safety protocol resulting in the patients
receiving a set of interventions that were in addition to the typical
standard of care. In most cases these interventions were greater intensity
than VA BIC intervention. Specifically, the staff psychiatrist helped the
patient with treatment engagement and notified providers of acute
concerns. The providers implemented further interventions to address
the patient’s acute symptoms, and these factors may have contaminated
our results. Aligned with this concern, our study was conducted during
the SARS-COV-2 pandemic, and its related physical distancing re­
strictions. It is possible that patients in the standard care arm may have
experienced unintended therapeutic benefit from the study assessments,
especially during times of social isolation. Several patients shared with
study staff how much they appreciated speaking with the assessor even
though our independent outcomes assessments were not designed to
have a therapeutic benefit.
Experts have emphasized that researchers face unique challenges in
conducting suicide research [29]. Importantly, investigators must strike
a balance between the need for frequent assessments to ensure safety,
and the risk that these frequent assessments may have unintended
therapeutic effects [29]. Certainly, an investigator must intervene in the
case of acute suicide risk or other safety concerns [29]. Mental health
researchers may face additional challenges in studying behavioral
change interventions such as ours. There is concern that behavioral
change interventions and assessments may interact and bias results [43].
Experts have found that assessments can be used to facilitate therapeutic
change [44]. These observations suggest that suicide researchers may
need to consider alternative strategies that can effectively manage sui­
cide risk during the course of a study, while also ensuring the integrity of
the research results. For example, one consideration may be decreasing
the frequency of the assessment visits, or further standardizing the
management of patients who trigger safety protocols for concerning
responses.
4.1. Strengths and limitations
We studied a highly promising brief suicide prevention strategy (VA
BIC) in an integrated care setting. Although this population is at risk for
suicide, little research has been done to understand the role of suicide
prevention strategies in this setting. The virtual aspect of VA BIC may
increase its applicability and scalability in primary care or integrated
care settings. Concerns have been raised about the potential challenges
posed by remote research during the SARS-COV-2 pandemic, but we had
exceptional study retention and relatively good recruitment [45].
Although one patient assigned to VA BIC was lost to follow-up, the pa­
tient received more than 80% of the intervention components.
Our study has several limitations. First, we had a small sample size.
As such, the study was not powered to detect statistically significant
difference, and we cannot draw conclusions regarding whether VA BIC
has any positive benefit on suicidal ideation or hopelessness in patients
who are seen in an integrated care setting. Our study included veterans
who access VA care. Because of the VA’s investment in mental health
services, patients who access VA care may have better access to mental
health treatment as compared to their civilian counterparts [46]. Sec­
ond, most patients were of white race. While there were a few patients
from underrepresented minority groups, these patients were all allo­
cated to standard care. Third, we included a broad population of patients
at risk for suicide, and the symptom acuity of enrolled patients was
markedly lower than that of a comparable inpatient sample [23]. While
this approach increased the generalizability of our findings, it made it
more difficult to demonstrate meaningful differences in outcomes in
such a small sample. Yet, despite this limitation, we did find that VA BIC
was associated with medium to large effects on suicidal ideation. It may
be useful in the future to conduct an adequately powered trial that can
73
General Hospital Psychiatry 75 (2022) 68–74
assess the effect of VA BIC on suicidal behaviors, and to determine
whether baseline suicide risk modifies this effect. It is possible that
primary care settings may benefit from selective prevention strategies
that can target a broader population of at-risk patients Finally, we
observed that there were some important differences in patient char­
acteristics between VA BIC and standard care. For example, more
women were in the standard care arm. There may be multiple unfore­
seen interactions between VA BIC, measures of social connectedness and
engagement, and gender. However, our study was not designed to
explore these hypotheses.
4.2. Conclusion
It is feasible to conduct a virtual trial of VA BIC in an integrated care
clinic. The intervention can be easily administered using a virtual plat­
form. Future research should further clarify the role of VA BIC as a
suicide prevention strategy in integrated care settings using an
adequately powered design.
Grant support and role of funder
This work was funded by the VA National Center for Patient Safety
Center of Inquiry Program, Ann Arbor, MI (PSCI-WRJ-Dr Shiner) and the
VA Office of Rural Health, Veterans Rural Health Resource Center,
White River Junction VT (ORH 15532). Dr. Riblet has support from the
Department of Veterans Affairs Clinical Science Research & Develop­
ment Career Development Award Program (MHBC-007-19F). The sup­
porters had no role in the design, analysis, interpretation, or publication
of this study. The views expressed in this article do not necessarily
represent the views of the Department of Veterans Affairs or of the
United States Government.
CRediT authorship contribution statement
Natalie B. Riblet: Conceptualization, Methodology, Validation,
Visualization, Writing – original draft, Writing – review & editing.
Lauren Kenneally: Investigation, Writing – original draft, Writing –
review & editing. Susan Stevens: Investigation, Writing – original draft,
Writing – review & editing. Bradley V. Watts: Conceptualization,
Methodology, Visualization, Writing – review & editing. Jiang Gui:
Formal analysis, Software, Data curation, Validation, Writing – review &
editing. Jenna Forehand: Investigation, Writing – original draft,
Writing – review & editing. Sarah Cornelius: Data curation, Writing –
original draft, Writing – review & editing. Glenna S. Rousseau: Re­
sources, Writing – review & editing. Jonathan C. Schwartz: Resources,
Writing – review & editing. Brian Shiner: Conceptualization, Method­
ology, Funding acquisition, Project administration, Resources, Super­
vision, Visualization, Writing – original draft, Writing – review &
editing.
Declaration of Competing Interest
The authors have no known conflict of interest to disclose.
Acknowledgements
We want to thank the patients who participated in this study. No
additional individuals were involved in this work outside of the authors.
Appendix A. Supplementary data
Supplementary data to this article can be found online at https://doi.
org/10.1016/j.genhosppsych.2022.02.002.
N.B. Riblet et al. General Hospital Psychiatry 75 (2022) 68–74

References [23] Riblet NB, Stevens SP, Watts BV, Gui J, Forehand J, Cornelius S, et al. A pilot
randomized trial of a brief intervention to prevent suicide after inpatient
psychiatric discharge. Psychiatr Serv 2021;72(11):1320–3. https://doi.org/
[1] Hedegaard H, Curtin SC, Warner M. Suicide mortality in the United States, 1999-
10.1176/appi.ps.202000537.
2019. NCHS Data Brief 2021;398:1–8. https://www.cdc.gov/nchs/data/datab
[24] Brenner LA, Barnes SM. Facilitating treatment engagement during high-risk
riefs/db398-H.pdf.
transition periods: a potential suicide prevention strategy. Am J Public Health
[2] Chung D, Hadzi-Pavlovic D, Wang M, Swaraj S, Olfson M, Large M. Meta-analysis
2012;102 Suppl 1(Suppl. 1):S12–4. https://doi.org/10.2105/ajph.2011.300581.
of suicide rates in the first week and the first month after psychiatric
[25] Calati R, Ferrari C, Brittner M, Oasi O, Olié E, Carvalho AF, et al. Suicidal thoughts
hospitalisation. BMJ Open 2019;9(3):e023883. https://doi.org/10.1136/bmjopen-
and behaviors and social isolation: a narrative review of the literature. J Affect
2018-023883.
Disord 2019;245:653–67. https://doi.org/10.1016/j.jad.2018.11.022.
[3] Ahmedani BK, Westphal J, Autio K, Elsiss F, Peterson EL, Beck A, et al. Variation in
[26] Fässberg MM, van Orden KA, Duberstein P, Erlangsen A, Lapierre S, Bodner E, et al.
patterns of health care before suicide: a population case-control study. Prev Med
A systematic review of social factors and suicidal behavior in older adulthood. Int J
2019;127:105796. https://doi.org/10.1016/j.ypmed.2019.105796.
Environ Res Public Health 2012;9(3):722–45. https://doi.org/10.3390/
[4] Luoma JB, Martin CE, Pearson JL. Contact with mental health and primary care
ijerph9030722.
providers before suicide: a review of the evidence. Am J Psychiatry 2002;159(6):
[27] Carter G, Milner A, McGill K, Pirkis J, Kapur N, Spittal MJ. Predicting suicidal
909–16. https://doi.org/10.1176/appi.ajp.159.6.909.
behaviours using clinical instruments: systematic review and meta-analysis of
[5] Dueweke AR, Bridges AJ. Suicide interventions in primary care: a selective review
positive predictive values for risk scales. Br J Psychiatry Jun 2017;210(6):387–95.
of the evidence. Fam Syst Health 2018;36(3):289–302. https://doi.org/10.1037/
https://doi.org/10.1192/bjp.bp.116.182717.
fsh0000349.
[28] Simpson SA, Goans C, Loh R, Ryall K, Middleton MCA, Dalton A. Suicidal ideation
[6] Platt S, Niederkrotenthaler T. Suicide prevention programs. Crisis 2020;41(1):
is insensitive to suicide risk after emergency department discharge: performance
S99–124. https://doi.org/10.1027/0227-5910/a000671.
characteristics of the Columbia-suicide severity rating scale screener. Acad Emerg
[7] Alexopoulos GS, Reynolds 3rd CF, Bruce ML, Katz IR, Raue PJ, Mulsant BH, et al.
Med Jun 2021;28(6):621–9. https://doi.org/10.1111/acem.14198.
Reducing suicidal ideation and depression in older primary care patients: 24-month
[29] Schatten HT, Gaudiano BA, Primack JM, et al. Monitoring, assessing, and
outcomes of the PROSPECT study. Am J Psychiatry 2009;166(8):882–90. https://
responding to suicide risk in clinical research. J Abnorm Psychol 2020;129(1):
doi.org/10.1176/appi.ajp.2009.08121779.
64–9. https://doi.org/10.1037/abn0000489.
[8] Bruce ML, Ten Have TR, Reynolds 3rd CF, Katz II, Schulberg HC, Mulsant BH, et al.
[30] Machin D, Campbell MJ, Tan SB, Tan SH. Sample sizes for clinical, laboratory and
Reducing suicidal ideation and depressive symptoms in depressed older primary
epidemiology studies. 4th ed. John Wiley and Sons; 2018. p. 251–67.
care patients: a randomized controlled trial. JAMA 2004;291(9):1081–91. https://
[31] National Institute of Health, National Center for Complementary and Integrative
doi.org/10.1001/jama.291.9.1081.
Health. Pilot Studies: Common uses and misuses. Available at, https://www.nccih.
[9] Okolie C, Dennis M, Simon Thomas E, John A. A systematic review of interventions
nih.gov/grants/pilot-studies-common-uses-and-misuses. Accessed February 8,
to prevent suicidal behaviors and reduce suicidal ideation in older people. Int
2022.
Psychogeriatr 2017;29(11):1801–24. https://doi.org/10.1017/
[32] Sheehan DV, Lecrubier Y, Sheehan KH, Amorim P, Janavs J, Weiller E, et al. The
s1041610217001430.
Mini-international neuropsychiatric interview (M.I.N.I.): the development and
[10] Richards JE, Parrish R, Lee A, Bradley K, Caldeiro R. An integrated care approach
validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10.
to identifying and treating the suicidal person in primary care. Psychiatr Times
J Clin Psychiatry 1998;59(20):22–57.
2019;36(1):9–15. https://www.psychiatrictimes.com/view/integrated-care-app
[33] American Psychiatric Association. Diagnostic and statistical manual of mental
roach-identifying-and-treating-suicidal-person-primary-care.
disorders. 5th ed. 2013. https://doi.org/10.1176/appi.books.9780890425596.
[11] Unutzer J, Tang L, Oishi S, Katon W, Williams Jr JW, Hunkeler E, et al. Reducing
[34] Beck AT, Steer RA. Manual for Beck scale for suicide ideation. Psychological
suicidal ideation in depressed older primary care patients. J Am Geriatr Soc 2006;
Corporation. 1991. p. 1–24.
54:1550–6. https://doi.org/10.1111/j.1532-5415.2006.00882.x.
[35] Posner K, Brown GK, Stanley B, et al. The Columbia-suicide severity rating scale:
[12] Mann JJ, Michel CA, Auerbach RP. Improving suicide prevention through
initial validity and internal consistency findings from three multisite studies with
evidence-based strategies: a systematic review. Am J Psychiatry 2021. https://doi.
adolescents and adults. Am J Psychiatry 2011;168(12):1266–77. https://doi.org/
org/10.1176/appi.ajp.2020.20060864. appiajp202020060864.
10.1176/appi.ajp.2011.10111704.
[13] Milner A, Witt K, Pirkis J, Hetrick S, Robinson J, Currier D, et al. The effectiveness
[36] Beck AT, Steer RA. Manual for Beck hopelessness scale. Psychological Corporation.
of suicide prevention delivered by GPs: a systematic review and meta-analysis.
1988. p. 1–29.
J Affect Disord 2017;210:294–302. https://doi.org/10.1016/j.jad.2016.12.035.
[37] Van Orden KA, Cukrowicz KC, Witte TK, Joiner Jr TE. Thwarted belongingness and
[14] Riblet NBV, Shiner B, Young-Xu Y, Watts BV. Strategies to prevent death by
perceived burdensomeness: construct validity and psychometric properties of the
suicide: meta-analysis of randomised controlled trials. Br J Psychiatry 2017;210
interpersonal needs questionnaire. Psychol Assess 2012;24(1):197–215. https://
(6):396–402. https://doi.org/10.1192/bjp.bp.116.187799.
doi.org/10.1037/a0025358.
[15] Tsiga E, Panagopoulou E, Sevdalis N, Montgomery A, Benos A. The influence of
[38] Zimet GD, Dahlem NW, Zimet SG, Farley GK. The multidimensional scale of
time pressure on adherence to guidelines in primary care: an experimental study.
perceived social support. J Pers Assess 1988;52(1):30–41. https://doi.org/
BMJ Open 2013;3(4):e002700. https://doi.org/10.1136/bmjopen-2013-002700.
10.1207/s15327752jpa5201_2.
[16] von dem Knesebeck O, Koens S, Marx G, Scherer M. Perceptions of time constraints
[39] Petkov J, Harvey P, Battersby M. The internal consistency and construct validity of
among primary care physicians in Germany. BMC Fam Pract 2019;20(1):142.
the partners in Health Scale: validation of a patient rated chronic condition self-
https://doi.org/10.1186/s12875-019-1033-5.
management measure. Qual Life Res 2010;19(7):1079–85. https://doi.org/
[17] Katzelnick DJ, Williams MD. Large-scale dissemination of collaborative care and
10.1007/s11136-010-9661-1.
implications for psychiatry. Psychiatr Serv 2015;66(9):904–6. https://doi.org/
[40] Stanley B, Green KL, Ghahramanlou-Holloway M, Brenner LA, Brown GK. The
10.1176/appi.ps.201400529.
construct and measurement of suicide-related coping. Psychiatr Res 2017;258:
[18] Dueweke AR, Rojas SM, Anastasia EA, Bridges AJ. Can brief behavioral health
189–93.
interventions reduce suicidal and self-harm ideation in primary care patients? Fam
[41] Schober P, Vetter TR. Repeated measures designs and analysis of longitudinal data:
Syst Health 2017;35(3):376–81. https://doi.org/10.1037/fsh0000287.
if at first you do not succeed-try, try again. Anesth Analg 2018;127(2):569–75.
[19] Raymond C, Myers S, Daly R, Murray D, Lyne J. Care pathways in a suicide crisis
[42] Lakens D. Calculating and reporting effect sizes to facilitate cumulative science: a
assessment nurse (SCAN) service. Int J Nurs Pract 2020;26(1):e12798. https://doi.
practical primer for t-tests and ANOVAs. Front Psychol 2013;4(863). https://doi.
org/10.1111/ijn.1279.
org/10.3389/fpsyg.2013.00863.
[20] Dueweke A. Preliminary outcomes, acceptability, and feasibility of a brief crisis
[43] McCambridge J, Butor-Bhavsar K, Witton J, Elbourne D. Can research assessments
response planning intervention for reducing suicide risk in primary care behavioral
themselves cause bias in behaviour change trials? A systematic review of evidence
health patients (Order No. 22588196). Health & Medical Collection; Nursing &
from Solomon 4-group studies. PLoS One 2011;6(10):e25223. https://doi.org/
Allied Health Premium; 2019 [Doctoral Dissertation], https://www.proquest.com/
10.1371/journal.pone.0025223.
dissertations-theses/preliminary-outcomes-acceptability-feasibility/docview/
[44] Finn SE, Fischer CT, Handler L, (editors).. Collaborative/therapeutic assessment:
2311959709/se-2?accountid=30099https://scholarworks.uark.edu/etd/3359.
Basic concepts, history, and research. In: Collaborative/therapeutic assessment: A
[21] Fleischmann A, Bertolote JM, Wasserman D, De Leo D, Bolhari J, Botega NJ, et al.
casebook and guide. John Wiley & Sons Inc; 2012. p. 1–24.
Effectiveness of brief intervention and contact for suicide attempters: a randomized
[45] McDermott MM, Newman AB. Remote research and clinical trial integrity during
controlled trial in five countries. Bull World Health Organ 2008;86(9):703–9.
and after the coronavirus pandemic. JAMA 2021;325(19):1935–6.
https://doi.org/10.2471/blt.07.046995.
[46] Leung LB, Rubenstein LV, Yoon J, et al. Veterans health administration investments
[22] Riblet NB, Shiner B, Schnurr P, et al. A pilot study of an intervention to prevent
in primary care and mental health integration improved care access. Health Aff
suicide after psychiatric hospitalization. J Nerv Ment Dis 2019;207(12):1031–8.
2019;38(8):1281–8. https://doi.org/10.1377/hlthaff.2019.00270.
https://doi.org/10.1097/nmd.0000000000001061.

74