1

RUNNING HEAD: IMPROVING UPTAKE OF CRISIS RESOURCES

NOTE: This manuscript is “in press” at the Journal of Behavioral and Cognitive Therapy
as of 12/25/2022. CITATION: Cohen, K., Dobias, M., Morris, R., & Schleider, J.L. (2022, in
press). Improving uptake of mental health crisis resources: Randomized test of a
single-session intervention embedded in social media. Journal of Behavioral and
Cognitive Therapy.

Improving uptake of mental health crisis resources: Randomized test of a single-session

intervention embedded in social media

Katherine Cohen, M.A.a, Mallory Dobias, M.A.a, Rob Morris, Ph.D.b, & Jessica Schleider, Ph.D.a*

a
Department of Psychology, Stony Brook Universityb Department of Psychology, Stony Brook
University; mallory.dobias@stonybrook.edu
b
Koko AI, Inc; rob@kokocares.org

*Corresponding author: jessica.schleider@stonybrook.edu

Abbreviations
DMHI: Digital Mental Health Intervention
SSI: Single Session Intervention
IMPROVING UPTAKE OF CRISIS RESOURCES 2

Abstract

Young people often experience difficulties accessing mental health support. In moments of
crisis, many young people search for mental health-related information or support on social
media platforms. When users search or post crisis-related content (e.g., “suicide”) on these
platforms, many are programmed to automatically provide the user with crisis hotlines. Little
research has examined whether young people use crisis hotlines when they are automatically
shared, or whether other resource-provision strategies may better support hope and
help-seeking. Methods: Users flagged as being potentially in-crisis by social media platforms
were referred to Koko—a nonprofit that partners with online platforms to provide crisis support.
Users were randomized to receive either a typical crisis response (988 crisis hotline provision)
or a one-minute, enhanced crisis response Single-Session Intervention (SSI). Results: Users
who received the enhanced crisis response SSI reported greater decreases in hopelessness ten
minutes later, compared to users who received the typical crisis response (t(153) = -2.16, p =
0.03, d = -0.35; 95% CI, -0.67, -0.03). Users who received the SSI were more than twice as
likely to report using the resources provided to them, compared to users who received the
typical crisis response (78.02% vs. 38.64%; χ2(1) = 27.02, p < 0.001, V = 0.28). Conclusion: An
enhanced crisis response SSI embedded within social media platforms can reduce users’
hopelessness and dramatically increase young people’s odds of accessing mental health
resources in moments of crisis.

Keywords: Single-session intervention, social media, crisis response, help-seeking

IMPROVING UPTAKE OF CRISIS RESOURCES 3

Improving uptake of mental health crisis resources: Randomized test of a single-session

intervention embedded in social media

Introduction

Young adults report alarming rates of psychological distress (Wang et al, 2020). Rates of
18-25 years old indicating severe psychological distress increased from 7.7% in 2008 to 13.1%
in 2017 (Twenge et al., 2019). Given the ubiquity of digital technologies among young adults
(Curtis et al., 2019), providing mental health support through digital mediums—particularly
platforms that young adults already naturalistically frequent, such as social media
platforms—could improve accessibility. Over 80% of young adults search online or via social
media for mental health information, suggesting that many are already using digital resources to
support their mental health (Pretorius et al., 2019).
Despite progress in the development of effective digital mental health interventions
(DMHIs) for young adults, real-world uptake is low (Lattie et al., 2021; Levin et al., 2022). One
explanation may be that accessing existing DHMIs requires additional user effort (e.g.,
downloading an app, requiring a subscription fee). Embedding DMHIs in online spaces that
young adults already frequent for mental health information-seeking purposes may overcome
user-burden barriers, capitalizing on moments of high motivation or perceived need (e.g., when
searching for mental health information online; when posting about suicidal thoughts).
Many social media apps use automated approaches to connecting users to mental
health crisis resources. For example, when users on Facebook, Instagram, or Tumblr search for
terms related to “suicide,” the platforms’ mental health response protocols are activated, and
they are presented with a crisis hotline number (e.g., 9-8-8). However, research has not
examined whether users subsequently use crisis hotlines when they are presented in this
context, nor whether this approach improves mental health outcomes. Qualitative research with
social media users suggests that this typical, automated approach to crisis resource-provision is
perceived as impersonal, insensitive, and insufficient by platform users (Biddle et al., 2020), who
prefer receiving resources personalized to their needs (Pretorius et al., 2020). Thus, research
must evaluate the assumed benefits and unintended harms of social media platforms’ standard
approaches to supporting users who may need mental health support.
Some social media platforms provide users flagged as “in crisis” with lists of multiple
resources (beyond a single hotline), presumably to increase odds that users will find one of
them helpful. However, it cannot be assumed that simply providing more resources to these
automated support protocols will necessarily help. Indeed, the authors of one recent study
suggested that providing resources that do not meet users’ needs may be harmful. In this study,
users who received referrals to in-person care following online mental health screeners were
more likely to later endorse suicidal intent. The authors of this study suggested that this could
be because the prospect of initiating in-person care was overwhelming to individuals (Jacobson
et al., 2022). No research has examined how online, automated crisis response protocols might
be optimized for positive real-world outcomes—that is, how they might be redesigned to
empower users and increase uptake of supports. Such research could inform critical decisions
made by large social media platforms and policy-makers, boosting millions of social media
users’ odds of connecting with care in moments of need.
IMPROVING UPTAKE OF CRISIS RESOURCES 4

Design principles from brief interventions—created to boost clinically-relevant outcomes

like hope, agency, and motivation—may offer a promising path toward optimizing automated
crisis response protocols. Several online single-session interventions (SSIs), or DMHIs
designed for completion in a single interaction, have improved each of these outcomes via
self-guided, 5-to-25-minute programs (Dobias et al., 2021; Schleider et al., 2022; Schleider &
Weisz, 2018; Sung et al., 2021). Shared design elements across effective SSIs include: (1)
using scientific information (e.g., neuroscience research findings, statistics) to normalize users’
problems and experiences, (2) offering users choices to boost autonomy and leverage personal
expertise of their own mental health needs, (3) including testimonials from others facing similar
challenges to enhance message credibility, and (4) empowering users to help others (i.e., peer
support). Applying theory-driven SSI design principles to automated crisis response protocols
may similarly empower users and encourage crisis resource uptake. However, this possibility
has not been formally evaluated. Furthermore, it is unclear whether college-aged users would
benefit from crisis response protocols that are specifically tailored for their developmental stage,
or if they would benefit equally from supports designed for users of any age.
The current randomized experiment compared two strategies for presenting mental
health-related information to users identified as potentially ‘at-risk’ of mental health crisis:
“support as usual” (automated provision of the 9-8-8 crisis hotline) versus an “enhanced crisis
response SSI” using design features drawn from evidence-based mental health SSIs.
Specifically, we tested whether adapting these four design principles decreased hopelessness
and increased crisis resource uptake, relative to standard resource provision among identified
social media users. This study was conducted on social media platforms where most users are
adolescents and young adults. We also evaluated whether SSI effects were similar among a
sub-sample of college-aged users (18-25 years old). Results of this study may inform
best-practices for automated, online mental health crisis response procedures for young people
using social media.

Methods

Koko Platform

We used data gathered from the Koko platform (https://www.kokocares.org/). Koko is a

nonprofit organization that uses machine learning techniques to connect users on social media
networks with crisis resources. Koko uses >1,300 keywords to detect terms such as ‘self-harm’
or ‘depression’ as well as slang and obfuscations, such as ‘sewer-slide’ and ‘'s3lf h@rm’
(Kshirsagar et al., 2017). When these words are detected, users are connected with myriad
self-help or peer-led evidence-based supports (Doré et al., 2017; Morris et al., 2015). Koko uses
barrier reduction and motivation enhancement techniques to improve users’ odds of utilizing
these resources (Jaroszewski et al., 2019).

Recruitment

Users were recruited to the Koko platform through partnered social media networks,
including Tumblr, Discord, Telegram, and Facebook Messenger. Recruitment took place via two
IMPROVING UPTAKE OF CRISIS RESOURCES 5

methods: 1) within-platform advertisements invited users to engage with Koko using messages
such as “24/7 peer support for your community. Always free, completely anonymous.” and 2)
when words are detected as crisis-related by Koko’s algorithms, users were shown an in-app
overlay of the Koko platform. Data for this study were collected between August 26 and
September 14, 2022.

Onboarding

After being referred to Koko’s platform, users were presented with Koko’s privacy policy
and terms of service, which inform users that anonymized data may be shared for research
purposes. As all data were part of a completely anonymous program evaluation, this study was
deemed as nonhuman subjects research in consultation with the institutional review board at
Stony Brook University.
All users who agreed to the privacy policy and terms of service (regardless of
recruitment method) were asked, “What are you struggling with?” and their free-text response
was scanned for crisis-related words. If their response passed Koko’s moderation systems, the
user was excluded from this study. If their response triggered Koko’s crisis model, the user was
sent to the crisis flow. Specifically, they received the following message: “Thanks for sharing
that… I'm concerned about you though. It sounds like you're feeling pretty down or you're in a
tough situation, is that right?” If the user agreed, they were asked to indicate what they are
struggling with: suicidal thoughts, self-harm, eating disorder, abuse, or something else. Users
indicated whether they were struggling with suicidal thoughts (n = 154), self-harm (n = 76),
eating disorders (n = 11), abuse (n = 38), or something else (n = 95).. These users (n = 374)
were then individually randomized to either the enhanced crisis response SSI (n = 190) or the
control condition (n = 184). Figure 1 illustrates the flow of participants in the study.

Enhanced Crisis Response SSI

Users in the “enhanced crisis response SSI” condition received information through a
brief (~1 minute) micro-intervention, based on design principles in the Single-Session
Intervention literature known to motivate hope, agency, and openness to treatment-seeking
(Schleider et al., 2020; Schleider & Weisz, 2018). Users in this condition received 1) a
psychoeducational validation statement (e.g., “A lot of Koko users struggle with self-harm. In
fact, 17% of young adults have purposefully harmed themselves at some point.”); 2) a
testimonial (e.g., “I have been struggling for years, but this [course] really changed me. We need
more things like this in the world.”); 3) the option to create a safety plan; 4) issue-specific links to
evidence-based “mini courses,” or online single-session interventions (described in Dobias et
al., 2022); and 5) global crisis lines. The testimonial that users were presented with differed
based on the concern they indicated struggling with. For example, users who indicated they
were struggling with an eating disorder were presented with the following message: “this course
felt very honest, and it helped me a lot to think about food + my body image in a healthier way.
Thanks!” Users were asked if they would use the resources they were presented with (i.e., “Be
honest, how likely are you to try the resources I just shared?”). Individuals who answered “very
likely” continued using the Koko platform as usual. Individuals who answered “not likely ...” were
IMPROVING UPTAKE OF CRISIS RESOURCES 6

presented with an interactive Barrier Reduction Intervention as described in Jaroszewski et al.

(2019). Users were notified that they would be contacted in ten minutes to check-in with them.
Appendix A includes representative screenshots portraying the enhanced crisis response SSI.

Control Condition (Crisis Response As Usual)

Users in the control condition received only crisis line information (e.g., “If you or
someone you know is struggling, help is available. If you're in the US: Call or Text 988 Suicide &
Crisis Lifeline. Otherwise, try IASP: https://www.iasp.info/crisis-centres-helplines/.”) The control
condition reflects the typical response that users receive when they search for mental health
topics on social media platforms. Users were notified that they would be contacted in ten
minutes to check-in with them. Appendix B includes representative screenshots portraying the
control condition.

Measures

Demographics. At the 10-minute follow-up, users indicated their age, gender identity
(man/boy, woman/girl, nonbinary, questioning, or other) and sexual identity (bisexual,
gay/lesbian, heterosexual, questioning, pansexual, queer, or other).

Platform. Users were categorized based on the platform that referred them to Koko.
Potential platforms included Discord, Tumblr, Facebook, and Telegram.

Beck Hopelessness Scale 4 Item (BHS-4). At pre-randomization and 10-minute

follow-up, users were asked to rate their agreement with four statements related to
hopelessness (e.g., “I feel that my future is hopeless and that things will not improve”) on
a 4-point Likert scale from Absolutely Disagree to Absolutely Agree (Perczel Forintos et
al., 2013). Responses were summed to create a total score. Higher scores indicated
higher levels of hopelessness. Users completed this scale immediately prior to
randomization and at the ten-minute follow up. The BHS-4 served as the present study’s
index of pre-treatment and immediate-post-treatment clinically-relevant distress.
Previous studies have used the BHS-4 as an index of pre-intervention psychological
distress (Kuyken, 2004; Cohen & Schleider, 2021). Additionally, ​evidence from a
previous study suggests that small changes in immediate post-intervention outcomes are
anchored in personally perceptible changes (Schleider et al., 2019). Internal consistency
prior to randomization was α = 0.77. Internal consistency at the ten-minute follow up was
α = 0.89.

Resources. At the ten-minute follow up, users were asked to self-report whether or not
they used any of the crisis or mental health resources that Koko provided. Users were
also asked to rate how helpful they believed the resources were on a 5-point Likert scale
from Very Helpful to Very Unhelpful. Higher scores indicated higher levels of perceived
helpfulness. The short-time span of our follow-up (10 minutes) is intentional and
informed by work demonstrating the importance of minute-to-minute changes in suicidal
IMPROVING UPTAKE OF CRISIS RESOURCES 7

thoughts. For example, research indicates that approximately 24% of individuals spend
less than 5 minutes between the decision to attempt suicide and the actual attempt
(Simon et al, 2001). As such, it may be possible that even small, positive changes over a
course of several minutes can provide substantial value.

Data Analysis

The RStudio Statistical Program was used to complete data analyses (Allaire, 2012). For
each statistical test, a p-value of < 0.05 was considered statistically significant.

College-Aged Subsample. Along with conducting analyses on the full sample of users
who engaged in the Koko platform, we conducted analyses on the subsample of users
classified as college-aged (between the ages of 18-25). We ran the same set of
analyses to see if there were effects for the college-aged population that differed from
the larger sample.

Participant Characteristics. We report users’ demographic characteristics using means,

standard deviations, and percentages. To determine whether randomization between the
treatment and control conditions was successful, we conducted a t-test for age and
chi-square tests for gender identity, sexual identity, and platform. Because participants
answered demographic questions only at the ten-minute follow-up, demographic data
could not be obtained from participants who did not complete the follow-up.

Uptake Patterns. We report follow-up rates and resource uptake using means, standard
deviations, and percentages. To determine whether follow-up rates or uptake of
resources differed between the treatment and control conditions, we conducted
chi-square tests. We report participants’ perceived helpfulness of resources using means
and standard deviations. To determine whether perceived helpfulness of resources
differed between the treatment and control conditions, we conducted a chi-square test.

Effectiveness of Enhanced Crisis Response SSI. To test our main hypothesis

(between-group effects on hopelessness), we conducted a linear regression that
examined whether the treatment versus the control led to differential changes in
hopelessness from baseline to ten-minutes post-intervention. We used listwise deletion
for participants lacking ten-minute follow-up data. We chose this method rather than
imputing the missing data or using the last observation carried forward because only the
dependent variable has missing values and there are no auxiliary variables (Jakobsen et
al., 2017).

Results

Participant Characteristics
IMPROVING UPTAKE OF CRISIS RESOURCES 8

Full Sample. A total of 374 users were randomized to a condition. The control condition
included 184 users. The enhanced crisis response SSI included 190 users. Figure 1
demonstrates the flow of participants in the study for the full sample. Table 1 includes
descriptive statistics for the participants who completed the 10-minute follow-up (n =
184). The average age was 17.93 (SD = 4.92, Range = 13-44). Participants’ gender
identities included woman/girl (n = 62, 41.06%), man/boy (n = 38, 25.17%), nonbinary (n
= 22, 14.57%), questioning (n = 10, 6.62%), and other (n = 19, 12.58%). Participants’
sexual identities included pansexual (n = 15, 27.78%), queer (n = 14, 7.41%),
heterosexual (n = 10, 18.52%), bisexual (n = 8, 14.81%), gay/lesbian (n = 5, 9.26%),
questioning (n = 3, 5.56%), and other (n = 9, 16.67%). Participants joined the Koko
platform from Tumblr (n = 288, 81.13%), Discord (n = 55, 15.49%), Facebook (n = 6,
1.7%), and Telegram (n = 6, 1.7%). Analyses indicated no significant differences
between participant characteristics in the control and the enhanced treatment response,
indicating successful randomization.

College-Aged Subsample. A total of 54 users that were randomized to a condition

indicated in the 10-minute follow up that they were between 18-25 years old. The control
condition included 26 college-aged users. The enhanced crisis response SSI condition
included 28 college-aged users. Table 2 includes descriptive statistics for the
college-aged sample of randomized participants. The average age of participants in this
group was 20.48 (SD = 1.81). Participants’ gender identites included woman/girl (n = 25,
46.3%), man/boy (n = 15, 27.78%), nonbinary (n = 8, 14.81%), questioning (n = 4,
7.41%), and other (n = 2, 3.70%). Participants’ sexual identities included bisexual (n =
35, 23.18%), pansexual (n = 26, 17.22%), heterosexual (n = 23, 15.23%), gay/lesbian (n
= 18, 11.92%), questioning (n = 11, 7.28%), queer (n = 10, 6.62%), and other (n = 28,
18.64%). Participants joined the Koko platform from Tumblr (n = 51, 94.44%), Discord (n
= 2, 3.7%), and Facebook (n = 1, 1.85%). Analyses indicated no significant differences
between participant characteristics in the control and the enhanced treatment response,
indicating successful randomization.

Dropout

Of the participants who were randomly assigned to a condition, 51.83% completed the
ten-minute follow-up survey. There was no evidence of differential dropout between participants
assigned to different conditions or participants from different platforms.

Uptake and Helpfulness of Resources

Full Sample. Table 3 presents means and standard deviations for all program outcomes
by experimental condition at baseline and 10-minute follow-up. Participants in the
enhanced crisis response SSI reported greater uptake of resources (78.02% utilized
resources) compared to participants in the control condition (38.64% utilized resources),
(χ2(1) = 27.02, p < 0.001, V = 0.28). Participants in the enhanced crisis response SSI
condition did not report statistically different rates of perceived helpfulness of resources
IMPROVING UPTAKE OF CRISIS RESOURCES 9

compared to participants in the control condition, (t(183) = -1.82, p = 0.07, d = -0.27;

95%, -0.56, 0.02).

College-Aged Sample. Table 4 presents means and standard deviations for all program
outcomes by experimental condition at baseline and 10-minute follow-up. Participants in
the enhanced crisis response SSI condition reported greater uptake of resources
(85.71% utilized resources) compared to participants in the control condition (38.46%
utilized resources), though the number of participants was too small to conduct a test for
statistical significance. Participants in the enhanced crisis response SSI condition did not
report statistically different rates of perceived helpfulness of resources compared to
participants in the control condition, (t(53) = -1.23, p = 0.22, d = -0.34; 95% CI, -0.87,
0.2).

Hopelessness

Full Sample. Participants in the enhanced crisis response SSI condition reported greater
decreases in hopelessness, from baseline to 10-minutes post-intervention, compared to
participants in the control condition (t(153) = -2.16, p = 0.03, d = -0.35; 95% CI, -0.67,
-0.03). Regarding within-group effects, there was a statistically significant within-group
decrease in hopelessness among participants in the enhanced crisis response SSI
condition (t(71)=-2.47, p = 0.02, d = -0.41; 95% CI, -0.73, -0.08). In contrast, there was
not a significant difference between pre and post hopelessness scores among
participants in the control condition (t(72)=-0.39, p = 0.69, d = -0.06; 95% CI, -0.37,
0.25).

College-Aged Subsample. Participants in the enhanced crisis response SSI condition

reported greater decreases in hopelessness, from baseline to 10-minutes
post-intervention, compared to participants in the control condition (t(52) = -2.36, p =
0.02, d = -0.65; 95% CI, -1.2, -0.09). Regarding within-group effects, there was a
statistically significant within-group decrease in hopelessness among participants in the
enhanced crisis response SSI condition (t(26)=-3.04; p = 0.01, d = -0.81; 95% CI, -1.36,
-0.27). In contrast, there was not a significant difference between pre and post
hopelessness scores among participants in the control condition (t(23)=-0.63, p = 0.53, d
= -0.18; 95% CI, -0.72, 0.37).

Discussion

Young people’s mental distress frequently goes untreated or unaddressed (Gulliver et

al., 2010), creating a clear need to evaluate and optimize methods for offering mental health
support to individuals with acute, in-the-moment needs. Given the ubiquity of social media
usage among young people, we tested whether an enhanced, social media-embedded
single-session intervention (SSI) could outperform broadly-implemented automated strategies
for connecting users with crisis support. Social media users identified as being in crisis were
randomized to either receive a typical crisis response (a crisis hotline) or an enhanced crisis
 IMPROVING UPTAKE OF CRISIS RESOURCES
10
response SSI integrating evidence-based design features for improving hope and agency
(Schleider et al., 2020). The typical crisis response and the enhanced crisis response SSI were
both delivered in a context (social media platforms that young people regularly use) and at a
time (after indicating acute distress) in which young people may be most likely to benefit from
resources. However, the enhanced crisis response SSI also delivered resources in a manner
that was adapted to the user’s needs and created with evidence-based design principles
(Schleider et al., 2020). Our findings suggest that the enhanced crisis response SSI proved
superior to the typical automated crisis response; users in the enhanced crisis response
condition reported greater decreases in hopelessness compared to users in the typical crisis
response condition. Twice as many users in the enhanced crisis response condition reported
using the resources provided to them (78.02%) compared to users in the typical crisis response
condition (38.64%). Results were similar in the analyses limited to adults aged 18-25,
suggesting that the enhanced response is beneficial to users both within and outside the typical
college-aged developmental stage.
The utility of these findings is broad. Social media companies and online platforms may
consider them when designing crisis response procedures. We do not suggest that social media
websites or online platforms are responsible for the mental health of their users, but we contend
that these companies may improve their users’ likelihoods of accessing help by presenting
resources in an evidence-based manner. This includes adopting the design principles of the
enhanced crisis response SSI: addressing barriers to resource utilization, validating users’
experiences, promoting autonomy by offering choices, including testimonials from others, and
offering users the opportunity provide peer support (Jaroszewski et al., 2019; Schleider et al.,
2020). Our findings may also inform crisis response protocols in other contexts—for instance, in
research studies when participants endorse risk of self-harm. Using the principles outlined
above may improve the safety of the research procedures by increasing the odds that
participants use the resources provided. For example, researchers may provide participants with
the option to create a digital safety plan or complete a digital SSI developed for youth
experiencing self-injurious thoughts and behaviors (Dobias et al., 2021; Fox & Wang, 2020). In
colleges and universities, brief interventions could be integrated into learning management
systems such as Canvas or Blackboard, where interventions could be triggered when users
submit assignments or make discussion posts that include crisis-related words.
Although users in the enhanced crisis response SSI used resources more, no
differences in perceived helpfulness of resources emerged across groups. This may point to the
fact that the enhanced treatment response does not necessarily improve the quality of the
resources, but the likelihood that users will engage with them (an outcome of at least equal
importance: Supports can help only if users access them).
Notably, hopelessness levels non-significantly increased from 12.48 (SD = 3.02) at
baseline to 12.8 (SD = 3.41) at ten-minute follow-up among control group participants. However,
the increase in average scores from baseline to ten-minute follow-up in this sample was likely
due to chance and should therefore be interpreted with caution. Future research should
consider mixed-methods data regarding the potential harms of standard automated crisis
responses.
Several limitations of this study warrant mention. First, data analyses only included
participants who completed the follow-up. Relatedly, demographic data were obtained only from
 IMPROVING UPTAKE OF CRISIS RESOURCES
11
participants who completed the follow-up, preventing tests of whether users with particular
identities were differentially likely to complete the follow-up assessment. A previous study found
that in the context of brief, online interventions, dropout does not significantly differ between
participants by identity (Cohen & Schleider, 2022). Nonetheless, examining differential dropout
rates between demographic groups in the specific context of crisis-response interventions may
be useful. Second, we do not have objective data regarding whether the users actually used the
resources they were provided with or which resource they used, only whether they reported that
they used some resource. It is possible that users reported that they used resources without
actually using them due to social desirability bias. However, our use of a realistic control group
mitigated this likelihood. Third, most users came from one platform (Tumblr). With a larger
sample of users from multiple platforms, future research may identify platform-specific factors to
consider when optimizing crisis response procedures. Fourth, while our total sample size is
larger than the average sample size in randomized control trials with young people (Weisz et al.,
2017), the subsample of participants in the college-age range is small (n=54). Future studies
may limit recruitment specifically to students attending college. Lastly, the current study was not
designed to easily collect data regarding how many users were excluded at each stage of the
recruitment process (e.g., how many users did not agree to the terms of use, how many users
did not indicate they were struggling with a crisis-related issue, etc.) Future studies could
consider methods for collecting this data prior to the start of the study.

Conclusion

When embedded within social media platforms, a 1-minute enhanced crisis response
SSI more than doubled users’ odds of accessing mental health resources (from 38.6% to
78.1%) compared to standard, broadly-implemented crisis response procedures (providing a
crisis hotline alone). Likewise, the enhanced crisis response SSI led to larger reductions in
hopelessness compared to standard crisis response protocols. Results were consistent across
college-aged and non-college-aged social media users, suggesting broad utility of our brief,
enhanced approach. This study suggests an evidence-based strategy for improving social
media users’ likelihood of connecting with mental health support at precise moments of
perceived need, carrying immediately-actionable implications for policy and practice.

Acknowledgments
The present study was supported by a research grant from Hopelab to RM and JLS. Outside of this
work, JLS has received funding from the National Institute of Health Office of the Director
(DP5OD028123), National Institute of Mental Health (R43MH128075), National Science Foundation
(2141710), Health Research and Services Association (U3NHP45406-01-00), Upswing Fund for
Adolescent Mental Health, Society for Clinical Child and Adolescent Psychology, and the
Klingenstein Third Generation Foundation. The preparation of this article was supported in part by
the Implementation Research Institute (IRI), at the George Warren Brown School of Social Work,
Washington University in St. Louis; through an award from the National Institute of Mental Health
(R25MH080916; JLS is an IRI Fellow).

Conflicts of Interest
 IMPROVING UPTAKE OF CRISIS RESOURCES
12
KC is a contracted researcher for Koko. MLD is a contracted researcher for Koko. JLS serves
on the Scientific Advisory Board for Walden Wise and the Clinical Advisory Board for Koko, is
Co-Founder and Co-Director of Single Session Support Solutions. Inc., and receives book
royalties from New Harbinger, Oxford University Press, and Little Brown Book Group.

Author Contributions
JLS and RM conceptualized the project and developed the enhanced crisis response SSI. KC,
JLS, and RM contributed to study design. RM oversaw data collection. KC conducted primary
analyses and drafted the initial version of the manuscript. KC, MLD, and JLS contributed to
manuscript edits and revisions of subsequent drafts. All authors approved the final version of
the manuscript prior to submission.
 IMPROVING UPTAKE OF CRISIS RESOURCES
13

References

Allaire, J. (2012). RStudio: Integrated development environment for R. Boston, MA, 770(394),

165–171.

Biddle, L., Derges, J., Goldsmith, C., Donovan, J. L., & Gunnell, D. (2020). Online help for

people with suicidal thoughts provided by charities and healthcare organisations: A

qualitative study of users’ perceptions. Social Psychiatry and Psychiatric Epidemiology,

55(9), 1157–1166. https://doi.org/10.1007/s00127-020-01852-6

Burke, T. A., Bettis, A. H., Barnicle, S. C., Wang, S. B., & Fox, K. R. (2021). Disclosure of

Self-Injurious Thoughts and Behaviors Across Sexual and Gender Identities. Pediatrics,

148(4), e2021050255. https://doi.org/10.1542/peds.2021-050255

Cohen, K. A., & Schleider, J. L. (2022). Adolescent dropout from brief digital mental health

interventions within and beyond randomized trials. Internet Interventions, 27, 100496.

https://doi.org/10.1016/j.invent.2022.100496

Curtis, B. L., Ashford, R. D., Magnuson, K. I., & Ryan-Pettes, S. R. (2019). Comparison of

Smartphone Ownership, Social Media Use, and Willingness to Use Digital Interventions

Between Generation Z and Millennials in the Treatment of Substance Use:

Cross-Sectional Questionnaire Study. Journal of Medical Internet Research, 21(4),

e13050. https://doi.org/10.2196/13050

Dobias, M. L., Morris, R. R., & Schleider, J. L. (2022). Single-Session Interventions Embedded

Within Tumblr: Acceptability, Feasibility, and Utility Study. JMIR Formative Research,

6(7), e39004. https://doi.org/10.2196/39004

Dobias, M. L., Schleider, J. L., Jans, L., & Fox, K. R. (2021). An online, single-session

intervention for adolescent self-injurious thoughts and behaviors: Results from a

randomized trial. Behaviour Research and Therapy, 147, 103983.

https://doi.org/10.1016/j.brat.2021.103983
 IMPROVING UPTAKE OF CRISIS RESOURCES
14
Doré, B. P., Morris, R. R., Burr, D. A., Picard, R. W., & Ochsner, K. N. (2017). Helping Others

Regulate Emotion Predicts Increased Regulation of One’s Own Emotions and

Decreased Symptoms of Depression. Personality & Social Psychology Bulletin, 43(5),

729–739. https://doi.org/10.1177/0146167217695558

Fox, K. R., Bettis, A. H., Burke, T. A., Hart, E. A., & Wang, S. B. (2022). Exploring Adolescent

Experiences with Disclosing Self-Injurious Thoughts and Behaviors Across Settings.

Research on Child and Adolescent Psychopathology, 50(5), 669–681.

https://doi.org/10.1007/s10802-021-00878-x

Fox, K., & Wang, S. B. (2020). Online Methods in Adolescent Self-Injury Research: Challenges

and Recommendations. Retrieved from osf.io/g6cha

Gulliver, A., Griffiths, K. M., & Christensen, H. (2010). Perceived barriers and facilitators to

mental health help-seeking in young people: A systematic review. BMC Psychiatry, 10,

113. https://doi.org/10.1186/1471-244X-10-113

Jacobson, N. C., Yom-Tov, E., Lekkas, D., Heinz, M., Liu, L., & Barr, P. J. (2022). Impact of

online mental health screening tools on help-seeking, care receipt, and suicidal ideation

and suicidal intent: Evidence from internet search behavior in a large U.S. cohort.

Journal of Psychiatric Research, 145, 276–283.

https://doi.org/10.1016/j.jpsychires.2020.11.010

Jakobsen, J. C., Gluud, C., Wetterslev, J., & Winkel, P. (2017). When and how should multiple

imputation be used for handling missing data in randomised clinical trials - a practical

guide with flowcharts. BMC medical research methodology, 17(1), 162.

https://doi.org/10.1186/s12874-017-0442-1

Jaroszewski, A. C., Morris, R. R., & Nock, M. K. (2019). Randomized controlled trial of an online

machine learning-driven risk assessment and intervention platform for increasing the use

of crisis services. Journal of Consulting and Clinical Psychology, 87(4), 370–379.

https://doi.org/10.1037/ccp0000389
 IMPROVING UPTAKE OF CRISIS RESOURCES
15
Kshirsagar, R., Morris, R., & Bowman, S. (2017). Detecting and Explaining Crisis. Proceedings

of the Fourth Workshop on Computational Linguistics and Clinical Psychology — From

Linguistic Signal to Clinical Reality, 66–73. https://doi.org/10.18653/v1/W17-3108

Lattie, E. G., Cohen, K. A., Hersch, E., Williams, K. D. A., Kruzan, K. P., MacIver, C., Hermes,

J., Maddi, K., Kwasny, M., & Mohr, D. C. (2021). Uptake and effectiveness of a

self-guided mobile app platform for college student mental health. Internet Interventions,

27, 100493. https://doi.org/10.1016/j.invent.2021.100493

Levin, M. E., Hicks, E. T., & Krafft, J. (2022). Pilot evaluation of the stop, breathe & think

mindfulness app for student clients on a college counseling center waitlist. Journal of

American College Health, 70(1), 165–173.

https://doi.org/10.1080/07448481.2020.1728281

Morris, R. R., Schueller, S. M., & Picard, R. W. (2015). Efficacy of a Web-based, crowdsourced

peer-to-peer cognitive reappraisal platform for depression: Randomized controlled trial.

Journal of Medical Internet Research, 17(3), e72. https://doi.org/10.2196/jmir.4167

Ortiz, A., & Eder, S. (2022, July 15). The U.S. Has a New Crisis Hotline: 988. Is It Prepared for a

Surge in Calls? The New York Times.

https://www.nytimes.com/2022/07/15/us/988-mental-health-lifeline.html

Perczel Forintos, D., Rózsa, S., Pilling, J., & Kopp, M. (2013). Proposal for a short version of the

Beck Hopelessness Scale based on a national representative survey in Hungary.

Community Mental Health Journal, 49(6), 822–830.

https://doi.org/10.1007/s10597-013-9619-1

Pretorius, C., Chambers, D., Cowan, B., & Coyle, D. (2019). Young People Seeking Help Online

for Mental Health: Cross-Sectional Survey Study. JMIR Mental Health, 6(8), e13524.

https://doi.org/10.2196/13524

Pretorius, C., McCashin, D., Kavanagh, N., & Coyle, D. (2020). Searching for Mental Health: A

Mixed-Methods Study of Young People’s Online Help-seeking. Proceedings of the 2020

 IMPROVING UPTAKE OF CRISIS RESOURCES
16
CHI Conference on Human Factors in Computing Systems, 1–13.

https://doi.org/10.1145/3313831.3376328

Schleider, J. L., Abel, M. R., & Weisz, J. R. (2019). Do Immediate Gains Predict Long-Term

Symptom Change? Findings from a Randomized Trial of a Single-Session Intervention

for Youth Anxiety and Depression. Child psychiatry and human development, 50(5),

868–881. https://doi.org/10.1007/s10578-019-00889-2

Schleider, J. L., Dobias, M. L., Sung, J. Y., & Mullarkey, M. C. (2020). Future Directions in

Single-Session Youth Mental Health Interventions. Journal of Clinical Child and

Adolescent Psychology: The Official Journal for the Society of Clinical Child and

Adolescent Psychology, American Psychological Association, Division 53, 49(2),

264–278. https://doi.org/10.1080/15374416.2019.1683852

Schleider, J. L., Mullarkey, M. C., Fox, K. R., Dobias, M. L., Shroff, A., Hart, E. A., & Roulston,

C. A. (2022). A randomized trial of online single-session interventions for adolescent

depression during COVID-19. Nature Human Behaviour, 6(2), 258–268.

https://doi.org/10.1038/s41562-021-01235-0

Schleider, J. L., & Weisz, J. R. (2018). Parent Expectancies and Preferences for Mental Health

Treatment: The Roles of Emotion Mind-Sets and Views of Failure. Journal of Clinical

Child & Adolescent Psychology, 1–17. https://doi.org/10.1080/15374416.2017.1405353

Simon, O. R., Swann, A. C., Powell, K. E., Potter, L. B., Kresnow, M. J., & O'Carroll, P. W.

(2001). Characteristics of impulsive suicide attempts and attempters. Suicide &

life-threatening behavior, 32(1 Suppl), 49–59.

https://doi.org/10.1521/suli.32.1.5.49.24212

Sung, J. Y., Mumper, E., & Schleider, J. L. (2021). Empowering Anxious Parents to Manage

Child Avoidance Behaviors: Randomized Control Trial of a Single-Session Intervention

for Parental Accommodation. JMIR Mental Health, 8(7), e29538.

https://doi.org/10.2196/29538
 IMPROVING UPTAKE OF CRISIS RESOURCES
17
Twenge, J. M., Cooper, A. B., Joiner, T. E., Duffy, M. E., & Binau, S. G. (2019). Age, period, and

cohort trends in mood disorder indicators and suicide-related outcomes in a nationally

representative dataset, 2005-2017. Journal of Abnormal Psychology, 128(3), 185–199.

https://doi.org/10.1037/abn0000410

Wang, X., Hegde, S., Son, C., Keller, B., Smith, A., & Sasangohar, F. (2020). Investigating

Mental Health of US College Students During the COVID-19 Pandemic: Cross-Sectional

Survey Study. Journal of Medical Internet Research, 22(9), e22817.

https://doi.org/10.2196/22817

Weisz, J. R., Kuppens, S., Ng, M. Y., Eckshtain, D., Ugueto, A. M., Vaughn-Coaxum, R.,

Jensen-Doss, A., Hawley, K. M., Krumholz Marchette, L. S., Chu, B. C., Weersing, V. R.,

& Fordwood, S. R. (2017). What five decades of research tells us about the effects of

youth psychological therapy: A multilevel meta-analysis and implications for science and

practice. The American psychologist, 72(2), 79–117. https://doi.org/10.1037/a0040360

 IMPROVING UPTAKE OF CRISIS RESOURCES
18
Figure 1. CONSORT Diagram for Full Sample.
 IMPROVING UPTAKE OF CRISIS RESOURCES
19
Table 1. Participant Characteristics in Full Sample.

Control (N = 91) Enhanced Crisis

Response (N = 93)

Age M = 17.49, SD = 4.67 M = 18.39, SD = 5.16

Gender Identity

Man/Boy 23 (29.49%) 15 (20.55%)

Nonbinary 12 (15.38%) 10 (13.7%)

Other 7 (8.97%) 12 (16.44%)

Questioning 8 (10.26%) 2 (2.74%)

Woman/Girl 28 (35.9%) 34 (46.58%)

Sexual Identity

Bisexual 19 (24.36%) 16 (21.92%)

Gay/Lesbian 8 (10.26%) 10 (13.7%)

Heterosexual 14 (17.95%) 9 (12.33%)

Other 12 (15.38%) 16 (21.92%)

Pansexual 14 (17.95%) 12 (16.44%)

Queer 4 (5.13%) 6 (8.22%)

Questioning 7 (8.97%) 4 (5.48%)

Control (N = 173) Enhanced Crisis

Response SSI (N =
182)

Platform

Discord 27 (15.61%) 28 (15.38%)

Facebook 2 (1.16%) 4 (2.2%)

Telegram 1 (0.58%) 5 (2.75%)

Tumblr 143 (82.66%) 145 (79.67%)

 IMPROVING UPTAKE OF CRISIS RESOURCES
20
Table 2. Participant Characteristics in College-Aged Sample.

Control (N = 26) Enhanced Crisis

Response SSI (N = 28)

Age M = 20.69 , SD = 1.98 M = 20.29, SD = 1.65

Gender Identity

Man/Boy 8 (30.77%) 7 (25%)

Nonbinary 5 (19.23%) 3 (10.71%)

Other 1 (3.85%) 1 (3.57%)

Questioning 2 (7.69%) 2 (7.14%)

Woman/Girl 10 (38.46%) 15 (53.57%)

Sexual Identity

Bisexual 3 (11.54%) 5 (17.86%)

Gay/Lesbian 3 (11.54%) 2 (7.14%)

Heterosexual 6 (23.08%) 4 (14.29%)

Other 4 (15.38%) 5 (17.86%)

Pansexual 6 (23.08%) 9 (32.14%)

Queer 1 (3.85%) 3 (10.71%)

Questioning 3 (11.54%) 0

Platform

Discord 2 (7.69%) 0 (0%)

Facebook 0 (0%) 1 (3.57%)

Tumblr 24 (92.31%) 27 (96.43%)

 IMPROVING UPTAKE OF CRISIS RESOURCES
21
Table 3. Means and Standard Deviations for Full Sample.

Control Enhanced Crisis Response SSI

N Mean/ SD N Mean/ SD
Proportion Proportion

Hopelessnessa

Baseline 162 12.48 3.02 174 12.31 2.87

10-Minute Post 80 12.8 3.41 74 11.54 3.36

Resource Utilizationb

10-Minute Post 88 (34/88) 91 (71/91)

38.64% 78.02%

Perceived
Helpfulnessc

10-Minute Post 91 3.02 1.26 93 3.37 1.29

a
Range: 4-16. Higher scores indicate greater levels of hopelessness.
b
Users indicated whether or not they used the resources that were provided to them.
c
Range: 1-5. Higher scores indicate greater levels of perceived helpfulness.
 IMPROVING UPTAKE OF CRISIS RESOURCES
22
Table 4. Means and Standard Deviations for College-Aged Sample.

Control Enhanced Crisis Response SSI

N Mean/ SD N Mean/ SD
Proportion Proportion

Hopelessnessa

Baseline 24 13.33 3.19 25 12.59 2.09

10-Minute Post 26 13.23 3.47 28 10.53 3.28

Resource Utilizationb

10-Minute Post 26 (10/26) 28 (24/28)

38.46% 85.71%

Perceived
Helpfulnessc

10-Minute Post 26 3.27 1.19 28 3.68 1.25

a
Range: 4-16. Higher scores indicate greater levels of hopelessness.
b
Users indicated whether or not they used the resources that were provided to them.
c
Range: 1-5. Higher scores indicate greater levels of perceived helpfulness.
 IMPROVING UPTAKE OF CRISIS RESOURCES
23
Appendices

Appendix A. Screenshot of enhanced crisis response SSI.

 IMPROVING UPTAKE OF CRISIS RESOURCES
24
Appendix B. Screenshot of control condition.