Journal of American College Health

ISSN: (Print) (Online) Journal homepage: https://www.tandfonline.com/loi/vach20

College student engagement with mental health

apps: analysis of barriers to sustained use

Jennifer Melcher , Erica Camacho , Sarah Lagan & John Torous

To cite this article: Jennifer Melcher , Erica Camacho , Sarah Lagan & John Torous (2020):
College student engagement with mental health apps: analysis of barriers to sustained use, Journal
of American College Health, DOI: 10.1080/07448481.2020.1825225

To link to this article: https://doi.org/10.1080/07448481.2020.1825225

Published online: 13 Oct 2020.

Submit your article to this journal

View related articles

View Crossmark data

Full Terms & Conditions of access and use can be found at

https://www.tandfonline.com/action/journalInformation?journalCode=vach20
JOURNAL OF AMERICAN COLLEGE HEALTH
https://doi.org/10.1080/07448481.2020.1825225

MAJOR ARTICLE

College student engagement with mental health apps: analysis of barriers

to sustained use
Jennifer Melcher, BA, Erica Camacho, MS, Sarah Lagan, BA, and John Torous, MD
Division of Digital Psychiatry, Department of Psychiatry, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston,
Massachusetts, USA

ABSTRACT ARTICLE HISTORY

Objective: College students’demand for mental health resources is straining the services offered Received 5 May 2020
by colleges. While mobile apps demonstrate potential to help, students’ engagement with these Revised 28 July 2020
apps remains low. This study examines why college students show poor engagement with mental Accepted 13 September 2020
health apps and how apps may be adapted to suit this population. Participants: Participants
KEYWORDS
were a convenience sample of 100 college students. Methods: Qualitative data was gathered College students;
through individual online interviews concerningattitudes toward mental health apps, and quantita- engagement; mental health;
tive data was gathered through a survey about phone and app use. Results: Students were inter- mobile apps
ested in mental health apps. 53% haddownloaded an app at one point, but only 19% currently
used a mental health app. Stress and costdrove mental health app choices. Responses around
engagement centered on: Data privacy, user interface, credibility, and customization. Conclusions:
Students have specific wants for mental health apps including safety, simplicity, credibility, and
customizability.

Introduction mobile apps, as 96% of US adults between the ages of 18 to

An increasing number of college students are experiencing
mental health conditions and seeking help. While this trend
is not novel, it continues upward at an alarming rate.1,2 The
percentage of students with diagnosed mental health condi-
tions expanded from 21.9% to 35.5% between 2007 and
2017.2 The American College Health Association (ACHA)
reported that within the past 12 months, 45.1% of students
felt so depressed that it was difficult to function, and 13.3%
had seriously considered suicide.3 Additionally, levels of
stigma surrounding mental health have decreased, and more
students are willing to seek help.4 This combination of
increased rates of mental health problems, and decreased
stigma leaves counseling centers facing higher clinical loads
than ever before.5–7 Between 2007 and 2017, students who
had sought treatment within the past year increased from
19% to 34% with college counseling centers standing out as
the most common location for students to access mental
health services.2 Today, counseling centers are overwhelmed
with student demand and many are limiting care to try to
help as many students as possible.6,8 To provide more care,
some counseling centers are turning toward digital solutions,
particularly mobile apps.6
Mental health mobile apps have rapidly emerged as a
possible resource to provide widely accessible and affordable
care, especially on college campuses.9–11 Mobile apps are
generally inexpensive and can be easily disseminated.12
College students seem well suited to adopt mental health
29 already own a smartphone.13 Some college counseling
centers are already suggesting and implementing mobile
apps as mental health resources for students.6,14
While mobile apps show potential for use in college
counseling settings, obstacles persist. User engagement is
now recognized as a critical challenge for all health apps,
and especially mental health ones.15–20 One review of
100,000 users’ engagement trajectories across different app
studies, including one for depression, found the median
engagement was only 5.5 days in the first 12 weeks of each
study.18 Dropout rates from app clinic studies may be as
high as nearly 50% when accounting for risk of bias, por-
tending even lower rates in real world use.21 Indeed, a study
of popular mental health apps with over 10,000 downloads
found that user retention was just around 4% two weeks
after the initial download.17
Previous studies have attempted to understand students’
interest and engagement with mental health apps andshown
mixed results.22–24 In one survey of 741 college students,
only 7.3% had ever used a mental health app, and only
26.1% were open to the idea of using a mental health app.23
Of those who had used a mental health app, only 24% con-
tinued to use the app for four weeks or longer.23 In another
study, recruitment for mental health app use posed the larg-
est obstacle to the study, suggesting serious acceptability con-
cerns for mental health apps among college students.24 While
these studies highlight the challenges that mental health apps

CONTACT John Torous jtorous@bidmc.harvard.edu Division of Digital Psychiatry, Department of Psychiatry, Beth Israel Deaconess Medical Center, Harvard
Medical School, 330 Brookline Ave, Boston, MA, 02446, USA.
ß 2020 Taylor & Francis Group, LLC
2 J. MELCHER ET AL.
targeting college students face, understanding of the core rea-
sons for low engagement and utilization remain unknown.
Current research offers potential solutions to make apps
more appealing and useful, though these findings are often
based on small samples and not easily generalizable to col-
lege students.25–28 App ease of use may be a primary driver
of engagement although that definition will vary for each
individual depending on their engagement style.29,30 But a
recent review of digital interventions for college students
reported that 51% of studies did not present any usability
outcomes, and only 4% examined implementation of these
interventions on college campuses.29 To date there is a
dearth of data about what college students want in mental
health apps and what factor would improve engagement.
This study explores college students’ attitudes and opinions
toward mental health apps and their perceptions of the limi-
tations of existing mental health apps in order to discover
how to engage this population. Given previous research in
this area, we anticipated that college students would be open
to engaging with mental health apps given the prevalence of
mobile technology among this age range. Additionally, we
expected that the primary reason for lack of user engage-
ment with mental health apps would be due to the user
interface of current app offerings, particularly the usability
and appearance of apps.
Method
100 U.S. college students (70% female) aged 18
25 com-
pleted a survey and participated in a 20-minute conversation
with a researcher about their attitudes and opinions toward
mental health mobile apps. Participants were recruited
through online posts on college Reddit pages and through
word of mouth. Interested students contacted the research
team for more information and to verify that they were cur-
rently enrolled in a college through the use of their college
email address. Participants signed up for a time slot and
agreed to either come inperson or connect over a video con-
ferencing platform. The day before their participation, par-
ticipants were emailed a survey about their general phone
use and attitudes toward mental health apps (Appendix A).
Then, during their time slot, a researcher asked a variety of
questions related to mental health app use. Researchers fol-
lowed a script of questions about app use and experience
with and attitudes toward mental health apps (Appendix B),
but participants were encouraged to elaborate on any topic
or personal experience of mental health app use. At the end
of each session, participants were sent a $20 Amazon gift card
to their email for their participation, regardless of answers
provided or level or participation. During each session, the
researcher took typed notes including basic responses and atti-
tudes as well as direct quotes that were strong representations
of a participant’s attitude. The researcher revised and
improved these notes following each session. After all the ses-
sions were completed, the three researchers who conducted
these sessions met to compile notes and review them to agree
on major categories. Notes were categorized and then
reviewed for emergent themes. Finally, the notes were coded
by the researchers according to these themes. This protocol
was approved by Beth Israel Deaconess Medical Center’s
Institutional Review Board (IRB) 2019P000985.
Results
Survey results revealed that all (n ¼ 100) participants had
smartphones and downloaded apps on their phones.
Additionally, students spent an average of 3.7 hours on their
phone daily. While 53% of participants had downloaded a
mental health app at some point, only 19% of participants
currently used any mental health apps. Responses outlined
student comfort level with different mental health app fea-
tures (Figure 1). Students also noted the top benefits as well as
top concerns they could see in mental health apps.Students
saw the potential to share information with a clinician and the
time savings of apps as favorable with 53% reporting those
two features as the top benefits of mental health apps. In terms
of concerns, privacy was a top concern for mental health apps
among 74% of students. This was followed by the accuracy of
recommendations from an app with60% of studentslisting this
as a top concern. Finally, 54% of students listed the cost of
apps as among their top concerns (Tables 1 and 2).
Reasons for adopting mental health apps
Qualitative responsesfrom conversations with students
revealed student reasons for adopting mental health apps as
well as issues around engagement with mental health apps.
Students cited that reasons for adopting the mental health
apps they used centered around school stress and cost.
School stress
Students expressed that stress brought on by their academic
course load as well as the life transition to college spurred
their interest in mental health apps with 30% mentioning
these as their reason for seeking out apps. Coping with
stress and getting to sleep were the two principal reasons
that these apps were used during stressful times. Students
also mentioned that these apps were primarily used during
acute periods of stress and that their use tended to drop off
some as stress ebbed.
Cost
The cost of mental health mobile apps was a major factor in
students’ decisions to download a mental health mobile app.
51% of students mentioned that cost was a driving factor
behind their app choice. Many of these students said that
they looked first to see if an app was free before evaluating
whether they should download it. However, students were
aware that even free apps often had in-app purchases or a
required subscription to access content within the app.
Participants felt particularly drawn to free apps because they
had little to no income as full time students and had to pri-
oritize tuition payments over other non-essential purchases.
 JOURNAL OF AMERICAN COLLEGE HEALTH 3

Figure 1. College student comfort levels for various mental health app features.

Table 1. Illustrative quotes of student decisions to download mental health apps.

School Stress  “When I was younger, I was against meditation, but after the trials of college I was willing to do anything to take care of myself.”
 “Last year I had a rough freshman year so I would use it more then. Like before I went to sleep or when
I was stressed out or when I first woke up, but this year I’m less stressed and anxious so I use it less.”
 “The app was useful during my stressful transition. I used it during my freshman year to fall asleep and to do breathing exercises.”
Cost  “I used it on a free trial basis last year but once it was free (from my college), I jumped onto it again.”
 “I only have the free version now but once I’m out of college and have a job I want to get the premium version.”
 “The free version practically does nothing, but the pro version charges you per month. I’m sure there are people that
are willing to do it, but the majority of students aren’t going to pay 20 dollars a month.”

Table 2. Illustrative quotes of qualitative themes around students’ app engagement.

Data Privacy  “I don’t read the privacy policy, but I wish I did”
 “I try to be aware. I think it’s a bit like the Wild West out there right now. I wish there was more awareness of data privacy.”
 “I think it should be made clear exactly how the app is using the data –I’m okay with it as long as the app is transparent.”
User Interface  “In other apps, the Fitbit one, I really like the interface because it’s so clean and it’s very easy to use. There was a point
where I didn’t use the watch, just the app, but logging activity was so simple and so clean. When apps add a bunch
of extra stuff, it’s too much. They get very complicated.”
 “The better it looks, the easier it is to use, the more colors, the more likely I am to download it.”
 “I want something simplistic. It would be fast and user friendly”
Credibility and Utility  “I’m less comfortable if the app itself is providing the content. I would be more convinced of credibility by links and references.”
 “What I dislike is so many appear to be legit resources, but it would be better to have an app that has some research
base on what they’re trying to sell you on.”
 “If I don’t gain anything new then I don’t see the point in using it.”
 “I would trust an app supported by my university more than a random app I found online”
Customization  “I have a hard time trusting all the different apps. How do I know that will help me specifically? I know it helps a ton of people but for
me it’s different. How do I know that’ll help me?”
 “You can use the features that apply to you, but it should give you the option to opt out.”

Engagement with mental health apps
Student responses around engagement with apps fell into a
few main themes: Data privacy, user interface, credibility,
and customization.
Data privacy
Participants expressed varying levels of awareness surround-
ing their data privacy with 59% mentioning at least some
awareness and caution around their personal data. Some stu-
dents remained unaware of how their data may be collected
by apps, while a few were diligent about protecting their data
privacy. Above all, the majority of participants expressed that
they wished they were more careful about protecting their
data, but often encountered barriers such as privacy policies
that were difficult to understand, or apps with poor privacy
policies that they needed to download anyway for either
school or communication purposes. Participants often linked
their understanding of the app’s privacy policy to their trust
of an app and noted that trusted apps or apps they had a
“connection” with were more likely to have their buy-in.
User interface
Participants felt strongly that a good user interface was one
of the deciding factors in determining which apps they
4 J. MELCHER ET AL.
remained engaged with. The Apple app store or Google Play
store shows screenshots of the app in use and many students
cited these screenshots as their deciding factor on whether
or not to download an app. Once downloaded, participants
wanted apps to be easy to navigate and involve short inter-
actions. That is, responding to app prompts and notifica-
tions as well as inputting information into the app should
be quick and easy. Additionally, apps with too many fea-
tures become crowded and messy. A simple and clean
appearance was most appealing to users.
Credibility and utility
Participants noted that mental health apps could provide
very important information to users, but that that informa-
tion would need to come from a credible source so that
users could expect to learn reliable information. Skepticism
about the legitimacy of the apps or the information on the
apps was frequently mentioned. Some students noted that
because anyone can make an app, knowing what a good
resource is can be challenging, and apps with supporting
research would be more credible.
Customization
The ability for a user to customize and personalize an app
surfaced as an important component for college student
engagement with 28% of students listing this as a feature
they want to see in a mental health app. Participants
expressed that although they would be glad to have a variety
of features, they would only be using the ones that they
needed or wanted and did not want to be bothered by the
others. The ability to opt out of features was important.
Further, many participants noted that one particularly
important aspect of customization is the ability to schedule
notifications. While push notifications can serve as helpful
reminders of activities that need to be completed within the
app, participants wanted to have these notifications adjusted
for their own daily schedule. They noted that when notifica-
tions interrupted them when they were otherwise occupied,
they were unlikely to go into the app and more likely to
eventually drop off of the app if that pattern continued.
Discussion
Quantitative results showed that students were interested in
using mental health apps with 53% having downloaded an
app at one point. However, only 19% of students claimed
that they currently had an app for their mental health
downloaded on their phone that they still visited. With even
fewer reporting meaningful use of apps, it is clear that
bridging the gap between potential, students’ interest, and
current low levels of use must focus on increas-
ing engagement.
Student interest and initial choice of mental health apps
was prompted by school-induced stress and their limited
budgets. Students remarked that academic or college transi-
tion stress is what often sparked their initial interest in men-
tal health apps. In terms of the apps that they chose to try,
students expressed that price was the first aspect they looked
at when deciding whether to download an app. Quantitative
results supported this with 54% of students reporting that
cost was among their top concerns for mental health apps.
Most students were only willing to use free apps or the free
trials of apps that required paid subscriptions. Tuition pay-
ments and living expenses were financial burdens that these
students prioritized above all other purchases, even for men-
tal health. In a few cases, students were willing to pay a few
dollars for an app if it had great reviews on the app store,
or if it had been recommended to them.
College student’s engagement with mental health apps is
multidimensional and influenced by at least four themes:
Data privacy, user interface, credibility and utility, and cus-
tomization. While barriers to app engagement among college
students in our study appears to mirror that of different
populations, interviews with these college students provide
insights into how apps can be customized in order to
increase engagement.10,11,15 Participants cited data privacy as
an important factor in their app use. This was comple-
mented by quantitative findings which show that 74% of
students cited privacy as among their top concerns for men-
tal health apps. These findings align with previous research
among both clinicians and college students who expressed
that apprehension and lack of knowledge about data privacy
to be a concern with mental health apps.10,11 Because mental
health apps often capture sensitive information, students
were especially frustrated with incomprehensible or compli-
cated privacy policies. Previous research shows that mental
health app privacy policies often lack essential privacy pro-
tections.31,32 Students’ awareness of the lack of privacy
among many apps also complements quantitative survey
responses, specifically concerning discomfort over sharing
location and social information with apps. 69% of students
were either uncomfortable or very uncomfortable sharing
their social information, and 58% were uncomfortable or
very uncomfortable sharing their location with apps. While
students want privacy protections, they are not likely to read
a full privacy policy and want other reassurances that they
could use the app safely. Some students suggested bullet-
point notes summarizing the privacy policy within the app
or other methods to convey that the app would be trust-
worthy. Trust and connection with apps were imperative for
students to maintain long-term use with apps, while apps
that felt “sketchy” or unsafe were likely to be dropped.
Next, college students asserted that a good user interface
in an app was vital for sustained engagement. Previous
research on college student engagement with mental health
apps found that discontinuation of app use was frequently
attributed to poor user experiences.14,33 In this study, stu-
dents further elaborated on what a good user interface
would entail, primarily advising that apps be kept simple.
Students were quick to cite apps they loved that had a sim-
ple and clean look such as the Fitbit app and the Apple
Health app. Students also noted that they often made judge-
ments about the user interface of an app before even down-
loading it by scrolling through the screenshots of the app on
the App store or Google Play store. However, more than

just a clean design, students also wanted apps to be easy to
navigate and keep only the necessary features. Apps that
were crowded and overly complicated were quickly aban-
doned. Survey results also supported student comments on
the need for simplicity and quick interactions with apps as
44% of students listed time savings was a benefit they would
see in mental health app use. College students struggle to
engage with apps that lack a nice user interface and simple
features. Clean designs straightforward functionality may be
the key to unlocking sustained student interest in mental
health mobile apps.
Additionally, college students want mental health apps to
have credible and useful information. College students today
are well aware that they live in a world filled with false
information.34 They are mindful that today anyone can put
an app on the app store and make it look legitimate.35
Survey results demonstrate that students are cognizant of
false information with 60% of respondents reporting that
accuracy of recommendations from mental health apps was
among their top concerns for using these apps. Students
noted that having apps come from credible sources and
have a clear foundation in research could increase their trust
and engagement with mental health apps. Participants men-
tioned that some additional ways to increase credibility of
apps would be to have their university choose and suggest
apps, or to have links out to credible resources such as
scholarly journals or reliable websites. Expanding on cred-
ibility, students noted that information in these apps should
also be useful. They were frustrated by mental health apps
that pushed the same information, tips, or skills repeatedly.
Once they felt like they were not gaining or not learning
anything new, students reported often deleted their mental
health apps. Thus, app engagement within the college stu-
dent population could be improved by ensuring that stu-
dents were accessing credible and useful information
through mental health apps.
Finally, even with apps, students still want personalized
care. This means that for an app to engage college students,
it should be customizable. Giving users the autonomy to opt
in or opt out of features can help them access the care they
need as well as avoid clutter or unnecessary features. One
component of customization mentioned by many students
was the ability to customize the timing of push notifications.
Students emphasized that push notifications from an app are
only helpful when they come at the right time, and that this
may be different for each person. A student who receives a
push notification while they are busy is more likely to swipe
it away and forget about it. However, if a notification is
received when a student has free time, they are more likely to
enter the app. Customizing app features and timing of app
notifications could help students build the habit of actually
using the app and contribute to sustained use.
Mental health apps may serve as an important tool for
colleges to increase the scope of their mental health resour-
ces for students. These findings implicate the importance of
research on acceptability of digital interventions within the
college population. Colleges considering offering an app to
all of its students may consider assessing apps’ usability and
JOURNAL OF AMERICAN COLLEGE HEALTH 5
acceptability among students prior to dissemination. Because
students expressed more willingness to use an app endorsed
by their institution, future research may also explore how a
college-customized app is received by students in compari-
son to the app without such backing. Finally, research with
mental health apps should include feedback from partici-
pants to further understand the evolving requirements for
mental health apps and better bridge the gap between app
research and app design.
Limitations
Like all studies, ours has several limitations. Different stu-
dents had different levels of experience with mental health
apps and results presented are an aggregate. Our broad
inclusion criteria allowed any student to partake and did not
take into account a diagnosis. Sample bias in partaking in
online research must also be considered although we also
offered in-person meetings for students in an effort to min-
imize this bias. Additionally, research advertisements men-
tioning mental health apps may have interested those
individuals who were already familiar with mental health
apps and this could have skewed results. Finally, this group
of participants was comprised of 70% female students, while
the national population of college students is 57% female.36
Conclusion
Mental health apps show potential for use by college stu-
dents to increase access to services and on-demand care.
However, like other populations, college students show poor
engagement thus far with mental health apps. Mental health
apps designed for use by all populations do not fit the needs
of college students and lead to drop-offs in use and engage-
ment. These survey and interview results reveal students
have specific wants and needs for mental health mobile apps
including wanted apps to be safe and secure, simple to navi-
gate, credible and informative, and customizable.
Conflict of interest disclosure
The authors have no conflicts of interest to report. JT reports unrelated
research support from Otsuka. The authors confirm that the research
presented in this article met the ethical guidelines, including adherence
to the legal requirements, of the United States and received approval
from the Beth Israel Deaconess Medical Center Institutional
Review Board.
Funding
This work was supported by a philanthropic gift from Jeremy Wertheimer.
References
1. Duffy ME, Twenge JM, Joiner TE. Trends in mood and anxiety
symptoms and suicide-related outcomes among U.S. undergradu-
ates, 2007-2018: Evidence From Two National Surveys. J Adolesc
Health. 2019;65(5):590–598. doi:10.1016/j.jadohealth.2019.04.033.
2. Lipson SK, Lattie EG, Eisenberg D. Increased rates of mental
health service utilization by U.S. college students: 10-year
6 J. MELCHER ET AL.
population-level trends (2007-2017). Psychiatr Serv. 2019;70(1):
60–63. doi:10.1176/appi.ps.201800332.
3. American college health association national college health
assessment spring 2019. American College Health Association;
2019. https://www.acha.org/documents/ncha/NCHA-II_SPRING_
2019_US_REFERENCE_GROUP_EXECUTIVE_SUMMARY.pdf.
4. Oswalt SB, Lederer AM, Chestnut-Steich K, Day C, Halbritter A,
Ortiz D. Trends in college students’ mental health diagnoses and
utilization of services, 2009-2015. J Am Coll Health. 2020;68(1):
41–51. doi:10.1080/07448481.2018.1515748.
5. Xiao H, Carney DM, Youn SJ, et al. Are we in crisis? National
mental health and treatment trends in college counseling centers.
Psychol Serv. 2017;14(4):407–415. doi:10.1037/ser0000130.
6. LeViness P, Bershad C, Gorman K, Braun L, Murray T. The
association for university and college counseling center directors:
Annual survey 2018. https://www.aucccd.org/assets/documents/
Survey/2018%20AUCCCD%20Survey-Public-June%2012-FINAL.pdf.
7. The Healthy Minds Study 2018-2019 Data Report. https://health-
ymindsnetwork.org/wp-content/uploads/2019/09/HMS_national-
2018-19.pdf.
8. Center for Collegiate Mental Health. 2019 Annual Report.
January 2020. https://ccmh.psu.edu/files/2020/01/2019-CCMH-
Annual-Report.pdf.
9. Eisenberg D, Golberstein E, Hunt JB. Mental health and aca-
demic success in college. BE J Econ Anal Policy. 2009;9(1):40.
doi:10.2202/1935-1682.2191.
10. Lattie EG, Lipson SK, Eisenberg D. Technology and College
Student Mental Health: Challenges and Opportunities. Front
Psychiatry. 2019;10:246. doi:10.3389/fpsyt.2019.00246.
11. Lattie EG, Nicholas J, Knapp AA, Skerl JJ, Kaiser SM, Mohr DC.
Opportunities for and Tensions Surrounding the Use of
Technology-Enabled Mental Health Services in Community
Mental Health Care. Adm Policy Ment Health. 2020;47(1):
138–149. doi:10.1007/s10488-019-00979-2.
12. Donker T, Blankers M, Hedman E, Ljotsson B, Petrie K,
Christensen H. Economic evaluations of internet interventions
for mental health: a systematic review. Psychol Med. 2015;45(16):
3357–3376. doi:10.1017/S0033291715001427.
13. Smartphone ownership and internet ssage continues to climb in
emerging economies. Pew Res Cent Glob Attitudes Proj.
February 2016. https://www.pewresearch.org/global/2016/02/22/
smartphone-ownership-and-internet-usage-continues-to-climb-in-
emerging-economies/. Accessed April 20, 2020.
14. Johnson KF, Kalkbrenner MT. The utilization of technological
innovations to support college student mental health: Mobile
health communication. J Technol Hum Serv. 2017;35(4):314–339.
doi:10.1080/15228835.2017.1368428.
15. Torous J, Nicholas J, Larsen ME, Firth J, Christensen H. Clinical
Review of User Engagement With Mental Health Smartphone
Apps: Evidence, Theory and Improvements. Evid Based Ment
Health. 2018;21(3):116–119. doi:10.1136/eb-2018-102891.
16. 24% of users abandon an app after one use. http://info.localytics.
com/blog/24-of-users-abandon-an-app-after-one-use. Accessed
April 20, 2020.
17. Baumel A, Muench F, Edan S, Kane JM. Objective user engage-
ment with mental health apps: Systematic search and panel-
based usage analysis. J Med Internet Res. 2019;21(9):e14567. doi:
10.2196/14567.
18. Pratap A, Neto EC, Snyder P, et al. Indicators of retention in
remote digital health studies: a cross-study evaluation of 100,000
participants. Npj Digit Med. 2020;3(1):1–10. doi:10.1038/s41746-
020-0224-8.
19. Torous J, Staples P, Slaters L, et al. Characterizing smartphone
engagement for schizophrenia: Results of a naturalist mobile
health study. Clin Schizophr Relat Psychoses. August 2017; doi:10.
3371/CSRP.JTPS.071317 [Online ahead of print].
20. Flett JAM, Hayne H, Riordan BC, Thompson LM, Conner TS.
Mobile mindfulness meditation: a randomised controlled trial of
the effect of two popular apps on mental health. Mindfulness.
2019;10(5):863–876. doi:10.1007/s12671-018-1050-9.
21. Torous J, Lipschitz J, Ng M, Firth J. Dropout rates in clinical tri-
als of smartphone apps for depressive symptoms: A systematic
review and meta-analysis. J Affect Disord. 2020;263:413–419. doi:
10.1016/j.jad.2019.11.167.
22. Palmer KM, Henderson SG. College students’ attitudes about
mental health-related content in mobile apps. J Technol Behav
Sci. 2019;4(4):381–389. doi:10.1007/s41347-019-00102-0.
23. Kern A, Hong V, Song J, Lipson SK, Eisenberg D. Mental health
apps in a college setting: openness, usage, and attitudes. Mhealth.
2018;4(:20doi:10.21037/mhealth.2018.06.01.
24. Levin ME, Hicks ET, Krafft J. Pilot evaluation of the stop,
breathe & think mindfulness app for student clients on a college
counseling center waitlist. J Am Coll Health J ACH. March 2020;:
1–9. doi:10.1080/07448481.2020.1728281 [Online ahead of print].
25. Byron P. ‘Apps are cool but generally pretty pointless’:
LGBTIQ þ young people’s mental health app ambivalence - Paul
Byron, 2019. Media Int Aust. April 2019. http://journals.sagepub.
com/doi/10.1177/1329878X19844034?icid=int.sj-full-text.similar-
articles.2. Accessed April 22, 2020.
26. Montague AE, Varcin KJ, Parker AG. Putting technology into youth
mental health practice: Young people’s perspectives. SAGE Open. April
2015. http://journals.sagepub.com/doi/10.1177/2158244015581019?icid=
int.sj-related-articles.similar-articles.4. Accessed April 22, 2020.
27. Pywell J, Vijaykumar S, Dodd A, Coventry L. Barriers to older
adults’ uptake of mobile-based mental health interventions. Digit
Health. 2020;6:2055207620905422. February http://journals.sage-
pub.com/doi/10.1177/2055207620905422?icid=int.sj-related-articles.
similar-articles.6. Accessed April 22, 2020. doi:10.1177/
2055207620905422.
28. Kenny R, Dooley B, Fitzgerald A. Developing mental health
mobile apps: Exploring adolescents’ perspectives. Health
Informatics J. 2016;22(2):265–275. doi:10.1177/1460458214555041.
29. Haan M, Lugtig P, Toepoel V. Can we predict device use? An
investigation into mobile device use in surveys. Int J Soc Res
Methodol. 2019;22(5):517–531. doi:10.1080/13645579.2019.1593340.
30. Fleming T, Merry S, Stasiak K, et al. The importance of user seg-
mentation for designing digital therapy for adolescent mental
health: Findings from scoping processes. JMIR Ment Health.
2019;6(5):e12656doi:10.2196/12656.
31. Parker L, Halter V, Karliychuk T, Grundy Q. How private is
your mental health app data? An empirical study of mental
health app privacy policies and practices. Int J Law Psychiatry.
2019;64:198–204. doi:10.1016/j.ijlp.2019.04.002.
32. Robillard JM, Feng TL, Sporn AB, et al. Availability, readability,
and content of privacy policies and terms of agreements of men-
tal health apps. Internet Interv. 2019;17:100243doi:10.1016/j.
invent.2019.100243.
33. Alqahtani F, Orji R. Insights from user reviews to improve men-
tal health apps. Health Informatics J. 2020;26(3):2042–2066.
January doi:10.1177/1460458219896492.
34. Leeder C. How college students evaluate and share “fake news”
stories. Libr Inf Sci Res. 2019;41(3):100967. doi:10.1016/j.lisr.
2019.100967.
35. Larsen ME, Huckvale K, Nicholas J, et al. Using science to sell
apps: Evaluation of mental health app store quality claims. Npj
Digit Med. 2019;2(1):1–6. doi:10.1038/s41746-019-0093-1.
36. Enrollment. 2019. National Center for Education Statistics;
National Center for Education Statistics. https://nces.ed.gov/fast-
facts/display.asp?id=98.
 JOURNAL OF AMERICAN COLLEGE HEALTH 7

Appendix A Coaching for Healthy Living (exercise, sleep, diet)

(Very uncomfortable) 1 2 3 4 5 (Very comfortable)
Survey
What is your age in years? Mindfulness or Therapy Exercises
___________________ (Very uncomfortable) 1 2 3 4 5 (Very comfortable)
Please choose your gender:
Communicating with My Clinician About My Mental Health
 Female (Very uncomfortable) 1 2 3 4 5 (Very comfortable)
 Male
 Prefer not to say Choose up to 3 top concerns you would have about mental health apps:
 Other
 Privacy
Do you currently own any type of phone?  Accuracy of recommendations from app
 Hard to use
 Yes  Sharing information with clinician
 No  Cost
 Time
If yes, is your phone a smartphone?
 Hard to set up
 Yes  Other ________
 No
Choose up to 3 top benefits you may see in mental health apps:
 I don’t own a phone
 Privacy
If yes, what type of smartphone?
 Accuracy of recommendations from app
___________________
 Hard to use
Do you use your phone for text messaging?  Sharing information with clinician
 Cost
 Yes  Time
 No  Hard to set up
Do you download apps on your phone?  Other ________
What is your average daily “screen time”? Look in phone settings if
 Yes
 No your phone offers this – write N/A if not.
________________________
Have you ever downloaded an app for your mental health?

 Yes
 No Appendix B
Do you currently use any apps for your mental health? Interview script
 Yes
 No
1. What gets you to download an app?
How comfortable or uncomfortable would you feel about a mental 2. What stops you from downloading an app?
health app gathering or sending the following data from your smart- 3. How important is data privacy to you when you download apps?
phone in the context of your care? 4. Have you ever downloaded an app for your mental health
or wellness?
Appointment Reminders  If you have downloaded an app for your mental health or
(Very uncomfortable) 1 2 3 4 5 (Very comfortable) wellness, what was your experience like with that app?
5. What is your perception of the available mental health apps?
Medication Reminders 6. What would your ideal mental health app help you do, and do for
you? What would a mental health app customized to you include?
(Very uncomfortable) 1 2 3 4 5 (Very comfortable) A lot of the survey asks about your comfort level with different infor-
mation gathering from apps; do you normally allow apps access to
Symptom Surveys other data on your phone? (such as location, camera, contacts access)
(Very uncomfortable) 1 2 3 4 5 (Very comfortable) 7. Do you have any concerns about mental health apps that were
not mentioned on this survey or that we haven’t talked about?
Your Location 8. How would you feel about using an app that a counselor sug-
(Very uncomfortable) 1 2 3 4 5 (Very comfortable) gested to you and then checking in with them about the infor-
mation you put in between sessions?
Your Social Information (call and text logs without any 9. How often would you be willing to fill out surveys - like short
phone numbers or content) mood surveys, or quick 1-2-minute surveys?
10. Do you wear any smart watches or other wearable technology?
(Very uncomfortable) 1 2 3 4 5 (Very comfortable) 11. What is your average daily screen time?