Scalable Mental Health Analysis in the Clinical Whitespace via Natural
Language Processing
Glen Coppersmith1 , Casey Hilland1 , Ophir Frieder2 , Ryan Leary1
Abstract— Our increasingly digital life provides a wealth of
data about our behavior, beliefs, mood, and well-being. This
data provides some insight into the lives of patients outside the
healthcare setting, and in aggregate can be insightful for the
person’s mental health and emotional crisis. Here, we introduce
this community to some of the recent advancement in using
natural language processing and machine learning to provide
insight into mental health of both individuals and populations.
We advocate using these linguistic signals as a supplement to
those that are collected in the health care system, filling in some
of the so-called “whitespace” between visits.
I. INTRODUCTION
Mental illness causes a tremendous strain on the human
race, emotionally, cognitively, and financially – and we un-
derstand relatively little about it compared to many physical
illnesses and ailments. $3.5 trillion is estimated global cost
of mental illness [1]. 1 in 5 Americans (19%) experience
a mental illness each year [2], between 4% and 8% have
thoughts of suicide and 1% die by suicide; 42000 died by
suicide last year in the United States [3], [4]. In the U.S., an
estimated 3500 adults aged 35-64 attempt suicide per month.
This represents a 28+% increase since 1999 and reaches
across all geographic regions of the United States [5].
At the crux of many of these issues is emotional crisis
– e.g., a suicide attempt or the loss of sobriety. These are
behaviors that have a high emotional and financial toll,
and are ill understood. Much of what we do know about
emotional crisis and mental health comes from interactions
with the healthcare system, but as Figure 1 illustrates, that is
a limited view of a patient’s life. As our lives are increasingly
digitally mediated (by smartphones, calendars, Internet of
Things devices, etc.), we are increasing the signals relevant
to our mental and behavioral health that can be analyzed.
We focus on a single piece of this signal here, language, but
this is one of many relevant signals that comprise our digital
life; Other examples include geolocation, activity, temporal
patterns of device usage. The primary purpose of this paper is
to bring work relevant to the prevention, personalization, and
scalable measurement of the whitespace to this community.
This research has reached a level of maturity that it may
be useful for filling a gap in health care monitoring, and
a critical gap in mental healthcare. Qntfy is a company
devoted to the research and development of analysis of
this whitespace data. The team’s interdisciplinary approach
depends on their diversity of skills from psychology, data
science, mathematics and computer science.
1 Qntfy {glen,casey,ryan}@qntfy.com
2 Georgetown University ophir@ir.cs.georgetown.edu
978-1-5090-4179-4/17/$31.00 ©2017 IEEE 393
Fig. 1. An example person’s interactions with the health care system (red
hashes), and Facebook posts (blue) over four years (x-axis). This user is
from the University of Pennsylvania’s data set linking social media data
and medical records [6], [7]. This paper discusses some of the scientific
advances that may provide clinically useful information from the analysis
of digital life data (e.g., Facebook) from the whitespace between health care
visits.
Whitespace information provides a lens through which
we can analyze psychological phenomena like emotional
crisis, suicide attempts, and drug relapse. This is particularly
powerful for mental health, which is, by definition, when the
user interacts with the rest of the world, and thus notoriously
difficult to assess in a lab or formal health care setting. In
particular, linguistic signals from the whitespace are powerful
because they are objectively recorded in the moment in a
manner conducive to automated analysis, rather than recalled
(with imprecision and bias) after the event. Encoded in
these signals are directly observable psychologically relevant
phenomena (e.g., social media interactions with friends),
indirectly observable phenomena (e.g., social events that
the user may or may not have attended), and further latent
signals (subconscious language choices). These signals can
empirically inform caregivers, clinicians, and patients about
wellness outside of clinical interactions. Thus, they are likely
best used as a supplement to existing health care data
rather than to the exclusion of trained professionals and
clinical measurements. The application of natural language
processing (NLP) to data in the whitespace between clinical
encounters has progressed sufficiently to warrant design
discussions around integration into existing systems of care.
To illustrate some of the power of even straightforward
analytics in this space, we also present some novel analysis
using some of these psychological classifiers on the com-
munications of a population and demonstrate correlations
between language usage and real world events affecting that
population.
A. Linguistic Signals
Linguistics provide an interesting lens through which one
can examine and research mental health. Early studies fo-
cused on the writings of those with mental health conditions,
but with the emergence of computational linguistics and cor-
pora analysis, this capability was accelerated. The Linguistic
Inquiry Word Count (LIWC) is a psychometrically validated
lexicon, which associates words with given psychological
concepts [8]. Some of these associations are obvious – the
use of words like “sister” or “father” evoke a psychological
concept of family, thus any text analyzed with LIWC
containing these words would have the family LIWC
concept associated with it. Some of the most interesting
signals from this sort of linguistic analysis are non-obvious.
For example, people who are challenged with depression tend
to use first person pronouns (e.g., “I”) more frequently than
matched controls [9], [10], [11].
With the uptake of social media, the data available for this
manner of analysis has also dramatically increased. Recently,
a community of interest around this work at the intersection
of computational linguistics and clinical psychology has
produced a wealth of interesting analytical and qualitative
papers covering this space [12], [13], [14]. For brevity, we
refer the interested reader to the archives of these conferences
at http://clpsych.org.
B. Population-level analysis
Much of the potential power of this sort of analysis is
its scalability and affordability, which makes the analysis of
large populations possible. There are two general approaches
for this population-level analysis:
• (1) Maintaining models for each individual in a pop-
ulation and using aggregates of these models to make
inferences about the population.
• (2) Aggregate the data for the population, then score
this data once in aggregate.
Illustrative examples of (1) can be seen for looking at
estimating suicidal ideation as compared to observed sui-
cide rates [15] and post traumatic stress rates [16], both
of geographic populations in the United States. Models to
estimate which users were experiencing suicidal ideation (or
post traumatic stress, respectively) from the language of their
social media posts were created and deployed against a large
geographic sample of Twitter data. The former segmented
the population to look at demographics well-represented in
Twitter (particularly women aged 14-24), while the latter
made no attempt to control for demographic differences
induced by studying Twitter users. Despite this, both provide
evidence indicating known trends and demonstrate face-valid
results.
There has been a wide array of progress in the sort of
algorithms and models necessary to support the analysis like
(1) from linguistic signals. In the past few years, research
examining the language usage on social media from users
with a wide array of mental health conditions supports
this approach. Research has been published on models for
suicidal ideation and risk, [17], [15], [18], [19], [20], [21]
postpartum depression [22], clinical depression [23], anxiety,
attention deficit hyperactivity (ADHD), bipolar, borderline
personality, eating disorders, obsessive compulsive (OCD),
post traumatic stress (PTSD), schizophrenia, and seasonal
affective disorder [11]. A number of groups have also
demonstrated the power of incorporating other information
394
(e.g., temporal information) in with the linguistic signals for
more robust classifiers [24] and a deeper understanding of
mental health phenomena (e.g., related to onset, variability
over time, or variability leading up to an emotional crisis)
[25].
The second (2) approach to population level analysis has
primarily been used to study the populations defined by
geography. For example, the psychological classifiers lan-
guage of Twitter users has been shown to be more predictive
of county-level risk of heart disease than the traditionally
examined set of demographics (e.g., age, race, ethnicity,
gender) for the condition [26]. Interestingly, even including
these demographics with the language data provided no sta-
tistically significant benefit to the accuracy of risk prediction.
Similarly, the psychological well-being of counties has also
been assessed, showing improved accuracy of predicting life
satisfaction over standard demographic and socio-economic
data [27].
II. DATA
Frequently, the population is defined by geography, likely
both for its convenience (Twitter makes it easy to collect ge-
ographic information) and its intuitive interpretations (states
have different customs, regulations, and laws that influence
the behavior of interest). We apply similar approaches to
a different kind of population – a company. Many large
enterprises are concerned with the well-being of their em-
ployees, especially those aspects that effect productivity and
burnout. Given the progress of computational linguistics to
quantify mental health signals, analysis of intracompany
communications may provide some of this insight. We have
been asked to analyze the internal chat, communications, and
filesharing for a few groups, and show one such analysis in
Figure 2.
III. METHODS
We use a textual emotion classifier from [17] and a
sentiment classifier from [28] to analyze this internal com-
pany data. Each of these classifiers produces a probability
distribution over the possible labels for each message (e.g.,
‘positive’, ‘negative’, or ‘neutral’ sentiment). This allows
us to aggregate all the messages sent on a given day by
summing the probabilities associated with each outcome.
We focus on the emotion- and sentiment-bearing portions
of the data, (i.e., ignoring those labeled with ‘no emotion’
or ‘neutral sentiment’). By excluding these from the anal-
ysis and focusing on relative proportions of messages with
emotions/sentiment we factor out effects and fluctuations due
to volume of communication activity alone. This, in effect,
provides a barometer for the mix of emotions and sentiments
expressed on a given day by the company.
IV. RESULTS
Many of these emotion and sentiment plots over time
produce interesting results, consistent with what would be
expected. Here, we present in detail a poignant anecdote that
illustrates the possible insight from population-level analysis.
The composition of emotions and sentiments is calculated for
each day of communications for the company (regardless of
author). A rolling mean over one week is plotted in each
of the plots in Figure 2. This company exhibits increases
in ‘joy’ around the major holidays and after the first major
software release on a new project, (top of Figure 2) and
exhibit increases in negative sentiment leading up to each
major deadline (bottom).
V. DISCUSSION
Examining the population-level analysis of a company’s
data with straightforward NLP techniques yields some im-
mediately interpretable and face-valid results. This company
appears to have more posts with negative sentiment leading
up to a big deliverable or deployment. This does not explain
all of the peaks in the negative sentiment in Figure 2, it is
clear that none of the deliverables come without an increase
in negative sentiment in communications (likely related to
stress). The explainable peaks in joy around national holidays
and the first deliverable on a particular project are noticeable
and intuitively explainable – the inherent reward earned
by reaching a professional goal that results in a functional
product for the first time. Subsequent releases and updates
don’t seem to have the same increases in joy, though.
This analysis was intended to be illustrative of the sort of
population-level analysis now possible. While the classifiers
used here were for emotion and sentiment, many of the
mental health classifiers mentioned in the introduction are
equally conducive to these sorts of analyses. They could
provide information about population-level mental health at
a previously impractical temporal granularity, for example.
VI. E THICS
The ethics of using such aggregated public data to inform
science have begun to be explored. Generally, the ethics
and public opinion seem to support using public social
media data to further mental health research [29], [30].
One natural extension of the individual-level analysis is
for screening applications, though there are still questions
remaining about the most ethical way to proceed. At its
crux is a tension between saving lives of those at risk and
violating their privacy (and the privacy of those estimated
to be at risk) to do so. While the technology would support
such screening applications, a public discourse over the most
appropriate balance between screening and intervention is
likely warranted.
VII. C AVEATS
This class of computational linguistic techniques does not
work for everyone – in particular, many older individuals
do not generate sufficient volume of textual data to permit
adequate assessments. Subsequently, this work focuses on
digital natives who allow their communications to be digi-
tally mediated. Ultimately, we expect as time progresses that
the population of digital natives will continue to grow while
the number of digital immigrants and those disconnected will
continue to shrink.
395
VIII. CONCLUSION
Language is just one of many possible signals in the
whitespace between healthcare interactions that can be used
for the assessment and treatment of physical and mental
health conditions. We advocate for the use of these signals
in combination with signals already collected by the health
care system, rather than as a replacement. Aligning these
new data streams from the analysis of everyday language
and behaviors, can yield information that will empower
clinicians and their patients to rapidly personalize treatment,
assessment, diagnosis, and intervention all at a scale not
previously possible.
ACKNOWLEDGMENT
The authors would like to express their gratitude to the rest
of Qntfy and the CLPsych community for continual interest
and passion for advancing the science and understanding of
mental health and well-being.
R EFERENCES
[1] D. E. Bloom, E. Cafiero, E. Jané-Llopis, S. Abrahams-Gessel, L. R.
Bloom, S. Fathima, A. B. Feigl, T. Gaziano, A. Hamandi, M. Mowafi,
A. Pandya, K. Prettner, L. Rosenberg, B. Seligman, A. Stein, and
C. Weinstein, “The global economic burden of noncommunicable
diseases,” World Economic Forum, Geneva, Tech. Rep., 2011.
[2] SAMHSA, “Substance abuse and mental health services adminis-
tration,” in Results from the 2013 National Survey on Drug Use
and Health: Mental Health Findings, NSDUH Series H-49, HHS
Publication No. (SMA) 14-4887, Rockville, MD, 2014.
[3] S. C. Curtin, M. Warner, and H. Hedegaard, “Increase in suicide in the
united states, 1999-2014.” NCHS data brief, no. 241, pp. 1–8, 2016.
[4] M. K. Nock, G. Borges, E. J. Bromet, J. Alonso, M. Angermeyer,
A. Beautrais, R. Bruffaerts, W. T. Chiu, G. De Girolamo, S. Gluzman,
et al., “Cross-national prevalence and risk factors for suicidal ideation,
plans and attempts,” The British Journal of Psychiatry, vol. 192, no. 2,
pp. 98–105, 2008.
[5] E. Sullivan, J. L. Annest, F. Luo, T. Simon, and L. Dahlberg, “Suicide
among adults aged 35–64 years, United States, 1999–2010,” Center
for Disease Control and Prevention, Morbidity and Mortality Weekly
Report, 2013.
[6] K. A. Padrez, L. Ungar, H. A. Schwartz, R. J. Smith, S. Hill,
T. Antanavicius, D. M. Brown, P. Crutchley, D. A. Asch, and R. M.
Merchant, “Linking social media and medical record data: a study of
adults presenting to an academic, urban emergency department,” BMJ
quality & safety, pp. bmjqs–2015, 2015.
[7] R. J. Smith, P. Crutchley, H. A. Schwartz, L. Ungar, F. Shofer, K. A.
Padrez, and R. M. Merchant, “Variations in facebook posting patterns
across validated patient health conditions: A prospective cohort study,”
Journal of Medical Internet Research, vol. 19, no. 1, p. e7, 2017.
[8] J. W. Pennebaker, C. K. Chung, M. Ireland, A. Gonzales, and R. J.
Booth, The development and psychometric properties of LIWC2007.
Austin, TX: LIWC.net, 2007.
[9] C. Chung and J. Pennebaker, “The psychological functions of function
words,” Social Communication, 2007.
[10] M. De Choudhury, M. Gamon, S. Counts, and E. Horvitz, “Predicting
depression via social media,” in Proceedings of the 7th International
AAAI Conference on Weblogs and Social Media (ICWSM), 2013.
[11] G. Coppersmith, M. Dredze, C. Harman, and K. Hollingshead, “From
ADHD to SAD: Analyzing the language of mental health on Twitter
through self-reported diagnoses,” in Proceedings of the Workshop on
Computational Linguistics and Clinical Psychology: From Linguistic
Signal to Clinical Reality. Denver, Colorado, USA: North American
Chapter of the Association for Computational Linguistics, June 2015.
[12] P. Resnik, R. Resnik, and M. Mitchell, Eds., Proceedings of the
Workshop on Computational Linguistics and Clinical Psychology:
From Linguistic Signal to Clinical Reality. Baltimore, Maryland,
USA: Association for Computational Linguistics, June 2014. [Online].
Available: http://www.aclweb.org/anthology/W14-32
Fig. 2. Rolling mean over a one week window of the proportion of messages with emotion that were tagged as “joy” (top) and those with sentiment
tagged as “negative” (bottom) from a group’s internal communications data. Major deadlines are overlayed as dotted lines and major holidays overlayed
as dashed lines.
[13] M. Mitchell, G. Coppersmith, and K. Hollingshead, Eds., Proceedings
of the Workshop on Computational Linguistics and Clinical Psychol-
ogy: From Linguistic Signal to Clinical Reality. Denver, Colorado,
USA: North American Association for Computational Linguistics,
June 2015.
[14] K. Hollingshead and L. Ungar, Eds., Proceedings of the Workshop on
Computational Linguistics and Clinical Psychology: From Linguistic
Signal to Clinical Reality. San Diego, California, USA: North
American Association for Computational Linguistics, June 2016.
[15] G. Coppersmith, R. Leary, E. Whyne, and T. Wood, “Quantifying sui-
cidal ideation via language usage on social media,” in Joint Statistics
Meetings Proceedings, Statistical Computing Section. JSM, 2015.
[16] G. Coppersmith, C. Harman, and M. Dredze, “Measuring post trau-
matic stress disorder in Twitter,” in Proceedings of the 8th Interna-
tional AAAI Conference on Weblogs and Social Media (ICWSM), 2014.
[17] G. Coppersmith, K. Ngo, R. Leary, and T. Wood, “Exploratory data
analysis of social media prior to a suicide attempt,” in Proceedings of
the Workshop on Computational Linguistics and Clinical Psychology:
From Linguistic Signal to Clinical Reality. San Diego, California,
USA: North American Chapter of the Association for Computational
Linguistics, June 2016.
[18] A. Wood, J. Shiffman, R. Leary, and G. Coppersmith, “Discovering
shifts to suicidal ideation from mental health content in social media,”
in Proceedings of the SIGCHI Conference on Human Factors in
Computing Systems. ACM, 2016.
[19] E. Kiciman, M. Kumar, G. Coppersmith, M. Dredze, and
M. De Choudhury, “Discovering shifts to suicidal ideation from
mental health content in social media,” in Proceedings of the SIGCHI
Conference on Human Factors in Computing Systems. ACM, 2016.
[20] A. Cohan, S. Young, and N. Goharian, “Triaging mental health forum
posts,” in Proceedings of the 3rd Workshop on Computational Lin-
guistics and Clinical Psychology: From Linguistic Signal to Clinical
Reality, San Diego, California, USA, June, vol. 16, 2016.
[21] M. Kumar, M. Dredze, G. Coppersmith, and M. De Choudhury, “De-
tecting changes in suicide content manifested in social media following
celebrity suicides,” in Proceedings of the 26th ACM conference on
Hypertext and hypermedia. ACM, 2015.
[22] M. De Choudhury, S. Counts, and E. Horvitz, “Predicting postpartum
changes in emotion and behavior via social media,” in Proceedings of
396
the ACM Annual Conference on Human Factors in Computing Systems
(CHI). ACM, 2013, pp. 3267–3276.
[23] H. A. Schwartz, J. Eichstaedt, M. L. Kern, G. Park, M. Sap, D. Still-
well, M. Kosinski, and L. Ungar, “Towards assessing changes in degree
of depression through Facebook,” in Proceedings of the ACL Workshop
on Computational Linguistics and Clinical Psychology, 2014.
[24] P. Resnik, W. Armstrong, L. Claudino, T. Nguyen, V.-A. Nguyen, and
J. Boyd-Graber, “The University of Maryland CLPsych 2015 shared
task system,” in Proceedings of the Workshop on Computational Lin-
guistics and Clinical Psychology: From Linguistic Signal to Clinical
Reality. Denver, Colorado, USA: North American Chapter of the
Association for Computational Linguistics, June 2015.
[25] K. Hollingshead, H. A. Schwartz, G. Coppersmith, F. Almoradasi,
A. Benton, J. Craley, P. Crutchley, D. Hovy, M. Ireland, B. S. Kim,
L. Kim, R. Merchant, M. Mitchell, P. Resnik, M. Rouhizadeh, and
L. Ungar, “Detecting risk and protective factors of mental health using
social media,” Center for Language and Speech Processing Technical
Reports, in prep.
[26] J. C. Eichstaedt, H. A. Schwartz, M. L. Kern, G. Park, D. R.
Labarthe, R. M. Merchant, S. Jha, M. Agrawal, L. A. Dziurzynski,
M. Sap, C. Weeg, E. E. Larson, L. H. Ungar, and M. E. P. Seligman,
“Psychological language on Twitter predicts county-level heart disease
mortality,” Psychological Science, vol. 26, no. 2, pp. 159–169, 2015.
[27] H. A. Schwartz, J. C. Eichstaedt, M. L. Kern, L. Dziurzynski, R. E.
Lucas, M. Agrawal, G. J. Park, S. K. Lakshmikanth, S. Jha, M. E. P.
Seligman, and L. H. Ungar, “Characterizing geographic variation in
well-being using tweets,” in Proceedings of the 8th International AAAI
Conference on Weblogs and Social Media (ICWSM), 2013.
[28] C. J. Hutto and E. Gilbert, “Vader: A parsimonious rule-based model
for sentiment analysis of social media text,” in Eighth International
AAAI Conference on Weblogs and Social Media, 2014.
[29] M. Conway, “Ethical issues in using twitter for public health surveil-
lance and research: developing a taxonomy of ethical concepts from
the research literature,” Journal of medical Internet research, vol. 16,
no. 12, p. e290, 2014.
[30] J. Mikal, S. Hurst, and M. Conway, “Ethical issues in using twitter
for population-level depression monitoring: a qualitative study,” BMC
medical ethics, vol. 17, no. 1, p. 1, 2016.