On Social media mining for

pharmacovigilance and other matters

Health Language Processing Lab - Penn IBI

Graciela Gonzalez-Hernandez @gracielagon
email: gragon@pennmedicine.upenn.edu

ADR Pipeline Funded by NLM/NIH 5R01LM011176

Timeline study funded in part by AbbVie and Penn IBI.

AMIA 2017 – SMM4H Workshop

 Social media as an “online health report”?

 26% of internet users actively discuss health

information. Of that group … 1

– 30% changed behavior as a result

– 42% discussed current medical conditions

 “Extrapolating” this to Twitter... 2,3

– Given 317 million active monthly users (Q3 2015): about 24

million would change their health behavior
– Given 350,000 tweets/minute: about 38,220 tweets / minute
about their current medical conditions

1
http://www.pewinternet.org/fact-sheets/health-fact-sheet/ 2http://www.statisticbrain.com/twitter-statistics/
2
http://www.statista.com/statistics/282087/number-of-monthly-active-twitter-users/
3
http://www.internetlivestats.com/twitter-statistics/
SM data for pharmacovigilance: Significance
 Patient reporting brings different perspective (34
studies) more detail, info on severity and impact of ADRs in
daily life. (PMID 27558545).
 Abundant adverse event reports in SM. (29 studies that
compared SM to other sources) showed a higher frequency
of adverse events was found in social media and that this
was particularly true for ‘symptom’ related and ‘mild’
adverse events. (PMID 26271492).
SM data for pharmacovigilance: challenges
 Incompleteness:
• Not all health conditions may be revealed
• Complete data about individual cases may be difficult to obtain: taking
drug X, but dosage, frequency etc. information may be missing
• Participants from the cohort may dropout at higher rates
 Accessibility:
• Data from social media is not easily collected: API limitations
• Not easily “digestable” once collected: challenging to process using
automatic methods
• Data collection methods may have to be changed frequently over time
 Authenticity:
• Bots – a large portion of social media is now generated by bots,
making it harder to mine reliable data
• Automatic processing of postings is often misled: postings mentioning
a drug might not necessarily mean intake.
Social Media pipeline for ADE detection

Concept
extractio
Data Concept n
Mapping
collection
Signal
Detection
Annotation

Classification
Drug-ADE pair
A taste of Twitter ADR lingo

 HA! Not if you're on #Seroquil. EXTREMELY vivid dreams that

stay in conscious memory. Very #Freaky! Any idea why?
 I'm def suing cymbalta. I can't wait until its out of my system.
Get out!!!!!!! Nowwww!!!!! You turn peaceful people into the
hulk!. (c0034634 – Rage)
 Apparently, Baclofen greatly exacerbates the "AD" part of my
ADHD. Average length of focus today: about 30 seconds.
(c0235198 – cerebration impaired)
 The 100mg tabs of trazodone my gp prescribed are too much,
now that I don't take them every night. Still zombieish after an
hour awake
 Gone from 50mg to 150mg of Serequel last night. Could barely
wake up this morning and I feel like my body is made of lead
 Data collection and annotation tasks
 Phonetic spelling variants for capturing misspelled
medication names1
• Seroquel -> siroquil, seroquil etc.
 Binary (yes/no) and full (concept span + mapping) ADE
annotations2,3
 Multiple trained annotators + pharmacology expert to
resolve annotation disagreements
 Datasets shared for research purposes
(www.healthlanguageprocessing.com)

1 Pimpalkhute et al. Phonetic Spelling Filter for Keyword Selection. AMIA Jt Summits Transl Sci Proc. 2014.
2 O’Connor et al. Pharmacovigilance on Twitter. AMIA Annu Symp Proc. 2014.
3 Ginn et al. Mining Twitter for adverse drug reaction mentions. BioTxtM. 2014.
Annotation example

Indication: Indication: ADR: drowsiness

depression (C001157) (C0013144)
mania (C0338831)

Works to calm mania or depression but zonks me and scares

me about diabetes issues reported.

Other:
diabetes Adverse reaction:
Indication:
crying (C0010399)
emotional indifference
(C0001726)
stops me from crying most of the time, blocks most of my
feelings
Text classification

 Text classification tasks:

• Adverse event mention classification1
• Drug abuse classification2
• Medication intake mention classification3
• Safety signal classification4
 Training data is used to “teach” a program to
discriminate
• Generate a large set of features, representing semantic
properties (e.g., sentiment, polarity, and topic), from short text1
• Combine training data from different corpora in attempts to
boost classification accuracies
1 Sarker and Gonzalez. Portable automatic text classification. J Biomed Inform. 2015.
2 Sarker et al. Social media mining for toxicovigilance. Drug Saf. 2016.
3 Klein et al. Detecting Personal Medication Intake in Twitter. ACL BioNLP Workshop 2017
4 Patki et al. Mining adverse drug .. going beyond extraction. BioLinkSig. 2014.
Concept extraction: finding AEs
 Given a set of postings, need to find a specific
concepts automatically
 Again, a program is “trained” to find (extract)
mentions of AEs and other concepts
 Our approach using conditional random fields and
deep learning clusters outperforms lexicon based
approaches1
• Traditional, lexicon-based approaches perform poorly on social media

1 Nikfarjam et al. Pharmacovigilance from social media.. sequence labeling with

word embedding cluster features. JAMIA. 2015.
Unsupervised learned clusters
Cluster# Topic Examples of clustered words

c1 Drug abilify, adderall, ambien, ativan, aspirin, citalopram, effexor, paxil,

…
c2 Signs/Symptoms hangover, headache, rash, hive, …

c3 Signs/Symptoms anxiety, depression, disorder, ocd, mania, stabilizer, …

c4 Drug dosage 1000mg, 100mg, .10, 10mg, 600mg, 0.25, .05, ...

c5 Treatment anti-depressant, antidepressant, drug, med, medication, medicine,

treat, …

c6 Family member brother, dad, daughter, father, husband, mom, mother, son, wife, …

c7 Date 1992, 2011, 2012, 23rd, 8th, april, aug, august, december, …
Concept mapping: translating AEs
 Given a (potential) concept mention from the
consumer, find its likely corresponding term
 AE given in user language is mapped to its UMLS
concept id (MedDRA -Medical Dictionary for
Regulatory Activities)
 Extremely difficult due to colloquialisms: the level
of detail and ambiguity precludes fine-grained
mapping.
 To make the problem “solvable”: group similar
effects into a high-level concept
MedDRA
Normalization – HSA
• We propose a system called hybrid semantic analysis (HSA)
that combines rule-based and semantic matching algorithms
• Maps to user-generated mentions to concept IDs in the
Unified Medical Language System
• The semantic matching component of HSA is adaptive in
nature:
• It adapts to a particular normalization task by combining a set of text-
based resources and semantic relatedness measures
• A regression model is used to combine the different semantic
relatedness measures

Emadzadeh E, Sarker A, Nikfarjam A, and Gonzalez G, Hybrid Semantic Analysis for Mapping
Adverse Drug Reaction Mentions in Tweets to Medical Terminology, Annual Symposium of
the American Medical Informatics Association, 2017, Nov 2017.
Performane
Precision Recall F-Measure
Syntactic 88.0 35.7 50.8
LSA-PubM-Dental 83.6 38.2 52.4
LSA-PubM-Nursing 83.1 38.6 52.7
LSA-UMLS-Defs 86.5 40.3 55.0
LSA-Reuters 81.5 44.9 57.9
LSA-PubM-Systematic 83.6 44.4 58.0
LSA-ADR-Tweets 84.6 47.7 61.0
MetaMap 82.6 18.7 30.5
HSA 82.3 50.2 62.4
Social media analysis pipeline drug by drug
 Collect data from Twitter using the Public API (
https://dev.twitter.com/streaming/public) using the keywords
humira, adalimumab, and common misspellings
 Process tweets with ADRMine system to extract mentions of
ADEs.
 Map to standard concepts in the Unified Medical Language
System (UMLS) using automatic a lexical comparison +
manual annotation.
 UMLS concepts categorized by frequency.
 Run disproportionality analyses: proportional reporting ratio
(PRR) and reporting odds ratio (ROR).
 Compare against other sources: pharmacovigilance data,
Micromedex, Lexicomp, Clinical Pharmacology, as well as
systematic reviews/meta-analyses, cohort studies and case-
control studies in the published literature.
Prescription drug abuse mentions

 About to be cracked on adderall to survive today

 I'm just gonna shower and overdose on Seroquel so I’ll sleep
until morning.
 Popped Adderall tonight hahahah let’s finish this 100 page
paper
 An oxycodone high from snorting lasts for one hour, if it is
swallowed, your looking at three hour high.

Sarker A, O'Connor K, Ginn R, Scotch M, Smith K, Malone D, Gonzalez G. Social media mining for
toxicovigilance: automatic monitoring of prescription medication abuse from Twitter, Drug Safety,
2016 Mar;39(3):231-40. (PMID: 26748505)
Adderall® vs. oxycodone abuse patterns
 Supervised classification to investigate patterns of
abuse-related tweets1

1 Sarker et al. Social media mining for toxicovigilance. Drug Saf. 2016.
Next steps: Mining temporal data

 Address the gap in longitudinal social media based

public health surveillance

 Develop natural language processing (NLP), machine

learning, and information retrieval (IR) methods to help
accurately identify a cohort of interest and collect their
social media timelines

 Perform preliminary analyses of the extracted health

timelines to identify limitations, and establish future
research goals.
Proof of Concept: Pregnancy Data
 Collect tweets announcing pregnancy
• Based on search queries
• Time period: Jan 2014 to Sept 2015
• Example query: “i am * weeks/months pregnant”
• Query count: 18
 Not all were actually pregnancy announcements
• “…I look like Im 3 months pregnant”
• CLASSIFICATION: N-grams and synsets, sentiment,
word clusters
 Annotation for just the announcement is relatively
straightforward, but the classification task is not
Sarker A, Chandrashekar P, Magge A, Cai H, Klein A, Gonzalez G
Discovering Cohorts of Pregnant Women From Social Media for Safety Surveillance and Analysis
J Med Internet Res 2017;19(10):e361 URL: http://www.jmir.org/2017/10/e361
DOI: 10.2196/jmir.8164 PMID: 29084707
POC: temporal references

•Extract pregnancy timeframe

•Tag trimester: combination of term and pattern
matching.
• “I'm officially 20 weeks pregnant….” : 2nd trimester.
•Challenge: cover ambiguous and relative time
cases:
• “next week is gone b my last week pregnant who want
to make a bet lol”
Other related work
 Determine relative safety of nutritional
products from Amazon reviews
• Sullivan R, Sarker A, O’Connor K, Goodin A, Karlsrud M, Gonzalez G.
Finding potentially unsafe nutritional supplements from user reviews
with topic modeling. In Proceedings of the Pacific Symposium on
Biocomputing. 2016. pp 528—539 (PMID 26776215).

 Use topic modeling on timelines to predict

the safety of medications consumed
• Jian Yang, PhD Dissertation, Defended Oct 2017
(paper in preparation)
Thank you!

gragon@pennmedicine.upenn.edu
Twitter: @gracielagon
HLP lab: 
https://healthlanguageprocessing.org