.

Identification and evaluation of patient adverse drug event reports

A project report submitted to

DEPARTMENT OF PHARMACY, STAREX UNIVERSITY

BACHELOR OF PHARMACY

Submitted By

RAVI

Roll no. 180701045

Under the guidance of

Ms. Sunaina

(Assistant Prof. in Pharmacy Department)

STAREX UNIVERSITY
Gurugram
.

Certificate
This is certified by the investigations described in this report entitled
“Identification and evaluation of patient adverse drug event reports ”. In
Partial fulfillment of requirement of degree of Bachelor of Pharmacy
Were carried out in the Starex University NH-48, BINOLA, P.O
BHORA KALAN, GURUGRAM , HARYANA122413 Brijesh Kumar
under my direct guidance under

Guide :- Ms. Sunaina

Head of the Department :- Dr. Neelam Dhankar
School of pharmaceutical sciences
.

Declaration
I Hereby declare that the thesis entitled “ Identification and evaluation of
patient adverse drug event reports” is project carried out by me under
the supervision of “ Ms. Sunaina”. This work has not been submitted
earlier, in full or in part, for the award of any degree, diploma or
Fellowship in any institution
.

ACKNOWLEGMENT
We feel highly privileged to express our deep sense of gratitude to all
Those who helped us during our project work. First of all, we would to
thank the supreme power. Without his grace this project would Never
have come to see the light of the day. We express our gratitude And
reverence to the preceptor and Project Guide Ms. Sunaina ,Lecturer in
B.PHARMACY for her advice, guidance and support which Helped us in
completing our project. She has taken pain to go through The project
and make necessary correction as and when needed. We Would like to
express our grateful thanks for the help and advice Given to us . We are
also highly thankful to our Institution for providing necessary facilities
and infrastructure.

Ravi
B.Pharm(8th sem)
.

Contents
➢ Front page
➢ Certificate
➢ Declaration
➢ ACKNOWLEDGEMENT
➢ Abstract
➢ Introduction
➢ Related work
➢ Research Method
➢ Experience and result
➢ Conclusions and Contribution
➢ Aknowledgement
➢ Refrence
.

Abstract :-
Social media offer insights of patients’ medical problems such as drug
side effects and treatment failures.Patient reports of adverse drug events
from social media have great potential to improve current practice of
pharmacovigilance. However, extracting patient adverse drug event
reports from social media continues to be an important challenge for
health informatics research. In this study, we develop aresearch
framework with advanced natural language processing techniques for
integrated and high performance patient reported adverse drug event
extraction. The framework consists of medical entity extraction for
recognizing patient discussions of drug and events, adverse drug event
extraction with shortest dependency path kernel based statistical learning
method and semantic filtering with information from medical knowledge
bases, and report source classification to tease out noise. To evaluate the
proposed framework, a series of experiments were conducted on a test
bed encompassing about postings from major diabetes and heart disease
forums in the United States. The results reveal that each component of
the framework significantly contributes to its overall effectiveness. Our
framework significantly out performs prior work.
.

Introduction:-
In recent years, a growing number of patients are sharing their
experiences of healthcare on the Internet. This body of
information is described as ‘‘cloud of patient experience”. The
increasing availability of patients’ accounts of their care on blogs,
social networks, and forums presents an intriguing opportunity to
advance the patient-centered care agenda. Patients with chronic
diseases such as hypertension, heart diseases, diabetes, and cancer
utilize the social media to share their diagnosis, treatment
opinions, medications and side effects . Patient self-reports on
social media frequently capture medical issues and side effects
that clinicians often miss or downgrade. Clinicians’ failure to note
those issues results in the occurrence of drug non-compliance and
preventable adverse events . Mining social media has been
considered as a new approach for collecting evidence for drug
side effects, drug compliance and drug effectiveness. It can
enhance the capture of subjective elements of drug safety and
.

treatment management,providing important insights for clinical

practitioners.

An adverse drug reaction (ADR) can be defined as ‘an

appreciably harmful or unpleasant reaction resulting from
an intervention related to the use of a medicinal product;
adverse effects usually predict hazard from future
administration and warrant prevention, or specific
treatment, or alteration of the dosage regimen, or
withdrawal of the product’.
.

Related work :
• Pharmacovigilance in health social media
Social media has developed to be a fundamental
information foundation for pharmacovigilance studies
where a growing number of people post adverse
reactions to medical drugs that are previously
unreported

• Biomedical relation extraction

Automatically extracting biomedical information has
been the subject of significant research efforts due to
the rapid growth in biomedical development and
discovery . Biomedical relation extraction techniques
are often developed to identify relations such as gene-
.

disease relations and protein interactions .

Biomedical relation extraction techniques are often

categorized into four types:
co-occurrence analysis, rule-based approaches,
statistical learning approach and hybrid approach,
which utilizes both rules and statistical learning.

• Research gaps and questions

Based on our review, we have identified several research
gaps.First, little advanced statistical learning based relation
extraction has been adopted in health social media adverse drug
event research. Prior pharmacovigilance research in health social
media employed co-occurrence analysis based relation extraction
techniques to extract adverse drug events. Co-occurrence analysis
only utilizes lexical features in the sentences. Neglecting the
syntactical information and semantic information in the sentence,
this approach could generate significant false positive results
.

causing by negations and true drug indications. Second, the

adverse drug events discussed in health social media may come
from a variety of sources such as patients, news, research, and
stories from third-hand accounts, which result in redundancy and
noise. While more researchers started to be aware of the
importance of patients’ voice in drug safety reporting , few prior
studies in social media pharmacovigilance research identified
patient adverse drug event reports based on true patient
experiences. The value of health social media, an open and
popular platform for patient to speak out their problems and
demands, has not been fully explored.
Based on the research gaps identified, we proposed the following
research questions:

1. How can we develop an integrated and scalable research

framework for mining patient reported adverse drug events from
patient forums?

2. How can statistical learning techniques augmented with health-

relevant semantic filtering improve the extraction of adverse drug
.

events as compared to other baseline methods?

3. How can we identify true patient reported adverse drug events
among noisy forum discussions?
.

• Research Method
Our proposed research framework for pharmacovigilance in
health social media is illustrated in Fig. 1. Major components
are explained in detail below.
• Patient forum data collection
We developed an automated crawler program to download
web pages from online patient forums. An extractor program
was written to extract specific fields in patient discussions.
Collected information includes post ID (the unique identifier
of a post in the forum), URL, topic title, post author’s ID (the
unique identifier of a user in the forum), post date, and post
conten

• Data preprocessing

Data preprocessing prepares the raw data for subsequent

analysis. Data preprocessing consists of two steps: text
cleaning and sentence boundary detection. We developed
.

regular expression base approach to remove URLs, duplicate

punctuations, and personally identifiable information such
as email addresses, social security numbers, and phone
numbers from the text. As our study focuses on sentence
level information extraction and processing, we segment
each post into sentences with a natural language processing
toolkit, OpenNLP.1 OpenNLP provides state-of-the-art
machine learning based sentence boundary detection
algorithm.Each post we obtain from the crawler can then be
segmented into individual sentences with OpenNLP.

• Adverse drug event extraction

Patients’ adverse drug event discussions in forums are more
informal and colloquial than biomedical literature or clinical
notes, which require medical knowledge and complex
linguistic techniques to interpret. Based on our review of
prior biomedical relation extraction studies, a hybrid
approach with both statistical machine learning methods and
rule-based filtering achieved satisfying performance. Our
approach incorporates the statistical learning method for
relation detection and semantic information from medical
.

and linguistic knowledge bases to identify adverse

drug events from drug indications and negated ADEs. The
statistical learning and semantic filtering components for
adverse drug event extraction.
.

• Semantic filtering

Shortest dependency path kernels can detect related drug and

medical events. However, this method cannot precisely
capture negation in sentences and differentiate drug
indication relations from adverse drug events. Most prior
studies neglected the importance of filtering out drug
indications and negated ADEs for analysis, leading to a low
precision. To address these issues, we develop a semantic
filtering algorithm, which utilizes thesemantic knowledge in
drug safety database to remove drug indications and rules
from negation detection tool to filter out negated ADEs.

• Research hypotheses

Based on the research gaps identified in the literature review,

we believe our proposed framework can significantly
improve the performance of patient adverse drug event
extraction in health social media. In particular, we propose
.

the following hypotheses:

H1a. Statistical learning methods in adverse drug event
extraction will outperform the co-occurrence analysis based
approach.
H1b. Semantic filtering in adverse drug event extraction will
further improve the performance of adverse drug event
extraction.
H2. Report source classification (RSC) can improve the
results of patient adverse drug event report extraction as
compared to not accounting for report source issues.
.

• Experiment and results

• Research test bed

Chronic diseases, such as diabetes and heart

diseases rely on patient self-management. Many
online health forums have emerged to provide
chronic disease patients with an anonymous
connection to an understanding audience where they
can ask questions, gain knowledge, and share
frustrations about their treatments. Our research test
bed is developed from major diabetespatient forums
and heart disease discussion boards in the United
States, including American Diabetes Association
online community, Diabetes Forums, Diabetes
Forum, Heart Disease, Heart Rhythm and Coronary
Heart Disease discussion boards fromMedHelp.
.

The American Diabetes Association (ADA) is a

United Statesbased association working to fight the
consequences of diabetes and to help those affected
by diabetes.

• Evaluation metrics

We adopt standard machine-learning and text

analysis evaluation metrics, precision, recall and f-
measure, to evaluate the performances of our
framework. These metrics have been widely used in
information extraction and health social media
studies

• Experiments

In this study, we conduct our experiments on

extracting patient reports of adverse drug events by
.

performing three tasks: medical entity extraction,

adverse drug event extraction, and report source
classification. We conduct 5-fold cross validation to
obtain the evaluation results for adverse drug event
extraction and report source classification. For each
forum, each time 80% of labeled data and all the
unlabeled sentences in our test bed are used as
training set and 20% of labeled data are used as test
set.

2. Adverse drug event extraction

To conduct relation detection, we randomly selected

400 sentences with at least one drug entity and one
medical event entity from each forum for
annotation. With this approach, we focus on
determine the relation of drugs and medical events
in the same sentences. Relations of drugs and
.

medical events across sentences in the same post are

not considered in this study.

• Results and discussions

.

2.Adverse drug event extraction

We compared the baseline co-occurrence method

(CO) against statistical learning method (SL) as well
as against our proposed adverse drug event
extraction method including statistical learning and
semantic filtering (SL + SF).

3. Hypothesis testing

In order to test our hypotheses, we conducted pair-

wise one tailed t-tests on F-measure. Bootstrapping
.

is used for statistical testing. Bootstrapping was

performed by randomly selecting 40 instances for
testing and remaining 360 for training, 50 times.
We evaluated the F-measure using pair wise t-tests
across the samples (n = 50). The p values for the
tests of our hypotheses for adverse drug event
extraction and report source classification are
presented.
.

• Conclusions and contributions

The advent of social media offers insights into

healthcare unfiltered by traditional methods of
healthcare data collection. Applying natural
language processing techniques to extract patient
reports of adverse drug events from social media has
great potential to improve clinical and scientific
knowledge of pharmacovigilance. In this study, we
develop a research framework for
pharmacovigilance in social media to identify
patient reported adverse drug events. It consists of
medical entity extraction for recognizing patient
discussions of drug and events, adverse drug
.

event extraction with shortest dependency path

kernel based statistical learning method and
semantic filtering with information from medical
knowledge bases, and report source classification to
tease out noise.
The major contribution of our research is the design
and evaluation of our research framework for
pharmacovigilance research in health social media.
It incorporates the state-of-the-art naturallanguage
processing techniques and effectively addresses the
challenges in extracting patient adverse drug event
reports from social media. This framework can be
applied to analyze treatments of different diseases
and extract patient intelligence on other medial
related topics.
.

• Conflict of interest

The authors declare that there are no conflicts of

interest.
.

• Acknowledgements

This material is based upon work supported by a

subcontract from Caduceus Intelligence Corporation with
grant funds provided by the United States National
Science Foundation

(IIP-1417181). We gratefully acknowledge the

contribution of Dr. Randall Brown and Ms. Chanadda
Chinthammit for their advices from clinical and
pharmaceutical perspectives in this study. We thank Jing
Liu for her analytical work on the MedHelp forum. We also
appreciate the research assistance provided by fellow
members of Healthcare Project team in University of
Arizona’s Artificial Intelligence Lab.
.

References
[1] A.R. Miller, C. Tucker, Active social media
management: the case of health care, Inform. Syst.
Res. 24 (1) (2013) 52–70.
[2] J.J. Mao, A. Chung, A. Benton, S. Hill, L. Ungar,
C.E. Leonard, S. Hennessy, J.H. Holmes, Online
discussion of drug side effects and discontinuation
among breast cancer survivors, Pharmacoepidemiol.
Drug Saf. 22 (3) (2013) 256–262.
[3] E. Basch, The missing voice of patients in drug-
safety reporting, N. Engl. J. Med.
362 (10) (2010) 865–869.
[4] M. Hauben, A. Bate, Decision support methods
for the detection of adverseevents in post-marketing
data, Drug Discov. Today 14 (7) (2009) 343–357.
[5] A. Bate, S. Evans, Quantitative signal detection
.

using spontaneous ADR

reporting, Pharmacoepidemiol. Drug Saf. 18 (6)
(2009) 427–436.
[6] I.R. Edwards, M. Lindquist, Social media and
networks in pharmacovigilance,Drug Saf. 34 (4)
(2011) 267–271.
[7] R. Chunara, J.R. Andrews, J.S. Brownstein,
Social and news media enable estimation of
epidemiological patterns early in the 2010 Haitian
cholera outbreak, Am. J. Trop. Med. Hyg. 86 (1)
(2012) 39–45.
[8] R. Harpaz, W. DuMouchel, N.H. Shah, D.
Madigan, P. Ryan, C. Friedman, Novel data-mining
methodologies for adverse drug event discovery and
analysis, Clin. Pharmacol. Ther. 91 (6) (2012)
1010–1021.
[9] R. Leaman, L. Wojtulewicz, R. Sullivan, A.
Skariah, J. Yang, G. Gonzalez, Towards internet-
age pharmacovigilance: extracting adverse drug
.

reactions from user posts to health-related social

networks, in: Proceedings of the 2010 Workshop on
Biomedical Natural Language Processing,
Association for Computational Linguistics, 2010,
pp. 117–125.
[10] A. Benton, L. Ungar, S. Hill, S. Hennessy, J.
Mao, A. Chung, C.E. Leonard, J.H. Holmes,
Identifying potential adverse effects using the web:
a new approach to medical hypothesis generation, J.
Biomed. Inform. 44 (6) (2011) 989–996.
[11] A. Nikfarjam, G.H. Gonzalez, Pattern mining
for extraction of mentions ofadverse drug reactions
from user comments, AMIA Annual Symposium
Proceedings, vol. 2011, American Medical
Informatics Association, 2011, p. 1019.
[12] A. Yates, N. Goharian, Adrtrace: detecting
expected and unexpected adverse drug reactions
from user reviews on social media sites, in:
Advances in Information Retrieval, Springer, 2013,
.

pp. 816–819.
[13] X. Liu, H. Chen, Azdrugminer: an information
extraction system for mining patient-reported
adverse drug events in online patient forums, in:
Smart Health, Springer, 2013, pp. 134–150.
[14] J. Bian, U. Topaloglu, F. Yu, Towards large-
scale twitter mining for drug-related adverse events,
in: Proceedings of the 2012 International Workshop
on Smart Health and Wellbeing, ACM, 2012, pp.
25–32.
[15] A. Sarker, G. Gonzalez, Portable automatic text
classification for adverse drug reaction detection via
multi-corpus training, J. Biomed. Inform. 53 (2015)
196–207.
[16] I. Segura-Bedmar, P. Martínez, R. Revert, J.
Moreno-Schneider, Exploring spanish health social
media for detecting drug effects, BMC Med. Inform.
Decis.Making 15 (Suppl. 2) (2015) S6.
[17] B.W. Chee, R. Berlin, B. Schatz, Predicting
.

adverse drug events from personal health messages,

AMIA Annual Symposium Proceedings, 2011,
American Medical Informatics Association, 2011,
p. 217.
[18] H. Wu, H. Fang, S. Stanhope, et al., Exploiting
online discussions to discover unrecognized drug
side effects, Method Inform. Med. 52 (2) (2013)
152–159.

[19] D.A. Lindberg, B.L. Humphreys, A.T.

McCray, The unified medical language system,
Methods Inform. Med. 32 (4) (1993) 281–291.
[20] J. Hadzi-Puric, J. Grmusa, Automatic drug
adverse reaction discovery from parenting websites
using disproportionality methods, in: Proceedings of
the 2012 International Conference on Advances in
Social Networks Analysis and Mining (ASONAM
2012), IEEE Computer Society, 2012, pp. 792–797.
[21] M. Kuhn, M. Campillos, I. Letunic, L.J. Jensen,
.

P. Bork, A side effect resource to capture

phenotypic effects of drugs, Mol. Syst. Biol. 6 (1)
(2010) 343.
[22] C.C. Yang, H. Yang, L. Jiang, M. Zhang, Social
media mining for drug safetysignal detection, in:
Proceedings of the 2012 International Workshop on
Smart Health and Wellbeing, 2012,pp. 33–40.
[23] H. Gurulingappa, L. Toldo, A.M. Rajput, J.A.
Kors, A. Taweel, Y. Tayrouz, Automatic detection
of adverse events to predict drug label changes using
text and data mining techniques,
Pharmacoepidemiol. Drug Saf. 22 (11) (2013)1189–
1194.
[24] Q.-C. Bui, S. Katrenko, P.M. Sloot, A hybrid
approach to extract protein–protein
interactions, Bioinformatics 27 (2) (2011) 259–265.
[25] K. Fundel, R. Küffner, R. Zimmer, Relex–
relation extraction using dependency parse trees,
.

Bioinformatics 23 (3) (2007) 365–371.