2016 IEEE International Conference on Bioinformatics and Biomedicine (BIBM)
Biomedical Event Extraction Based on Distributed
Representation and Deep Learning
Anran Wang, Jian Wang*, Hongfei Lin, Jianhai Zhang, Zhihao Yang, Kan Xu
School of Computer Science and Technology, Dalian University of Technology, Dalian, China
E-mail: * wangjian@dlut.edu.cn
Abstract—The two main problems of biomedical event
extraction are trigger identification and argument detection which
can both be considered as classification problems. In this paper,
we propose a distributed representation method, which combines
context, consisted by dependency-based word embedding, and
task-based features represented in a distributed way on deep
learning models to realize biomedical event extraction. The
experimental results on Multi-Level Event Extraction (MLEE)
corpus show higher F-scores compared to the state-of-the-art SVM
method. This demonstrates that our proposed method is effective
for biomedical event extraction.
Keywords—Biomedical event extraction; Distributed
representation; Deep learning; Convolutional neural network
I. INTRODUCTION
A biomedical event is normally described by event type,
trigger and arguments. The goal of extraction is to detect and
classify the type of trigger and the event arguments. Due to rich
feature engineering, support machine vector (SVM) previously
has reached the best performance [1]. However, massive feature
engineering and semantic gap from SVM created obstacles
when exploring more semantic features. Meanwhile, the deep
learning methods have achieved considerable progress on some
natural language processing tasks such as sentence
classification[2]. In this paper, we proposed a distributed
representation method on deep learning models. Our goal is to
realize event trigger identification and event argument detection
to complete biomedical event extraction.
II. METHOD
Our main work proceeded as follows. First, we trained
dependency-based word embedding from all available PubMed
abstracts. Then, we formed the distributed representation for
deep learning models for the two subtasks.
A. Event Trigger Identification
The aim of trigger identification is to predict the trigger type
of words. First, we concatenated word embedding trained
previously for words in the window size around candidate
triggers. Then, we multiplied the topic distribution of every word
acquired by Latent Dirichlet Allocation (LDA)[3] to represent
the approximate sentence topic distribution. Third, we randomly
initialized two vector tables for the POS of words and the
distance of words to closest entity in the dependency tree.
Finally, the concatenation of them formed the representation of
candidate triggers and was fed to the deep neural network. A
softmax layer was used to realize classification.
978-1-5090-1610-5/16/$31.00 ©2016 IEEE
B. Event Argument Detection
Event argument detection is the step for examining whether
entities have relation with the triggers and detecting the relation
type. We converted the word embeddings of the words on the
dependency-path between trigger and entity to a matrix as
representation of the logical semantic relations. Then we
concatenated the matrix with their POS and relative distance
vectors. We also added the type of trigger and entities. Finally,
we trained Convolutional neural network (CNN) with the input
layer formed by these distributed representations.
III. EXPERIMENT
We evaluated our proposed method on MLEE corpus [4],
and it achieved a better F-score compare to the baseline on
trigger identification and event extraction. This indicated our
method’s efficiency. We also conducted feature analysis tests
and the results showed that task-based features contributed to the
performance.
IV. CONCLUSION
In this paper, we have presented a distributed representation
method that combines dependent context formed by word
embedding with task-based features of biomedical text. Then we
fed it to deep learning models to realize biomedical event
extraction. This method avoided the problems of semantic gap
and dimension disaster from traditional one-hot representation
methods and achieved promising results with few manually
extracted features, instead of complex feature engineering.
ACKNOWLEDGMENT
This work is supported by the grants from the National Key
Technology R&D Program (2015BAF20B02), Natural Science
Foundation of China (61572098, 61572102, and 61602078), and
Liaoning Province Natural Science Foundation of China
(2014020003).
REFERENCES
[1] D. Zhou, D. Zhong, “A semi-supervised learning framework for
biomedical event extraction based on hidden topics,” Artificial
intelligence in medicine, vol. 64(1), pp. 51-58, 2015.
[2] M. Ma, L. Huang, B. Xiang, B. Zhou, “Dependency-based Convolutional
Neural Networks for Sentence Embedding,” Meeting of the Association
for Computational Linguistics and the International Joint Conference on
Natural Language Processing, 2015.
[3] D.M. Blei，A.Y. Ng，M.I. Jordan, “Latent dirichlet allocation,” Journal
of Machine Learning Research, vol. 3, pp. 993-1022, 2003.
[4] S. Pyysalo, T. Ohta, M Miwa, H.C. Cho, J.I. Tsujii, et al., "Event
extraction across multiple levels of biological organization,"
Bioinformatics, vol. 28(18), pp. i575-i581, 2012.
775